Skip to content

apt-get update does not run cleanly on a brand-new cluster #622

Description

@kynnjo

Right after creating a new cluster, I log in to it with starcluster sshmaster <name-of-cluster>, and immediately run the command apt-get update (as the very first command after login). The output from this command reveals several problems:

# apt-get update
Get:1 http://archive.cloudera.com squeeze-cdh3u5 Release.gpg [198 B]
Get:2 http://archive.cloudera.com squeeze-cdh3u5 Release [2,405 B]
Ign http://us-east-1.ec2.archive.ubuntu.com raring Release.gpg
Ign http://us-east-1.ec2.archive.ubuntu.com raring-updates Release.gpg
Ign http://us-east-1.ec2.archive.ubuntu.com raring Release
Ign http://us-east-1.ec2.archive.ubuntu.com raring-updates Release
Ign http://us-east-1.ec2.archive.ubuntu.com raring/main Sources/DiffIndex
Ign http://us-east-1.ec2.archive.ubuntu.com raring/universe Sources/DiffIndex
Ign http://us-east-1.ec2.archive.ubuntu.com raring/main amd64 Packages/DiffIndex
Ign http://us-east-1.ec2.archive.ubuntu.com raring/universe amd64 Packages/DiffIndex
Ign http://us-east-1.ec2.archive.ubuntu.com raring-updates/main Sources/DiffIndex
Ign http://us-east-1.ec2.archive.ubuntu.com raring-updates/universe Sources/DiffIndex
Ign http://us-east-1.ec2.archive.ubuntu.com raring-updates/main amd64 Packages/DiffIndex
Ign http://us-east-1.ec2.archive.ubuntu.com raring-updates/universe amd64 Packages/DiffIndex
Get:3 http://archive.cloudera.com squeeze-cdh3u5/contrib Sources [5,001 B]
Ign http://security.ubuntu.com raring-security Release.gpg
Get:4 http://archive.cloudera.com squeeze-cdh3u5/contrib amd64 Packages [14.0 kB]
Hit http://ppa.launchpad.net raring Release.gpg
Ign http://security.ubuntu.com raring-security Release
Ign http://us-east-1.ec2.archive.ubuntu.com raring/main Translation-en_US
Ign http://us-east-1.ec2.archive.ubuntu.com raring/main Translation-en
Ign http://us-east-1.ec2.archive.ubuntu.com raring/universe Translation-en_US
Ign http://us-east-1.ec2.archive.ubuntu.com raring/universe Translation-en
Ign http://us-east-1.ec2.archive.ubuntu.com raring-updates/main Translation-en_US
Ign http://us-east-1.ec2.archive.ubuntu.com raring-updates/main Translation-en
Ign http://us-east-1.ec2.archive.ubuntu.com raring-updates/universe Translation-en_US
Ign http://us-east-1.ec2.archive.ubuntu.com raring-updates/universe Translation-en
Err http://us-east-1.ec2.archive.ubuntu.com raring/main Sources
  404  Not Found [IP: 54.152.129.43 80]
Err http://us-east-1.ec2.archive.ubuntu.com raring/universe Sources
  404  Not Found [IP: 54.152.129.43 80]
Err http://us-east-1.ec2.archive.ubuntu.com raring/main amd64 Packages
  404  Not Found [IP: 54.152.129.43 80]
Err http://us-east-1.ec2.archive.ubuntu.com raring/universe amd64 Packages
  404  Not Found [IP: 54.152.129.43 80]
Err http://us-east-1.ec2.archive.ubuntu.com raring-updates/main Sources
  404  Not Found [IP: 54.152.129.43 80]
Ign http://security.ubuntu.com raring-security/main Sources/DiffIndex
Err http://us-east-1.ec2.archive.ubuntu.com raring-updates/universe Sources
  404  Not Found [IP: 54.152.129.43 80]
Err http://us-east-1.ec2.archive.ubuntu.com raring-updates/main amd64 Packages
  404  Not Found [IP: 54.152.129.43 80]
Err http://us-east-1.ec2.archive.ubuntu.com raring-updates/universe amd64 Packages
  404  Not Found [IP: 54.152.129.43 80]
Hit http://ppa.launchpad.net raring Release.gpg
Ign http://security.ubuntu.com raring-security/universe Sources/DiffIndex
Ign http://security.ubuntu.com raring-security/main amd64 Packages/DiffIndex
Get:5 http://ppa.launchpad.net raring Release.gpg [316 B]
Ign http://security.ubuntu.com raring-security/universe amd64 Packages/DiffIndex
Hit http://ppa.launchpad.net raring Release
Hit http://ppa.launchpad.net raring Release
Get:6 http://ppa.launchpad.net raring Release [9,753 B]
Ign http://archive.cloudera.com squeeze-cdh3u5/contrib Translation-en_US
Hit http://ppa.launchpad.net raring/main Sources
Ign http://archive.cloudera.com squeeze-cdh3u5/contrib Translation-en
Hit http://ppa.launchpad.net raring/main amd64 Packages
Hit http://ppa.launchpad.net raring/main Sources
Hit http://ppa.launchpad.net raring/main amd64 Packages
Get:7 http://ppa.launchpad.net raring/main Sources [860 B]
Get:8 http://ppa.launchpad.net raring/main amd64 Packages [1,072 B]
Ign http://security.ubuntu.com raring-security/main Translation-en_US
Ign http://security.ubuntu.com raring-security/main Translation-en
Ign http://security.ubuntu.com raring-security/universe Translation-en_US
Ign http://security.ubuntu.com raring-security/universe Translation-en
Err http://security.ubuntu.com raring-security/main Sources
  404  Not Found [IP: 91.189.88.161 80]
Err http://security.ubuntu.com raring-security/universe Sources
  404  Not Found [IP: 91.189.88.161 80]
Err http://security.ubuntu.com raring-security/main amd64 Packages
  404  Not Found [IP: 91.189.88.161 80]
Err http://security.ubuntu.com raring-security/universe amd64 Packages
  404  Not Found [IP: 91.189.88.161 80]
Ign http://ppa.launchpad.net raring/main Translation-en_US
Ign http://ppa.launchpad.net raring/main Translation-en
Ign http://ppa.launchpad.net raring/main Translation-en_US
Ign http://ppa.launchpad.net raring/main Translation-en
Ign http://ppa.launchpad.net raring/main Translation-en_US
Ign http://ppa.launchpad.net raring/main Translation-en
Fetched 33.7 kB in 6s (5,163 B/s)
W: Failed to fetch http://us-east-1.ec2.archive.ubuntu.com/ubuntu/dists/raring/main/source/Sources  404  Not Found [IP: 54.152.129.43 80]

W: Failed to fetch http://us-east-1.ec2.archive.ubuntu.com/ubuntu/dists/raring/universe/source/Sources  404  Not Found [IP: 54.152.129.43 80]

W: Failed to fetch http://us-east-1.ec2.archive.ubuntu.com/ubuntu/dists/raring/main/binary-amd64/Packages  404  Not Found [IP: 54.152.129.43 80]

W: Failed to fetch http://us-east-1.ec2.archive.ubuntu.com/ubuntu/dists/raring/universe/binary-amd64/Packages  404  Not Found [IP: 54.152.129.43 80]

W: Failed to fetch http://us-east-1.ec2.archive.ubuntu.com/ubuntu/dists/raring-updates/main/source/Sources  404  Not Found [IP: 54.152.129.43 80]

W: Failed to fetch http://us-east-1.ec2.archive.ubuntu.com/ubuntu/dists/raring-updates/universe/source/Sources  404  Not Found [IP: 54.152.129.43 80]

W: Failed to fetch http://us-east-1.ec2.archive.ubuntu.com/ubuntu/dists/raring-updates/main/binary-amd64/Packages  404  Not Found [IP: 54.152.129.43 80]

W: Failed to fetch http://us-east-1.ec2.archive.ubuntu.com/ubuntu/dists/raring-updates/universe/binary-amd64/Packages  404  Not Found [IP: 54.152.129.43 80]

W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/raring-security/main/source/Sources  404  Not Found [IP: 91.189.88.161 80]

W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/raring-security/universe/source/Sources  404  Not Found [IP: 91.189.88.161 80]

W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/raring-security/main/binary-amd64/Packages  404  Not Found [IP: 91.189.88.161 80]

W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/raring-security/universe/binary-amd64/Packages  404  Not Found [IP: 91.189.88.161 80]

E: Some index files failed to download. They have been ignored, or old ones used instead.

Indeed, if I then attempt to install fastqc, apt-get complains of not being able to authenticate some packages:

# apt-get install fastqc
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following extra packages will be installed:
  libjbzip2-java libsam-java
Suggested packages:
  picard-tools
The following NEW packages will be installed:
  fastqc libjbzip2-java libsam-java
0 upgraded, 3 newly installed, 0 to remove and 1 not upgraded.
Need to get 547 kB of archives.
After this operation, 1,115 kB of additional disk space will be used.
Do you want to continue [Y/n]? 
WARNING: The following packages cannot be authenticated!
  libjbzip2-java libsam-java fastqc
Install these packages without verification [y/N]? n
E: Some packages could not be authenticated

Maybe StarCluster's default configuration for apt-get is out-of-date?

On my desktop, my StarCluster version is as follows:

% starcluster --version
StarCluster - (http://star.mit.edu/cluster) (v. 0.95.6)
Software Tools for Academics and Researchers (STAR)
Please submit bug reports to starcluster@mit.edu

0.95.6

My cluster's distro is:

% uname -ar
Linux master 3.8.0-35-generic #50-Ubuntu SMP Tue Dec 3 01:24:59 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

EDIT: FWIW, the following is the distro for my desktop (not the cluster):

% uname -ar
Linux xxx 3.16.0-4-amd64 #1 SMP Debian 3.16.43-2 (2017-04-30) x86_64 GNU/Linux

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions