In Docker, apt-get install fails with "Failed to fetch http://archive.ubuntu.com/ ... 404 Not Found" errors. Why? How can we get past it?

Question

My team uses Docker (with ubuntu:14.04 base image) for local development and we often have to rebuild some or all of our images. But we often get failures downloading packages with apt-get install, even immediately after running apt-get -y update. For instance, today I see

Err http://archive.ubuntu.com/ubuntu/ trusty-security/main libxml2 amd64 2.9.1+dfsg1-3ubuntu4.7
  404  Not Found [IP: 91.189.88.161 80]
Err http://archive.ubuntu.com/ubuntu/ trusty-security/main libxml2-dev amd64 2.9.1+dfsg1-3ubuntu4.7
  404  Not Found [IP: 91.189.88.161 80]
Fetched 84.7 MB in 1min 6s (1281 kB/s)
Unable to correct missing packages.
E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/libx/libxml2/libxml2_2.9.1+dfsg1-3ubuntu4.7_amd64.deb  404  Not Found [IP: 91.189.88.161 80]

E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/libx/libxml2/libxml2-dev_2.9.1+dfsg1-3ubuntu4.7_amd64.deb  404  Not Found [IP: 91.189.88.161 80]

E: Aborting install.

Apparently the specific version of a particular package has been deleted from the archive and replaced with a slightly differently named patch version. For instance, the above error is looking for libxml2_2.9.1+dfsg1-3ubuntu4.7_amd64.deb but the version on the server is libxml2_2.9.1+dfsg1-3ubuntu4.8_amd64.deb.

Often this is solvable by removing the base image (docker rmi ubuntu:14.04) and rebuilding; the newly downloaded ubuntu image has the correct patch number and finds the right archive file. But even this doesn't always work -- probably due to a delay between a new minor upgrade to Ubuntu's dependency db and the deployment of that new ubuntu:14.04 image onto Docker Hub.

We've tried using apt-get flags --fix-missing and --fix-broken and those don't consistently work either.

Any other ideas?

apt-get install fails with Not Found error because package removed from repository is a similar problem but the accepted answer is unacceptable because it's not possible to be automated. Our daily development process, including automatic build and deploy, is all scripted and using Docker and it's not practical to hack around inside a Dockerfile every time a particular archive goes missing (then remove the hack after a few hours or days).

---

In response to @prateek05, here's the /etc/apt/sources.list from the official ubuntu:14.04 docker image:

root@72daa1942714:/# cat /etc/apt/sources.list
# See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to
# newer versions of the distribution.

deb http://archive.ubuntu.com/ubuntu/ trusty main restricted
deb-src http://archive.ubuntu.com/ubuntu/ trusty main restricted

## Major bug fix updates produced after the final release of the
## distribution.
deb http://archive.ubuntu.com/ubuntu/ trusty-updates main restricted
deb-src http://archive.ubuntu.com/ubuntu/ trusty-updates main restricted

## Uncomment the following two lines to add software from the 'universe'
## repository.
## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team. Also, please note that software in universe WILL NOT receive any
## review or updates from the Ubuntu security team.
deb http://archive.ubuntu.com/ubuntu/ trusty universe
deb-src http://archive.ubuntu.com/ubuntu/ trusty universe
deb http://archive.ubuntu.com/ubuntu/ trusty-updates universe
deb-src http://archive.ubuntu.com/ubuntu/ trusty-updates universe

## N.B. software from this repository may not have been tested as
## extensively as that contained in the main release, although it includes
## newer versions of some applications which may provide useful features.
## Also, please note that software in backports WILL NOT receive any review
## or updates from the Ubuntu security team.
# deb http://archive.ubuntu.com/ubuntu/ trusty-backports main restricted
# deb-src http://archive.ubuntu.com/ubuntu/ trusty-backports main restricted

deb http://archive.ubuntu.com/ubuntu/ trusty-security main restricted
deb-src http://archive.ubuntu.com/ubuntu/ trusty-security main restricted
deb http://archive.ubuntu.com/ubuntu/ trusty-security universe
deb-src http://archive.ubuntu.com/ubuntu/ trusty-security universe
# deb http://archive.ubuntu.com/ubuntu/ trusty-security multiverse
# deb-src http://archive.ubuntu.com/ubuntu/ trusty-security multiverse

Answer 1

The issue could be potentially with the ubuntu sources

Check /etc/apt/sources.list

If you see deb http://archive.ubuntu.com/ubuntu main universe restricted multiverse , that could be the potential issue.

Fix that by replacing it with deb http://archive.ubuntu.com/ubuntu/ trusty main universe restricted multiverse

Alternatively it could be the mirror itself doesn't respond. archive.ubuntu.com

Name:   archive.ubuntu.com
Address: 91.189.88.152
Name:   archive.ubuntu.com
Address: 91.189.88.161
Name:   archive.ubuntu.com
Address: 91.189.88.149

Replace archive.ubuntu.com with a more trusted mirror say us.archive.ubuntu.com

Name:   us.archive.ubuntu.com
Address: 91.189.91.23
Name:   us.archive.ubuntu.com
Address: 91.189.91.26

---

(edit by the oiriginal asker):

Thanks, prateek05! My Dockerfile now starts:

FROM ubuntu:14.04
RUN sed -i'' 's/archive\.ubuntu\.com/us\.archive\.ubuntu\.com/' /etc/apt/sources.list
RUN apt-get -y update

and it seems to be working. But since this is a sporadic issue, only time will tell...

Answer 2

In my case the problem was caused by the parent image since it had not cleared the apt cache properly.

I solve the problem including cleaning commands before the first apt update ...

RUN apt clean && \
    rm -rf /var/lib/apt/lists/* && \
    apt update && \
    ...

Hope this helps

Answer 3

found something that works!

so i found this: https://www.mail-archive.com/[email protected]/msg5682159.html

The solution is to create a pinning_file with the contents

# Written by ubuntu-advantage-tools
Package: *
Pin: release o=UbuntuESM, n=trusty
Pin-Priority: never

then add

COPY pinning_file /etc/apt/preferences.d/ubuntu-esm-infra-trusty

to you Dockerfile before you run the sudo apt-get -y update

Answer 4

I was able to fix this error only after adding some extra arguments to apt-get to deal with http issues:

sudo apt-get \
  -o Acquire::BrokenProxy="true" \
  -o Acquire::http::No-Cache="true" \ 
  -o Acquire::http::Pipeline-Depth="0" install \
    ignition

Answer 5

What worked for me ...

I had the statements in two lines, but when i merged to one it worked (don't know if it was cache related ...)

it was

RUN apt-get -y update && apt-get upgrade -y

# Install tools && libraries
RUN apt-get -y install --fix-missing apt-utils iputils-ping nano wget dialog \
    build-essential git zip  \

And then i changed to

RUN apt-get -y update && apt-get upgrade -y \
    && apt-get -y install --fix-missing apt-utils nano wget \
    git zip  \
    mysql-client \