Docker.service fails to start, error "failed to mount overlay: no such device" on Raspbian

Question

I recently had a problem with my raspbian installation. Probably some upgrade messed up my SSD boot. After resolving it, I noticed that docker.service refuses to start. On the service's log it says

Apr 23 19:24:08 raspberrypi systemd[1]: Starting Docker Application Container Engine...
Apr 23 19:24:08 raspberrypi dockerd[1629]: time="2020-04-23T19:24:08.702984211+01:00" level=info msg="Starting up"
Apr 23 19:24:08 raspberrypi dockerd[1629]: time="2020-04-23T19:24:08.706657044+01:00" level=info msg="parsed scheme: \"unix\"" module=grpc
Apr 23 19:24:08 raspberrypi dockerd[1629]: time="2020-04-23T19:24:08.707181266+01:00" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
Apr 23 19:24:08 raspberrypi dockerd[1629]: time="2020-04-23T19:24:08.707520970+01:00" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock 0  <
Apr 23 19:24:08 raspberrypi dockerd[1629]: time="2020-04-23T19:24:08.707855840+01:00" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
Apr 23 19:24:08 raspberrypi dockerd[1629]: time="2020-04-23T19:24:08.711273840+01:00" level=info msg="parsed scheme: \"unix\"" module=grpc
Apr 23 19:24:08 raspberrypi dockerd[1629]: time="2020-04-23T19:24:08.711450507+01:00" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
Apr 23 19:24:08 raspberrypi dockerd[1629]: time="2020-04-23T19:24:08.711545396+01:00" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock 0  <
Apr 23 19:24:08 raspberrypi dockerd[1629]: time="2020-04-23T19:24:08.711626674+01:00" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
Apr 23 19:24:08 raspberrypi dockerd[1629]: time="2020-04-23T19:24:08.721460748+01:00" level=error msg="failed to mount overlay: no such device" storage-driver=overlay2
Apr 23 19:24:08 raspberrypi dockerd[1629]: time="2020-04-23T19:24:08.721569822+01:00" level=error msg="[graphdriver] prior storage driver overlay2 failed: driver not supported"
Apr 23 19:24:08 raspberrypi dockerd[1629]: failed to start daemon: error initializing graphdriver: driver not supported
Apr 23 19:24:08 raspberrypi systemd[1]: docker.service: Main process exited, code=exited, status=1/FAILURE
Apr 23 19:24:08 raspberrypi systemd[1]: docker.service: Failed with result 'exit-code'.
Apr 23 19:24:08 raspberrypi systemd[1]: Failed to start Docker Application Container Engine.

I wasn't able to find anything relevant online. Ideally I would like to avoid losing my docker images

Edit: The kernel version was 4.19.114-v7l+, however there were no modules installed for this kernel under /lib/module. Running a "sudo apt install --reinstall raspberrypi-bootloader raspberrypi-kernel" fixed it.

Answer 1

In case you have an old kernel, update the kernel.

Check if modules are installed for the kernel under /lib/module.

sudo apt install --reinstall raspberrypi-bootloader raspberrypi-kernel

should fix it.

I had the same problem in a VM on a Xen host, and fixed it by installing pygrub and its own kernel inside the VM: https://stackoverflow.com/a/71763164/1069083

Answer 2

Your kernel might not support overlayl2:

1. Try to upgrade your kernel (check if it has support for overlay2)
2. Or use another driver like aufs or devicemapper for Debian > 9, and overlay for Ubuntu.

NB.: devicemapper is tagged as deprecated for Docker.

https://docs.docker.com/storage/storagedriver/select-storage-driver/

After your changes, you also might have to remove /var/lib/docker before restarting the service.

Good luck !