'Docker container can’t startup with a 166G(nfs/efs) volume

my docker container can’t startup with a 166G (nfs/efs) volume, I use the EFS mounted to the EC2 with amazon-efs-utils tool, and then volume to the docker, below is my docker-compose file

services:      
    app:
        image: xxxx
        restart: always
        environment:
            - ASPNETCORE_ENVIRONMENT=Production
        ports:
            - "8002:8002"
        volumes:
            - "./BinaryObjects/Static:/app/App_Data/BinaryObjects/Static"
            - "./BinaryObjects/Temp:/app/App_Data/BinaryObjects/Temp"

the /BinaryObjects is the root which EFS mounted to, and /Static folder has the 166G files, and /Temp folder has 10M files, if I remove the /Static in volumes, and the docker container can startup. so I think it is not the permission issue. I did the same thing in our staging env, and it works well, the different between staging and production is the /Static folder in staging has only 2G files. I have tried that created the nfs volumes first,

docker volume create \
    --driver local \
    --opt type=nfs \
    --opt o=addr=xxxx.amazonaws.com,nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport \
    --opt device=:/Static 
    efs-static

docker volume create \
    --driver local \
    --opt type=nfs \
    --opt o=addr=xxxx.amazonaws.com,nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport \
    --opt device=:/Temp 
    efs-temp

and no luck, the docker container can't startup either, and works if I remove the efs-static volume, it is just stuck, and no error message, please refer to below screenshots. image

Does someone know why? I want the docker can startup with my 166G efs volume. Thanks.

my docker info

Client:
 Context:    default
 Debug Mode: false

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 31
 Server Version: 20.10.7
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version:
 runc version:
 init version:
 Security Options:
  apparmor
  seccomp
   Profile: default
 Kernel Version: 5.4.0-1059-aws
 Operating System: Ubuntu 18.04.6 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 15.18GiB
 Name: ip-172-31-50-32
 ID: XWJL:SUKJ:DVZL:SVII:JFHJ:W7ND:LZJ5:2LKU:ASBA:ILEB:HF54:KLCV
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false


Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source