How to use gsutil in startup-script on container-os image in GCP?

Question

I'm creating my GCE instance with gcloud beta compute instances create-with-container and the following startup-script:

#! /bin/bash
gsutil cp gs://artifacts.<PROJECT-ID>.appspot.com/some-file .
docker-credential-gcr configure-docker
docker pull gcr.io/<PROJECT-ID>/image:tag
docker run --name compose --rm -v $PWD:$PWD -w $PWD -v /var/run/docker.sock:/var/run/docker.sock docker/compose:1.26.0 up -d --force-recreate

but I am ending with the following:

Jun 23 15:22:08 app startup-script[389]: INFO Found startup-script in metadata.
Jun 23 15:22:08 app startup-script[389]: INFO startup-script: /var/lib/google/startup-DqGmUt/tmpPyYsVd: line 2: gsutil: command not found
Jun 23 15:22:09 app startup-script[389]: INFO startup-script: ERROR: Unable to save docker config: mkdir /root/.docker: read-only file system

I tried installing gsutil also in the startup-script but it bails out with the same problem of /root being RO. Is it actually possible to do this on a COS image?

Answer 1

As the documentation [1] it seems gsutil tool runs on Linux/Unix, MAC OS and also in windows. So it seems COS image don't support gsutil. Also I hope link [2] and [3] could be helpful.

Please have documentation [4] as there have some limitation for COS image.

[1] https://cloud.google.com/storage/docs/gsutil_install#specifications

[2] `gsutil cp` from Storage to compute instance running container doesn't copy files

[3] https://github.com/GoogleCloudPlatform/gsutil/issues/453

[4] https://cloud.google.com/container-optimized-os/docs/concepts/features-and-benefits#limitations

Answer 2

There is a way to use tools like gsutil and gcloud in a Container Optimized OS. You can access the toolbox to use gsutil to access buckets and then copy the downloaded data form the toolbox directory to your actual VM.

As soon as you started the toolbox you can exchange files between the toolbox and your vm with either /var/lib/toolbox/USER-gcr.io_google-containers_toolbox-20200404-00/ to get data from the toolbox while in your vm or /media/root/ while in the toolbox to get files from the vm.

More information https://cloud.google.com/container-optimized-os/docs/how-to/toolbox

Answer 3

For gcloud beta compute instances create with --create-disk=...image=projects/cos-cloud/global/images/cos-85-13310-1416-9,..., I used a containerized gsutil in the startup script:

#!/bin/bash
docker run -v /tmp:/tmp google/cloud-sdk:slim gsutil cp gs://BUCKET/file /tmp/
cat /tmp/file