'Parallel upload of files from a directory in Google Cloud Storage

I'm trying to write a Python utility function which uploads files from a directory to a destination folder. Although, the documentation provides a snippet with blob.upload_from_filename() for a single file and then it jumps to gsutil -m solution for multiple files.

So far, I have found only 2 suboptimal solutions:

  1. Simple iteration of the single file operation
  2. Use of subprocess and hardcode the gsutil command.

Is there any elegant solution for this issue or should I use Python's multiprocessing?



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source