'How to write an xarray object to s3 in an asynchronous routine?
I have an xarray object in an asynchronous task, and would like to write it to s3. I thought that to_zarr would do the trick, as zarr uses fsspec, which supports async in s3fs. However, how to use? It seems that to_zarr actually writes something synchronously immediately even when compute=True? And then I get a dask object... is there any way to avoid dask?
I am writing relatively small arrays -- but lots of them. I am receiving data from several websocket streams, filling into fixed chunks, and then triggering a task to write full tasks to s3. How can I avoid blocking operations?
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
