'H5PY storage device disconnections management
I have a Python program that collects data from electronic devices at a defined frequency. Each time I get new data, I have to store them into an HDF5 file that must be saved on a USB drive.
I want to manage this drive disconnections, but every time I unplug the USB drive while h5py has a handle on it, it crashes without letting me catch the exception.
Below is an example of the behaviour:
import random
import h5py
try:
h5f = h5py.File(r"E:\data.h5", "a")
try:
dset = h5f["dataset"]
except KeyError: # dset does not exist yet => create it
h5f.create_dataset("dataset", (0,), maxshape=(None,), chunks=(1024,))
dset = h5f["dataset"]
input("waiting...")
dset.resize((dset.shape[0] + 1,))
dset[-1] = random.random()
except Exception:
print("exception raised")
else:
h5f.close()
If I run this script and unplug the E: drive when waiting... is displayed, then I got the following output:
$ python tmp.py
waiting...
exception raised
Traceback (most recent call last):
File "h5py\_objects.pyx", line 193, in h5py._objects.ObjectID.__dealloc__
RuntimeError: Can't decrement id ref count (file write failed: time = Wed May 11 16:09:20 2022
, filename = 'E:\data.h5', file descriptor = 3, errno = 22, error message = 'Invalid argument', buf = 000002573736C038, total write size = 272, bytes this sub-write = 272, bytes actually written = 18446744073709551615, offset = 800)
Exception ignored in: 'h5py._objects.ObjectID.__dealloc__'
Traceback (most recent call last):
File "h5py\_objects.pyx", line 193, in h5py._objects.ObjectID.__dealloc__
RuntimeError: Can't decrement id ref count (file write failed: time = Wed May 11 16:09:20 2022
, filename = 'E:\data.h5', file descriptor = 3, errno = 22, error message = 'Invalid argument', buf = 000002573736C038, total write size = 272, bytes this sub-write = 272, bytes actually written = 18446744073709551615, offset = 800)
When this happens, I have not found any way to catch the h5py exceptions in order to keep my program alive and continue processing.
Is there anything I can do?
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
