'Deleting HDF file if writing does not complete (Pandas)?
I am saving a Dataframe as an HDF file in Pandas, using:
store = pandas.HDFStore('myfile.h5', table=True)
store.append('df', df, chunksize=100000)
store.close()
I have this code in a Luigi pipeline, and it works fine - except when there is some error in the writing process (such as a mixed type column), such that myfile.h5 appears on the drive but isn't fully written. Luigi thinks that task is complete, and then the next task fails because the myfile.h5 isn't written fully.
How can I tell Python to delete myfile.h5 if store.append doesn't fully complete?
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|