'How to update HDF5 table with partial data?
I'm wondering how one might update an HDF5 table when one only has partial data? For example, suppose the following df is stored in an HDF5 table.
import pandas as pd
df = pd.DataFrame([
[100,90,80,70,36,45],
[101,78,65,88,55,78],
[92,77,42,79,43,32],
[103,98,76,54,45,65]],
index = pd.date_range(start='2022-01-01', periods=4)
)
df.columns = pd.MultiIndex.from_tuples(
(("mkf", "Open"),
("mkf", "Close"),
("tdf", "Open"),
("tdf","Close"),
("ghi","Open"),
("ghi", "Close"))
)
df
mkf tdf ghi
Open Close Open Close Open Close
2022-01-01 100 90 80 70 36 45
2022-01-02 101 78 65 88 55 78
2022-01-03 92 77 42 79 43 32
2022-01-04 103 98 76 54 45 65
store = pd.HDFStore('store.h5')
store.append('data', df)
Next, suppose I obtain partial data (e.g. data for mkf and tdf but not ghi).
df1 = pd.DataFrame([
[70,80,90,70],
[91,68,45,88],
[92,47,32,79],
[43,38,77,74]],
index = pd.date_range(start='2022-01-05', periods=4)
)
df1.columns = pd.MultiIndex.from_tuples((("mkf", "Open"),
("mkf", "Close"),
("tdf", "Open"),
("tdf","Close"),
)
)
df1
mkf tdf
Open Close Open Close
2022-01-05 70 80 90 70
2022-01-06 91 68 45 88
2022-01-07 92 47 32 79
2022-01-08 43 38 77 74
How can I update store? I tried the following but got a ValueError:
store.append('data',df1)
ValueError: cannot match existing table structure for [(mkf, Open),(mkf, Close),(tdf, Open),(tdf, Close),(ghi, Open),(ghi, Close)] on appending data
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
