'Writing a long array into a CSV column
Currently I wish to create a CSV dataset for use later on (as memory issues prevent other things), with three columns: id, category, signal.
ID and category are both integers while signal is a float32 array with length ~10001 and shape ~(10001, 401) for most cases.
I can create a dataframe for each of the samples easily as follows:
pd.DataFrame([{'id': i, 'category': y, 'signal': signal}])
However when calling to_csv('../dataset.csv', mode='a', header=False, index=False) it produces the output:
0,7,[[ 1.24496163e-03+0.0000000e+00j 1.17749616e-04+0.0000000e+00j
8.92068347e-05+0.0000000e+00j ... -3.55575146e-04+0.0000000e+00j
-3.53000971e-04+0.0000000e+00j -1.35809695e-03+0.0000000e+00j]
[-1.20925368e-03+2.0470628e-05j -5.52154524e-05-3.7754296e-05j
-7.05739149e-05-7.2288349e-05j ... 2.85753951e-04+4.1393949e-05j
1.60366966e-04-2.7969104e-04j 1.15900207e-03+2.3369875e-04j]
[ 1.19788689e-03+3.3121873e-05j -6.23480264e-06+4.2733776e-05j
4.23972924e-05+8.2590450e-05j ... -1.44452570e-04+8.9506881e-05j
-1.42675699e-05+1.8458923e-04j -9.61950631e-04-1.2690650e-04j]
...
[-1.11952168e-05+1.8826026e-06j 3.46628576e-06-2.0878474e-06j
-1.54314409e-06-2.8386687e-06j ... -1.41124383e-06-2.3261330e-06j
-5.40845519e-07-2.5624531e-06j 1.46134671e-05-1.2955890e-06j]
[ 8.70186886e-06-2.8798408e-06j -3.47046830e-06+4.1321928e-06j
1.99560964e-06-2.9683741e-07j ... -3.09421989e-06+8.9018937e-07j
-1.19136712e-06+2.9354976e-06j -1.56483602e-05+6.8048553e-07j]
[-6.49670437e-06+0.0000000e+00j 4.49650315e-06+0.0000000e+00j
-2.71528188e-06+0.0000000e+00j ... 6.58240697e-06+0.0000000e+00j
3.21866310e-06+0.0000000e+00j 1.57587510e-05+0.0000000e+00j]]
With the long array on multiple lines and most of the array "cut out". Is there potentially a solution to this?
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
