'Python Pandas DataFrame stops working after certain row
I have a few programs where I would like to import data from CSV files, do some calculations and then output to a new CSV.
However, when I run the following code and the input file is larger than ~ 100 rows, the code stops working after row 29 and then resumes at some point later on.
Note, the ".. .." you see in the output are in fact in the output file. This is not a placeholder. The dataframe creation seems to stop at 29 and then resumewith 156. In between nothing but the one row with "..".
If I use an input file with say 40 rows only, then those work correctly.
Can someone help me out here? Not sure where the problem is.
Thank you all for your help
Alex
import csv
import pandas as pd
from tkinter import *
from tkinter import filedialog as fd
root = Tk()
ImportFile = fd.askopenfilename()
with open(ImportFile, newline='') as csvfile:
reader = csv.DictReader(csvfile)
df = pd.DataFrame(columns=["ColA", "ColB", "ColC", "ColD"])
for row in reader:
global ColA
ColA = float(row['ColA'])
global ColB
ColB = float(row['ColB'])
global ColC
ColC = float(row['ColC'])
header = ['ColA', 'ColB', 'ColC', 'ColD']
data = [ColA, ColB, ColC, "0"]
df.loc[len(df)] = data
print(df)
f = open("batchoutput.csv", "w", encoding="UTF-8")
f.write(str(df))
f.close()
root.destroy()
Output:
ColA ColB ColC ColD
0 10.0 10.0 164.00 0
1 10.0 11.0 176.00 0
2 10.0 11.0 171.00 0
3 10.0 617.0 183.00 0
4 10.0 12.0 18.00 0
5 10.0 618.0 182.00 0
6 10.0 607.0 175.00 0
7 10.0 615.0 184.00 0
8 10.0 616.0 187.00 0
9 10.0 614.0 18.00 0
10 10.0 616.0 19.00 0
11 10.0 616.0 18.00 0
12 10.0 865.0 168.00 0
13 10.0 869.0 134.00 0
14 10.0 868.0 138.00 0
15 10.0 10.0 164.00 0
16 10.0 11.0 176.00 0
17 10.0 11.0 171.00 0
18 10.0 617.0 183.00 0
19 10.0 12.0 18.00 0
20 10.0 618.0 182.00 0
21 10.0 607.0 175.00 0
22 10.0 615.0 184.00 0
23 10.0 616.0 187.00 0
24 10.0 614.0 18.00 0
25 10.0 616.0 19.00 0
26 10.0 616.0 18.00 0
27 10.0 11.0 199.00 0
28 10.0 618.0 188.00 0
29 10.0 10.0 198.00 0
.. ... ... ... ...
157 10.0 618.0 188.00 0
158 10.0 10.0 198.00 0
159 10.0 11.0 3.07 0
160 10.0 617.0 151.00 0
161 10.0 619.0 156.00 0
162 10.0 616.0 17.00 0
163 10.0 619.0 16.00 0
164 10.0 13.0 153.00 0
165 10.0 618.0 11.00 0
166 10.0 616.0 17.00 0
167 10.0 614.0 167.00 0
168 10.0 615.0 171.00 0
169 10.0 616.0 185.00 0
170 10.0 618.0 189.00 0
171 10.0 11.0 174.00 0
172 10.0 861.0 11.00 0
173 10.0 856.0 174.00 0
174 10.0 863.0 172.00 0
175 10.0 81.0 173.00 0
176 10.0 861.0 169.00 0
177 10.0 81.0 17.00 0
178 10.0 864.0 169.00 0
179 10.0 863.0 172.00 0
180 10.0 870.0 182.00 0
181 10.0 864.0 18.00 0
182 10.0 863.0 18.00 0
183 10.0 81.0 17.00 0
184 10.0 865.0 168.00 0
185 10.0 869.0 134.00 0
186 10.0 868.0 138.00 0
[187 rows x 4 columns]
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
