'Stop Pandas from converting int to float due to an insertion in another column
I have a DataFrame with two columns: a column of int and a column of str.
- I understand that if I insert
NaNinto theintcolumn, Pandas will convert all theintintofloatbecause there is noNaNvalue for anint. - However, when I insert
Noneinto thestrcolumn, Pandas converts all myinttofloatas well. This doesn't make sense to me - why does the value I put in column 2 affect column 1?
Here's a simple working example):
import pandas as pd
df = pd.DataFrame()
df["int"] = pd.Series([], dtype=int)
df["str"] = pd.Series([], dtype=str)
df.loc[0] = [0, "zero"]
print(df)
print()
df.loc[1] = [1, None]
print(df)
The output is:
int str
0 0 zero
int str
0 0.0 zero
1 1.0 NaN
Is there any way to make the output the following:
int str
0 0 zero
int str
0 0 zero
1 1 NaN
without recasting the first column to int.
I prefer using
intinstead offloatbecause the actual data in that column are integers. If there's not workaround, I'll just usefloatthough.I prefer not having to recast because in my actual code, I don't
store the actualdtype.I also need the data inserted row-by-row.
Solution 1:[1]
As of pandas 1.0.0 I believe you have another option, which is to first use convert_dtypes. This converts the dataframe columns to dtypes that support pd.NA, avoiding the issues with NaN/None.
...
df = df.convert_dtypes()
df.loc[1] = [1, None]
print(df)
# int str
# 0 0 zero
# 1 1 NaN
Solution 2:[2]
If you use DataFrame.append to add the data, the dtypes are preserved, and you do not have to recast or rely on object:
In [157]: df
Out[157]:
int str
0 0 zero
In [159]: df.append(pd.DataFrame([[1, None]], columns=['int', 'str']), ignore_index=True)
Out[159]:
int str
0 0 zero
1 1 None
Solution 3:[3]
right after
df = pd.DataFrame()
add the below and it will initialize the entire series to int. This worked for me.
df['int'] = 0
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | totalhack |
| Solution 2 | fuglede |
| Solution 3 | QuentinJS |
