Entire Pandas Column Read in as NAN from read_csv()

Question

I'm having trouble reading in my CSV file with Pandas. It's reading in a stock dataset which contains the Date and OHLCV columns, but for some reason, it's reading in the Open column as all Nan's.

I've read in other posts that this could be a "non-UTF-8" encoding issue, so I've since re-encoded it to UTF-8 via the Notepad++ route, but still the same problem. When I print the dtypes of the columns, they all say float64 (except for volume as those are integers) which is good, so why would I be getting all nan's for the Open column, if they ARE float64's?

To reproduce, download the dataset I'm using from my google drive link here, and then place it in the same folder as this below minimal example script. I imagine it's got something to do with this dataset, but I've used it elsewhere without this issue before:

from pandas import read_csv, to_datetime

# ---------------------------------------------------------------------------- #
#                            Import/Prepare Dataset                            #
# ---------------------------------------------------------------------------- #
# Read in df
datapath = "./SPY5min.csv"
df = read_csv(datapath)
df['time'] = to_datetime(df['time'])
# Rename the columns
df = df.rename(columns={"time": "Datetime",
                        "open":" Open",
                        "high": "High",
                        "low": "Low",
                        "close": "Close",
                        "volume": "Volume"})
# Set the index
df = df.set_index("Datetime")
# For some reason, "Open" isn't being recognized as a column,
# so using this below fix to make it "seen" (though I suspect
# this is where the problem originates)
df = df.reindex(columns=['Open', 'High', 'Low', 'Close', 'Volume'])
# Preview the dtypes of the columns
print(df.dtypes)
# Attempt at a fix to convert open column to float64, same as the
# others
df['Open'] = df['Open'].astype('float64')
print(df.isnull().sum())
# ------------------------ End Import/Prepare Dataset ------------------------ #

https://drive.google.com/file/d/1TuUmnLr_8lQ8beZFDahjMGEsC07bJEO1/view?usp=sharing

Answer 1

Looks like there's a space before Open in your df.rename code, e.g. " Open" which should be "Open"