read_csv() got an unexpected keyword argument 'on_bad_lines'

Question

I am trying to read multiple dataframes in an list like the following:

for i in excels:
    df2 = pd.read_csv(i, on_bad_lines='skip')
    dfs.append(df2)

and it is working fine while running it locally but when on the deployement it shows the following error:

TypeError at /rfid-dumpdownload/
read_csv() got an unexpected keyword argument 'on_bad_lines'

on_bad_lines is even in the documentation then why is it not accepting it ?

Answer 1

Reason is use older pandas version, under pandas 1.4.0:

on_bad_lines{‘error’, ‘warn’, ‘skip’} or callable, default ‘error’

Specifies what to do upon encountering a bad line (a line with too many fields). Allowed values are :

        ‘error’, raise an Exception when a bad line is encountered.

        ‘warn’, raise a warning when a bad line is encountered and skip that line.

        ‘skip’, skip bad lines without raising or warning when they are encountered.

New in version 1.3.0:

    callable, function with signature (bad_line: list[str]) -> list[str] | None that will process a single bad line. bad_line is a list of strings split by the sep. If the function returns None, the bad line will be ignored. If the function returns a new list of strings with more elements than expected, a ParserWarning will be emitted while dropping extra elements. Only supported when engine="python"

New in version 1.4.0.