Why and when use append() instead of concat() in Pandas?

Question

I have read all topics on append vs concat here, but still: Why should I use append if concat has the same options and funcionallity?

Please correct me if I am mistaken.

• append() : can append two data frames at once; it's the same as concat in axis=0
• concat() : can concatenate multiple DFs

To concatenate multiple DFs, should I use append() with a for-loop? Is it faster?

Let's assume that I am opening DFs from different files like:

df = pd.DataFrame()
for file in file_folder:
    df = df.append(pd.read_csv(file))

OR 

df = pd.DataFrame()
for file in file_folder:
   df = pd.concat([df, pd.read_csv(file)])

The output is the same. So why?

EDIT: to speed it up should I do:

df_list = []
for file in file_folder:

    df_list.append(pd.read_csv(file))

#and then use concat

df_all = pd.concat(df_list)`

right?

Answer 1

append is a convenience method which calls concat under the hood. If you look at the implementation of the append method, you will see that.

def append(...
    ...
    if isinstance(other, (list, tuple)):
        to_concat = [self, *other]
    else:
        to_concat = [self, other]
    return concat(
        to_concat,
        ignore_index=ignore_index,
        verify_integrity=verify_integrity,
        sort=sort,
    )

As for the performance. Both of these called over and over in a loop can be computationally expensive. You should just create a list and do one concatenation after you are done looping.

From docs:

iteratively appending rows to a DataFrame can be more computationally intensive than a single concatenate. A better solution is to append those rows to a list and then concatenate the list with the original DataFrame all at once.

Answer 2

1. It is an arbitrary choice to either use .append() or .concat()
   For example if you have to merge two dataframes on axis=1 then go with concat and on the other hand if you have to merge two dataframes on axis=0 go with whatever choice you prefer to use or works best for you!
   

2. Personally I would reccomend you to go with the pandas default DF merger(concat) to avoid any delays, latencies and most importantly bugs in you code.
   

Hope so that you find this information useful and answers you question!
Happy Coding!