Reindexing only valid with uniquely valued Index objects

Question

Irun this code

esg_fm_barron = pd.concat([barron_clean.drop(columns = "10 year return", inplace = False),ESG_fixed.drop(columns = 'Name',inplace = False), financial_clean.drop(columns = 'Name',inplace = False)], axis = 'columns', join = 'inner')
esg_fm_barron.rename(columns={'Average (Current)': "Total ESG Score"}, inplace=True)
esg_fm_barron.head(3)

but is get this error : Reindexing only valid with uniquely valued Index objects does anyone know a solution ? thanks

Answer 1

When you run pd.concat, each source DataFrame must have unique index.

First identify which source DataFrame has a non-unique index. For each source DataFrame (assuming it is df) run:

df.index.is_unique()

The false result means that the index in this DataFrame is non-unique. Then drop rows with duplicated index values:

df = df.loc[~df.index.duplicated(keep='first')]

In order not to loose original data, maybe you should save the result under a new temporary DataFrame and then use for concatenation these temporary DataFrames.