I'm trying to remove certain words from a column on each row in my dataframe

Question

I'm still trying to understand how pandas works please bear with me. In this exercise, I,m trying to access a particular column ['Without Stop Words'] on each row which has a list of words. I wish to remove certain words from each row of that column. the words to be removed have been specified in a dictionary called {'stop_words_dict'}. here's my code, but the dataframe seems to be unchanged after running it.

def stop_words_remover(df):
    # your code here
    df['Without Stop Words']= df['Tweets'].str.lower().str.split()
    for i, r in df.iterrows():
        for word in df['Without Stop Words']:
            if word in stop_words_dict.items():
                df['Without Stop Words'][i] = df['Without Stop Words'].str.remove(word)
                
    return df

this is how the input looks like

INPUT EXPECTED OUTPUT

Answer 1

In Pandas, it's generally a bad idea to loop over your dataframe row by row to try to change rows. Instead, try using methods like .apply().

An example for stopwords, together with list comprehension:

test['Tweets'].apply(lambda x: [item for item in x if item not in stop_words_dict.items()])

See https://stackoverflow.com/a/29523440/12904151 for more context.