'ValueError: setting an array element with a sequence in pyspark.pandas

I am trying to migrate pandas to pyspark.pandas library.

The below code was working fine with pandas library.

FYI. pdf is a pyspark.pandas dataframe.

While running the below code :

for x in ["a", "b", "c"]: 
   pdf[x] = np.where( 
     (pdf["p"] <=constants.100) & (pdf["z"]=="ABC" ), pdf["k"], pdf[x] )

I got the below error :

File "<__array_function__ internals>", line 6, in where
ValueError: setting an array element with a sequence.

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Solution	Source