'condition should be a Column dataframe PySpark
When using df_hdr_join.count() > 0 in when statement, it gives an error 'condition should be a Column'. I tried following.
df_result = df.withColumn('NUM', when(df_lbr_join.count() > 0, lit(monotonically_increasing_id())).otherwise(when(df_hdr_join.count() > 0, lit(monotonically_increasing_id())).otherwise(lit(None))))
I need to check the size of the dataframe inside when statement and increment the value. How this should be done?
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|