'PySpark Union throws IllegalArgumentException: transpose requires all collections have the same size
it's the first time I've seen this error when trying to union two Spark Dataframes. I am trying to extract fraud records from existing data and then take a difference to get non-fraud cases and union the two but get the following error:
Here's a snapshot of the DFs:
Happy to provide more info on DF operations if needed. PySpark: 2.4.5 Python: 3.6.9
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|