'How spark.sql.shuffle.partitions affect the results?
When I was set spark.sql.shuffle.partitions to 3, in the unit test I can get the result I want, but when I set it to 1, the results would change, seems like it will cause the data loss, how can I avoid this problem if I need spark.sql.shuffle.partitions is 1
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|