'How can I visualize a large data using python libraries from a spark.sql.dataframe?
I have a problem in visualizing data using matplotlib, my project is for Bigdata Analytics, Now i have a spark dataframe that has the result but when i want to convert this df to pandas.df, it cannot collect all the data because is too large. So I cannot do the visualization without converting spark.df to pandas.df. When i search I found that we should do partitions in spark.df or something like this.... So please if someone knows how to visualize a large amount of data from spark.df, help me ! Thank You.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|