'How can I convert a list of lists in a Dataframe in Pyspark?
I have a list of list of type:
[[1, 'A', 'aa'], [2 'B', 'bb'], [3, 'C', 'cc']]
I want to get the next dataframe:
| id | col1 | col2 |
|---|---|---|
| 1 | A | aa |
| 2 | B | bb |
| 3 | C | cc |
This solution is somewhat similar ,but list has data from same columns https://stackoverflow.com/a/46891396/6463651
Solution 1:[1]
Just use the list to create the dataframe directly.
data = [[1, 'A', 'aa'], [2, 'B', 'bb'], [3, 'C', 'cc']]
df = spark.createDataFrame(data, ['id', 'col1', 'col2'])
df.show(truncate=False)
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | 过过招 |
