'Pyspark : How to concat two dataframes in Pyspark
df_1 :
| NBB1 |
|---|
| 776 |
And df_2
| NBB2 |
|---|
| 4867 |
I will to obtain this dataframe in Pyspark df :
| NBB1 | NBB2 |
|---|---|
| 776 | 4867 |
Solution 1:[1]
You need to perform a crossJoin between the two dataframes.
See below for details -
from pyspark.sql import Row
df1 = spark.createDataFrame([Row(NBB1 = 776)])
df1.show()
#Output
+----+
|NBB1|
+----+
| 776|
+----+
df2 = spark.createDataFrame([Row(NBB2 = 4867)])
df2.show()
#Output
+----+
|NBB2|
+----+
|4867|
+----+
df1.crossJoin(df2).show()
#Output
+----+----+
|NBB1|NBB2|
+----+----+
| 776|4867|
+----+----+
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | DKNY |