'Text is not interpreting the null in source dataframe(csv) as blank/null in the resultant dataframe(txt.)

I read the source data from a CSV file to a dataframe. Some columns have null values as seen in dataframe A.

Now, dataframe B is the resultant data frame while writing the data in text. As you see, the text format gives an error. The null values are ignored and have been merged in a single column which is correct.

However, dataframe C is what I am expecting. Are there any ways through which I could achieve dataframe C?

Dataframes A, B, C are below:

sample data

A)
+---+---+---+----+----+----+----+--------+
|_c0|_c1|_c2| _c3| _c4| _c5| _c6|     _c7|
+---+---+---+----+----+----+----+--------+
|abc|  1|  2|null|null|null|null|10/12/22|
+---+---+---+----+----+----+----+--------+

B)
+----------------+
|            data|
+----------------+
|abc,1,2,10/12/22|
+----------------+

However, this is what I am expecting.
C)
+--------------------+
|            data    |
+--------------------+
|abc,1,,,,,2,10/12/22|
+--------------------+

code snippet

file_location = "/FileStore/tables/sample_data.csv"
file_type = "csv"
infer_schema = "false"
first_row_is_header = "false"
delimiter = "," 

df = spark.read.format(file_type)
  .option("inferSchema", infer_schema)
  .option("header", first_row_is_header)
  .option("sep", delimiter)
  .load(file_location)
df.show() # dataframe A

from pyspark.sql.functions import concat_ws
single_column_df = df.select(concat_ws(",", *df.columns).alias('data'))
destination = "/FileStore/tables/output10.txt"
file_format = "text"
single_column_df.write.format(file_format).save(destination)
single_column_df.show() # dataframe B


Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source