'pyspark.sql.functions.lit() not nullable conversion [duplicate]
As I create a new column with F.lit(1), while calling printSchema() I get
column_name: integer (nullable = false)
as lit function docs is quite scarce, do you think there is any simple mapping that can be done to turn it into nullable = true?
Solution 1:[1]
Okay, in this scenario (only some specific column mapping, nothing in bulk) it seems like
df.schema['column_name'].nullable = True
does the trick. Nevertheless df.printSchema() isn't updated, although df.schema is.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | mckraqs |