'Set user defined value in Spark dataframe/dataset

I have a dataframe in which few columns may have some missing values and these columns are not of same type i.e. they may be of StringType, DoubleType, IntegerType or any other possible data type.

Right now i am trying to set missing values for Double type of columns like below code:

double missingValue = Double.parseDouble(userDefinedvalue);
dataset = dataset.na().fill(missingValue, columns);

Is there any best way which work for any dataType of columns or combination of any dataType of column

javaapache-sparkapache-spark-sqlapache-spark-dataset


Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source

Related Questions