'How do I get the datatype of a column in apache spark using java
I want to get the datatype of a particular column in Apache Spark using java. Majority of the implementations I have been seeing are done in Scala and it is not working for java.
Here is what I have.
String[] preTasks = (String[]) JSONObject.parseObject(taskNode.getPreTasks(), List.class).toArray(new String[0]);
Dataset<Row> dataset = SparkDataframeUtil.loadDatasetRows(buildPath(preTasks[0]), sparkSession);
I want to get the data type of the column selected by the user on the frontend. Let's say the user selected a column in a dataset. Let's call that column getColumn, if that column is a string, I don't want to perform some arithmetic function else, I want to perform arithmetic function. dtypes is not working and schema('columnName').dataType.typeName is not working either.
In summary, I want to determine if a datatype of a column is a string, an integer or double or a boolean.
I have gone through this and it has not helped.
What do you think I can do?
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
