'How to select elements from json column except unwanted columns in spark
I have various columns in Spark DataFrame, they are nested json columns. In configuration i will provide a list of columns and fields to remove from json. For ex:
Map("column1" -> Array("PathName"), "column2" -> Array("Type", "Seq"))
What i need to do is to create new DataFrame with same columns but except json fields provided in config. So from below dfs i need to filter out what its in config.
Source
Destination
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|