'typesafe joins in spark for 3.x
In old versions of Spark 3.x
import org.apache.calcite.sql.JoinType
Calcite was exposed.
That meant a JOIN could not only be executed in a strongly typed fashion for dataframes (notice I do not mean datasets) by feeding: JoinType.LEFT.name.
Calcite is no longer exposed for 3.x.
Therefore, currently one would need to fall back to stringly typed join types (which is a bad thing I think). Is there a similar possibility for another class I am overlooking (which actually is exposed)? I am a bit reluctant to manually add calcite in my user code as this might result in conflicting classes (but it would be a viable option).
Solution 1:[1]
import org.apache.hadoop.hive.ql.parse.JoinType
seems to do the trick
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | General Grievance |