'Hive Java UDF on Big Query
We have trying to convert the Hive Scripts (DDL CTAS) statements to Big Query equivalent. However the Hive scripts are using complex hive UDF which are not supported in Big Query.
I came across the below link "Hive Storage Handler on BigQuery" which provides a potential solution by linking Hive tables to Big Query. That way we can run the hive scripts with the UDFs and resulting data will be stored in Big Query. Link - https://github.com/GoogleCloudDataproc/hive-bigquery-storage-handler
The issue is the repository mentions its no longer maintained and suggests to use the big query connector - https://cloud.google.com/dataproc/docs/concepts/connectors/bigquery
There is no working example of how to use mentioned latest big query connector with hive. There are examples available for MR and Spark in the link above.
Any help in this matter is appreciated.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|