'Problem with executing sql query in BigQuery using ExecuteGroovyScript in NiFi
I have ExecuteGroovyScript 1.15.3 processor.
Additional classpath property points to java bigquery client and its dependencies.
List of jars:
api-common-1.9.0.jar
auto-value-annotations-1.7.jar
checker-compat-qual-2.5.5.jar
commons-codec-1.11.jar
commons-logging-1.2.jar
error_prone_annotations-2.3.4.jar
failureaccess-1.0.1.jar
gax-1.56.0.jar
gax-httpjson-0.73.0.jar
google-api-client-1.30.9.jar
google-api-services-bigquery-v2-rev20200324-1.30.9.jar
google-auth-library-credentials-0.20.0.jar
google-auth-library-oauth2-http-0.20.0.jar
google-cloud-bigquery-1.111.1.jar
google-cloud-core-1.93.4.jar
google-cloud-core-http-1.93.4.jar
google-http-client-1.34.2.jar
google-http-client-appengine-1.34.2.jar
google-http-client-jackson2-1.34.2.jar
google-oauth-client-1.30.5.jar
grpc-context-1.29.0.jar
gson-2.8.6.jar
guava-29.0-android.jar
httpclient-4.5.11.jar
httpcore-4.4.13.jar
j2objc-annotations-1.3.jar
jackson-core-2.10.2.jar
javax.annotation-api-1.3.2.jar
jsr305-3.0.2.jar
listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar
opencensus-api-0.24.0.jar
opencensus-contrib-http-util-0.24.0.jar
proto-google-common-protos-1.17.0.jar
proto-google-iam-v1-0.13.0.jar
protobuf-java-3.11.4.jar
protobuf-java-util-3.11.4.jar
threetenbp-1.4.3.jar
Also processor has CTL.credentials property, value is controller service of GCPCredentialsControllerService type.
Script that instantiates BigQuery:
def creds = CTL.credentials.getGoogleCredentials()
def bigQuery = new BigQueryOptions.DefaultBigQueryFactory().create(
BigQueryOptions.newBuilder()
.setProjectId(ProjectId.evaluateAttributeExpressions().value)
.setCredentials(creds)
.build());
And I got error:
groovy.lang.MissingMethodException: No signature of method:
com.google.cloud.bigquery.BigQueryOptions$Builder.setCredentials() is applicable for argument types:
(com.google.auth.oauth2.ServiceAccountCredentials) values: [ServiceAccountCredentials{....}]
Possible solutions: setCredentials(com.google.auth.Credentials)
This is very strange because ServiceAccountCredentials extends Credentials.
So I tried to explicitly cast it:
def creds = CTL.credentials.getGoogleCredentials() as Credentials
I got error:
org.codehaus.groovy.runtime.typehandling.GroovyCastException: Cannot cast object 'ServiceAccountCredentials{....}'
with class 'com.google.auth.oauth2.ServiceAccountCredentials' to class 'com.google.auth.Credentials'
Then I tried to cast to ServiceAccountCredentials and I got:
org.codehaus.groovy.runtime.typehandling.GroovyCastException: Cannot cast object 'ServiceAccountCredentials{....}'
with class 'com.google.auth.oauth2.ServiceAccountCredentials' to class 'com.google.auth.oauth2.ServiceAccountCredentials'
Why am I getting a GroovyCastException? What could be the problem? Maybe there is an easier way to execute sql?
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
