'Problem with executing sql query in BigQuery using ExecuteGroovyScript in NiFi

I have ExecuteGroovyScript 1.15.3 processor.

Additional classpath property points to java bigquery client and its dependencies. List of jars:

 api-common-1.9.0.jar
 auto-value-annotations-1.7.jar
 checker-compat-qual-2.5.5.jar
 commons-codec-1.11.jar
 commons-logging-1.2.jar
 error_prone_annotations-2.3.4.jar
 failureaccess-1.0.1.jar
 gax-1.56.0.jar
 gax-httpjson-0.73.0.jar
 google-api-client-1.30.9.jar
 google-api-services-bigquery-v2-rev20200324-1.30.9.jar
 google-auth-library-credentials-0.20.0.jar
 google-auth-library-oauth2-http-0.20.0.jar
 google-cloud-bigquery-1.111.1.jar
 google-cloud-core-1.93.4.jar
 google-cloud-core-http-1.93.4.jar
 google-http-client-1.34.2.jar
 google-http-client-appengine-1.34.2.jar
 google-http-client-jackson2-1.34.2.jar
 google-oauth-client-1.30.5.jar
 grpc-context-1.29.0.jar
 gson-2.8.6.jar
 guava-29.0-android.jar
 httpclient-4.5.11.jar
 httpcore-4.4.13.jar
 j2objc-annotations-1.3.jar
 jackson-core-2.10.2.jar
 javax.annotation-api-1.3.2.jar
 jsr305-3.0.2.jar
 listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar
 opencensus-api-0.24.0.jar
 opencensus-contrib-http-util-0.24.0.jar
 proto-google-common-protos-1.17.0.jar
 proto-google-iam-v1-0.13.0.jar
 protobuf-java-3.11.4.jar
 protobuf-java-util-3.11.4.jar
 threetenbp-1.4.3.jar

Also processor has CTL.credentials property, value is controller service of GCPCredentialsControllerService type.

Script that instantiates BigQuery:

def creds = CTL.credentials.getGoogleCredentials()

def bigQuery = new BigQueryOptions.DefaultBigQueryFactory().create(
    BigQueryOptions.newBuilder()
        .setProjectId(ProjectId.evaluateAttributeExpressions().value)
        .setCredentials(creds)
        .build());

And I got error:

groovy.lang.MissingMethodException: No signature of method: 
com.google.cloud.bigquery.BigQueryOptions$Builder.setCredentials() is applicable for argument types:
(com.google.auth.oauth2.ServiceAccountCredentials) values: [ServiceAccountCredentials{....}]
Possible solutions: setCredentials(com.google.auth.Credentials)

This is very strange because ServiceAccountCredentials extends Credentials. So I tried to explicitly cast it:

def creds = CTL.credentials.getGoogleCredentials() as Credentials

I got error:

org.codehaus.groovy.runtime.typehandling.GroovyCastException: Cannot cast object 'ServiceAccountCredentials{....}'
with class 'com.google.auth.oauth2.ServiceAccountCredentials' to class 'com.google.auth.Credentials'

Then I tried to cast to ServiceAccountCredentials and I got:

org.codehaus.groovy.runtime.typehandling.GroovyCastException: Cannot cast object 'ServiceAccountCredentials{....}'
with class 'com.google.auth.oauth2.ServiceAccountCredentials' to class 'com.google.auth.oauth2.ServiceAccountCredentials'

Why am I getting a GroovyCastException? What could be the problem? Maybe there is an easier way to execute sql?



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source