'Authenticate Spark to GCS with HMAC key

We have a Spark application accessing GCS using the GCP connector. We would like to authenticate using a service account HMAC key. Is this possible?

We have tried a few of the authentication configurations here but none seems to work.

Here's an example of what we are trying to do

    val spark = SparkSession.builder()
      .config("google.cloud.auth.client.id", "HMAC key id")
      .config("google.cloud.auth.client.secret", "HMAC key secret")
      .master("local[*]")
      .appName("Test App")
      .getOrCreate()

    df.write.format("parquet")
      .save("gs://test-project/")

We have tried the keyfile JSON which works, but HMAC would be a bit more convenient for us.



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source