'Is it possible to connect to scylla using cassandra connector 3.10 using Catalog?
I am trying to do a simple select in a scylla database as follows:
spark.read.table("Scylla.config.parameters").show(5)
I have created a variable with the settings that I will use as follows:
val sparkConf: SparkConf = new SparkConf()
.set(s"spark.sql.extensions", "com.datastax.spark.connector.CassandraSparkExtensions")
.set(s"spark.sql.catalog.Scylla", "com.datastax.spark.connector.datasource.CassandraCatalog")
.set(s"spark.sql.catalog.Scylla.spark.cassandra.connection.host", Settings.cassandraServerAddress)
.set(s"spark.sql.catalog.Scylla.spark.sql.catalog.spark.cassandra.auth.username", Settings.User)
.set(s"spark.sql.catalog.Scylla.spark.cassandra.auth.password", Settings.Pass)
.set(s"spark.sql.catalog.Scylla.spark.cassandra.connection.port",Settings.Port)
val sparkSession: SparkSession = SparkSession
.builder()
.config(sparkConf)
.appName(Settings.appname)
.getOrCreate()
Then I make the call to start the Spark Session:
val spark: SparkSession = Spark.sparkSession
But when the engine tries to execute the query, it returns the error:
Node node01-xxx-scylla.xxxxxxxx.com/xxx.x.xxx.xxx:9042 requires authentication (org.apache.cassandra.auth.PasswordAuthenticator), but no authenticator configured
Am I doing something wrong? Or is there an incompatibility of the connector version with the version of scylla I use? The version of scylla I am using is 2021.1.5-0.20210818.fc817c0cd
@Edit
I have identified the problem. In my db user configuration, I had too much text in the connection string. After adjustment the problem stopped.
Before:
.set(s"spark.sql.catalog.Scylla.spark.sql.catalog.spark.cassandra.auth.username", Settings.User)
After:
.set(s"spark.sql.catalog.Scylla.spark.cassandra.auth.username", Settings.User)
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
