Connecting to CosmosDB from Databricks

Question

Following: https://docs.microsoft.com/en-us/azure/cosmos-db/sql/create-sql-api-spark

But getting error while creating db on Cosmos

# Configure Catalog Api to be used
spark.conf.set("spark.sql.catalog.cosmosCatalog", "com.azure.cosmos.spark.CosmosCatalog")
spark.conf.set("spark.sql.catalog.cosmosCatalog.spark.cosmos.accountEndpoint", cosmosEndpoint)
spark.conf.set("spark.sql.catalog.cosmosCatalog.spark.cosmos.accountKey", cosmosMasterKey)

# create a cosmos database using catalog api
spark.sql("CREATE DATABASE IF NOT EXISTS cosmosCatalog.{};".format(cosmosDatabaseName))

Error:

java.lang.RuntimeException: Client initialization failed. Check if the endpoint is reachable and if your auth token is valid. More info: https://aka.ms/cosmosdb-tsg-service-unavailable-java

Answer 1

java.lang.RuntimeException: Client initialization failed. Check if the endpoint is reachable and if your auth token is valid. More info: https://aka.ms/cosmosdb-tsg-service-unavailable-java

This exception come if the SDK is not able to talk to the Cosmos DB instance. It normally points to firewall rule is blocking the requests.

To avoid this exception and SDK can communicate to the Cosmos DB account execute the following command from where the application is hosted. If it fails, this points to a firewall rule blocking the request. If it succeeds the SDK should be able to communicate to the Cosmos DB account.

telnet myCosmosDbAccountName.documents.azure.com 443