AWS Document DB connection from Spark running in AWS EMR

Question

I am trying to connect to AWS DocumentDB from spark job/shell running on EMR

Command used to launch spark-shell

spark-shell --conf "spark.mongodb.output.uri=mongodb://adminuser:Password@docdb-2022-03-20-22-06-33.cluster-xxxyyyzzz.us-east-1.docdb.amazonaws.com:27017/test.spark/?replicaSet=rs0&readPreference=secondaryPreferred&retryWrites=false" \
--conf "spark.mongodb.input.uri=mongodb://adminuser:Password@docdb-2022-03-20-22-06-33.cluster-xxxxyyyzzz.us-east-1.docdb.amazonaws.com:27017/test.spark/?replicaSet=rs0&readPreference=secondaryPreferred&retryWrites=false" \
--packages org.mongodb.spark:mongo-spark-connector_2.11:2.4.3

After launching spark-shell running below listed commands, which command 4 is failing:

scala> import com.mongodb.spark.config._
scala> val readConfig = ReadConfig(Map("collection" -> "spark", "readPreference.name" -> "secondaryPreferred"), Some(ReadConfig(sc)))
scala> val customRdd = MongoSpark.load(sc, readConfig)
scala> println(customRdd.count) 

com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting for a server that matches com.mongodb.client.internal.MongoClientDelegate$1@2a25dd06. Client view of cluster state is {type=REPLICA_SET, servers=[{address=docdb-2022-03-20-22-06-33.xxxyyyzzz.us-east-1.docdb.amazonaws.com:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketReadTimeoutException: Timeout while receiving message}, caused by {java.net.SocketTimeoutException: Read timed out}}]

Have already tried this suggestion