'Spark/MongoDB connector with $or pipeline syntax

I'm trying to request a MongoDB collection using Spark with the mongo-spark-connector. Here is the code :

val document = Document.parse("{$or: [{key1:\"A\", key2:\"500\"}, {key1:\"A\", key2:\"100\"}] }")

val mongoDf = mongoCollectionRdd.withPipeline(Seq(document)).toDF(mongoCollectionSchema)

But I get the following error :

com.mongodb.MongoCommandException: Command failed with error 40324 (Location40324): 'Unrecognized pipeline stage name: '$or'' on server hostname.fr:27017.

I ran the query through the mongo shell and the execution is fine and I get the expected result. Here are the versions of the different components :

Scala : 2.12.11

Spark : 3.2.0

MongoDB : 4.2.17

Any ideas ?



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source