'How to fix NoClassDefFoundError while trying to use Spark Streaming?

I'm trying to run a java program which uses spark streaming as follows:

JavaStreamingContext javaStreamingContext = new JavaStreamingContext(sparkContext, Durations.seconds(2));
        javaStreamingContext.checkpoint("checkpoint");

        String hostname = (onServer) ? "stream-host" : "localhost";
        JavaReceiverInputDStream<String> lines =
                javaStreamingContext.socketTextStream(hostname, 9000);

Whenever i try to run it, i get an error with the following message:

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/spark/streaming/api/java/JavaStreamingContext

Caused by: java.lang.ClassNotFoundException: org.apache.spark.streaming.api.java.JavaStreamingContext
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583)
    at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
    ... 2 more

I have looked in multiple places but I don't know what's causing these errors. How can I fix this issue?

Here's the relevant part from my pom.xml file for reference:

 <!-- Do not remove any of the dependencies related to Spark and Maven plugins -->
    <dependencies>
        <!-- https://mvnrepository.com/artifact/org.apache.spark/spark-core -->
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-core_2.13</artifactId>
            <version>3.2.0</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.apache.spark/spark-sql -->
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-sql_2.13</artifactId>
            <version>3.2.0</version>
            <scope>provided</scope>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.apache.spark/spark-streaming -->
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-streaming_2.13</artifactId>
            <version>3.2.0</version>
            <scope>provided</scope>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.apache.maven.plugins/maven-compiler-plugin -->
        <dependency>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>3.8.1</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.apache.maven.plugins/maven-jar-plugin -->
        <dependency>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-jar-plugin</artifactId>
            <version>3.2.2</version>
        </dependency>
    </dependencies>

apache-spark spark-streaming

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution	Source

'How to fix NoClassDefFoundError while trying to use Spark Streaming?

Sources

Related Questions