'How to access Spark Web UI?

I'm running a Spark application locally of 4 nodes. when I'm running my Application it displays my driver having this address 10.0.2.15:

INFO Utils: Successfully started service 'SparkUI' on port 4040.
INFO SparkUI: Bound SparkUI to 0.0.0.0, and started at http://10.0.2.15:4040

at the end of running it displays :

INFO SparkUI: Stopped Spark web UI at http://10.0.2.15:4040
INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
INFO MemoryStore: MemoryStore cleared
INFO BlockManager: BlockManager stopped
INFO BlockManagerMaster: BlockManagerMaster stopped
INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
INFO SparkContext: Successfully stopped SparkContext

I tried to access the Spark Web by: 10.0.2.15:4040 but the page is inaccessible. Trying with the below address also didn't helped:

 http://localhost:18080

Using ping 10.0.2.15 the result is:

Send a request 'Ping' 10.0.2.15 with 32 bytes of data

Waiting time exceeded

Waiting time exceeded

Waiting time exceeded

Waiting time exceeded

Ping statistics for 10.0.2.15: Packages: sent = 4, received = 0, lost = 4 (100% loss)

Checked the availability of the port 4040 using netstat -a to verify which ports are available. The result is as follow:

   Active connexion:

    Active       local address        Remote address                      state

    TCP          127.0.0.1:4040      DESKTOP-FF4U.....:0                 Listening

PS.: Knowning that my code is running succesfully. What could be the reason?

apache-spark

Solution 1:^[1]

The web ui is only accessible while the Spark application is running. Once the Spark application has finished so has the UI.

To review the logs of already finished and currently running Spark applications you have to use the Spark History server.

Solution 2:^[2]

So if you are running your spark code on a scala IDE you need to enter the below line.

scala.io.StdIn.readLine()

Later go to the web and type localhost:4040 or localhost:4041.

Cheers!

Solution 3:^[3]

In Java, Before closing SparkContext, put

Scanner scanner = new Scanner(System.in); 
scanner.nextLine();

and then close the SparkContext, as

sparkContext.close();

And then access the spark UI at http://localhost:4040

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution	Source
Solution 1	Jacek Laskowski
Solution 2	Balasankar Sajja
Solution 3	Neeraj Sharma

'How to access Spark Web UI?

Solution 1:[1]

Solution 2:[2]

Solution 3:[3]

Sources

Related Questions

Solution 1:^[1]

Solution 2:^[2]

Solution 3:^[3]