spark kernel reveal-08-spark's Web monitoring page_html/css_WEB-ITnose

You can see the initialization UI code in SparkContext:

// Initialize the Spark UI<strong>private</strong>[spark] <strong>val </strong>ui: Option[SparkUI] =  <strong>if </strong>(conf.getBoolean("spark.ui.enabled", <strong>true</strong>)) {    Some(SparkUI.createLiveUI(<strong>this</strong>, conf, listenerBus, jobProgressListener,      env.securityManager,appName))  } <strong>else </strong>{    // For tests, do not enable the UI    None  }// Bind the UI before starting the task scheduler to communicate// the bound port to the cluster manager properlyui.foreach(_.bind())

Create SparkUI object:

<strong>def </strong>createLiveUI(    sc: SparkContext,    conf: SparkConf,    listenerBus: SparkListenerBus,    jobProgressListener: JobProgressListener,    securityManager: SecurityManager,    appName: String): SparkUI =  {  create(Some(sc), conf, listenerBus, securityManager, appName,    jobProgressListener = Some(jobProgressListener))}

Enter the create method:

Enter SparkUI:

SparkUI inherits WebUI:

The WebUI class defines the def initialize() method

SparkUI’s initialize() implementation method:

Above Code analysis: Add page tab page

1. JobsTab:

2. StagesTab:

3. StorageTab:

4. EnvironmentTab:

5. ExecutorsTab:

6. createStaticHandler:

7. createRedirectHandler:

When the execution is completed After the initialize() method, we return to the Initialize the spark UI of SparkContext:

bind port:

from As can be seen from the above code, the local JettyServer with port number 4040 is started

Analysis of the above code:

1 , create a ContextHandlerCollection and set the handlers to the ContextHandlerCollection

2. Add a filter:

3. Create Jetty Server and bind the port number and create a QueuedThreadPool, set it to the Server:

4. Start the Jetty server. If the startup fails, an exception will be thrown:

5. Try to start startServiceOnPort again several times. If the startup fails, change the new port number (the rule is 1 oldPort) and restart:

After the final startup is successful, the information will be stored in ServerInfo: