Check Job History
You can use the following resources to view job history:
Spark history server UI: view information about Spark jobs that have completed.
- On an Ambari-managed cluster, in the Ambari Services tab, select Spark.
- Click Quick Links.
- Choose the Spark history server UI.
Ambari displays a list of jobs.
- Click "App ID" for job details.
Spark history server web UI: view information about Spark jobs that have completed.
In a browser window, navigate to the history server web UI. The default host port is
YARN web UI: view job history and time spent in various stages of the job:
yarn logscommand: list the contents of all log files from all containers associated with the specified application.
yarn logs -applicationId <app_id>.
Hadoop Distributed File System (HDFS) shell or API: view container log files.