Apache Spark Component Guide
Also available as:
PDF
loading table of contents...

Livy API Reference for Interactive Sessions

GET

GET /sessions returns all active interactive sessions.

Request ParameterDescriptionType
fromStarting index for fetching sessionsint
sizeNumber of sessions to fetchint
ResponseDescriptionType
fromStarting index of fetched sessionsint
totalNumber of sessions fetchedint
sessionsSession listlist

The following response shows zero active sessions:

      {"from":0,"total":0,"sessions":[]}

GET /sessions/{sessionId} returns information about the specified session.

GET /sessions/{sessionId}/state returns the state of the specified session:

ResponseDescriptionType
idA non-negative integer that represents a specific sessionint
stateCurrent state of the sessionstring

GET /sessions/{sessionId}/logs retrieves log records for the specified session.

Request ParametersDescriptionType
fromOffsetint
sizeMaximum number of log records to retrieveint
ResponseDescriptionType
idA non-negative integer that represents a specific sessionint
fromOffset from the start of the log fileint
sizeNumber of log records retrievedint
logLog recordslist of strings

GET /sessions/{sessionId}/statements returns all the statements in a session.

ResponseDescriptionType
statementsList of statements in the specified sessionlist

GET /sessions/{sessionId}/statements/{statementId} returns a specified statement in a session.

ResponseDescriptionType
statement object (for more information see "Livy Objects for Interactive Sessions")Statementstatement object

POST

POST /sessions creates a new interactive Scala, Python, or R shell in the cluster.

Request ParameterDescriptionType
kindSession kind (required)session kind
proxyUserUser ID to impersonate when starting the sessionstring
jarsJar files to be used in this sessionlist of strings
pyFilesPython files to be used in this sessionlist of strings
filesOther files to be used in this sessionlist of strings
driverMemoryAmount of memory to use for the driver processstring
driverCoresNumber of cores to use for the driver processint
executorMemoryAmount of memory to use for each executor processstring
executorCoresNumber of cores to use for each executor processint
numExecutorsNumber of executors to launch for this sessionint
archivesArchives to be used in this sessionlist of strings
queueThe name of the YARN queue to which the job should be submittedstring
nameName of this sessionstring
confSpark configuration propertiesMap of key=value
heartbeatTimeoutInSecondTimeout in second to which session be orphanedint
ResponseDescriptionType
session object (for more information see "Livy Objects for Interactive Sessions")The created sessionsession object

The following response shows a PySpark session in the process of starting:

      {"id":0,"state":"starting","kind":"pyspark","log":[]}

POST /sessions/{sessionId}/statements runs a statement in a session.

Request ParameterDescriptionType
codeThe code to executestring
ResponseDescriptionType
statement object (for more information see "Livy Objects for Interactive Sessions")Result of an execution statementstatement object

POST /sessions/{sessionId}/statements/{statementId}/cancel cancels the specified statement in the session.

ResponseDescriptionType
cancellation messageReports "cancelled"string

DELETE

DELETE /sessions/{sessionId} terminates the session.