@InterfaceAudience.Public @InterfaceStability.Unstable public class ApplicationMaster extends Object
This class is meant to act as an example on how to write yarn-based application masters.
The ApplicationMaster is started on a container by the
ResourceManager's launcher. The first thing that the
ApplicationMaster needs to do is to connect and register itself
ResourceManager. The registration sets up information
ResourceManager regarding what host:port the
ApplicationMaster is listening on to provide any form of functionality to a
client as well as a tracking url that a client can use to keep track of
status/job history if needed. However, in the distributedshell, trackingurl
and appMasterHost:appMasterRpcPort are not supported.
ApplicationMaster needs to send a heartbeat to the
ResourceManager at regular intervals to inform the
ResourceManager that it is up and alive. The
ApplicationMasterProtocol.allocate(org.apache.hadoop.yarn.api.protocolrecords.AllocateRequest) to the
ResourceManager from the
ApplicationMaster acts as a heartbeat.
For the actual handling of the job, the
ApplicationMaster has to
AllocateRequest for the
required no. of containers using
ResourceRequest with the necessary
resource specifications such as node location, computational
(memory/disk/cpu) resource requirements. The
responds with an
AllocateResponse that informs the
ApplicationMaster of the set of newly allocated containers,
completed containers as well as current state of available resources.
For each allocated container, the
ApplicationMaster can then set
up the necessary launch context via
ContainerLaunchContext to specify
the allocated container id, local resources required by the executable, the
environment to be setup for the executable, commands to execute, etc. and
StartContainerRequest to the
launch and execute the defined commands on the given allocated container.
ApplicationMaster can monitor the launched container by
either querying the
ApplicationMasterProtocol.allocate(org.apache.hadoop.yarn.api.protocolrecords.AllocateRequest) to get updates on completed containers or via
ContainerManagementProtocol by querying for the status of the allocated
After the job has been completed, the
ApplicationMaster has to
FinishApplicationMasterRequest to the
ResourceManager to inform it that the
ApplicationMaster has been completed.
|Modifier and Type||Field and Description|
|Constructor and Description|
|Modifier and Type||Method and Description|
Parse command line options
Main run function for the application master
protected int numTotalContainers
public boolean init(String args) throws org.apache.commons.cli.ParseException, IOException
args- Command line args
public void run() throws YarnException, IOException, InterruptedException
protected boolean finish()
Copyright © 2016 Apache Software Foundation. All rights reserved.