@InterfaceAudience.Public @InterfaceStability.Unstable public class ApplicationMaster extends Object
This class is meant to act as an example on how to write yarn-based application masters.
The ApplicationMaster is started on a container by the
ResourceManager
's launcher. The first thing that the
ApplicationMaster
needs to do is to connect and register itself
with the ResourceManager
. The registration sets up information
within the ResourceManager
regarding what host:port the
ApplicationMaster is listening on to provide any form of functionality to a
client as well as a tracking url that a client can use to keep track of
status/job history if needed. However, in the distributedshell, trackingurl
and appMasterHost:appMasterRpcPort are not supported.
The ApplicationMaster
needs to send a heartbeat to the
ResourceManager
at regular intervals to inform the
ResourceManager
that it is up and alive. The
ApplicationMasterProtocol.allocate(org.apache.hadoop.yarn.api.protocolrecords.AllocateRequest)
to the ResourceManager
from the
ApplicationMaster
acts as a heartbeat.
For the actual handling of the job, the ApplicationMaster
has to
request the ResourceManager
via AllocateRequest
for the
required no. of containers using ResourceRequest
with the necessary
resource specifications such as node location, computational
(memory/disk/cpu) resource requirements. The ResourceManager
responds with an AllocateResponse
that informs the
ApplicationMaster
of the set of newly allocated containers,
completed containers as well as current state of available resources.
For each allocated container, the ApplicationMaster
can then set
up the necessary launch context via ContainerLaunchContext
to specify
the allocated container id, local resources required by the executable, the
environment to be setup for the executable, commands to execute, etc. and
submit a StartContainerRequest
to the ContainerManagementProtocol
to
launch and execute the defined commands on the given allocated container.
The ApplicationMaster
can monitor the launched container by
either querying the ResourceManager
using
ApplicationMasterProtocol.allocate(org.apache.hadoop.yarn.api.protocolrecords.AllocateRequest)
to get updates on completed containers or via
the ContainerManagementProtocol
by querying for the status of the allocated
container's ContainerId
.
After the job has been completed, the ApplicationMaster
has to
send a FinishApplicationMasterRequest
to the
ResourceManager
to inform it that the
ApplicationMaster
has been completed.
Modifier and Type | Field and Description |
---|---|
protected ApplicationAttemptId |
appAttemptID |
protected AtomicInteger |
numAllocatedContainers |
protected AtomicInteger |
numRequestedContainers |
protected int |
numTotalContainers |
Constructor and Description |
---|
ApplicationMaster() |
protected ApplicationAttemptId appAttemptID
protected int numTotalContainers
protected AtomicInteger numAllocatedContainers
protected AtomicInteger numRequestedContainers
public ApplicationMaster()
public boolean init(String[] args) throws org.apache.commons.cli.ParseException, IOException
args
- Command line argsorg.apache.commons.cli.ParseException
IOException
public void run() throws YarnException, IOException, InterruptedException
protected boolean finish()
Copyright © 2015 Apache Software Foundation. All rights reserved.