org.apache.hadoop.yarn.server.uam

Class UnmanagedAMPoolManager

java.lang.Object

org.apache.hadoop.service.AbstractService

org.apache.hadoop.yarn.server.uam.UnmanagedAMPoolManager

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.hadoop.service.Service

@InterfaceAudience.Public
@InterfaceStability.Unstable
public class UnmanagedAMPoolManager
extends org.apache.hadoop.service.AbstractService

A service that manages a pool of UAM managers in UnmanagedApplicationManager.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service

org.apache.hadoop.service.Service.STATE

Field Summary

Fields

Modifier and Type	Field and Description
static org.slf4j.Logger	LOG

Constructor Summary

Constructors

Constructor and Description
UnmanagedAMPoolManager(ExecutorService threadpool)

Method Summary

Methods

Modifier and Type	Method and Description
void	allocateAsync(String uamId, org.apache.hadoop.yarn.api.protocolrecords.AllocateRequest request, org.apache.hadoop.yarn.util.AsyncCallback<org.apache.hadoop.yarn.api.protocolrecords.AllocateResponse> callback) AllocateAsync to an UAM.
String	createAndRegisterNewUAM(org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterRequest registerRequest, org.apache.hadoop.conf.Configuration conf, String queueName, String submitter, String appNameSuffix, boolean keepContainersAcrossApplicationAttempts) Create a new UAM and register the application, without specifying uamId and appId.
protected UnmanagedApplicationManager	createUAM(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.yarn.api.records.ApplicationId appId, String queueName, String submitter, String appNameSuffix, boolean keepContainersAcrossApplicationAttempts) Creates the UAM instance.
org.apache.hadoop.yarn.api.protocolrecords.FinishApplicationMasterResponse	finishApplicationMaster(String uamId, org.apache.hadoop.yarn.api.protocolrecords.FinishApplicationMasterRequest request) Finish an UAM/application.
Set<String>	getAllUAMIds() Get the id of all running UAMs.
boolean	hasUAMId(String uamId) Return whether an UAM exists.
org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier>	launchUAM(String uamId, org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.yarn.api.records.ApplicationId appId, String queueName, String submitter, String appNameSuffix, boolean keepContainersAcrossApplicationAttempts) Launch a new UAM, using the provided uamId and appId.
void	reAttachUAM(String uamId, org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.yarn.api.records.ApplicationId appId, String queueName, String submitter, String appNameSuffix, org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier> uamToken) Re-attach to an existing UAM, using the provided uamIdentifier.
org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterResponse	registerApplicationMaster(String uamId, org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterRequest registerRequest) Register application master for the UAM.
protected void	serviceStart()
protected void	serviceStop() Normally we should finish all applications before stop.

Methods inherited from class org.apache.hadoop.service.AbstractService

close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, serviceInit, setConfig, start, stop, toString, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

LOG

public static final org.slf4j.Logger LOG

Constructor Detail

UnmanagedAMPoolManager

public UnmanagedAMPoolManager(ExecutorService threadpool)

Method Detail

serviceStart

protected void serviceStart()
                     throws Exception

Overrides:

serviceStart in class org.apache.hadoop.service.AbstractService

Throws:

Exception

serviceStop

protected void serviceStop()
                    throws Exception

Normally we should finish all applications before stop. If there are still UAMs running, force kill all of them. Do parallel kill because of performance reasons. TODO: move waiting for the kill to finish into a separate thread, without blocking the serviceStop.

Overrides:

serviceStop in class org.apache.hadoop.service.AbstractService

Throws:

Exception

createAndRegisterNewUAM

public String createAndRegisterNewUAM(org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterRequest registerRequest,
                             org.apache.hadoop.conf.Configuration conf,
                             String queueName,
                             String submitter,
                             String appNameSuffix,
                             boolean keepContainersAcrossApplicationAttempts)
                               throws org.apache.hadoop.yarn.exceptions.YarnException,
                                      IOException

Create a new UAM and register the application, without specifying uamId and appId. We will ask for an appId from RM and use it as the uamId.

Parameters:

registerRequest - RegisterApplicationMasterRequest

conf - configuration for this UAM

queueName - queue of the application

submitter - submitter name of the UAM

appNameSuffix - application name suffix for the UAM

keepContainersAcrossApplicationAttempts - keep container flag for UAM recovery.

Returns:

uamId for the UAM

Throws:

org.apache.hadoop.yarn.exceptions.YarnException - if registerApplicationMaster fails

IOException - if registerApplicationMaster fails

See Also:

#setKeepContainersAcrossApplicationAttempts(boolean)

launchUAM

public org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier> launchUAM(String uamId,
                                                                                                    org.apache.hadoop.conf.Configuration conf,
                                                                                                    org.apache.hadoop.yarn.api.records.ApplicationId appId,
                                                                                                    String queueName,
                                                                                                    String submitter,
                                                                                                    String appNameSuffix,
                                                                                                    boolean keepContainersAcrossApplicationAttempts)
                                                                                                      throws org.apache.hadoop.yarn.exceptions.YarnException,
                                                                                                             IOException

Launch a new UAM, using the provided uamId and appId.

Parameters:

uamId - uam Id

conf - configuration for this UAM

appId - application id for the UAM

queueName - queue of the application

submitter - submitter name of the UAM

appNameSuffix - application name suffix for the UAM

keepContainersAcrossApplicationAttempts - keep container flag for UAM recovery.

Returns:

UAM token

Throws:

org.apache.hadoop.yarn.exceptions.YarnException - if fails

IOException - if fails

See Also:

#setKeepContainersAcrossApplicationAttempts(boolean)

reAttachUAM

public void reAttachUAM(String uamId,
               org.apache.hadoop.conf.Configuration conf,
               org.apache.hadoop.yarn.api.records.ApplicationId appId,
               String queueName,
               String submitter,
               String appNameSuffix,
               org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier> uamToken)
                 throws org.apache.hadoop.yarn.exceptions.YarnException,
                        IOException

Re-attach to an existing UAM, using the provided uamIdentifier.

Parameters:

uamId - uam Id

conf - configuration for this UAM

appId - application id for the UAM

queueName - queue of the application

submitter - submitter name of the UAM

appNameSuffix - application name suffix for the UAM

uamToken - UAM token

Throws:

org.apache.hadoop.yarn.exceptions.YarnException - if fails

IOException - if fails

createUAM

protected UnmanagedApplicationManager createUAM(org.apache.hadoop.conf.Configuration conf,
                                    org.apache.hadoop.yarn.api.records.ApplicationId appId,
                                    String queueName,
                                    String submitter,
                                    String appNameSuffix,
                                    boolean keepContainersAcrossApplicationAttempts)

Creates the UAM instance. Pull out to make unit test easy.

Parameters:

conf - Configuration

appId - application id

queueName - queue of the application

submitter - submitter name of the application

appNameSuffix - application name suffix

keepContainersAcrossApplicationAttempts - keep container flag for UAM

Returns:

the UAM instance

registerApplicationMaster

public org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterResponse registerApplicationMaster(String uamId,
                                                                                                     org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterRequest registerRequest)
                                                                                                       throws org.apache.hadoop.yarn.exceptions.YarnException,
                                                                                                              IOException

Register application master for the UAM.

Parameters:

uamId - uam Id

registerRequest - RegisterApplicationMasterRequest

Returns:

register response

Throws:

org.apache.hadoop.yarn.exceptions.YarnException - if register fails

IOException - if register fails

allocateAsync

public void allocateAsync(String uamId,
                 org.apache.hadoop.yarn.api.protocolrecords.AllocateRequest request,
                 org.apache.hadoop.yarn.util.AsyncCallback<org.apache.hadoop.yarn.api.protocolrecords.AllocateResponse> callback)
                   throws org.apache.hadoop.yarn.exceptions.YarnException,
                          IOException

AllocateAsync to an UAM.

Parameters:

uamId - uam Id

request - AllocateRequest

callback - callback for response

Throws:

org.apache.hadoop.yarn.exceptions.YarnException - if allocate fails

IOException - if allocate fails

finishApplicationMaster

public org.apache.hadoop.yarn.api.protocolrecords.FinishApplicationMasterResponse finishApplicationMaster(String uamId,
                                                                                                 org.apache.hadoop.yarn.api.protocolrecords.FinishApplicationMasterRequest request)
                                                                                                   throws org.apache.hadoop.yarn.exceptions.YarnException,
                                                                                                          IOException

Finish an UAM/application.

Parameters:

uamId - uam Id

request - FinishApplicationMasterRequest

Returns:

FinishApplicationMasterResponse

Throws:

org.apache.hadoop.yarn.exceptions.YarnException - if finishApplicationMaster call fails

IOException - if finishApplicationMaster call fails

getAllUAMIds

public Set<String> getAllUAMIds()

Get the id of all running UAMs.

Returns:

uamId set

hasUAMId

public boolean hasUAMId(String uamId)

Return whether an UAM exists.

Parameters:

uamId - uam Id

Returns:

UAM exists or not