org.apache.hadoop.yarn.server.resourcemanager.rmapp

Class RMAppImpl

java.lang.Object

org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl

All Implemented Interfaces:

org.apache.hadoop.yarn.event.EventHandler<RMAppEvent>, Recoverable, RMApp

public class RMAppImpl
extends Object
implements RMApp, Recoverable

Constructor Summary

Constructors

Constructor and Description
RMAppImpl(org.apache.hadoop.yarn.api.records.ApplicationId applicationId, RMContext rmContext, org.apache.hadoop.conf.Configuration config, String name, String user, String queue, org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext submissionContext, YarnScheduler scheduler, ApplicationMasterService masterService, long submitTime, String applicationType, Set<String> applicationTags, List<org.apache.hadoop.yarn.api.records.ResourceRequest> amReqs)
RMAppImpl(org.apache.hadoop.yarn.api.records.ApplicationId applicationId, RMContext rmContext, org.apache.hadoop.conf.Configuration config, String name, String user, String queue, org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext submissionContext, YarnScheduler scheduler, ApplicationMasterService masterService, long submitTime, String applicationType, Set<String> applicationTags, List<org.apache.hadoop.yarn.api.records.ResourceRequest> amReqs, ApplicationPlacementContext placementContext, long startTime)

Method Summary

Modifier and Type	Method and Description
void	aggregateLogReport(org.apache.hadoop.yarn.api.records.NodeId nodeId, org.apache.hadoop.yarn.server.api.protocolrecords.LogAggregationReport report)
org.apache.hadoop.yarn.api.records.ApplicationReport	createAndGetApplicationReport(String clientUserName, boolean allowAccess) To get the status of an application in the RM, this method can be used.
org.apache.hadoop.yarn.api.records.YarnApplicationState	createApplicationState() Create the external user-facing state of ApplicationMaster from the current state of the RMApp.
String	getAmNodeLabelExpression() Return the node label expression of the AM container.
List<org.apache.hadoop.yarn.api.records.ResourceRequest>	getAMResourceRequests()
Map<org.apache.hadoop.yarn.api.records.ApplicationAttemptId,RMAppAttempt>	getAppAttempts() RMApp can have multiple application attempts RMAppAttempt.
org.apache.hadoop.yarn.api.records.ApplicationId	getApplicationId() The application id for this RMApp.
ApplicationPlacementContext	getApplicationPlacementContext() Get the application -> queue placement context
org.apache.hadoop.yarn.api.records.Priority	getApplicationPriority() Get priority of the application.
Map<String,String>	getApplicationSchedulingEnvs() Get the application scheduling environment variables.
org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext	getApplicationSubmissionContext() The application submission context for this RMApp
Set<String>	getApplicationTags() Get tags for the application
Map<org.apache.hadoop.yarn.api.records.ApplicationTimeoutType,Long>	getApplicationTimeouts()
String	getApplicationType() Returns the application type
String	getAppNodeLabelExpression()
org.apache.hadoop.ipc.CallerContext	getCallerContext()
org.apache.hadoop.yarn.server.api.records.AppCollectorData	getCollectorData() The timeline collector information for the application.
org.apache.hadoop.yarn.api.records.CollectorInfo	getCollectorInfo() The timeline collector information to be sent to AM.
RMAppAttempt	getCurrentAppAttempt() RMApp can have multiple application attempts RMAppAttempt.
StringBuilder	getDiagnostics() the diagnostics information for the application master.
org.apache.hadoop.yarn.api.records.FinalApplicationStatus	getFinalApplicationStatus() The final finish state of the AM when unregistering as in FinishApplicationMasterRequest.setFinalApplicationStatus(FinalApplicationStatus).
long	getFinishTime() The finish time of the RMApp
String	getLogAggregationFailureMessagesForNM(org.apache.hadoop.yarn.api.records.NodeId nodeId)
Map<org.apache.hadoop.yarn.api.records.NodeId,org.apache.hadoop.yarn.server.api.protocolrecords.LogAggregationReport>	getLogAggregationReportsForApp()
org.apache.hadoop.yarn.api.records.LogAggregationStatus	getLogAggregationStatusForAppReport()
int	getMaxAppAttempts() The number of max attempts of the application.
String	getName() The name of the application as set in ApplicationSubmissionContext.setApplicationName(String).
int	getNextAttemptId()
int	getNumFailedAppAttempts()
String	getOriginalTrackingUrl() The original tracking url for the application master.
float	getProgress() Progress of application.
String	getQueue() Each Application is submitted to a queue decided by ApplicationSubmissionContext.setQueue(String).
Set<org.apache.hadoop.yarn.api.records.NodeId>	getRanNodes() Nodes on which the containers for this RMApp ran.
RMAppState	getRecoveredFinalState()
org.apache.hadoop.yarn.api.records.ReservationId	getReservationId()
RMAppAttempt	getRMAppAttempt(org.apache.hadoop.yarn.api.records.ApplicationAttemptId appAttemptId) RMApp can have multiple application attempts RMAppAttempt.
RMAppMetrics	getRMAppMetrics() Get RMAppMetrics of the RMApp.
long	getStartTime() the start time of the application.
RMAppState	getState() The current state of the RMApp.
long	getSubmitTime() the submit time of the application.
String	getTrackingUrl() The tracking url for the application master.
String	getUser() The user who submitted this application.
void	handle(RMAppEvent event)
boolean	isAppFinalStateStored() Check whether this application's state has been saved to the state store.
boolean	isAppInCompletedStates() To verify whether app has reached in its completing/completed states.
static boolean	isAppInFinalState(RMApp rmApp)
protected void	onInvalidStateTransition(RMAppEventType rmAppEventType, RMAppState state) catch the InvalidStateTransition.
int	pullRMNodeUpdates(Map<RMNode,org.apache.hadoop.yarn.api.records.NodeUpdateType> upNodes) To receive the collection of all RMNodes whose updates have been received by the RMApp.
void	recover(RMStateStore.RMState state)
void	removeCollectorData()
void	setApplicationPriority(org.apache.hadoop.yarn.api.records.Priority applicationPriority)
void	setCollectorData(org.apache.hadoop.yarn.server.api.records.AppCollectorData incomingData)
void	setQueue(String queue) Reflects a change in the application's queue from the one specified in the ApplicationSubmissionContext.
void	setSystemClock(org.apache.hadoop.yarn.util.Clock clock)
void	startTimelineCollector() Starts the application level timeline collector for this app.
void	stopTimelineCollector() Stops the application level timeline collector for this app.
void	updateApplicationTimeout(Map<org.apache.hadoop.yarn.api.records.ApplicationTimeoutType,Long> updateTimeout)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

RMAppImpl

public RMAppImpl(org.apache.hadoop.yarn.api.records.ApplicationId applicationId,
                 RMContext rmContext,
                 org.apache.hadoop.conf.Configuration config,
                 String name,
                 String user,
                 String queue,
                 org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext submissionContext,
                 YarnScheduler scheduler,
                 ApplicationMasterService masterService,
                 long submitTime,
                 String applicationType,
                 Set<String> applicationTags,
                 List<org.apache.hadoop.yarn.api.records.ResourceRequest> amReqs)

RMAppImpl

public RMAppImpl(org.apache.hadoop.yarn.api.records.ApplicationId applicationId,
                 RMContext rmContext,
                 org.apache.hadoop.conf.Configuration config,
                 String name,
                 String user,
                 String queue,
                 org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext submissionContext,
                 YarnScheduler scheduler,
                 ApplicationMasterService masterService,
                 long submitTime,
                 String applicationType,
                 Set<String> applicationTags,
                 List<org.apache.hadoop.yarn.api.records.ResourceRequest> amReqs,
                 ApplicationPlacementContext placementContext,
                 long startTime)

Method Detail

startTimelineCollector

public void startTimelineCollector()

Starts the application level timeline collector for this app. This should be used only if the timeline service v.2 is enabled.

stopTimelineCollector

public void stopTimelineCollector()

Stops the application level timeline collector for this app. This should be used only if the timeline service v.2 is enabled.

getApplicationId

public org.apache.hadoop.yarn.api.records.ApplicationId getApplicationId()

Description copied from interface: RMApp

The application id for this RMApp.

Specified by:

getApplicationId in interface RMApp

Returns:

the ApplicationId for this RMApp.

getApplicationSubmissionContext

public org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext getApplicationSubmissionContext()

Description copied from interface: RMApp

The application submission context for this RMApp

Specified by:

getApplicationSubmissionContext in interface RMApp

Returns:

the ApplicationSubmissionContext for this RMApp

getFinalApplicationStatus

public org.apache.hadoop.yarn.api.records.FinalApplicationStatus getFinalApplicationStatus()

Description copied from interface: RMApp

The final finish state of the AM when unregistering as in FinishApplicationMasterRequest.setFinalApplicationStatus(FinalApplicationStatus).

Specified by:

getFinalApplicationStatus in interface RMApp

Returns:

the final finish state of the AM as set in FinishApplicationMasterRequest.setFinalApplicationStatus(FinalApplicationStatus).

getState

public RMAppState getState()

Description copied from interface: RMApp

The current state of the RMApp.

Specified by:

getState in interface RMApp

Returns:

the current state RMAppState for this application.

getUser

public String getUser()

Description copied from interface: RMApp

The user who submitted this application.

Specified by:

getUser in interface RMApp

Returns:

the user who submitted the application.

getProgress

public float getProgress()

Description copied from interface: RMApp

Progress of application.

Specified by:

getProgress in interface RMApp

Returns:

the progress of the RMApp.

getRMAppAttempt

public RMAppAttempt getRMAppAttempt(org.apache.hadoop.yarn.api.records.ApplicationAttemptId appAttemptId)

Description copied from interface: RMApp

RMApp can have multiple application attempts RMAppAttempt. This method returns the RMAppAttempt corresponding to ApplicationAttemptId.

Specified by:

getRMAppAttempt in interface RMApp

Parameters:

appAttemptId - the application attempt id

Returns:

the RMAppAttempt corresponding to the ApplicationAttemptId.

getQueue

public String getQueue()

Description copied from interface: RMApp

Each Application is submitted to a queue decided by ApplicationSubmissionContext.setQueue(String). This method returns the queue to which an application was submitted.

Specified by:

getQueue in interface RMApp

Returns:

the queue to which the application was submitted to.

setQueue

public void setQueue(String queue)

Description copied from interface: RMApp

Reflects a change in the application's queue from the one specified in the ApplicationSubmissionContext.

Specified by:

setQueue in interface RMApp

Parameters:

queue - the new queue name

getCollectorData

public org.apache.hadoop.yarn.server.api.records.AppCollectorData getCollectorData()

Description copied from interface: RMApp

The timeline collector information for the application. It should be used only if the timeline service v.2 is enabled.

Specified by:

getCollectorData in interface RMApp

Returns:

the data for the application's collector, including collector address, RM ID, version and collector token. Return null if the timeline service v.2 is not enabled.

setCollectorData

public void setCollectorData(org.apache.hadoop.yarn.server.api.records.AppCollectorData incomingData)

getCollectorInfo

public org.apache.hadoop.yarn.api.records.CollectorInfo getCollectorInfo()

Description copied from interface: RMApp

The timeline collector information to be sent to AM. It should be used only if the timeline service v.2 is enabled.

Specified by:

getCollectorInfo in interface RMApp

Returns:

collector info, including collector address and collector token. Return null if the timeline service v.2 is not enabled.

removeCollectorData

public void removeCollectorData()

getName

public String getName()

Description copied from interface: RMApp

The name of the application as set in ApplicationSubmissionContext.setApplicationName(String).

Specified by:

getName in interface RMApp

Returns:

the name of the application.

getCurrentAppAttempt

public RMAppAttempt getCurrentAppAttempt()

Description copied from interface: RMApp

RMApp can have multiple application attempts RMAppAttempt. This method returns the current RMAppAttempt.

Specified by:

getCurrentAppAttempt in interface RMApp

Returns:

the current RMAppAttempt

getAppAttempts

public Map<org.apache.hadoop.yarn.api.records.ApplicationAttemptId,RMAppAttempt> getAppAttempts()

Description copied from interface: RMApp

RMApp can have multiple application attempts RMAppAttempt. This method returns the all RMAppAttempts for the RMApp.

Specified by:

getAppAttempts in interface RMApp

Returns:

all RMAppAttempts for the RMApp.

pullRMNodeUpdates

public int pullRMNodeUpdates(Map<RMNode,org.apache.hadoop.yarn.api.records.NodeUpdateType> upNodes)

Description copied from interface: RMApp

To receive the collection of all RMNodes whose updates have been received by the RMApp. Updates can be node becoming lost or becoming healthy etc. The method clears the information from the RMApp. So each call to this method gives the delta from the previous call.

Specified by:

pullRMNodeUpdates in interface RMApp

Parameters:

upNodes - Map into which the updates are transferred, with each node updates as the key, and the NodeUpdateType for that update as the corresponding value.

Returns:

the number of nodes added to the Map

createAndGetApplicationReport

public org.apache.hadoop.yarn.api.records.ApplicationReport createAndGetApplicationReport(String clientUserName,
                                                                                          boolean allowAccess)

Description copied from interface: RMApp

To get the status of an application in the RM, this method can be used. If full access is not allowed then the following fields in the report will be stubbed:

• host - set to "N/A"
• RPC port - set to -1
• client token - set to "N/A"
• diagnostics - set to "N/A"
• tracking URL - set to "N/A"
• original tracking URL - set to "N/A"
• resource usage report - all values are -1

Specified by:

createAndGetApplicationReport in interface RMApp

Parameters:

clientUserName - the user name of the client requesting the report

allowAccess - whether to allow full access to the report

Returns:

the ApplicationReport detailing the status of the application.

getFinishTime

public long getFinishTime()

Description copied from interface: RMApp

The finish time of the RMApp

Specified by:

getFinishTime in interface RMApp

Returns:

the finish time of the application.,

getStartTime

public long getStartTime()

Description copied from interface: RMApp

the start time of the application.

Specified by:

getStartTime in interface RMApp

Returns:

the start time of the application.

getSubmitTime

public long getSubmitTime()

Description copied from interface: RMApp

the submit time of the application.

Specified by:

getSubmitTime in interface RMApp

Returns:

the submit time of the application.

getTrackingUrl

public String getTrackingUrl()

Description copied from interface: RMApp

The tracking url for the application master.

Specified by:

getTrackingUrl in interface RMApp

Returns:

the tracking url for the application master.

getOriginalTrackingUrl

public String getOriginalTrackingUrl()

Description copied from interface: RMApp

The original tracking url for the application master.

Specified by:

getOriginalTrackingUrl in interface RMApp

Returns:

the original tracking url for the application master.

getDiagnostics

public StringBuilder getDiagnostics()

Description copied from interface: RMApp

the diagnostics information for the application master.

Specified by:

getDiagnostics in interface RMApp

Returns:

the diagnostics information for the application master.

getMaxAppAttempts

public int getMaxAppAttempts()

Description copied from interface: RMApp

The number of max attempts of the application.

Specified by:

getMaxAppAttempts in interface RMApp

Returns:

the number of max attempts of the application.

handle

public void handle(RMAppEvent event)

Specified by:

handle in interface org.apache.hadoop.yarn.event.EventHandler<RMAppEvent>

recover

public void recover(RMStateStore.RMState state)

Specified by:

recover in interface Recoverable

getNumFailedAppAttempts

public int getNumFailedAppAttempts()

getApplicationType

public String getApplicationType()

Description copied from interface: RMApp

Returns the application type

Specified by:

getApplicationType in interface RMApp

Returns:

the application type.

getApplicationTags

public Set<String> getApplicationTags()

Description copied from interface: RMApp

Get tags for the application

Specified by:

getApplicationTags in interface RMApp

Returns:

tags corresponding to the application

isAppFinalStateStored

public boolean isAppFinalStateStored()

Description copied from interface: RMApp

Check whether this application's state has been saved to the state store.

Specified by:

isAppFinalStateStored in interface RMApp

Returns:

the flag indicating whether the applications's state is stored.

createApplicationState

public org.apache.hadoop.yarn.api.records.YarnApplicationState createApplicationState()

Description copied from interface: RMApp

Create the external user-facing state of ApplicationMaster from the current state of the RMApp.

Specified by:

createApplicationState in interface RMApp

Returns:

the external user-facing state of ApplicationMaster.

isAppInFinalState

public static boolean isAppInFinalState(RMApp rmApp)

isAppInCompletedStates

public boolean isAppInCompletedStates()

Description copied from interface: RMApp

To verify whether app has reached in its completing/completed states.

Specified by:

isAppInCompletedStates in interface RMApp

Returns:

True/False to confirm whether app is in final states

getApplicationPlacementContext

public ApplicationPlacementContext getApplicationPlacementContext()

Description copied from interface: RMApp

Get the application -> queue placement context

Specified by:

getApplicationPlacementContext in interface RMApp

Returns:

ApplicationPlacementContext

getRecoveredFinalState

public RMAppState getRecoveredFinalState()

getRanNodes

public Set<org.apache.hadoop.yarn.api.records.NodeId> getRanNodes()

Description copied from interface: RMApp

Nodes on which the containers for this RMApp ran.

Specified by:

getRanNodes in interface RMApp

Returns:

the set of nodes that ran any containers from this RMApp Add more node on which containers for this RMApp ran

getRMAppMetrics

public RMAppMetrics getRMAppMetrics()

Description copied from interface: RMApp

Get RMAppMetrics of the RMApp.

Specified by:

getRMAppMetrics in interface RMApp

Returns:

metrics

setSystemClock

@InterfaceAudience.Private
public void setSystemClock(org.apache.hadoop.yarn.util.Clock clock)

getReservationId

public org.apache.hadoop.yarn.api.records.ReservationId getReservationId()

Specified by:

getReservationId in interface RMApp

getAMResourceRequests

public List<org.apache.hadoop.yarn.api.records.ResourceRequest> getAMResourceRequests()

Specified by:

getAMResourceRequests in interface RMApp

getLogAggregationReportsForApp

public Map<org.apache.hadoop.yarn.api.records.NodeId,org.apache.hadoop.yarn.server.api.protocolrecords.LogAggregationReport> getLogAggregationReportsForApp()

Specified by:

getLogAggregationReportsForApp in interface RMApp

aggregateLogReport

public void aggregateLogReport(org.apache.hadoop.yarn.api.records.NodeId nodeId,
                               org.apache.hadoop.yarn.server.api.protocolrecords.LogAggregationReport report)

getLogAggregationStatusForAppReport

public org.apache.hadoop.yarn.api.records.LogAggregationStatus getLogAggregationStatusForAppReport()

Specified by:

getLogAggregationStatusForAppReport in interface RMApp

getLogAggregationFailureMessagesForNM

public String getLogAggregationFailureMessagesForNM(org.apache.hadoop.yarn.api.records.NodeId nodeId)

getAppNodeLabelExpression

public String getAppNodeLabelExpression()

Specified by:

getAppNodeLabelExpression in interface RMApp

getAmNodeLabelExpression

public String getAmNodeLabelExpression()

Description copied from interface: RMApp

Return the node label expression of the AM container.

Specified by:

getAmNodeLabelExpression in interface RMApp

getCallerContext

public org.apache.hadoop.ipc.CallerContext getCallerContext()

Specified by:

getCallerContext in interface RMApp

getNextAttemptId

@InterfaceAudience.Private
public int getNextAttemptId()

getApplicationTimeouts

public Map<org.apache.hadoop.yarn.api.records.ApplicationTimeoutType,Long> getApplicationTimeouts()

Specified by:

getApplicationTimeouts in interface RMApp

updateApplicationTimeout

public void updateApplicationTimeout(Map<org.apache.hadoop.yarn.api.records.ApplicationTimeoutType,Long> updateTimeout)

getApplicationPriority

public org.apache.hadoop.yarn.api.records.Priority getApplicationPriority()

Description copied from interface: RMApp

Get priority of the application.

Specified by:

getApplicationPriority in interface RMApp

Returns:

priority

setApplicationPriority

public void setApplicationPriority(org.apache.hadoop.yarn.api.records.Priority applicationPriority)

getApplicationSchedulingEnvs

public Map<String,String> getApplicationSchedulingEnvs()

Description copied from interface: RMApp

Get the application scheduling environment variables.

Specified by:

getApplicationSchedulingEnvs in interface RMApp

Returns:

Map of envs related to application scheduling preferences.

onInvalidStateTransition

protected void onInvalidStateTransition(RMAppEventType rmAppEventType,
                                        RMAppState state)

catch the InvalidStateTransition.

Parameters:

state -

rmAppEventType -