Package org.apache.hadoop.yarn.service.client

Class ServiceClient

java.lang.Object

org.apache.hadoop.service.AbstractService

org.apache.hadoop.service.CompositeService

org.apache.hadoop.yarn.client.api.AppAdminClient

org.apache.hadoop.yarn.service.client.ServiceClient

All Implemented Interfaces:

Closeable, AutoCloseable, Service, org.apache.hadoop.yarn.service.conf.SliderExitCodes, org.apache.hadoop.yarn.service.conf.YarnServiceConstants, org.apache.hadoop.yarn.service.exceptions.LauncherExitCodes

@Public
@Unstable
public class ServiceClient
extends AppAdminClient
implements org.apache.hadoop.yarn.service.conf.SliderExitCodes, org.apache.hadoop.yarn.service.conf.YarnServiceConstants

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.service.CompositeService

org.apache.hadoop.service.CompositeService.CompositeServiceShutdownHook

Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service

Service.STATE

Field Summary

Fields

Modifier and Type	Field	Description
protected YarnClient	yarnClient

Fields inherited from class org.apache.hadoop.yarn.client.api.AppAdminClient

DEFAULT_CLASS_NAME, DEFAULT_TYPE, UNIT_TEST_CLASS_NAME, UNIT_TEST_TYPE, YARN_APP_ADMIN_CLIENT_PREFIX

Fields inherited from class org.apache.hadoop.service.CompositeService

STOP_ONLY_STARTED_SERVICES

Fields inherited from interface org.apache.hadoop.yarn.service.exceptions.LauncherExitCodes

EXIT_CLIENT_INITIATED_SHUTDOWN, EXIT_COMMAND_ARGUMENT_ERROR, EXIT_CONFLICT, EXIT_CONNECTIVITY_PROBLEM, EXIT_EXCEPTION_THROWN, EXIT_FALSE, EXIT_FORBIDDEN, EXIT_FOUND, EXIT_INTERNAL_ERROR, EXIT_INTERRUPTED, EXIT_MOVED, EXIT_NOT_ACCEPTABLE, EXIT_NOT_FOUND, EXIT_NOT_MODIFIED, EXIT_OPERATION_NOT_ALLOWED, EXIT_OTHER_FAILURE, EXIT_SERVICE_UNAVAILABLE, EXIT_SUCCESS, EXIT_TASK_LAUNCH_FAILURE, EXIT_UNAUTHORIZED, EXIT_UNIMPLEMENTED, EXIT_UNSUPPORTED_VERSION, EXIT_USAGE

Fields inherited from interface org.apache.hadoop.yarn.service.conf.SliderExitCodes

_EXIT_CODE_BASE, EXIT_APPLICATION_IN_USE, EXIT_BAD_CONFIGURATION, EXIT_BAD_STATE, EXIT_DEPLOYMENT_FAILED, EXIT_INSTANCE_EXISTS, EXIT_PROCESS_FAILED, EXIT_TIMED_OUT, EXIT_UNKNOWN_INSTANCE, EXIT_YARN_SERVICE_FAILED, EXIT_YARN_SERVICE_FINISHED_WITH_ERROR, EXIT_YARN_SERVICE_KILLED

Fields inherited from interface org.apache.hadoop.yarn.service.conf.YarnServiceConstants

APP_CONF_DIR, APP_LIB_DIR, APP_RESOURCES_DIR, APP_TYPE, CONTAINER_STATE_REPORT_AS_SERVICE_STATE, CONTENT, DEPENDENCY_DIR, DEPENDENCY_DIR_PERMISSIONS, DEPENDENCY_LOCALIZED_DIR_LINK, DEPENDENCY_TAR_GZ_FILE_EXT, DEPENDENCY_TAR_GZ_FILE_NAME, ERR_FILE, HADOOP_USER_NAME, KEYTAB_DIR, KEYTAB_LOCATION, OUT_FILE, PRINCIPAL, PROPERTY_LIB_DIR, RESOURCE_DIR, SERVICE_BASE_DIRECTORY, SERVICE_CORE_JAR, SERVICES_DIRECTORY, SERVICES_PUBLIC_DIRECTORY, STDERR_AM, STDOUT_AM, SUBMITTED_CONF_DIR, SYSPROP_LOG_DIR, SYSPROP_LOG4J_CONFIGURATION, TMP_DIR_PREFIX, UPGRADE_DIR, YARN_SERVICE_LOG4J_FILENAME

Constructor Summary

Constructors

Constructor	Description
ServiceClient()

Method Summary

Modifier and Type	Method	Description
int	actionBuild(Service service)
int	actionCancelUpgrade(String appName)	Cancels the upgrade of the service.
int	actionCleanUp(String appName, String userName)	Operation to be performed by the RM after an application has completed.
ApplicationId	actionCreate(Service service)
int	actionDecommissionInstances(String appName, List<String> componentInstances)	Decommission component instances of a long running service.
int	actionDependency(String destinationFolder, boolean overwrite)
int	actionDestroy(String serviceName)	Remove the specification and all application data for a YARN application.
int	actionFlex(String serviceName, Map<String,String> componentCountStrings)	Change the number of running containers for a component of a YARN application / long-running service.
int	actionLaunch(String fileName, String serviceName, Long lifetime, String queue)	Launch a new YARN application.
int	actionSave(String fileName, String serviceName, Long lifetime, String queue)	Save the specification for a YARN application / long-running service.
int	actionStart(String serviceName)	Start a YARN application from a previously saved specification.
ApplicationId	actionStartAndGetId(String serviceName)
int	actionStop(String serviceName)	Stop a YARN application (attempt to stop gracefully before killing the application).
int	actionStop(String serviceName, boolean waitForAppStopped)
int	actionUpgrade(Service service, List<Container> compInstances)
int	actionUpgradeComponents(String appName, List<String> components)	Upgrade components of a long running service.
int	actionUpgradeExpress(String appName, File path)	Express upgrade a long running service.
int	actionUpgradeExpress(Service service)
int	actionUpgradeInstances(String appName, List<String> componentInstances)	Upgrade component instances of a long running service.
protected Map<String,String>	addAMEnv()
protected Path	addJarResource(String serviceName, Map<String,LocalResource> localResources)
static File	compressFiles(Collection<File> files, File output, String bundleRoot)	Compress (tar) the input files to the output file.
ServiceState	convertState(YarnApplicationState state)
protected org.apache.hadoop.yarn.service.ClientAMProtocol	createAMProxy(String serviceName, ApplicationReport appReport)
int	enableFastLaunch(String destinationFolder)	Upload AM dependencies to HDFS.
Map<String,Long>	flexByRestService(String serviceName, Map<String,Long> componentCounts)
ApplicationId	getAppId(String serviceName)
ComponentContainers[]	getContainers(String appName, List<String> components, String version, List<ContainerState> containerStates)
String	getInstances(String appName, List<String> components, String version, List<String> containerStates)
Service	getStatus(String serviceName)
String	getStatusString(String appIdOrName)	Get detailed app specific status string for a YARN application.
YarnClient	getYarnClient()
int	initiateUpgrade(String appName, String fileName, boolean autoFinalize)	Initiate upgrade of a long running service.
int	initiateUpgrade(Service service)
Service	loadAppJsonFromLocalFS(String fileName, String serviceName, Long lifetime, String queue)
protected void	serviceInit(Configuration configuration)	All initialization code needed by a service.
protected void	serviceStop()	Actions called during the transition to the STOPPED state.
String	updateLifetime(String serviceName, long lifetime)

Methods inherited from class org.apache.hadoop.yarn.client.api.AppAdminClient

createAppAdminClient

Methods inherited from class org.apache.hadoop.service.CompositeService

addIfService, addService, getServices, removeService, serviceStart

Methods inherited from class org.apache.hadoop.service.AbstractService

close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, setConfig, start, stop, toString, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Details

yarnClient

protected YarnClient yarnClient

Constructor Details

ServiceClient

public ServiceClient()

Method Details

serviceInit

protected void serviceInit(Configuration configuration)
                    throws Exception

Description copied from class: AbstractService

All initialization code needed by a service. This method will only ever be called once during the lifecycle of a specific service instance. Implementations do not need to be synchronized as the logic in AbstractService.init(Configuration) prevents re-entrancy. The base implementation checks to see if the subclass has created a new configuration instance, and if so, updates the base class value

Overrides:

serviceInit in class CompositeService

Parameters:

configuration - configuration

Throws:

Exception - on a failure -these will be caught, possibly wrapped, and will trigger a service stop

serviceStop

protected void serviceStop()
                    throws Exception

Description copied from class: AbstractService

Actions called during the transition to the STOPPED state. This method will only ever be called once during the lifecycle of a specific service instance. Implementations do not need to be synchronized as the logic in AbstractService.stop() prevents re-entrancy. Implementations MUST write this to be robust against failures, including checks for null references -and for the first failure to not stop other attempts to shut down parts of the service.

Overrides:

serviceStop in class CompositeService

Throws:

Exception - if needed -these will be caught and logged.

loadAppJsonFromLocalFS

public Service loadAppJsonFromLocalFS(String fileName,
 String serviceName,
 Long lifetime,
 String queue)
                               throws IOException,
YarnException

Throws:

IOException

YarnException

actionSave

public int actionSave(String fileName,
 String serviceName,
 Long lifetime,
 String queue)
               throws IOException,
YarnException

Description copied from class: AppAdminClient

Save the specification for a YARN application / long-running service. The application may be started later.

Specified by:

actionSave in class AppAdminClient

Parameters:

fileName - specification of application to save

serviceName - name of the application

lifetime - lifetime of the application

queue - queue of the application

Returns:

exit code

Throws:

IOException - IOException

YarnException - exception in client or server

actionBuild

public int actionBuild(Service service)
                throws YarnException,
IOException

Throws:

YarnException

IOException

actionUpgradeExpress

public int actionUpgradeExpress(String appName,
 File path)
                         throws IOException,
YarnException

Description copied from class: AppAdminClient

Express upgrade a long running service.

Specified by:

actionUpgradeExpress in class AppAdminClient

Parameters:

appName - the name of the application

path - specification of application upgrade to save.

Returns:

exit code

Throws:

IOException - io error occur.

YarnException - exceptions from yarn servers.

actionUpgradeExpress

public int actionUpgradeExpress(Service service)
                         throws YarnException,
IOException

Throws:

YarnException

IOException

initiateUpgrade

public int initiateUpgrade(String appName,
 String fileName,
 boolean autoFinalize)
                    throws IOException,
YarnException

Description copied from class: AppAdminClient

Initiate upgrade of a long running service.

Specified by:

initiateUpgrade in class AppAdminClient

Parameters:

appName - the name of the application.

fileName - specification of application upgrade to save.

autoFinalize - when true, finalization of upgrade will be done automatically.

Returns:

exit code

Throws:

IOException - IOException

YarnException - exception in client or server

initiateUpgrade

public int initiateUpgrade(Service service)
                    throws YarnException,
IOException

Throws:

YarnException

IOException

actionUpgradeInstances

public int actionUpgradeInstances(String appName,
 List<String> componentInstances)
                           throws IOException,
YarnException

Description copied from class: AppAdminClient

Upgrade component instances of a long running service.

Specified by:

actionUpgradeInstances in class AppAdminClient

Parameters:

appName - the name of the application.

componentInstances - the name of the component instances.

Returns:

exit code.

Throws:

IOException - io error occur.

YarnException - exceptions from yarn servers.

actionUpgradeComponents

public int actionUpgradeComponents(String appName,
 List<String> components)
                            throws IOException,
YarnException

Description copied from class: AppAdminClient

Upgrade components of a long running service.

Specified by:

actionUpgradeComponents in class AppAdminClient

Parameters:

appName - the name of the application.

components - the name of the components.

Returns:

exit code.

Throws:

IOException - io error occur.

YarnException - exceptions from yarn servers.

actionCancelUpgrade

public int actionCancelUpgrade(String appName)
                        throws IOException,
YarnException

Description copied from class: AppAdminClient

Cancels the upgrade of the service.

Specified by:

actionCancelUpgrade in class AppAdminClient

Parameters:

appName - the name of the application

Returns:

exit code

Throws:

IOException - io error occur.

YarnException - exceptions from yarn servers.

actionDecommissionInstances

public int actionDecommissionInstances(String appName,
 List<String> componentInstances)
                                throws IOException,
YarnException

Description copied from class: AppAdminClient

Decommission component instances of a long running service.

Specified by:

actionDecommissionInstances in class AppAdminClient

Parameters:

appName - the name of the application.

componentInstances - the name of the component instances.

Returns:

exit code.

Throws:

IOException - io error occur.

YarnException - exceptions from yarn servers.

actionCleanUp

public int actionCleanUp(String appName,
 String userName)
                  throws IOException,
YarnException

Description copied from class: AppAdminClient

Operation to be performed by the RM after an application has completed.

Specified by:

actionCleanUp in class AppAdminClient

Parameters:

appName - the name of the application.

userName - the name of the user.

Returns:

exit code

Throws:

IOException - io error occur.

YarnException - exceptions from yarn servers.

getInstances

public String getInstances(String appName,
 List<String> components,
 String version,
 List<String> containerStates)
                    throws IOException,
YarnException

Specified by:

getInstances in class AppAdminClient

Throws:

IOException

YarnException

getContainers

public ComponentContainers[] getContainers(String appName,
 List<String> components,
 String version,
 List<ContainerState> containerStates)
                                    throws IOException,
YarnException

Throws:

IOException

YarnException

actionUpgrade

public int actionUpgrade(Service service,
 List<Container> compInstances)
                  throws IOException,
YarnException

Throws:

IOException

YarnException

actionLaunch

public int actionLaunch(String fileName,
 String serviceName,
 Long lifetime,
 String queue)
                 throws IOException,
YarnException

Description copied from class: AppAdminClient

Launch a new YARN application.

Specified by:

actionLaunch in class AppAdminClient

Parameters:

fileName - specification of application

serviceName - name of the application

lifetime - lifetime of the application

queue - queue of the application

Returns:

exit code

Throws:

IOException - IOException

YarnException - exception in client or server

actionCreate

public ApplicationId actionCreate(Service service)
                           throws IOException,
YarnException

Throws:

IOException

YarnException

actionFlex

public int actionFlex(String serviceName,
 Map<String,String> componentCountStrings)
               throws YarnException,
IOException

Description copied from class: AppAdminClient

Change the number of running containers for a component of a YARN application / long-running service.

Specified by:

actionFlex in class AppAdminClient

Parameters:

serviceName - the name of the application

componentCountStrings - map of component name to new component count or amount to change existing component count (e.g. 5, +5, -5)

Returns:

exit code

Throws:

YarnException - exception in client or server

IOException - IOException

flexByRestService

public Map<String,Long> flexByRestService(String serviceName,
 Map<String,Long> componentCounts)
                                   throws YarnException,
IOException

Throws:

YarnException

IOException

actionStop

public int actionStop(String serviceName)
               throws YarnException,
IOException

Description copied from class: AppAdminClient

Stop a YARN application (attempt to stop gracefully before killing the application). In the case of a long-running service, the service may be restarted later.

Specified by:

actionStop in class AppAdminClient

Parameters:

serviceName - the name of the application

Returns:

exit code

Throws:

YarnException - exception in client or server

IOException - IOException

actionStop

public int actionStop(String serviceName,
 boolean waitForAppStopped)
               throws YarnException,
IOException

Throws:

YarnException

IOException

actionDestroy

public int actionDestroy(String serviceName)
                  throws YarnException,
IOException

Description copied from class: AppAdminClient

Remove the specification and all application data for a YARN application. The application cannot be running.

Specified by:

actionDestroy in class AppAdminClient

Parameters:

serviceName - the name of the application

Returns:

exit code

Throws:

YarnException - exception in client or server

IOException - IOException

compressFiles

public static File compressFiles(Collection<File> files,
 File output,
 String bundleRoot)
                          throws IOException

Compress (tar) the input files to the output file.

Parameters:

files - The files to compress

output - The resulting output file (should end in .tar.gz)

bundleRoot -

Throws:

IOException

addAMEnv

@VisibleForTesting
protected Map<String,String> addAMEnv()
                               throws IOException

Throws:

IOException

addJarResource

protected Path addJarResource(String serviceName,
 Map<String,LocalResource> localResources)
                       throws IOException,
YarnException

Throws:

IOException

YarnException

actionStart

public int actionStart(String serviceName)
                throws YarnException,
IOException

Description copied from class: AppAdminClient

Start a YARN application from a previously saved specification. In the case of a long-running service, the service must have been previously launched/started and then stopped, or previously saved but not started.

Specified by:

actionStart in class AppAdminClient

Parameters:

serviceName - the name of the application

Returns:

exit code

Throws:

YarnException - exception in client or server

IOException - IOException

actionStartAndGetId

public ApplicationId actionStartAndGetId(String serviceName)
                                  throws YarnException,
IOException

Throws:

YarnException

IOException

updateLifetime

public String updateLifetime(String serviceName,
 long lifetime)
                      throws YarnException,
IOException

Throws:

YarnException

IOException

convertState

public ServiceState convertState(YarnApplicationState state)

getStatusString

public String getStatusString(String appIdOrName)
                       throws IOException,
YarnException

Description copied from class: AppAdminClient

Get detailed app specific status string for a YARN application.

Specified by:

getStatusString in class AppAdminClient

Parameters:

appIdOrName - appId or appName

Returns:

status string

Throws:

IOException - IOException

YarnException - exception in client or server

getStatus

public Service getStatus(String serviceName)
                  throws IOException,
YarnException

Throws:

IOException

YarnException

getYarnClient

public YarnClient getYarnClient()

enableFastLaunch

public int enableFastLaunch(String destinationFolder)
                     throws IOException,
YarnException

Description copied from class: AppAdminClient

Upload AM dependencies to HDFS. This makes future application launches faster since the dependencies do not have to be uploaded on each launch.

Specified by:

enableFastLaunch in class AppAdminClient

Parameters:

destinationFolder - an optional HDFS folder where dependency tarball will be uploaded

Returns:

exit code

Throws:

IOException - IOException

YarnException - exception in client or server

actionDependency

public int actionDependency(String destinationFolder,
 boolean overwrite)

createAMProxy

protected org.apache.hadoop.yarn.service.ClientAMProtocol createAMProxy(String serviceName,
 ApplicationReport appReport)
                                                                 throws IOException,
YarnException

Throws:

IOException

YarnException

getAppId

public ApplicationId getAppId(String serviceName)
                       throws IOException,
YarnException

Throws:

IOException

YarnException