org.apache.hadoop.service.launcher

Class ServiceLauncher<S extends Service>

java.lang.Object

org.apache.hadoop.service.launcher.ServiceLauncher<S>

Type Parameters:

S - service class to cast the generated service to.

All Implemented Interfaces:

Thread.UncaughtExceptionHandler, LauncherArguments, LauncherExitCodes

public class ServiceLauncher<S extends Service>
extends Object
implements LauncherExitCodes, LauncherArguments, Thread.UncaughtExceptionHandler

A class to launch any YARN service by name. It's designed to be subclassed for custom entry points. Workflow:

1. An instance of the class is created. It must be of the type Service
2. If it implements LaunchableService.bindArgs(Configuration, List), it is given the binding args off the CLI after all general configuration arguments have been stripped.
3. Its Service.init(Configuration) and Service.start() methods are called.
4. If it implements it, LaunchableService.execute() is called and its return code used as the exit code.
5. Otherwise: it waits for the service to stop, assuming that the Service.start() method spawns one or more thread to perform work
6. If any exception is raised and provides an exit code, that is, it implements ExitCodeProvider, the return value of ExitCodeProvider.getExitCode(), becomes the exit code of the command.

Error and warning messages are logged to stderr.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	ServiceLauncher.MinimalGenericOptionsParser A generic options parser which does not parse any of the traditional Hadoop options.

Field Summary

Fields

Modifier and Type	Field and Description
protected static String[]	DEFAULT_CONFIGS List of the standard configurations to create (and so load in properties).
static String	NAME The name of this class.
protected static int	SHUTDOWN_PRIORITY Priority for the shutdown hook: 30.
static String	USAGE_MESSAGE Usage message.
protected static String	USAGE_NAME
protected static String	USAGE_SERVICE_ARGUMENTS

Fields inherited from interface org.apache.hadoop.service.launcher.LauncherExitCodes

EXIT_BAD_CONFIGURATION, EXIT_CLIENT_INITIATED_SHUTDOWN, EXIT_COMMAND_ARGUMENT_ERROR, EXIT_CONNECTIVITY_PROBLEM, EXIT_EXCEPTION_THROWN, EXIT_FAIL, EXIT_FORBIDDEN, EXIT_INTERRUPTED, EXIT_NOT_ACCEPTABLE, EXIT_NOT_FOUND, EXIT_OPERATION_NOT_ALLOWED, EXIT_OTHER_FAILURE, EXIT_SERVICE_CREATION_FAILURE, EXIT_SERVICE_LIFECYCLE_EXCEPTION, EXIT_SERVICE_UNAVAILABLE, EXIT_SUCCESS, EXIT_TASK_LAUNCH_FAILURE, EXIT_UNAUTHORIZED, EXIT_UNIMPLEMENTED, EXIT_UNSUPPORTED_VERSION, EXIT_USAGE

Fields inherited from interface org.apache.hadoop.service.launcher.LauncherArguments

ARG_CONF, ARG_CONF_PREFIXED, ARG_CONF_SHORT, ARG_CONFCLASS, ARG_CONFCLASS_PREFIXED, ARG_CONFCLASS_SHORT, E_PARSE_FAILED

Constructor Summary

Constructors

Constructor and Description
ServiceLauncher(String serviceClassName) Create an instance of the launcher.
ServiceLauncher(String serviceName, String serviceClassName) Create an instance of the launcher.

Method Summary

Methods

Modifier and Type	Method and Description
protected void	bindCommandOptions() Set the commandOptions field to the result of createOptions(); protected for subclasses and test access.
protected static ExitUtil.ExitException	convertToExitException(Throwable thrown) Convert an exception to an ExitException.
protected int	coreServiceLaunch(Configuration conf, List<String> processedArgs, boolean addShutdownHook, boolean execute) Launch the service.
protected Configuration	createConfiguration() Override point: create the base configuration for the service.
protected org.apache.hadoop.util.GenericOptionsParser	createGenericOptionsParser(Configuration conf, String[] argArray) Override point: create a generic options parser or subclass thereof.
protected org.apache.commons.cli.Options	createOptions() Override point: create an options instance to combine with the standard options set.
protected void	error(String message, Throwable thrown) Report an error.
protected void	exit(ExitUtil.ExitException ee) Exit the JVM using an exception for the exit code and message, invoking ExitUtil.terminate(ExitUtil.ExitException).
protected void	exit(int exitCode, String message) Exit the JVM.
protected static void	exitWithMessage(int status, String message) Exit with a printed message.
protected static void	exitWithUsageMessage() Exit with the usage exit code LauncherExitCodes.EXIT_USAGE and message USAGE_MESSAGE.
List<String>	extractCommandOptions(Configuration conf, List<String> args) Extract the command options and apply them to the configuration, building an array of processed arguments to hand down to the service.
protected ClassLoader	getClassLoader() Override point: get the classloader to use.
Configuration	getConfiguration() Get the configuration constructed from the command line arguments.
protected List<String>	getConfigurationsToCreate() Override point: Get a list of configuration classes to create.
S	getService() Get the service.
ExitUtil.ExitException	getServiceException() Get the exit exception used to end this service.
int	getServiceExitCode() The exit code from a successful service execution.
String	getServiceName() Get the service name via Service.getName().
protected String	getUsageMessage() Get the usage message, ideally dynamically.
Service	instantiateService(Configuration conf) Instantiate the service defined in serviceClassName.
ExitUtil.ExitException	launchService(Configuration conf, List<String> processedArgs, boolean addShutdownHook, boolean execute) Launch a service catching all exceptions and downgrading them to exit codes after logging.
void	launchServiceAndExit(List<String> args) Launch the service and exit.
int	loadConfigurationClasses() This creates all the configurations defined by getConfigurationsToCreate() , ensuring that the resources have been pushed in.
static void	main(String[] args) This is the JVM entry point for the service launcher.
protected List<String>	parseCommandArgs(Configuration conf, List<String> args) Parse the command arguments, extracting the service class as the last element of the list (after extracting all the rest).
protected void	registerFailureHandling() Override point: register this class as the handler for the control-C and SIGINT interrupts.
protected ServiceLaunchException	serviceCreationFailure(Exception exception) Generate an exception announcing a failure to create the service.
static void	serviceMain(List<String> argsList) The real main function, which takes the arguments as a list.
static void	serviceMain(String... args) Varargs version of the entry point for testing and other in-JVM use.
protected void	setService(S s) Setter is to give subclasses the ability to manipulate the service.
protected static String	startupShutdownMessage(String classname, List<String> args) Build a log message for starting up and shutting down.
String	toString()
void	uncaughtException(Thread thread, Throwable exception) Handler for uncaught exceptions: terminate the service.
protected void	verifyConfigurationFilesExist(String[] filenames) Verify that all the specified filenames exist.
protected void	warn(String text) Print a warning message.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

SHUTDOWN_PRIORITY

protected static final int SHUTDOWN_PRIORITY

Priority for the shutdown hook: 30.

See Also:

Constant Field Values

NAME

public static final String NAME

The name of this class.

See Also:

Constant Field Values

USAGE_NAME

protected static final String USAGE_NAME

See Also:

Constant Field Values

USAGE_SERVICE_ARGUMENTS

protected static final String USAGE_SERVICE_ARGUMENTS

See Also:

Constant Field Values

USAGE_MESSAGE

public static final String USAGE_MESSAGE

Usage message. Text: "Usage: ServiceLauncher [--conf <conf file>] [--hadoopconf <configuration classname>] service-classname <service arguments>"

See Also:

Constant Field Values

DEFAULT_CONFIGS

protected static final String[] DEFAULT_CONFIGS

List of the standard configurations to create (and so load in properties). The values are Hadoop, HDFS and YARN configurations.

Constructor Detail

ServiceLauncher

public ServiceLauncher(String serviceClassName)

Create an instance of the launcher.

Parameters:

serviceClassName - classname of the service

ServiceLauncher

public ServiceLauncher(String serviceName,
               String serviceClassName)

Create an instance of the launcher.

Parameters:

serviceName - name of service for text messages

serviceClassName - classname of the service

Method Detail

getService

public final S getService()

Get the service. Null until coreServiceLaunch(Configuration, List, boolean, boolean) has completed.

Returns:

the service

setService

protected void setService(S s)

Setter is to give subclasses the ability to manipulate the service.

Parameters:

s - the new service

getConfiguration

public final Configuration getConfiguration()

Get the configuration constructed from the command line arguments.

Returns:

the configuration used to create the service

getServiceExitCode

public final int getServiceExitCode()

The exit code from a successful service execution.

Returns:

the exit code.

getServiceException

public final ExitUtil.ExitException getServiceException()

Get the exit exception used to end this service.

Returns:

an exception, which will be null until the service has exited (and System.exit has not been called)

toString

public String toString()

Overrides:

toString in class Object

launchServiceAndExit

public void launchServiceAndExit(List<String> args)

Launch the service and exit.

1. Parse the command line.
2. Build the service configuration from it.
3. Start the service.
.
4. If it is a LaunchableService: execute it
5. Otherwise: wait for it to finish.
6. Exit passing the status code to the exit(int, String) method.

Parameters:

args - arguments to the service. arg[0] is assumed to be the service classname.

bindCommandOptions

protected void bindCommandOptions()

Set the commandOptions field to the result of createOptions(); protected for subclasses and test access.

getUsageMessage

protected String getUsageMessage()

Get the usage message, ideally dynamically.

Returns:

the usage message

createOptions

protected org.apache.commons.cli.Options createOptions()

Override point: create an options instance to combine with the standard options set. Important. Synchronize uses of OptionBuilder with OptionBuilder.class

Returns:

the new options

createConfiguration

protected Configuration createConfiguration()

Override point: create the base configuration for the service. Subclasses can override to create HDFS/YARN configurations etc.

Returns:

the configuration to use as the service initializer.

getConfigurationsToCreate

protected List<String> getConfigurationsToCreate()

Override point: Get a list of configuration classes to create.

Returns:

the array of configs to attempt to create. If any are off the classpath, that is logged

loadConfigurationClasses

public int loadConfigurationClasses()

This creates all the configurations defined by getConfigurationsToCreate() , ensuring that the resources have been pushed in. If one cannot be loaded it is logged and the operation continues except in the case that the class does load but it isn't actually a subclass of Configuration.

Throws:

ExitUtil.ExitException - if a loaded class is of the wrong type

launchService

public ExitUtil.ExitException launchService(Configuration conf,
                                   List<String> processedArgs,
                                   boolean addShutdownHook,
                                   boolean execute)

Launch a service catching all exceptions and downgrading them to exit codes after logging. Sets serviceException to this value.

Parameters:

conf - configuration to use

processedArgs - command line after the launcher-specific arguments have been stripped out.

addShutdownHook - should a shutdown hook be added to terminate this service on shutdown. Tests should set this to false.

execute - execute/wait for the service to stop.

Returns:

an exit exception, which will have a status code of 0 if it worked

coreServiceLaunch

protected int coreServiceLaunch(Configuration conf,
                    List<String> processedArgs,
                    boolean addShutdownHook,
                    boolean execute)
                         throws Exception

Launch the service. All exceptions that occur are propagated upwards. If the method returns a status code, it means that it got as far starting the service, and if it implements LaunchableService, that the method LaunchableService.execute() has completed. After this method returns, the service can be retrieved returned by getService().

Parameters:

conf - configuration

processedArgs - arguments after the configuration parameters have been stripped out.

addShutdownHook - should a shutdown hook be added to terminate this service on shutdown. Tests should set this to false.

execute - execute/wait for the service to stop

Throws:

ClassNotFoundException - classname not on the classpath

IllegalAccessException - not allowed at the class

InstantiationException - not allowed to instantiate it

InterruptedException - thread interrupted

ExitUtil.ExitException - any exception defining the status code.

Exception - any other failure -if it implements ExitCodeProvider then it defines the exit code for any containing exception

instantiateService

public Service instantiateService(Configuration conf)

Instantiate the service defined in serviceClassName. Sets the configuration field to the the value of conf, and the service field to the service created.

Parameters:

conf - configuration to use

convertToExitException

protected static ExitUtil.ExitException convertToExitException(Throwable thrown)

Convert an exception to an ExitException. This process may just be a simple pass through, otherwise a new exception is created with an exit code, the text of the supplied exception, and the supplied exception as an inner cause.

1. If is already the right type, pass it through.
2. If it implements ExitCodeProvider.getExitCode(), the exit code is extracted and used in the new exception.
3. Otherwise, the exit code LauncherExitCodes.EXIT_EXCEPTION_THROWN is used.

Parameters:

thrown - the exception thrown

Returns:

an ExitException with a status code

serviceCreationFailure

protected ServiceLaunchException serviceCreationFailure(Exception exception)

Generate an exception announcing a failure to create the service.

Parameters:

exception - inner exception.

Returns:

a new exception, with the exit code LauncherExitCodes.EXIT_SERVICE_CREATION_FAILURE

registerFailureHandling

protected void registerFailureHandling()

Override point: register this class as the handler for the control-C and SIGINT interrupts. Subclasses can extend this with extra operations, such as an exception handler:

  Thread.setDefaultUncaughtExceptionHandler(
     new YarnUncaughtExceptionHandler());
 

uncaughtException

public void uncaughtException(Thread thread,
                     Throwable exception)

Handler for uncaught exceptions: terminate the service.

Specified by:

uncaughtException in interface Thread.UncaughtExceptionHandler

Parameters:

thread - thread

exception - exception

getServiceName

public String getServiceName()

Get the service name via Service.getName(). If the service is not instantiated, the classname is returned instead.

Returns:

the service name

warn

protected void warn(String text)

Print a warning message.

This tries to log to the log's warn() operation. If the log at that level is disabled it logs to system error

Parameters:

text - warning text

error

protected void error(String message,
         Throwable thrown)

Report an error.

This tries to log to LOG.error().

If that log level is disabled disabled the message is logged to system error along with thrown.toString()

Parameters:

message - message for the user

thrown - the exception thrown

exit

protected void exit(int exitCode,
        String message)

Exit the JVM. This is method can be overridden for testing, throwing an exception instead. Any subclassed method MUST raise an ExitException instance/subclass. The service launcher code assumes that after this method is invoked, no other code in the same method is called.

Parameters:

exitCode - code to exit

exit

protected void exit(ExitUtil.ExitException ee)

Exit the JVM using an exception for the exit code and message, invoking ExitUtil.terminate(ExitUtil.ExitException). This is the standard way a launched service exits. An error code of 0 means success -nothing is printed. If ExitUtil.disableSystemExit() has been called, this method will throw the exception. The method may be subclassed for testing

Parameters:

ee - exit exception

Throws:

ExitUtil.ExitException - if ExitUtil exceptions are disabled

getClassLoader

protected ClassLoader getClassLoader()

Override point: get the classloader to use.

Returns:

the classloader for loading a service class.

extractCommandOptions

public List<String> extractCommandOptions(Configuration conf,
                                 List<String> args)

Extract the command options and apply them to the configuration, building an array of processed arguments to hand down to the service.

Parameters:

conf - configuration to update.

args - main arguments. args[0]is assumed to be the service classname and is skipped.

Returns:

the remaining arguments

Throws:

ExitUtil.ExitException - if JVM exiting is disabled.

parseCommandArgs

protected List<String> parseCommandArgs(Configuration conf,
                            List<String> args)

Parse the command arguments, extracting the service class as the last element of the list (after extracting all the rest). The field commandOptions field must already have been set.

Parameters:

conf - configuration to use

args - command line argument list

Returns:

the remaining arguments

Throws:

ServiceLaunchException - if processing of arguments failed

createGenericOptionsParser

protected org.apache.hadoop.util.GenericOptionsParser createGenericOptionsParser(Configuration conf,
                                                                     String[] argArray)
                                                                          throws IOException

Override point: create a generic options parser or subclass thereof.

Parameters:

conf - Hadoop configuration

argArray - array of arguments

Returns:

a generic options parser to parse the arguments

Throws:

IOException - on any failure

verifyConfigurationFilesExist

protected void verifyConfigurationFilesExist(String[] filenames)

Verify that all the specified filenames exist.

Parameters:

filenames - a list of files

Throws:

ServiceLaunchException - if a file is not found

startupShutdownMessage

protected static String startupShutdownMessage(String classname,
                            List<String> args)

Build a log message for starting up and shutting down.

Parameters:

classname - the class of the server

args - arguments

exitWithMessage

protected static void exitWithMessage(int status,
                   String message)

Exit with a printed message.

Parameters:

status - status code

message - message message to print before exiting

Throws:

ExitUtil.ExitException - if exceptions are disabled

exitWithUsageMessage

protected static void exitWithUsageMessage()

Exit with the usage exit code LauncherExitCodes.EXIT_USAGE and message USAGE_MESSAGE.

Throws:

ExitUtil.ExitException - if exceptions are disabled

main

public static void main(String[] args)

This is the JVM entry point for the service launcher. Converts the arguments to a list, then invokes serviceMain(List)

Parameters:

args - command line arguments.

serviceMain

public static void serviceMain(String... args)

Varargs version of the entry point for testing and other in-JVM use. Hands off to serviceMain(List)

Parameters:

args - command line arguments.

serviceMain

public static void serviceMain(List<String> argsList)

The real main function, which takes the arguments as a list. Argument 0 MUST be the service classname

Parameters:

argsList - the list of arguments