org.apache.hadoop.registry.server.integration

Class RMRegistryOperationsService

java.lang.Object

org.apache.hadoop.service.AbstractService

org.apache.hadoop.service.CompositeService

org.apache.hadoop.registry.client.impl.zk.CuratorService

org.apache.hadoop.registry.client.impl.zk.RegistryOperationsService

org.apache.hadoop.registry.server.services.RegistryAdminService

org.apache.hadoop.registry.server.integration.RMRegistryOperationsService

All Implemented Interfaces:

Closeable, AutoCloseable, RegistryConstants, RegistryOperations, RegistryBindingSource, org.apache.hadoop.service.Service

@InterfaceAudience.LimitedPrivate(value="YARN")
@InterfaceStability.Evolving
public class RMRegistryOperationsService
extends RegistryAdminService

Handle RM events by updating the registry

These actions are all implemented as event handlers to operations which come from the RM.

This service is expected to be executed by a user with the permissions to manipulate the entire registry,

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.registry.server.services.RegistryAdminService

RegistryAdminService.AsyncPurge, RegistryAdminService.NodeSelector, RegistryAdminService.PurgePolicy

Nested classes/interfaces inherited from class org.apache.hadoop.service.CompositeService

org.apache.hadoop.service.CompositeService.CompositeServiceShutdownHook

Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service

org.apache.hadoop.service.Service.STATE

Field Summary

Fields inherited from class org.apache.hadoop.registry.server.services.RegistryAdminService

executor, USER_HOMEDIR_ACL_PERMISSIONS

Fields inherited from class org.apache.hadoop.service.CompositeService

STOP_ONLY_STARTED_SERVICES

Fields inherited from interface org.apache.hadoop.registry.client.api.RegistryConstants

DEFAULT_REGISTRY_CLIENT_JAAS_CONTEXT, DEFAULT_REGISTRY_ENABLED, DEFAULT_REGISTRY_SECURE, DEFAULT_REGISTRY_SYSTEM_ACCOUNTS, DEFAULT_REGISTRY_USER_ACCOUNTS, DEFAULT_REGISTRY_ZK_QUORUM, DEFAULT_ZK_CONNECTION_TIMEOUT, DEFAULT_ZK_REGISTRY_ROOT, DEFAULT_ZK_RETRY_CEILING, DEFAULT_ZK_RETRY_INTERVAL, DEFAULT_ZK_RETRY_TIMES, DEFAULT_ZK_SESSION_TIMEOUT, KEY_REGISTRY_CLIENT_AUTH, KEY_REGISTRY_CLIENT_AUTHENTICATION_ID, KEY_REGISTRY_CLIENT_AUTHENTICATION_PASSWORD, KEY_REGISTRY_CLIENT_JAAS_CONTEXT, KEY_REGISTRY_ENABLED, KEY_REGISTRY_KERBEROS_REALM, KEY_REGISTRY_SECURE, KEY_REGISTRY_SYSTEM_ACCOUNTS, KEY_REGISTRY_USER_ACCOUNTS, KEY_REGISTRY_ZK_CONNECTION_TIMEOUT, KEY_REGISTRY_ZK_QUORUM, KEY_REGISTRY_ZK_RETRY_CEILING, KEY_REGISTRY_ZK_RETRY_INTERVAL, KEY_REGISTRY_ZK_RETRY_TIMES, KEY_REGISTRY_ZK_ROOT, KEY_REGISTRY_ZK_SESSION_TIMEOUT, PATH_SYSTEM_SERVICES, PATH_USER_SERVICES, PATH_USERS, REGISTRY_CLIENT_AUTH_ANONYMOUS, REGISTRY_CLIENT_AUTH_DIGEST, REGISTRY_CLIENT_AUTH_KERBEROS, REGISTRY_PREFIX, SUBPATH_COMPONENTS, ZK_PREFIX

Constructor Summary

Constructors

Constructor and Description
RMRegistryOperationsService(String name)
RMRegistryOperationsService(String name, RegistryBindingSource bindingSource)

Method Summary

Methods

Modifier and Type	Method and Description
RegistryAdminService.PurgePolicy	getPurgeOnCompletionPolicy()
void	onAMContainerFinished(org.apache.hadoop.yarn.api.records.ContainerId containerId) Actions to take when the AM container is completed
void	onApplicationAttemptAdded(org.apache.hadoop.yarn.api.records.ApplicationAttemptId appAttemptId)
void	onApplicationAttemptRegistered(org.apache.hadoop.yarn.api.records.ApplicationAttemptId attemptId, String host, int rpcport, String trackingurl)
void	onApplicationAttemptUnregistered(org.apache.hadoop.yarn.api.records.ApplicationAttemptId attemptId) Actions to take when an application attempt is completed
void	onApplicationCompleted(org.apache.hadoop.yarn.api.records.ApplicationId id) Actions to take when an application is completed
void	onApplicationLaunched(org.apache.hadoop.yarn.api.records.ApplicationId id)
void	onApplicationMasterRegistered(org.apache.hadoop.yarn.api.records.ApplicationAttemptId attemptId) Actions to take as an AM registers itself with the RM.
void	onContainerFinished(org.apache.hadoop.yarn.api.records.ContainerId id) Actions to take when the AM container is completed
void	onStateStoreEvent(org.apache.hadoop.yarn.api.records.ApplicationId applicationId, String user) This is the event where the user is known, so the user directory can be created
protected void	purgeAppAttemptRecords(org.apache.hadoop.yarn.api.records.ApplicationAttemptId attemptId) remove all application attempt entries
Future<Integer>	purgeRecordsAsync(String path, String id, String persistencePolicyMatch) Queue an async operation to purge all matching records under a base path.
Future<Integer>	purgeRecordsAsync(String path, String id, String persistencePolicyMatch, RegistryAdminService.PurgePolicy purgePolicy, org.apache.curator.framework.api.BackgroundCallback callback) Queue an async operation to purge all matching records under a base path.
protected void	serviceInit(org.apache.hadoop.conf.Configuration conf) Extend the parent service initialization by verifying that the service knows —in a secure cluster— the realm in which it is executing.
void	setPurgeOnCompletionPolicy(RegistryAdminService.PurgePolicy purgeOnCompletionPolicy)

Methods inherited from class org.apache.hadoop.registry.server.services.RegistryAdminService

aclsForUser, createDirAsync, createRootRegistryPaths, getExecutor, homeDir, initUserRegistry, initUserRegistryAsync, purge, serviceStart, serviceStop, stopExecutor, submit, verifyRealmValidity

Methods inherited from class org.apache.hadoop.registry.client.impl.zk.RegistryOperationsService

bind, delete, exists, getClientAcls, list, mknode, resolve, stat, validatePath

Methods inherited from class org.apache.hadoop.registry.client.impl.zk.CuratorService

addWriteAccessor, bindingDiagnosticDetails, buildConnectionString, buildSecurityDiagnostics, clearWriteAccessors, createEnsembleProvider, createFullPath, dumpPath, dumpRegistryRobustly, getBindingSource, getRegistrySecurity, isSecure, maybeCreate, operationFailure, operationFailure, supplyBindingInformation, toString, zkCreate, zkDelete, zkGetACLS, zkList, zkMkParentPath, zkMkPath, zkPathExists, zkPathMustExist, zkRead, zkSet, zkStat, zkUpdate

Methods inherited from class org.apache.hadoop.service.CompositeService

addIfService, addService, getServices, removeService

Methods inherited from class org.apache.hadoop.service.AbstractService

close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, setConfig, start, stop, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.hadoop.registry.client.api.RegistryOperations

addWriteAccessor, clearWriteAccessors

Methods inherited from interface org.apache.hadoop.service.Service

close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, registerServiceListener, start, stop, unregisterServiceListener, waitForServiceToStop

Constructor Detail

RMRegistryOperationsService

public RMRegistryOperationsService(String name)

RMRegistryOperationsService

public RMRegistryOperationsService(String name,
                           RegistryBindingSource bindingSource)

Method Detail

serviceInit

protected void serviceInit(org.apache.hadoop.conf.Configuration conf)
                    throws Exception

Extend the parent service initialization by verifying that the service knows —in a secure cluster— the realm in which it is executing. It needs this to properly build up the user names and hence their access rights.

Overrides:

serviceInit in class RegistryAdminService

Parameters:

conf - configuration of the service

Throws:

Exception

getPurgeOnCompletionPolicy

public RegistryAdminService.PurgePolicy getPurgeOnCompletionPolicy()

setPurgeOnCompletionPolicy

public void setPurgeOnCompletionPolicy(RegistryAdminService.PurgePolicy purgeOnCompletionPolicy)

onApplicationAttemptRegistered

public void onApplicationAttemptRegistered(org.apache.hadoop.yarn.api.records.ApplicationAttemptId attemptId,
                                  String host,
                                  int rpcport,
                                  String trackingurl)
                                    throws IOException

Throws:

IOException

onApplicationLaunched

public void onApplicationLaunched(org.apache.hadoop.yarn.api.records.ApplicationId id)
                           throws IOException

Throws:

IOException

onApplicationMasterRegistered

public void onApplicationMasterRegistered(org.apache.hadoop.yarn.api.records.ApplicationAttemptId attemptId)
                                   throws IOException

Actions to take as an AM registers itself with the RM.

Parameters:

attemptId - attempt ID

Throws:

IOException - problems

onAMContainerFinished

public void onAMContainerFinished(org.apache.hadoop.yarn.api.records.ContainerId containerId)
                           throws IOException

Actions to take when the AM container is completed

Parameters:

containerId - container ID

Throws:

IOException - problems

purgeAppAttemptRecords

protected void purgeAppAttemptRecords(org.apache.hadoop.yarn.api.records.ApplicationAttemptId attemptId)

remove all application attempt entries

Parameters:

attemptId - attempt ID

onApplicationAttemptUnregistered

public void onApplicationAttemptUnregistered(org.apache.hadoop.yarn.api.records.ApplicationAttemptId attemptId)
                                      throws IOException

Actions to take when an application attempt is completed

Parameters:

attemptId - application ID

Throws:

IOException - problems

onApplicationCompleted

public void onApplicationCompleted(org.apache.hadoop.yarn.api.records.ApplicationId id)
                            throws IOException

Actions to take when an application is completed

Parameters:

id - application ID

Throws:

IOException - problems

onApplicationAttemptAdded

public void onApplicationAttemptAdded(org.apache.hadoop.yarn.api.records.ApplicationAttemptId appAttemptId)

onStateStoreEvent

public void onStateStoreEvent(org.apache.hadoop.yarn.api.records.ApplicationId applicationId,
                     String user)
                       throws IOException

This is the event where the user is known, so the user directory can be created

Parameters:

applicationId - application ID

user - username

Throws:

IOException - problems

onContainerFinished

public void onContainerFinished(org.apache.hadoop.yarn.api.records.ContainerId id)
                         throws IOException

Actions to take when the AM container is completed

Parameters:

id - container ID

Throws:

IOException - problems

purgeRecordsAsync

public Future<Integer> purgeRecordsAsync(String path,
                                String id,
                                String persistencePolicyMatch)

Queue an async operation to purge all matching records under a base path.

1. Uses a depth first search
2. A match is on ID and persistence policy, or, if policy==-1, any match
3. If a record matches then it is deleted without any child searches
4. Deletions will be asynchronous if a callback is provided

Parameters:

path - base path

id - ID for service record.id

persistencePolicyMatch - ID for the persistence policy to match: no match, no delete.

Returns:

a future that returns the #of records deleted

purgeRecordsAsync

public Future<Integer> purgeRecordsAsync(String path,
                                String id,
                                String persistencePolicyMatch,
                                RegistryAdminService.PurgePolicy purgePolicy,
                                org.apache.curator.framework.api.BackgroundCallback callback)

Queue an async operation to purge all matching records under a base path.

1. Uses a depth first search
2. A match is on ID and persistence policy, or, if policy==-1, any match
3. If a record matches then it is deleted without any child searches
4. Deletions will be asynchronous if a callback is provided

Parameters:

path - base path

id - ID for service record.id

persistencePolicyMatch - ID for the persistence policy to match: no match, no delete.

purgePolicy - how to react to children under the entry

callback - an optional callback

Returns:

a future that returns the #of records deleted