Package org.apache.hadoop.yarn.client.api.impl

Class SharedCacheClientImpl

java.lang.Object
org.apache.hadoop.service.AbstractService
org.apache.hadoop.yarn.client.api.SharedCacheClient
org.apache.hadoop.yarn.client.api.impl.SharedCacheClientImpl
All Implemented Interfaces:
Closeable, AutoCloseable, org.apache.hadoop.service.Service
@Private @Unstable public class SharedCacheClientImpl extends SharedCacheClient
An implementation of the SharedCacheClient API.

  • Nested Class Summary

    Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service

    org.apache.hadoop.service.Service.STATE

  • Constructor Summary

    Constructors
    Constructor
    Description
    SharedCacheClientImpl()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    protected org.apache.hadoop.yarn.api.ClientSCMProtocol
    createClientProxy()
     
    String
    getFileChecksum(org.apache.hadoop.fs.Path sourceFile)
    A convenience method to calculate the checksum of a specified file.
    void
    release(org.apache.hadoop.yarn.api.records.ApplicationId applicationId, String resourceKey)
    The method to release a resource with the SharedCacheManager. This method is called once an application is no longer using a claimed resource in the shared cache.
    protected void
    serviceInit(org.apache.hadoop.conf.Configuration conf)
     
    protected void
    serviceStart()
     
    protected void
    serviceStop()
     
    protected void
    stopClientProxy()
     
    org.apache.hadoop.yarn.api.records.URL
    use(org.apache.hadoop.yarn.api.records.ApplicationId applicationId, String resourceKey)
    The method to claim a resource with the SharedCacheManager. The client uses a checksum to identify the resource and an ApplicationId to identify which application will be using the resource.

    Methods inherited from class org.apache.hadoop.yarn.client.api.SharedCacheClient

    createSharedCacheClient

    Methods inherited from class org.apache.hadoop.service.AbstractService

    close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, setConfig, start, stop, toString, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

  • Constructor Details

    • SharedCacheClientImpl

      public SharedCacheClientImpl()

  • Method Details

    • serviceInit

      protected void serviceInit(org.apache.hadoop.conf.Configuration conf) throws Exception
      Overrides:
      serviceInit in class org.apache.hadoop.service.AbstractService
      Throws:
      Exception

    • serviceStart

      protected void serviceStart() throws Exception
      Overrides:
      serviceStart in class org.apache.hadoop.service.AbstractService
      Throws:
      Exception

    • serviceStop

      protected void serviceStop() throws Exception
      Overrides:
      serviceStop in class org.apache.hadoop.service.AbstractService
      Throws:
      Exception

    • createClientProxy

      @VisibleForTesting protected org.apache.hadoop.yarn.api.ClientSCMProtocol createClientProxy()

    • stopClientProxy

      @VisibleForTesting protected void stopClientProxy()

    • use

      public org.apache.hadoop.yarn.api.records.URL use(org.apache.hadoop.yarn.api.records.ApplicationId applicationId, String resourceKey) throws org.apache.hadoop.yarn.exceptions.YarnException
      Description copied from class: SharedCacheClient

      The method to claim a resource with the SharedCacheManager. The client uses a checksum to identify the resource and an ApplicationId to identify which application will be using the resource.

      The SharedCacheManager responds with whether or not the resource exists in the cache. If the resource exists, a URL to the resource in the shared cache is returned. If the resource does not exist, null is returned instead.

      Once a URL has been returned for a resource, that URL is safe to use for the lifetime of the application that corresponds to the provided ApplicationId.

      Specified by:
      use in class SharedCacheClient
      Parameters:
      applicationId - ApplicationId of the application using the resource
      resourceKey - the key (i.e. checksum) that identifies the resource
      Returns:
      URL to the resource, or null if it does not exist
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException

    • release

      public void release(org.apache.hadoop.yarn.api.records.ApplicationId applicationId, String resourceKey) throws org.apache.hadoop.yarn.exceptions.YarnException
      Description copied from class: SharedCacheClient

      The method to release a resource with the SharedCacheManager. This method is called once an application is no longer using a claimed resource in the shared cache. The client uses a checksum to identify the resource and an ApplicationId to identify which application is releasing the resource.

      Note: This method is an optimization and the client is not required to call it for correctness.

      Specified by:
      release in class SharedCacheClient
      Parameters:
      applicationId - ApplicationId of the application releasing the resource
      resourceKey - the key (i.e. checksum) that identifies the resource
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException

    • getFileChecksum

      public String getFileChecksum(org.apache.hadoop.fs.Path sourceFile) throws IOException
      Description copied from class: SharedCacheClient
      A convenience method to calculate the checksum of a specified file.
      Specified by:
      getFileChecksum in class SharedCacheClient
      Parameters:
      sourceFile - A path to the input file
      Returns:
      A hex string containing the checksum digest
      Throws:
      IOException