Package org.apache.hadoop.yarn.client.api

Class SharedCacheClient

java.lang.Object
org.apache.hadoop.service.AbstractService
org.apache.hadoop.yarn.client.api.SharedCacheClient
All Implemented Interfaces:
Closeable, AutoCloseable, org.apache.hadoop.service.Service
Direct Known Subclasses:
SharedCacheClientImpl
@Public @Unstable public abstract class SharedCacheClient extends org.apache.hadoop.service.AbstractService
This is the client for YARN's shared cache.

  • Nested Class Summary

    Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service

    org.apache.hadoop.service.Service.STATE

  • Constructor Summary

    Constructors
    Constructor
    Description
    SharedCacheClient(String name)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    static SharedCacheClient
    createSharedCacheClient()
     
    abstract String
    getFileChecksum(org.apache.hadoop.fs.Path sourceFile)
    A convenience method to calculate the checksum of a specified file.
    abstract void
    release(org.apache.hadoop.yarn.api.records.ApplicationId applicationId, String resourceKey)
    The method to release a resource with the SharedCacheManager. This method is called once an application is no longer using a claimed resource in the shared cache.
    abstract org.apache.hadoop.yarn.api.records.URL
    use(org.apache.hadoop.yarn.api.records.ApplicationId applicationId, String resourceKey)
    The method to claim a resource with the SharedCacheManager. The client uses a checksum to identify the resource and an ApplicationId to identify which application will be using the resource.

    Methods inherited from class org.apache.hadoop.service.AbstractService

    close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, serviceInit, serviceStart, serviceStop, setConfig, start, stop, toString, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

  • Constructor Details

    • SharedCacheClient

      @Private public SharedCacheClient(String name)

  • Method Details

    • createSharedCacheClient

      @Public public static SharedCacheClient createSharedCacheClient()

    • use

      @Public @Unstable public abstract org.apache.hadoop.yarn.api.records.URL use(org.apache.hadoop.yarn.api.records.ApplicationId applicationId, String resourceKey) throws org.apache.hadoop.yarn.exceptions.YarnException

      The method to claim a resource with the SharedCacheManager. The client uses a checksum to identify the resource and an ApplicationId to identify which application will be using the resource.

      The SharedCacheManager responds with whether or not the resource exists in the cache. If the resource exists, a URL to the resource in the shared cache is returned. If the resource does not exist, null is returned instead.

      Once a URL has been returned for a resource, that URL is safe to use for the lifetime of the application that corresponds to the provided ApplicationId.

      Parameters:
      applicationId - ApplicationId of the application using the resource
      resourceKey - the key (i.e. checksum) that identifies the resource
      Returns:
      URL to the resource, or null if it does not exist
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException

    • release

      @Public @Unstable public abstract void release(org.apache.hadoop.yarn.api.records.ApplicationId applicationId, String resourceKey) throws org.apache.hadoop.yarn.exceptions.YarnException

      The method to release a resource with the SharedCacheManager. This method is called once an application is no longer using a claimed resource in the shared cache. The client uses a checksum to identify the resource and an ApplicationId to identify which application is releasing the resource.

      Note: This method is an optimization and the client is not required to call it for correctness.

      Parameters:
      applicationId - ApplicationId of the application releasing the resource
      resourceKey - the key (i.e. checksum) that identifies the resource
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException

    • getFileChecksum

      @Public @Unstable public abstract String getFileChecksum(org.apache.hadoop.fs.Path sourceFile) throws IOException
      A convenience method to calculate the checksum of a specified file.
      Parameters:
      sourceFile - A path to the input file
      Returns:
      A hex string containing the checksum digest
      Throws:
      IOException