Package org.apache.hadoop.hdfs.server.aliasmap

Class InMemoryAliasMap

java.lang.Object
org.apache.hadoop.hdfs.server.aliasmap.InMemoryAliasMap
All Implemented Interfaces:
Configurable, org.apache.hadoop.hdfs.server.aliasmap.InMemoryAliasMapProtocol
@Public @Unstable public class InMemoryAliasMap extends Object implements org.apache.hadoop.hdfs.server.aliasmap.InMemoryAliasMapProtocol, Configurable
InMemoryAliasMap is an implementation of the InMemoryAliasMapProtocol for use with LevelDB.

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static interface 
    org.apache.hadoop.hdfs.server.aliasmap.InMemoryAliasMap.CheckedFunction2<T1,T2,R>
    CheckedFunction is akin to Function but specifies an IOException.

    Nested classes/interfaces inherited from interface org.apache.hadoop.hdfs.server.aliasmap.InMemoryAliasMapProtocol

    org.apache.hadoop.hdfs.server.aliasmap.InMemoryAliasMapProtocol.IterationResult

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    close()
     
    static void
    completeBootstrapTransfer(File aliasMap)
    Extract the aliasmap archive to complete the bootstrap process.
    static org.apache.hadoop.hdfs.protocol.Block
    fromBlockBytes(byte[] blockDbFormat)
     
    static org.apache.hadoop.hdfs.protocol.ProvidedStorageLocation
    fromProvidedStorageLocationBytes(byte[] providedStorageLocationDbFormat)
     
    String
    getBlockPoolId()
    Get the associated block pool id.
    Configuration
    getConf()
    Return the configuration used by this object.
    static InMemoryAliasMap
    init(Configuration conf, String blockPoolID)
     
    org.apache.hadoop.hdfs.server.aliasmap.InMemoryAliasMapProtocol.IterationResult
    list(Optional<org.apache.hadoop.hdfs.protocol.Block> marker)
    List the next batch of FileRegions in the alias map starting from the given marker.
    Optional<org.apache.hadoop.hdfs.protocol.ProvidedStorageLocation>
    read(org.apache.hadoop.hdfs.protocol.Block block)
    Gets the ProvidedStorageLocation associated with the specified block.
    void
    setConf(Configuration conf)
    Set the configuration to be used by this object.
    static byte[]
    toProtoBufBytes(org.apache.hadoop.hdfs.protocol.Block block)
     
    static byte[]
    toProtoBufBytes(org.apache.hadoop.hdfs.protocol.ProvidedStorageLocation providedStorageLocation)
     
    static void
    transferForBootstrap(javax.servlet.http.HttpServletResponse response, Configuration conf, InMemoryAliasMap aliasMap)
    Transfer this aliasmap for bootstrapping standby Namenodes.
    void
    write(org.apache.hadoop.hdfs.protocol.Block block, org.apache.hadoop.hdfs.protocol.ProvidedStorageLocation providedStorageLocation)
    Stores the block and it's associated ProvidedStorageLocation in the alias map.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Method Details

    • setConf

      public void setConf(Configuration conf)
      Description copied from interface: Configurable
      Set the configuration to be used by this object.
      Specified by:
      setConf in interface Configurable
      Parameters:
      conf - configuration to be used

    • getConf

      public Configuration getConf()
      Description copied from interface: Configurable
      Return the configuration used by this object.
      Specified by:
      getConf in interface Configurable
      Returns:
      Configuration

    • init

      @Nonnull public static InMemoryAliasMap init(Configuration conf, String blockPoolID) throws IOException
      Throws:
      IOException

    • list

      public org.apache.hadoop.hdfs.server.aliasmap.InMemoryAliasMapProtocol.IterationResult list(Optional<org.apache.hadoop.hdfs.protocol.Block> marker) throws IOException
      Description copied from interface: org.apache.hadoop.hdfs.server.aliasmap.InMemoryAliasMapProtocol
      List the next batch of FileRegions in the alias map starting from the given marker. To retrieve all FileRegions stored in the alias map, multiple calls to this function might be required.
      Specified by:
      list in interface org.apache.hadoop.hdfs.server.aliasmap.InMemoryAliasMapProtocol
      Parameters:
      marker - the next block to get fileregions from.
      Returns:
      the InMemoryAliasMapProtocol.IterationResult with a set of FileRegions and the next marker.
      Throws:
      IOException

    • read

      @Nonnull public Optional<org.apache.hadoop.hdfs.protocol.ProvidedStorageLocation> read(@Nonnull org.apache.hadoop.hdfs.protocol.Block block) throws IOException
      Description copied from interface: org.apache.hadoop.hdfs.server.aliasmap.InMemoryAliasMapProtocol
      Gets the ProvidedStorageLocation associated with the specified block.
      Specified by:
      read in interface org.apache.hadoop.hdfs.server.aliasmap.InMemoryAliasMapProtocol
      Parameters:
      block - the block to lookup
      Returns:
      the associated ProvidedStorageLocation.
      Throws:
      IOException

    • write

      public void write(@Nonnull org.apache.hadoop.hdfs.protocol.Block block, @Nonnull org.apache.hadoop.hdfs.protocol.ProvidedStorageLocation providedStorageLocation) throws IOException
      Description copied from interface: org.apache.hadoop.hdfs.server.aliasmap.InMemoryAliasMapProtocol
      Stores the block and it's associated ProvidedStorageLocation in the alias map.
      Specified by:
      write in interface org.apache.hadoop.hdfs.server.aliasmap.InMemoryAliasMapProtocol
      Throws:
      IOException

    • getBlockPoolId

      public String getBlockPoolId()
      Description copied from interface: org.apache.hadoop.hdfs.server.aliasmap.InMemoryAliasMapProtocol
      Get the associated block pool id.
      Specified by:
      getBlockPoolId in interface org.apache.hadoop.hdfs.server.aliasmap.InMemoryAliasMapProtocol
      Returns:
      the block pool id associated with the Namenode running the in-memory alias map.

    • close

      public void close() throws IOException
      Throws:
      IOException

    • fromProvidedStorageLocationBytes

      @Nonnull public static org.apache.hadoop.hdfs.protocol.ProvidedStorageLocation fromProvidedStorageLocationBytes(@Nonnull byte[] providedStorageLocationDbFormat) throws org.apache.hadoop.thirdparty.protobuf.InvalidProtocolBufferException
      Throws:
      org.apache.hadoop.thirdparty.protobuf.InvalidProtocolBufferException

    • fromBlockBytes

      @Nonnull public static org.apache.hadoop.hdfs.protocol.Block fromBlockBytes(@Nonnull byte[] blockDbFormat) throws org.apache.hadoop.thirdparty.protobuf.InvalidProtocolBufferException
      Throws:
      org.apache.hadoop.thirdparty.protobuf.InvalidProtocolBufferException

    • toProtoBufBytes

      public static byte[] toProtoBufBytes(@Nonnull org.apache.hadoop.hdfs.protocol.ProvidedStorageLocation providedStorageLocation) throws IOException
      Throws:
      IOException

    • toProtoBufBytes

      public static byte[] toProtoBufBytes(@Nonnull org.apache.hadoop.hdfs.protocol.Block block) throws IOException
      Throws:
      IOException

    • transferForBootstrap

      public static void transferForBootstrap(javax.servlet.http.HttpServletResponse response, Configuration conf, InMemoryAliasMap aliasMap) throws IOException
      Transfer this aliasmap for bootstrapping standby Namenodes. The map is transferred as a tar.gz archive. This archive needs to be extracted on the standby Namenode.
      Parameters:
      response - http response.
      conf - configuration to use.
      aliasMap - aliasmap to transfer.
      Throws:
      IOException

    • completeBootstrapTransfer

      public static void completeBootstrapTransfer(File aliasMap) throws IOException
      Extract the aliasmap archive to complete the bootstrap process. This method has to be called after the aliasmap archive is transfered from the primary Namenode.
      Parameters:
      aliasMap - location of the aliasmap.
      Throws:
      IOException