Package | Description |
---|---|
org.apache.hadoop.conf |
Configuration of system parameters.
|
org.apache.hadoop.fs |
Implementations of
AbstractFileSystem for hdfs
over rpc and hdfs over web. |
org.apache.hadoop.fs.adl |
Supporting classes for metrics instrumentation.
|
org.apache.hadoop.fs.azure |
A distributed implementation of
FileSystem for reading and writing files on
Azure Block Storage. |
org.apache.hadoop.fs.ftp | |
org.apache.hadoop.fs.s3native |
A distributed implementation of
FileSystem for reading and writing files on
Amazon S3. |
org.apache.hadoop.fs.viewfs | |
org.apache.hadoop.hdfs.server.aliasmap | |
org.apache.hadoop.hdfs.server.common.blockaliasmap.impl | |
org.apache.hadoop.hdfs.server.namenode | |
org.apache.hadoop.hdfs.web.oauth2 |
OAuth2-based WebHDFS authentication.
|
org.apache.hadoop.io |
Generic i/o code for use when reading and writing data to the network,
to databases, and to files.
|
org.apache.hadoop.io.compress | |
org.apache.hadoop.io.serializer |
This package provides a mechanism for using different serialization frameworks
in Hadoop.
|
org.apache.hadoop.io.serializer.avro |
This package provides Avro serialization in Hadoop.
|
org.apache.hadoop.mapred | |
org.apache.hadoop.mapred.join | |
org.apache.hadoop.mapred.pipes | |
org.apache.hadoop.mapreduce.lib.db | |
org.apache.hadoop.mapreduce.lib.join | |
org.apache.hadoop.mapreduce.lib.partition | |
org.apache.hadoop.mapreduce.tools | |
org.apache.hadoop.net |
Network-related classes.
|
org.apache.hadoop.record |
(DEPRECATED) Hadoop record I/O contains classes and a record description language
translator for simplifying serialization and deserialization of records in a
language-neutral manner.
|
org.apache.hadoop.security.authorize | |
org.apache.hadoop.tools |
Command-line tools for MapReduce.
|
org.apache.hadoop.tools.dynamometer |
The Dynamometer tool is used to stress test the HDFS NameNode by setting
up a realistic environment consistent of fake DataNodes and a real NameNode
with a real FS namesystem and near-real user workload (RPCs).
|
org.apache.hadoop.util |
Common utilities.
|
org.apache.hadoop.yarn.client.cli | |
org.apache.hadoop.yarn.util |
Modifier and Type | Class and Description |
---|---|
class |
Configured
Base class for things that may be configured with a
Configuration . |
Modifier and Type | Class and Description |
---|---|
class |
ChecksumFileSystem
Abstract Checksumed FileSystem.
|
class |
FileSystem
An abstract base class for a fairly generic filesystem.
|
class |
FilterFileSystem
A
FilterFileSystem contains
some other file system, which it uses as
its basic file system, possibly transforming
the data along the way or providing additional
functionality. |
class |
LocalFileSystem
Implement the FileSystem API for the checksumed local filesystem.
|
class |
RawLocalFileSystem
Implement the FileSystem API for the raw local filesystem.
|
class |
Trash
Provides a trash facility which supports pluggable Trash policies.
|
class |
TrashPolicy
This interface is used for implementing different Trash policies.
|
Modifier and Type | Class and Description |
---|---|
class |
AdlFileSystem
A FileSystem to access Azure Data Lake Store.
|
Modifier and Type | Class and Description |
---|---|
class |
NativeAzureFileSystem
A
FileSystem for reading and writing files stored on Windows Azure. |
class |
WasbFsck
An fsck tool implementation for WASB that does various admin/cleanup/recovery
tasks on the WASB file system.
|
Modifier and Type | Class and Description |
---|---|
class |
FTPFileSystem
A
FileSystem backed by an FTP client provided by Apache Commons Net. |
Modifier and Type | Class and Description |
---|---|
class |
NativeS3FileSystem
Deprecated.
Replaced by the S3A client.
|
Modifier and Type | Class and Description |
---|---|
class |
ViewFileSystem
ViewFileSystem (extends the FileSystem interface) implements a client-side
mount table.
|
Modifier and Type | Class and Description |
---|---|
class |
InMemoryAliasMap
InMemoryAliasMap is an implementation of the InMemoryAliasMapProtocol for
use with LevelDB.
|
Modifier and Type | Class and Description |
---|---|
class |
LevelDBFileRegionAliasMap
A LevelDB based implementation of
BlockAliasMap . |
class |
TextFileRegionAliasMap
This class is used for block maps stored as text files,
with a specified delimiter.
|
Modifier and Type | Class and Description |
---|---|
class |
FileSystemImage
Create FSImage from an external namespace.
|
class |
FixedBlockMultiReplicaResolver
Resolver mapping all files to a configurable, uniform blocksize
and replication.
|
class |
FixedBlockResolver
Resolver mapping all files to a configurable, uniform blocksize.
|
class |
SingleUGIResolver
Map all owners/groups in external system to a single user in FSImage.
|
Modifier and Type | Class and Description |
---|---|
class |
AccessTokenProvider
Provide an OAuth2 access token to be used to authenticate http calls in
WebHDFS.
|
class |
ConfCredentialBasedAccessTokenProvider
Obtain an access token via a a credential (provided through the
Configuration) using the
Client Credentials Grant workflow.
|
class |
ConfRefreshTokenBasedAccessTokenProvider
Supply a access token obtained via a refresh token (provided through the
Configuration using the second half of the
Authorization Code Grant workflow.
|
class |
CredentialBasedAccessTokenProvider
Obtain an access token via the credential-based OAuth2 workflow.
|
Modifier and Type | Class and Description |
---|---|
class |
AbstractMapWritable
Abstract base class for MapWritable and SortedMapWritable
Unlike org.apache.nutch.crawl.MapWritable, this class allows creation of
MapWritable<Writable, MapWritable> so the CLASS_TO_ID and ID_TO_CLASS
maps travel with the class instead of being static.
|
class |
EnumSetWritable<E extends Enum<E>>
A Writable wrapper for EnumSet.
|
class |
GenericWritable
A wrapper for Writable instances.
|
class |
MapWritable
A Writable Map.
|
class |
ObjectWritable
A polymorphic Writable that writes an instance with it's class name.
|
class |
SortedMapWritable<K extends WritableComparable<? super K>>
A Writable SortedMap.
|
class |
WritableComparator
A Comparator for
WritableComparable s. |
Modifier and Type | Class and Description |
---|---|
class |
BZip2Codec
This class provides output and input streams for bzip2 compression
and decompression.
|
class |
DefaultCodec |
class |
GzipCodec
This class creates gzip compressors/decompressors.
|
class |
PassthroughCodec
This is a special codec which does not transform the output.
|
Modifier and Type | Class and Description |
---|---|
class |
WritableSerialization
A
Serialization for Writable s that delegates to
Writable.write(java.io.DataOutput) and
Writable.readFields(java.io.DataInput) . |
Modifier and Type | Class and Description |
---|---|
class |
AvroReflectSerialization
Serialization for Avro Reflect classes.
|
class |
AvroSerialization<T>
Base class for providing serialization to Avro types.
|
class |
AvroSpecificSerialization
Serialization for Avro Specific classes.
|
Modifier and Type | Class and Description |
---|---|
class |
JobClient
JobClient is the primary interface for the user-job to interact
with the cluster. |
Modifier and Type | Class and Description |
---|---|
class |
WrappedRecordReader<K extends WritableComparable,U extends Writable>
Proxy class for a RecordReader participating in the join framework.
|
Modifier and Type | Class and Description |
---|---|
class |
Submitter
The main entry point and job submitter.
|
Modifier and Type | Class and Description |
---|---|
class |
DataDrivenDBInputFormat<T extends DBWritable>
A InputFormat that reads input data from an SQL table.
|
class |
DBInputFormat<T extends DBWritable>
A InputFormat that reads input data from an SQL table.
|
class |
OracleDataDrivenDBInputFormat<T extends DBWritable>
A InputFormat that reads input data from an SQL table in an Oracle db.
|
Modifier and Type | Class and Description |
---|---|
class |
CompositeRecordReader<K extends WritableComparable<?>,V extends Writable,X extends Writable>
A RecordReader that can effect joins of RecordReaders sharing a common key
type and partitioning.
|
class |
InnerJoinRecordReader<K extends WritableComparable<?>>
Full inner join.
|
class |
JoinRecordReader<K extends WritableComparable<?>>
Base class for Composite joins returning Tuples of arbitrary Writables.
|
class |
MultiFilterRecordReader<K extends WritableComparable<?>,V extends Writable>
Base class for Composite join returning values derived from multiple
sources, but generally not tuples.
|
class |
OuterJoinRecordReader<K extends WritableComparable<?>>
Full outer join.
|
class |
OverrideRecordReader<K extends WritableComparable<?>,V extends Writable>
Prefer the "rightmost" data source for this key.
|
Modifier and Type | Class and Description |
---|---|
class |
BinaryPartitioner<V>
Partition
BinaryComparable keys using a configurable part of
the bytes array returned by BinaryComparable.getBytes() . |
class |
InputSampler<K,V>
Utility for collecting samples and writing a partition file for
TotalOrderPartitioner . |
class |
KeyFieldBasedComparator<K,V>
This comparator implementation provides a subset of the features provided
by the Unix/GNU Sort.
|
class |
KeyFieldBasedPartitioner<K2,V2>
Defines a way to partition keys based on certain key fields (also see
KeyFieldBasedComparator . |
class |
TotalOrderPartitioner<K,V>
Partitioner effecting a total order by reading split points from
an externally generated source.
|
Modifier and Type | Class and Description |
---|---|
class |
CLI
Interprets the map reduce cli options
|
Modifier and Type | Class and Description |
---|---|
class |
AbstractDNSToSwitchMapping
This is a base class for DNS to Switch mappings.
|
class |
CachedDNSToSwitchMapping
A cached implementation of DNSToSwitchMapping that takes an
raw DNSToSwitchMapping and stores the resolved network location in
a cache.
|
class |
ScriptBasedMapping
This class implements the
DNSToSwitchMapping interface using a
script configured via the
CommonConfigurationKeysPublic.NET_TOPOLOGY_SCRIPT_FILE_NAME_KEY option. |
class |
SocksSocketFactory
Specialized SocketFactory to create sockets with a SOCKS proxy
|
class |
TableMapping
Simple
DNSToSwitchMapping implementation that reads a 2 column text
file. |
Modifier and Type | Class and Description |
---|---|
class |
RecordComparator
Deprecated.
Replaced by Avro.
|
Modifier and Type | Interface and Description |
---|---|
interface |
ImpersonationProvider |
Modifier and Type | Class and Description |
---|---|
class |
DefaultImpersonationProvider |
Modifier and Type | Class and Description |
---|---|
class |
DistCp
DistCp is the main driver-class for DistCpV2.
|
Modifier and Type | Class and Description |
---|---|
class |
Client
Client for submitting a Dynamometer YARN application, and optionally, a
workload MapReduce job.
|
Modifier and Type | Interface and Description |
---|---|
interface |
Tool
A tool interface that supports handling of generic command-line options.
|
Modifier and Type | Class and Description |
---|---|
class |
LogsCLI |
class |
SchedConfCLI
CLI for modifying scheduler configuration.
|
Modifier and Type | Class and Description |
---|---|
class |
ResourceCalculatorProcessTree
Interface class to obtain process resource usage
NOTE: This class should not be used by external users, but only by external
developers to extend and include their own process-tree implementation,
especially for platforms other than Linux and Windows.
|
Copyright © 2020 Apache Software Foundation. All rights reserved.