General

• Overview
• Single Node Setup
• Cluster Setup
• Commands Reference
• FileSystem Shell
• Hadoop Compatibility
• Interface Classification
• FileSystem Specification

Common

• CLI Mini Cluster
• Native Libraries
• Proxy User
• Rack Awareness
• Secure Mode
• Service Level Authorization
• HTTP Authentication
• Credential Provider API
• Hadoop KMS
• Tracing

HDFS

• Architecture
• User Guide
• Commands Reference
• NameNode HA With QJM
• NameNode HA With NFS
• Observer NameNode
• Federation
• ViewFs
• Snapshots
• Edits Viewer
• Image Viewer
• Permissions and HDFS
• Quotas and HDFS
• HFTP
• libhdfs (C API)
• WebHDFS (REST API)
• HttpFS
• Short Circuit Local Reads
• Centralized Cache Management
• NFS Gateway
• Rolling Upgrade
• Extended Attributes
• Transparent Encryption
• Multihoming
• Storage Policies
• Memory Storage Support
• Upgrade Domain
• DataNode Admin
• Router Federation

MapReduce

• Tutorial
• Commands Reference
• Compatibility with 1.x
• Encrypted Shuffle
• Pluggable Shuffle/Sort
• Distributed Cache Deploy
• Support for YARN Shared Cache

MapReduce REST APIs

• MR Application Master
• MR History Server

YARN

• Architecture
• Commands Reference
• Capacity Scheduler
• Fair Scheduler
• ResourceManager Restart
• ResourceManager HA
• Node Labels
• Web Application Proxy
• Timeline Server
• Timeline Service V.2
• Writing YARN Applications
• YARN Application Security
• NodeManager
• DockerContainerExecutor
• Running Applications in Docker Containers
• Using CGroups
• Secure Containers
• Registry
• Reservation System
• Graceful Decommission
• Opportunistic Containers
• YARN Federation
• Shared Cache
• Resource Profiles

YARN REST APIs

• Introduction
• Resource Manager
• Node Manager
• Timeline Server
• Timeline Service V.2

Hadoop Compatible File Systems

• Amazon S3
• Azure Blob Storage
• OpenStack Swift

Auth

• Overview
• Examples
• Configuration
• Building

Tools

• Hadoop Streaming
• Hadoop Archives
• Hadoop Archive Logs
• DistCp
• GridMix
• Rumen
• Resource Estimator Service
• Scheduler Load Simulator
• Hadoop Benchmarking

Reference

• Changelog and Release Notes
• API docs
• Metrics

Configuration

• core-default.xml
• hdfs-default.xml
• hdfs-rbf-default.xml
• mapred-default.xml
• yarn-default.xml
• Deprecated Properties

Apache Hadoop 2.10.1

Apache Hadoop 2.10.1 is a minor release in the 2.x.y release line, building upon the previous stable release 2.4.1.

Here is a short overview of the major features and improvements.

• Common

  • Authentication improvements when using an HTTP proxy server. This is useful when accessing WebHDFS via a proxy server.
  • A new Hadoop metrics sink that allows writing directly to Graphite.
  • Specification work related to the Hadoop Compatible Filesystem (HCFS) effort.

• HDFS

  • Support for POSIX-style filesystem extended attributes. See the user documentation for more details.
  • Using the OfflineImageViewer, clients can now browse an fsimage via the WebHDFS API.
  • The NFS gateway received a number of supportability improvements and bug fixes. The Hadoop portmapper is no longer required to run the gateway, and the gateway is now able to reject connections from unprivileged ports.
  • The SecondaryNameNode, JournalNode, and DataNode web UIs have been modernized with HTML5 and Javascript.

• YARN

  • YARN’s REST APIs now support write/modify operations. Users can submit and kill applications through REST APIs.
  • The timeline store in YARN, used for storing generic and application-specific information for applications, supports authentication through Kerberos.
  • The Fair Scheduler supports dynamic hierarchical user queues, user queues are created dynamically at runtime under any specified parent-queue.

Getting Started

The Hadoop documentation includes the information you need to get started using Hadoop. Begin with the Single Node Setup which shows you how to set up a single-node Hadoop installation. Then move on to the Cluster Setup to learn how to set up a multi-node Hadoop installation.