General

• Overview
• Single Node Setup
• Cluster Setup
• Hadoop Commands Reference
• FileSystem Shell
• Hadoop Compatibility
• Interface Classification
• FileSystem Specification

Common

• CLI Mini Cluster
• Native Libraries
• Proxy User
• Rack Awareness
• Secure Mode
• Service Level Authorization
• HTTP Authentication
• Hadoop KMS
• Tracing

HDFS

• HDFS User Guide
• HDFS Commands Reference
• High Availability With QJM
• High Availability With NFS
• Federation
• ViewFs Guide
• HDFS Snapshots
• HDFS Architecture
• Edits Viewer
• Image Viewer
• Permissions and HDFS
• Quotas and HDFS
• HFTP
• C API libhdfs
• WebHDFS REST API
• HttpFS Gateway
• Short Circuit Local Reads
• Centralized Cache Management
• HDFS NFS Gateway
• HDFS Rolling Upgrade
• Extended Attributes
• Transparent Encryption
• HDFS Support for Multihoming
• Archival Storage, SSD & Memory

MapReduce

• MapReduce Tutorial
• MapReduce Commands Reference
• Compatibilty between Hadoop 1.x and Hadoop 2.x
• Encrypted Shuffle
• Pluggable Shuffle/Sort
• Distributed Cache Deploy

MapReduce REST APIs

• MR Application Master
• MR History Server

YARN

• Overview
• YARN Architecture
• Capacity Scheduler
• Fair Scheduler
• ResourceManager Restart
• ResourceManager HA
• Web Application Proxy
• YARN Timeline Server
• Writing YARN Applications
• YARN Commands
• NodeManager Restart
• DockerContainerExecutor
• Using CGroups
• Secure Containers
• Registry

YARN REST APIs

• Introduction
• Resource Manager
• Node Manager

Hadoop Compatible File Systems

• Amazon S3
• Azure Blob Storage
• OpenStack Swift

Auth

• Overview
• Examples
• Configuration
• Building

Tools

• Hadoop Streaming
• Hadoop Archives
• DistCp
• GridMix
• Rumen
• Scheduler Load Simulator

Reference

• Release Notes
• API docs
• Common CHANGES.txt
• HDFS CHANGES.txt
• MapReduce CHANGES.txt
• YARN CHANGES.txt
• Metrics

Configuration

• core-default.xml
• hdfs-default.xml
• mapred-default.xml
• yarn-default.xml
• Deprecated Properties

Apache Hadoop 2.7.0

Apache Hadoop 2.7.0 is a minor release in the 2.x.y release line, building upon the previous stable release 2.6.0.

This release is not yet ready for production use. Production users should use 2.7.1 release and beyond.

Here is a short overview of the major features and improvements.

• Common

  • Authentication improvements when using an HTTP proxy server. This is useful when accessing WebHDFS via a proxy server.
  • A new Hadoop metrics sink that allows writing directly to Graphite.
  • Specification work related to the Hadoop Compatible Filesystem (HCFS) effort.

• HDFS

  • Support for POSIX-style filesystem extended attributes. See the user documentation for more details.
  • Using the OfflineImageViewer, clients can now browse an fsimage via the WebHDFS API.
  • The NFS gateway received a number of supportability improvements and bug fixes. The Hadoop portmapper is no longer required to run the gateway, and the gateway is now able to reject connections from unprivileged ports.
  • The SecondaryNameNode, JournalNode, and DataNode web UIs have been modernized with HTML5 and Javascript.

• YARN

  • YARN’s REST APIs now support write/modify operations. Users can submit and kill applications through REST APIs.
  • The timeline store in YARN, used for storing generic and application-specific information for applications, supports authentication through Kerberos.
  • The Fair Scheduler supports dynamic hierarchical user queues, user queues are created dynamically at runtime under any specified parent-queue.

Getting Started

The Hadoop documentation includes the information you need to get started using Hadoop. Begin with the Single Node Setup which shows you how to set up a single-node Hadoop installation. Then move on to the Cluster Setup to learn how to set up a multi-node Hadoop installation.