General

• Overview
• Single Node Setup
• Cluster Setup
• Commands Reference
• FileSystem Shell
• Compatibility Specification
• Downstream Developer's Guide
• Admin Compatibility Guide
• Interface Classification
• FileSystem Specification

Common

• CLI Mini Cluster
• Native Libraries
• Proxy User
• Rack Awareness
• Secure Mode
• Service Level Authorization
• HTTP Authentication
• Credential Provider API
• Hadoop KMS
• Tracing
• Unix Shell Guide

HDFS

• Architecture
• User Guide
• Commands Reference
• NameNode HA With QJM
• NameNode HA With NFS
• Federation
• ViewFs
• Snapshots
• Edits Viewer
• Image Viewer
• Permissions and HDFS
• Quotas and HDFS
• libhdfs (C API)
• WebHDFS (REST API)
• HttpFS
• Short Circuit Local Reads
• Centralized Cache Management
• NFS Gateway
• Rolling Upgrade
• Extended Attributes
• Transparent Encryption
• Multihoming
• Storage Policies
• Memory Storage Support
• Synthetic Load Generator
• Erasure Coding
• Disk Balancer
• Upgrade Domain
• DataNode Admin
• Router Federation
• Provided Storage

MapReduce

• Tutorial
• Commands Reference
• Compatibility with 1.x
• Encrypted Shuffle
• Pluggable Shuffle/Sort
• Distributed Cache Deploy
• Support for YARN Shared Cache

MapReduce REST APIs

• MR Application Master
• MR History Server

YARN

• Architecture
• Commands Reference
• Capacity Scheduler
• Fair Scheduler
• ResourceManager Restart
• ResourceManager HA
• Resource Model
• Node Labels
• Web Application Proxy
• Timeline Server
• Timeline Service V.2
• Writing YARN Applications
• YARN Application Security
• NodeManager
• Running Applications in Docker Containers
• Using CGroups
• Secure Containers
• Registry
• Reservation System
• Graceful Decommission
• Opportunistic Containers
• YARN Federation
• Shared Cache
• Using GPU
• Using FPGA
• Placement Constraints

YARN REST APIs

• Introduction
• Resource Manager
• Node Manager
• Timeline Server
• Timeline Service V.2

YARN Service

• Overview
• QuickStart
• Concepts
• Yarn Service API
• Service Discovery
• System Services

Hadoop Compatible File Systems

• Aliyun OSS
• Amazon S3
• Azure Blob Storage
• Azure Data Lake Storage
• OpenStack Swift

Auth

• Overview
• Examples
• Configuration
• Building

Tools

• Hadoop Streaming
• Hadoop Archives
• Hadoop Archive Logs
• DistCp
• GridMix
• Rumen
• Resource Estimator Service
• Scheduler Load Simulator
• Hadoop Benchmarking

Reference

• Changelog and Release Notes
• Java API docs
• Unix Shell API
• Metrics

Configuration

• core-default.xml
• hdfs-default.xml
• hdfs-rbf-default.xml
• mapred-default.xml
• yarn-default.xml
• Deprecated Properties

Apache Hadoop 3.1.3

Apache Hadoop 3.1.3 incorporates a number of significant enhancements over the previous major release line (hadoop-2.x).

This release is generally available (GA), meaning that it represents a point of API stability and quality that we consider production-ready.

Overview

This release is a maintainance release. Users are encouraged to read the full set of changelog. This page provides some of the major changes.

Failover for LdapGroupsMapping - HADOOP-15950

DelegationTokenAuthenticationFilter get wrong doAsUser since it does not decode URL - HADOOP-15922

Service AM support for decommissioning component instances - YARN-8761

Avoid allocating opportunistic containers to previous/removed/non-exist application attempt - YARN-9238

Standby does not trigger edit log rolling when in-progress edit log tailing is enabled - HDFS-14317

Getting Started

The Hadoop documentation includes the information you need to get started using Hadoop. Begin with the Single Node Setup which shows you how to set up a single-node Hadoop installation. Then move on to the Cluster Setup to learn how to set up a multi-node Hadoop installation.