These release notes cover new developer and user-facing incompatibilities, important issues, features, and major improvements.
Change update_requests field id to 7. This matches the current field id in branch-2 and branch-2.8.
The YARN Shared Cache provides the facility to upload and manage shared application resources to HDFS in a safe and scalable manner. YARN applications can leverage resources uploaded by other applications or previous runs of the same application without having to re-upload and localize identical files multiple times. This will save network resources and reduce YARN application startup time.
HDFS Router-based Federation adds a RPC routing layer that provides a federated view of multiple HDFS namespaces. This is similar to the existing ViewFS and HDFS federation functionality, except the mount table is managed on the server-side by the routing layer rather than on the client. This simplifies access to a federated cluster for existing HDFS clients.
See HDFS-10467 and the HDFS Router-based Federation documentation for more details.
The OrgQueue extension to the capacity scheduler provides a programmatic way to change configurations by providing a REST API that users can call to modify queue configurations. This enables automation of queue configuration management by administrators in the queue’s administer_queue ACL.
MapReduce support for the YARN shared cache allows MapReduce jobs to take advantage of additional resource caching. This saves network bandwidth between the job submission client as well as within the YARN cluster itself. This will reduce job submission time and overall job runtime.
A change in configuration for launching Docker containers under YARN. Docker container capabilities, mounts, networks and allowing privileged container have to specified in the container-executor.cfg. By default, all of the above are turned off. This change will break existing setups launching Docker containers under YARN. Please refer to the Docker containers under YARN documentation for more information.
This patch changes the default build and test environment in the following ways:
* Switch from Ubuntu “Trusty” 14.04 to Ubuntu “Xenial” 16.04 * Switch from Oracle JDK 8 to OpenJDK 8 * Adds OpenJDK 9 to the build environment
ReconfigurationTaskStatus’ API scope is reduced to LimitedPrivate, and its dependency on com.google.com.base.Optional is replaced by java.util.Optional.
The first version of Resource Estimator service, a tool that captures the historical resource usage of an app and predicts its future resource requirement.
In hadoop-mapreduce-client-core module, the type of some public LOG variables were changed from org.apache.commons.logging.Log to org.slf4j.Logger. In the public methods that accepts logger, the logger was changed from org.apache.commons.logging.Log to org.slf4j.Logger.
WARNING: No release note provided for this change.
This allows the Application Master to ask the Scheduler to change the ExecutionType of a running/allocated container.
Add ErasureCodingPolicyId to each OP_ADD edit log op.
This change reverses the default delegation token format implemented by HADOOP-12563, but preserves the capability to read the new delegation token format. When the new format becomes default, then MR deployment jobs runs will be compatible with releases that contain this change.