Apache Hadoop 0.18.3 Release Notes

These release notes cover new developer and user-facing incompatibilities, important issues, features, and major improvements.

Included librecordio in release for use by xerces-c (ant -Dlibrecordio=true -Dxercescroot=<path to the xerces-c root>)

Added a new conf property dfs.namenode.decommission.nodes.per.interval so that NameNode checks decommission status of x nodes for every y seconds, where x is the value of dfs.namenode.decommission.nodes.per.interval and y is the value of dfs.namenode.decommission.interval.

WARNING: No release note provided for this change.

TestDistributedUpgrade used succeed for wrong reasons.

fix memory leak of user/group information in fuse-dfs

  1. Only datanode’s offerService thread shutdown the datanode to avoid deadlock;
  2. Datanode checks disk in case of failure on creating a block file.

This patch HADOOP-4620.patch (1) solves the hanging problem on map side with empty input and nonempty output — this map task generates output properly to intermediate files similar to other map tasks. (2) solves the problem of hanging reducer with empty input to reduce task and nonempty output — this reduce task doesn’t generate output if input to reduce task is empty.

Improve how RPC server reads and writes large buffers. Avoids soft-leak of direct buffers and excess copies in NIO layer.

A long (unexpected) delay at datanodes could make subsequent block reports from many datanode at the same time.

Revert tmp files handling on DataNodes back to 0.17. sync() introduced in 0.18 has less gaurantees.

Fix couple of JavaDoc warnings.