Apache Hadoop Changelog

Release 0.22.1 - Unreleased (as of 2017-08-28)

INCOMPATIBLE CHANGES:

JIRA Summary Priority Component Reporter Contributor
HADOOP-6453 Hadoop wrapper script shouldn’t ignore an existing JAVA_LIBRARY_PATH Minor scripts Chad Metcalf

NEW FEATURES:

JIRA Summary Priority Component Reporter Contributor
HADOOP-7119 add Kerberos HTTP SPNEGO authentication support to Hadoop JT/NN/DN/TT web-consoles Major security Alejandro Abdelnur Alejandro Abdelnur
HADOOP-7937 Forward port SequenceFile#syncFs and friends from Hadoop 1.x Major io Eli Collins Tom White
MAPREDUCE-3837 Job tracker is not able to recover job in case of crash and after that no user can submit job. Major . Mayank Bansal Mayank Bansal

IMPROVEMENTS:

JIRA Summary Priority Component Reporter Contributor
HADOOP-6995 Allow wildcards to be used in ProxyUsers configurations Minor security Todd Lipcon Todd Lipcon
HDFS-1601 Pipeline ACKs are sent as lots of tiny TCP packets Major datanode Todd Lipcon Todd Lipcon
HADOOP-7272 Remove unnecessary security related info logs Major ipc, security Suresh Srinivas Suresh Srinivas
HDFS-2246 Shortcut a local client reads to a Datanodes files directly Major . Sanjay Radia Jitendra Nath Pandey
HADOOP-7338 LocalDirAllocator improvements for MR-2178 Major . Todd Lipcon Benoy Antony
MAPREDUCE-4403 Adding test case for resubmission of jobs in TestRecoveryManager Minor jobtracker Mayank Bansal Mayank Bansal
MAPREDUCE-4405 Adding test case for HierarchicalQueue in TestJobQueueClient Minor client Mayank Bansal Mayank Bansal
MAPREDUCE-4349 Distributed Cache gives inconsistent result if cache Archive files get deleted from task tracker Minor . Mayank Bansal Mayank Bansal
MAPREDUCE-2353 Make the MR changes to reflect the API changes in SecureIO library Major security, task, tasktracker Devaraj Das Benoy Antony
MAPREDUCE-1521 Protection against incorrectly configured reduces Major jobtracker Arun C Murthy Mahadev konar

BUG FIXES:

JIRA Summary Priority Component Reporter Contributor
MAPREDUCE-2420 JobTracker should be able to renew delegation token over HTTP Major . Boris Shkolnik Boris Shkolnik
MAPREDUCE-2452 Delegation token cancellation shouldn’t hold global JobTracker lock Major jobtracker Devaraj Das Devaraj Das
HADOOP-7621 alfredo config should be in a file not readable by users Critical security Alejandro Abdelnur Aaron T. Myers
HDFS-2698 BackupNode is downloading image from NameNode for every checkpoint Major namenode Konstantin Shvachko Konstantin Shvachko
HDFS-1910 when dfs.name.dir and dfs.name.edits.dir are same fsimage will be saved twice every time Minor namenode Gokul
MAPREDUCE-3593 MAPREDUCE Impersonation is not working in 22 Major job submission Mayank Bansal Mayank Bansal
MAPREDUCE-3725 Hadoop 22 hadoop job -list returns user name as NULL Major client Mayank Bansal Mayank Bansal
HDFS-2718 Optimize OP_ADD in edits loading Major namenode Konstantin Shvachko Konstantin Shvachko
HDFS-2877 If locking of a storage dir fails, it will remove the other NN’s lock file on exit Major namenode Todd Lipcon Todd Lipcon
HADOOP-7680 TestHardLink fails on Mac OS X, when gnu stat is in path Major . Milind Bhandarkar Milind Bhandarkar
HDFS-2991 failure to load edits: ClassCastException Blocker namenode Todd Lipcon Todd Lipcon
MAPREDUCE-4164 Hadoop 22 Exception thrown after task completion causes its reexecution Major tasktracker Mayank Bansal Mayank Bansal
MAPREDUCE-3863 0.22 branch mvn deploy is not publishing hadoop-streaming JAR Critical build Alejandro Abdelnur Benoy Antony
HDFS-3368 Missing blocks due to bad DataNodes coming up and down. Major namenode Konstantin Shvachko Konstantin Shvachko
MAPREDUCE-2178 Race condition in LinuxTaskController permissions handling Major security, task-controller Todd Lipcon Benoy Antony
MAPREDUCE-4314 Synchronization in JvmManager for 0.22 branch Major tasktracker Konstantin Shvachko Benoy Antony
MAPREDUCE-2377 task-controller fails to parse configuration if it doesn’t end in \n Major task-controller Todd Lipcon Benoy Antony
MAPREDUCE-4318 TestRecoveryManager should not use raw and deprecated configuration parameters. Major test Konstantin Shvachko Benoy Antony
MAPREDUCE-4360 Capacity Scheduler Hierarchical leaf queue does not honor the max capacity of container queue Major . Mayank Bansal Mayank Bansal
HDFS-1584 Need to check TGT and renew if needed when fetching delegation tokens using HFTP Major security Kan Zhang Benoy Antony
HDFS-3402 Fix hdfs scripts for secure datanodes Minor scripts, security Benoy Antony Benoy Antony
HADOOP-7115 Add a cache for getpwuid_r and getpwgid_r calls Major . Arun C Murthy Alejandro Abdelnur
MAPREDUCE-4404 Adding Test case for TestMRJobClient to verify the user name Minor client Mayank Bansal Mayank Bansal
MAPREDUCE-5706 toBeDeleted parent directories aren’t being cleaned up Major security Robert Kanter Robert Kanter

SUB-TASKS:

JIRA Summary Priority Component Reporter Contributor
HDFS-2886 CreateEditLogs should generate a realistic edit log. Major namenode Konstantin Shvachko Konstantin Shvachko
HADOOP-8381 Substitute _HOST with hostname for HTTP principals Minor security Benoy Antony Benoy Antony
HADOOP-8383 TestKerberosAuthenticator fails Minor security Benoy Antony Benoy Antony

OTHER:

JIRA Summary Priority Component Reporter Contributor
MAPREDUCE-4240 Revert MAPREDUCE-2767 Minor security Benoy Antony Benoy Antony
MAPREDUCE-4243 Modify mapreduce build to include task-controller Minor build Benoy Antony Benoy Antony
MAPREDUCE-4244 Fix an issue related to do with setting of correct groups for tasks Minor security Benoy Antony Benoy Antony
MAPREDUCE-4246 Failure in deleting user directories in Secure hadoop Major security Benoy Antony Benoy Antony
MAPREDUCE-4247 TestTaskTrackerLocalization fails Minor security Benoy Antony Benoy Antony
MAPREDUCE-4248 TestRecoveryManager fails Minor security Benoy Antony Benoy Antony
MAPREDUCE-4249 Fix failures in streaming test TestFileArgs Minor security Benoy Antony Benoy Antony
HADOOP-8357 Restore security in Hadoop 0.22 branch Major security Konstantin Shvachko Benoy Antony