Apache Hadoop Changelog

Release 2.7.5 - 2017-12-14

IMPORTANT ISSUES:

JIRA Summary Priority Component Reporter Contributor
YARN-6959 RM may allocate wrong AM Container for new attempt Major capacity scheduler, fairscheduler, scheduler Yuqi Wang Yuqi Wang

IMPROVEMENTS:

JIRA Summary Priority Component Reporter Contributor
HDFS-8829 Make SO_RCVBUF and SO_SNDBUF size configurable for DataTransferProtocol sockets and allow configuring auto-tuning Major datanode He Tianyi He Tianyi
HADOOP-13442 Optimize UGI group lookups Major . Daryn Sharp Daryn Sharp
MAPREDUCE-6937 Backport MAPREDUCE-6870 to branch-2 while preserving compatibility Major . Zhe Zhang Peter Bacsko
HADOOP-14827 Allow StopWatch to accept a Timer parameter for tests Minor common, test Erik Krogen Erik Krogen
HDFS-12131 Add some of the FSNamesystem JMX values as metrics Minor hdfs, namenode Erik Krogen Erik Krogen
HDFS-8865 Improve quota initialization performance Major . Kihwal Lee Kihwal Lee
HDFS-12420 Add an option to disallow ‘namenode format -force’ Major . Ajay Kumar Ajay Kumar
MAPREDUCE-6975 Logging task counters Major task Prabhu Joseph Prabhu Joseph
HDFS-12823 Backport HDFS-9259 “Make SO_SNDBUF size configurable at DFSClient” to branch-2.7 Major hdfs, hdfs-client Erik Krogen Erik Krogen

BUG FIXES:

JIRA Summary Priority Component Reporter Contributor
MAPREDUCE-6165 [JDK8] TestCombineFileInputFormat failed on JDK8 Minor . Wei Yan Akira Ajisaka
HDFS-8797 WebHdfsFileSystem creates too many connections for pread Major webhdfs Jing Zhao Jing Zhao
HDFS-9003 ForkJoin thread pool leaks Major . Kihwal Lee Kihwal Lee
HDFS-9107 Prevent NN’s unrecoverable death spiral after full GC Critical namenode Daryn Sharp Daryn Sharp
MAPREDUCE-6750 TestHSAdminServer.testRefreshSuperUserGroups is failing Minor test Kihwal Lee Kihwal Lee
HDFS-10738 Fix TestRefreshUserMappings.testRefreshSuperUserGroupsConfiguration test failure Major test Rakesh R Rakesh R
HADOOP-14702 Fix formatting issue and regression caused by conversion from APT to Markdown Minor documentation Doris Gu Doris Gu
HDFS-12157 Do fsyncDirectory(..) outside of FSDataset lock Critical datanode Vinayakumar B Vinayakumar B
MAPREDUCE-6931 Remove TestDFSIO “Total Throughput” calculation Critical benchmarks, test Dennis Huo Dennis Huo
HADOOP-14867 Update HDFS Federation setup document, for incorrect property name for secondary name node http address Major . Bharat Viswanadham Bharat Viswanadham
MAPREDUCE-6957 shuffle hangs after a node manager connection timeout Major mrv2 Jooseong Kim Jooseong Kim
HDFS-12323 NameNode terminates after full GC thinking QJM unresponsive if full GC is much longer than timeout Major namenode, qjm Erik Krogen Erik Krogen
YARN-5195 RM intermittently crashed with NPE while handling APP_ATTEMPT_REMOVED event when async-scheduling enabled in CapacityScheduler Major resourcemanager Karam Singh sandflee
HADOOP-14902 LoadGenerator#genFile write close timing is incorrectly calculated Major fs Jason Lowe Hanisha Koneru
YARN-7084 TestSchedulingMonitor#testRMStarts fails sporadically Major . Jason Lowe Jason Lowe
HDFS-12578 TestDeadDatanode#testNonDFSUsedONDeadNodeReReg failing in branch-2.7 Blocker test Xiao Chen Ajay Kumar
HADOOP-14919 BZip2 drops records when reading data in splits Critical . Aki Tanaka Jason Lowe
HDFS-12832 INode.getFullPathName may throw ArrayIndexOutOfBoundsException lead to NameNode exit Critical namenode DENG FEI Konstantin Shvachko
HDFS-12638 Delete copy-on-truncate block along with the original block, when deleting a file being truncated Blocker hdfs Jiandan Yang Konstantin Shvachko

TESTS:

JIRA Summary Priority Component Reporter Contributor
HDFS-9153 Pretty-format the output for DFSIO Major . Kai Zheng Kai Zheng
HDFS-12596 Add TestFsck#testFsckCorruptWhenOneReplicaIsCorrupt back to branch-2.7 Major test Xiao Chen Xiao Chen

SUB-TASKS:

JIRA Summary Priority Component Reporter Contributor
HDFS-10711 Optimize FSPermissionChecker group membership check Major hdfs Daryn Sharp Daryn Sharp
HADOOP-14881 LoadGenerator should use Time.monotonicNow() to measure durations Major . Chetna Chaudhari Bharat Viswanadham
YARN-5402 Fix NoSuchMethodError in ClusterMetricsInfo Major webapp Weiwei Yang Weiwei Yang

OTHER:

JIRA Summary Priority Component Reporter Contributor
HDFS-10984 Expose nntop output as metrics Major namenode Siddharth Wagle Siddharth Wagle