Apache Hadoop Changelog

Release 2.9.0 - 2017-11-17

INCOMPATIBLE CHANGES:

JIRA Summary Priority Component Reporter Contributor
YARN-4784 Fairscheduler: defaultQueueSchedulingPolicy should not accept FIFO Major fairscheduler Yufei Gu Yufei Gu
HADOOP-13588 ConfServlet should respect Accept request header Major conf Weiwei Yang Weiwei Yang
MAPREDUCE-6776 yarn.app.mapreduce.client.job.max-retries should have a more useful default Major client Daniel Templeton Miklos Szegedi
YARN-5388 Deprecate and remove DockerContainerExecutor Critical nodemanager Daniel Templeton Daniel Templeton
HADOOP-12705 Upgrade Jackson 2.2.3 to 2.7.8 Major build Steve Loughran Sean Mackrory
HADOOP-13050 Upgrade to AWS SDK 1.11.45 Blocker build, fs/s3 Steve Loughran Steve Loughran
HADOOP-13119 Add ability to secure log servlet using proxy users Major . Jeffrey E Rodriguez Yuanbo Liu
HADOOP-14174 Set default ADLS access token provider type to ClientCredential Major fs/adl John Zhuge John Zhuge
HDFS-11402 HDFS Snapshots should capture point-in-time copies of OPEN files Major hdfs Manoj Govindassamy Manoj Govindassamy
YARN-2962 ZKRMStateStore: Limit the number of znodes under a znode Critical resourcemanager Karthik Kambatla Varun Saxena
HADOOP-14419 Remove findbugs report from docs profile Minor . Andras Bokor Andras Bokor
YARN-6127 Add support for work preserving NM restart when AMRMProxy is enabled Major amrmproxy, nodemanager Subru Krishnan Botong Huang
HADOOP-14539 Move commons logging APIs over to slf4j in hadoop-common Major . Akira Ajisaka Wenxin He
HADOOP-14260 Configuration.dumpConfiguration should redact sensitive information Major conf, security Vihang Karajgaonkar John Zhuge
YARN-6623 Add support to turn off launching privileged containers in the container-executor Blocker nodemanager Varun Vasudev Varun Vasudev
YARN-5049 Extend NMStateStore to save queued container information Major nodemanager, resourcemanager Konstantinos Karanasos Arun Suresh

NEW FEATURES:

JIRA Summary Priority Component Reporter Contributor
HADOOP-12321 Make JvmPauseMonitor an AbstractService Major . Steve Loughran Sunil Govindan
HDFS-9525 hadoop utilities need to support provided delegation tokens Blocker security Allen Wittenauer Heesoo Kim
HADOOP-12702 Add an HDFS metrics sink Major metrics Daniel Templeton Daniel Templeton
HADOOP-12847 hadoop daemonlog should support https and SPNEGO for Kerberized cluster Major security Wei-Chiu Chuang Wei-Chiu Chuang
YARN-2928 YARN Timeline Service v.2: alpha 1 Critical timelineserver Sangjin Lee Sangjin Lee
MAPREDUCE-6331 [Umbrella] Make MapReduce work with Timeline Service Nextgen (YARN-2928) Major . Vinod Kumar Vavilapalli Sangjin Lee
HADOOP-12747 support wildcard in libjars argument Major util Sangjin Lee Sangjin Lee
MAPREDUCE-6690 Limit the number of resources a single map reduce job can submit for localization Major . Chris Trezzo Chris Trezzo
HADOOP-13396 Allow pluggable audit loggers in KMS Major kms Xiao Chen Xiao Chen
HDFS-9820 Improve distcp to support efficient restore to an earlier snapshot Major distcp Yongjun Zhang Yongjun Zhang
YARN-4597 Introduce ContainerScheduler and a SCHEDULED state to NodeManager container lifecycle Major nodemanager Chris Douglas Arun Suresh
HADOOP-13578 Add Codec for ZStandard Compression Major . churro morales churro morales
HADOOP-13933 Add haadmin -getAllServiceState option to get the HA state of all the NameNodes/ResourceManagers Major tools Surendra Singh Lilhore Surendra Singh Lilhore
YARN-5910 Support for multi-cluster delegation tokens Minor security Clay B. Jian He
YARN-5864 YARN Capacity Scheduler - Queue Priorities Major . Wangda Tan Wangda Tan
HDFS-11194 Maintain aggregated peer performance metrics on NameNode Major namenode Xiaobing Zhou Arpit Agarwal
MAPREDUCE-6871 Allow users to specify racks and nodes for strict locality for AMs Major client Robert Kanter Robert Kanter
HDFS-11417 Add datanode admin command to get the storage info. Major . Surendra Singh Lilhore Surendra Singh Lilhore
YARN-679 add an entry point that can start any Yarn service Major api Steve Loughran Steve Loughran
HDFS-10480 Add an admin command to list currently open files Major . Kihwal Lee Manoj Govindassamy
YARN-4161 Capacity Scheduler : Assign single or multiple containers per heart beat driven by configuration Major capacity scheduler Mayank Bansal Wei Yan
YARN-5355 YARN Timeline Service v.2: alpha 2 Critical timelineserver Sangjin Lee Vrushali C
HADOOP-13345 S3Guard: Improved Consistency for S3A Major fs/s3 Chris Nauroth Chris Nauroth
HDFS-7877 [Umbrella] Support maintenance state for datanodes Major datanode, namenode Ming Ma Ming Ma
YARN-2915 Enable YARN RM scale out via federation using multiple RM’s Major nodemanager, resourcemanager Sriram Rao Subru Krishnan
YARN-1492 truly shared cache for jars (jobjar/libjar) Major . Sangjin Lee Chris Trezzo
HDFS-10467 Router-based HDFS federation Major fs Íñigo Goiri Íñigo Goiri
YARN-5734 OrgQueue for easy CapacityScheduler queue configuration management Major . Min Shen Min Shen
MAPREDUCE-5951 Add support for the YARN Shared Cache Major . Chris Trezzo Chris Trezzo
YARN-6871 Add additional deSelects params in RMWebServices#getAppReport Major resourcemanager, router Giovanni Matteo Fumarola Tanuj Nayak
MAPREDUCE-6732 mapreduce tasks for YARN Timeline Service v.2: alpha 2 Major . Sangjin Lee Vrushali C
HADOOP-14840 Tool to estimate resource requirements of an application pipeline based on prior executions Major tools Subru Krishnan Rui Li
YARN-3813 Support Application timeout feature in YARN. Major scheduler nijel Rohith Sharma K S
YARN-2877 Extend YARN to support distributed scheduling Major nodemanager, resourcemanager Sriram Rao Konstantinos Karanasos
YARN-5220 Scheduling of OPPORTUNISTIC containers through YARN RM Major resourcemanager Konstantinos Karanasos Konstantinos Karanasos

IMPROVEMENTS:

JIRA Summary Priority Component Reporter Contributor
YARN-2280 Resource manager web service fields are not accessible Trivial resourcemanager Krisztian Horvath Krisztian Horvath
HDFS-9267 TestDiskError should get stored replicas through FsDatasetTestUtils. Minor test Lei (Eddy) Xu Lei (Eddy) Xu
HDFS-9491 Tests should get the number of pending async delets via FsDatasetTestUtils Minor test Tony Wu Tony Wu
HADOOP-12625 Add a config to disable the /logs endpoints Major security Robert Kanter Robert Kanter
YARN-4341 add doc about timeline performance tool usage Major . Chang Li Chang Li
HDFS-9281 Change TestDeleteBlockPool to not explicitly use File to check block pool existence. Minor test Lei (Eddy) Xu Lei (Eddy) Xu
HDFS-8477 describe dfs.ha.zkfc.port in hdfs-default.xml Minor . Kanaka Kumar Avvaru Kanaka Kumar Avvaru
HDFS-9350 Avoid creating temprorary strings in Block.toString() and getBlockName() Minor performance Staffan Friberg Staffan Friberg
HADOOP-12566 Add NullGroupMapping Major . Daniel Templeton Daniel Templeton
YARN-2934 Improve handling of container’s stderr Critical . Gera Shegalov Naganarasimha G R
HADOOP-12663 Remove Hard-Coded Values From FileSystem.java Trivial fs BELUGA BEHR BELUGA BEHR
HADOOP-12683 Add number of samples in last interval in snapshot of MutableStat Minor metrics Vikram Srivastava Vikram Srivastava
HADOOP-8887 Use a Maven plugin to build the native code using CMake Minor build Colin P. McCabe Colin P. McCabe
HADOOP-12713 Disable spurious checkstyle checks Major . Andrew Wang Andrew Wang
HADOOP-12662 The build should fail if a -Dbundle option fails Minor . Kai Zheng Kai Zheng
HDFS-9576 HTrace: collect position/length information on read operations Major hdfs-client, tracing Zhe Zhang Zhe Zhang
YARN-4603 FairScheduler should mention user requested queuename in error message when failed in queue ACL check Trivial fairscheduler Tao Jie Tao Jie
HDFS-9674 The HTrace span for OpWriteBlock should record the maxWriteToDisk time Major datanode, tracing Colin P. McCabe Colin P. McCabe
YARN-4496 Improve HA ResourceManager Failover detection on the client Major client, resourcemanager Arun Suresh Jian He
YARN-3542 Re-factor support for CPU as a resource using the new ResourceHandler mechanism Critical nodemanager Sidharta Seethana Varun Vasudev
HDFS-9541 Add hdfsStreamBuilder API to libhdfs to support defaultBlockSizes greater than 2 GB Major libhdfs Colin P. McCabe Colin P. McCabe
YARN-4462 FairScheduler: Disallow preemption from a queue Major fairscheduler Tao Jie Tao Jie
HDFS-9677 Rename generationStampV1/generationStampV2 to legacyGenerationStamp/generationStamp Minor namenode Jing Zhao Mingliang Liu
MAPREDUCE-6431 JobClient should be an AutoClosable Major . André Kelpe Haibo Chen
HDFS-7764 DirectoryScanner shouldn’t abort the scan if one directory had an error Major datanode Rakesh R Rakesh R
YARN-4647 Make RegisterNodeManagerRequestPBImpl thread-safe Major nodemanager Karthik Kambatla Karthik Kambatla
YARN-4649 Add additional logging to some NM state store operations Minor . Sidharta Seethana Sidharta Seethana
HADOOP-12759 RollingFileSystemSink should eagerly rotate directories Critical . Daniel Templeton Daniel Templeton
YARN-4628 Display application priority in yarn top Minor . Bibin A Chundatt Bibin A Chundatt
YARN-4655 Log uncaught exceptions/errors in various thread pools in YARN Major . Sidharta Seethana Sidharta Seethana
HDFS-9637 Tests for RollingFileSystemSink Major test Daniel Templeton Daniel Templeton
YARN-4569 Remove incorrect part of maxResources in FairScheduler documentation Major documentation Ray Chiang Ray Chiang
MAPREDUCE-6634 Log uncaught exceptions/errors in various thread pools in mapreduce Major . Sidharta Seethana Sidharta Seethana
HADOOP-12817 Enable TLS v1.1 and 1.2 Major security Robert Kanter Robert Kanter
MAPREDUCE-6627 Add machine-readable output to mapred job -history command Major client Robert Kanter Robert Kanter
YARN-4708 Missing default mapper type in TimelineServer performance test tool usage Minor timelineserver Kai Sasaki Kai Sasaki
HADOOP-12829 StatisticsDataReferenceCleaner swallows interrupt exceptions Minor fs Gregory Chanan Gregory Chanan
MAPREDUCE-6640 mapred job -history command should be able to take Job ID Major client Robert Kanter Robert Kanter
YARN-4697 NM aggregation thread pool is not bound by limits Critical nodemanager Haibo Chen Haibo Chen
YARN-4579 Allow DefaultContainerExecutor container log directory permissions to be configurable Major yarn Ray Chiang Ray Chiang
HADOOP-12841 Update s3-related properties in core-default.xml Minor fs/s3 Wei-Chiu Chuang Wei-Chiu Chuang
YARN-4718 Rename variables in SchedulerNode to reduce ambiguity post YARN-1011 Major scheduler Karthik Kambatla Íñigo Goiri
HDFS-9521 TransferFsImage.receiveFile should account and log separate times for image download and fsync to disk Minor . Wellington Chevreuil Wellington Chevreuil
HADOOP-11404 Clarify the “expected client Kerberos principal is null” authorization message Minor security Stephen Chu Stephen Chu
YARN-4719 Add a helper library to maintain node state and allows common queries Major scheduler Karthik Kambatla Karthik Kambatla
HDFS-9928 Make HDFS commands guide up to date Major documentation Wei-Chiu Chuang Wei-Chiu Chuang
YARN-4560 Make scheduler error checking message more user friendly Trivial yarn Ray Chiang Ray Chiang
HDFS-9579 Provide bytes-read-by-network-distance metrics at FileSystem.Statistics level Major . Ming Ma Ming Ma
YARN-4732 *ProcessTree classes have too many whitespace issues Trivial . Karthik Kambatla Gabor Liptak
HADOOP-12952 /BUILDING example of zero-docs dist should skip javadocs Trivial build, documentation Steve Loughran Steve Loughran
YARN-4436 DistShell ApplicationMaster.ExecBatScripStringtPath is misspelled Trivial applications/distributed-shell Daniel Templeton Matt LaMantia
YARN-4639 Remove dead code in TestDelegationTokenRenewer added in YARN-3055 Trivial resourcemanager Daniel Templeton Daniel Templeton
YARN-4607 Pagination support for AppAttempt page TotalOutstandingResource Requests table Minor . Bibin A Chundatt Bibin A Chundatt
YARN-4769 Add support for CSRF header in the dump capacity scheduler logs and kill app buttons in RM web UI Major . Varun Vasudev Varun Vasudev
HADOOP-12984 Add GenericTestUtils.getTestDir method and use it for temporary directory in tests Major build, test Steve Loughran Steve Loughran
YARN-4541 Change log message in LocalizedResource#handle() to DEBUG Minor . Ray Chiang Ray Chiang
YARN-1297 FairScheduler: Move some logs to debug and check if debug logging is enabled Major fairscheduler Sandy Ryza Yufei Gu
YARN-5003 Add container resource to RM audit log Major resourcemanager, scheduler Nathan Roberts Nathan Roberts
HADOOP-13068 Clean up RunJar and related test class Trivial util Andras Bokor Andras Bokor
YARN-4577 Enable aux services to have their own custom classpath/jar file Major . Xuan Gong Xuan Gong
YARN-5053 More informative diagnostics when applications killed by a user Major resourcemanager Jason Lowe Eric Badger
MAPREDUCE-6686 Add a way to download the job config from the mapred CLI Major client Robert Kanter Robert Kanter
HADOOP-12782 Faster LDAP group name resolution with ActiveDirectory Major security Wei-Chiu Chuang Wei-Chiu Chuang
MAPREDUCE-6696 Add a configuration to limit the number of map tasks allowed per job. Major job submission zhihai xu zhihai xu
YARN-4878 Expose scheduling policy and max running apps over JMX for Yarn queues Major yarn Yufei Gu Yufei Gu
HDFS-9782 RollingFileSystemSink should have configurable roll interval Major . Daniel Templeton Daniel Templeton
HADOOP-13160 Suppress checkstyle JavadocPackage check for test source Minor . John Zhuge John Zhuge
YARN-4766 NM should not aggregate logs older than the retention policy Major log-aggregation, nodemanager Haibo Chen Haibo Chen
HADOOP-13197 Add non-decayed call metrics for DecayRpcScheduler Major ipc, metrics Xiaoyu Yao Xiaoyu Yao
HADOOP-12291 Add support for nested groups in LdapGroupsMapping Major security Gautam Gopalakrishnan Esther Kundin
HDFS-7541 Upgrade Domains in HDFS Major . Ming Ma Ming Ma
HDFS-10256 Use GenericTestUtils.getTestDir method in tests for temporary directories Major build, test Vinayakumar B Vinayakumar B
YARN-5083 YARN CLI for AM logs does not give any error message if entered invalid am value Major yarn Sumana Sathish Jian He
YARN-5082 Limit ContainerId increase in fair scheduler if the num of node app reserved reached the limit Major . sandflee sandflee
YARN-4958 The file localization process should allow for wildcards to reduce the application footprint in the state store Critical nodemanager Daniel Templeton Daniel Templeton
MAPREDUCE-6719 The list of -libjars archives should be replaced with a wildcard in the distributed cache to reduce the application footprint in the state store Critical distributed-cache Daniel Templeton Daniel Templeton
MAPREDUCE-6721 mapreduce.reduce.shuffle.memory.limit.percent=0.0 should be legal to enforce shuffle to disk Major mrv2, task Gera Shegalov Gera Shegalov
HADOOP-13034 Log message about input options in distcp lacks some items Minor tools/distcp Takashi Ohnishi Takashi Ohnishi
HDFS-10568 Reuse ObjectMapper instance in CombinedHostsFileReader and CombinedHostsFileWriter Major hdfs-client Yiqun Lin Yiqun Lin
HADOOP-13337 Update maven-enforcer-plugin version to 1.4.1 Major build Tsuyoshi Ozawa Tsuyoshi Ozawa
YARN-4568 Fix message when NodeManager runs into errors initializing the recovery directory Major yarn Ray Chiang Ray Chiang
MAPREDUCE-6717 Remove deprecated StringUtils.getFormattedTimeWithDiff Minor . Akira Ajisaka Shen Yinjie
HDFS-10387 DataTransferProtocol#writeBlock missing some javadocs Minor datanode, hdfs Yongjun Zhang John Zhuge
YARN-5339 passing file to -out for YARN log CLI doesnt give warning or error code Major . Sumana Sathish Xuan Gong
YARN-5303 Clean up ContainerExecutor JavaDoc Minor nodemanager Daniel Templeton Daniel Templeton
MAPREDUCE-6652 Add configuration property to prevent JHS from loading jobs with a task count greater than X Major jobhistoryserver Haibo Chen Haibo Chen
YARN-5181 ClusterNodeTracker: add method to get list of nodes matching a specific resourceName Major scheduler Karthik Kambatla Karthik Kambatla
YARN-4996 Make TestNMReconnect.testCompareRMNodeAfterReconnect() scheduler agnostic, or better yet parameterized Minor resourcemanager, test Daniel Templeton Kai Sasaki
MAPREDUCE-6365 Refactor JobResourceUploader#uploadFilesInternal Minor . Chris Trezzo Chris Trezzo
HADOOP-13354 Update WASB driver to use the latest version (4.2.0) of SDK for Microsoft Azure Storage Clients Major fs/azure Sivaguru Sankaridurg Sivaguru Sankaridurg
YARN-5460 Change container runtime type logging in DelegatingLinuxContainerRuntime to debug Trivial yarn Shane Kumpf Shane Kumpf
HADOOP-13458 LoadBalancingKMSClientProvider#doOp should log IOException stacktrace Trivial kms Wei-Chiu Chuang Chen Liang
MAPREDUCE-6748 Enhance logging for Cluster.java around InetSocketAddress Minor . sarun singla Vrushali C
YARN-4910 Fix incomplete log info in ResourceLocalizationService Trivial . Jun Gong Jun Gong
HADOOP-13380 TestBasicDiskValidator should not write data to /tmp Minor . Lei (Eddy) Xu Yufei Gu
MAPREDUCE-6751 Add debug log message when splitting is not possible due to unsplittable compression Minor client, mrv1, mrv2 Peter Vary Peter Vary
YARN-5455 Update Javadocs for LinuxContainerExecutor Major nodemanager Daniel Templeton Daniel Templeton
YARN-4702 FairScheduler: Allow setting maxResources for ad hoc queues Major fairscheduler Karthik Kambatla Daniel Templeton
HDFS-10645 Make block report size as a metric and add this metric to datanode web ui Major datanode, ui Yuanbo Liu Yuanbo Liu
YARN-4491 yarn list command to support filtering by tags Minor client Steve Loughran Varun Saxena
HDFS-10784 Implement WebHdfsFileSystem#listStatusIterator Major webhdfs Andrew Wang Andrew Wang
HDFS-10822 Log DataNodes in the write pipeline Trivial hdfs-client John Zhuge John Zhuge
YARN-5616 Clean up WeightAdjuster Major fairscheduler Yufei Gu Yufei Gu
HDFS-10778 Add -format option to make the output of FileDistribution processor human-readable in OfflineImageViewer Major tools Yiqun Lin Yiqun Lin
HDFS-10847 Complete the document for FileDistribution processor in OfflineImageViewer Minor documentation Yiqun Lin Yiqun Lin
HDFS-10837 Standardize serializiation of WebHDFS DirectoryListing Major webhdfs Andrew Wang Andrew Wang
HADOOP-13411 Checkstyle suppression by annotation or comment Minor . John Zhuge John Zhuge
HADOOP-13412 Move dev-support/checkstyle/suppressions.xml to hadoop-build-tools Trivial . John Zhuge John Zhuge
HDFS-10823 Implement HttpFSFileSystem#listStatusIterator Major httpfs Andrew Wang Andrew Wang
MAPREDUCE-6632 Master.getMasterAddress() should be updated to use YARN-4629 Minor applicationmaster Daniel Templeton Daniel Templeton
YARN-5400 Light cleanup in ZKRMStateStore Trivial resourcemanager Daniel Templeton Daniel Templeton
MAPREDUCE-6718 add progress log to JHS during startup Minor jobhistoryserver Haibo Chen Haibo Chen
MAPREDUCE-6638 Do not attempt to recover progress from previous job attempts if spill encryption is enabled Major applicationmaster Karthik Kambatla Haibo Chen
HADOOP-13628 Support to retrieve specific property from configuration via REST API Major conf Weiwei Yang Weiwei Yang
HDFS-10683 Make class Token$PrivateToken private Minor . John Zhuge John Zhuge
HADOOP-13688 Stop bundling HTML source code in javadoc JARs Major build Andrew Wang Andrew Wang
YARN-5551 Ignore file backed pages from memory computation when smaps is enabled Minor . Rajesh Balamohan Rajesh Balamohan
YARN-5466 DefaultContainerExecutor needs JavaDocs Minor nodemanager Daniel Templeton Daniel Templeton
HADOOP-13522 Add %A and %a formats for fs -stat command to print permissions Major fs Alex Garbarini Alex Garbarini
HADOOP-13737 Cleanup DiskChecker interface Major util Arpit Agarwal Arpit Agarwal
HADOOP-13702 Add a new instrumented read-write lock Major common Jingcheng Du Jingcheng Du
MAPREDUCE-6792 Allow user’s full principal name as owner of MapReduce staging directory in JobSubmissionFiles#JobStagingDir() Major client Santhosh G Nayak Santhosh G Nayak
YARN-5575 Many classes use bare yarn. properties instead of the defined constants Major . Daniel Templeton Daniel Templeton
HDFS-11049 The description of dfs.block.replicator.classname is not clear Minor documentation Yiqun Lin Yiqun Lin
HDFS-11069 Tighten the authorization of datanode RPC Major datanode, security Kihwal Lee Kihwal Lee
YARN-4456 Clean up Lint warnings in nodemanager Minor nodemanager Daniel Templeton Daniel Templeton
YARN-4668 Reuse objectMapper instance in Yarn Major timelineclient Yiqun Lin Yiqun Lin
YARN-4907 Make all MockRM#waitForState consistent. Major resourcemanager Yufei Gu Yufei Gu
YARN-4396 Log the trace information on FSAppAttempt#assignContainer Major applications, fairscheduler Yiqun Lin Yiqun Lin
HDFS-11088 Quash unnecessary safemode WARN message during NameNode startup Trivial . Andrew Wang Yiqun Lin
YARN-4998 Minor cleanup to UGI use in AdminService Trivial resourcemanager Daniel Templeton Daniel Templeton
HDFS-10756 Expose getTrashRoot to HTTPFS and WebHDFS Major encryption, httpfs, webhdfs Xiao Chen Yuanbo Liu
YARN-5356 NodeManager should communicate physical resource capability to ResourceManager Major nodemanager, resourcemanager Nathan Roberts Íñigo Goiri
HADOOP-13802 Make generic options help more consistent, and aligned Minor . Grant Sohn Grant Sohn
HDFS-9482 Replace DatanodeInfo constructors with a builder pattern Major . Brahma Reddy Battula Brahma Reddy Battula
HADOOP-13687 Provide a unified dependency artifact that transitively includes the cloud storage modules shipped with Hadoop. Major build Chris Nauroth Chris Nauroth
YARN-5552 Add Builder methods for common yarn API records Major . Arun Suresh Tao Jie
YARN-4033 In FairScheduler, parent queues should also display queue status Major fairscheduler Siqi Li Siqi Li
HADOOP-13427 Eliminate needless uses of FileSystem#{exists(), isFile(), isDirectory()} Major fs Steve Loughran Mingliang Liu
YARN-5375 invoke MockRM#drainEvents implicitly in MockRM methods to reduce test failures Major resourcemanager sandflee sandflee
YARN-5722 FairScheduler hides group resolution exceptions when assigning queue Major fairscheduler Wilfred Spiegelenburg Wilfred Spiegelenburg
HADOOP-11552 Allow handoff on the server side for RPC requests Major ipc Siddharth Seth Siddharth Seth
HADOOP-13605 Clean up FileSystem javadocs, logging; improve diagnostics on FS load Major fs Steve Loughran Steve Loughran
HADOOP-13018 Make Kdiag check whether hadoop.token.files points to existent and valid files Major . Ravi Prakash Ravi Prakash
YARN-5890 FairScheduler should log information about AM-resource-usage and max-AM-share for queues Major fairscheduler Yufei Gu Yufei Gu
HDFS-7588 Improve the HDFS Web UI browser to allow chowning / chmoding, creating dirs and uploading files Major ui, webhdfs Ravi Prakash Ravi Prakash
YARN-4997 Update fair scheduler to use pluggable auth provider Major fairscheduler Daniel Templeton Tao Jie
MAPREDUCE-6787 Allow job_conf.xml to be downloadable on the job overview page in JHS Major jobhistoryserver Haibo Chen Haibo Chen
HDFS-11211 Add a time unit to the DataNode client trace format Minor datanode Akira Ajisaka Jagadesh Kiran N
HDFS-10206 Datanodes not sorted properly by distance when the reader isn’t a datanode Major . Ming Ma Nanda kumar
HADOOP-13709 Ability to clean up subprocesses spawned by Shell when the process exits Major . Eric Badger Eric Badger
HDFS-10930 Refactor: Wrap Datanode IO related operations Major datanode Xiaoyu Yao Xiaoyu Yao
HDFS-10959 Adding per disk IO statistics and metrics in DataNode. Major datanode Xiaoyu Yao Xiaoyu Yao
HDFS-10913 Introduce fault injectors to simulate slow mirrors Major datanode, test Xiaobing Zhou Xiaobing Zhou
HADOOP-13911 Remove TRUSTSTORE_PASSWORD related scripts from KMS Minor kms Xiao Chen John Zhuge
HADOOP-13863 Azure: Add a new SAS key mode for WASB. Major fs/azure Dushyanth Dushyanth
HDFS-10917 Collect peer performance statistics on DataNode. Major datanode Xiaobing Zhou Xiaobing Zhou
YARN-5969 FairShareComparator: Cache value of getResourceUsage for better performance Major fairscheduler zhangshilong zhangshilong
HDFS-11279 Cleanup unused DataNode#checkDiskErrorAsync() Minor . Xiaoyu Yao Hanisha Koneru
YARN-6015 AsyncDispatcher thread name can be set to improved debugging Major . Ajith S Ajith S
HADOOP-13953 Make FTPFileSystem’s data connection mode and transfer mode configurable Major fs Xiao Chen Xiao Chen
HDFS-11299 Support multiple Datanode File IO hooks Major hdfs Hanisha Koneru Hanisha Koneru
YARN-5849 Automatically create YARN control group for pre-mounted cgroups Minor . Miklos Szegedi Miklos Szegedi
HDFS-11339 Support File IO sampling for Datanode IO profiling hooks Major datanode Hanisha Koneru Hanisha Koneru
HDFS-11274 Datanode should only check the failed volume upon IO errors Major datanode Xiaoyu Yao Xiaoyu Yao
HDFS-11306 Print remaining edit logs from buffer if edit log can’t be rolled. Major ha, namenode Wei-Chiu Chuang Wei-Chiu Chuang
HADOOP-13496 Include file lengths in Mismatch in length error for distcp Minor . Ted Yu Ted Yu
YARN-6028 Add document for container metrics Major documentation, nodemanager Weiwei Yang Weiwei Yang
MAPREDUCE-6728 Give fetchers hint when ShuffleHandler rejects a shuffling connection Major mrv2 Haibo Chen Haibo Chen
HDFS-10534 NameNode WebUI should display DataNode usage histogram Major namenode, ui Zhe Zhang Kai Sasaki
HADOOP-14003 Make additional KMS tomcat settings configurable Major kms Andrew Wang Andrew Wang
HDFS-11374 Skip FSync in Test util CreateEditsLog to speed up edit log generation Minor hdfs Hanisha Koneru Hanisha Koneru
HDFS-9884 Use doxia macro to generate in-page TOC of HDFS site documentation Major documentation Masatake Iwasaki Masatake Iwasaki
YARN-6131 FairScheduler: Lower update interval for faster tests Major fairscheduler Karthik Kambatla Karthik Kambatla
YARN-6106 Document FairScheduler ‘allowPreemptionFrom’ queue property Minor fairscheduler Yufei Gu Yufei Gu
YARN-4658 Typo in o.a.h.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler comment Major . Daniel Templeton Udai Kiran Potluri
MAPREDUCE-6644 Use doxia macro to generate in-page TOC of MapReduce site documentation Major documentation Masatake Iwasaki Masatake Iwasaki
HDFS-11112 Journal Nodes should refuse to format non-empty directories Major . Arpit Agarwal Yiqun Lin
HADOOP-14050 Add process name to kms process Minor kms, scripts Rushabh S Shah Rushabh S Shah
HADOOP-14053 Update the link to HTrace SpanReceivers Minor documentation Akira Ajisaka Yiqun Lin
HADOOP-12097 Allow port range to be specified while starting webapp Major . Varun Saxena Varun Saxena
MAPREDUCE-6404 Allow AM to specify a port range for starting its webapp Major applicationmaster Varun Saxena Varun Saxena
MAPREDUCE-6842 Update the links in PiEstimator document Minor documentation Akira Ajisaka Jung Yoo
HDFS-11390 Add process name to httpfs process Minor httpfs, scripts John Zhuge Weiwei Yang
HDFS-11409 DatanodeInfo getNetworkLocation and setNetworkLocation shoud use volatile instead of synchronized Minor performance Chen Liang Chen Liang
YARN-6061 Add an UncaughtExceptionHandler for critical threads in RM Major resourcemanager Yufei Gu Yufei Gu
YARN-4753 Use doxia macro to generate in-page TOC of YARN site documentation Major documentation Masatake Iwasaki Masatake Iwasaki
HDFS-11333 Print a user friendly error message when plugins are not found Minor namenode Wei-Chiu Chuang Wei-Chiu Chuang
YARN-6174 Log files pattern should be same for both running and finished container Major yarn Sumana Sathish Xuan Gong
HDFS-11375 Display the volume storage type in datanode UI Minor datanode, ui Surendra Singh Lilhore Surendra Singh Lilhore
YARN-6125 The application attempt’s diagnostic message should have a maximum size Critical resourcemanager Daniel Templeton Andras Piros
HDFS-11406 Remove unused getStartInstance and getFinalizeInstance in FSEditLogOp Trivial . Andrew Wang Alison Yu
HDFS-11438 Fix typo in error message of StoragePolicyAdmin tool Trivial . Alison Yu Alison Yu
YARN-6194 Cluster capacity in SchedulingPolicy is updated only on allocation file reload Major fairscheduler Karthik Kambatla Yufei Gu
HADOOP-14097 Remove Java6 specific code from GzipCodec.java Minor . Akira Ajisaka Elek, Marton
HADOOP-13817 Add a finite shell command timeout to ShellBasedUnixGroupsMapping Minor security Harsh J Harsh J
HDFS-11295 Check storage remaining instead of node remaining in BlockPlacementPolicyDefault.chooseReplicaToDelete() Major namenode X. Liang Elek, Marton
HADOOP-14083 KMS should support old SSL clients Minor kms John Zhuge John Zhuge
HADOOP-14127 Add log4j configuration to enable logging in hadoop-distcp’s tests Minor test Xiao Chen Xiao Chen
HDFS-11466 Change dfs.namenode.write-lock-reporting-threshold-ms default from 1000ms to 5000ms Major namenode Andrew Wang Andrew Wang
YARN-6189 Improve application status log message when RM restarted when app is in NEW state Major . Yesha Vora Junping Du
HDFS-11432 Federation : Support fully qualified path for Quota/Snapshot/cacheadmin/cryptoadmin commands Major federation Brahma Reddy Battula Brahma Reddy Battula
HDFS-11418 HttpFS should support old SSL clients Minor httpfs John Zhuge John Zhuge
HDFS-11461 DataNode Disk Outlier Detection Major hdfs Hanisha Koneru Hanisha Koneru
HADOOP-13930 Azure: Add Authorization support to WASB Major fs/azure Dushyanth Sivaguru Sankaridurg
HDFS-11494 Log message when DN is not selected for block replication Minor . Yiqun Lin Yiqun Lin
HDFS-8741 Proper error msg to be printed when invalid operation type is given to WebHDFS operations Minor webhdfs Archana T Surendra Singh Lilhore
HADOOP-14108 CLI MiniCluster: add an option to specify NameNode HTTP port Minor . Takanobu Asanuma Takanobu Asanuma
HDFS-10838 Last full block report received time for each DN should be easily discoverable Major ui Arpit Agarwal Surendra Singh Lilhore
HDFS-11477 Simplify file IO profiling configuration Minor . Hanisha Koneru Hanisha Koneru
YARN-6287 RMCriticalThreadUncaughtExceptionHandler.rmContext should be final Minor resourcemanager Daniel Templeton Corey Barker
HADOOP-14150 Implement getHomeDirectory() method in NativeAzureFileSystem Critical fs/azure Namit Maheshwari Santhosh G Nayak
YARN-6300 NULL_UPDATE_REQUESTS is redundant in TestFairScheduler Minor . Daniel Templeton Yuanbo Liu
YARN-6042 Dump scheduler and queue state information into FairScheduler DEBUG log Major fairscheduler Yufei Gu Yufei Gu
HDFS-11511 Support Timeout when checking single disk Major hdfs Hanisha Koneru Hanisha Koneru
HDFS-11517 Expose slow disks via DataNode JMX Major hdfs Hanisha Koneru Hanisha Koneru
HADOOP-14169 Implement listStatusIterator, listLocatedStatus for ViewFs Minor viewfs Erik Krogen Erik Krogen
HDFS-11547 Add logs for slow BlockReceiver while writing data to disk Major datanode Xiaobing Zhou Xiaobing Zhou
MAPREDUCE-6865 Fix typo in javadoc for DistributedCache Trivial . Attila Sasvari Attila Sasvari
YARN-6309 Fair scheduler docs should have the queue and queuePlacementPolicy elements listed in bold so that they’re easier to see Minor fairscheduler Daniel Templeton esmaeil mirzaee
HADOOP-13945 Azure: Add Kerberos and Delegation token support to WASB client. Major fs/azure Santhosh G Nayak Santhosh G Nayak
HDFS-11545 Propagate DataNode’s slow disks info to the NameNode via Heartbeat Major . Hanisha Koneru Hanisha Koneru
YARN-6284 hasAlreadyRun should be final in ResourceManager.StandByTransitionRunnable Major resourcemanager Daniel Templeton Laura Adams
HADOOP-14213 Move Configuration runtime check for hadoop-site.xml to initialization Major . Jonathan Eagles Jonathan Eagles
HDFS-11534 Add counters for number of blocks in pending IBR Major hdfs Xiaobing Zhou Xiaobing Zhou
YARN-5956 Refactor ClientRMService to unify error handling across apis Minor resourcemanager Kai Sasaki Kai Sasaki
YARN-6379 Remove unused argument in ClientRMService Trivial . Kai Sasaki Kai Sasaki
HADOOP-14233 Delay construction of PreCondition.check failure message in Configuration#set Major . Jonathan Eagles Jonathan Eagles
HADOOP-14240 Configuration#get return value optimization Major . Jonathan Eagles Jonathan Eagles
YARN-6339 Improve performance for createAndGetApplicationReport Major . yunjiong zhao yunjiong zhao
HDFS-11170 Add builder-based create API to FileSystem Major . Sammi Chen Sammi Chen
YARN-6329 Remove unnecessary TODO comment from AppLogAggregatorImpl.java Minor . Akira Ajisaka victor bertschinger
HDFS-9705 Refine the behaviour of getFileChecksum when length = 0 Minor . Kai Zheng Sammi Chen
HDFS-11551 Handle SlowDiskReport from DataNode at the NameNode Major hdfs Hanisha Koneru Hanisha Koneru
HDFS-11603 Improve slow mirror/disk warnings in BlockReceiver Major datanode Arpit Agarwal Arpit Agarwal
HDFS-11560 Expose slow disks via NameNode JMX Major namenode Hanisha Koneru Hanisha Koneru
HDFS-9651 All web UIs should include a robots.txt file Minor . Lars Francke Lars Francke
HDFS-11628 Clarify the behavior of HDFS Mover in documentation Major documentation Xiaobing Zhou Xiaobing Zhou
YARN-6381 FSAppAttempt has several variables that should be final Major fairscheduler Daniel Templeton Ameet Zaveri
HDFS-11302 Improve Logging for SSLHostnameVerifier Major security Xiaoyu Yao Chen Liang
HADOOP-14104 Client should always ask namenode for kms provider path. Major kms Rushabh S Shah Rushabh S Shah
YARN-5797 Add metrics to the node manager for cleaning the PUBLIC and PRIVATE caches Major . Chris Trezzo Chris Trezzo
HADOOP-14276 Add a nanosecond API to Time/Timer/FakeTimer Minor util Erik Krogen Erik Krogen
YARN-6195 Export UsedCapacity and AbsoluteUsedCapacity to JMX Major capacityscheduler, metrics, yarn Benson Qiu Benson Qiu
HDFS-11558 BPServiceActor thread name is too long Minor datanode Tsz Wo Nicholas Sze Xiaobing Zhou
HADOOP-14246 Authentication Tokens should use SecureRandom instead of Random and 256 bit secrets Major security Robert Kanter Robert Kanter
HDFS-11648 Lazy construct the IIP pathname Major . Daryn Sharp Daryn Sharp
HADOOP-14274 Azure: Simplify Ranger-WASB policy model Major fs/azure Sivaguru Sankaridurg Sivaguru Sankaridurg
MAPREDUCE-6673 Add a test example job that grows in memory usage over time Major test Karthik Kambatla Karthik Kambatla
HADOOP-11794 Enable distcp to copy blocks in parallel Major tools/distcp dhruba borthakur Yongjun Zhang
YARN-6406 Remove SchedulerRequestKeys when no more pending ResourceRequest Major . Arun Suresh Arun Suresh
HDFS-11634 Optimize BlockIterator when iterating starts in the middle. Major . Konstantin Shvachko Konstantin Shvachko
HADOOP-14141 Store KMS SSL keystore password in catalina.properties Minor kms John Zhuge John Zhuge
YARN-6164 Expose Queue Configurations per Node Label through YARN client api Major . Benson Qiu Benson Qiu
YARN-6392 Add submit time to Application Summary log Minor resourcemanager zhihai xu zhihai xu
HADOOP-12856 FileUtil.checkDest() and RawLocalFileSystem.mkdirs() to throw stricter IOEs; RawLocalFS contract tests to verify Minor fs Steve Loughran Steve Loughran
HDFS-11384 Add option for balancer to disperse getBlocks calls to avoid NameNode’s rpc.CallQueueLength spike Major balancer & mover yunjiong zhao Konstantin Shvachko
HDFS-8873 Allow the directoryScanner to be rate-limited Major datanode Nathan Roberts Daniel Templeton
HDFS-11722 Change Datanode file IO profiling sampling to percentage Major hdfs Hanisha Koneru Hanisha Koneru
HDFS-11687 Add new public encryption APIs required by Hive Major encryption Andrew Wang Lei (Eddy) Xu
HADOOP-14383 Implement FileSystem that reads from HTTP / HTTPS endpoints Major fs Haohui Mai Haohui Mai
YARN-6457 Allow custom SSL configuration to be supplied in WebApps Major webapp, yarn Sanjay M Pujare Sanjay M Pujare
HADOOP-14216 Improve Configuration XML Parsing Performance Major . Jonathan Eagles Jonathan Eagles
HDFS-11800 Document output of ‘hdfs count -u’ should contain PATHNAME Minor hdfs Xiaobing Zhou Xiaobing Zhou
HADOOP-14413 Add Javadoc comment for jitter parameter on CachingGetSpaceUsed Trivial . Erik Krogen Erik Krogen
HADOOP-14417 Update default SSL cipher list for KMS Minor kms, security John Zhuge John Zhuge
HDFS-11816 Update default SSL cipher list for HttpFS Minor httpfs, security John Zhuge John Zhuge
HDFS-11641 Reduce cost of audit logging by using FileStatus instead of HdfsFileStatus Major hdfs Daryn Sharp Daryn Sharp
HADOOP-14242 Make KMS Tomcat SSL property sslEnabledProtocols and clientAuth configurable Major kms John Zhuge John Zhuge
HDFS-11579 Make HttpFS Tomcat SSL property sslEnabledProtocols and clientAuth configurable Major httpfs John Zhuge John Zhuge
YARN-6493 Print requested node partition in assignContainer logs Major . Jonathan Hung Jonathan Hung
HADOOP-14407 DistCp - Introduce a configurable copy buffer size Major tools/distcp Omkar Aradhya K S Omkar Aradhya K S
YARN-6582 FSAppAttempt demand can be updated atomically in updateDemand() Major . Karthik Kambatla Karthik Kambatla
HDFS-11421 Make WebHDFS’ ACLs RegEx configurable Major webhdfs Harsh J Harsh J
HDFS-11891 DU#refresh should print the path of the directory when an exception is caught Minor . Chen Liang Chen Liang
HADOOP-14442 Owner support for ranger-wasb integration Major fs, fs/azure Varada Hemeswari Varada Hemeswari
YARN-6497 Method length of ResourceManager#serviceInit() is too long Minor resourcemanager Yufei Gu Gergely Novák
HDFS-11383 Intern strings in BlockLocation and ExtendedBlock Major . Misha Dmitriev Misha Dmitriev
YARN-6208 Improve the log when FinishAppEvent sent to the NodeManager which didn’t run the application Minor . Akira Ajisaka Akira Ajisaka
HADOOP-14440 Add metrics for connections dropped Major . Eric Badger Eric Badger
HADOOP-14491 Azure has messed doc structure Major documentation, fs/azure Mingliang Liu Mingliang Liu
HDFS-11914 Add more diagnosis info for fsimage transfer failure. Major . Yongjun Zhang Yongjun Zhang
HDFS-11840 Log HDFS Mover exception message of exit to its own log Minor balancer & mover LiXin Ge LiXin Ge
HDFS-11861 ipc.Client.Connection#sendRpcRequest should log request name Trivial ipc John Zhuge John Zhuge
HADOOP-14465 LdapGroupsMapping - support user and group search base Major common, security Shwetha G S Shwetha G S
HADOOP-14310 RolloverSignerSecretProvider.LOG should be @VisibleForTesting Minor security Daniel Templeton Arun Shanmugam Kumar
HDFS-11907 Add metric for time taken by NameNode resource check Major . Chen Liang Chen Liang
HADOOP-14503 Make RollingAverages a mutable metric Major common Hanisha Koneru Hanisha Koneru
HADOOP-14506 Add create() contract test that verifies ancestor dir creation Minor fs Aaron Fabbri Sean Mackrory
HADOOP-14523 OpensslAesCtrCryptoCodec.finalize() holds excessive amounts of memory Major . Misha Dmitriev Misha Dmitriev
HADOOP-14524 Make CryptoCodec Closeable so it can be cleaned up proactively Major . Xiao Chen Xiao Chen
HADOOP-14424 Add CRC32C performance test. Minor common LiXin Ge LiXin Ge
HDFS-11345 Document the configuration key for FSNamesystem lock fairness Minor documentation, namenode Zhe Zhang Erik Krogen
HDFS-11992 Replace commons-logging APIs with slf4j in FsDatasetImpl Major . Akira Ajisaka hu xiaodong
HDFS-11993 Add log info when connect to datanode socket address failed Major hdfs-client chencan chencan
HADOOP-14536 Update azure-storage sdk to version 5.3.0 Major fs/azure Mingliang Liu Georgi Chalakov
YARN-6738 LevelDBCacheTimelineStore should reuse ObjectMapper instances Major timelineserver Zoltan Haindrich Zoltan Haindrich
HADOOP-14515 Specifically configure zookeeper-related log levels in KMS log4j Major kms Xiao Chen Xiao Chen
HDFS-11881 NameNode consumes a lot of memory for snapshot diff report generation Major hdfs, snapshots Manoj Govindassamy Manoj Govindassamy
HADOOP-14611 NetworkTopology.DEFAULT_HOST_LEVEL is unused Trivial . Daniel Templeton Chen Liang
YARN-6751 Display reserved resources in web UI per queue Major fairscheduler, webapp Abdullah Yousufi Abdullah Yousufi
HDFS-12042 Lazy initialize AbstractINodeDiffList#diffs for snapshots to reduce memory consumption Major . Misha Dmitriev Misha Dmitriev
HDFS-12078 Add time unit to the description of property dfs.namenode.stale.datanode.interval in hdfs-default.xml Minor documentation, hdfs Weiwei Yang Weiwei Yang
YARN-6752 Display reserved resources in web UI per application Major fairscheduler Abdullah Yousufi Abdullah Yousufi
YARN-6746 SchedulerUtils.checkResourceRequestMatchingNodePartition() is dead code Minor scheduler Daniel Templeton Deepti Sawhney
YARN-6410 FSContext.scheduler should be final Minor fairscheduler Daniel Templeton Yeliang Cang
YARN-6764 Simplify the logic in FairScheduler#attemptScheduling Trivial fairscheduler Yufei Gu Yufei Gu
HADOOP-14443 Azure: Support retry and client side failover for authorization, SASKey and delegation token generation Major fs/azure Santhosh G Nayak Santhosh G Nayak
HADOOP-14535 wasb: implement high-performance random access and seek of block blobs Major fs/azure Thomas Marquardt Thomas Marquardt
HADOOP-14629 Improve exception checking in FileContext related JUnit tests Major fs, test Andras Bokor Andras Bokor
YARN-6689 PlacementRule should be configurable Major . Jonathan Hung Jonathan Hung
HDFS-12130 Optimizing permission check for getContentSummary Major namenode Chen Liang Chen Liang
HDFS-12137 DN dataset lock should be fair Critical datanode Daryn Sharp Daryn Sharp
YARN-6280 Introduce deselect query param to skip ResourceRequest from getApp/getApps REST API Major resourcemanager, restapi Lantao Jin Lantao Jin
HDFS-12138 Remove redundant ‘public’ modifiers from BlockCollection Trivial namenode Chen Liang Chen Liang
HADOOP-14640 Azure: Support affinity for service running on localhost and reuse SPNEGO hadoop.auth cookie for authorization, SASKey and delegation token generation Major fs/azure Santhosh G Nayak Santhosh G Nayak
YARN-6778 In ResourceWeights, weights and setWeights() should be final Minor scheduler Daniel Templeton Daniel Templeton
HDFS-12067 Correct dfsadmin commands usage message to reflects IPC port Major . steven-wugang steven-wugang
HADOOP-14666 Tests use assertTrue(….equals(…)) instead of assertEquals() Minor test Daniel Templeton Daniel Templeton
HADOOP-14659 UGI getShortUserName does not need to search the Subject Major common Daryn Sharp Daryn Sharp
YARN-6768 Improve performance of yarn api record toString and fromString Major . Jonathan Eagles Jonathan Eagles
YARN-6779 DominantResourceFairnessPolicy.DominantResourceFairnessComparator.calculateShares() should be @VisibleForTesting Trivial fairscheduler Daniel Templeton Yeliang Cang
YARN-6845 Variable scheduler of FSLeafQueue duplicates the one of its parent FSQueue. Trivial fairscheduler Yufei Gu Yufei Gu
HADOOP-14681 Remove MockitoMaker class Major test Andras Bokor Andras Bokor
HDFS-12143 Improve performance of getting and removing inode features Major namenode Daryn Sharp Daryn Sharp
HDFS-12171 Reduce IIP object allocations for inode lookup Major namenode Daryn Sharp Daryn Sharp
HDFS-12190 Enable ‘hdfs dfs -stat’ to display access time Major hdfs, shell Yongjun Zhang Yongjun Zhang
YARN-6864 FSPreemptionThread cleanup for readability Minor fairscheduler Daniel Templeton Daniel Templeton
YARN-5892 Support user-specific minimum user limit percentage in Capacity Scheduler Major capacityscheduler Eric Payne Eric Payne
HADOOP-14455 ViewFileSystem#rename should support be supported within same nameservice with different mountpoints Major viewfs Brahma Reddy Battula Brahma Reddy Battula
HADOOP-14690 RetryInvocationHandler$RetryInfo should override toString() Minor . Akira Ajisaka Yeliang Cang
HADOOP-14709 Fix checkstyle warnings in ContractTestUtils Minor test Steve Loughran Thomas Marquardt
MAPREDUCE-6914 Tests use assertTrue(….equals(…)) instead of assertEquals() Minor test Daniel Templeton Daniel Templeton
YARN-6832 Tests use assertTrue(….equals(…)) instead of assertEquals() Minor test Daniel Templeton Daniel Templeton
HADOOP-14706 Adding a helper method to determine whether a log is Log4j implement Minor util Wenxin He Wenxin He
HDFS-12251 Add document for StreamCapabilities Major . Lei (Eddy) Xu Lei (Eddy) Xu
YARN-6634 [API] Refactor ResourceManager WebServices to make API explicit Critical resourcemanager Subru Krishnan Giovanni Matteo Fumarola
YARN-6802 Add Max AM Resource and AM Resource Usage to Leaf Queue View in FairScheduler WebUI Major fairscheduler YunFan Zhou YunFan Zhou
HDFS-12264 DataNode uses a deprecated method IoUtils#cleanup. Major . Ajay Kumar Ajay Kumar
YARN-6757 Refactor the usage of yarn.nodemanager.linux-container-executor.cgroups.mount-path Minor nodemanager Miklos Szegedi Miklos Szegedi
YARN-6811 [ATS1.5] All history logs should be kept under its own User Directory. Major timelineclient, timelineserver Rohith Sharma K S Rohith Sharma K S
YARN-6879 TestLeafQueue.testDRFUserLimits() has commented out code Trivial capacity scheduler, test Daniel Templeton Angela Wang
MAPREDUCE-6923 Optimize MapReduce Shuffle I/O for small partitions Major . Robert Schmidtke Robert Schmidtke
YARN-6884 AllocationFileLoaderService.loadQueue() has an if without braces Trivial fairscheduler Daniel Templeton weiyuan
HADOOP-14627 Support MSI and DeviceCode token provider in ADLS Major fs/adl Atul Sikaria Atul Sikaria
HADOOP-14741 Refactor curator based ZooKeeper communication into common library Major . Subru Krishnan Íñigo Goiri
YARN-6917 Queue path is recomputed from scratch on every allocation Minor capacityscheduler Jason Lowe Eric Payne
HADOOP-14662 Update azure-storage sdk to version 5.4.0 Major fs/azure Thomas Marquardt Thomas Marquardt
YARN-3254 HealthReport should include disk full information Major nodemanager Akira Ajisaka Suma Shivaprasad
HDFS-12117 HttpFS does not seem to support SNAPSHOT related methods for WebHDFS REST Interface Major httpfs Wellington Chevreuil Wellington Chevreuil
YARN-7053 Move curator transaction support to ZKCuratorManager Major . Jonathan Hung Jonathan Hung
HADOOP-14251 Credential provider should handle property key deprecation Critical security John Zhuge John Zhuge
YARN-7049 FSAppAttempt preemption related fields have confusing names Major fairscheduler Karthik Kambatla Karthik Kambatla
YARN-6999 Add log about how to solve Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster Minor documentation, security Linlin Zhou Linlin Zhou
YARN-7037 Optimize data transfer with zero-copy approach for containerlogs REST API in NMWebServices Major nodemanager Tao Yang Tao Yang
MAPREDUCE-6937 Backport MAPREDUCE-6870 to branch-2 while preserving compatibility Major . Zhe Zhang Peter Bacsko
YARN-6780 ResourceWeights.toString() cleanup Minor scheduler Daniel Templeton weiyuan
YARN-5547 NMLeveldbStateStore should be more tolerant of unknown keys Major nodemanager Jason Lowe Ajith S
HDFS-12182 BlockManager.metaSave does not distinguish between “under replicated” and “missing” blocks Trivial hdfs Wellington Chevreuil Wellington Chevreuil
HADOOP-14688 Intern strings in KeyVersion and EncryptedKeyVersion Major kms Xiao Chen Xiao Chen
HADOOP-14827 Allow StopWatch to accept a Timer parameter for tests Minor common, test Erik Krogen Erik Krogen
HDFS-12131 Add some of the FSNamesystem JMX values as metrics Minor hdfs, namenode Erik Krogen Erik Krogen
HADOOP-14844 Remove requirement to specify TenantGuid for MSI Token Provider Major fs/adl Atul Sikaria Atul Sikaria
YARN-6799 Remove the duplicated code in CGroupsHandlerImp.java Trivial nodemanager Yufei Gu weiyuan
HADOOP-14520 WASB: Block compaction for Azure Block Blobs Major fs/azure Georgi Chalakov Georgi Chalakov
HADOOP-14839 DistCp log output should contain copied and deleted files and directories Major tools/distcp Konstantin Shaposhnikov Yiqun Lin
HADOOP-14843 Improve FsPermission symbolic parsing unit test coverage Minor fs Jason Lowe Bharat Viswanadham
YARN-7057 FSAppAttempt#getResourceUsage doesn’t need to consider resources queued for preemption Major fairscheduler Karthik Kambatla Karthik Kambatla
HADOOP-14864 FSDataInputStream#unbuffer UOE should include stream class name Minor fs John Zhuge Bharat Viswanadham
HDFS-12441 Suppress UnresolvedPathException in namenode log Minor . Kihwal Lee Kihwal Lee
HDFS-12472 Add JUNIT timeout to TestBlockStatsMXBean Minor . Lei (Eddy) Xu Bharat Viswanadham
MAPREDUCE-6958 Shuffle audit logger should log size of shuffle transfer Minor . Jason Lowe Jason Lowe
HADOOP-7308 Remove unused TaskLogAppender configurations from log4j.properties Major conf Todd Lipcon Todd Lipcon
HDFS-12496 Make QuorumJournalManager timeout properties configurable Major . Ajay Kumar Ajay Kumar
YARN-7045 Remove FSLeafQueue#addAppSchedulable Major fairscheduler Yufei Gu Sen Zhao
HDFS-12530 Processor argument in Offline Image Viewer should be case insensitive Minor tools Hanisha Koneru Hanisha Koneru
YARN-7240 Add more states and transitions to stabilize the NM Container state machine Major . Arun Suresh kartheek muthyala
YARN-65 Reduce RM app memory footprint once app has completed Major resourcemanager Jason Lowe Manikandan R
YARN-6333 Improve doc for minSharePreemptionTimeout, fairSharePreemptionTimeout and fairSharePreemptionThreshold Major fairscheduler Yufei Gu Chetna Chaudhari
HADOOP-14095 Document caveats about the default JavaKeyStoreProvider in KMS Major documentation, kms Xiao Chen Xiao Chen
YARN-4879 Enhance Allocate Protocol to Identify Requests Explicitly Major applications, resourcemanager Subru Krishnan Subru Krishnan
HDFS-12420 Add an option to disallow ‘namenode format -force’ Major . Ajay Kumar Ajay Kumar
HADOOP-14521 KMS client needs retry logic Major . Rushabh S Shah Rushabh S Shah
YARN-2162 add ability in Fair Scheduler to optionally configure maxResources in terms of percentage Major fairscheduler, scheduler Ashwin Shankar Yufei Gu
YARN-7207 Cache the RM proxy server address Major RM Yufei Gu Yufei Gu
HADOOP-14920 KMSClientProvider won’t work with KMS delegation token retrieved from non-Java client. Major kms Xiaoyu Yao Xiaoyu Yao
YARN-6930 Admins should be able to explicitly enable specific LinuxContainerRuntime in the NodeManager Major nodemanager Vinod Kumar Vavilapalli Shane Kumpf
HDFS-12603 Enable async edit logging by default Major namenode Andrew Wang Andrew Wang
HDFS-12642 Log block and datanode details in BlockRecoveryWorker Major datanode Xiao Chen Xiao Chen
HADOOP-14938 Configuration.updatingResource map should be initialized lazily Major . Misha Dmitriev Misha Dmitriev
YARN-6608 Backport all SLS improvements from trunk to branch-2 Major . Carlo Curino Carlo Curino
HADOOP-14880 [KMS] Document&test missing KMS client side configs Minor . Wei-Chiu Chuang Gabor Bota
HDFS-12619 Do not catch and throw unchecked exceptions if IBRs fail to process Minor namenode Wei-Chiu Chuang Wei-Chiu Chuang
YARN-7359 TestAppManager.testQueueSubmitWithNoPermission() should be scheduler agnostic Minor . Haibo Chen Haibo Chen
HADOOP-14944 Add JvmMetrics to KMS Major kms Xiao Chen Xiao Chen
YARN-7261 Add debug message for better download latency monitoring Major nodemanager Yufei Gu Yufei Gu
YARN-7357 Several methods in TestZKRMStateStore.TestZKRMStateStoreTester.TestZKRMStateStoreInternal should have @Override annotations Trivial resourcemanager Daniel Templeton Sen Zhao
YARN-4163 Audit getQueueInfo and getApplications calls Major . Chang Li Chang Li
HADOOP-9657 NetUtils.wrapException to have special handling for 0.0.0.0 addresses and :0 ports Minor net Steve Loughran Varun Saxena
YARN-7389 Make TestResourceManager Scheduler agnostic Major test Robert Kanter Robert Kanter
YARN-7358 TestZKConfigurationStore and TestLeveldbConfigurationStore should explicitly set capacity scheduler Minor resourcemanager Haibo Chen Haibo Chen
YARN-7262 Add a hierarchy into the ZKRMStateStore for delegation token znodes to prevent jute buffer overflow Major . Robert Kanter Robert Kanter
HADOOP-14870 backport HADOOP-14553 parallel tests to branch-2 Major fs/azure, test Steve Loughran Steve Loughran
HADOOP-14992 Upgrade Avro patch version Major build Chris Douglas Bharat Viswanadham
YARN-6413 FileSystem based Yarn Registry implementation Major amrmproxy, api, resourcemanager Ellen Hui Ellen Hui
HDFS-12744 More logs when short-circuit read is failed and disabled Major datanode Weiwei Yang Weiwei Yang
MAPREDUCE-6975 Logging task counters Major task Prabhu Joseph Prabhu Joseph
HADOOP-14112 Über-jira adl:// Azure Data Lake Phase I: Stabilization Major fs/adl Steve Loughran John Zhuge
HADOOP-14552 Über-jira: WASB client phase II: performance and testing Major fs/azure Steve Loughran Thomas Marquardt
HADOOP-13738 DiskChecker should perform some disk IO Major . Arpit Agarwal Arpit Agarwal

BUG FIXES:

JIRA Summary Priority Component Reporter Contributor
HADOOP-7256 Resource leak during failure scenario of closing of resources. Minor util ramkrishna.s.vasudevan ramkrishna.s.vasudevan
YARN-524 TestYarnVersionInfo failing if generated properties doesn’t include an SVN URL Minor api Steve Loughran Steve Loughran
YARN-1471 The SLS simulator is not running the preemption policy for CapacityScheduler Minor . Carlo Curino Carlo Curino
HADOOP-11703 git should ignore .DS_Store files on Mac OS X Major . Abin Shahab Abin Shahab
HDFS-8797 WebHdfsFileSystem creates too many connections for pread Major webhdfs Jing Zhao Jing Zhao
HDFS-9107 Prevent NN’s unrecoverable death spiral after full GC Critical namenode Daryn Sharp Daryn Sharp
HDFS-9034 “StorageTypeStats” Metric should not count failed storage. Major namenode Archana T Surendra Singh Lilhore
YARN-4156 TestAMRestart#testAMBlacklistPreventsRestartOnSameNode assumes CapacityScheduler Major . Anubhav Dhoot Anubhav Dhoot
YARN-4417 Make RM and Timeline-server REST APIs more consistent Major . Wangda Tan Wangda Tan
YARN-4522 Queue acl can be checked at app submission Major . Jian He Jian He
YARN-4530 LocalizedResource trigger a NPE Cause the NodeManager exit Major . tangshangwen tangshangwen
HADOOP-12655 TestHttpServer.testBindAddress bind port range is wider than expected Major . Wei-Chiu Chuang Wei-Chiu Chuang
HADOOP-12573 TestRPC.testClientBackOff failing Major test Steve Loughran Xiao Chen
HADOOP-12653 Use SO_REUSEADDR to avoid getting “Address already in use” when using kerberos and attempting to bind to any port on the local IP address Major net Colin P. McCabe Colin P. McCabe
YARN-4571 Make app id/name available to the yarn authorizer provider for better auditing Major . Jian He Jian He
YARN-4551 Address the duplication between StatusUpdateWhenHealthy and StatusUpdateWhenUnhealthy transitions Minor nodemanager Karthik Kambatla Sunil Govindan
HDFS-9517 Fix missing @Test annotation on TestDistCpUtils.testUnpackAttributes Trivial distcp Wei-Chiu Chuang Wei-Chiu Chuang
HDFS-9624 DataNode start slowly due to the initial DU command operations Major . Yiqun Lin Yiqun Lin
YARN-4611 Fix scheduler load simulator to support multi-layer network location Major . Ming Ma Ming Ma
YARN-4584 RM startup failure when AM attempts greater than max-attempts Critical . Bibin A Chundatt Bibin A Chundatt
HDFS-9618 Fix mismatch between log level and guard in BlockManager#computeRecoveryWorkForBlocks Minor namenode Masatake Iwasaki Masatake Iwasaki
YARN-4497 RM might fail to restart when recovering apps whose attempts are missing Critical . Jun Gong Jun Gong
YARN-4612 Fix rumen and scheduler load simulator handle killed tasks properly Major . Ming Ma Ming Ma
MAPREDUCE-6620 Jobs that did not start are shown as starting in 1969 in the JHS web UI Major jobhistoryserver Daniel Templeton Haibo Chen
YARN-4625 Make ApplicationSubmissionContext and ApplicationSubmissionContextInfo more consistent Major . Xuan Gong Xuan Gong
YARN-4594 container-executor fails to remove directory tree when chmod required Major nodemanager Colin P. McCabe Colin P. McCabe
YARN-4669 Fix logging statements in resource manager’s Application class Trivial . Sidharta Seethana Sidharta Seethana
YARN-4629 Distributed shell breaks under strong security Major applications/distributed-shell, security Daniel Templeton Daniel Templeton
HDFS-9608 Disk IO imbalance in HDFS with heterogeneous storages Major . Wei Zhou Wei Zhou
YARN-4689 FairScheduler: Cleanup preemptContainer to be more readable Trivial fairscheduler Karthik Kambatla Kai Sasaki
YARN-4651 movetoqueue option does not documented in ‘YARN Commands’ Major documentation Takashi Ohnishi Takashi Ohnishi
YARN-4648 Move preemption related tests from TestFairScheduler to TestFairSchedulerPreemption Major fairscheduler Karthik Kambatla Kai Sasaki
YARN-4729 SchedulerApplicationAttempt#getTotalRequiredResources can throw an NPE Major scheduler Karthik Kambatla Karthik Kambatla
YARN-4701 When task logs are not available, port 8041 is referenced instead of port 8042 Major yarn Haibo Chen Haibo Chen
HDFS-9858 RollingFileSystemSink can throw an NPE on non-secure clusters Major . Daniel Templeton Daniel Templeton
YARN-4731 container-executor should not follow symlinks in recursive_unlink_children Blocker . Bibin A Chundatt Colin P. McCabe
HADOOP-10321 TestCompositeService should cover all enumerations of adding a service to a parent service Major . Karthik Kambatla Ray Chiang
YARN-4737 Add CSRF filter support in YARN Major nodemanager, resourcemanager, webapp Jonathan Maron Jonathan Maron
YARN-4762 NMs failing on DelegatingLinuxContainerRuntime init with LCE on Blocker . Vinod Kumar Vavilapalli Sidharta Seethana
YARN-4764 Application submission fails when submitted queue is not available in scheduler xml Major . Bibin A Chundatt Bibin A Chundatt
HADOOP-12895 SSLFactory#createSSLSocketFactory exception message is wrong Trivial . Wei-Chiu Chuang Wei-Chiu Chuang
HADOOP-12906 AuthenticatedURL should convert a 404/Not Found into an FileNotFoundException. Minor io, security Steve Loughran Steve Loughran
HDFS-9947 Block#toString should not output information from derived classes Minor . Colin P. McCabe Colin P. McCabe
YARN-4816 SystemClock API broken in 2.9.0 Major . Siddharth Seth Siddharth Seth
HDFS-9780 RollingFileSystemSink doesn’t work on secure clusters Critical . Daniel Templeton Daniel Templeton
HADOOP-12888 Shell to disable bash and setsid support when running under JVM security manager Major security Costin Leau Costin Leau
YARN-4593 Deadlock in AbstractService.getConfig() Major yarn Steve Loughran Steve Loughran
HDFS-10173 Typo in DataXceiverServer Trivial datanode Michael Han Michael Han
YARN-4812 TestFairScheduler#testContinuousScheduling fails intermittently Major fairscheduler Karthik Kambatla Karthik Kambatla
MAPREDUCE-6110 JobHistoryServer CLI throws NullPointerException with job ids that do not exist Minor jobhistoryserver Li Lu Kai Sasaki
MAPREDUCE-6535 TaskID default constructor results in NPE on toString() Major mrv2 Daniel Templeton Daniel Templeton
MAPREDUCE-6655 Fix a typo (STRICT_IE6) in Encrypted Shuffle Trivial documentation Wei-Chiu Chuang Wei-Chiu Chuang
HADOOP-12954 Add a way to change hadoop.security.token.service.use_ip Major security Robert Kanter Robert Kanter
YARN-4657 Javadoc comment is broken for Resources.multiplyByAndAddTo() Trivial . Daniel Templeton Daniel Templeton
YARN-4880 Running TestZKRMStateStorePerf with real zookeeper cluster throws NPE Major . Rohith Sharma K S Sunil Govindan
YARN-4609 RM Nodes list page takes too much time to load Major webapp Bibin A Chundatt Bibin A Chundatt
YARN-4906 Capture container start/finish time in container metrics Major . Jian He Jian He
HDFS-10192 Namenode safemode not coming out during failover Major namenode Brahma Reddy Battula Brahma Reddy Battula
MAPREDUCE-6647 MR usage counters use the resources requested instead of the resources allocated Major . Haibo Chen Haibo Chen
HADOOP-13006 FileContextMainOperationsBaseTest.testListStatusThrowsExceptionForNonExistentFile() doesnt run Minor test Steve Loughran Kai Sasaki
YARN-4927 TestRMHA#testTransitionedToActiveRefreshFail fails with FairScheduler Major test Karthik Kambatla Bibin A Chundatt
YARN-4562 YARN WebApp ignores the configuration passed to it for keystore settings Major . Sergey Shelukhin Sergey Shelukhin
YARN-4897 dataTables_wrapper change min height Minor . Bibin A Chundatt Bibin A Chundatt
HDFS-8356 Document missing properties in hdfs-default.xml Major documentation Ray Chiang Ray Chiang
YARN-4810 NM applicationpage cause internal error 500 Major . Bibin A Chundatt Bibin A Chundatt
HDFS-10273 Remove duplicate logSync() and log message in FSN#enterSafemode() Minor . Vinayakumar B Vinayakumar B
HDFS-10282 The VolumeScanner should warn about replica files which are misplaced Major datanode Colin P. McCabe Colin P. McCabe
HDFS-10283 o.a.h.hdfs.server.namenode.TestFSImageWithSnapshot#testSaveLoadImageWithAppending fails intermittently Major test Mingliang Liu Mingliang Liu
HDFS-10306 SafeModeMonitor should not leave safe mode if name system is starting active service Major namenode Mingliang Liu Mingliang Liu
YARN-4935 TestYarnClient#testSubmitIncorrectQueue fails with FairScheduler Major test Yufei Gu Yufei Gu
MAPREDUCE-2398 MRBench: setting the baseDir parameter has no effect Minor benchmarks Michael Noll Wilfred Spiegelenburg
YARN-4976 Missing NullPointer check in ContainerLaunchContextPBImpl causes RM to die Major resourcemanager Giovanni Matteo Fumarola Giovanni Matteo Fumarola
YARN-4795 ContainerMetrics drops records Major resourcemanager Daniel Templeton Daniel Templeton
YARN-4983 JVM and UGI metrics disappear after RM is once transitioned to standby mode Major . Li Lu Li Lu
HADOOP-13012 yetus-wrapper should fail sooner when download fails Minor yetus Steven K. Wong Steven K. Wong
HADOOP-12469 distcp should not ignore the ignoreFailures option Critical tools/distcp Gera Shegalov Mingliang Liu
MAPREDUCE-6677 LocalContainerAllocator doesn’t specify resource of the containers allocated. Major mr-am Haibo Chen Haibo Chen
YARN-5002 getApplicationReport call may raise NPE for removed queues Critical . Sumana Sathish Jian He
HADOOP-13118 Fix IOUtils#cleanup and IOUtils#closeStream javadoc Trivial io Wei-Chiu Chuang Wei-Chiu Chuang
HADOOP-10694 Remove synchronized input streams from Writable deserialization Major io Gopal V Gopal V
HDFS-10410 RedundantEditLogInputStream#LOG is set to wrong class Minor . John Zhuge John Zhuge
HDFS-10208 Addendum for HDFS-9579: to handle the case when client machine can’t resolve network path Major . Ming Ma Ming Ma
MAPREDUCE-6701 application master log can not be available when clicking jobhistory’s am logs link Critical jobhistoryserver chenyukang Haibo Chen
HDFS-10404 Fix formatting of CacheAdmin command usage help text Major caching Yiqun Lin Yiqun Lin
MAPREDUCE-6657 job history server can fail on startup when NameNode is in start phase Major jobhistoryserver Haibo Chen Haibo Chen
HDFS-10360 DataNode may format directory and lose blocks if current/VERSION is missing Major datanode Wei-Chiu Chuang Wei-Chiu Chuang
YARN-5076 YARN web interfaces lack XFS protection Major nodemanager, resourcemanager, timelineserver Jonathan Maron Jonathan Maron
YARN-5126 Remove CHANGES.txt from branch-2 Major . Akira Ajisaka Akira Ajisaka
YARN-5112 Excessive log warnings for directory permission issue on NM recovery. Major . Jian He Jian He
YARN-4979 FSAppAttempt demand calculation considers demands at multiple locality levels different Major fairscheduler zhihai xu zhihai xu
MAPREDUCE-6703 Add flag to allow MapReduce AM to request for OPPORTUNISTIC containers Major . Arun Suresh Arun Suresh
YARN-5131 Distributed shell AM fails when extra container arrives during finishing Major . Sumana Sathish Wangda Tan
YARN-4866 FairScheduler: AMs can consume all vcores leading to a livelock when using FAIR policy Major fairscheduler Karthik Kambatla Yufei Gu
HDFS-10463 TestRollingFileSystemSinkWithHdfs needs some cleanup Critical . Daniel Templeton Daniel Templeton
HDFS-10449 TestRollingFileSystemSinkWithHdfs#testFailedClose() fails on branch-2 Major test Takanobu Asanuma Takanobu Asanuma
HDFS-10468 HDFS read ends up ignoring an interrupt Major . Siddharth Seth Jing Zhao
MAPREDUCE-6240 Hadoop client displays confusing error message Major client Mohammad Kamrul Islam Gera Shegalov
YARN-4308 ContainersAggregated CPU resource utilization reports negative usage in first few heartbeats Major nodemanager Sunil Govindan Sunil Govindan
HDFS-10508 DFSInputStream should set thread’s interrupt status after catching InterruptException from sleep Major . Jing Zhao Jing Zhao
HADOOP-13243 TestRollingFileSystemSink.testSetInitialFlushTime() fails intermittently Minor test Daniel Templeton Daniel Templeton
YARN-5077 Fix FSLeafQueue#getFairShare() for queues with zero fairshare Major . Yufei Gu Yufei Gu
HDFS-10437 ReconfigurationProtocol not covered by HDFSPolicyProvider. Major namenode Chris Nauroth Arpit Agarwal
MAPREDUCE-6197 Cache MapOutputLocations in ShuffleHandler Major . Siddharth Seth Junping Du
YARN-5266 Wrong exit code while trying to get app logs using regex via CLI Critical yarn Sumana Sathish Xuan Gong
HDFS-10561 test_native_mini_dfs fails by NoClassDefFoundError Major native, test Akira Ajisaka Akira Ajisaka
HDFS-10555 Unable to loadFSEdits due to a failure in readCachePoolInfo Critical caching, namenode Uma Maheswara Rao G Uma Maheswara Rao G
MAPREDUCE-5803 Counters page display all task neverthless of task type( Map or Reduce) Minor jobhistoryserver Rohith Sharma K S Kai Sasaki
YARN-5182 MockNodes.newNodes creates one more node per rack than requested Major test Karthik Kambatla Karthik Kambatla
YARN-5282 Fix typos in CapacityScheduler documentation Trivial documentation Ray Chiang Ray Chiang
HDFS-10572 Fix TestOfflineEditsViewer#testGenerated Blocker test Xiaoyu Yao Surendra Singh Lilhore
YARN-5296 NMs going OutOfMemory because ContainerMetrics leak in ContainerMonitorImpl Major nodemanager Karam Singh Junping Du
YARN-5294 Pass remote ip address down to YarnAuthorizationProvider Major . Jian He Jian He
YARN-4366 Fix Lint Warnings in YARN Common Major yarn Daniel Templeton Daniel Templeton
YARN-5362 TestRMRestart#testFinishedAppRemovalAfterRMRestart can fail Major . Jason Lowe sandflee
HDFS-10617 PendingReconstructionBlocks.size() should be synchronized Major . Eric Badger Eric Badger
YARN-5383 Fix findbugs for nodemanager & checkstyle warnings in nodemanager.ContainerExecutor Major nodemanager Vrushali C Vrushali C
MAPREDUCE-6733 MapReduce JerseyTest tests failing with “java.net.BindException: Address already in use” Critical test Vinod Kumar Vavilapalli Vinod Kumar Vavilapalli
HADOOP-13254 Create framework for configurable disk checkers Major util Yufei Gu Yufei Gu
YARN-5272 Handle queue names consistently in FairScheduler Major fairscheduler Wilfred Spiegelenburg Wilfred Spiegelenburg
YARN-5213 Fix a bug in LogCLIHelpers which cause TestLogsCLI#testFetchApplictionLogs fails intermittently Major test Rohith Sharma K S Xuan Gong
YARN-5441 Fixing minor Scheduler test case failures Major . Subru Krishnan Subru Krishnan
YARN-5440 Use AHSClient in YarnClient when TimelineServer is running Major . Xuan Gong Xuan Gong
YARN-5203 Return ResourceRequest JAXB object in ResourceManager Cluster Applications REST API Major . Subru Krishnan Ellen Hui
HDFS-9276 Failed to Update HDFS Delegation Token for long running application in HA mode Major fs, ha, security Liangliang Gu Liangliang Gu
YARN-5436 Race in AsyncDispatcher can cause random test failures in Tez (probably YARN also) Major . Zhiyuan Yang Zhiyuan Yang
YARN-5444 Fix failing unit tests in TestLinuxContainerExecutorWithMocks Major nodemanager Yufei Gu Yufei Gu
HADOOP-13443 KMS should check the type of underlying keyprovider of KeyProviderExtension before falling back to default Minor kms Anthony Young-Garner Anthony Young-Garner
YARN-5333 Some recovered apps are put into default queue when RM HA Major . Jun Gong Jun Gong
HADOOP-13353 LdapGroupsMapping getPassward shouldn’t return null when IOException throws Major security Zhaohao Liang Wei-Chiu Chuang
HADOOP-13403 AzureNativeFileSystem rename/delete performance improvements Major fs/azure Subramanyam Pattipaka Subramanyam Pattipaka
HDFS-10457 DataNode should not auto-format block pool directory if VERSION is missing Major datanode Wei-Chiu Chuang Wei-Chiu Chuang
HDFS-4176 EditLogTailer should call rollEdits with a timeout Major ha, namenode Todd Lipcon Lei (Eddy) Xu
HADOOP-13476 CredentialProviderFactory fails at class loading from libhdfs (JNI) Major fs/s3 Aaron Fabbri Aaron Fabbri
HADOOP-13461 NPE in KeyProvider.rollNewVersion Minor . Colm O hEigeartaigh Colm O hEigeartaigh
HADOOP-13441 Document LdapGroupsMapping keystore password properties Minor security Wei-Chiu Chuang Yuanbo Liu
YARN-4833 For Queue AccessControlException client retries multiple times on both RM Major . Bibin A Chundatt Bibin A Chundatt
HDFS-9696 Garbage snapshot records lingering forever Critical . Kihwal Lee Kihwal Lee
YARN-5521 TestCapacityScheduler#testKillAllAppsInQueue fails randomly Major . Varun Saxena sandflee
HADOOP-13437 KMS should reload whitelist and default key ACLs when hot-reloading Major kms Xiao Chen Xiao Chen
YARN-5475 Test failed for TestAggregatedLogFormat on trunk Major . Junping Du Jun Gong
YARN-5523 Yarn running container log fetching causes OutOfMemoryError Major log-aggregation Prasanth Jayachandran Xuan Gong
HADOOP-11786 Fix Javadoc typos in org.apache.hadoop.fs.FileSystem Trivial documentation Chen He Andras Bokor
YARN-5526 DrainDispacher#ServiceStop blocked if setDrainEventsOnStop invoked Major . sandflee sandflee
HDFS-10763 Open files can leak permanently due to inconsistent lease update Critical . Kihwal Lee Kihwal Lee
YARN-5533 JMX AM Used metrics for queue wrong when app submited to nodelabel partition Major . Bibin A Chundatt Bibin A Chundatt
MAPREDUCE-6762 ControlledJob#toString failed with NPE when job status is not successfully updated Major . Weiwei Yang Weiwei Yang
HADOOP-13526 Add detailed logging in KMS for the authentication failure of proxy user Minor kms Suraj Acharya Suraj Acharya
HDFS-8312 Trash does not descent into child directories to check for permissions Critical fs, security Eric Yang Weiwei Yang
YARN-5537 Intermittent test failure of TestAMRMClient#testAMRMClientWithContainerResourceChange Major . Varun Saxena Bibin A Chundatt
YARN-5430 Return container’s ip and host from NM ContainerStatus call Major . Jian He Jian He
YARN-5373 NPE listing wildcard directory in containerLaunch Blocker nodemanager Haibo Chen Daniel Templeton
MAPREDUCE-6628 Potential memory leak in CryptoOutputStream Major security Mariappan Asokan Mariappan Asokan
MAPREDUCE-6777 Typos in 4 log messages Trivial . Mehran Hassani Mehran Hassani
YARN-5642 Typos in 9 log messages Trivial . Mehran Hassani Mehran Hassani
YARN-5657 Fix TestDefaultContainerExecutor Major test Akira Ajisaka Arun Suresh
HADOOP-13602 Fix some warnings by findbugs in hadoop-maven-plugin Major . Tsuyoshi Ozawa Tsuyoshi Ozawa
YARN-4973 YarnWebParams next.fresh.interval should be next.refresh.interval Minor webapp Daniel Templeton Daniel Templeton
YARN-5662 Provide an option to enable ContainerMonitor Major . Jian He Jian He
HADOOP-11780 Prevent IPC reader thread death Critical ipc Daryn Sharp Daryn Sharp
HADOOP-13164 Optimize S3AFileSystem::deleteUnnecessaryFakeDirectories Minor fs/s3 Rajesh Balamohan Rajesh Balamohan
YARN-5693 Reduce loglevel to Debug in ContainerManagementProtocolProxy and AMRMClientImpl Major yarn Yufei Gu Yufei Gu
YARN-5678 Log demand as demand in FSLeafQueue and FSParentQueue Major fairscheduler Yufei Gu Yufei Gu
YARN-5672 FairScheduler: wrong queue name in log when adding application Minor fairscheduler Wilfred Spiegelenburg Wilfred Spiegelenburg
YARN-4767 Network issues can cause persistent RM UI outage Critical webapp Daniel Templeton Daniel Templeton
MAPREDUCE-6789 Fix TestAMWebApp failure Major test Akira Ajisaka Daniel Templeton
HADOOP-13690 Fix typos in core-default.xml Minor documentation Yiqun Lin Yiqun Lin
HDFS-10969 Fix typos in hdfs-default.xml Minor documentation Yiqun Lin Yiqun Lin
YARN-5057 resourcemanager.security.TestDelegationTokenRenewer fails in trunk Major . Yongjun Zhang Jason Lowe
HADOOP-13697 LogLevel#main throws exception if no arguments provided Major . Mingliang Liu Mingliang Liu
HDFS-10699 Log object instance get incorrectly in TestDFSAdmin Minor . Yiqun Lin Yiqun Lin
HDFS-10423 Increase default value of httpfs maxHttpHeaderSize Minor hdfs Nicolae Popa Nicolae Popa
HADOOP-13626 Remove distcp dependency on FileStatus serialization Major tools/distcp Chris Douglas Chris Douglas
YARN-5711 Propogate exceptions back to client when using hedging RM failover provider Critical applications, resourcemanager Subru Krishnan Subru Krishnan
YARN-5754 Null check missing for earliest in FifoPolicy Major fairscheduler Yufei Gu Yufei Gu
HDFS-11040 Add documentation for HDFS-9820 distcp improvement Major distcp Yongjun Zhang Yongjun Zhang
YARN-5753 fix NPE in AMRMClientImpl.getMatchingRequests() Major yarn Haibo Chen Haibo Chen
HDFS-9929 Duplicate keys in NAMENODE_SPECIFIC_KEYS Minor namenode Akira Ajisaka Akira Ajisaka
YARN-5752 TestLocalResourcesTrackerImpl#testLocalResourceCache times out Major . Eric Badger Eric Badger
YARN-5710 Fix inconsistent naming in class ResourceRequest Trivial yarn Yufei Gu Yufei Gu
YARN-5686 DefaultContainerExecutor random working dir algorigthm skews results Minor . Miklos Szegedi Vrushali C
MAPREDUCE-6798 Fix intermittent failure of TestJobHistoryParsing.testJobHistoryMethods() Major jobhistoryserver Haibo Chen Haibo Chen
YARN-5757 RM REST API documentation is not up to date Trivial resourcemanager, yarn Miklos Szegedi Miklos Szegedi
YARN-5420 Delete org.apache.hadoop.yarn.server.resourcemanager.resource.Priority as its not necessary Minor resourcemanager Sunil Govindan Sunil Govindan
YARN-5172 Update yarn daemonlog documentation due to HADOOP-12847 Trivial documentation Wei-Chiu Chuang Wei-Chiu Chuang
YARN-4388 Cleanup “mapreduce.job.hdfs-servers” from yarn-default.xml Minor yarn Junping Du Junping Du
YARN-2306 Add test for leakage of reservation metrics in fair scheduler Minor fairscheduler Hong Zhiguo Hong Zhiguo
YARN-4743 FairSharePolicy breaks TimSort assumption Major fairscheduler Zephyr Guo Zephyr Guo
YARN-5809 AsyncDispatcher possibly invokes multiple shutdown thread when handling exception Major . Jian He Jian He
YARN-5805 Add isDebugEnabled check for debug logs in nodemanager Minor . Bibin A Chundatt Bibin A Chundatt
YARN-5788 Apps not activiated and AM limit resource in UI and REST not updated after -replaceLabelsOnNode Major . Bibin A Chundatt Bibin A Chundatt
MAPREDUCE-6765 MR should not schedule container requests in cases where reducer or mapper containers demand resource larger than the maximum supported Minor mr-am Haibo Chen Haibo Chen
HDFS-11095 BlockManagerSafeMode should respect extension period default config value (30s) Minor namenode Mingliang Liu Mingliang Liu
YARN-4862 Handle duplicate completed containers in RMNodeImpl Major resourcemanager Rohith Sharma K S Rohith Sharma K S
YARN-5377 Fix TestQueuingContainerManager.testKillMultipleOpportunisticContainers Major . Rohith Sharma K S Konstantinos Karanasos
MAPREDUCE-6782 JHS task page search based on each individual column not working Major jobhistoryserver Bibin A Chundatt Ajith S
HADOOP-13789 Hadoop Common includes generated test protos in both jar and test-jar Major build, common Sean Busbey Sean Busbey
YARN-5823 Update NMTokens in case of requests with only opportunistic containers Blocker . Konstantinos Karanasos Konstantinos Karanasos
YARN-5862 TestDiskFailures.testLocalDirsFailures failed Major . Yufei Gu Yufei Gu
YARN-5453 FairScheduler#update may skip update demand resource of child queue/app if current demand reached maxResource Major fairscheduler sandflee sandflee
YARN-5843 Incorrect documentation for timeline service entityType/events REST end points Minor . Bibin A Chundatt Bibin A Chundatt
YARN-5834 TestNodeStatusUpdater.testNMRMConnectionConf compares nodemanager wait time to the incorrect value Trivial . Miklos Szegedi Chang Li
YARN-5545 Fix issues related to Max App in capacity scheduler Major capacity scheduler Bibin A Chundatt Bibin A Chundatt
YARN-5825 ProportionalPreemptionalPolicy could use readLock over LeafQueue instead of synchronized block Major capacity scheduler Sunil Govindan Sunil Govindan
YARN-5874 RM -format-state-store and -remove-application-from-state-store commands fail with NPE Critical resourcemanager Varun Saxena Varun Saxena
YARN-5873 RM crashes with NPE if generic application history is enabled Critical resourcemanager Varun Saxena Varun Saxena
MAPREDUCE-6811 TestPipeApplication#testSubmitter fails after HADOOP-13802 Major test Brahma Reddy Battula Brahma Reddy Battula
YARN-5891 yarn rmadmin -help contains a misspelled ResourceManager Trivial resourcemanager Grant Sohn Grant Sohn
YARN-5870 Expose getApplications API in YarnClient with GetApplicationsRequest parameter Major client Gour Saha Jian He
YARN-3538 TimelineServer doesn’t catch/translate all exceptions raised Minor timelineserver Steve Loughran Steve Loughran
YARN-5904 Reduce the number of default server threads for AMRMProxyService Minor nodemanager Subru Krishnan Subru Krishnan
MAPREDUCE-6793 io.sort.factor code default and mapred-default.xml values inconsistent Trivial task Gera Shegalov Prabhu Joseph
YARN-5911 DrainDispatcher does not drain all events on stop even if setDrainEventsOnStop is true Major . Varun Saxena Varun Saxena
YARN-5918 Handle Opportunistic scheduling allocate request failure when NM is lost Major . Bibin A Chundatt Bibin A Chundatt
YARN-5920 Fix deadlock in TestRMHA.testTransitionedToStandbyShouldNotHang Major test Rohith Sharma K S Varun Saxena
HADOOP-13833 TestSymlinkHdfsFileSystem#testCreateLinkUsingPartQualPath2 fails after HADOOP13605 Critical test Brahma Reddy Battula Brahma Reddy Battula
YARN-5851 TestContainerManagerSecurity testContainerManager[1] failed Major nodemanager Haibo Chen Haibo Chen
MAPREDUCE-6565 Configuration to use host name in delegation token service is not read from job.xml during MapReduce job execution. Major . Chris Nauroth Li Lu
YARN-5942 “Overridden” is misspelled as “overriden” in FairScheduler.md Trivial site Daniel Templeton Heather Sutherland
YARN-5901 Fix race condition in TestGetGroups beforeclass setup() Major yarn Haibo Chen Haibo Chen
YARN-5929 Missing scheduling policy in the FS queue metric. Major . Yufei Gu Yufei Gu
HADOOP-13675 Bug in return value for delete() calls in WASB Major fs/azure Dushyanth Dushyanth
MAPREDUCE-6571 JobEndNotification info logs are missing in AM container syslog Minor applicationmaster Prabhu Joseph Haibo Chen
HADOOP-13859 TestConfigurationFieldsBase fails for fields that are DEFAULT values of skipped properties. Major common Haibo Chen Haibo Chen
YARN-5932 Retrospect moveApplicationToQueue in align with YARN-5611 Major capacity scheduler, resourcemanager Sunil Govindan Sunil Govindan
YARN-5136 Error in handling event type APP_ATTEMPT_REMOVED to the scheduler Major . tangshangwen Wilfred Spiegelenburg
MAPREDUCE-6817 The format of job start time in JHS is different from those of submit and finish time Major jobhistoryserver Haibo Chen Haibo Chen
YARN-5963 Spelling errors in logging and exceptions for node manager, client, web-proxy, common, and app history code Trivial client, nodemanager Grant Sohn Grant Sohn
HADOOP-13867 FilterFileSystem should override rename(.., options) to take effect of Rename options called via FilterFileSystem implementations Major . Vinayakumar B Vinayakumar B
YARN-4752 FairScheduler should preempt for a ResourceRequest and all preempted containers should be on the same node Major fairscheduler Karthik Kambatla Karthik Kambatla
HDFS-11226 cacheadmin,cryptoadmin and storagepolicyadmin should support generic options Minor tools Archana T Brahma Reddy Battula
MAPREDUCE-6822 should set HADOOP_JOB_HISTORYSERVER_HEAPSIZE only if it’s empty on branch2 Major scripts Fei Hui
YARN-5999 AMRMClientAsync will stop if any exceptions thrown on allocate call Major . Jian He Jian He
HADOOP-13831 Correct check for error code to detect Azure Storage Throttling and provide retries Major fs/azure Gaurav Kanade Gaurav Kanade
HADOOP-13508 FsPermission string constructor does not recognize sticky bit Major . Atul Sikaria Atul Sikaria
HDFS-11253 FileInputStream leak on failure path in BlockSender Major datanode Arpit Agarwal Arpit Agarwal
YARN-4330 MiniYARNCluster is showing multiple Failed to instantiate default resource calculator warning messages. Blocker test, yarn Steve Loughran Varun Saxena
YARN-5903 Fix race condition in TestResourceManagerAdministrationProtocolPBClientImpl beforeclass setup method Major yarn Haibo Chen Haibo Chen
YARN-5774 MR Job stuck in ACCEPTED status without any progress in Fair Scheduler if set yarn.scheduler.minimum-allocation-mb to 0. Blocker resourcemanager Yufei Gu Yufei Gu
YARN-6000 Make AllocationFileLoaderService.Listener public Major fairscheduler, yarn Tao Jie Tao Jie
YARN-6026 A couple of spelling errors in the docs Trivial documentation Grant Sohn Grant Sohn
HADOOP-13940 Document the missing envvars commands Minor documentation Yiqun Lin Yiqun Lin
HADOOP-13943 TestCommonConfigurationFields#testCompareXmlAgainstConfigurationClass fails after HADOOP-13863 Major test Brahma Reddy Battula Brahma Reddy Battula
YARN-5962 Spelling errors in logging and exceptions for resource manager code Trivial resourcemanager Grant Sohn Grant Sohn
YARN-5257 Fix unreleased resources and null dereferences Major . Yufei Gu Yufei Gu
YARN-6001 Improve moveApplicationQueues command line Major client Sunil Govindan Sunil Govindan
YARN-4882 Change the log level to DEBUG for recovering completed applications Major resourcemanager Rohith Sharma K S Daniel Templeton
HDFS-11251 ConcurrentModificationException during DataNode#refreshVolumes Major . Jason Lowe Manoj Govindassamy
HDFS-11267 Avoid redefinition of storageDirs in NNStorage and cleanup its accessors in Storage Major . Manoj Govindassamy Manoj Govindassamy
HADOOP-13942 Build failure due to errors of javadoc build in hadoop-azure Major fs/azure Kai Sasaki Kai Sasaki
YARN-5988 RM unable to start in secure setup Blocker . Ajith S Ajith S
MAPREDUCE-6715 Fix Several Unsafe Practices Major . Yufei Gu Yufei Gu
HDFS-11282 Document the missing metrics of DataNode Volume IO operations Minor documentation Yiqun Lin Yiqun Lin
YARN-6041 Opportunistic containers : Combined patch for branch-2 Major . Arun Suresh Arun Suresh
YARN-6066 Opportunistic containers minor fixes: API annotations and config parameter changes Minor . Arun Suresh Arun Suresh
YARN-6068 Log aggregation get failed when NM restart even with recovery Blocker . Junping Du Junping Du
YARN-6073 Misuse of format specifier in Preconditions.checkArgument Trivial . Yongjun Zhang Yuanbo Liu
YARN-5899 Debug log in AbstractCSQueue#canAssignToThisQueue needs improvement Trivial capacity scheduler Ying Zhang Ying Zhang
YARN-6054 TimelineServer fails to start when some LevelDb state files are missing. Critical . Ravi Prakash Ravi Prakash
YARN-6022 Revert changes of AbstractResourceRequest Blocker . Wangda Tan Wangda Tan
YARN-4148 When killing app, RM releases app’s resource before they are released by NM Major resourcemanager Jun Gong Jason Lowe
YARN-6079 simple spelling errors in yarn test code Trivial test Grant Sohn vijay
HADOOP-13903 Improvements to KMS logging to help debug authorization errors Minor kms Tristan Stevens Tristan Stevens
YARN-6072 RM unable to start in secure mode Blocker resourcemanager Bibin A Chundatt Ajith S
YARN-6081 LeafQueue#getTotalPendingResourcesConsideringUserLimit should deduct reserved from pending to avoid unnecessary preemption of reserved container Critical . Wangda Tan Wangda Tan
HADOOP-13928 TestAdlFileContextMainOperationsLive.testGetFileContext1 runtime error Major fs/adl, test John Zhuge John Zhuge
HDFS-11307 The rpc to portmap service for NFS has hardcoded timeout. Major . Jitendra Nath Pandey Mukul Kumar Singh
YARN-6057 yarn.scheduler.minimum-allocation-* descriptions are incorrect about behavior when a request is out of bounds Minor . Bibin A Chundatt Julia Sommer
HADOOP-13976 Path globbing does not match newlines Major . Eric Badger Eric Badger
HDFS-11209 SNN can’t checkpoint when rolling upgrade is not finalized Critical rolling upgrades Xiaoyu Yao Xiaoyu Yao
HDFS-10733 NameNode terminated after full GC thinking QJM is unresponsive. Major namenode, qjm Konstantin Shvachko Vinitha Reddy Gankidi
HDFS-11316 TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails in trunk Minor . Yiqun Lin Yiqun Lin
HDFS-11287 Storage class member storageDirs should be private to avoid unprotected access by derived classes Major . Manoj Govindassamy Manoj Govindassamy
YARN-6110 Fix opportunistic containers documentation Minor documentation Akira Ajisaka Akira Ajisaka
YARN-6104 RegistrySecurity overrides zookeeper sasl system properties Major . Billie Rinaldi Billie Rinaldi
HADOOP-13858 TestGridmixMemoryEmulation and TestResourceUsageEmulators fail on the environment other than Linux or Windows Major test Akira Ajisaka Akira Ajisaka
YARN-6117 SharedCacheManager does not start up Major . Chris Trezzo Chris Trezzo
YARN-6082 Invalid REST api response for getApps since queueUsagePercentage is coming as INF Critical . Sunil Govindan Sunil Govindan
HDFS-11365 Log portnumber in PrivilegedNfsGatewayStarter Minor nfs Mukul Kumar Singh Mukul Kumar Singh
MAPREDUCE-6808 Log map attempts as part of shuffle handler audit log Major . Jonathan Eagles Gergő Pásztor
HDFS-11369 Change exception message in StorageLocationChecker Minor datanode Arpit Agarwal Arpit Agarwal
YARN-4975 Fair Scheduler: exception thrown when a parent queue marked ‘parent’ has configured child queues Major fairscheduler Ashwin Shankar Yufei Gu
HDFS-11364 Add a test to verify Audit log entries for setfacl/getfacl commands over FS shell Major hdfs, test Manoj Govindassamy Manoj Govindassamy
HADOOP-13988 KMSClientProvider does not work with WebHDFS and Apache Knox w/ProxyUser Major common, kms Greg Senia Xiaoyu Yao
HADOOP-14029 Fix KMSClientProvider for non-secure proxyuser use case Major kms Xiaoyu Yao Xiaoyu Yao
YARN-5641 Localizer leaves behind tarballs after container is complete Major . Eric Badger Eric Badger
HDFS-11378 Verify multiple DataNodes can be decommissioned/maintenance at the same time Major hdfs Manoj Govindassamy Manoj Govindassamy
YARN-6103 Log updates for ZKRMStateStore Trivial . Bibin A Chundatt Daniel Sturman
HDFS-11335 Remove HdfsClientConfigKeys.DFS_CLIENT_SLOW_IO_WARNING_THRESHOLD_KEY usage from DNConf Major . Manoj Govindassamy Manoj Govindassamy
HDFS-11363 Need more diagnosis info when seeing Slow waitForAckedSeqno Major . Yongjun Zhang Xiao Chen
HDFS-11387 Socket reuse address option is not honored in PrivilegedNfsGatewayStarter Major nfs Mukul Kumar Singh Mukul Kumar Singh
HADOOP-14044 Synchronization issue in delegation token cancel functionality Major . Hrishikesh Gadre Hrishikesh Gadre
MAPREDUCE-6338 MR AppMaster does not honor ephemeral port range Major mr-am, mrv2 Frank Nguyen Frank Nguyen
HDFS-11377 Balancer hung due to no available mover threads Major balancer & mover yunjiong zhao yunjiong zhao
YARN-6145 Improve log message on fail over Major . Jian He Jian He
YARN-6031 Application recovery has failed when node label feature is turned off during RM recovery Minor scheduler Ying Zhang Ying Zhang
YARN-6137 Yarn client implicitly invoke ATS client which accesses HDFS Major . Yesha Vora Li Lu
HADOOP-13433 Race in UGI.reloginFromKeytab Major security Duo Zhang Duo Zhang
YARN-6112 UpdateCallDuration is calculated only when debug logging is enabled Major fairscheduler Yufei Gu Yufei Gu
YARN-6144 FairScheduler: preempted resources can become negative Blocker fairscheduler, resourcemanager Miklos Szegedi Miklos Szegedi
YARN-6118 Add javadoc for Resources.isNone Minor scheduler Karthik Kambatla Andres Perez
YARN-6166 Unnecessary INFO logs in AMRMClientAsyncImpl$CallbackHandlerThread.run Trivial . Grant W Grant W
HADOOP-14055 SwiftRestClient includes pass length in exception if auth fails Minor security Marcell Hegedus Marcell Hegedus
HDFS-11403 Zookeper ACLs on NN HA enabled clusters to be handled consistently Major hdfs Laszlo Puskas Hanisha Koneru
HADOOP-13233 help of stat is confusing Trivial documentation, fs Xiaohe Lan Attila Bukor
HADOOP-14058 Fix NativeS3FileSystemContractBaseTest#testDirWithDifferentMarkersWorks Minor fs/s3, test Akira Ajisaka Yiqun Lin
HDFS-11084 Add a regression test for sticky bit support of OIV ReverseXML processor Major tools Wei-Chiu Chuang Wei-Chiu Chuang
HDFS-11391 Numeric usernames do no work with WebHDFS FS (write access) Major webhdfs Pierre Villard Pierre Villard
YARN-4212 FairScheduler: Can’t create a DRF queue under a FAIR policy queue Major . Arun Suresh Yufei Gu
YARN-6177 Yarn client should exit with an informative error message if an incompatible Jersey library is used at client Major . Weiwei Yang Weiwei Yang
YARN-6171 ConcurrentModificationException on FSAppAttempt.containersToPreempt Major fairscheduler Miklos Szegedi Miklos Szegedi
HDFS-11410 Use the cached instance when edit logging SetAclOp, SetXAttrOp and RemoveXAttrOp Major namenode Xiao Chen Xiao Chen
YARN-6188 Fix OOM issue with decommissioningNodesWatcher in the case of clusters with large number of nodes Major resourcemanager Ajay Jadhav Ajay Jadhav
HDFS-11177 ‘storagepolicies -getStoragePolicy’ command should accept URI based path. Major shell Surendra Singh Lilhore Surendra Singh Lilhore
HDFS-11404 Increase timeout on TestShortCircuitLocalRead.testDeprecatedGetBlockLocalPathInfoRpc Major . Eric Badger Eric Badger
MAPREDUCE-6825 YARNRunner#createApplicationSubmissionContext method is longer than 150 lines Trivial . Chris Trezzo Gergely Novák
YARN-6210 FS: Node reservations can interfere with preemption Major fairscheduler Karthik Kambatla Karthik Kambatla
YARN-6211 Synchronization improvement for moveApplicationAcrossQueues and updateApplicationPriority Major . Bibin A Chundatt Bibin A Chundatt
YARN-6222 TestFairScheduler.testReservationMetrics is flaky Major fairscheduler Yufei Gu Yufei Gu
HADOOP-14114 S3A can no longer handle unencoded + in URIs Minor fs/s3 Sean Mackrory Sean Mackrory
HDFS-10620 StringBuilder created and appended even if logging is disabled Major namenode Staffan Friberg Staffan Friberg
HADOOP-14116 FailoverOnNetworkExceptionRetry does not wait when failover on certain exception Major . Jian He Jian He
HDFS-11433 Document missing usages of OfflineEditsViewer processors Minor documentation, tools Yiqun Lin Yiqun Lin
HDFS-11462 Fix occasional BindException in TestNameNodeMetricsLogger Major test Arpit Agarwal Arpit Agarwal
YARN-6172 FSLeafQueue demand update needs to be atomic Major resourcemanager Varun Saxena Miklos Szegedi
HADOOP-14119 Remove unused imports from GzipCodec.java Minor . Akira Ajisaka Yiqun Lin
MAPREDUCE-6841 Fix dead link in MapReduce tutorial document Minor documentation Akira Ajisaka Victor Nee
YARN-6231 FairSchedulerTestBase helper methods should call scheduler.update to avoid flakiness Major . Arun Suresh Karthik Kambatla
HDFS-11479 Socket re-use address option should be used in SimpleUdpServer Major nfs Mukul Kumar Singh Mukul Kumar Singh
HADOOP-14131 kms.sh creates bogus dir for tomcat logs Minor kms John Zhuge John Zhuge
MAPREDUCE-6852 Job#updateStatus() failed with NPE due to race condition Major . Junping Du Junping Du
MAPREDUCE-6753 Variable in byte printed directly in mapreduce client Major client Nemo Chen Kai Sasaki
HADOOP-6801 io.sort.mb and io.sort.factor were renamed and moved to mapreduce but are still in CommonConfigurationKeysPublic.java and used in SequenceFile.java Minor . Erik Steffl Harsh J
YARN-6263 NMTokenSecretManagerInRM.createAndGetNMToken is not thread safe Major yarn Haibo Chen Haibo Chen
YARN-6218 Fix TestAMRMClient when using FairScheduler Minor . Miklos Szegedi Miklos Szegedi
HDFS-11476 Fix NPE in FsDatasetImpl#checkAndUpdate Major datanode Xiaobing Zhou Xiaobing Zhou
YARN-6271 yarn rmadin -getGroups returns information from standby RM Critical yarn Sumana Sathish Jian He
YARN-6248 user is not removed from UsersManager’s when app is killed with pending container requests. Major . Eric Payne Eric Payne
HADOOP-14026 start-build-env.sh: invalid docker image name Major build Gergő Pásztor Gergő Pásztor
HDFS-11441 Add escaping to error message in KMS web UI Minor security Aaron T. Myers Aaron T. Myers
YARN-5665 Enhance documentation for yarn.resourcemanager.scheduler.class property Trivial documentation Miklos Szegedi Yufei Gu
MAPREDUCE-6855 Specify charset when create String in CredentialsTestJob Minor . Akira Ajisaka Kai Sasaki
HDFS-11508 Fix bind failure in SimpleTCPServer & Portmap where bind fails because socket is in TIME_WAIT state Major nfs Mukul Kumar Singh Mukul Kumar Singh
MAPREDUCE-6839 TestRecovery.testCrashed failed Major test Gergő Pásztor Gergő Pásztor
YARN-6207 Move application across queues should handle delayed event processing Major capacity scheduler Bibin A Chundatt Bibin A Chundatt
YARN-6297 TestAppLogAggregatorImp.verifyFilesUploaded() should check # of filed uploaded with that of files expected Major . Haibo Chen Haibo Chen
YARN-6165 Intra-queue preemption occurs even when preemption is turned off for a specific queue. Major capacity scheduler, scheduler preemption Eric Payne Eric Payne
YARN-6264 AM not launched when a single vcore is available on the cluster Major fairscheduler Yufei Gu Yufei Gu
YARN-6310 OutputStreams in AggregatedLogFormat.LogWriter can be left open upon exceptions Major yarn Haibo Chen Haibo Chen
YARN-6321 TestResources test timeouts are too aggressive Major test Jason Lowe Eric Badger
HDFS-11340 DataNode reconfigure for disks doesn’t remove the failed volumes Major . Manoj Govindassamy Manoj Govindassamy
HDFS-11512 Increase timeout on TestShortCircuitLocalRead#testSkipWithVerifyChecksum Minor . Eric Badger Eric Badger
HDFS-11499 Decommissioning stuck because of failing recovery Major hdfs, namenode Lukas Majercak Lukas Majercak
HDFS-11395 RequestHedgingProxyProvider#RequestHedgingInvocationHandler hides the Exception thrown from NameNode Major ha Nanda kumar Nanda kumar
HDFS-11526 Fix confusing block recovery message Minor datanode Wei-Chiu Chuang Yiqun Lin
HADOOP-14170 FileSystemContractBaseTest is not cleaning up test directory clearly Major fs Mingliang Liu Mingliang Liu
YARN-6328 Fix a spelling mistake in CapacityScheduler Trivial capacity scheduler Jin Yibo Jin Yibo
HDFS-11420 Edit file should not be processed by the same type processor in OfflineEditsViewer Major tools Yiqun Lin Yiqun Lin
YARN-6294 ATS client should better handle Socket closed case Major timelineclient Sumana Sathish Li Lu
YARN-6332 Make RegistrySecurity use short user names for ZK ACLs Major . Billie Rinaldi Billie Rinaldi
YARN-4051 ContainerKillEvent lost when container is still recovering and application finishes Critical nodemanager sandflee sandflee
HDFS-11533 reuseAddress option should be used for child channels in Portmap and SimpleTcpServer Major nfs Mukul Kumar Singh Mukul Kumar Singh
HDFS-10394 move declaration of okhttp version from hdfs-client to hadoop-project POM Minor build Steve Loughran Xiaobing Zhou
HDFS-11516 Admin command line should print message to stderr in failure case Minor . Kai Sasaki Kai Sasaki
YARN-6217 TestLocalCacheDirectoryManager test timeout is too aggressive Major test Jason Lowe Miklos Szegedi
YARN-6353 Clean up OrderingPolicy javadoc Minor resourcemanager Daniel Templeton Daniel Templeton
HADOOP-14059 typo in s3a rename(self, subdir) error message Minor . Steve Loughran Steve Loughran
HDFS-6648 Order of namenodes in ConfiguredFailoverProxyProvider is undefined Major ha, hdfs-client Rafal Wojdyla Íñigo Goiri
HDFS-11132 Allow AccessControlException in contract tests when getFileStatus on subdirectory of existing files Major fs/adl, test Vishwajeet Dusane Vishwajeet Dusane
HADOOP-14204 S3A multipart commit failing, “UnsupportedOperationException at java.util.Collections$UnmodifiableList.sort” Critical fs/s3 Steve Loughran Steve Loughran
HADOOP-14205 No FileSystem for scheme: adl Major fs/adl John Zhuge John Zhuge
HDFS-11561 HttpFS doc errors Trivial documentation, httpfs, test Yuanbo Liu Yuanbo Liu
HADOOP-9631 ViewFs should use underlying FileSystem’s server side defaults Major fs, viewfs Lohit Vijayarenu Erik Krogen
HADOOP-14214 DomainSocketWatcher::add()/delete() should not self interrupt while looping await() Critical hdfs-client Mingliang Liu Mingliang Liu
HADOOP-14195 CredentialProviderFactory$getProviders is not thread-safe Major security Vihang Karajgaonkar Vihang Karajgaonkar
HADOOP-14211 FilterFs and ChRootedFs are too aggressive about enforcing “authorityNeeded” Major viewfs Erik Krogen Erik Krogen
YARN-6360 Prevent FS state dump logger from cramming other log files Major fairscheduler Yufei Gu Yufei Gu
YARN-6334 TestRMFailover#testAutomaticFailover always passes even when it should fail Major . Yufei Gu Yufei Gu
MAPREDUCE-6866 Fix getNumMapTasks() documentation in JobConf Minor documentation Joe Mészáros Joe Mészáros
MAPREDUCE-6868 License check for jdiff output files should be ignored Major build Akira Ajisaka Akira Ajisaka
HDFS-10506 OIV’s ReverseXML processor cannot reconstruct some snapshot details Major tools Colin P. McCabe Akira Ajisaka
HDFS-11486 Client close() should not fail fast if the last block is being decommissioned Major . Wei-Chiu Chuang Wei-Chiu Chuang
YARN-6359 TestRM#testApplicationKillAtAcceptedState fails rarely due to race condition Major test Robert Kanter Robert Kanter
YARN-5368 Memory leak in timeline server Critical timelineserver Wataru Yukawa Jonathan Eagles
YARN-6050 AMs can’t be scheduled on racks or nodes Major . Robert Kanter Robert Kanter
HDFS-11571 Typo in DataStorage exception message Minor datanode Daniel Templeton Anna Budai
YARN-5685 RM configuration allows all failover methods to disabled when automatic failover is enabled Critical resourcemanager Daniel Templeton Daniel Templeton
HADOOP-14247 FileContextMainOperationsBaseTest should clean up test root path Minor fs, test Mingliang Liu Mingliang Liu
YARN-6352 Header injections are possible in application proxy servlet Major resourcemanager, security Naganarasimha G R Naganarasimha G R
MAPREDUCE-6862 Fragments are not handled correctly by resource limit checking Minor . Chris Trezzo Chris Trezzo
MAPREDUCE-6873 MR Job Submission Fails if MR framework application path not on defaultFS Minor mrv2 Erik Krogen Erik Krogen
HADOOP-14256 [S3A DOC] Correct the format for “Seoul” example Minor documentation, fs/s3 Brahma Reddy Battula Brahma Reddy Battula
MAPREDUCE-6850 Shuffle Handler keep-alive connections are closed from the server side Major . Jonathan Eagles Jonathan Eagles
MAPREDUCE-6836 exception thrown when accessing the job configuration web UI Minor webapps Sangjin Lee Haibo Chen
HDFS-11592 Closing a file has a wasteful preconditions in NameNode Major namenode Eric Badger Eric Badger
YARN-6354 LeveldbRMStateStore can parse invalid keys when recovering reservations Major resourcemanager Jason Lowe Jason Lowe
YARN-5703 ReservationAgents are not correctly configured Major capacity scheduler, resourcemanager Sean Po Manikandan R
HADOOP-14268 Fix markdown itemization in hadoop-aws documents Minor documentation, fs/s3 Akira Ajisaka Akira Ajisaka
HADOOP-14272 Azure: WasbRemoteCallHelper should use String equals for comparison. Major fs/azure Santhosh G Nayak Santhosh G Nayak
HADOOP-14273 Azure: NativeAzureFileSystem should respect config for kerberosSupportEnabled flag Major fs/azure Santhosh G Nayak Santhosh G Nayak
YARN-6436 TestSchedulingPolicy#testParseSchedulingPolicy timeout is too low Major test Jason Lowe Eric Badger
YARN-6004 Refactor TestResourceLocalizationService#testDownloadingResourcesOnContainer so that it is less than 150 lines Trivial test Chris Trezzo Chris Trezzo
YARN-6420 RM startup failure due to wrong order in nodelabel editlog Critical . Bibin A Chundatt Bibin A Chundatt
MAPREDUCE-6824 TaskAttemptImpl#createCommonContainerLaunchContext is longer than 150 lines Trivial . Chris Trezzo Chris Trezzo
YARN-6403 Invalid local resource request can raise NPE and make NM exit Major nodemanager Tao Yang Tao Yang
HDFS-11538 Move ClientProtocol HA proxies into hadoop-hdfs-client Blocker hdfs-client Andrew Wang Huafeng Wang
YARN-6437 TestSignalContainer#testSignalRequestDeliveryToNM fails intermittently Major test Jason Lowe Jason Lowe
YARN-6448 Continuous scheduling thread crashes while sorting nodes Major . Yufei Gu Yufei Gu
MAPREDUCE-6846 Fragments specified for libjar paths are not handled correctly Minor . Chris Trezzo Chris Trezzo
HDFS-11131 TestThrottledAsyncChecker#testCancellation is flaky Major test Arpit Agarwal Arpit Agarwal
HADOOP-13996 Fix some release build issues Blocker build Andrew Wang Andrew Wang
HDFS-11362 StorageDirectory should initialize a non-null default StorageDirType Minor hdfs Hanisha Koneru Hanisha Koneru
HDFS-11608 HDFS write crashed with block size greater than 2 GB Critical hdfs-client Xiaobing Zhou Xiaobing Zhou
YARN-6288 Exceptions during aggregated log writes are mishandled Critical log-aggregation Akira Ajisaka Akira Ajisaka
YARN-6368 Decommissioning an NM results in a -1 exit code Minor . Miklos Szegedi Miklos Szegedi
HADOOP-14066 VersionInfo should be marked as public API Critical common Thejas M Nair Akira Ajisaka
YARN-6343 Docker docs MR example is broken Major nodemanager Daniel Templeton Prashant Jha
HADOOP-14293 Initialize FakeTimer with a less trivial value Major test Andrew Wang Andrew Wang
HADOOP-13545 Upgrade HSQLDB to 2.3.4 Minor build Giovanni Matteo Fumarola Giovanni Matteo Fumarola
YARN-6461 TestRMAdminCLI has very low test timeouts Major test Jason Lowe Eric Badger
YARN-6463 correct spelling mistake in FileSystemRMStateStore Trivial . Yeliang Cang Yeliang Cang
YARN-6439 Fix ReservationSystem creation of default ReservationQueue Major . Carlo Curino Carlo Curino
HDFS-11630 TestThrottledAsyncCheckerTimeout fails intermittently in Jenkins builds Major hdfs Hanisha Koneru Hanisha Koneru
HDFS-11163 Mover should move the file blocks to default storage once policy is unset Major balancer & mover Surendra Singh Lilhore Surendra Singh Lilhore
YARN-6450 TestContainerManagerWithLCE requires override for each new test added to ContainerManagerTest Major test Jason Lowe Jason Lowe
YARN-3760 FSDataOutputStream leak in AggregatedLogFormat.LogWriter.close() Critical nodemanager Daryn Sharp Haibo Chen
YARN-6216 Unify Container Resizing code paths with Container Updates making it scheduler agnostic Major capacity scheduler, fairscheduler, resourcemanager Arun Suresh Arun Suresh
YARN-5994 TestCapacityScheduler.testAMLimitUsage fails intermittently Major . Eric Badger Eric Badger
YARN-6433 Only accessible cgroup mount directories should be selected for a controller Major nodemanager Miklos Szegedi Miklos Szegedi
YARN-6480 Timeout is too aggressive for TestAMRestart.testPreemptedAMRestartOnRMRestart Major . Eric Badger Eric Badger
HADOOP-14311 Add python2.7-dev to Dockerfile Major . Allen Wittenauer Allen Wittenauer
YARN-6304 Skip rm.transitionToActive call to RM if RM is already active. Major resourcemanager Rohith Sharma K S Rohith Sharma K S
HDFS-11615 FSNamesystemLock metrics can be inaccurate due to millisecond precision Major hdfs Erik Krogen Erik Krogen
HADOOP-14318 Remove non-existent setfattr command option from FileSystemShell.md Minor documentation Doris Gu Doris Gu
HADOOP-14315 Python example in the rack awareness document doesn’t work due to bad indentation Minor documentation Kengo Seki Kengo Seki
HADOOP-13997 Typo in metrics docs Trivial documentation Daniel Templeton Ana Krasteva
YARN-6302 Fail the node if Linux Container Executor is not configured properly Minor . Miklos Szegedi Miklos Szegedi
HDFS-11660 TestFsDatasetCache#testPageRounder fails intermittently with AssertionError Major test Andrew Wang Andrew Wang
HDFS-11685 TestDistributedFileSystem.java fails to compile Major test John Zhuge John Zhuge
YARN-6501 FSSchedulerNode.java fails to compile with JDK7 Major resourcemanager John Zhuge John Zhuge
YARN-6453 fairscheduler-statedump.log gets generated regardless of service Blocker fairscheduler, scheduler Allen Wittenauer Yufei Gu
YARN-6153 keepContainer does not work when AM retry window is set Major resourcemanager kyungwan nam kyungwan nam
HDFS-11689 New exception thrown by DFSClient#isHDFSEncryptionEnabled broke hacky hive code Major . Yongjun Zhang Yongjun Zhang
YARN-5889 Improve and refactor user-limit calculation in capacity scheduler Major capacity scheduler Sunil Govindan Sunil Govindan
YARN-6500 Do not mount inaccessible cgroups directories in CgroupsLCEResourcesHandler Major nodemanager Miklos Szegedi Miklos Szegedi
HDFS-11691 Add a proper scheme to the datanode links in NN web UI Major . Kihwal Lee Kihwal Lee
HADOOP-14341 Support multi-line value for ssl.server.exclude.cipher.list Major . John Zhuge John Zhuge
YARN-5617 AMs only intended to run one attempt can be run more than once Major resourcemanager Jason Lowe Jason Lowe
YARN-6510 Fix profs stat file warning caused by process names that includes parenthesis Major . Wilfred Spiegelenburg Wilfred Spiegelenburg
HADOOP-14351 Azure: RemoteWasbAuthorizerImpl and RemoteSASKeyGeneratorImpl should not use Kerberos interactive user cache Major fs/azure Santhosh G Nayak Santhosh G Nayak
HADOOP-14346 CryptoOutputStream throws IOException on flush() if stream is closed Major . Pierre Lacave Pierre Lacave
HDFS-11709 StandbyCheckpointer should handle an non-existing legacyOivImageDir gracefully Critical ha, namenode Zhe Zhang Erik Krogen
YARN-5894 fixed license warning caused by de.ruedigermoeller:fst:jar:2.24 Blocker yarn Haibo Chen Haibo Chen
HADOOP-14320 TestIPC.testIpcWithReaderQueuing fails intermittently Major . Eric Badger Eric Badger
YARN-6536 TestAMRMClient.testAMRMClientWithSaslEncryption fails intermittently Major . Eric Badger Jason Lowe
YARN-6520 Fix warnings from Spotbugs in hadoop-yarn-client Major . Weiwei Yang Weiwei Yang
HDFS-11609 Some blocks can be permanently lost if nodes are decommissioned while dead Blocker namenode Kihwal Lee Kihwal Lee
HDFS-8498 Blocks can be committed with wrong size Critical hdfs-client Daryn Sharp Jing Zhao
HDFS-11714 Newly added NN storage directory won’t get initialized and cause space exhaustion Critical . Kihwal Lee Kihwal Lee
HDFS-11593 Change SimpleHttpProxyHandler#exceptionCaught log level from info to debug Minor datanode Xiaoyu Yao Xiaobing Zhou
HADOOP-14371 License error in TestLoadBalancingKMSClientProvider.java Major . hu xiaodong hu xiaodong
HADOOP-14369 NetworkTopology calls expensive toString() when logging Major . Íñigo Goiri Íñigo Goiri
YARN-6519 Fix warnings from Spotbugs in hadoop-yarn-server-resourcemanager Major resourcemanager Weiwei Yang Weiwei Yang
YARN-6481 Yarn top shows negative container number in FS Major yarn Yufei Gu Tao Jie
HADOOP-14306 TestLocalFileSystem tests have very low timeouts Major . Eric Badger Eric Badger
HADOOP-14372 TestSymlinkLocalFS timeouts are too low Major . Eric Badger Eric Badger
HDFS-11739 Fix regression in tests caused by YARN-679 Major test Steve Loughran Steve Loughran
HADOOP-14207 “dfsadmin -refreshCallQueue” fails with DecayRpcScheduler Blocker rpc-server Surendra Singh Lilhore Surendra Singh Lilhore
HDFS-11702 Remove indefinite caching of key provider uri in DFSClient Major hdfs-client Rushabh S Shah Rushabh S Shah
YARN-3839 Quit throwing NMNotYetReadyException Major nodemanager Karthik Kambatla Manikandan R
HADOOP-14374 License error in GridmixTestUtils.java Major . lixinglong lixinglong
HADOOP-14100 Upgrade Jsch jar to latest version to fix vulnerability in old versions Critical . Vinayakumar B Vinayakumar B
YARN-5301 NM mount cpu cgroups failed on some systems Major . sandflee Miklos Szegedi
HADOOP-14377 Increase Common test timeouts from 1 second to 10 seconds Major . Eric Badger Eric Badger
YARN-3742 YARN RM will shut down if ZKClient creation times out Major resourcemanager Wilfred Spiegelenburg Daniel Templeton
HADOOP-14373 License error In org.apache.hadoop.metrics2.util.Servers Major . hu xiaodong hu xiaodong
YARN-6552 Increase YARN test timeouts from 1 second to 10 seconds Major . Eric Badger Eric Badger
MAPREDUCE-6882 Increase MapReduce test timeouts from 1 second to 10 seconds Major . Eric Badger Eric Badger
HADOOP-14405 Fix performance regression due to incorrect use of DataChecksum Major native, performance LiXin Ge LiXin Ge
HDFS-11745 Increase HDFS test timeouts from 1 second to 10 seconds Major . Eric Badger Eric Badger
HDFS-11755 Underconstruction blocks can be considered missing Major . Nathan Roberts Nathan Roberts
YARN-6571 Fix JavaDoc issues in SchedulingPolicy Trivial fairscheduler Daniel Templeton Weiwei Yang
HADOOP-14361 Azure: NativeAzureFileSystem.getDelegationToken() call fails sometimes when invoked concurrently Major fs/azure Trupti Dhavle Santhosh G Nayak
HADOOP-14410 Correct spelling of ‘beginning’ and variants Trivial . Dongtao Zhang Dongtao Zhang
YARN-5543 ResourceManager SchedulingMonitor could potentially terminate the preemption checker thread Major capacityscheduler, resourcemanager Min Shen Min Shen
YARN-6380 FSAppAttempt keeps redundant copy of the queue Major fairscheduler Daniel Templeton Daniel Templeton
HDFS-11674 reserveSpaceForReplicas is not released if append request failed due to mirror down and replica recovered Critical datanode Vinayakumar B Vinayakumar B
YARN-6580 Incorrect logger for FairSharePolicy Minor fairscheduler Yufei Gu Vrushali C
HADOOP-14376 Memory leak when reading a compressed file using the native library Major common, io Eli Acherkan Eli Acherkan
HDFS-11818 TestBlockManager.testSufficientlyReplBlocksUsesNewRack fails intermittently Major . Eric Badger Nathan Roberts
HDFS-11644 Support for querying outputstream capabilities Major erasure-coding Andrew Wang Manoj Govindassamy
YARN-6598 History server getApplicationReport NPE when fetching report for pre-2.8 job Blocker timelineserver Jason Lowe Jason Lowe
YARN-6603 NPE in RMAppsBlock Major resourcemanager Jason Lowe Jason Lowe
HDFS-11833 HDFS architecture documentation describes outdated placement policy Minor . dud Chen Liang
HADOOP-14416 Path starting with ‘wasb:///’ not resolved correctly while authorizing with WASB-Ranger Major fs/azure, security Sivaguru Sankaridurg Sivaguru Sankaridurg
HDFS-11827 NPE is thrown when log level changed in BlockPlacementPolicyDefault#chooseRandom() method Major . xupeng xupeng
HADOOP-14412 HostsFileReader#getHostDetails is very expensive on large clusters Major util Jason Lowe Jason Lowe
HADOOP-14427 Avoid reloading of Configuration in ViewFileSystem creation. Major viewfs Vinayakumar B Vinayakumar B
HDFS-11842 TestDataNodeOutlierDetectionViaMetrics UT fails Major . Yesha Vora Hanisha Koneru
YARN-6577 Remove unused ContainerLocalization classes Minor nodemanager ZhangBing Lin ZhangBing Lin
YARN-6618 TestNMLeveldbStateStoreService#testCompactionCycle can fail if compaction occurs more than once Minor test Jason Lowe Jason Lowe
YARN-6249 TestFairSchedulerPreemption fails inconsistently. Major fairscheduler, resourcemanager Sean Po Tao Jie
YARN-6602 Impersonation does not work if standby RM is contacted first Blocker client Robert Kanter Robert Kanter
HDFS-11863 Document missing metrics for blocks count in pending IBR Minor documentation Yiqun Lin Yiqun Lin
HDFS-11849 JournalNode startup failure exception should be logged in log file Major journal-node Surendra Singh Lilhore Surendra Singh Lilhore
YARN-6584 Correct license headers in hadoop-common, hdfs, yarn and mapreduce Trivial . Yeliang Cang Yeliang Cang
HDFS-11864 Document Metrics to track usage of memory for writes Major documentation Brahma Reddy Battula Yiqun Lin
YARN-6615 AmIpFilter drops query parameters on redirect Major . Wilfred Spiegelenburg Wilfred Spiegelenburg
HADOOP-14449 The ASF Header in ComparableVersion.java and SSLHostnameVerifier.java is not correct Minor common, documentation ZhangBing Lin ZhangBing Lin
HADOOP-14166 Reset the DecayRpcScheduler AvgResponseTime metric to zero when queue is not used Major common Surendra Singh Lilhore Surendra Singh Lilhore
HDFS-11661 GetContentSummary uses excessive amounts of memory Blocker namenode Nathan Roberts Wei-Chiu Chuang
YARN-6141 ppc64le on Linux doesn’t trigger __linux get_executable codepath Major nodemanager Sonia Garudi Ayappan
HADOOP-14399 Configuration does not correctly XInclude absolute file URIs Blocker conf Andrew Wang Jonathan Eagles
HADOOP-14430 the accessTime of FileStatus returned by SFTPFileSystem’s getFileStatus method is always 0 Trivial fs Hongyuan Li Hongyuan Li
HDFS-11445 FSCK shows overall health status as corrupt even one replica is corrupt Critical . Brahma Reddy Battula Brahma Reddy Battula
YARN-6643 TestRMFailover fails rarely due to port conflict Major test Robert Kanter Robert Kanter
HDFS-11817 A faulty node can cause a lease leak and NPE on accessing data Critical . Kihwal Lee Kihwal Lee
YARN-6641 Non-public resource localization on a bad disk causes subsequent containers failure Major . Kuhu Shukla Kuhu Shukla
HADOOP-14431 ModifyTime of FileStatus returned by SFTPFileSystem’s getFileStatus method is wrong Major fs Hongyuan Li Hongyuan Li
HDFS-11078 Fix NPE in LazyPersistFileScrubber Major . Íñigo Goiri Íñigo Goiri
HADOOP-14464 hadoop-aws doc header warning #5 line wrapped Trivial documentation, fs/s3 John Zhuge John Zhuge
HDFS-11659 TestDataNodeHotSwapVolumes.testRemoveVolumeBeingWritten fail due to no DataNode available for pipeline recovery. Major datanode Lei (Eddy) Xu Lei (Eddy) Xu
YARN-6366 Refactor the NodeManager DeletionService to support additional DeletionTask types. Major nodemanager, yarn Shane Kumpf Shane Kumpf
HDFS-5042 Completed files lost after power failure Critical . Dave Latham Vinayakumar B
YARN-6649 RollingLevelDBTimelineServer throws RuntimeException if object decoding ever fails runtime exception Critical . Jonathan Eagles Jonathan Eagles
HADOOP-9849 License information is missing for native CRC32 code Critical . Timothy St. Clair Andrew Wang
HDFS-11893 Fix TestDFSShell.testMoveWithTargetPortEmpty failure. Major test Konstantin Shvachko Brahma Reddy Battula
HADOOP-14460 Azure: update doc for live and contract tests Major documentation, fs/azure Mingliang Liu Mingliang Liu
HDFS-11741 Long running balancer may fail due to expired DataEncryptionKey Major balancer & mover Wei-Chiu Chuang Wei-Chiu Chuang
HDFS-11856 Ability to re-add Upgrading Nodes (remote) to pipeline for future pipeline updates Major hdfs-client, rolling upgrades Vinayakumar B Vinayakumar B
HDFS-11905 Fix license header inconsistency in hdfs Trivial . Yeliang Cang Yeliang Cang
HADOOP-14436 Remove the redundant colon in ViewFs.md Major documentation maobaolong maobaolong
HADOOP-14474 Use OpenJDK 7 instead of Oracle JDK 7 to avoid oracle-java7-installer failures Major build Akira Ajisaka Akira Ajisaka
HDFS-11899 ASF License warnings generated intermittently in trunk Major . Yiqun Lin Yiqun Lin
HADOOP-14428 s3a: mkdir appears to be broken Blocker fs/s3 Aaron Fabbri Mingliang Liu
HDFS-11928 Segment overflow in FileDistributionCalculator Major tools LiXin Ge LiXin Ge
HDFS-10816 TestComputeInvalidateWork#testDatanodeReRegistration fails due to race between test and replication monitor Major . Eric Badger Eric Badger
HADOOP-14478 Optimize NativeAzureFsInputStream for positional reads Major fs/azure Rajesh Balamohan Rajesh Balamohan
HADOOP-14472 Azure: TestReadAndSeekPageBlobAfterWrite fails intermittently Major fs/azure, test Mingliang Liu Mingliang Liu
HDFS-11932 BPServiceActor thread name is not correctly set Major hdfs Chen Liang Chen Liang
HDFS-11708 Positional read will fail if replicas moved to different DNs after stream is opened Critical hdfs-client Vinayakumar B Vinayakumar B
HDFS-11929 Document missing processor of hdfs oiv_legacy command Minor documentation, tools LiXin Ge LiXin Ge
HDFS-11711 DN should not delete the block On “Too many open files” Exception Critical datanode Brahma Reddy Battula Brahma Reddy Battula
MAPREDUCE-6676 NNBench should Throw IOException when rename and delete fails Major . Brahma Reddy Battula Brahma Reddy Battula
HADOOP-14500 Azure: TestFileSystemOperationExceptionHandling{,MultiThreaded} fails Major fs/azure, test Mingliang Liu Rajesh Balamohan
HADOOP-14283 Upgrade AWS SDK to 1.11.134 Critical fs/s3 Aaron Fabbri Aaron Fabbri
HADOOP-14511 WritableRpcEngine.Invocation#toString NPE on null parameters Minor ipc John Zhuge John Zhuge
YARN-6585 RM fails to start when upgrading from 2.7 to 2.8 for clusters with node labels. Blocker . Eric Payne Sunil Govindan
YARN-6703 RM startup failure with old state store due to version mismatch Critical . Bibin A Chundatt Varun Saxena
HADOOP-14501 Switch from aalto-xml to woodstox to handle odd XML features Blocker conf Andrew Wang Jonathan Eagles
HDFS-11967 TestJMXGet fails occasionally Major . Arpit Agarwal Arpit Agarwal
HDFS-11947 When constructing a thread name, BPOfferService may print a bogus warning message Minor datanode Tsz Wo Nicholas Sze Weiwei Yang
MAPREDUCE-6895 Job end notification not send due to YarnRuntimeException Major applicationmaster yunjiong zhao yunjiong zhao
HADOOP-14486 TestSFTPFileSystem#testGetAccessTime test failure using openJDK 1.8.0 Major fs Sonia Garudi Hongyuan Li
MAPREDUCE-6897 Add Unit Test to make sure Job end notification get sent even appMaster stop get YarnRuntimeException Minor . Junping Du Gergely Novák
YARN-6517 Fix warnings from Spotbugs in hadoop-yarn-common Major . Weiwei Yang Weiwei Yang
YARN-6719 Fix findbugs warnings in SLSCapacityScheduler.java Major . Akira Ajisaka Akira Ajisaka
HADOOP-14540 Replace MRv1 specific terms in HostsFileReader Minor documentation Akira Ajisaka hu xiaodong
HDFS-11995 HDFS Architecture documentation incorrectly describes writing to a local temporary file. Minor documentation Chris Nauroth Nanda kumar
HDFS-11736 OIV tests should not write outside ‘target’ directory. Major . Konstantin Shvachko Yiqun Lin
YARN-6713 Fix dead link in the Javadoc of FairSchedulerEventLog.java Minor documentation Akira Ajisaka Weiwei Yang
HADOOP-14533 Size of args cannot be less than zero in TraceAdmin#run as its linkedlist Trivial common, tracing Weisen Han Weisen Han
HDFS-11960 Successfully closed files can stay under-replicated. Critical . Kihwal Lee Kihwal Lee
HADOOP-14429 FTPFileSystem#getFsAction always returns FsAction.NONE Major fs Hongyuan Li Hongyuan Li
HDFS-12010 TestCopyPreserveFlag fails consistently because of mismatch in access time Major hdfs, test Mukul Kumar Singh Mukul Kumar Singh
HADOOP-14568 GenericTestUtils#waitFor missing parameter verification Major test Yiqun Lin Yiqun Lin
HADOOP-14146 KerberosAuthenticationHandler should authenticate with SPN in AP-REQ Major security Daryn Sharp Daryn Sharp
YARN-5876 TestResourceTrackerService#testGracefulDecommissionWithApp fails intermittently on trunk Major . Varun Saxena Robert Kanter
HADOOP-14543 ZKFC should use getAversion() while setting the zkacl Major . Brahma Reddy Battula Brahma Reddy Battula
YARN-5006 ResourceManager quit due to ApplicationStateData exceed the limit size of znode in zk Critical resourcemanager dongtingting Bibin A Chundatt
HADOOP-14461 Azure: handle failure gracefully in case of missing account access key Major fs/azure Mingliang Liu Mingliang Liu
HDFS-12040 TestFsDatasetImpl.testCleanShutdownOfVolume fails Major test Akira Ajisaka hu xiaodong
HADOOP-14594 ITestS3AFileOperationCost::testFakeDirectoryDeletion to uncomment metric assertions Major fs/s3 Mingliang Liu Mingliang Liu
YARN-6467 CSQueueMetrics needs to update the current metrics for default partition only Major capacity scheduler Naganarasimha G R Manikandan R
HADOOP-14024 KMS JMX endpoint throws ClassNotFoundException Critical kms Andrew Wang John Zhuge
HDFS-12043 Add counters for block re-replication Major . Chen Liang Chen Liang
YARN-6344 Add parameter for rack locality delay in CapacityScheduler Major capacityscheduler Konstantinos Karanasos Konstantinos Karanasos
MAPREDUCE-6905 Fix meaningless operations in TestDFSIO in some situation. Major tools/rumen LiXin Ge LiXin Ge
HDFS-12079 Description of dfs.block.invalidate.limit is incorrect in hdfs-default.xml Minor documentation Weiwei Yang Weiwei Yang
MAPREDUCE-6909 LocalJobRunner fails when run on a node from multiple users Blocker client Jason Lowe Jason Lowe
HADOOP-13414 Hide Jetty Server version header in HTTP responses Major security Vinayakumar B Surendra Singh Lilhore
MAPREDUCE-6911 TestMapreduceConfigFields.testCompareXmlAgainstConfigurationClass fails consistently Major . Eric Badger Eric Badger
YARN-6708 Nodemanager container crash after ext3 folder limit Critical . Bibin A Chundatt Bibin A Chundatt
HADOOP-14563 LoadBalancingKMSClientProvider#warmUpEncryptedKeys swallows IOException Major . Rushabh S Shah Rushabh S Shah
YARN-6428 Queue AM limit is not honored in CS always Major . Bibin A Chundatt Bibin A Chundatt
YARN-6770 [Docs] A small mistake in the example of TimelineClient Trivial docs Jinjiang Ling Jinjiang Ling
HADOOP-10829 Iteration on CredentialProviderFactory.serviceLoader is thread-unsafe Major security Benoy Antony Benoy Antony
HADOOP-14581 Restrict setOwner to list of user when security is enabled in wasb Major fs/azure Varada Hemeswari Varada Hemeswari
YARN-6809 Fix typo in ResourceManagerHA.md Trivial documentation Akira Ajisaka Yeliang Cang
YARN-6797 TimelineWriter does not fully consume the POST response Major timelineclient Jason Lowe Jason Lowe
HDFS-11502 Datanode UI should display hostname based on JMX bean instead of window.location.hostname Major hdfs Jeffrey E Rodriguez Jeffrey E Rodriguez
HADOOP-14646 FileContextMainOperationsBaseTest#testListStatusFilterWithSomeMatches never runs Minor test Andras Bokor Andras Bokor
HADOOP-14658 branch-2 compilation is broken in hadoop-azure Blocker build, fs/azure Sunil Govindan Sunil Govindan
MAPREDUCE-6697 Concurrent task limits should only be applied when necessary Major mrv2 Jason Lowe Nathan Roberts
YARN-6654 RollingLevelDBTimelineStore backwards incompatible after fst upgrade Blocker . Jonathan Eagles Jonathan Eagles
YARN-6805 NPE in LinuxContainerExecutor due to null PrivilegedOperationException exit code Major nodemanager Jason Lowe Jason Lowe
YARN-6792 Incorrect XML convertion in NodeIDsInfo and LabelsToNodesInfo Blocker resourcemanager Giovanni Matteo Fumarola Giovanni Matteo Fumarola
YARN-6769 Make schedulables without demand less needy in FairSharePolicy#compare Major fairscheduler YunFan Zhou YunFan Zhou
YARN-6759 Fix TestRMRestart.testRMRestartWaitForPreviousAMToFinish failure Major . Naganarasimha G R Naganarasimha G R
YARN-3260 AM attempt fail to register before RM processes launch event Critical resourcemanager Jason Lowe Bibin A Chundatt
HDFS-12140 Remove BPOfferService lock contention to get block pool id Critical datanode Daryn Sharp Daryn Sharp
HDFS-12112 TestBlockManager#testBlockManagerMachinesArray sometimes fails with NPE Minor . Wei-Chiu Chuang Wei-Chiu Chuang
YARN-6714 IllegalStateException while handling APP_ATTEMPT_REMOVED event when async-scheduling enabled in CapacityScheduler Major . Tao Yang Tao Yang
MAPREDUCE-6889 Add Job#close API to shutdown MR client services. Major . Rohith Sharma K S Rohith Sharma K S
MAPREDUCE-6910 MapReduceTrackingUriPlugin can not return the right URI of history server with HTTPS Major jobhistoryserver Lantao Jin Lantao Jin
HDFS-12154 Incorrect javadoc description in StorageLocationChecker#check Major . Nanda kumar Nanda kumar
YARN-6798 Fix NM startup failure with old state store due to version mismatch Major nodemanager Ray Chiang Botong Huang
HADOOP-14637 GenericTestUtils.waitFor needs to check condition again after max wait time Major . Daniel Templeton Daniel Templeton
YARN-6819 Application report fails if app rejected due to nodesize Major . Bibin A Chundatt Bibin A Chundatt
HADOOP-14669 GenericTestUtils.waitFor should use monotonic time Trivial test Jason Lowe Daniel Templeton
HDFS-12158 Secondary Namenode’s web interface lack configs for X-FRAME-OPTIONS protection Major namenode Mukul Kumar Singh Mukul Kumar Singh
HDFS-12139 HTTPFS liststatus returns incorrect pathSuffix for path of file Major httpfs Yongjun Zhang Yongjun Zhang
YARN-6837 Null LocalResource visibility or resource type can crash the nodemanager Major . Jinjiang Ling Jinjiang Ling
HDFS-11472 Fix inconsistent replica size after a data pipeline failure Critical datanode Wei-Chiu Chuang Erik Krogen
HDFS-11742 Improve balancer usability after HDFS-8818 Blocker . Kihwal Lee Kihwal Lee
HDFS-12176 dfsadmin shows DFS Used%: NaN% if the cluster has zero block. Trivial . Wei-Chiu Chuang Weiwei Yang
YARN-6844 AMRMClientImpl.checkNodeLabelExpression() has wrong error message Minor . Daniel Templeton Manikandan R
YARN-6150 TestContainerManagerSecurity tests for Yarn Server are flakey Major test Daniel Sturman Daniel Sturman
YARN-6307 Refactor FairShareComparator#compare Major fairscheduler Yufei Gu Yufei Gu
HADOOP-14578 Bind IPC connections to kerberos UPN host for proxy users Major ipc Daryn Sharp Daryn Sharp
HDFS-11896 Non-dfsUsed will be doubled on dead node re-registration Blocker . Brahma Reddy Battula Brahma Reddy Battula
YARN-6870 Fix floating point inaccuracies in resource availability check in AllocationBasedResourceUtilizationTracker Major api, nodemanager Brook Zhou Brook Zhou
YARN-5728 TestMiniYarnClusterNodeUtilization.testUpdateNodeUtilization timeout Major test Akira Ajisaka Akira Ajisaka
HADOOP-14677 mvn clean compile fails Major build Andras Bokor Andras Bokor
HDFS-12082 BlockInvalidateLimit value is incorrectly set after namenode heartbeat interval reconfigured Major hdfs, namenode Weiwei Yang Weiwei Yang
YARN-6628 Unexpected jackson-core-2.2.3 dependency introduced Blocker timelineserver Jason Lowe Jonathan Eagles
HADOOP-14644 Increase max heap size of Maven javadoc plugin Major test Andras Bokor Andras Bokor
MAPREDUCE-6921 TestUmbilicalProtocolWithJobToken#testJobTokenRpc fails Major . Sonia Garudi Sonia Garudi
HADOOP-14676 Wrong default value for “fs.df.interval” Major common, conf, fs Konstantin Shvachko Sherwood Zheng
HADOOP-14701 Configuration can log misleading warnings about an attempt to override final parameter Major conf Andrew Sherman Andrew Sherman
YARN-5731 Preemption calculation is not accurate when reserved containers are present in queue. Major capacity scheduler Sunil Govindan Wangda Tan
HADOOP-14683 FileStatus.compareTo binary compatible issue Blocker . Sergey Shelukhin Akira Ajisaka
HADOOP-14702 Fix formatting issue and regression caused by conversion from APT to Markdown Minor documentation Doris Gu Doris Gu
YARN-6872 Ensure apps could run given NodeLabels are disabled post RM switchover/restart Major resourcemanager Sunil Govindan Sunil Govindan
HDFS-12217 HDFS snapshots doesn’t capture all open files when one of the open files is deleted Major snapshots Manoj Govindassamy Manoj Govindassamy
YARN-6846 Nodemanager can fail to fully delete application local directories when applications are killed Critical nodemanager Jason Lowe Jason Lowe
YARN-6678 Handle IllegalStateException in Async Scheduling mode of CapacityScheduler Major capacityscheduler Tao Yang Tao Yang
YARN-6895 [FairScheduler] Preemption reservation may cause regular reservation leaks Blocker fairscheduler Miklos Szegedi Miklos Szegedi
HADOOP-14722 Azure: BlockBlobInputStream position incorrect after seek Major fs/azure Thomas Marquardt Thomas Marquardt
YARN-6951 Fix debug log when Resource Handler chain is enabled Minor . Yang Wang Yang Wang
HADOOP-14727 Socket not closed properly when reading Configurations with BlockReaderRemote Blocker conf Xiao Chen Jonathan Eagles
YARN-6920 Fix resource leak that happens during container re-initialization. Major nodemanager Arun Suresh Arun Suresh
HDFS-12198 Document missing namenode metrics that were added recently Minor documentation Yiqun Lin Yiqun Lin
HADOOP-14680 Azure: IndexOutOfBoundsException in BlockBlobInputStream Minor fs/azure Rajesh Balamohan Thomas Marquardt
MAPREDUCE-6927 MR job should only set tracking url if history was successfully written Major . Eric Badger Eric Badger
HDFS-10326 Disable setting tcp socket send/receive buffers for write pipelines Major datanode, hdfs Daryn Sharp Daryn Sharp
HADOOP-14598 Blacklist Http/HttpsFileSystem in FsUrlStreamHandlerFactory Major fs/azure, test Steve Loughran Steve Loughran
HDFS-12157 Do fsyncDirectory(..) outside of FSDataset lock Critical datanode Vinayakumar B Vinayakumar B
HDFS-12278 LeaseManager operations are inefficient in 2.8. Blocker namenode Rushabh S Shah Rushabh S Shah
HADOOP-14743 CompositeGroupsMapping should not swallow exceptions Major security Wei-Chiu Chuang Wei-Chiu Chuang
HADOOP-14754 TestCommonConfigurationFields failed: core-default.xml has 2 wasb properties missing in classes Minor common, fs/azure John Zhuge John Zhuge
YARN-5927 BaseContainerManagerTest::waitForNMContainerState timeout accounting is not accurate Trivial . Miklos Szegedi Kai Sasaki
YARN-6967 Limit application attempt’s diagnostic message size thoroughly Major resourcemanager Chengbing Liu Chengbing Liu
HDFS-11303 Hedged read might hang infinitely if read data from all DN failed Major hdfs-client Chen Zhang Chen Zhang
YARN-6996 Change javax.cache library implementation from JSR107 to Apache Geronimo Blocker . Ray Chiang Ray Chiang
YARN-6987 Log app attempt during InvalidStateTransition Major . Jonathan Eagles Jonathan Eagles
HADOOP-14773 Extend ZKCuratorManager API for more reusability Major . Íñigo Goiri Íñigo Goiri
YARN-6965 Duplicate instantiation in FairSchedulerQueueInfo Minor fairscheduler Masahiro Tanaka Masahiro Tanaka
YARN-7020 TestAMRMProxy#testAMRMProxyTokenRenewal is flakey Major . Robert Kanter Robert Kanter
MAPREDUCE-6940 Copy-paste error in the TaskAttemptUnsuccessfulCompletionEvent constructor Minor . Oleg Danilov Oleg Danilov
MAPREDUCE-6936 Remove unnecessary dependency of hadoop-yarn-server-common from hadoop-mapreduce-client-common Major mrv2 Haibo Chen Haibo Chen
YARN-7007 NPE in RM while using YarnClient.getApplications() Major . Lingfeng Su Lingfeng Su
HDFS-12325 SFTPFileSystem operations should restore cwd Major . Namit Maheshwari Chen Liang
HDFS-11738 Hedged pread takes more time when block moved from initial locations Major hdfs-client Vinayakumar B Vinayakumar B
YARN-2416 InvalidStateTransitonException in ResourceManager if AMLauncher does not receive response for startContainers() call in time Critical resourcemanager Jian Fang Jonathan Eagles
YARN-7048 Fix tests faking kerberos to explicitly set ugi auth type Major yarn Daryn Sharp Daryn Sharp
HADOOP-14687 AuthenticatedURL will reuse bad/expired session cookies Critical common Daryn Sharp Daryn Sharp
YARN-6251 Do async container release to prevent deadlock during container updates Major . Arun Suresh Arun Suresh
YARN-7090 testRMRestartAfterNodeLabelDisabled get failed when CapacityScheduler is configured Major test Yesha Vora Wangda Tan
YARN-7074 Fix NM state store update comment Minor nodemanager Botong Huang Botong Huang
YARN-6640 AM heartbeat stuck when responseId overflows MAX_INT Blocker . Botong Huang Botong Huang
HDFS-12319 DirectoryScanner will throw IllegalStateException when Multiple BP’s are present Blocker . Brahma Reddy Battula Brahma Reddy Battula
HDFS-12299 Race Between update pipeline and DN Re-Registration Critical . Brahma Reddy Battula Brahma Reddy Battula
YARN-7052 RM SchedulingMonitor gives no indication why the spawned thread crashed. Critical yarn Eric Payne Eric Payne
YARN-7087 NM failed to perform log aggregation due to absent container Blocker log-aggregation Jason Lowe Jason Lowe
HDFS-12215 DataNode#transferBlock does not create its daemon in the xceiver thread group Major datanode Lei (Eddy) Xu Lei (Eddy) Xu
YARN-7051 Avoid concurrent modification exception in FifoIntraQueuePreemptionPlugin Critical capacity scheduler, scheduler preemption, yarn Eric Payne Eric Payne
YARN-7099 ResourceHandlerModule.parseConfiguredCGroupPath only works for privileged yarn users. Minor nodemanager Miklos Szegedi Miklos Szegedi
YARN-7112 TestAMRMProxy is failing with invalid request Major . Jason Lowe Jason Lowe
HDFS-12293 DataNode should log file name on disk error Major datanode Wei-Chiu Chuang Ajay Kumar
YARN-7076 yarn application -list -appTypes <appType> is not working Blocker . Jian He Jian He
YARN-5816 TestDelegationTokenRenewer#testCancelWithMultipleAppSubmissions is still flakey Minor resourcemanager, test Daniel Templeton Robert Kanter
MAPREDUCE-6641 TestTaskAttempt fails in trunk Major test Tsuyoshi Ozawa Haibo Chen
YARN-6756 ContainerRequest#executionTypeRequest causes NPE Critical . Jian He Jian He
HDFS-12191 Provide option to not capture the accessTime change of a file to snapshot if no other modification has been done to this file Major hdfs, namenode Yongjun Zhang Yongjun Zhang
YARN-6982 Potential issue on setting AMContainerSpec#tokenConf to null before app is completed Major . Rohith Sharma K S Manikandan R
HDFS-12336 Listing encryption zones still fails when deleted EZ is not a direct child of snapshottable directory Minor encryption, hdfs Wellington Chevreuil Wellington Chevreuil
HADOOP-14814 Fix incompatible API change on FsServerDefaults to HADOOP-14104 Blocker . Junping Du Junping Du
MAPREDUCE-6931 Remove TestDFSIO “Total Throughput” calculation Critical benchmarks, test Dennis Huo Dennis Huo
YARN-7077 TestAMSimulator and TestNMSimulator fail Major test Akira Ajisaka Akira Ajisaka
YARN-7116 CapacityScheduler Web UI: Queue’s AM usage is always show on per-user’s AM usage. Major capacity scheduler, webapp Wangda Tan Wangda Tan
HDFS-12317 HDFS metrics render error in the page of Github Minor documentation, metrics Yiqun Lin Yiqun Lin
HADOOP-14824 Update ADLS SDK to 2.2.2 for MSI fix Major fs/adl Atul Sikaria Atul Sikaria
YARN-7120 CapacitySchedulerPage NPE in “Aggregate scheduler counts” section Minor . Eric Payne Eric Payne
YARN-7023 Incorrect ReservationId.compareTo() implementation Minor reservation system Oleg Danilov Oleg Danilov
HADOOP-14820 Wasb mkdirs security checks inconsistent with HDFS Major fs/azure Sivaguru Sankaridurg Sivaguru Sankaridurg
YARN-7164 TestAMRMClientOnRMRestart fails sporadically with bind address in use Major test Jason Lowe Jason Lowe
YARN-6992 Kill application button is visible even if the application is FINISHED in RM UI Major . Sumana Sathish Suma Shivaprasad
HDFS-12357 Let NameNode to bypass external attribute provider for special user Major . Yongjun Zhang Yongjun Zhang
HDFS-12369 Edit log corruption due to hard lease recovery of not-closed file which has snapshots Major namenode Xiao Chen Xiao Chen
HADOOP-14842 Hadoop 2.8.2 release build process get stuck due to java issue Blocker build Junping Du Junping Du
YARN-6219 NM web server related UT fails with “NMWebapps failed to start.” Major . Yesha Vora Jason Lowe
HDFS-12407 Journal nodes fails to shutdown cleanly if JournalNodeHttpServer or JournalNodeRpcServer fails to start Major . Ajay Kumar Ajay Kumar
YARN-7185 ContainerScheduler should only look at availableResource for GUARANTEED containers when OPPORTUNISTIC container queuing is enabled. Blocker yarn Sumana Sathish Tan, Wangda
HADOOP-14867 Update HDFS Federation setup document, for incorrect property name for secondary name node http address Major . Bharat Viswanadham Bharat Viswanadham
YARN-4727 Unable to override the $HADOOP_CONF_DIR env variable for container Major nodemanager Terence Yim Jason Lowe
MAPREDUCE-6957 shuffle hangs after a node manager connection timeout Major mrv2 Jooseong Kim Jooseong Kim
YARN-7163 RMContext need not to be injected to webapp and other Always Running services. Blocker resourcemanager Rohith Sharma K S Rohith Sharma K S
HDFS-12424 Datatable sorting on the Datanode Information page in the Namenode UI is broken Major . Shawna Martell Shawna Martell
HDFS-12323 NameNode terminates after full GC thinking QJM unresponsive if full GC is much longer than timeout Major namenode, qjm Erik Krogen Erik Krogen
YARN-7149 Cross-queue preemption sometimes starves an underserved queue Major capacity scheduler Eric Payne Eric Payne
YARN-7192 Add a pluggable StateMachine Listener that is notified of NM Container State changes Major . Arun Suresh Arun Suresh
MAPREDUCE-6960 Shuffle Handler prints disk error stack traces for every read failure. Major . Kuhu Shukla Kuhu Shukla
HDFS-11799 Introduce a config to allow setting up write pipeline with fewer nodes than replication factor Major . Yongjun Zhang Brahma Reddy Battula
YARN-7196 Fix finicky TestContainerManager tests Major . Arun Suresh Arun Suresh
YARN-6771 Use classloader inside configuration class to make new classes Major . Jongyoul Lee Jongyoul Lee
HDFS-12526 FSDirectory should use Time.monotonicNow for durations Minor . Chetna Chaudhari Bharat Viswanadham
YARN-6968 Hardcoded absolute pathname in DockerLinuxContainerRuntime Major nodemanager Miklos Szegedi Eric Badger
HDFS-12371 “BlockVerificationFailures” and “BlocksVerified” show up as 0 in Datanode JMX Major metrics Sai Nukavarapu Hanisha Koneru
MAPREDUCE-6964 BaileyBorweinPlouffe should use Time.monotonicNow for measuring durations Minor examples Chetna Chaudhari Chetna Chaudhari
YARN-5195 RM intermittently crashed with NPE while handling APP_ATTEMPT_REMOVED event when async-scheduling enabled in CapacityScheduler Major resourcemanager Karam Singh sandflee
YARN-7034 DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client environment variables to container-executor Blocker nodemanager Miklos Szegedi Miklos Szegedi
MAPREDUCE-6966 DistSum should use Time.monotonicNow for measuring durations Minor . Chetna Chaudhari Chetna Chaudhari
MAPREDUCE-6965 QuasiMonteCarlo should use Time.monotonicNow for measuring durations Minor examples Chetna Chaudhari Chetna Chaudhari
MAPREDUCE-6967 gridmix/SleepReducer should use Time.monotonicNow for measuring durations Minor . Chetna Chaudhari Chetna Chaudhari
HDFS-12531 Fix conflict in the javadoc of UnderReplicatedBlocks.java in branch-2 Minor documentation Akira Ajisaka Bharat Viswanadham
YARN-7153 Remove duplicated code in AMRMClientAsyncImpl.java Minor client Sen Zhao Sen Zhao
HADOOP-14901 ReuseObjectMapper in Hadoop Common Minor . Hanisha Koneru Hanisha Koneru
YARN-7118 AHS REST API can return NullPointerException Major . Prabhu Joseph Billie Rinaldi
HDFS-12495 TestPendingInvalidateBlock#testPendingDeleteUnknownBlocks fails intermittently Major . Eric Badger Eric Badger
HADOOP-14891 Remove references to Guava Objects.toStringHelper Major . Jonathan Eagles Jonathan Eagles
HADOOP-14822 hadoop-project/pom.xml is executable Minor . Akira Ajisaka Ajay Kumar
YARN-7157 Add admin configuration to filter per-user’s apps in secure cluster Major webapp Sunil Govindan Sunil Govindan
YARN-7257 AggregatedLogsBlock reports a bad ‘end’ value as a bad ‘start’ value Major log-aggregation Jason Lowe Jason Lowe
YARN-7248 NM returns new SCHEDULED container status to older clients Blocker nodemanager Jason Lowe Arun Suresh
HADOOP-14902 LoadGenerator#genFile write close timing is incorrectly calculated Major fs Jason Lowe Hanisha Koneru
YARN-7260 yarn.router.pipeline.cache-max-size is missing in yarn-default.xml Major . Rohith Sharma K S Jason Lowe
YARN-7084 TestSchedulingMonitor#testRMStarts fails sporadically Major . Jason Lowe Jason Lowe
YARN-6625 yarn application -list returns a tracking URL for AM that doesn’t work in secured and HA environment Major amrmproxy Yufei Gu Yufei Gu
YARN-7044 TestContainerAllocation#testAMContainerAllocationWhenDNSUnavailable fails Major capacity scheduler, test Wangda Tan Akira Ajisaka
YARN-7226 Whitelisted variables do not support delayed variable expansion Major nodemanager Jason Lowe Jason Lowe
YARN-7258 Add Node and Rack Hints to Opportunistic Scheduler Major . Arun Suresh kartheek muthyala
YARN-7285 ContainerExecutor always launches with priorities due to yarn-default property Minor nodemanager Jason Lowe Jason Lowe
YARN-7245 Max AM Resource column in Active Users Info section of Capacity Scheduler UI page should be updated per-user Major capacity scheduler, yarn Eric Payne Eric Payne
YARN-7309 TestClientRMService#testUpdateApplicationPriorityRequest and TestClientRMService#testUpdatePriorityAndKillAppWithZeroClusterResource test functionality not supported by FairScheduler Major test Robert Kanter Robert Kanter
HADOOP-14912 FairCallQueue may defer servicing calls Major ipc Daryn Sharp Daryn Sharp
MAPREDUCE-6951 Improve exception message when mapreduce.jobhistory.webapp.address is in wrong format Major applicationmaster Prabhu Joseph Prabhu Joseph
HADOOP-13556 Change Configuration.getPropsWithPrefix to use getProps instead of iterator Major . Larry McCay Larry McCay
HADOOP-13102 Update GroupsMapping documentation to reflect the new changes Major documentation Anu Engineer Esther Kundin
YARN-7270 Fix unsafe casting from long to int for class Resource and its sub-classes Major resourcemanager Yufei Gu Yufei Gu
YARN-7269 Tracking URL in the app state does not get redirected to ApplicationMaster for Running applications Critical . Sumana Sathish Tan, Wangda
YARN-7321 Backport container-executor changes from YARN-6852 to branch-2 Major . Varun Vasudev Varun Vasudev
HDFS-12659 Update TestDeadDatanode#testNonDFSUsedONDeadNodeReReg to increase heartbeat recheck interval Minor . Ajay Kumar Ajay Kumar
HDFS-12485 expunge may fail to remove trash from encryption zone Major . Wei-Chiu Chuang Wei-Chiu Chuang
HADOOP-14949 TestKMS#testACLs fails intermittently Major kms, test Xiao Chen Xiao Chen
YARN-7124 LogAggregationTFileController deletes/renames while file is open Critical nodemanager Daryn Sharp Jason Lowe
YARN-7333 container-executor fails to remove entries from a directory that is not writable or executable Critical . Jason Lowe Jason Lowe
YARN-7341 TestRouterWebServiceUtil#testMergeMetrics is flakey Major federation Robert Kanter Robert Kanter
HADOOP-14958 CLONE - Fix source-level compatibility after HADOOP-11252 Blocker . Junping Du Junping Du
YARN-7294 TestSignalContainer#testSignalRequestDeliveryToNM fails intermittently with Fair scheduler Major . Miklos Szegedi Miklos Szegedi
YARN-7170 Improve bower dependencies for YARN UI v2 Critical webapp Sunil Govindan Sunil Govindan
YARN-7355 TestDistributedShell should be scheduler agnostic Major . Haibo Chen Haibo Chen
HADOOP-14942 DistCp#cleanup() should check whether jobFS is null Minor . Ted Yu Andras Bokor
HDFS-12683 DFSZKFailOverController re-order logic for logging Exception Major . Bharat Viswanadham Bharat Viswanadham
HADOOP-14966 Handle JDK-8071638 for hadoop-common Blocker . Bibin A Chundatt Bibin A Chundatt
HDFS-12695 Add a link to HDFS router federation document in site.xml Minor documentation Yiqun Lin Yiqun Lin
YARN-7382 NoSuchElementException in FairScheduler after failover causes RM crash Blocker fairscheduler Robert Kanter Robert Kanter
YARN-7385 TestFairScheduler#testUpdateDemand and TestFSLeafQueue#testUpdateDemand are failing with NPE Major test Robert Kanter Yufei Gu
YARN-7339 LocalityMulticastAMRMProxyPolicy should handle cancel request properly Minor . Botong Huang Botong Huang
HDFS-12710 HTTPFS HTTP max header size env variable is not respected in branch-2 Minor . Istvan Fajth Istvan Fajth
HDFS-9914 Fix configurable WebhDFS connect/read timeout Blocker hdfs-client, webhdfs Xiaoyu Yao Xiaoyu Yao
YARN-7375 Possible NPE in RMWebapp when HA is enabled and the active RM fails Major . Chandni Singh Chandni Singh
HADOOP-14986 Enforce JDK limitations Major build Chris Douglas Chris Douglas
HADOOP-14991 Add missing figures to Resource Estimator tool Major . Subru Krishnan Rui Li
YARN-7299 Fix TestDistributedScheduler Major . Jason Lowe Arun Suresh
YARN-6747 TestFSAppStarvation.testPreemptionEnable fails intermittently Major . Sunil Govindan Miklos Szegedi
YARN-7244 ShuffleHandler is not aware of disks that are added Major . Kuhu Shukla Kuhu Shukla
HADOOP-14990 Clean up jdiff xml files added for 2.8.2 release Blocker . Subru Krishnan Junping Du
HADOOP-14919 BZip2 drops records when reading data in splits Critical . Aki Tanaka Jason Lowe
HDFS-12699 TestMountTable fails with Java 7 Major . Íñigo Goiri Íñigo Goiri
YARN-7146 Many RM unit tests failing with FairScheduler Major test Robert Kanter Robert Kanter
MAPREDUCE-6999 Fix typo “onf” in DynamicInputChunk.java Trivial . fang zhenyi fang zhenyi
YARN-7364 Queue dash board in new YARN UI has incorrect values Critical webapp Sunil Govindan Sunil Govindan
YARN-7370 Preemption properties should be refreshable Major capacity scheduler, scheduler preemption Eric Payne Gergely Novák
YARN-7400 incorrect log preview displayed in jobhistory server ui Major yarn Santhosh B Gowda Xuan Gong
HADOOP-15013 Fix ResourceEstimator findbugs issues Blocker . Allen Wittenauer Arun Suresh
YARN-7432 Fix DominantResourceFairnessPolicy serializable findbugs issues Blocker . Allen Wittenauer Daniel Templeton
YARN-7434 Router getApps REST invocation fails with multiple RMs Critical . Subru Krishnan Íñigo Goiri
YARN-4742 [Umbrella] Enhancements to Distributed Scheduling Major . Arun Suresh Arun Suresh
YARN-5085 Add support for change of container ExecutionType Major . Arun Suresh Arun Suresh
HDFS-12783 [branch-2] “dfsrouter” should use hdfsScript Major . Brahma Reddy Battula Brahma Reddy Battula
YARN-7453 Fix issue where RM fails to switch to active after first successful start Blocker resourcemanager Rohith Sharma K S Rohith Sharma K S
HADOOP-15025 Ensure singleton for ResourceEstimatorService Major . Subru Krishnan Rui Li
HADOOP-15026 Rebase ResourceEstimator start/stop scripts for branch-2 Major . Subru Krishnan Rui Li
YARN-6102 RMActiveService context to be updated with new RMContext on failover Critical . Ajith S Rohith Sharma K S
HADOOP-15030 [branch-2] Include hadoop-cloud-storage-project in the main hadoop pom modules Critical . Subru Krishnan Subru Krishnan
YARN-7475 Fix Container log link in new YARN UI Major . Sunil Govindan Sunil Govindan
YARN-7476 Fix miscellaneous issues in ATSv2 after merge to branch-2 Major . Varun Saxena Varun Saxena
HADOOP-15036 Update LICENSE.txt for HADOOP-14840 Major . Arun Suresh Arun Suresh
HDFS-13054 Handling PathIsNotEmptyDirectoryException in DFSClient delete call Major hdfs-client Nanda kumar Nanda kumar

TESTS:

JIRA Summary Priority Component Reporter Contributor
HDFS-9153 Pretty-format the output for DFSIO Major . Kai Zheng Kai Zheng
HDFS-9300 TestDirectoryScanner.testThrottle() is still a little flakey Major balancer & mover, test Daniel Templeton Daniel Templeton
YARN-4704 TestResourceManager#testResourceAllocation() fails when using FairScheduler Major fairscheduler, test Ray Chiang Yufei Gu
HADOOP-12701 Run checkstyle on test source files Minor . John Zhuge John Zhuge
HADOOP-13178 TestShellBasedIdMapping.testStaticMapUpdate doesn’t work on OS X Major test Allen Wittenauer Kai Sasaki
YARN-5024 TestContainerResourceUsage#testUsageAfterAMRestartWithMultipleContainers random failure Major test Bibin A Chundatt Bibin A Chundatt
YARN-5005 TestRMWebServices#testDumpingSchedulerLogs fails randomly Major . Bibin A Chundatt Bibin A Chundatt
YARN-5118 Tests fails with localizer port bind exception. Major test Brahma Reddy Battula Brahma Reddy Battula
YARN-4989 TestWorkPreservingRMRestart#testCapacitySchedulerRecovery fails intermittently Major test Rohith Sharma K S Ajith S
YARN-5208 Run TestAMRMClient TestNMClient TestYarnClient TestClientRMTokens TestAMAuthorization tests with hadoop.security.token.service.use_ip enabled Blocker test Rohith Sharma K S Rohith Sharma K S
YARN-5343 TestContinuousScheduling#testSortedNodes fails intermittently Minor . sandflee Yufei Gu
YARN-2398 TestResourceTrackerOnHA crashes Major test Jason Lowe Ajith S
YARN-5656 Fix ReservationACLsTestBase Major . Sean Po Sean Po
YARN-4555 TestDefaultContainerExecutor#testContainerLaunchError fails on non-english locale environment Minor nodemanager, test Takashi Ohnishi Takashi Ohnishi
YARN-4363 In TestFairScheduler, testcase should not create FairScheduler redundantly Trivial fairscheduler Tao Jie Tao Jie
MAPREDUCE-6831 Flaky test TestJobImpl.testKilledDuringKillAbort Major mrv2 Peter Bacsko Peter Bacsko
HDFS-11290 TestFSNameSystemMBean should wait until JMX cache is cleared Major test Akira Ajisaka Erik Krogen
HDFS-11570 Unit test for NameNodeStatusMXBean Major hdfs, test Hanisha Koneru Hanisha Koneru
HADOOP-14218 Replace assertThat with assertTrue in MetricsAsserts Minor . Akira Ajisaka Akira Ajisaka
YARN-5548 Use MockRMMemoryStateStore to reduce test failures Major . Bibin A Chundatt Bibin A Chundatt
HADOOP-14245 Use Mockito.when instead of Mockito.stub Minor test Akira Ajisaka Andras Bokor
YARN-5349 TestWorkPreservingRMRestart#testUAMRecoveryOnRMWorkPreservingRestart fail intermittently Minor . sandflee Jason Lowe
YARN-6240 TestCapacityScheduler.testRefreshQueuesWithQueueDelete fails randomly Major test Sunil Govindan Naganarasimha G R
HDFS-11988 Verify HDFS Snapshots with open files captured are safe across truncates and appends on current version file Major hdfs, snapshots Manoj Govindassamy Manoj Govindassamy
MAPREDUCE-6953 Skip the testcase testJobWithChangePriority if FairScheduler is used Major client Peter Bacsko Peter Bacsko

SUB-TASKS:

JIRA Summary Priority Component Reporter Contributor
YARN-3063 Bootstrap TimelineServer Next Gen Module Major timelineserver Zhijie Shen Zhijie Shen
YARN-3030 [Collector wireup] Set up timeline collector with basic request serving structure and lifecycle Major timelineserver Sangjin Lee Sangjin Lee
YARN-3041 [Data Model] create overall data objects of TS next gen Major timelineserver Sangjin Lee Zhijie Shen
YARN-3240 [Data Mode] Implement client API to put generic entities Major timelineserver Zhijie Shen Zhijie Shen
YARN-3087 [Collector implementation] the REST server (web server) for per-node collector does not work if it runs inside node manager Major timelineserver Sangjin Lee Li Lu
YARN-3125 [Event producers] Change distributed shell to use new timeline service Major timelineserver Zhijie Shen Junping Du
YARN-3210 [Source organization] Refactor timeline collector according to new code organization Major timelineserver Li Lu Li Lu
YARN-3264 [Storage implementation] Create backing storage write interface and a POC only file based storage implementation Major timelineserver Vrushali C Vrushali C
YARN-3039 [Collector wireup] Implement timeline app-level collector service discovery Major timelineserver Sangjin Lee Junping Du
YARN-3333 rename TimelineAggregator etc. to TimelineCollector Major timelineserver Sangjin Lee Sangjin Lee
YARN-3377 TestTimelineServiceClientIntegration fails Minor timelineserver Sangjin Lee Sangjin Lee
YARN-3034 [Collector wireup] Implement RM starting its timeline collector Major timelineserver Sangjin Lee Naganarasimha G R
YARN-3040 [Data Model] Make putEntities operation be aware of the app’s context Major timelineserver Sangjin Lee Zhijie Shen
YARN-3374 Collector’s web server should randomly bind an available port Major timelineserver Zhijie Shen Zhijie Shen
YARN-3334 [Event Producers] NM TimelineClient container metrics posting to new timeline service. Major nodemanager Junping Du Junping Du
YARN-3391 Clearly define flow ID/ flow run / flow version in API and storage Major timelineserver Zhijie Shen Zhijie Shen
MAPREDUCE-6327 [Event producers] Implement MapReduce AM writing MR events/counters to v2 ATS Major . Sangjin Lee Junping Du
YARN-3390 Reuse TimelineCollectorManager for RM Major timelineserver Zhijie Shen Zhijie Shen
YARN-3431 Sub resources of timeline entity needs to be passed to a separate endpoint. Major timelineserver Zhijie Shen Zhijie Shen
MAPREDUCE-6335 convert load test driver to timeline service v.2 Major . Sangjin Lee Sangjin Lee
YARN-3551 Consolidate data model change according to the backend implementation Major timelineserver Zhijie Shen Zhijie Shen
YARN-3562 unit tests failures and issues found from findbug from earlier ATS checkins Minor timelineserver Sangjin Lee Naganarasimha G R
YARN-3134 [Storage implementation] Exploiting the option of using Phoenix to access HBase backend Major timelineserver Zhijie Shen Li Lu
YARN-3529 Add miniHBase cluster and Phoenix support to ATS v2 unit tests Major timelineserver Li Lu Li Lu
YARN-3634 TestMRTimelineEventHandling and TestApplication are broken Major timelineserver Sangjin Lee Sangjin Lee
MAPREDUCE-6337 add a mode to replay MR job history files to the timeline service Major . Sangjin Lee Sangjin Lee
YARN-3411 [Storage implementation] explore & create the native HBase schema for writes Critical timelineserver Sangjin Lee Vrushali C
MAPREDUCE-6370 Timeline service v2 load generator needs to write event id Major . Li Lu Li Lu
YARN-3726 Fix TestHBaseTimelineWriterImpl unit test failure by fixing it’s test data Major timelineserver Vrushali C Vrushali C
YARN-3721 build is broken on YARN-2928 branch due to possible dependency cycle Blocker timelineserver Sangjin Lee Li Lu
YARN-3276 Refactor and fix null casting in some map cast for TimelineEntity (old and new) and fix findbug warnings Major timelineserver Junping Du Junping Du
YARN-3044 [Event producers] Implement RM writing app lifecycle events to ATS Major timelineserver Sangjin Lee Naganarasimha G R
YARN-3801 [JDK-8][YARN-2928] Exclude jdk.tools from hbase-client and hbase-testing-util Major timelineserver Tsuyoshi Ozawa Tsuyoshi Ozawa
YARN-3706 Generalize native HBase writer for additional tables Minor timelineserver Joep Rottinghuis Joep Rottinghuis
YARN-3792 Test case failures in TestDistributedShell and some issue fixes related to ATSV2 Major timelineserver Naganarasimha G R Naganarasimha G R
YARN-3051 [Storage abstraction] Create backing storage read interface for ATS readers Major timelineserver Sangjin Lee Varun Saxena
YARN-3047 [Data Serving] Set up ATS reader with basic request serving structure and lifecycle Major timelineserver Sangjin Lee Varun Saxena
YARN-3836 add equals and hashCode to TimelineEntity and other classes in the data model Major timelineserver Sangjin Lee Li Lu
YARN-3445 Cache runningApps in RMNode for getting running apps on given NodeId Major nodemanager, resourcemanager Junping Du Junping Du
YARN-3116 [Collector wireup] We need an assured way to determine if a container is an AM container on NM Major nodemanager, timelineserver Zhijie Shen Giovanni Matteo Fumarola
YARN-3949 ensure timely flush of timeline writes Major timelineserver Sangjin Lee Sangjin Lee
YARN-3908 Bugs in HBaseTimelineWriterImpl Major timelineserver Zhijie Shen Vrushali C
YARN-3993 Change to use the AM flag in ContainerContext determine AM container Major timelineserver Zhijie Shen Sunil Govindan
YARN-3984 Rethink event column key issue Major timelineserver Zhijie Shen Vrushali C
YARN-3049 [Storage Implementation] Implement storage reader interface to fetch raw data from HBase backend Major timelineserver Sangjin Lee Zhijie Shen
YARN-3906 split the application table from the entity table Major timelineserver Sangjin Lee Sangjin Lee
YARN-3904 Refactor timelineservice.storage to add support to online and offline aggregation writers Major timelineserver Li Lu Li Lu
YARN-3045 [Event producers] Implement NM writing container lifecycle events to ATS Major timelineserver Sangjin Lee Naganarasimha G R
YARN-4025 Deal with byte representations of Longs in writer code Major timelineserver Vrushali C Sangjin Lee
YARN-4064 build is broken at TestHBaseTimelineWriterImpl.java Blocker timelineserver Sangjin Lee Sangjin Lee
YARN-3814 REST API implementation for getting raw entities in TimelineReader Major timelineserver Varun Saxena Varun Saxena
YARN-4058 Miscellaneous issues in NodeManager project Minor timelineserver Naganarasimha G R Naganarasimha G R
YARN-2884 Proxying all AM-RM communications Major nodemanager, resourcemanager Carlo Curino Kishore Chaliparambil
YARN-4102 Add a “skip existing table” mode for timeline schema creator Major timelineserver Li Lu Li Lu
YARN-3901 Populate flow run data in the flow_run & flow activity tables Major timelineserver Vrushali C Vrushali C
YARN-4074 [timeline reader] implement support for querying for flows and flow runs Major timelineserver Sangjin Lee Sangjin Lee
YARN-4075 [reader REST API] implement support for querying for flows and flow runs Major timelineserver Sangjin Lee Varun Saxena
YARN-4203 Add request/response logging & timing for each REST endpoint call Minor timelineserver Vrushali C Varun Saxena
YARN-4210 HBase reader throws NPE if Get returns no rows Major timelineserver Varun Saxena Varun Saxena
YARN-3864 Implement support for querying single app and all apps for a flow run Blocker timelineserver Varun Saxena Varun Saxena
YARN-4178 [storage implementation] app id as string in row keys can cause incorrect ordering Major timelineserver Sangjin Lee Varun Saxena
YARN-4237 Support additional queries for ATSv2 Web UI Major . Varun Saxena Varun Saxena
YARN-4179 [reader implementation] support flow activity queries based on time Minor timelineserver Sangjin Lee Varun Saxena
YARN-4129 Refactor the SystemMetricPublisher in RM to better support newer events Major resourcemanager Naganarasimha G R Naganarasimha G R
YARN-4221 Store user in app to flow table Major timelineserver Varun Saxena Varun Saxena
YARN-4053 Change the way metric values are stored in HBase Storage Major timelineserver Varun Saxena Varun Saxena
YARN-4297 TestJobHistoryEventHandler and TestRMContainerAllocator failing on YARN-2928 branch Major . Varun Saxena Varun Saxena
YARN-3862 Support for fetching specific configs and metrics based on prefixes Major timelineserver Varun Saxena Varun Saxena
HDFS-9129 Move the safemode block count into BlockManager Major namenode Haohui Mai Mingliang Liu
HDFS-9414 Refactor reconfiguration of ClientDatanodeProtocol for reusability Major . Xiaobing Zhou Xiaobing Zhou
YARN-4356 ensure the timeline service v.2 is disabled cleanly and has no impact when it’s turned off Critical timelineserver Sangjin Lee Sangjin Lee
YARN-4450 TestTimelineAuthenticationFilter and TestYarnConfigurationFields fail Major timelineserver Sangjin Lee Sangjin Lee
HDFS-9371 Code cleanup for DatanodeManager Major namenode Jing Zhao Jing Zhao
YARN-4460 [Bug fix] RM fails to start when SMP is enabled Major timelineserver Li Lu Li Lu
YARN-4445 Unify the term flowId and flowName in timeline v2 codebase Major timelineserver Li Lu Zhan Zhang
YARN-1856 cgroups based memory monitoring for containers Major nodemanager Karthik Kambatla Varun Vasudev
YARN-4350 TestDistributedShell fails for V2 scenarios Major timelineserver Sangjin Lee Naganarasimha G R
YARN-3586 RM only get back addresses of Collectors that NM needs to know. Critical resourcemanager, timelineserver Junping Du Junping Du
YARN-2882 Add an OPPORTUNISTIC ExecutionType Major nodemanager Konstantinos Karanasos Konstantinos Karanasos
YARN-3480 Recovery may get very slow with lots of services with lots of app-attempts Major resourcemanager Jun Gong Jun Gong
HDFS-9498 Move code that tracks blocks with future generation stamps to BlockManagerSafeMode Major namenode Mingliang Liu Mingliang Liu
YARN-4550 some tests in TestContainerLanch fails on non-english locale environment Minor nodemanager, test Takashi Ohnishi Takashi Ohnishi
YARN-4335 Allow ResourceRequests to specify ExecutionType of a request ask Major nodemanager, resourcemanager Konstantinos Karanasos Konstantinos Karanasos
YARN-3995 Some of the NM events are not getting published due race condition when AM container finishes in NM Major nodemanager, timelineserver Naganarasimha G R Naganarasimha G R
YARN-4200 Refactor reader classes in storage to nest under hbase specific package name Minor timelineserver Vrushali C Li Lu
YARN-4553 Add cgroups support for docker containers Major yarn Sidharta Seethana Sidharta Seethana
YARN-4526 Make SystemClock singleton so AppSchedulingInfo could use it Major scheduler Karthik Kambatla Karthik Kambatla
HDFS-9542 Move BlockIdManager from FSNamesystem to BlockManager Major namenode Jing Zhao Jing Zhao
YARN-4622 TestDistributedShell fails for v2 test cases after modifications for 1.5 Major timelineserver Naganarasimha G R Naganarasimha G R
YARN-4578 Directories that are mounted in docker containers need to be more restrictive/container-specific Major yarn Sidharta Seethana Sidharta Seethana
YARN-4574 TestAMRMClientOnRMRestart fails on trunk Major client, test Takashi Ohnishi Takashi Ohnishi
YARN-4613 TestClientRMService#testGetClusterNodes fails occasionally Major test Jason Lowe Takashi Ohnishi
HDFS-9094 Add command line option to ask NameNode reload configuration. Major namenode Xiaobing Zhou Xiaobing Zhou
YARN-4238 correctly set createdTime and remove modifiedTime when publishing entities Major timelineserver Varun Saxena Varun Saxena
YARN-4224 Support fetching entities by UID and change the REST interface to conform to current REST APIs’ in YARN Major timelineserver Varun Saxena Varun Saxena
YARN-4644 TestRMRestart fails and findbugs issue in YARN-2928 branch Major timelineserver Varun Saxena Varun Saxena
YARN-4633 TestRMRestart.testRMRestartAfterPreemption fails intermittently in trunk Major test Rohith Sharma K S Bibin A Chundatt
YARN-4615 TestAbstractYarnScheduler#testResourceRequestRecoveryToTheRightAppAttempt fails occasionally Major test Jason Lowe Sunil Govindan
YARN-4446 Refactor reader API for better extensibility Major timelineserver Varun Saxena Varun Saxena
YARN-4409 Fix javadoc and checkstyle issues in timelineservice code Major timelineserver Varun Saxena Varun Saxena
YARN-3367 Replace starting a separate thread for post entity with event loop in TimelineClient Major timelineserver Junping Du Naganarasimha G R
YARN-4684 TestYarnCLI#testGetContainers failing in CN locale Major yarn Bibin A Chundatt Bibin A Chundatt
HDFS-9691 TestBlockManagerSafeMode#testCheckSafeMode fails intermittently Major test Mingliang Liu Mingliang Liu
HADOOP-12710 Remove dependency on commons-httpclient for TestHttpServerLogs Major . Wei-Chiu Chuang Wei-Chiu Chuang
YARN-4700 ATS storage has one extra record each time the RM got restarted Major timelineserver Li Lu Naganarasimha G R
MAPREDUCE-6546 reconcile the two versions of the timeline service performance tests Minor . Sangjin Lee Sangjin Lee
HDFS-1477 Support reconfiguring dfs.heartbeat.interval and dfs.namenode.heartbeat.recheck-interval without NN restart Major namenode Patrick Kling Xiaobing Zhou
YARN-4062 Add the flush and compaction functionality via coprocessors and scanners for flow run table Major timelineserver Vrushali C Vrushali C
YARN-4712 CPU Usage Metric is not captured properly in YARN-2928 Major timelineserver Naganarasimha G R Naganarasimha G R
HADOOP-12926 lz4.c does not detect 64-bit mode properly Major native Alan Burlison Alan Burlison
YARN-4687 Document Reservation ACLs Minor . Sean Po Sean Po
YARN-4805 Don’t go through all schedulers in ParameterizedTestBase Major . Karthik Kambatla Karthik Kambatla
YARN-4711 NM is going down with NPE’s due to single thread processing of events by Timeline client Critical timelineserver Naganarasimha G R Naganarasimha G R
HDFS-9349 Support reconfiguring fs.protected.directories without NN restart Major namenode Xiaobing Zhou Xiaobing Zhou
YARN-4811 Generate histograms in ContainerMetrics for actual container resource usage Major . Varun Vasudev Varun Vasudev
YARN-3461 Consolidate flow name/version/run defaults Major timelineserver Zhijie Shen Sangjin Lee
HADOOP-12753 S3A JUnit tests failing if using HTTP proxy Minor fs/s3 Zoran Rajic Zoran Rajic
YARN-3863 Support complex filters in TimelineReader Major . Varun Saxena Varun Saxena
HDFS-10209 Support enable caller context in HDFS namenode audit log without restart namenode Major . Xiaoyu Yao Xiaobing Zhou
HDFS-10286 Fix TestDFSAdmin#testNameNodeGetReconfigurableProperties Major . Xiaoyu Yao Xiaobing Zhou
HDFS-10284 o.a.h.hdfs.server.blockmanagement.TestBlockManagerSafeMode.testCheckSafeMode fails intermittently Major test Mingliang Liu Mingliang Liu
YARN-2883 Queuing of container requests in the NM Major nodemanager, resourcemanager Konstantinos Karanasos Konstantinos Karanasos
YARN-4890 Unit test intermittent failure: TestNodeLabelContainerAllocation#testQueueUsedCapacitiesUpdate Major . Wangda Tan Sunil Govindan
HDFS-10207 Support enable Hadoop IPC backoff without namenode restart Major . Xiaoyu Yao Xiaobing Zhou
YARN-4968 A couple of AM retry unit tests need to wait SchedulerApplicationAttempt stopped. Major . Wangda Tan Wangda Tan
YARN-3816 [Aggregation] App-level aggregation and accumulation for YARN system metrics Major timelineserver Junping Du Li Lu
YARN-4846 Random failures for TestCapacitySchedulerPreemption#testPreemptionPolicyShouldRespectAlreadyMarkedKillableContainers Major . Bibin A Chundatt Bibin A Chundatt
YARN-2885 Create AMRMProxy request interceptor for distributed scheduling decisions for queueable containers Major nodemanager, resourcemanager Konstantinos Karanasos Arun Suresh
YARN-4991 Fix ContainerRequest Constructor to set nodelabelExpression correctly Major test Bibin A Chundatt Bibin A Chundatt
YARN-4966 Improve yarn logs to fetch container logs without specifying nodeId Major . Xuan Gong Xuan Gong
YARN-4807 MockAM#waitForState sleep duration is too long Major . Karthik Kambatla Yufei Gu
YARN-3998 Add support in the NodeManager to re-launch containers Major . Jun Gong Jun Gong
YARN-4986 Add a check in the coprocessor for table to operated on Major timelineserver Vrushali C Vrushali C
YARN-5014 Ensure non-metric values are returned as is for flow run table from the coprocessor Major timelineserver Vrushali C Vrushali C
YARN-3150 [Documentation] Documenting the timeline service v2 Major . Zhijie Shen Sangjin Lee
MAPREDUCE-6424 Store MR counters as timeline metrics instead of event Major . Junping Du Naganarasimha G R
YARN-4447 Provide a mechanism to represent complex filters and parse them at the REST layer Major timelineserver Varun Saxena Varun Saxena
MAPREDUCE-6688 Store job configurations in Timeline Service v2 Major applicationmaster Junping Du Varun Saxena
YARN-4920 ATS/NM should support a link to dowload/get the logs in text format Major yarn Xuan Gong Xuan Gong
YARN-4905 Improve “yarn logs” command-line to optionally show log metadata also Major . Xuan Gong Xuan Gong
YARN-4595 Add support for configurable read-only mounts when launching Docker containers Major yarn Billie Rinaldi Billie Rinaldi
YARN-5045 hbase unit tests fail due to dependency issues Blocker timelineserver Sangjin Lee Sangjin Lee
YARN-4778 Support specifying resources for task containers in SLS Major . Wangda Tan Wangda Tan
YARN-4842 “yarn logs” command should not require the appOwner argument Major . Ram Venkatesh Xuan Gong
YARN-5073 Refactor startContainerInternal() in ContainerManager to remove unused parameter Minor nodemanager, resourcemanager Konstantinos Karanasos Konstantinos Karanasos
HDFS-9389 Add maintenance states to AdminStates Major . Ming Ma Ming Ma
YARN-2888 Corrective mechanisms for rebalancing NM container queues Major nodemanager, resourcemanager Konstantinos Karanasos Arun Suresh
YARN-4738 Notify the RM about the status of OPPORTUNISTIC containers Major nodemanager, resourcemanager Konstantinos Karanasos Konstantinos Karanasos
YARN-5075 Fix findbugs warning in hadoop-yarn-common module Major . Akira Ajisaka Arun Suresh
YARN-4412 Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers Major nodemanager, resourcemanager Arun Suresh Arun Suresh
YARN-5102 timeline service build fails with java 8 Blocker timelineserver Sangjin Lee Sangjin Lee
YARN-5090 Add End-to-End test-cases for DistributedScheduling using MiniYarnCluster Major . Arun Suresh Arun Suresh
YARN-5096 timelinereader has a lot of logging that’s not useful Minor timelineserver Sangjin Lee Sangjin Lee
YARN-4913 Yarn logs should take a -out option to write to a directory Major . Xuan Gong Xuan Gong
HDFS-2173 saveNamespace should not throw IOE when only one storage directory fails to write VERSION file Major . Todd Lipcon Andras Bokor
YARN-5110 Fix OpportunisticContainerAllocator to insert complete HostAddress in issued ContainerTokenIds Major . Arun Suresh Konstantinos Karanasos
YARN-5050 Code cleanup for TestDistributedShell Major timelineserver Li Lu Li Lu
YARN-5016 Add support for a minimum retry interval for container retries Major . Varun Vasudev Jun Gong
HDFS-7766 Add a flag to WebHDFS op=CREATE to not respond with a 307 redirect Major ui, webhdfs Ravi Prakash Ravi Prakash
YARN-5093 created time shows 0 in most REST output Critical timelineserver Sangjin Lee Varun Saxena
YARN-5018 Online aggregation logic should not run immediately after collectors got started Major timelineserver Li Lu Li Lu
YARN-5115 Avoid setting CONTENT-DISPOSITION header in the container-logs web-service Major . Xuan Gong Xuan Gong
YARN-5097 NPE in Separator.joinEncoded() Critical timelineserver Sangjin Lee Vrushali C
YARN-5095 flow activities and flow runs are populated with wrong timestamp when RM restarts w/ recovery enabled Critical timelineserver Sangjin Lee Varun Saxena
YARN-5138 fix “no findbugs output file” error for hadoop-yarn-server-timelineservice-hbase-tests Major timelineserver Vrushali C Vrushali C
YARN-5109 timestamps are stored unencoded causing parse errors Blocker timelineserver Sangjin Lee Varun Saxena
YARN-5089 Improve “yarn log” command-line “logFiles” option to support regex Major . Xuan Gong Xuan Gong
YARN-5127 Expose ExecutionType in Container api record Major . Arun Suresh Hitesh Sharma
YARN-5117 QueuingContainerManager does not start GUARANTEED Container even if Resources are available Major . Arun Suresh Konstantinos Karanasos
YARN-5111 YARN container system metrics are not aggregated to application Critical timelineserver Sangjin Lee Naganarasimha G R
YARN-4007 Add support for different network setups when launching the docker container Major nodemanager Varun Vasudev Sidharta Seethana
YARN-5141 Get Container logs for the Running application from Yarn Logs CommandLine Major . Xuan Gong Xuan Gong
YARN-5105 entire time series is returned for YARN container system metrics (CPU and memory) Major timelineserver Sangjin Lee Varun Saxena
YARN-5088 Improve “yarn log” command-line to read the last K bytes for the log files Major . Xuan Gong Xuan Gong
YARN-5162 Fix Exceptions thrown during registerAM call when Distributed Scheduling is Enabled Major . Arun Suresh Hitesh Sharma
YARN-5189 Make HBaseTimeline[Reader|Writer]Impl default and move FileSystemTimeline*Impl Major timelineserver Joep Rottinghuis Joep Rottinghuis
HDFS-9877 HDFS Namenode UI: Fix browsing directories that need to be encoded Major ui Ravi Prakash Ravi Prakash
HDFS-7767 Use the noredirect flag in WebHDFS to allow web browsers to upload files via the NN UI Major ui, webhdfs Ravi Prakash Ravi Prakash
YARN-5142 fix findbugs warnings/errors for hadoop-yarn-server-timelineservice-hbase-tests Major timelineserver Vrushali C Vrushali C
YARN-5167 Escaping occurences of encodedValues Critical timelineserver Joep Rottinghuis Sangjin Lee
YARN-5180 Allow ResourceRequest to specify an enforceExecutionType flag Major . Arun Suresh Arun Suresh
YARN-5176 More test cases for queuing of containers at the NM Major . Konstantinos Karanasos Konstantinos Karanasos
YARN-5204 Properly report status of killed/stopped queued containers Major . Konstantinos Karanasos Konstantinos Karanasos
YARN-5210 NPE in Distributed Shell while publishing DS_CONTAINER_START event and other miscellaneous issues Major timelineserver Varun Saxena Varun Saxena
YARN-5191 Rename the “download=true” option for getLogs in NMWebServices and AHSWebServices Major . Xuan Gong Xuan Gong
YARN-5170 Eliminate singleton converters and static method access Major timelineserver Joep Rottinghuis Joep Rottinghuis
HDFS-7987 Allow files / directories to be moved Major ui Ravi Prakash Ravi Prakash
YARN-5052 [Documentation] Update timeline service v2 documentation to capture information about filters Major timelineserver Varun Saxena Varun Saxena
YARN-5124 Modify AMRMClient to set the ExecutionType in the ResourceRequest Major . Arun Suresh Arun Suresh
YARN-5212 Run existing ContainerManager tests using QueuingContainerManagerImpl Major . Konstantinos Karanasos Konstantinos Karanasos
YARN-4887 AM-RM protocol changes for identifying resource-requests explicitly Major applications, resourcemanager Subru Krishnan Subru Krishnan
YARN-5223 Container line in yarn logs output for a live application should include the hostname for the container Major . Siddharth Seth Xuan Gong
YARN-5122 “yarn logs” for running containers should print an explicit footer saying that the log may be incomplete Major . Vinod Kumar Vavilapalli Jian He
YARN-5243 fix several rebase and other miscellaneous issues before merge Major timelineserver Sangjin Lee Sangjin Lee
YARN-5070 upgrade HBase version for first merge Critical timelineserver Sangjin Lee Vrushali C
YARN-5251 Yarn CLI to obtain App logs for last ‘n’ bytes fails with ‘java.io.IOException’ and for ‘n’ bytes fails with NumberFormatException Blocker . Sumana Sathish Xuan Gong
HDFS-10328 Add per-cache-pool default replication num configuration Minor caching xupeng xupeng
YARN-5171 Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node Major . Arun Suresh Íñigo Goiri
MAPREDUCE-6720 Inconsistent values of counters across tasks and job reported to timeline service. Major applicationmaster Varun Saxena Varun Saxena
YARN-5300 Exclude generated federation protobuf sources from YARN Javadoc/findbugs build Minor nodemanager, resourcemanager Subru Krishnan Subru Krishnan
YARN-5174 [documentation] several updates/corrections to timeline service documentation Major timelineserver Sangjin Lee Sangjin Lee
YARN-5227 yarn logs command: no need to specify -applicationId when specifying containerId Major . Jian He Gergely Novák
YARN-5224 Logs for a completed container are not available in the yarn logs output for a live application Major . Siddharth Seth Xuan Gong
YARN-5316 fix hadoop-aws pom not to do the exclusion Major timelineserver Sangjin Lee Sangjin Lee
YARN-5233 Support for specifying a path for ATS plugin jars Major timelineserver Li Lu Li Lu
YARN-5200 Improve yarn logs to get Container List Major . Xuan Gong Xuan Gong
YARN-5299 Log Docker run command when container fails Major yarn Varun Vasudev Varun Vasudev
YARN-4759 Fix signal handling for docker containers Major yarn Sidharta Seethana Shane Kumpf
YARN-5363 For AM containers, or for containers of running-apps, “yarn logs” incorrectly only (tries to) shows syslog file-type by default Major log-aggregation Vinod Kumar Vavilapalli Vinod Kumar Vavilapalli
YARN-5298 Mount usercache and NM filecache directories into Docker container Major yarn Varun Vasudev Sidharta Seethana
YARN-5361 Obtaining logs for completed container says ’file belongs to a running container ’ at the end Critical . Sumana Sathish Xuan Gong
YARN-5156 YARN_CONTAINER_FINISHED of YARN_CONTAINERs will always have running state Major timelineserver Li Lu Vrushali C
YARN-5350 Distributed Scheduling: Ensure sort order of allocatable nodes returned by the RM is not lost Major . Arun Suresh Arun Suresh
YARN-5392 Replace use of Priority in the Scheduling infrastructure with an opaque ShedulerRequestKey Major . Arun Suresh Arun Suresh
YARN-5351 ResourceRequest should take ExecutionType into account during comparison Major . Konstantinos Karanasos Konstantinos Karanasos
YARN-3662 Federation Membership State Store internal APIs Major nodemanager, resourcemanager Subru Krishnan Subru Krishnan
YARN-5229 Refactor #isApplicationEntity and #getApplicationEvent from HBaseTimelineWriterImpl Minor timelineserver Joep Rottinghuis Vrushali C
YARN-5113 Refactoring and other clean-up for distributed scheduling Major . Arun Suresh Konstantinos Karanasos
YARN-5458 Rename DockerStopCommandTest to TestDockerStopCommand Trivial . Shane Kumpf Shane Kumpf
YARN-5443 Add support for docker inspect command Major yarn Shane Kumpf Shane Kumpf
YARN-5226 remove AHS enable check from LogsCLI#fetchAMContainerLogs Major . Xuan Gong Xuan Gong
YARN-5459 Add support for docker rm Minor yarn Shane Kumpf Shane Kumpf
YARN-5406 In-memory based implementation of the FederationMembershipStateStore Major nodemanager, resourcemanager Subru Krishnan Ellen Hui
YARN-5390 Federation Subcluster Resolver Major nodemanager, resourcemanager Carlo Curino Ellen Hui
YARN-5429 Fix @return related javadoc warnings in yarn-api Major . Vrushali C Vrushali C
YARN-4888 Changes in scheduler to identify resource-requests explicitly by allocation-id Major resourcemanager Subru Krishnan Subru Krishnan
YARN-5307 Federation Application State Store internal APIs Major nodemanager, resourcemanager Subru Krishnan Subru Krishnan
YARN-3664 Federation PolicyStore internal APIs Major nodemanager, resourcemanager Subru Krishnan Subru Krishnan
YARN-5470 Differentiate exactly match with regex in yarn log CLI Major . Xuan Gong Xuan Gong
YARN-5408 Compose Federation membership/application/policy APIs into an uber FederationStateStore API Major nodemanager, resourcemanager Subru Krishnan Ellen Hui
YARN-5394 Remove bind-mount /etc/passwd for Docker containers Major yarn Zhankun Tang Zhankun Tang
YARN-5407 In-memory based implementation of the FederationApplicationStateStore, FederationPolicyStateStore Major nodemanager, resourcemanager Subru Krishnan Ellen Hui
YARN-5137 Make DiskChecker pluggable in NodeManager Major nodemanager Ray Chiang Yufei Gu
YARN-5519 Add SubClusterId in AddApplicationHomeSubClusterResponse for Router Failover Major nodemanager, resourcemanager Giovanni Matteo Fumarola Ellen Hui
YARN-5514 Clarify DecommissionType.FORCEFUL comment Minor documentation Robert Kanter Vrushali C
YARN-3672 Create Facade for Federation State and Policy Store Major nodemanager, resourcemanager Subru Krishnan Subru Krishnan
YARN-5467 InputValidator for the FederationStateStore internal APIs Major nodemanager, resourcemanager Subru Krishnan Giovanni Matteo Fumarola
YARN-4676 Automatic and Asynchronous Decommissioning Nodes Status Tracking Major resourcemanager Daniel Zhi Daniel Zhi
YARN-3673 Create a FailoverProxy for Federation services Major nodemanager, resourcemanager Subru Krishnan Subru Krishnan
YARN-5457 Refactor DistributedScheduling framework to pull out common functionality Major resourcemanager Arun Suresh Arun Suresh
YARN-5042 Mount /sys/fs/cgroup into Docker containers as read only mount Major yarn Varun Vasudev luhuichun
YARN-5564 Fix typo in RM_SCHEDULER_RESERVATION_THRESHOLD_INCREMENT_MULTIPLE Trivial fairscheduler Ray Chiang Ray Chiang
YARN-5557 Add localize API to the ContainerManagementProtocol Major . Jian He Jian He
YARN-5327 API changes required to support recurring reservations in the YARN ReservationSystem Major resourcemanager Subru Krishnan Sangeetha Abdu Jyothi
YARN-4889 Changes in AMRMClient for identifying resource-requests explicitly Major resourcemanager Subru Krishnan Arun Suresh
YARN-3671 Integrate Federation services with ResourceManager Major nodemanager, resourcemanager Subru Krishnan Subru Krishnan
HDFS-9392 Admins support for maintenance state Major . Ming Ma Ming Ma
HDFS-10813 DiskBalancer: Add the getNodeList method in Command Minor balancer & mover Yiqun Lin Yiqun Lin
YARN-5221 Expose UpdateResourceRequest API to allow AM to request for change in container properties Major . Arun Suresh Arun Suresh
YARN-5596 Fix failing unit test in TestDockerContainerRuntime Minor nodemanager, yarn Sidharta Seethana Sidharta Seethana
YARN-5612 Return SubClusterId in FederationStateStoreFacade#addApplicationHomeSubCluster for Router Failover Major nodemanager, resourcemanager Giovanni Matteo Fumarola Giovanni Matteo Fumarola
YARN-5601 Make the RM epoch base value configurable Major nodemanager, resourcemanager Subru Krishnan Subru Krishnan
YARN-5264 Store all queue-specific information in FSQueue Major fairscheduler Yufei Gu Yufei Gu
YARN-5576 Allow resource localization while container is running Major . Jian He Jian He
YARN-5323 Policies APIs (for Router and AMRMProxy policies) Major nodemanager, resourcemanager Carlo Curino Carlo Curino
YARN-5620 Core changes in NodeManager to support re-initialization of Containers with new launchContext Major . Arun Suresh Arun Suresh
YARN-4091 Add REST API to retrieve scheduler activity Major capacity scheduler, resourcemanager Sunil Govindan Chen Ge
YARN-5637 Changes in NodeManager to support Container rollback and commit Major . Arun Suresh Arun Suresh
YARN-3140 Improve locks in AbstractCSQueue/LeafQueue/ParentQueue Major resourcemanager, scheduler Wangda Tan Wangda Tan
YARN-3141 Improve locks in SchedulerApplicationAttempt/FSAppAttempt/FiCaSchedulerApp Major resourcemanager, scheduler Wangda Tan Wangda Tan
YARN-5324 Stateless Federation router policies implementation Major nodemanager, resourcemanager Carlo Curino Carlo Curino
YARN-5609 Expose upgrade and restart API in ContainerManagementProtocol Major . Arun Suresh Arun Suresh
HDFS-9895 Remove unnecessary conf cache from DataNode Major datanode Xiaobing Zhou Xiaobing Zhou
YARN-3142 Improve locks in AppSchedulingInfo Major resourcemanager, scheduler Wangda Tan Varun Saxena
YARN-4205 Add a service for monitoring application life time out Major scheduler nijel Rohith Sharma K S
YARN-5486 Update OpportunisticContainerAllocatorAMService::allocate method to handle OPPORTUNISTIC container requests Major resourcemanager Arun Suresh Konstantinos Karanasos
YARN-5384 Expose priority in ReservationSystem submission APIs Major capacity scheduler, fairscheduler, resourcemanager Sean Po Sean Po
YARN-5702 Refactor TestPBImplRecords so that we can reuse for testing protocol records in other YARN modules Major . Subru Krishnan Subru Krishnan
YARN-3139 Improve locks in AbstractYarnScheduler/CapacityScheduler/FairScheduler Major resourcemanager, scheduler Wangda Tan Wangda Tan
HDFS-10893 Refactor TestDFSShell by setting up MiniDFSCluser once for all commands test Major test Mingliang Liu Mingliang Liu
HADOOP-13627 Have an explicit KerberosAuthException for UGI to throw, text from public constants Major security Steve Loughran Xiao Chen
YARN-5325 Stateless ARMRMProxy policies implementation Major nodemanager, resourcemanager Carlo Curino Carlo Curino
YARN-5638 Introduce a collector timestamp to uniquely identify collectors creation order in collector discovery Major timelineserver Li Lu Li Lu
YARN-5699 Retrospect yarn entity fields which are publishing in events info fields. Major . Rohith Sharma K S Rohith Sharma K S
HDFS-9390 Block management for maintenance states Major . Ming Ma Ming Ma
YARN-5561 [Atsv2] : Support for ability to retrieve apps/app-attempt/containers and entities via REST Major timelinereader Rohith Sharma K S Rohith Sharma K S
YARN-4911 Bad placement policy in FairScheduler causes the RM to crash Major fairscheduler Ray Chiang Ray Chiang
YARN-5047 Refactor nodeUpdate across schedulers Major capacityscheduler, fairscheduler, scheduler Ray Chiang Ray Chiang
YARN-3649 Allow configurable prefix for hbase table names (like prod, exp, test etc) Major timelineserver Vrushali C Vrushali C
YARN-5715 introduce entity prefix for return and sort order Critical timelineserver Sangjin Lee Rohith Sharma K S
YARN-4097 Create POC timeline web UI with new YARN web UI framework Major timelineserver Li Lu Li Lu
YARN-5799 Fix Opportunistic Allocation to set the correct value of Node Http Address Major resourcemanager Arun Suresh Arun Suresh
YARN-4765 Split TestHBaseTimelineStorage into multiple test classes Major timelineserver Varun Saxena Varun Saxena
YARN-5793 Trim configuration values in DockerLinuxContainerRuntime Minor nodemanager Tianyin Xu Tianyin Xu
YARN-5391 PolicyManager to tie together Router/AMRM Federation policies Major nodemanager, resourcemanager Carlo Curino Carlo Curino
YARN-5265 Make HBase configuration for the timeline service configurable Major timelineserver Joep Rottinghuis Joep Rottinghuis
YARN-2995 Enhance UI to show cluster resource utilization of various container Execution types Blocker resourcemanager Sriram Rao Konstantinos Karanasos
YARN-3359 Recover collector list when RM fails over Major resourcemanager Junping Du Li Lu
YARN-5716 Add global scheduler interface definition and update CapacityScheduler to use it. Major capacityscheduler Wangda Tan Wangda Tan
YARN-5833 Add validation to ensure default ports are unique in Configuration Major yarn Konstantinos Karanasos Konstantinos Karanasos
YARN-4329 Allow fetching exact reason as to why a submitted app is in ACCEPTED state in Fair Scheduler Major fairscheduler, resourcemanager Naganarasimha G R Yufei Gu
YARN-5611 Provide an API to update lifetime of an application. Major resourcemanager Rohith Sharma K S Rohith Sharma K S
HDFS-11119 Support for parallel checking of StorageLocations on DataNode startup Major datanode Arpit Agarwal Arpit Agarwal
HDFS-11114 Support for running async disk checks in DataNode Major datanode Arpit Agarwal Arpit Agarwal
YARN-5634 Simplify initialization/use of RouterPolicy via a RouterPolicyFacade Major nodemanager, resourcemanager Carlo Curino Carlo Curino
YARN-5792 adopt the id prefix for YARN, MR, and DS entities Major timelineserver Sangjin Lee Varun Saxena
HDFS-11148 Update DataNode to use StorageLocationChecker at startup Major datanode Arpit Agarwal Arpit Agarwal
YARN-5676 Add a HashBasedRouterPolicy, and small policies and test refactoring. Major nodemanager, resourcemanager Carlo Curino Carlo Curino
YARN-5872 Add AlwayReject policies for router and amrmproxy. Major nodemanager, resourcemanager Carlo Curino Carlo Curino
YARN-5905 Update the RM webapp host that is reported as part of Federation membership to current primary RM’s IP Minor federation, resourcemanager Subru Krishnan Subru Krishnan
YARN-5865 Retrospect updateApplicationPriority api to handle state store exception in align with YARN-5611 Major . Sunil Govindan Sunil Govindan
YARN-5649 Add REST endpoints for updating application timeouts Major scheduler Rohith Sharma K S Rohith Sharma K S
YARN-4206 Add Application timeouts in Application report and CLI Major scheduler nijel Rohith Sharma K S
HDFS-11149 Support for parallel checking of FsVolumes Major datanode Arpit Agarwal Arpit Agarwal
HDFS-8678 Bring back the feature to view chunks of files in the HDFS file browser Major ui Ravi Prakash Ravi Prakash
YARN-5761 Separate QueueManager from Scheduler Major capacityscheduler Xuan Gong Xuan Gong
YARN-5746 The state of the parentQueue and its childQueues should be synchronized. Major capacity scheduler, resourcemanager Xuan Gong Xuan Gong
YARN-5965 Retrospect ApplicationReport#getApplicationTimeouts Major scheduler Jian He Rohith Sharma K S
YARN-5739 Provide timeline reader API to list available timeline entity types for one application Major timelinereader Li Lu Li Lu
MAPREDUCE-6818 Remove direct reference to TimelineClientImpl Major . Li Lu Li Lu
YARN-5982 Simplify opportunistic container parameters and metrics Major . Konstantinos Karanasos Konstantinos Karanasos
YARN-5970 Validate application update timeout request parameters Major scheduler Rohith Sharma K S Rohith Sharma K S
YARN-5524 Yarn live log aggregation does not throw if command line arg is wrong Major log-aggregation Prasanth Jayachandran Xuan Gong
YARN-5650 Render Application Timeout value in web UI Major scheduler Rohith Sharma K S Akhil PB
HDFS-11182 Update DataNode to use DatasetVolumeChecker Major datanode Arpit Agarwal Arpit Agarwal
YARN-5976 Update hbase version to 1.2 (removes phoenix dependencies) Major . Vrushali C Vrushali C
YARN-5216 Expose configurable preemption policy for OPPORTUNISTIC containers running on the NM Major distributed-scheduling Arun Suresh Hitesh Sharma
YARN-5938 Refactoring OpportunisticContainerAllocator to use SchedulerRequestKey instead of Priority and other misc fixes Major . Arun Suresh Arun Suresh
YARN-5756 Add state-machine implementation for scheduler queues Major . Xuan Gong Xuan Gong
YARN-5931 Document timeout interfaces CLI and REST APIs Major scheduler Rohith Sharma K S Rohith Sharma K S
YARN-5529 Create new DiskValidator class with metrics Major nodemanager Ray Chiang Yufei Gu
YARN-6025 Fix synchronization issues of AbstractYarnScheduler#nodeUpdate and its implementations Major capacity scheduler, scheduler Naganarasimha G R Naganarasimha G R
YARN-5923 Unable to access logs for a running application if YARN_ACL_ENABLE is enabled Major . Xuan Gong Xuan Gong
YARN-5906 Update AppSchedulingInfo to use SchedulingPlacementSet Major . Wangda Tan Wangda Tan
YARN-6009 RM fails to start during an upgrade - Failed to load/recover state (YarnException: Invalid application timeout, value=0 for type=LIFETIME) Critical resourcemanager Gour Saha Rohith Sharma K S
YARN-5585 [Atsv2] Reader side changes for entity prefix and support for pagination via additional filters Critical timelinereader Rohith Sharma K S Rohith Sharma K S
YARN-6074 FlowRunEntity does not deserialize long values correctly Major timelinereader Rohith Sharma K S Rohith Sharma K S
YARN-3955 Support for application priority ACLs in queues of CapacityScheduler Major capacityscheduler Sunil Govindan Sunil Govindan
HDFS-9391 Update webUI/JMX to display maintenance state info Major . Ming Ma Manoj Govindassamy
YARN-5416 TestRMRestart#testRMRestartWaitForPreviousAMToFinish failed intermittently due to not wait SchedulerApplicationAttempt to be stopped Minor test, yarn Junping Du Junping Du
YARN-5980 Update documentation for single node hbase deploy Major timelineserver Vrushali C Vrushali C
YARN-6011 Add a new web service to list the files on a container in AHSWebService Major . Xuan Gong Xuan Gong
YARN-5378 Accommodate app-id->cluster mapping Major timelineserver Joep Rottinghuis Sangjin Lee
YARN-6016 Fix minor bugs in handling of local AMRMToken in AMRMProxy Minor federation Botong Huang Botong Huang
YARN-5556 CapacityScheduler: Support deleting queues without requiring a RM restart Major capacity scheduler Xuan Gong Naganarasimha G R
YARN-6064 Support fromId for flowRuns and flow/flowRun apps REST API’s Major timelinereader Rohith Sharma K S Rohith Sharma K S
HDFS-11259 Update fsck to display maintenance state info Major datanode, namenode Manoj Govindassamy Manoj Govindassamy
YARN-6094 Update the coprocessor to be a dynamically loaded one Major timelineserver Vrushali C Vrushali C
YARN-5831 Propagate allowPreemptionFrom flag all the way down to the app Major fairscheduler Karthik Kambatla Yufei Gu
HDFS-11296 Maintenance state expiry should be an epoch time and not jvm monotonic Major . Manoj Govindassamy Manoj Govindassamy
YARN-6099 Improve webservice to list aggregated log files Major . Xuan Gong Xuan Gong
YARN-5830 FairScheduler: Avoid preempting AM containers Major fairscheduler Karthik Kambatla Yufei Gu
YARN-3637 Handle localization sym-linking correctly at the YARN level Major . Chris Trezzo Chris Trezzo
YARN-6126 Obtaining app logs for Running application fails with “Unable to parse json from webservice. Error:” Major . Sumana Sathish Xuan Gong
YARN-6100 improve YARN webservice to output aggregated container logs Major . Xuan Gong Xuan Gong
YARN-5951 Changes to allow CapacityScheduler to use configuration store Major . Jonathan Hung Jonathan Hung
YARN-6108 Improve AHS webservice to accept NM address as a parameter to get container logs Major . Xuan Gong Xuan Gong
YARN-5258 Document Use of Docker with LinuxContainerExecutor Critical documentation Daniel Templeton Daniel Templeton
HADOOP-14032 Reduce fair call queue priority inversion Major ipc Daryn Sharp Daryn Sharp
HADOOP-14034 Allow ipc layer exceptions to selectively close connections Major ipc Daryn Sharp Daryn Sharp
HADOOP-14033 Reduce fair call queue lock contention Major ipc Daryn Sharp Daryn Sharp
YARN-6170 TimelineReaderServer should wait to join with HttpServer2 Minor timelinereader Sangjin Lee Sangjin Lee
HADOOP-13075 Add support for SSE-KMS and SSE-C in s3a filesystem Major fs/s3 Andrew Olson Steve Moist
YARN-6113 re-direct NM Web Service to get container logs for finished applications Major . Xuan Gong Xuan Gong
YARN-5966 AMRMClient changes to support ExecutionType update Major . Arun Suresh Arun Suresh
YARN-6156 AM blacklisting to consider node label partition Major . Bibin A Chundatt Bibin A Chundatt
HDFS-11265 Extend visualization for Maintenance Mode under Datanode tab in the NameNode UI Major datanode, namenode Manoj Govindassamy Elek, Marton
YARN-6163 FS Preemption is a trickle for severely starved applications Major fairscheduler Karthik Kambatla Karthik Kambatla
YARN-5798 Set UncaughtExceptionHandler for all FairScheduler threads Major fairscheduler Karthik Kambatla Yufei Gu
YARN-4675 Reorganize TimelineClient and TimelineClientImpl into separate classes for ATSv1.x and ATSv2 Major timelineserver Naganarasimha G R Naganarasimha G R
HADOOP-14040 Use shaded aws-sdk uber-JAR 1.11.86 Major build, fs/s3 Steve Loughran Steve Loughran
YARN-6193 FairScheduler might not trigger preemption when using DRF Major fairscheduler Karthik Kambatla Karthik Kambatla
YARN-6159 Documentation changes for TimelineV2Client Minor documentation Varun Saxena Naganarasimha G R
HDFS-11430 Separate class InnerNode from class NetworkTopology and make it extendable Major namenode Chen Liang Tsz Wo Nicholas Sze
HADOOP-14099 Split S3 testing documentation out into its own file Minor documentation, fs/s3 Steve Loughran Steve Loughran
HDFS-11411 Avoid OutOfMemoryError in TestMaintenanceState test runs Major datanode, namenode Manoj Govindassamy Manoj Govindassamy
HADOOP-14102 Relax error message assertion in S3A test ITestS3AEncryptionSSEC Minor fs/s3 Mingliang Liu Mingliang Liu
YARN-6069 CORS support in timeline v2 Major timelinereader Sreenath Somarajapuram Rohith Sharma K S
YARN-5959 RM changes to support change of container ExecutionType Major . Arun Suresh Arun Suresh
YARN-6228 EntityGroupFSTimelineStore should allow configurable cache stores. Major timelineserver Li Lu Li Lu
YARN-5410 Bootstrap Router server module Major nodemanager, resourcemanager Subru Krishnan Giovanni Matteo Fumarola
YARN-5946 Create YarnConfigurationStore interface and InMemoryConfigurationStore class Major . Jonathan Hung Jonathan Hung
YARN-6215 FairScheduler preemption and update should not run concurrently Major fairscheduler, test Sunil Govindan Tao Jie
YARN-6123 [YARN-5864] Add a test to make sure queues of orderingPolicy will be updated when childQueues is added or removed. Major . Wangda Tan Wangda Tan
YARN-6253 FlowAcitivityColumnPrefix.store(byte[] rowKey, …) drops timestamp Major . Haibo Chen Haibo Chen
YARN-6190 Validation and synchronization fixes in LocalityMulticastAMRMProxyPolicy Minor federation Botong Huang Botong Huang
YARN-6027 Support fromid(offset) filter for /flows API Major timelineserver Rohith Sharma K S Rohith Sharma K S
HDFS-11450 HDFS specific network topology classes with storage type info included Major namenode Chen Liang Chen Liang
HDFS-11412 Maintenance minimum replication config value allowable range should be [0, DefaultReplication] Major datanode, namenode Manoj Govindassamy Manoj Govindassamy
YARN-6247 Share a single instance of SubClusterResolver instead of instantiating one per AM Minor . Botong Huang Botong Huang
YARN-6256 Add FROM_ID info key for timeline entities in reader response. Major timelineserver Rohith Sharma K S Rohith Sharma K S
YARN-6237 Move UID constant to TimelineReaderUtils Major timelinereader Rohith Sharma K S Rohith Sharma K S
YARN-5669 Add support for Docker pull Major yarn Zhankun Tang luhuichun
YARN-5948 Implement MutableConfigurationManager for handling storage into configuration store Major . Jonathan Hung Jonathan Hung
YARN-1047 Expose # of pre-emptions as a queue counter Major fairscheduler Philip Zeyliger Karthik Kambatla
YARN-6281 Cleanup when AMRMProxy fails to initialize a new interceptor chain Minor . Botong Huang Botong Huang
YARN-6318 timeline service schema creator fails if executed from a remote machine Minor timelineserver Sangjin Lee Sangjin Lee
HDFS-11482 Add storage type demand to into DFSNetworkTopology#chooseRandom Major namenode Chen Liang Chen Liang
YARN-6314 Potential infinite redirection on YARN log redirection web service Major . Xuan Gong Xuan Gong
YARN-6313 yarn logs cli should provide logs for a completed container even when application is still running Major . Siddharth Seth Xuan Gong
HDFS-11514 DFSTopologyNodeImpl#chooseRandom optimizations Major namenode Chen Liang Chen Liang
YARN-6367 YARN logs CLI needs alway check containerLogsInfo/containerLogInfo before parse the JSON object from NMWebService Major . Siddharth Seth Xuan Gong
YARN-6146 Add Builder methods for TimelineEntityFilters Major timelineserver Rohith Sharma K S Haibo Chen
HADOOP-14120 needless S3AFileSystem.setOptionalPutRequestParameters in S3ABlockOutputStream putObject() Minor fs/s3 Steve Loughran Yuanbo Liu
HADOOP-14135 Remove URI parameter in AWSCredentialProvider constructors Major fs/s3 Mingliang Liu Mingliang Liu
HADOOP-14196 Azure Data Lake doc is missing required config entry Major fs/adl Atul Sikaria Atul Sikaria
HADOOP-14197 Fix ADLS doc for credential provider Major documentation, fs/adl John Zhuge John Zhuge
HADOOP-14230 TestAdlFileSystemContractLive fails to clean up Minor fs/adl, test John Zhuge John Zhuge
HADOOP-14038 Rename ADLS credential properties Minor fs/adl John Zhuge John Zhuge
HDFS-11577 Combine the old and the new chooseRandom for better performance Major namenode Chen Liang Chen Liang
YARN-6357 Implement putEntitiesAsync API in TimelineCollector Major ATSv2, timelineserver Joep Rottinghuis Haibo Chen
YARN-6342 Make TimelineV2Client’s drain timeout after stop configurable Major . Jian He Haibo Chen
YARN-6376 Exceptions caused by synchronous putEntities requests can be swallowed Critical ATSv2 Haibo Chen Haibo Chen
YARN-6414 ATSv2 HBase related tests fail due to guava version upgrade Major timelineserver Sonia Garudi Haibo Chen
YARN-6377 NMTimelinePublisher#serviceStop does not stop timeline clients Major yarn Haibo Chen Haibo Chen
YARN-5952 Create REST API for changing YARN scheduler configurations Major . Jonathan Hung Jonathan Hung
YARN-6109 Add an ability to convert ChildQueue to ParentQueue Major capacity scheduler Xuan Gong Xuan Gong
YARN-6424 TimelineCollector is not stopped when an app finishes in RM Critical timelineserver Varun Saxena Varun Saxena
YARN-5602 Utils for Federation State and Policy Store Major nodemanager, resourcemanager Giovanni Matteo Fumarola Giovanni Matteo Fumarola
HDFS-10882 Federation State Store Interface API Major fs Jason Kace Jason Kace
HADOOP-14290 Update SLF4J from 1.7.10 to 1.7.25 Major . Akira Ajisaka Akira Ajisaka
YARN-6372 Add default value for NM disk validator Major nodemanager Yufei Gu Yufei Gu
HADOOP-14301 Deprecate SharedInstanceProfileCredentialsProvider in branch-2. Major fs/s3 Mingliang Liu Mingliang Liu
HADOOP-14255 S3A to delete unnecessary fake directory objects in mkdirs() Major fs/s3 Mingliang Liu Mingliang Liu
YARN-6040 Introduce api independent PendingAsk to replace usage of ResourceRequest within Scheduler classes Major . Wangda Tan Wangda Tan
YARN-6203 Occasional test failure in TestWeightedRandomRouterPolicy Minor federation Botong Huang Carlo Curino
YARN-6432 FairScheduler: Reserve preempted resources for corresponding applications Major . Miklos Szegedi Miklos Szegedi
HADOOP-14321 Explicitly exclude S3A root dir ITests from parallel runs Minor fs/s3, test Steve Loughran Steve Loughran
HADOOP-14241 Add ADLS sensitive config keys to default list Minor fs, fs/adl, security John Zhuge John Zhuge
HADOOP-14324 Refine S3 server-side-encryption key as encryption secret; improve error reporting and diagnostics Blocker fs/s3 Steve Loughran Steve Loughran
HADOOP-14305 S3A SSE tests won’t run in parallel: Bad request in directory GetFileStatus Minor fs/s3, test Steve Loughran Steve Moist
YARN-3663 Federation State and Policy Store (DBMS implementation) Major nodemanager, resourcemanager Giovanni Matteo Fumarola Giovanni Matteo Fumarola
HADOOP-14349 Rename ADLS CONTRACT_ENABLE_KEY Minor fs/adl Mingliang Liu Mingliang Liu
YARN-6455 Enhance the timelinewriter.flush() race condition fix Major yarn Haibo Chen Haibo Chen
HDFS-7964 Add support for async edit logging Major namenode Daryn Sharp Daryn Sharp
YARN-4359 Update LowCost agents logic to take advantage of YARN-4358 Major capacityscheduler, fairscheduler, resourcemanager Carlo Curino Ishai Menache
YARN-6542 Fix the logger in TestAlignedPlanner and TestGreedyReservationAgent Major reservation system Subru Krishnan Subru Krishnan
YARN-5331 Extend RLESparseResourceAllocation with period for supporting recurring reservations in YARN ReservationSystem Major resourcemanager Subru Krishnan Sangeetha Abdu Jyothi
HDFS-9005 Provide configuration support for upgrade domain Major . Ming Ma Ming Ma
HDFS-9016 Display upgrade domain information in fsck Major . Ming Ma Ming Ma
HDFS-9922 Upgrade Domain placement policy status marks a good block in violation when there are decommissioned nodes Minor . Chris Trezzo Chris Trezzo
HDFS-10630 Federation State Store FS Implementation Major hdfs Íñigo Goiri Jason Kace
YARN-5411 Create a proxy chain for ApplicationClientProtocol in the Router Major nodemanager, resourcemanager Subru Krishnan Giovanni Matteo Fumarola
YARN-6374 Improve test coverage and add utility classes for common Docker operations Major nodemanager, yarn Shane Kumpf Shane Kumpf
YARN-6375 App level aggregation should not consider metric values reported in the previous aggregation cycle Major timelineserver Varun Saxena Varun Saxena
HDFS-11530 Use HDFS specific network topology to choose datanode in BlockPlacementPolicyDefault Major namenode Yiqun Lin Yiqun Lin
YARN-6565 Fix memory leak and finish app trigger in AMRMProxy Critical . Botong Huang Botong Huang
YARN-6234 Support multiple attempts on the node when AMRMProxy is enabled Major amrmproxy, federation, nodemanager Subru Krishnan Giovanni Matteo Fumarola
HADOOP-14384 Reduce the visibility of FileSystem#newFSDataOutputStreamBuilder before the API becomes stable Blocker fs Lei (Eddy) Xu Lei (Eddy) Xu
YARN-6563 ConcurrentModificationException in TimelineCollectorManager while stopping RM Major resourcemanager Rohith Sharma K S Haibo Chen
YARN-5413 Create a proxy chain for ResourceManager Admin API in the Router Major nodemanager, resourcemanager Subru Krishnan Giovanni Matteo Fumarola
YARN-6435 [ATSv2] Can’t retrieve more than 1000 versions of metrics in time series Critical timelineserver Rohith Sharma K S Vrushali C
YARN-6561 Update exception message during timeline collector aux service initialization Minor timelineserver Vrushali C Vrushali C
YARN-6306 NMClient API change for container upgrade Major . Jian He Arun Suresh
HADOOP-11572 s3a delete() operation fails during a concurrent delete of child entries Major fs/s3 Steve Loughran Steve Loughran
YARN-5949 Add pluggable configuration ACL policy interface and implementation Major . Jonathan Hung Jonathan Hung
YARN-6587 Refactor of ResourceManager#startWebApp in a Util class Major nodemanager, resourcemanager Giovanni Matteo Fumarola Giovanni Matteo Fumarola
YARN-6555 Store application flow context in NM state store for work-preserving restart Major timelineserver Vrushali C Rohith Sharma K S
YARN-5531 UnmanagedAM pool manager for federating application across clusters Major nodemanager, resourcemanager Subru Krishnan Botong Huang
HDFS-11446 TestMaintenanceState#testWithNNAndDNRestart fails intermittently Major . Yiqun Lin Yiqun Lin
YARN-6666 Fix unit test failure in TestRouterClientRMService Minor . Botong Huang Botong Huang
YARN-6484 [Documentation] Documenting the YARN Federation feature Major nodemanager, resourcemanager Subru Krishnan Carlo Curino
YARN-6658 Remove columnFor() methods of Columns in HBaseTimeline backend Major . Haibo Chen Haibo Chen
YARN-3666 Federation Intercepting and propagating AM- home RM communications Major nodemanager, resourcemanager Kishore Chaliparambil Botong Huang
YARN-6246 Identifying starved apps does not need the scheduler writelock Major fairscheduler Karthik Kambatla Karthik Kambatla
HDFS-11904 Reuse iip in unprotectedRemoveXAttrs calls Major . Xiao Chen Xiao Chen
HDFS-11359 DFSAdmin report command supports displaying maintenance state datanodes Major datanode, namenode Yiqun Lin Yiqun Lin
YARN-6316 Provide help information and documentation for TimelineSchemaCreator Major timelineserver Li Lu Haibo Chen
YARN-6575 Support global configuration mutation in MutableConfProvider Major . Jonathan Hung Jonathan Hung
HADOOP-14035 Reduce fair call queue backoff’s impact on clients Major ipc Daryn Sharp Daryn Sharp
YARN-6604 Allow metric TTL for Application table to be specified through cmd Major ATSv2 Haibo Chen Haibo Chen
YARN-5647 [ATSv2 Security] Collector side changes for loading auth filters and principals Major timelineserver Varun Saxena Varun Saxena
YARN-6511 Federation: transparently spanning application across multiple sub-clusters Major . Botong Huang Botong Huang
YARN-6679 Reduce Resource instance overhead via non-PBImpl Major resourcemanager Daryn Sharp Daryn Sharp
HADOOP-13174 Add more debug logs for delegation tokens and authentication Minor security Xiao Chen Xiao Chen
HADOOP-13854 KMS should log error details in KMSExceptionsProvider Major kms Xiao Chen Xiao Chen
YARN-6682 Improve performance of AssignmentInformation datastructures Major . Daryn Sharp Daryn Sharp
YARN-6638 [ATSv2 Security] Timeline reader side changes for loading auth filters and principals Major timelineserver Varun Saxena Varun Saxena
YARN-6707 [ATSv2] Update HBase version to 1.2.6 Major timelineserver Varun Saxena Vrushali C
HADOOP-14394 Provide Builder pattern for DistributedFileSystem.create Major fs Lei (Eddy) Xu Lei (Eddy) Xu
HADOOP-14289 Move log4j APIs over to slf4j in hadoop-common Major . Akira Ajisaka Akira Ajisaka
HADOOP-14395 Provide Builder pattern for DistributedFileSystem.append Major fs Lei (Eddy) Xu Lei (Eddy) Xu
HADOOP-14538 Fix TestFilterFileSystem and TestHarFileSystem failures after DistributedFileSystem.append API Major fs Lei (Eddy) Xu Lei (Eddy) Xu
YARN-6680 Avoid locking overhead for NO_LABEL lookups Major resourcemanager Daryn Sharp Daryn Sharp
HADOOP-14296 Move logging APIs over to slf4j in hadoop-tools Major . Akira Ajisaka Akira Ajisaka
YARN-6724 Add ability to blacklist sub-clusters when invoking Routing policies Major router Subru Krishnan Giovanni Matteo Fumarola
HDFS-11998 Enable DFSNetworkTopology as default Major namenode Chen Liang Chen Liang
HADOOP-14542 Add IOUtils.cleanupWithLogger that accepts slf4j logger API Major . Akira Ajisaka Chen Liang
HADOOP-14547 [WASB] the configured retry policy is not used for all storage operations. Major fs/azure Thomas Marquardt Thomas Marquardt
YARN-5648 [ATSv2 Security] Client side changes for authentication Major timelineserver Varun Saxena Varun Saxena
HADOOP-14549 Use GenericTestUtils.setLogLevel when available in hadoop-tools Major . Akira Ajisaka Wenxin He
YARN-3659 Federation: routing client invocations transparently to multiple RMs Major client, resourcemanager, router Giovanni Matteo Fumarola Giovanni Matteo Fumarola
HADOOP-14573 regression: Azure tests which capture logs failing with move to SLF4J Major fs/azure, test Steve Loughran Steve Loughran
HADOOP-14546 Azure: Concurrent I/O does not work when secure.mode is enabled Major fs/azure Thomas Marquardt Thomas Marquardt
HADOOP-14190 add more on s3 regions to the s3a documentation Minor documentation, fs/s3 Steve Loughran Steve Loughran
HADOOP-14609 NPE in AzureNativeFileSystemStore.checkContainer() if StorageException lacks an error code Major fs/azure Steve Loughran Steve Loughran
YARN-5311 Document graceful decommission CLI and usage Major documentation Junping Du Elek, Marton
HADOOP-14596 AWS SDK 1.11+ aborts() on close() if > 0 bytes in stream; logs error Blocker fs/s3 Steve Loughran Steve Loughran
YARN-6681 Eliminate double-copy of child queues in canAssignToThisQueue Major resourcemanager Daryn Sharp Daryn Sharp
HADOOP-14615 Add ServiceOperations.stopQuietly that accept slf4j logger API Major . Wenxin He Wenxin He
HADOOP-14617 Add ReflectionUtils.logThreadInfo that accept slf4j logger API Major . Wenxin He Wenxin He
HADOOP-14571 Deprecate public APIs relate to log4j1 Major . Akira Ajisaka Wenxin He
HADOOP-14587 Use GenericTestUtils.setLogLevel when available in hadoop-common Major . Wenxin He Wenxin He
YARN-5953 Create CLI for changing YARN configurations Major . Jonathan Hung Jonathan Hung
YARN-6776 Refactor ApplicaitonMasterService to move actual processing logic to a separate class Minor . Arun Suresh Arun Suresh
HADOOP-14638 Replace commons-logging APIs with slf4j in StreamPumper Major . Wenxin He Wenxin He
YARN-6801 NPE in RM while setting collectors map in NodeHeartbeatResponse Major timelineserver Vrushali C Vrushali C
YARN-6807 Adding required missing configs to Federation configuration guide based on e2e testing Major documentation, federation Subru Krishnan Tanuj Nayak
YARN-6815 [Bug] FederationStateStoreFacade return behavior should be consistent irrespective of whether caching is enabled or not Major nodemanager, resourcemanager Subru Krishnan Subru Krishnan
YARN-6821 Move FederationStateStore SQL DDL files from test resource to sbin Major nodemanager, resourcemanager Subru Krishnan Subru Krishnan
YARN-2113 Add cross-user preemption within CapacityScheduler’s leaf-queue Major capacity scheduler Vinod Kumar Vavilapalli Sunil Govindan
YARN-5292 NM Container lifecycle and state transitions to support for PAUSED container state. Major . Hitesh Sharma Hitesh Sharma
HADOOP-14642 wasb: add support for caching Authorization and SASKeys Major fs/azure Sivaguru Sankaridurg Sivaguru Sankaridurg
YARN-6777 Support for ApplicationMasterService processing chain of interceptors Major . Arun Suresh Arun Suresh
YARN-6775 CapacityScheduler: Improvements to assignContainers, avoid unnecessary canAssignToUser/Queue calls Major capacityscheduler Nathan Roberts Nathan Roberts
YARN-4455 Support fetching metrics by time range Major timelineserver Varun Saxena Varun Saxena
YARN-6850 Ensure that supplemented timestamp is stored only for flow run metrics Major timelineserver Vrushali C Varun Saxena
YARN-6733 Add table for storing sub-application entities Major timelineserver Vrushali C Vrushali C
HADOOP-14518 Customize User-Agent header sent in HTTP/HTTPS requests by WASB. Minor fs/azure Georgi Chalakov Georgi Chalakov
YARN-6804 Allow custom hostname for docker containers in native services Major yarn-native-services Billie Rinaldi Billie Rinaldi
YARN-6866 Minor clean-up and fixes in anticipation of YARN-2915 merge with trunk Major federation Subru Krishnan Botong Huang
YARN-5412 Create a proxy chain for ResourceManager REST API in the Router Major nodemanager, resourcemanager Subru Krishnan Giovanni Matteo Fumarola
YARN-6888 Refactor AppLevelTimelineCollector such that RM does not have aggregator threads created Major timelineserver Vrushali C Vrushali C
HADOOP-14678 AdlFilesystem#initialize swallows exception when getting user name Minor fs/adl John Zhuge John Zhuge
YARN-6734 Ensure sub-application user is extracted & sent to timeline service Major timelineserver Vrushali C Rohith Sharma K S
HDFS-12223 Rebasing HDFS-10467 Major fs Íñigo Goiri Íñigo Goiri
YARN-6902 Update Microsoft JDBC Driver for SQL Server version in License.txt Minor federation Botong Huang Botong Huang
HDFS-10687 Federation Membership State Store internal API Major hdfs Íñigo Goiri Jason Kace
YARN-5947 Create LeveldbConfigurationStore class using Leveldb as backing store Major . Jonathan Hung Jonathan Hung
HADOOP-14397 Pull up the builder pattern to FileSystem and add AbstractContractCreateTest for it Major common, fs, hdfs-client Lei (Eddy) Xu Lei (Eddy) Xu
YARN-6322 Disable queue refresh when configuration mutation is enabled Major . Jonathan Hung Jonathan Hung
HDFS-11826 Federation Namenode Heartbeat Major fs Íñigo Goiri Íñigo Goiri
YARN-6853 Add MySql Scripts for FederationStateStore Major . Giovanni Matteo Fumarola Giovanni Matteo Fumarola
HDFS-9388 Refactor decommission related code to support maintenance state for datanodes Major . Ming Ma Manoj Govindassamy
YARN-5977 ContainerManagementProtocol changes to support change of container ExecutionType Major . Arun Suresh kartheek muthyala
HADOOP-14126 remove jackson, joda and other transient aws SDK dependencies from hadoop-aws Minor build, fs/s3 Steve Loughran Steve Loughran
HDFS-10880 Federation Mount Table State Store internal API Major fs Jason Kace Íñigo Goiri
YARN-6957 Moving logging APIs over to slf4j in hadoop-yarn-server-sharedcachemanager Major . Yeliang Cang Yeliang Cang
YARN-6955 Handle concurrent register AM requests in FederationInterceptor Minor . Botong Huang Botong Huang
YARN-6873 Moving logging APIs over to slf4j in hadoop-yarn-server-applicationhistoryservice Major . Yeliang Cang Yeliang Cang
YARN-6897 Refactoring RMWebServices by moving some util methods to RMWebAppUtil Major . Giovanni Matteo Fumarola Giovanni Matteo Fumarola
HDFS-10646 Federation admin tool Major fs Íñigo Goiri Íñigo Goiri
HADOOP-14715 TestWasbRemoteCallHelper failing Major fs/azure, test Steve Loughran Esfandiar Manii
YARN-6970 Add PoolInitializationException as retriable exception in FederationFacade Major federation Giovanni Matteo Fumarola Giovanni Matteo Fumarola
YARN-6874 Supplement timestamp for min start/max end time columns in flow run table to avoid overwrite Major timelineserver Varun Saxena Vrushali C
YARN-6958 Moving logging APIs over to slf4j in hadoop-yarn-server-timelineservice Major . Yeliang Cang Yeliang Cang
HADOOP-14183 Remove service loader config file for wasb fs Minor fs/azure John Zhuge Esfandiar Manii
YARN-6130 [ATSv2 Security] Generate a delegation token for AM when app collector is created and pass it to AM via NM and RM Major timelineserver Varun Saxena Varun Saxena
YARN-6133 [ATSv2 Security] Renew delegation token for app automatically if an app collector is active Major timelineserver Varun Saxena Varun Saxena
YARN-6820 Restrict read access to timelineservice v2 data Major timelinereader Vrushali C Vrushali C
YARN-6896 Federation: routing REST invocations transparently to multiple RMs (part 1 - basic execution) Major . Giovanni Matteo Fumarola Giovanni Matteo Fumarola
YARN-6687 Validate that the duration of the periodic reservation is less than the periodicity Major reservation system Subru Krishnan Subru Krishnan
YARN-6905 Multiple HBaseTimelineStorage test failures due to missing FastNumberFormat Major timelineserver Sonia Garudi Haibo Chen
YARN-5978 ContainerScheduler and ContainerManager changes to support ExecType update Major . Arun Suresh kartheek muthyala
YARN-6741 Deleting all children of a Parent Queue on refresh throws exception Major capacity scheduler Naganarasimha G R Naganarasimha G R
HADOOP-14660 wasb: improve throughput by 34% when account limit exceeded Major fs/azure Thomas Marquardt Thomas Marquardt
YARN-7006 [ATSv2 Security] Changes for authentication for CollectorNodemanagerProtocol Major timelineserver Varun Saxena Varun Saxena
YARN-6900 ZooKeeper based implementation of the FederationStateStore Major federation, nodemanager, resourcemanager Subru Krishnan Íñigo Goiri
HDFS-11554 [Documentation] Router-based federation documentation Minor fs Íñigo Goiri Íñigo Goiri
HDFS-12312 Rebasing HDFS-10467 (2) Major . Íñigo Goiri Íñigo Goiri
YARN-7028 Backport HADOOP-14386 to YARN-5355 branch Major timelineserver Vrushali C Vrushali C
YARN-6988 container-executor fails for docker when command length > 4096 B Major yarn Eric Badger Eric Badger
YARN-7038 [Atsv2 Security] CollectorNodemanagerProtocol RPC interface doesn’t work when service authorization is enabled Major . Rohith Sharma K S Varun Saxena
HADOOP-14769 WASB: delete recursive should not fail if a file is deleted Major fs/azure Thomas Marquardt Thomas Marquardt
YARN-7041 Nodemanager NPE running jobs with security off Major timelineserver Aaron Gresch Varun Saxena
YARN-6134 [ATSv2 Security] Regenerate delegation token for app just before token expires if app collector is active Major timelineserver Varun Saxena Varun Saxena
YARN-6979 Add flag to notify all types of container updates to NM via NodeHeartbeatResponse Major . Arun Suresh kartheek muthyala
YARN-6861 Reader API for sub application entities Major timelinereader Rohith Sharma K S Rohith Sharma K S
YARN-6323 Rolling upgrade/config change is broken on timeline v2. Major timelineserver Li Lu Vrushali C
HDFS-10631 Federation State Store ZooKeeper implementation Major fs Íñigo Goiri Jason Kace
YARN-6047 Documentation updates for TimelineService v2 Major documentation, timelineserver Varun Saxena Rohith Sharma K S
MAPREDUCE-6838 [ATSv2 Security] Add timeline delegation token received in allocate response to UGI Major . Varun Saxena Varun Saxena
YARN-3053 [Security] Review and implement authentication in ATS v.2 Major timelineserver Sangjin Lee Varun Saxena
YARN-5603 Metrics for Federation StateStore Major . Subru Krishnan Ellen Hui
YARN-6923 Metrics for Federation Router Major . Giovanni Matteo Fumarola Giovanni Matteo Fumarola
YARN-7024 Fix issues on recovery in LevelDB store Major . Jonathan Hung Jonathan Hung
YARN-7047 Moving logging APIs over to slf4j in hadoop-yarn-server-nodemanager Major . Yeliang Cang Yeliang Cang
YARN-6876 Create an abstract log writer for extendability Major . Xuan Gong Xuan Gong
YARN-7010 Federation: routing REST invocations transparently to multiple RMs (part 2 - getApps) Major . Giovanni Matteo Fumarola Giovanni Matteo Fumarola
YARN-5219 When an export var command fails in launch_container.sh, the full container launch should fail Major . Hitesh Shah Sunil Govindan
HADOOP-14802 Add support for using container saskeys for all accesses Major fs/azure Sivaguru Sankaridurg Sivaguru Sankaridurg
YARN-7094 Document the current known issue with server-side NM graceful decom Blocker graceful Robert Kanter Robert Kanter
YARN-7095 Federation: routing getNode/getNodes/getMetrics REST invocations transparently to multiple RMs Major . Giovanni Matteo Fumarola Giovanni Matteo Fumarola
YARN-6726 Fix issues with docker commands executed by container-executor Major nodemanager Shane Kumpf Shane Kumpf
YARN-6877 Create an abstract log reader for extendability Major . Xuan Gong Xuan Gong
HADOOP-14103 Sort out hadoop-aws contract-test-options.xml Minor fs/s3, test Steve Loughran John Zhuge
YARN-7148 TestLogsCLI fails in trunk and branch-2 and javadoc error Major . Xuan Gong Xuan Gong
YARN-5328 Plan/ResourceAllocation data structure enhancements required to support recurring reservations in ReservationSystem Major resourcemanager Subru Krishnan Subru Krishnan
YARN-6978 Add updateContainer API to NMClient. Major . Arun Suresh kartheek muthyala
HADOOP-14774 S3A case “testRandomReadOverBuffer” failed due to improper range parameter Minor fs/s3 Yonger Yonger
YARN-7144 Log Aggregation controller should not swallow the exceptions when it calls closeWriter and closeReader. Major . Xuan Gong Xuan Gong
HDFS-12384 Fixing compilation issue with BanDuplicateClasses Major fs Íñigo Goiri Íñigo Goiri
YARN-6600 Introduce default and max lifetime of application at LeafQueue level Major capacity scheduler Rohith Sharma K S Rohith Sharma K S
YARN-6849 NMContainerStatus should have the Container ExecutionType. Major . Arun Suresh kartheek muthyala
HDFS-12335 Federation Metrics Major fs Giovanni Matteo Fumarola Íñigo Goiri
YARN-5330 SharingPolicy enhancements required to support recurring reservations in ReservationSystem Major resourcemanager Subru Krishnan Carlo Curino
YARN-7072 Add a new log aggregation file format controller Major . Xuan Gong Xuan Gong
YARN-7173 Container update RM-NM communication fix for backward compatibility Major . Arun Suresh Arun Suresh
YARN-7063 TestTimelineReaderWebServicesHBaseStorage fails with NoClassDefFoundError on TSv2 branch2 Major timelineserver Vrushali C Varun Saxena
YARN-6059 Update paused container state in the NM state store Blocker . Hitesh Sharma Hitesh Sharma
HDFS-12430 Rebasing HDFS-10467 After HDFS-12269 and HDFS-12218 Major fs Íñigo Goiri Íñigo Goiri
YARN-7194 Log aggregation status is always Failed with the newly added log aggregation IndexedFileFormat Major . Xuan Gong Xuan Gong
YARN-7162 Remove XML excludes file format Blocker graceful Robert Kanter Robert Kanter
YARN-7174 Add retry logic in LogsCLI when fetch running application logs Major . Xuan Gong Xuan Gong
HDFS-12450 Fixing TestNamenodeHeartbeat and support non-HA Major fs Íñigo Goiri Íñigo Goiri
YARN-6840 Implement zookeeper based store for scheduler configuration updates Major . Wangda Tan Jonathan Hung
HADOOP-14583 wasb throws an exception if you try to create a file and there’s no parent directory Minor fs/azure Steve Loughran Esfandiar Manii
HDFS-12473 Change hosts JSON file format Major . Ming Ma Ming Ma
HDFS-11035 Better documentation for maintenace mode and upgrade domain Major datanode, documentation Wei-Chiu Chuang Ming Ma
YARN-7046 Add closing logic to configuration store Major . Jonathan Hung Jonathan Hung
YARN-4266 Allow users to enter containers as UID:GID pair instead of by username Major yarn Sidharta Seethana luhuichun
MAPREDUCE-6947 Moving logging APIs over to slf4j in hadoop-mapreduce-examples Major . Gergely Novák Gergely Novák
HDFS-12381 [Documentation] Adding configuration keys for the Router Minor fs Íñigo Goiri Íñigo Goiri
HADOOP-14892 MetricsSystemImpl should use Time.monotonicNow for measuring durations Minor . Chetna Chaudhari Chetna Chaudhari
HADOOP-14881 LoadGenerator should use Time.monotonicNow() to measure durations Major . Chetna Chaudhari Bharat Viswanadham
YARN-7238 Documentation for API based scheduler configuration management Major . Jonathan Hung Jonathan Hung
HADOOP-14893 WritableRpcEngine should use Time.monotonicNow Minor . Chetna Chaudhari Chetna Chaudhari
HADOOP-14890 Move up to AWS SDK 1.11.199 Blocker build, fs/s3 Steve Loughran Steve Loughran
HDFS-12386 Add fsserver defaults call to WebhdfsFileSystem. Minor webhdfs Rushabh S Shah Rushabh S Shah
YARN-7252 Removing queue then failing over results in exception Critical . Jonathan Hung Jonathan Hung
YARN-7251 Misc changes to YARN-5734 Major . Jonathan Hung Jonathan Hung
YARN-6691 Update YARN daemon startup/shutdown scripts to include Router service Major nodemanager, resourcemanager Subru Krishnan Giovanni Matteo Fumarola
YARN-6962 Add support for updateContainers when allocating using FederationInterceptor Minor . Botong Huang Botong Huang
YARN-7250 Update Shared cache client api to use URLs Minor . Chris Trezzo Chris Trezzo
YARN-6509 Add a size threshold beyond which yarn logs will require a force option Major . Siddharth Seth Xuan Gong
HADOOP-14768 Honoring sticky bit during Deletion when authorization is enabled in WASB Major fs/azure Varada Hemeswari Varada Hemeswari
YARN-7259 Add size-based rolling policy to LogAggregationIndexedFileController Major . Xuan Gong Xuan Gong
YARN-6550 Capture launch_container.sh logs to a separate log file Major . Wangda Tan Suma Shivaprasad
HDFS-12580 Rebasing HDFS-10467 after HDFS-12447 Major fs Íñigo Goiri Íñigo Goiri
HADOOP-14220 Enhance S3GuardTool with bucket-info and set-capacity commands, tests Major fs/s3 Steve Loughran Steve Loughran
YARN-2037 Add work preserving restart support for Unmanaged AMs Major resourcemanager Karthik Kambatla Botong Huang
YARN-5329 Placement Agent enhancements required to support recurring reservations in ReservationSystem Blocker resourcemanager Subru Krishnan Carlo Curino
HADOOP-14845 Azure wasb: getFileStatus not making any auth checks Major fs/azure, security Sivaguru Sankaridurg Sivaguru Sankaridurg
YARN-2960 Add documentation for the YARN shared cache Major . Chris Trezzo Chris Trezzo
HDFS-12273 Federation UI Major fs Íñigo Goiri Íñigo Goiri
HDFS-12577 Rename Router tooling Major fs Íñigo Goiri Íñigo Goiri
HADOOP-14899 Restrict Access to setPermission operation when authorization is enabled in WASB Major fs/azure Kannapiran Srinivasan Kannapiran Srinivasan
YARN-7305 Merge YARN-5734 branch to branch-2 Major . Xuan Gong Xuan Gong
HADOOP-14913 Sticky bit implementation for rename() operation in Azure WASB Major fs, fs/azure Varada Hemeswari Varada Hemeswari
YARN-6033 Add support for sections in container-executor configuration file Major nodemanager Varun Vasudev Varun Vasudev
HADOOP-14935 Azure: POSIX permissions are taking effect in access() method even when authorization is enabled Major fs/azure Santhosh G Nayak Santhosh G Nayak
YARN-7275 NM Statestore cleanup for Container updates Blocker . Arun Suresh kartheek muthyala
YARN-7311 Fix TestRMWebServicesReservation parametrization for fair scheduler Blocker fairscheduler, reservation system Yufei Gu Yufei Gu
YARN-7295 Backport ATSv2 to branch-2 Critical . Varun Saxena Varun Saxena
YARN-7338 Support same origin policy for cross site scripting prevention. Major yarn-ui-v2 Vrushali C Sunil Govindan
HDFS-12620 Backporting HDFS-10467 to branch-2 Major . Íñigo Goiri Íñigo Goiri
YARN-4090 Make Collections.sort() more efficient by caching resource usage Major fairscheduler Xianyin Xin Yufei Gu
YARN-7353 Docker permitted volumes don’t properly check for directories Major yarn Eric Badger Eric Badger
YARN-7169 Backport new yarn-ui to branch2 code (starting with YARN-5355_branch2) Critical timelineclient, timelinereader, timelineserver Vrushali C Vrushali C
YARN-3661 Basic Federation UI Major nodemanager, resourcemanager Giovanni Matteo Fumarola Íñigo Goiri
YARN-4827 Document configuration of ReservationSystem for FairScheduler Blocker capacity scheduler Subru Krishnan Yufei Gu
YARN-5516 Add REST API for supporting recurring reservations Major resourcemanager Sangeetha Abdu Jyothi Sean Po
YARN-7178 Add documentation for Container Update API Blocker . Arun Suresh Arun Suresh
YARN-7380 Fix findbugs warning in timeline service on branch-2 Major timelineclient, timelinereader, timelineserver Vrushali C Vrushali C
YARN-7378 Documentation changes post branch-2 merge Major timelineclient, timelinereader, timelineserver Varun Saxena Vrushali C
YARN-7398 Update LICENSE.txt with respect to new YARN UI merge Blocker . Subru Krishnan Varun Saxena
YARN-7276 Federation Router Web Service fixes Major . Íñigo Goiri Íñigo Goiri
YARN-7289 Application lifetime does not work with FairScheduler Major resourcemanager Miklos Szegedi Miklos Szegedi
YARN-7286 Add support for docker to have no capabilities Major yarn Eric Badger Eric Badger
YARN-6627 Use deployed webapp folder to launch new YARN UI Major yarn-ui-v2 Sunil Govindan Sunil Govindan
HDFS-9754 Avoid unnecessary getBlockCollection calls in BlockManager Major namenode Jing Zhao Jing Zhao

OTHER:

JIRA Summary Priority Component Reporter Contributor
YARN-4535 Fix checkstyle error in CapacityScheduler.java Trivial . Rohith Sharma K S Naganarasimha G R
YARN-5297 Avoid printing a stack trace when recovering an app after the RM restarts Major . Siddharth Seth Junping Du
YARN-5717 Add tests for container-executor’s is_feature_enabled function Major yarn Sidharta Seethana Sidharta Seethana
YARN-5308 FairScheduler: Move continuous scheduling related tests to TestContinuousScheduling Major fairscheduler, test Karthik Kambatla Kai Sasaki
YARN-5822 Log ContainerRuntime initialization error in LinuxContainerExecutor Trivial nodemanager Sidharta Seethana Sidharta Seethana
YARN-5646 Add documentation and update config parameter names for scheduling of OPPORTUNISTIC containers Blocker . Konstantinos Karanasos Konstantinos Karanasos
YARN-6411 Clean up the overwrite of createDispatcher() in subclass of MockRM Minor resourcemanager Yufei Gu Yufei Gu
HADOOP-14344 Revert HADOOP-13606 swift FS to add a service load metadata file Major . John Zhuge John Zhuge
HDFS-11717 Add unit test for HDFS-11709 StandbyCheckpointer should handle non-existing legacyOivImageDir gracefully Minor ha, namenode Erik Krogen Erik Krogen
YARN-6969 Clean up unused code in class FairSchedulerQueueInfo Trivial fairscheduler Yufei Gu Larry Lo
YARN-6622 Document Docker work as experimental Blocker documentation Varun Vasudev Varun Vasudev
YARN-7203 Add container ExecutionType into ContainerReport Minor . Botong Huang Botong Huang
YARN-6499 Remove the doc about Schedulable#redistributeShare() Trivial fairscheduler Yufei Gu Chetna Chaudhari
HADOOP-13835 Move Google Test Framework code from mapreduce to hadoop-common Major test Varun Vasudev Varun Vasudev
YARN-7317 Fix overallocation resulted from ceiling in LocalityMulticastAMRMProxyPolicy Minor . Botong Huang Botong Huang