Apache Hadoop Changelog

Release 2.4.0 - 2014-04-07

INCOMPATIBLE CHANGES:

JIRA Summary Priority Component Reporter Contributor
HADOOP-8691 FsShell can print “Found xxx items” unnecessarily often Minor fs Jason Lowe Daryn Sharp
HDFS-6102 Lower the default maximum items per directory to fix PB fsimage loading Blocker namenode Andrew Wang Andrew Wang
HDFS-6055 Change default configuration to limit file name length in HDFS Major namenode Suresh Srinivas Chris Nauroth
HDFS-5804 HDFS NFS Gateway fails to mount and proxy when using Kerberos Major nfs Abin Shahab Abin Shahab
HDFS-5321 Clean up the HTTP-related configuration in HDFS Major . Haohui Mai Haohui Mai
HDFS-5138 Support HDFS upgrade in HA Blocker . Kihwal Lee Aaron T. Myers
MAPREDUCE-5036 Default shuffle handler port should not be 8080 Major . Sandy Ryza Sandy Ryza

IMPORTANT ISSUES:

JIRA Summary Priority Component Reporter Contributor

NEW FEATURES:

JIRA Summary Priority Component Reporter Contributor
HADOOP-10184 Hadoop Common changes required to support HDFS ACLs. Major fs, security Chris Nauroth Chris Nauroth
HDFS-5535 Umbrella jira for improved HDFS rolling upgrades Major datanode, ha, hdfs-client, namenode Nathan Roberts Tsz Wo Nicholas Sze
HDFS-4685 Implementation of ACLs in HDFS Major hdfs-client, namenode, security Sachin Jose Chris Nauroth

IMPROVEMENTS:

JIRA Summary Priority Component Reporter Contributor
HADOOP-10423 Clarify compatibility policy document for combination of new client and old server. Minor documentation Chris Nauroth Chris Nauroth
HADOOP-10386 Log proxy hostname in various exceptions being thrown in a HA setup Minor ha Arpit Gupta Haohui Mai
HADOOP-10383 InterfaceStability annotations should have RetentionPolicy.RUNTIME Major . Enis Soztutar Enis Soztutar
HADOOP-10379 Protect authentication cookies with the HttpOnly and Secure flags Major . Haohui Mai Haohui Mai
HADOOP-10374 InterfaceAudience annotations should have RetentionPolicy.RUNTIME Major . Enis Soztutar Enis Soztutar
HADOOP-10348 Deprecate hadoop.ssl.configuration in branch-2, and remove it in trunk Major . Haohui Mai Haohui Mai
HADOOP-10343 Change info to debug log in LossyRetryInvocationHandler Minor . Arpit Gupta Arpit Gupta
HADOOP-10333 Fix grammatical error in overview.html document Trivial . René Nyffenegger René Nyffenegger
HADOOP-10295 Allow distcp to automatically identify the checksum type of source files and use it for the target Major tools/distcp Jing Zhao Jing Zhao
HADOOP-10221 Add a plugin to specify SaslProperties for RPC protocol based on connection properties Major security Benoy Antony Benoy Antony
HADOOP-10211 Enable RPC protocol to negotiate SASL-QOP values between clients and servers Major security Benoy Antony Benoy Antony
HADOOP-10139 Update and improve the Single Cluster Setup document Major documentation Akira AJISAKA Akira AJISAKA
HADOOP-9454 Support multipart uploads for s3native Major fs/s3 Jordan Mendelson Akira AJISAKA
HDFS-6150 Add inode id information in the logs to make debugging easier Minor namenode Suresh Srinivas Suresh Srinivas
HDFS-6138 User Guide for how to use viewfs with federation Minor documentation Sanjay Radia Sanjay Radia
HDFS-6123 Improve datanode error messages Minor datanode Tsz Wo Nicholas Sze Tsz Wo Nicholas Sze
HDFS-6120 Fix and improve safe mode log messages Major namenode Arpit Agarwal Arpit Agarwal
HDFS-6090 Use MiniDFSCluster.Builder instead of deprecated constructors Minor test Akira AJISAKA Akira AJISAKA
HDFS-6085 Improve CacheReplicationMonitor log messages a bit Major namenode Colin Patrick McCabe Colin Patrick McCabe
HDFS-6084 Namenode UI - “Hadoop” logo link shouldn’t go to hadoop homepage Minor namenode Travis Thompson Travis Thompson
HDFS-6080 Improve NFS gateway performance by making rtmax and wtmax configurable Major nfs, performance Abin Shahab Abin Shahab
HDFS-6072 Clean up dead code of FSImage Major . Haohui Mai Haohui Mai
HDFS-6070 Cleanup use of ReadStatistics in DFSInputStream Trivial . Andrew Wang Andrew Wang
HDFS-6069 Quash stack traces when ACLs are disabled Trivial namenode Andrew Wang Chris Nauroth
HDFS-6044 Add property for setting the NFS look up time for users Minor nfs Brandon Li Brandon Li
HDFS-6043 Give HDFS daemons NFS3 and Portmap their own OPTS Major nfs Brandon Li Brandon Li
HDFS-6018 Exception recorded in LOG when IPCLoggerChannel#close is called Trivial . Jing Zhao Jing Zhao
HDFS-6006 Remove duplicate code in FSNameSystem#getFileInfo Trivial namenode Akira AJISAKA Akira AJISAKA
HDFS-5979 Typo and logger fix for fsimage PB code Minor . Andrew Wang Andrew Wang
HDFS-5939 WebHdfs returns misleading error code and logs nothing if trying to create a file with no DNs in cluster Major hdfs-client Yongjun Zhang Yongjun Zhang
HDFS-5935 New Namenode UI FS browser should throw smarter error messages Minor namenode Travis Thompson Travis Thompson
HDFS-5929 Add Block pool % usage to HDFS federated nn page Major federation Siqi Li Siqi Li
HDFS-5910 Enhance DataTransferProtocol to allow per-connection choice of encryption/plain-text Major security Benoy Antony Benoy Antony
HDFS-5908 Change AclFeature to capture list of ACL entries in an ImmutableList. Minor namenode Chris Nauroth Chris Nauroth
HDFS-5781 Use an array to record the mapping between FSEditLogOpCode and the corresponding byte value Minor namenode Jing Zhao Jing Zhao
HDFS-5776 Support ‘hedged’ reads in DFSClient Major hdfs-client Liang Xie Liang Xie
HDFS-5775 Consolidate the code for serialization in CacheManager Major namenode Haohui Mai Haohui Mai
HDFS-5768 Consolidate the serialization code in DelegationTokenSecretManager Major namenode Haohui Mai Haohui Mai
HDFS-5709 Improve NameNode upgrade with existing reserved paths and path components Major namenode Andrew Wang Andrew Wang
HDFS-5698 Use protobuf to serialize / deserialize FSImage Major namenode Haohui Mai Haohui Mai
HDFS-5318 Support read-only and read-write paths to shared replicas Major namenode Eric Sirianni
HDFS-5153 Datanode should send block reports for each storage in a separate message Major datanode Arpit Agarwal Arpit Agarwal
HDFS-4911 Reduce PeerCache timeout to be commensurate with dfs.datanode.socket.reuse.keepalive Minor . Colin Patrick McCabe Colin Patrick McCabe
HDFS-4370 Fix typo Blanacer in DataNode Major datanode Konstantin Shvachko Chu Tong
HDFS-4200 Reduce the size of synchronized sections in PacketResponder Major datanode Suresh Srinivas Andrew Wang
HDFS-3405 Checkpointing should use HTTP POST or PUT instead of GET-GET to send merged fsimages Major . Aaron T. Myers Vinayakumar B
MAPREDUCE-5773 Provide dedicated MRAppMaster syslog length limit Blocker mr-am Gera Shegalov Gera Shegalov
MAPREDUCE-5761 Add a log message like “encrypted shuffle is ON” in nodemanager logs Trivial . Yesha Vora Jian He
MAPREDUCE-5754 Preserve Job diagnostics in history Major jobhistoryserver, mr-am Gera Shegalov Gera Shegalov
MAPREDUCE-5732 Report proper queue when job has been automatically placed Major . Sandy Ryza Sandy Ryza
MAPREDUCE-5553 Add task state filters on Application/MRJob page for MR Application master Minor applicationmaster Paul Han Paul Han
MAPREDUCE-2349 speed up list[located]status calls from input formats Major task Joydeep Sen Sarma Siddharth Seth
YARN-1789 ApplicationSummary does not escape newlines in the app name Minor resourcemanager Akira AJISAKA Tsuyoshi Ozawa
YARN-1771 many getFileStatus calls made from node manager for localizing a public distributed cache resource Critical nodemanager Sangjin Lee Sangjin Lee
YARN-1570 Formatting the lines within 80 chars in YarnCommands.apt.vm Minor documentation Akira AJISAKA Akira AJISAKA
YARN-1536 Cleanup: Get rid of ResourceManager#get*SecretManager() methods and use the RMContext methods instead Minor resourcemanager Karthik Kambatla Anubhav Dhoot
YARN-1512 Enhance CS to decouple scheduling from node heartbeats Major . Arun C Murthy Arun C Murthy
YARN-1171 Add default queue properties to Fair Scheduler documentation Major documentation, scheduler Sandy Ryza Naren Koneru

BUG FIXES:

JIRA Summary Priority Component Reporter Contributor
HADOOP-10450 Build zlib native code bindings in hadoop.dll for Windows. Major io, native Chris Nauroth Chris Nauroth
HADOOP-10442 Group look-up can cause segmentation fault when certain JNI-based mapping module is used. Blocker . Kihwal Lee Kihwal Lee
HADOOP-10441 Namenode metric “rpc.RetryCache/NameNodeRetryCache.CacheHit” can’t be correctly processed by Ganglia Blocker metrics Jing Zhao Jing Zhao
HADOOP-10440 HarFsInputStream of HarFileSystem, when reading data, computing the position has bug Major fs guodongdong guodongdong
HADOOP-10425 Incompatible behavior of LocalFileSystem:getContentSummary Critical fs Brandon Li Tsz Wo Nicholas Sze
HADOOP-10422 Remove redundant logging of RPC retry attempts. Minor ipc Chris Nauroth Chris Nauroth
HADOOP-10395 TestCallQueueManager is flaky Minor test Arpit Agarwal Arpit Agarwal
HADOOP-10394 TestAuthenticationFilter is flaky Major test Arpit Agarwal Arpit Agarwal
HADOOP-10368 InputStream is not closed in VersionInfo ctor Minor util Ted Yu Tsuyoshi Ozawa
HADOOP-10361 Correct alignment in CLI output for ACLs. Minor fs Chris Nauroth Chris Nauroth
HADOOP-10355 TestLoadGenerator#testLoadGenerator fails Major . Akira AJISAKA Haohui Mai
HADOOP-10354 TestWebHDFS fails after merge of HDFS-4685 to trunk Major fs Yongjun Zhang Chris Nauroth
HADOOP-10353 FsUrlStreamHandlerFactory is not thread safe Major fs Tudor Scurtu Tudor Scurtu
HADOOP-10352 Recursive setfacl erroneously attempts to apply default ACL to files. Major fs Chris Nauroth Chris Nauroth
HADOOP-10346 Deadlock while logging tokens Blocker security Jason Lowe Jason Lowe
HADOOP-10338 Cannot get the FileStatus of the root inode from the new Globber Major . Andrew Wang Colin Patrick McCabe
HADOOP-10337 ConcurrentModificationException from MetricsDynamicMBeanBase.createMBeanInfo() Major metrics Liang Xie Liang Xie
HADOOP-10330 TestFrameDecoder fails if it cannot bind port 12345 Major test Arpit Agarwal Arpit Agarwal
HADOOP-10328 loadGenerator exit code is not reliable Major tools Arpit Gupta Haohui Mai
HADOOP-10327 Trunk windows build broken after HDFS-5746 Blocker native Vinayakumar B Vinayakumar B
HADOOP-10326 M/R jobs can not access S3 if Kerberos is enabled Major security Manuel DE FERRAN bc Wong
HADOOP-10320 Javadoc in InterfaceStability.java lacks final </ul> Trivial documentation René Nyffenegger René Nyffenegger
HADOOP-10314 The ls command help still shows outdated 0.16 format. Major . Kihwal Lee Rushabh S Shah
HADOOP-10301 AuthenticationFilter should return Forbidden for failed authentication Blocker security Daryn Sharp Daryn Sharp
HADOOP-10249 LdapGroupsMapping should trim ldap password read from file Major . Dilli Arumugam Dilli Arumugam
HADOOP-10191 Missing executable permission on viewfs internal dirs Blocker viewfs Gera Shegalov Gera Shegalov
HADOOP-10085 CompositeService should allow adding services while being inited Blocker . Karthik Kambatla Steve Loughran
HADOOP-10070 RPC client doesn’t use per-connection conf to determine server’s expected Kerberos principal name Major security Aaron T. Myers Aaron T. Myers
HADOOP-10015 UserGroupInformation prints out excessive ERROR warnings Minor security Haohui Mai Nicolas Liochon
HDFS-6237 TestDFSShell#testGet fails on Windows due to invalid file system path. Trivial hdfs-client, test Chris Nauroth Chris Nauroth
HDFS-6166 revisit balancer so_timeout Blocker balancer & mover Nathan Roberts Nathan Roberts
HDFS-6163 Fix a minor bug in the HA upgrade document Minor documentation Fengdong Yu Fengdong Yu
HDFS-6157 Fix the entry point of OfflineImageViewer for hdfs.cmd Major . Haohui Mai Haohui Mai
HDFS-6140 WebHDFS cannot create a file with spaces in the name after HA failover changes. Major webhdfs Chris Nauroth Chris Nauroth
HDFS-6135 In HDFS upgrade with HA setup, JournalNode cannot handle layout version bump when rolling back Blocker journal-node Jing Zhao Jing Zhao
HDFS-6131 Move HDFSHighAvailabilityWithNFS.apt.vm and HDFSHighAvailabilityWithQJM.apt.vm from Yarn to HDFS Major documentation Jing Zhao Jing Zhao
HDFS-6130 NPE when upgrading namenode from fsimages older than -32 Blocker namenode Fengdong Yu Haohui Mai
HDFS-6129 When a replica is not found for deletion, do not throw exception. Minor datanode Tsz Wo Nicholas Sze Tsz Wo Nicholas Sze
HDFS-6127 WebHDFS tokens cannot be renewed in HA setup Major ha Arpit Gupta Haohui Mai
HDFS-6117 Print file path information in FileNotFoundException Minor namenode Suresh Srinivas Suresh Srinivas
HDFS-6115 flush() should be called for every append on block scan verification log Minor datanode Vinayakumar B Vinayakumar B
HDFS-6107 When a block can’t be cached due to limited space on the DataNode, that block becomes uncacheable Major datanode Colin Patrick McCabe Colin Patrick McCabe
HDFS-6106 Reduce default for dfs.namenode.path.based.cache.refresh.interval.ms Major . Colin Patrick McCabe Colin Patrick McCabe
HDFS-6105 NN web UI for DN list loads the same jmx page multiple times. Major . Kihwal Lee Haohui Mai
HDFS-6100 DataNodeWebHdfsMethods does not failover in HA mode Major ha Arpit Gupta Haohui Mai
HDFS-6099 HDFS file system limits not enforced on renames. Major namenode Chris Nauroth Chris Nauroth
HDFS-6097 zero-copy reads are incorrectly disabled on file offsets above 2GB Major hdfs-client Colin Patrick McCabe Colin Patrick McCabe
HDFS-6096 TestWebHdfsTokens may timeout Minor test Tsz Wo Nicholas Sze Tsz Wo Nicholas Sze
HDFS-6094 The same block can be counted twice towards safe mode threshold Major namenode Arpit Agarwal Arpit Agarwal
HDFS-6089 Standby NN while transitioning to active throws a connection refused error when the prior active NN process is suspended Major ha Arpit Gupta Jing Zhao
HDFS-6079 Timeout for getFileBlockStorageLocations does not work Major hdfs-client Andrew Wang Andrew Wang
HDFS-6078 TestIncrementalBlockReports is flaky Minor test Arpit Agarwal Arpit Agarwal
HDFS-6077 running slive with webhdfs on secure HA cluster fails with unkown host exception Major . Arpit Gupta Jing Zhao
HDFS-6071 BlockReaderLocal doesn’t return -1 on EOF when doing a zero-length read on a short file Major . Colin Patrick McCabe Colin Patrick McCabe
HDFS-6068 Disallow snapshot names that are also invalid directory names Major snapshots Andrew Wang sathish
HDFS-6067 TestPread.testMaxOutHedgedReadPool is flaky Major hdfs-client Colin Patrick McCabe Colin Patrick McCabe
HDFS-6065 HDFS zero-copy reads should return null on EOF when doing ZCR Major hdfs-client Colin Patrick McCabe Colin Patrick McCabe
HDFS-6064 DFSConfigKeys.DFS_BLOCKREPORT_INTERVAL_MSEC_DEFAULT is not updated with latest block report interval of 6 hrs Minor datanode Vinayakumar B Vinayakumar B
HDFS-6063 TestAclCLI fails intermittently when running test 24: copyFromLocal Minor test, tools Colin Patrick McCabe Chris Nauroth
HDFS-6062 TestRetryCacheWithHA#testConcat is flaky Minor . Jing Zhao Jing Zhao
HDFS-6059 TestBlockReaderLocal fails if native library is not available Major . Akira AJISAKA Akira AJISAKA
HDFS-6058 Fix TestHDFSCLI failures after HADOOP-8691 change Major . Vinayakumar B Haohui Mai
HDFS-6057 DomainSocketWatcher.watcherThread should be marked as daemon thread Blocker hdfs-client Eric Sirianni Colin Patrick McCabe
HDFS-6053 Fix TestDecommissioningStatus and TestDecommission in branch-2 Major namenode Jing Zhao Jing Zhao
HDFS-6051 HDFS cannot run on Windows since short-circuit shared memory segment changes. Blocker hdfs-client Chris Nauroth Colin Patrick McCabe
HDFS-6047 TestPread NPE inside in DFSInputStream hedgedFetchBlockByteRange Major . stack stack
HDFS-6039 Uploading a File under a Dir with default acls throws “Duplicated ACLFeature” Major namenode Yesha Vora Chris Nauroth
HDFS-6033 PBImageXmlWriter incorrectly handles processing cache directives Major caching Aaron T. Myers Aaron T. Myers
HDFS-6028 Print clearer error message when user attempts to delete required mask entry from ACL. Trivial namenode Chris Nauroth Chris Nauroth
HDFS-6008 Namenode dead node link is giving HTTP error 500 Minor namenode Benoy Antony Benoy Antony
HDFS-5988 Bad fsimage always generated after upgrade Blocker namenode Andrew Wang Andrew Wang
HDFS-5982 Need to update snapshot manager when applying editlog for deleting a snapshottable directory Critical namenode Tassapol Athiapinya Jing Zhao
HDFS-5981 PBImageXmlWriter generates malformed XML Minor tools Haohui Mai Haohui Mai
HDFS-5962 Mtime and atime are not persisted for symbolic links Critical . Kihwal Lee Akira AJISAKA
HDFS-5961 OIV cannot load fsimages containing a symbolic link Critical . Kihwal Lee Kihwal Lee
HDFS-5959 Fix typo at section name in FSImageFormatProtobuf.java Minor . Akira AJISAKA Akira AJISAKA
HDFS-5949 New Namenode UI when trying to download a file, the browser doesn’t know the file name Minor namenode Travis Thompson Travis Thompson
HDFS-5948 TestBackupNode flakes with port in use error Major . Andrew Wang Haohui Mai
HDFS-5944 LeaseManager:findLeaseWithPrefixPath can’t handle path like /a/b/ right and cause SecondaryNameNode failed do checkpoint Major namenode zhaoyunjiong zhaoyunjiong
HDFS-5943 ‘dfs.namenode.https-address.ns1’ property is not used in federation setup Major . Yesha Vora Suresh Srinivas
HDFS-5941 add dfs.namenode.secondary.https-address and dfs.namenode.secondary.https-address in hdfs-default.xml Major documentation, namenode Haohui Mai Haohui Mai
HDFS-5934 New Namenode UI back button doesn’t work as expected Minor namenode Travis Thompson Travis Thompson
HDFS-5922 DN heartbeat thread can get stuck in tight loop Major datanode Aaron T. Myers Arpit Agarwal
HDFS-5915 Refactor FSImageFormatProtobuf to simplify cross section reads Major namenode Haohui Mai Haohui Mai
HDFS-5913 Nfs3Utils#getWccAttr() should check attr parameter against null Minor nfs Ted Yu Brandon Li
HDFS-5904 TestFileStatus fails intermittently on trunk and branch2 Major . Mit Desai Mit Desai
HDFS-5901 NameNode new UI doesn’t support IE8 and IE9 on windows 7 Major namenode Vinayakumar B Vinayakumar B
HDFS-5900 Cannot set cache pool limit of “unlimited” via CacheAdmin Major caching Tassapol Athiapinya Andrew Wang
HDFS-5895 HDFS cacheadmin -listPools has exit_code of 1 when the command returns 0 result. Major tools Tassapol Athiapinya Tassapol Athiapinya
HDFS-5893 HftpFileSystem.RangeHeaderUrlOpener uses the default URLConnectionFactory which does not import SSL certificates Major . Yesha Vora Haohui Mai
HDFS-5891 webhdfs should not try connecting the DN during redirection Major namenode, webhdfs Haohui Mai Haohui Mai
HDFS-5886 Potential null pointer deference in RpcProgramNfs3#readlink() Major nfs Ted Yu Brandon Li
HDFS-5881 Fix skip() of the short-circuit local reader (legacy). Critical . Kihwal Lee Kihwal Lee
HDFS-5879 Some TestHftpFileSystem tests do not close streams Major test Gera Shegalov Gera Shegalov
HDFS-5859 DataNode#checkBlockToken should check block tokens even if security is not enabled Major datanode Colin Patrick McCabe Colin Patrick McCabe
HDFS-5857 TestWebHDFS#testNamenodeRestart fails intermittently with NPE Major . Mit Desai Mit Desai
HDFS-5856 DataNode.checkDiskError might throw NPE Minor datanode Josh Elser Josh Elser
HDFS-5846 Assigning DEFAULT_RACK in resolveNetworkLocation method can break data resiliency Major namenode Nikola Vujic Nikola Vujic
HDFS-5843 DFSClient.getFileChecksum() throws IOException if checksum is disabled Major datanode Laurent Goujon Laurent Goujon
HDFS-5840 Follow-up to HDFS-5138 to improve error handling during partial upgrade failures Blocker ha, journal-node, namenode Aaron T. Myers Jing Zhao
HDFS-5828 BlockPlacementPolicyWithNodeGroup can place multiple replicas on the same node group when dfs.namenode.avoid.write.stale.datanode is true Major namenode Taylor, Buddy Taylor, Buddy
HDFS-5821 TestHDFSCLI fails for user names with the dash character Major test Gera Shegalov Gera Shegalov
HDFS-5807 TestBalancerWithNodeGroup.testBalancerWithNodeGroup fails intermittently on Branch-2 Major test Mit Desai Chen He
HDFS-5803 TestBalancer.testBalancer0 fails Major . Mit Desai Chen He
HDFS-5791 TestHttpsFileSystem should use a random port to avoid binding error during testing Major test Brandon Li Haohui Mai
HDFS-5790 LeaseManager.findPath is very slow when many leases need recovery Major namenode, performance Todd Lipcon Todd Lipcon
HDFS-5780 TestRBWBlockInvalidation times out intemittently on branch-2 Major . Mit Desai Mit Desai
HDFS-5767 NFS implementation assumes userName userId mapping to be unique, which is not true sometimes Blocker nfs Yongjun Zhang Yongjun Zhang
HDFS-5759 Web UI does not show up during the period of loading FSImage Major . Haohui Mai Haohui Mai
HDFS-5742 DatanodeCluster (mini cluster of DNs) fails to start Minor test Arpit Agarwal Arpit Agarwal
HDFS-5716 Allow WebHDFS to use pluggable authentication filter Major webhdfs Haohui Mai Haohui Mai
HDFS-5705 TestSecondaryNameNodeUpgrade#testChangeNsIDFails may fail due to ConcurrentModificationException Major datanode Ted Yu Ted Yu
HDFS-5626 dfsadmin -report shows incorrect cache values Major caching Stephen Chu Colin Patrick McCabe
HDFS-5516 WebHDFS does not require user name when anonymous http requests are disallowed. Major webhdfs Chris Nauroth Miodrag Radulovic
HDFS-5492 Port HDFS-2069 (Incorrect default trash interval in the docs) to trunk Minor documentation Akira AJISAKA Akira AJISAKA
HDFS-5339 WebHDFS URI does not accept logical nameservices when security is enabled Major webhdfs Stephen Chu Haohui Mai
HDFS-5244 TestNNStorageRetentionManager#testPurgeMultipleDirs fails Major test Jinghui Wang Jinghui Wang
HDFS-5064 Standby checkpoints should not block concurrent readers Major ha, namenode Aaron T. Myers Aaron T. Myers
HDFS-4858 HDFS DataNode to NameNode RPC should timeout Minor datanode Jagane Sundar Henry Wang
HDFS-3969 Small bug fixes and improvements for disk locations API Major hdfs-client Todd Lipcon Todd Lipcon
HDFS-3128 Unit tests should not use a test root in /tmp Minor test Eli Collins Andrew Wang
MAPREDUCE-5813 YarnChild does not load job.xml with mapreduce.job.classloader=true Blocker mrv2, task Gera Shegalov Gera Shegalov
MAPREDUCE-5810 TestStreamingTaskLog#testStreamingTaskLogWithHadoopCmd is failing Major contrib/streaming Mit Desai Akira AJISAKA
MAPREDUCE-5806 Log4j settings in container-log4j.properties cannot be overridden Major . Eugene Koifman Varun Vasudev
MAPREDUCE-5805 Unable to parse launch time from job history file Major jobhistoryserver Fengdong Yu Akira AJISAKA
MAPREDUCE-5795 Job should be marked as Failed if it is recovered from commit. Major . Yesha Vora Xuan Gong
MAPREDUCE-5794 SliveMapper always uses default FileSystem. Minor test Tsz Wo Nicholas Sze Tsz Wo Nicholas Sze
MAPREDUCE-5791 Shuffle phase is slow in Windows - FadviseFileRegion::transferTo does not read disks efficiently Major client Nikola Vujic Nikola Vujic
MAPREDUCE-5789 Average Reduce time is incorrect on Job Overview page Major jobhistoryserver, webapps Rushabh S Shah Rushabh S Shah
MAPREDUCE-5780 SliveTest always uses default FileSystem Minor test Tsz Wo Nicholas Sze Tsz Wo Nicholas Sze
MAPREDUCE-5778 JobSummary does not escape newlines in the job name Major jobhistoryserver Jason Lowe Akira AJISAKA
MAPREDUCE-5770 Redirection from AM-URL is broken with HTTPS_ONLY policy Major . Yesha Vora Jian He
MAPREDUCE-5769 Unregistration to RM should not be called if AM is crashed before registering with RM Major . Rohith Sharma K S Rohith Sharma K S
MAPREDUCE-5768 TestMRJobs.testContainerRollingLog fails on trunk Major . Zhijie Shen Gera Shegalov
MAPREDUCE-5766 Ping messages from attempts should be moved to DEBUG Minor applicationmaster Ramya Sunil Jian He
MAPREDUCE-5757 ConcurrentModificationException in JobControl.toList Major client Jason Lowe Jason Lowe
MAPREDUCE-5751 MR app master fails to start in some cases if mapreduce.job.classloader is true Major . Sangjin Lee Sangjin Lee
MAPREDUCE-5746 Job diagnostics can implicate wrong task for a failed job Major jobhistoryserver Jason Lowe Jason Lowe
MAPREDUCE-5699 Allow setting tags on MR jobs Major applicationmaster Karthik Kambatla Karthik Kambatla
MAPREDUCE-5688 TestStagingCleanup fails intermittently with JDK7 Major . Mit Desai Mit Desai
MAPREDUCE-5670 CombineFileRecordReader should report progress when moving to the next file Minor mrv2 Jason Lowe Chen He
MAPREDUCE-5570 Map task attempt with fetch failure has incorrect attempt finish time Major mr-am, mrv2 Jason Lowe Rushabh S Shah
MAPREDUCE-5028 Maps fail when io.sort.mb is set to high value Critical . Karthik Kambatla Karthik Kambatla
MAPREDUCE-4052 Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster. Major job submission xieguiming Jian He
YARN-1873 TestDistributedShell#testDSShell fails when the test cases are out of order Major . Mit Desai Mit Desai
YARN-1867 NPE while fetching apps via the REST API Blocker resourcemanager Karthik Kambatla Vinod Kumar Vavilapalli
YARN-1866 YARN RM fails to load state store with delegation token parsing error Blocker . Arpit Gupta Jian He
YARN-1859 WebAppProxyServlet will throw ApplicationNotFoundException if the app is no longer cached in RM Major . Zhijie Shen Zhijie Shen
YARN-1852 Application recovery throws InvalidStateTransitonException for FAILED and KILLED jobs Major resourcemanager Rohith Sharma K S Rohith Sharma K S
YARN-1849 NPE in ResourceTrackerService#registerNodeManager for UAM Blocker resourcemanager Karthik Kambatla Karthik Kambatla
YARN-1846 TestRM#testNMTokenSentForNormalContainer assumes CapacityScheduler Major . Robert Kanter Robert Kanter
YARN-1839 Capacity scheduler preempts an AM out. AM attempt 2 fails to launch task container with SecretManager$InvalidToken: No NMToken sent Critical applications, capacityscheduler Tassapol Athiapinya Jian He
YARN-1833 TestRMAdminService Fails in trunk and branch-2 : Assert Fails due to different count of UserGroups for currentUser() Major . Mit Desai Mit Desai
YARN-1830 TestRMRestart.testQueueMetricsOnRMRestart failure Major resourcemanager Karthik Kambatla Zhijie Shen
YARN-1824 Make Windows client work with Linux/Unix cluster Major . Jian He Jian He
YARN-1793 yarn application -kill doesn’t kill UnmanagedAMs Critical resourcemanager Karthik Kambatla Karthik Kambatla
YARN-1788 AppsCompleted/AppsKilled metric is incorrect when MR job is killed with yarn application -kill Critical resourcemanager Tassapol Athiapinya Varun Vasudev
YARN-1785 FairScheduler treats app lookup failures as ERRORs Major . bc Wong bc Wong
YARN-1783 yarn application does not make any progress even when no other application is running when RM is being restarted in the background Critical . Arpit Gupta Jian He
YARN-1774 FS: Submitting to non-leaf queue throws NPE Blocker fairscheduler Anubhav Dhoot Anubhav Dhoot
YARN-1768 yarn kill non-existent application is too verbose Minor client Hitesh Shah Tsuyoshi Ozawa
YARN-1760 TestRMAdminService assumes CapacityScheduler Trivial . Karthik Kambatla Karthik Kambatla
YARN-1758 MiniYARNCluster broken post YARN-1666 Blocker . Hitesh Shah Xuan Gong
YARN-1752 Unexpected Unregistered event at Attempt Launched state Major . Jian He Rohith Sharma K S
YARN-1748 hadoop-yarn-server-tests packages core-site.xml breaking downstream tests Blocker . Sravya Tirukkovalur Sravya Tirukkovalur
YARN-1742 Fix javadoc of parameter DEFAULT_NM_MIN_HEALTHY_DISKS_FRACTION Trivial documentation Akira AJISAKA Akira AJISAKA
YARN-1724 Race condition in Fair Scheduler when continuous scheduling is turned on Critical scheduler Sandy Ryza Sandy Ryza
YARN-1721 When moving app between queues in Fair Scheduler, grab lock on FSSchedulerApp Critical scheduler Sandy Ryza Sandy Ryza
YARN-1697 NodeManager reports negative running containers Major nodemanager Sandy Ryza Sandy Ryza
YARN-1692 ConcurrentModificationException in fair scheduler AppSchedulable Major scheduler Sangjin Lee Sangjin Lee
YARN-1689 RMAppAttempt is not killed when RMApp is at ACCEPTED Critical resourcemanager Deepesh Khandelwal Vinod Kumar Vavilapalli
YARN-1686 NodeManager.resyncWithRM() does not handle exception which cause NodeManger to Hang. Major nodemanager Rohith Sharma K S Rohith Sharma K S
YARN-1673 Valid yarn kill application prints out help message. Blocker client Tassapol Athiapinya Mayank Bansal
YARN-1672 YarnConfiguration is missing a default for yarn.nodemanager.log.retain-seconds Trivial nodemanager Karthik Kambatla Naren Koneru
YARN-1670 aggregated log writer can write more log data then it says is the log length Critical . Thomas Graves Mit Desai
YARN-1661 AppMaster logs says failing even if an application does succeed. Major applications/distributed-shell Tassapol Athiapinya Vinod Kumar Vavilapalli
YARN-1632 TestApplicationMasterServices should be under org.apache.hadoop.yarn.server.resourcemanager package Minor . Chen He Chen He
YARN-1617 Remove ancient comment and surround LOG.debug in AppSchedulingInfo.allocate Major scheduler Sandy Ryza Sandy Ryza
YARN-1591 TestResourceTrackerService fails randomly on trunk Major . Vinod Kumar Vavilapalli Tsuyoshi Ozawa
YARN-1590 _HOST doesn’t expand properly for RM, NM, ProxyServer and JHS Major resourcemanager Mohammad Kamrul Islam Mohammad Kamrul Islam
YARN-1553 Do not use HttpConfig.isSecure() in YARN Major . Haohui Mai Haohui Mai
YARN-1531 True up yarn command documentation Major documentation Akira AJISAKA Akira AJISAKA
YARN-1528 Allow setting auth for ZK connections Blocker resourcemanager Karthik Kambatla Karthik Kambatla
YARN-1470 Add audience annotation to MiniYARNCluster Major . Sandy Ryza Anubhav Dhoot
YARN-1444 RM crashes when node resource request sent without corresponding off-switch request Blocker client, resourcemanager Robert Grandl Wangda Tan
YARN-1417 RM may issue expired container tokens to AM while issuing new containers. Blocker . Omkar Vinit Joshi Jian He
YARN-1398 Deadlock in capacity scheduler leaf queue and parent queue for getQueueInfo and completedContainer call Blocker resourcemanager Sunil G Vinod Kumar Vavilapalli
YARN-1301 Need to log the blacklist additions/removals when YarnSchedule#allocate Minor . Zhijie Shen Tsuyoshi Ozawa
YARN-1285 Inconsistency of default “yarn.acl.enable” value Major . Zhijie Shen Kenji Kikushima
YARN-1206 AM container log link broken on NM web page Blocker . Jian He Rohith Sharma K S
YARN-1166 YARN ‘appsFailed’ metric should be of type ‘counter’ Blocker resourcemanager Srimanth Gunturi Zhijie Shen
YARN-1071 ResourceManager’s decommissioned and lost node count is 0 after restart Major resourcemanager Srimanth Gunturi Jian He
YARN-713 ResourceManager can exit unexpectedly if DNS is unavailable Critical resourcemanager Jason Lowe Jian He

TESTS:

JIRA Summary Priority Component Reporter Contributor
HADOOP-9525 Add tests that validate winutils chmod behavior on folders Major test, util Ivan Mitic Ivan Mitic
HDFS-5953 TestBlockReaderFactory fails if libhadoop.so has not been built Major . Ted Yu Akira AJISAKA
HDFS-5936 MiniDFSCluster does not clean data left behind by SecondaryNameNode. Major namenode, test Andrew Wang Binglin Chang
HDFS-5882 TestAuditLogs is flaky Minor . Jimmy Xiang Jimmy Xiang
HDFS-5672 TestHASafeMode#testSafeBlockTracking fails in trunk Major namenode Ted Yu Jing Zhao
YARN-1863 TestRMFailover fails with ‘AssertionError: null’ Blocker . Ted Yu Xuan Gong
YARN-1855 TestRMFailover#testRMWebAppRedirect fails in trunk Critical . Ted Yu Zhijie Shen
YARN-1854 Race condition in TestRMHA#testStartAndTransitions Blocker . Mit Desai Rohith Sharma K S

SUB-TASKS:

JIRA Summary Priority Component Reporter Contributor
HADOOP-10449 Fix the javac warnings in the security packages. Minor security Tsz Wo Nicholas Sze Tsz Wo Nicholas Sze
HADOOP-10437 Fix the javac warnings in the conf and the util package Minor conf, util Tsz Wo Nicholas Sze Tsz Wo Nicholas Sze
HADOOP-10407 Fix the javac warnings in the ipc package. Minor ipc Tsz Wo Nicholas Sze Tsz Wo Nicholas Sze
HADOOP-10399 FileContext API for ACLs. Major fs Chris Nauroth Vinayakumar B
HADOOP-10393 Fix hadoop-auth javac warnings Minor security Tsz Wo Nicholas Sze Tsz Wo Nicholas Sze
HADOOP-10285 Admin interface to swap callqueue at runtime Major . Chris Li Chris Li
HADOOP-10280 Make Schedulables return a configurable identity of user or group Major . Chris Li Chris Li
HADOOP-10278 Refactor to make CallQueue pluggable Major ipc Chris Li Chris Li
HDFS-6124 Add final modifier to class members Major . Suresh Srinivas Suresh Srinivas
HDFS-6086 Fix a case where zero-copy or no-checksum reads were not allowed even when the block was cached Major datanode Colin Patrick McCabe Colin Patrick McCabe
HDFS-6076 SimulatedDataSet should not create DatanodeRegistration with namenode layout version and type Minor datanode, test Tsz Wo Nicholas Sze Tsz Wo Nicholas Sze
HDFS-6061 Allow dfs.datanode.shared.file.descriptor.path to contain multiple entries and fall back when needed Major datanode Colin Patrick McCabe Colin Patrick McCabe
HDFS-6060 NameNode should not check DataNode layout version Major namenode Brandon Li Brandon Li
HDFS-6046 add dfs.client.mmap.enabled Major hdfs-client Colin Patrick McCabe Colin Patrick McCabe
HDFS-6040 fix DFSClient issue without libhadoop.so and some other ShortCircuitShm cleanups Blocker hdfs-client Colin Patrick McCabe Colin Patrick McCabe
HDFS-6038 Allow JournalNode to handle editlog produced by new release with future layoutversion Major journal-node, namenode Haohui Mai Jing Zhao
HDFS-5986 Capture the number of blocks pending deletion on namenode webUI Major namenode Suresh Srinivas Chris Nauroth
HDFS-5973 add DomainSocket#shutdown method Major hdfs-client Colin Patrick McCabe Colin Patrick McCabe
HDFS-5956 A file size is multiplied by the replication factor in ‘hdfs oiv -p FileDistribution’ option Major tools Akira AJISAKA Akira AJISAKA
HDFS-5950 The DFSClient and DataNode should use shared memory segments to communicate short-circuit information Major datanode, hdfs-client Colin Patrick McCabe Colin Patrick McCabe
HDFS-5942 Fix javadoc in OfflineImageViewer Minor documentation, tools Akira AJISAKA Akira AJISAKA
HDFS-5940 Minor cleanups to ShortCircuitReplica, FsDatasetCache, and DomainSocketWatcher Major hdfs-client Colin Patrick McCabe Colin Patrick McCabe
HDFS-5938 Make BlockReaderFactory#BlockReaderPeer a static class Trivial hdfs-client Colin Patrick McCabe Colin Patrick McCabe
HDFS-5898 Allow NFS gateway to login/relogin from its kerberos keytab Major nfs Jing Zhao Abin Shahab
HDFS-5868 Make hsync implementation pluggable Major datanode Taylor, Buddy Taylor, Buddy
HDFS-5866 ‘-maxSize’ and ‘-step’ option fail in OfflineImageViewer Major tools Akira AJISAKA Akira AJISAKA
HDFS-5847 Consolidate INodeReference into a separate section Major . Haohui Mai Jing Zhao
HDFS-5810 Unify mmap cache and short-circuit file descriptor cache Major hdfs-client Colin Patrick McCabe Colin Patrick McCabe
HDFS-5746 add ShortCircuitSharedMemorySegment Major datanode, hdfs-client Colin Patrick McCabe Colin Patrick McCabe
HDFS-5726 Fix compilation error in AbstractINodeDiff for JDK7 Minor namenode Jing Zhao Jing Zhao
HDFS-5715 Use Snapshot ID to indicate the corresponding Snapshot for a FileDiff/DirectoryDiff Major namenode Jing Zhao Jing Zhao
HDFS-5647 Merge INodeDirectory.Feature and INodeFile.Feature Major namenode Haohui Mai Haohui Mai
HDFS-5638 HDFS implementation of FileContext API for ACLs. Major hdfs-client Chris Nauroth Vinayakumar B
HDFS-5632 Add Snapshot feature to INodeDirectory Major namenode Jing Zhao Jing Zhao
HDFS-5623 NameNode: add tests for skipping ACL enforcement when permission checks are disabled, user is superuser or user is member of supergroup. Major namenode Chris Nauroth Chris Nauroth
HDFS-5554 Add Snapshot Feature to INodeFile Major namenode Jing Zhao Jing Zhao
HDFS-5537 Remove FileWithSnapshot interface Major namenode, snapshots Jing Zhao Jing Zhao
HDFS-5531 Combine the getNsQuota() and getDsQuota() methods in INode Minor namenode Tsz Wo Nicholas Sze Tsz Wo Nicholas Sze
HDFS-5483 NN should gracefully handle multiple block replicas on same DN Major namenode Arpit Agarwal Arpit Agarwal
HDFS-5286 Flatten INodeDirectory hierarchy: add DirectoryWithQuotaFeature Major namenode Tsz Wo Nicholas Sze Tsz Wo Nicholas Sze
HDFS-5285 Flatten INodeFile hierarchy: Add UnderContruction Feature Major namenode Tsz Wo Nicholas Sze Jing Zhao
HDFS-5167 Add metrics about the NameNode retry cache Minor ha, namenode Jing Zhao Tsuyoshi Ozawa
HDFS-4564 Webhdfs returns incorrect http response codes for denied operations Blocker webhdfs Daryn Sharp Daryn Sharp
MAPREDUCE-5787 Modify ShuffleHandler to support Keep-Alive Critical nodemanager Rajesh Balamohan Rajesh Balamohan
YARN-1893 Make ApplicationMasterProtocol#allocate AtMostOnce Blocker resourcemanager Xuan Gong Xuan Gong
YARN-1850 Make enabling timeline service configurable Major . Zhijie Shen Zhijie Shen
YARN-1838 Timeline service getEntities API should provide ability to get entities from given id Major . Srimanth Gunturi Billie Rinaldi
YARN-1821 NPE on registerNodeManager if the request has containers for UnmanagedAMs Blocker resourcemanager Karthik Kambatla Karthik Kambatla
YARN-1816 Succeeded application remains in accepted after RM restart Major . Arpit Gupta Jian He
YARN-1812 Job stays in PREP state for long time after RM Restarts Major . Yesha Vora Jian He
YARN-1811 RM HA: AM link broken if the AM is on nodes other than RM Major resourcemanager Robert Kanter Robert Kanter
YARN-1800 YARN NodeManager with java.util.concurrent.RejectedExecutionException Critical nodemanager Paul Isaychuk Varun Vasudev
YARN-1787 yarn applicationattempt/container print wrong usage information Major . Zhijie Shen Zhijie Shen
YARN-1781 NM should allow users to specify max disk utilization for local disks Major nodemanager Varun Vasudev Varun Vasudev
YARN-1780 Improve logging in timeline service Major . Zhijie Shen Zhijie Shen
YARN-1776 renewDelegationToken should survive RM failover Major . Zhijie Shen Zhijie Shen
YARN-1775 Create SMAPBasedProcessTree to get PSS information Major nodemanager Rajesh Balamohan Rajesh Balamohan
YARN-1766 When RM does the initiation, it should use loaded Configuration instead of bootstrap configuration. Major . Xuan Gong Xuan Gong
YARN-1765 Write test cases to verify that killApplication API works in RM HA Major . Xuan Gong Xuan Gong
YARN-1764 Handle RM fail overs after the submitApplication call. Major . Xuan Gong Xuan Gong
YARN-1761 RMAdminCLI should check whether HA is enabled before executes transitionToActive/transitionToStandby Major . Xuan Gong Xuan Gong
YARN-1749 Review AHS configs and sync them up with the timeline-service configs Major . Zhijie Shen Zhijie Shen
YARN-1734 RM should get the updated Configurations when it transits from Standby to Active Critical . Xuan Gong Xuan Gong
YARN-1732 Change types of related entities and primary filters in ATSEntity Major . Billie Rinaldi Billie Rinaldi
YARN-1730 Leveldb timeline store needs simple write locking Major . Billie Rinaldi Billie Rinaldi
YARN-1729 TimelineWebServices always passes primary and secondary filters as strings Major . Billie Rinaldi Billie Rinaldi
YARN-1719 ATSWebServices produces jersey warnings Major . Billie Rinaldi Billie Rinaldi
YARN-1717 Enable offline deletion of entries in leveldb timeline store Major . Billie Rinaldi Billie Rinaldi
YARN-1706 Create an utility function to dump timeline records to json Major . Zhijie Shen Zhijie Shen
YARN-1705 Reset cluster-metrics on transition to standby Major resourcemanager Karthik Kambatla Rohith Sharma K S
YARN-1704 Review LICENSE and NOTICE to reflect new levelDB releated libraries being used Blocker . Billie Rinaldi Billie Rinaldi
YARN-1698 Replace MemoryApplicationTimelineStore with LeveldbApplicationTimelineStore as default Major . Zhijie Shen Zhijie Shen
YARN-1690 Sending timeline entities+events from Distributed shell Major . Mayank Bansal Mayank Bansal
YARN-1687 Refactoring timeline classes to remove “app” related words Major . Zhijie Shen Zhijie Shen
YARN-1685 Bugs around log URL Major . Mayank Bansal Zhijie Shen
YARN-1684 Fix history server heap size in yarn script Major . Billie Rinaldi Billie Rinaldi
YARN-1676 Make admin refreshUserToGroupsMappings of configuration work across RM failover Major . Xuan Gong Xuan Gong
YARN-1669 Make admin refreshServiceAcls work across RM failover Major . Xuan Gong Xuan Gong
YARN-1668 Make admin refreshAdminAcls work across RM failover Major . Xuan Gong Xuan Gong
YARN-1667 Make admin refreshSuperUserGroupsConfiguration work across RM failover Major . Xuan Gong Xuan Gong
YARN-1666 Make admin refreshNodes work across RM failover Major . Xuan Gong Xuan Gong
YARN-1665 Set better defaults for HA configs for automatic failover Major resourcemanager Arpit Gupta Xuan Gong
YARN-1660 add the ability to set yarn.resourcemanager.hostname.rm-id instead of setting all the various host:port properties for RM Major resourcemanager Arpit Gupta Xuan Gong
YARN-1659 Define the ApplicationTimelineStore store as an abstraction for implementing different storage impls for storing timeline information Major . Billie Rinaldi Billie Rinaldi
YARN-1658 Webservice should redirect to active RM when HA is enabled. Major . Cindy Li Cindy Li
YARN-1641 ZK store should attempt a write periodically to ensure it is still Active Major resourcemanager Karthik Kambatla Karthik Kambatla
YARN-1640 Manual Failover does not work in secure clusters Blocker . Xuan Gong Xuan Gong
YARN-1639 YARM RM HA requires different configs on different RM hosts Major resourcemanager Arpit Gupta Xuan Gong
YARN-1637 Implement a client library for java users to post entities+events Major . Vinod Kumar Vavilapalli Zhijie Shen
YARN-1636 Implement timeline related web-services inside AHS for storing and retrieving entities+events Major . Vinod Kumar Vavilapalli Zhijie Shen
YARN-1635 Implement a Leveldb based ApplicationTimelineStore Major . Vinod Kumar Vavilapalli Billie Rinaldi
YARN-1634 Define an in-memory implementation of ApplicationTimelineStore Major . Vinod Kumar Vavilapalli Zhijie Shen
YARN-1633 Define user-faced entity, entity-info and event objects Major . Vinod Kumar Vavilapalli Zhijie Shen
YARN-1625 mvn apache-rat:check outputs warning message in YARN-321 branch Trivial . Shinichi Yamashita Shinichi Yamashita
YARN-1613 Fix config name YARN_HISTORY_SERVICE_ENABLED Major . Zhijie Shen Akira AJISAKA
YARN-1611 Make admin refresh of capacity scheduler configuration work across RM failover Major . Xuan Gong Xuan Gong
YARN-1605 Fix formatting issues with new module in YARN-321 branch Major . Vinod Kumar Vavilapalli Vinod Kumar Vavilapalli
YARN-1597 FindBugs warnings on YARN-321 branch Major . Vinod Kumar Vavilapalli Vinod Kumar Vavilapalli
YARN-1596 Javadoc failures on YARN-321 branch Major . Vinod Kumar Vavilapalli Vinod Kumar Vavilapalli
YARN-1595 Test failures on YARN-321 branch Major . Vinod Kumar Vavilapalli Vinod Kumar Vavilapalli
YARN-1594 YARN-321 branch needs to be updated after YARN-888 pom changes Major . Vinod Kumar Vavilapalli Vinod Kumar Vavilapalli
YARN-1588 Rebind NM tokens for previous attempt’s running containers to the new attempt Major . Jian He Jian He
YARN-1587 [YARN-321] Merge Patch for YARN-321 Major . Mayank Bansal Vinod Kumar Vavilapalli
YARN-1578 Fix how to read history file in FileSystemApplicationHistoryStore Major . Shinichi Yamashita Shinichi Yamashita
YARN-1577 Unmanaged AM is broken because of YARN-1493 Blocker . Jian He Jian He
YARN-1566 Change distributed-shell to retain containers from previous AppAttempt Major . Jian He Jian He
YARN-1555 [YARN-321] Failing tests in org.apache.hadoop.yarn.server.applicationhistoryservice.* Major . Vinod Kumar Vavilapalli Vinod Kumar Vavilapalli
YARN-1534 TestAHSWebApp failed in YARN-321 branch Major . Shinichi Yamashita Shinichi Yamashita
YARN-1525 Web UI should redirect to active RM when HA is enabled. Major . Xuan Gong Cindy Li
YARN-1521 Mark appropriate protocol methods with the idempotent annotation or AtMostOnce annotation Blocker . Xuan Gong Xuan Gong
YARN-1504 RM changes for moving apps between queues Major resourcemanager Sandy Ryza Sandy Ryza
YARN-1499 Fair Scheduler changes for moving apps between queues Major scheduler Sandy Ryza Sandy Ryza
YARN-1498 Common scheduler changes for moving apps between queues Major resourcemanager Sandy Ryza Sandy Ryza
YARN-1497 Expose moving apps between queues on the command line Major client Sandy Ryza Sandy Ryza
YARN-1496 Protocol additions to allow moving apps between queues Major scheduler Sandy Ryza Sandy Ryza
YARN-1493 Schedulers don’t recognize apps separately from app-attempts Major . Jian He Jian He
YARN-1490 RM should optionally not kill all containers when an ApplicationMaster exits Major . Vinod Kumar Vavilapalli Jian He
YARN-1461 RM API and RM changes to handle tags for running jobs Major resourcemanager Karthik Kambatla Karthik Kambatla
YARN-1459 RM services should depend on ConfigurationProvider during startup too Major resourcemanager Karthik Kambatla Xuan Gong
YARN-1428 RM cannot write the final state of RMApp/RMAppAttempt to the application history store in the transition to the final state Major . Zhijie Shen Zhijie Shen
YARN-1410 Handle RM fails over after getApplicationID() and before submitApplication(). Major . Bikas Saha Xuan Gong
YARN-1389 ApplicationClientProtocol and ApplicationHistoryProtocol should expose analogous APIs Major . Mayank Bansal Mayank Bansal
YARN-1379 [YARN-321] AHS protocols need to be in yarn proto package name after YARN-1170 Major . Vinod Kumar Vavilapalli Vinod Kumar Vavilapalli
YARN-1345 Removing FINAL_SAVING from YarnApplicationAttemptState Major . Zhijie Shen Zhijie Shen
YARN-1266 Implement PB service and client wrappers for ApplicationHistoryProtocol Major . Mayank Bansal Mayank Bansal
YARN-1242 Script changes to start AHS as an individual process Major . Zhijie Shen Mayank Bansal
YARN-1191 [YARN-321] Update artifact versions for application history service Major . Mayank Bansal Mayank Bansal
YARN-1123 [YARN-321] Adding ContainerReport and Protobuf implementation Major . Zhijie Shen Mayank Bansal
YARN-1041 Protocol changes for RM to bind and notify a restarted AM of existing containers Major resourcemanager Steve Loughran Jian He
YARN-1023 [YARN-321] Webservices REST API’s support for Application History Major . Devaraj K Zhijie Shen
YARN-1017 Document RM Restart feature Blocker resourcemanager Jian He Jian He
YARN-1007 [YARN-321] Enhance History Reader interface for Containers Major . Devaraj K Mayank Bansal
YARN-987 Adding ApplicationHistoryManager responsible for exposing reports to all clients Major . Mayank Bansal Mayank Bansal
YARN-986 RM DT token service should have service addresses of both RMs Blocker . Vinod Kumar Vavilapalli Karthik Kambatla
YARN-984 [YARN-321] Move classes from applicationhistoryservice.records.pb.impl package to applicationhistoryservice.records.impl.pb Major . Devaraj K Devaraj K
YARN-979 [YARN-321] Add more APIs related to ApplicationAttempt and Container in ApplicationHistoryProtocol Major . Mayank Bansal Mayank Bansal
YARN-978 [YARN-321] Adding ApplicationAttemptReport and Protobuf implementation Major . Mayank Bansal Mayank Bansal
YARN-975 Add a file-system implementation for history-storage Major . Zhijie Shen Zhijie Shen
YARN-974 RMContainer should collect more useful information to be recorded in Application-History Major . Zhijie Shen Zhijie Shen
YARN-967 [YARN-321] Command Line Interface(CLI) for Reading Application History Storage Data Major . Devaraj K Mayank Bansal
YARN-962 Update application_history_service.proto Major . Zhijie Shen Zhijie Shen
YARN-956 [YARN-321] Add a testable in-memory HistoryStorage Major . Vinod Kumar Vavilapalli Zhijie Shen
YARN-955 [YARN-321] Implementation of ApplicationHistoryProtocol Major . Vinod Kumar Vavilapalli Mayank Bansal
YARN-954 [YARN-321] History Service should create the webUI and wire it to HistoryStorage Major . Vinod Kumar Vavilapalli Zhijie Shen
YARN-953 [YARN-321] Enable ResourceManager to write history data Major . Vinod Kumar Vavilapalli Zhijie Shen
YARN-947 Defining the history data classes for the implementation of the reading/writing interface Major . Zhijie Shen Zhijie Shen
YARN-935 YARN-321 branch is broken due to applicationhistoryserver module’s pom.xml Major . Zhijie Shen Zhijie Shen
YARN-934 HistoryStorage writer interface for Application History Server Major . Zhijie Shen Zhijie Shen
YARN-930 Bootstrap ApplicationHistoryService module Major . Vinod Kumar Vavilapalli Vinod Kumar Vavilapalli
YARN-925 HistoryStorage Reader Interface for Application History Server Major . Mayank Bansal Mayank Bansal

OTHER:

JIRA Summary Priority Component Reporter Contributor
HDFS-6030 Remove an unused constructor in INode.java Trivial . Yongjun Zhang Yongjun Zhang
HDFS-6025 Update findbugsExcludeFile.xml Minor build Tsz Wo Nicholas Sze Tsz Wo Nicholas Sze
YARN-1891 Document NodeManager health-monitoring Minor . Varun Vasudev Varun Vasudev
YARN-1452 Document the usage of the generic application history and the timeline data service Major . Zhijie Shen Zhijie Shen