Hadoop 2.7.0 Release Notes

• YARN-3430. Blocker sub-task reported by Xuan Gong and fixed by Xuan Gong (resourcemanager , webapp , yarn)
  RMAppAttempt headroom data is missing in RM Web UI
  

• YARN-3393. Critical bug reported by Zhijie Shen and fixed by Zhijie Shen (timelineserver)
  Getting application(s) goes wrong when app finishes before starting the attempt
  

• YARN-3384. Minor bug reported by Naganarasimha G R and fixed by Naganarasimha G R
  TestLogAggregationService.verifyContainerLogs fails after YARN-2777
  

• YARN-3379. Major sub-task reported by Xuan Gong and fixed by Xuan Gong (resourcemanager , webapp , yarn)
  Missing data in localityTable and ResourceRequests table in RM WebUI
  

• YARN-3369. Blocker bug reported by Giovanni Matteo Fumarola and fixed by Brahma Reddy Battula (resourcemanager)
  Missing NullPointer check in AppSchedulingInfo causes RM to die
  

• YARN-3349. Minor improvement reported by zhihai xu and fixed by zhihai xu (test)
  Treat all exceptions as failure in TestFSRMStateStore#testFSRMStateStoreClientRetry
  

• YARN-3338. Blocker bug reported by Zhijie Shen and fixed by Zhijie Shen (build)
  Exclude jline dependency from YARN
  

• YARN-3336. Critical bug reported by zhihai xu and fixed by zhihai xu (resourcemanager)
  FileSystem memory leak in DelegationTokenRenewer
  

• YARN-3304. Blocker bug reported by Junping Du and fixed by Junping Du (nodemanager)
  ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
  

• YARN-3300. Major sub-task reported by Xuan Gong and fixed by Xuan Gong (resourcemanager)
  outstanding_resource_requests table should not be shown in AHS
  

• YARN-3296. Major bug reported by Hitesh Shah and fixed by Hitesh Shah
  yarn.nodemanager.container-monitor.process-tree.class is configurable but ResourceCalculatorProcessTree class is marked Private
  

• YARN-3295. Trivial bug reported by Masatake Iwasaki and fixed by Masatake Iwasaki (documentation)
  Fix documentation nits found in markdown conversion
  

• YARN-3287. Major bug reported by Jonathan Eagles and fixed by Daryn Sharp
  TimelineClient kerberos authentication failure uses wrong login context.
  

• YARN-3285. Major improvement reported by Masatake Iwasaki and fixed by Masatake Iwasaki (documentation)
  Convert branch-2 .apt.vm files of YARN to markdown
  

• YARN-3281. Minor bug reported by Chengbing Liu and fixed by Chengbing Liu (scripts)
  Add RMStateStore to StateMachine visualization list
  

• YARN-3275. Major bug reported by Eric Payne and fixed by Eric Payne
  CapacityScheduler: Preemption happening on non-preemptable queues
  

• YARN-3273. Major improvement reported by Jian He and fixed by Rohith
  Improve web UI to facilitate scheduling analysis and debugging
  

• YARN-3272. Major improvement reported by Jian He and fixed by Jian He
  Surface container locality info in RM web UI
  

• YARN-3270. Minor bug reported by Rohit Agarwal and fixed by Rohit Agarwal
  node label expression not getting set in ApplicationSubmissionContext
  

• YARN-3267. Major bug reported by Prakash Ramachandran and fixed by Chang Li
  Timelineserver applies the ACL rules after applying the limit on the number of records
  

• YARN-3265. Blocker sub-task reported by Wangda Tan and fixed by Wangda Tan (capacityscheduler , resourcemanager)
  CapacityScheduler deadlock when computing absolute max avail capacity (fix for trunk/branch-2)
  

• YARN-3262. Major improvement reported by Jian He and fixed by Jian He (yarn)
  Surface application outstanding resource requests table in RM web UI
  

• YARN-3256. Major bug reported by Anubhav Dhoot and fixed by Anubhav Dhoot
  TestClientToAMTokens#testClientTokenRace is not running against all Schedulers even when using ParameterizedSchedulerTestBase
  

• YARN-3255. Major bug reported by Konstantin Shvachko and fixed by Konstantin Shvachko (nodemanager , resourcemanager)
  RM, NM, JobHistoryServer, and WebAppProxyServer's main() should support generic options
  

• YARN-3249. Minor improvement reported by Ryu Kobayashi and fixed by Ryu Kobayashi (resourcemanager)
  Add a "kill application" button to Resource Manager's Web UI
  

• YARN-3242. Critical bug reported by zhihai xu and fixed by zhihai xu (resourcemanager)
  Asynchrony in ZK-close can lead to ZKRMStateStore watcher receiving events for old client
  

• YARN-3239. Major bug reported by Hitesh Shah and fixed by Jian He
  WebAppProxy does not support a final tracking url which has query fragments and params
  

• YARN-3238. Blocker bug reported by Jason Lowe and fixed by Jason Lowe
  Connection timeouts to nodemanagers are retried at multiple levels
  

• YARN-3237. Major bug reported by Rushabh S Shah and fixed by Rushabh S Shah
  AppLogAggregatorImpl fails to log error cause
  

• YARN-3236. Trivial improvement reported by zhihai xu and fixed by zhihai xu (resourcemanager)
  cleanup RMAuthenticationFilter#AUTH_HANDLER_PROPERTY.
  

• YARN-3231. Critical bug reported by Siqi Li and fixed by Siqi Li
  FairScheduler: Changing queueMaxRunningApps interferes with pending jobs
  

• YARN-3230. Major improvement reported by Jian He and fixed by Jian He
  Clarify application states on the web UI
  

• YARN-3227. Critical bug reported by Jonathan Eagles and fixed by Zhijie Shen
  Timeline renew delegation token fails when RM user's TGT is expired
  

• YARN-3222. Critical bug reported by Rohith and fixed by Rohith (resourcemanager)
  RMNodeImpl#ReconnectNodeTransition should send scheduler events in sequential order
  

• YARN-3217. Major task reported by Akira AJISAKA and fixed by Brahma Reddy Battula
  Remove httpclient dependency from hadoop-yarn-server-web-proxy
  

• YARN-3207. Major bug reported by Prakash Ramachandran and fixed by Zhijie Shen (timelineserver)
  secondary filter matches entites which do not have the key being filtered for.
  

• YARN-3203. Minor improvement reported by Brahma Reddy Battula and fixed by Brahma Reddy Battula
  Correct a log message in AuxServices
  

• YARN-3195. Minor improvement reported by Jagadesh Kiran N and fixed by Jagadesh Kiran N (client)
  Add -help to yarn logs and nodes CLI command
  

• YARN-3194. Blocker bug reported by Rohith and fixed by Rohith (resourcemanager)
  RM should handle NMContainerStatuses sent by NM while registering if NM is Reconnected node
  

• YARN-3191. Trivial bug reported by Rohith and fixed by Rohith (nodemanager)
  Log object should be initialized with its own class
  

• YARN-3187. Major improvement reported by Naganarasimha G R and fixed by Gururaj Shetty (capacityscheduler , documentation)
  Documentation of Capacity Scheduler Queue mapping based on user or group
  

• YARN-3182. Minor improvement reported by Ray Chiang and fixed by Ray Chiang
  Cleanup switch statement in ApplicationMasterLauncher#handle()
  

• YARN-3179. Minor improvement reported by Ray Chiang and fixed by Ray Chiang
  Update use of Iterator to Iterable
  

• YARN-3171. Minor sub-task reported by Jeff Zhang and fixed by Naganarasimha G R (timelineserver)
  Sort by Application id, AppAttempt & ContainerID doesn't work in ATS / RM web ui
  

• YARN-3164. Minor bug reported by Bibin A Chundatt and fixed by Bibin A Chundatt (resourcemanager)
  rmadmin command usage prints incorrect command name
  

• YARN-3160. Major bug reported by Chengbing Liu and fixed by Chengbing Liu (resourcemanager)
  Non-atomic operation on nodeUpdateQueue in RMNodeImpl
  

• YARN-3158. Major improvement reported by Devaraj K and fixed by Varun Saxena
  Correct log messages in ResourceTrackerService
  

• YARN-3157. Minor improvement reported by Bibin A Chundatt and fixed by Bibin A Chundatt (resourcemanager)
  Refactor the exception handling in ConverterUtils#to*Id
  

• YARN-3155. Minor bug reported by Li Lu and fixed by Li Lu
  Refactor the exception handling code for TimelineClientImpl's retryOn method
  

• YARN-3154. Blocker sub-task reported by Xuan Gong and fixed by Xuan Gong (nodemanager , resourcemanager)
  Should not upload partial logs for MR jobs or other "short-running' applications
  

• YARN-3151. Minor bug reported by Bibin A Chundatt and fixed by Rohith (client , resourcemanager)
  On Failover tracking url wrong in application cli for KILLED application
  

• YARN-3149. Trivial bug reported by Bibin A Chundatt and fixed by Bibin A Chundatt (resourcemanager)
  Typo in message for invalid application id
  

• YARN-3147. Major improvement reported by Steve Loughran and fixed by Steve Loughran (webapp)
  Clean up RM web proxy code
  

• YARN-3145. Major bug reported by Jian He and fixed by Tsuyoshi Ozawa
  ConcurrentModificationException on CapacityScheduler ParentQueue#getQueueUserAclInfo
  

• YARN-3144. Major improvement reported by Jonathan Eagles and fixed by Jonathan Eagles
  Configuration for making delegation token failures to timeline server not-fatal
  

• YARN-3143. Major bug reported by Kendall Thrapp and fixed by Jason Lowe (webapp)
  RM Apps REST API can return NPE or entries missing id and other fields
  

• YARN-3132. Major sub-task reported by Wangda Tan and fixed by Wangda Tan (api , client , resourcemanager)
  RMNodeLabelsManager should remove node from node-to-label mapping when node becomes deactivated
  

• YARN-3131. Major bug reported by Chang Li and fixed by Chang Li
  YarnClientImpl should check FAILED and KILLED state in submitApplication
  

• YARN-3124. Major sub-task reported by Wangda Tan and fixed by Wangda Tan (api , client , resourcemanager)
  Capacity Scheduler LeafQueue/ParentQueue should use QueueCapacities to track capacities-by-label
  

• YARN-3123. Major improvement reported by Zhijie Shen and fixed by Naganarasimha G R (client)
  Make YARN CLI show a single completed container even if the app is running
  

• YARN-3122. Major sub-task reported by Anubhav Dhoot and fixed by Anubhav Dhoot (nodemanager)
  Metrics for container's actual CPU usage
  

• YARN-3113. Major bug reported by Chang Li and fixed by Steve Loughran
  Release audit warning for "Sorting icons.psd"
  

• YARN-3108. Major improvement reported by Chang Li and fixed by Chang Li
  ApplicationHistoryServer doesn't process -D arguments
  

• YARN-3104. Major bug reported by Jason Lowe and fixed by Jason Lowe (resourcemanager)
  RM generates new AMRM tokens every heartbeat between rolling and activation
  

• YARN-3103. Blocker bug reported by Jason Lowe and fixed by Jason Lowe (client)
  AMRMClientImpl does not update AMRM token properly
  

• YARN-3101. Major bug reported by Anubhav Dhoot and fixed by Anubhav Dhoot (fairscheduler)
  In Fair Scheduler, fix canceling of reservations for exceeding max share
  

• YARN-3100. Major improvement reported by Jian He and fixed by Jian He
  Make YARN authorization pluggable
  

• YARN-3099. Major sub-task reported by Wangda Tan and fixed by Wangda Tan (api , client , resourcemanager)
  Capacity Scheduler LeafQueue/ParentQueue should use ResourceUsage to track used-resources-by-label.
  

• YARN-3098. Major sub-task reported by Wangda Tan and fixed by Wangda Tan (capacityscheduler)
  Create common QueueCapacities class in Capacity Scheduler to track capacities-by-labels of queues
  

• YARN-3094. Major bug reported by Jun Gong and fixed by Jun Gong (resourcemanager)
  reset timer for liveness monitors after RM recovery
  

• YARN-3092. Major sub-task reported by Wangda Tan and fixed by Wangda Tan (api , client , resourcemanager)
  Create common ResourceUsage class to track labeled resource usages in Capacity Scheduler
  

• YARN-3090. Major bug reported by Jason Lowe and fixed by Varun Saxena (nodemanager)
  DeletionService can silently ignore deletion task failures
  

• YARN-3089. Blocker bug reported by Jason Lowe and fixed by Eric Payne
  LinuxContainerExecutor does not handle file arguments to deleteAsUser
  

• YARN-3088. Major bug reported by Jason Lowe and fixed by Eric Payne (nodemanager)
  LinuxContainerExecutor.deleteAsUser can throw NPE if native executor returns an error
  

• YARN-3086. Minor improvement reported by Robert Metzger and fixed by Robert Metzger (test)
  Make NodeManager memory configurable in MiniYARNCluster
  

• YARN-3085. Major improvement reported by Jason Lowe and fixed by Rohith (resourcemanager)
  Application summary should include the application type
  

• YARN-3082. Major bug reported by Anubhav Dhoot and fixed by Anubhav Dhoot
  Non thread safe access to systemCredentials in NodeHeartbeatResponse processing
  

• YARN-3079. Major bug reported by zhihai xu and fixed by zhihai xu
  Scheduler should also update maximumAllocation when updateNodeResource.
  

• YARN-3078. Minor bug reported by sam liu and fixed by (log-aggregation)
  LogCLIHelpers lacks of a blank space before string 'does not exist'
  

• YARN-3077. Major improvement reported by Chun Chen and fixed by Chun Chen (resourcemanager)
  RM should create yarn.resourcemanager.zk-state-store.parent-path recursively
  

• YARN-3076. Major sub-task reported by Varun Saxena and fixed by Varun Saxena (client)
  Add API/Implementation to YarnClient to retrieve label-to-node mapping
  

• YARN-3075. Major sub-task reported by Varun Saxena and fixed by Varun Saxena (resourcemanager)
  NodeLabelsManager implementation to retrieve label to node mapping
  

• YARN-3074. Major bug reported by Jason Lowe and fixed by Varun Saxena (nodemanager)
  Nodemanager dies when localizer runner tries to write to a full disk
  

• YARN-3071. Trivial bug reported by Masatake Iwasaki and fixed by Masatake Iwasaki (documentation)
  Remove invalid char from sample conf in doc of FairScheduler
  

• YARN-3070. Minor test reported by Ted Yu and fixed by Junping Du
  TestRMAdminCLI#testHelp fails for transitionToActive command
  

• YARN-3064. Critical bug reported by Wangda Tan and fixed by Jian He (scheduler)
  TestRMRestart/TestContainerResourceUsage/TestNodeManagerResync failure with allocation timeout
  

• YARN-3058. Minor bug reported by Yi Liu and fixed by Yi Liu
  Fix error message of tokens' activation delay configuration
  

• YARN-3056. Trivial improvement reported by zhihai xu and fixed by zhihai xu (test)
  add verification for containerLaunchDuration in TestNodeManagerMetrics.
  

• YARN-3055. Blocker bug reported by Yi Liu and fixed by Daryn Sharp (security)
  The token is not renewed properly if it's shared by jobs (oozie) in DelegationTokenRenewer
  

• YARN-3029. Major bug reported by Steve Loughran and fixed by Varun Saxena (nodemanager)
  FSDownload.unpack() uses local locale for FS case conversion, may not work everywhere
  

• YARN-3028. Major sub-task reported by Jian He and fixed by Rohith (api , client , resourcemanager)
  Better syntax for replaceLabelsOnNode in RMAdmin CLI
  

• YARN-3027. Major bug reported by Anubhav Dhoot and fixed by Anubhav Dhoot
  Scheduler should use totalAvailable resource from node instead of availableResource for maxAllocation
  

• YARN-3024. Major bug reported by Chengbing Liu and fixed by Chengbing Liu (nodemanager)
  LocalizerRunner should give DIE action when all resources are localized
  

• YARN-3022. Major improvement reported by Anubhav Dhoot and fixed by Anubhav Dhoot
  Expose Container resource information from NodeManager for monitoring
  

• YARN-3019. Major sub-task reported by Jian He and fixed by Jian He (resourcemanager)
  Make work-preserving-recovery the default mechanism for RM recovery
  

• YARN-3015. Minor bug reported by Chris Nauroth and fixed by Varun Saxena (scripts)
  yarn classpath command should support same options as hadoop classpath.
  

• YARN-3014. Major sub-task reported by Wangda Tan and fixed by Wangda Tan
  Replaces labels on a host should update all NM's labels on that host
  

• YARN-3011. Major sub-task reported by Wang Hao and fixed by Varun Saxena (nodemanager)
  NM dies because of the failure of resource localization
  

• YARN-3010. Minor bug reported by Yi Liu and fixed by Yi Liu
  Fix recent findbug issue in AbstractYarnScheduler
  

• YARN-3005. Trivial improvement reported by Akira AJISAKA and fixed by
  [JDK7] Use switch statement for String instead of if-else statement in RegistrySecurity.java
  

• YARN-2998. Major sub-task reported by Anubhav Dhoot and fixed by Anubhav Dhoot (fairscheduler)
  Abstract out scheduler independent PlanFollower components
  

• YARN-2997. Major bug reported by Chengbing Liu and fixed by Chengbing Liu (nodemanager)
  NM keeps sending already-sent completed containers to RM until containers are removed from context
  

• YARN-2996. Major improvement reported by Yi Liu and fixed by Yi Liu (resourcemanager)
  Refine fs operations in FileSystemRMStateStore and few fixes
  

• YARN-2994. Major sub-task reported by Jian He and fixed by Jian He (resourcemanager)
  Document work-preserving RM restart
  

• YARN-2993. Major bug reported by Yi Liu and fixed by Yi Liu (resourcemanager)
  Several fixes (missing acl check, error log msg ...) and some refinement in AdminService
  

• YARN-2992. Blocker bug reported by Karthik Kambatla and fixed by Karthik Kambatla (resourcemanager)
  ZKRMStateStore crashes due to session expiry
  

• YARN-2991. Blocker bug reported by Zhijie Shen and fixed by Rohith
  TestRMRestart.testDecomissionedNMsMetricsOnRMRestart intermittently fails on trunk
  

• YARN-2990. Major bug reported by Karthik Kambatla and fixed by Karthik Kambatla (fairscheduler)
  FairScheduler's delay-scheduling always waits for node-local and rack-local delays, even for off-rack-only requests
  

• YARN-2988. Minor bug reported by Ted Yu and fixed by Ted Yu
  Graph#save() may leak file descriptors
  

• YARN-2987. Major bug reported by Jian He and fixed by Varun Saxena
  ClientRMService#getQueueInfo doesn't check app ACLs
  

• YARN-2984. Major sub-task reported by Karthik Kambatla and fixed by Karthik Kambatla (nodemanager)
  Metrics for container's actual memory usage
  

• YARN-2983. Major bug reported by Varun Saxena and fixed by Varun Saxena (resourcemanager)
  NPE possible in ClientRMService#getQueueInfo
  

• YARN-2978. Critical bug reported by Jason Tufo and fixed by Varun Saxena
  ResourceManager crashes with NPE while getting queue info
  

• YARN-2977. Major bug reported by Junping Du and fixed by Junping Du
  TestNMClient get failed intermittently
  

• YARN-2975. Blocker bug reported by Karthik Kambatla and fixed by Karthik Kambatla
  FSLeafQueue app lists are accessed without required locks
  

• YARN-2972. Major bug reported by Jason Lowe and fixed by Jason Lowe (resourcemanager)
  DelegationTokenRenewer thread pool never expands
  

• YARN-2971. Major sub-task reported by Jonathan Eagles and fixed by Jonathan Eagles (timelineserver)
  RM uses conf instead of token service address to renew timeline delegation tokens
  

• YARN-2970. Minor sub-task reported by Junping Du and fixed by Varun Saxena (api , client , resourcemanager)
  NodeLabel operations in RMAdmin CLI get missing in help command.
  

• YARN-2964. Blocker bug reported by Daryn Sharp and fixed by Jian He (resourcemanager)
  RM prematurely cancels tokens for jobs that submit jobs (oozie)
  

• YARN-2958. Blocker bug reported by Zhijie Shen and fixed by Varun Saxena (resourcemanager)
  RMStateStore seems to unnecessarily and wrongly store sequence number separately
  

• YARN-2957. Minor improvement reported by Ray Chiang and fixed by Ray Chiang
  Create unit test to automatically compare YarnConfiguration and yarn-default.xml
  

• YARN-2956. Minor bug reported by Remus Rusanu and fixed by Masatake Iwasaki (documentation)
  Some yarn-site index linked pages are difficult to discover because are not in the side bar
  

• YARN-2952. Major bug reported by Jian He and fixed by Rohith
  Incorrect version check in RMStateStore
  

• YARN-2950. Minor improvement reported by Harsh J and fixed by Dustin Cote (webapp)
  Change message to mandate, not suggest JS requirement on UI
  

• YARN-2949. Major task reported by Varun Vasudev and fixed by Varun Vasudev (documentation , nodemanager)
  Add documentation for CGroups
  

• YARN-2945. Major bug reported by Tsuyoshi Ozawa and fixed by Tsuyoshi Ozawa
  FSLeafQueue#assignContainer - document the reason for using both write and read locks
  

• YARN-2944. Minor sub-task reported by Chris Trezzo and fixed by Chris Trezzo
  InMemorySCMStore can not be instantiated with ReflectionUtils#newInstance
  

• YARN-2943. Major sub-task reported by Wangda Tan and fixed by Wangda Tan (resourcemanager)
  Add a node-labels page in RM web UI
  

• YARN-2940. Major improvement reported by Varun Saxena and fixed by Li Lu
  Fix new findbugs warnings in rest of the hadoop-yarn components
  

• YARN-2939. Major improvement reported by Varun Saxena and fixed by Li Lu
  Fix new findbugs warnings in hadoop-yarn-common
  

• YARN-2938. Major improvement reported by Varun Saxena and fixed by Varun Saxena
  Fix new findbugs warnings in hadoop-yarn-resourcemanager and hadoop-yarn-applicationhistoryservice
  

• YARN-2937. Major improvement reported by Varun Saxena and fixed by Varun Saxena
  Fix new findbugs warnings in hadoop-yarn-nodemanager
  

• YARN-2936. Major bug reported by Zhijie Shen and fixed by Varun Saxena
  YARNDelegationTokenIdentifier doesn't set proto.builder now
  

• YARN-2933. Major sub-task reported by Wangda Tan and fixed by Mayank Bansal (capacityscheduler)
  Capacity Scheduler preemption policy should only consider capacity without labels temporarily
  

• YARN-2932. Major bug reported by Eric Payne and fixed by Eric Payne
  Add entry for "preemptable" status (enabled/disabled) to scheduler web UI and queue initialize/refresh logging
  

• YARN-2931. Critical bug reported by Anubhav Dhoot and fixed by Anubhav Dhoot (nodemanager)
  PublicLocalizer may fail until directory is initialized by LocalizeRunner
  

• YARN-2930. Minor test reported by Ted Yu and fixed by Wangda Tan
  TestRMRestart#testRMRestartRecoveringNodeLabelManager sometimes fails against Java 7 & 8
  

• YARN-2927. Major sub-task reported by Ray Chiang and fixed by Ray Chiang
  InMemorySCMStore properties are inconsistent
  

• YARN-2924. Major sub-task reported by Wangda Tan and fixed by Wangda Tan (client)
  Node to labels mapping should not transfer to lowercase when adding from RMAdminCLI
  

• YARN-2922. Major bug reported by Jason Tufo and fixed by Rohith (capacityscheduler , resourcemanager , scheduler)
  ConcurrentModificationException in CapacityScheduler's LeafQueue
  

• YARN-2920. Major sub-task reported by Wangda Tan and fixed by Wangda Tan
  CapacityScheduler should be notified when labels on nodes changed
  

• YARN-2917. Critical bug reported by Rohith and fixed by Rohith (resourcemanager)
  Potential deadlock in AsyncDispatcher when system.exit called in AsyncDispatcher#dispatch and AsyscDispatcher#serviceStop from shutdown hook
  

• YARN-2914. Minor sub-task reported by Ted Yu and fixed by Varun Saxena
  Potential race condition in Singleton implementation of SharedCacheUploaderMetrics, CleanerMetrics, ClientSCMMetrics
  

• YARN-2912. Major bug reported by Steve Loughran and fixed by Varun Saxena (test)
  Jersey Tests failing with port in use
  

• YARN-2910. Major bug reported by Wilfred Spiegelenburg and fixed by Wilfred Spiegelenburg (fairscheduler)
  FSLeafQueue can throw ConcurrentModificationException
  

• YARN-2907. Trivial bug reported by Rohith and fixed by Rohith (resourcemanager)
  SchedulerNode#toString should print all resource detail instead of only memory.
  

• YARN-2906. Major bug reported by Jason Lowe and fixed by Jason Lowe (capacityscheduler)
  CapacitySchedulerPage shows HTML tags for a queue's Active Users
  

• YARN-2905. Blocker bug reported by Jason Lowe and fixed by Varun Saxena
  AggregatedLogsBlock page can infinitely loop if the aggregated log file is corrupted
  

• YARN-2899. Minor bug reported by Ming Ma and fixed by Ming Ma (nodemanager , test)
  Run TestDockerContainerExecutorWithMocks on Linux only
  

• YARN-2897. Major bug reported by Mit Desai and fixed by Mit Desai
  CrossOriginFilter needs more log statements
  

• YARN-2894. Major bug reported by Rohith and fixed by Rohith (resourcemanager)
  When ACL's are enabled, if RM switches then application can not be viewed from web.
  

• YARN-2891. Minor improvement reported by Dustin Cote and fixed by Dustin Cote (nodemanager)
  Failed Container Executor does not provide a clear error message
  

• YARN-2881. Major sub-task reported by Anubhav Dhoot and fixed by Anubhav Dhoot (fairscheduler)
  Implement PlanFollower for FairScheduler
  

• YARN-2880. Major sub-task reported by Wangda Tan and fixed by Rohith (resourcemanager)
  Add a test in TestRMRestart to make sure node labels will be recovered if it is enabled
  

• YARN-2878. Major improvement reported by Abin Shahab and fixed by Abin Shahab (documentation)
  Fix DockerContainerExecutor.apt.vm formatting
  

• YARN-2874. Blocker bug reported by Naganarasimha G R and fixed by Naganarasimha G R (resourcemanager)
  Dead lock in "DelegationTokenRenewer" which blocks RM to execute any further apps
  

• YARN-2870. Trivial bug reported by Masatake Iwasaki and fixed by Masatake Iwasaki (documentation , timelineserver)
  Update examples in document of Timeline Server
  

• YARN-2869. Major bug reported by Wangda Tan and fixed by Wangda Tan (capacityscheduler , resourcemanager)
  CapacityScheduler should trim sub queue names when parse configuration
  

• YARN-2865. Critical bug reported by Rohith and fixed by Rohith (resourcemanager)
  Application recovery continuously fails with "Application with id already present. Cannot duplicate"
  

• YARN-2861. Major bug reported by Zhijie Shen and fixed by Zhijie Shen
  Timeline DT secret manager should not reuse the RM's configs.
  

• YARN-2857. Critical bug reported by Mohammad Kamrul Islam and fixed by Mohammad Kamrul Islam
  ConcurrentModificationException in ContainerLogAppender
  

• YARN-2856. Critical bug reported by Rohith and fixed by Rohith (resourcemanager)
  Application recovery throw InvalidStateTransitonException: Invalid event: ATTEMPT_KILLED at ACCEPTED
  

• YARN-2854. Critical improvement reported by Zhijie Shen and fixed by Naganarasimha G R (timelineserver)
  The document about timeline service and generic service needs to be updated
  

• YARN-2847. Major bug reported by Jason Lowe and fixed by Olaf Flebbe (nodemanager)
  Linux native container executor segfaults if default banned user detected
  

• YARN-2837. Blocker new feature reported by Zhijie Shen and fixed by Zhijie Shen (timelineserver)
  Timeline server needs to recover the timeline DT when restarting
  

• YARN-2820. Major improvement reported by zhihai xu and fixed by zhihai xu (resourcemanager)
  Retry in FileSystemRMStateStore when FS's operations fail due to IOException.
  

• YARN-2816. Major bug reported by zhihai xu and fixed by zhihai xu (nodemanager)
  NM fail to start with NPE during container recovery
  

• YARN-2815. Major bug reported by Ferdinand Xu and fixed by Ferdinand Xu
  Remove jline from hadoop-yarn-server-common
  

• YARN-2811. Major bug reported by Siqi Li and fixed by Siqi Li
  In Fair Scheduler, reservation fulfillments shouldn't ignore max share
  

• YARN-2809. Major bug reported by Nathan Roberts and fixed by Nathan Roberts (nodemanager)
  Implement workaround for linux kernel panic when removing cgroup
  

• YARN-2808. Major bug reported by Gordon Wang and fixed by Naganarasimha G R (client)
  yarn client tool can not list app_attempt's container info correctly
  

• YARN-2807. Minor sub-task reported by Wangda Tan and fixed by Masatake Iwasaki (documentation , resourcemanager)
  Option "--forceactive" not works as described in usage of "yarn rmadmin -transitionToActive"
  

• YARN-2802. Major improvement reported by zhihai xu and fixed by zhihai xu (resourcemanager)
  ClusterMetrics to include AM launch and register delays
  

• YARN-2800. Major sub-task reported by Wangda Tan and fixed by Wangda Tan (client , resourcemanager)
  Remove MemoryNodeLabelsStore and add a way to enable/disable node labels feature
  

• YARN-2799. Minor improvement reported by zhihai xu and fixed by zhihai xu (test)
  cleanup TestLogAggregationService based on the change in YARN-90
  

• YARN-2797. Minor improvement reported by Karthik Kambatla and fixed by Karthik Kambatla
  TestWorkPreservingRMRestart should use ParametrizedSchedulerTestBase
  

• YARN-2786. Major sub-task reported by Wangda Tan and fixed by Wangda Tan (api , client , resourcemanager)
  Create yarn cluster CLI to enable list node labels collection
  

• YARN-2780. Minor improvement reported by Koji Noguchi and fixed by Eric Payne (resourcemanager)
  Log aggregated resource allocation in rm-appsummary.log
  

• YARN-2777. Major improvement reported by Ted Yu and fixed by Varun Saxena
  Mark the end of individual log in aggregated log
  

• YARN-2766. Major sub-task reported by Robert Kanter and fixed by Robert Kanter (timelineserver)
  ApplicationHistoryManager is expected to return a sorted list of apps/attempts/containers
  

• YARN-2765. Major sub-task reported by Jason Lowe and fixed by Jason Lowe (resourcemanager)
  Add leveldb-based implementation for RMStateStore
  

• YARN-2762. Minor sub-task reported by Rohith and fixed by Rohith (resourcemanager)
  RMAdminCLI node-labels-related args should be trimmed and checked before sending to RM
  

• YARN-2749. Major bug reported by Xuan Gong and fixed by Xuan Gong
  Some testcases from TestLogAggregationService fails in trunk
  

• YARN-2742. Minor bug reported by Sangjin Lee and fixed by Wei Yan (fairscheduler)
  FairSchedulerConfiguration should allow extra spaces between value and unit
  

• YARN-2738. Major sub-task reported by Anubhav Dhoot and fixed by Anubhav Dhoot (fairscheduler)
  Add FairReservationSystem for FairScheduler
  

• YARN-2735. Trivial bug reported by zhihai xu and fixed by zhihai xu (nodemanager)
  diskUtilizationPercentageCutoff and diskUtilizationSpaceCutoff are initialized twice in DirectoryCollection
  

• YARN-2731. Major bug reported by Carlo Curino and fixed by Carlo Curino
  Fixed RegisterApplicationMasterResponsePBImpl to properly invoke maybeInitBuilder
  

• YARN-2713. Major bug reported by Karthik Kambatla and fixed by Karthik Kambatla (resourcemanager)
  "RM Home" link in NM should point to one of the RMs in an HA setup
  

• YARN-2712. Major sub-task reported by Tsuyoshi Ozawa and fixed by Tsuyoshi Ozawa (resourcemanager)
  TestWorkPreservingRMRestart: Augment FS tests with queue and headroom checks
  

• YARN-2697. Major bug reported by Zhijie Shen and fixed by haosdent (resourcemanager)
  RMAuthenticationHandler is no longer useful
  

• YARN-2694. Major sub-task reported by Wangda Tan and fixed by Wangda Tan (capacityscheduler , resourcemanager)
  Ensure only single node labels specified in resource request / host, and node label expression only specified when resourceName=ANY
  

• YARN-2690. Major sub-task reported by Anubhav Dhoot and fixed by Anubhav Dhoot (fairscheduler)
  Make ReservationSystem and its dependent classes independent of Scheduler type
  

• YARN-2683. Major sub-task reported by Steve Loughran and fixed by Steve Loughran (api , resourcemanager)
  registry config options: document and move to core-default
  

• YARN-2679. Major improvement reported by zhihai xu and fixed by zhihai xu (nodemanager)
  Add metric for container launch duration
  

• YARN-2675. Major bug reported by zhihai xu and fixed by zhihai xu (nodemanager)
  containersKilled metrics is not updated when the container is killed during localization
  

• YARN-2669. Major improvement reported by Wei Yan and fixed by Wei Yan
  FairScheduler: queue names shouldn't allow periods
  

• YARN-2643. Trivial improvement reported by Sandy Ryza and fixed by Karthik Kambatla
  Don't create a new DominantResourceCalculator on every FairScheduler.allocate call
  

• YARN-2641. Major improvement reported by zhihai xu and fixed by zhihai xu (resourcemanager)
  Decommission nodes on -refreshNodes instead of next NM-RM heartbeat
  

• YARN-2637. Critical bug reported by Wangda Tan and fixed by Craig Welch (resourcemanager)
  maximum-am-resource-percent could be respected for both LeafQueue/User when trying to activate applications.
  

• YARN-2616. Major sub-task reported by Steve Loughran and fixed by Akshay Radia (client)
  Add CLI client to the registry to list, view and manipulate entries
  

• YARN-2604. Major improvement reported by Karthik Kambatla and fixed by Robert Kanter (scheduler)
  Scheduler should consider max-allocation-* in conjunction with the largest node
  

• YARN-2574. Major new feature reported by Subru Krishnan and fixed by Anubhav Dhoot (fairscheduler)
  Add support for FairScheduler to the ReservationSystem
  

• YARN-2543. Major sub-task reported by Zhijie Shen and fixed by Naganarasimha G R (timelineserver)
  Resource usage should be published to the timeline server as well
  

• YARN-2461. Minor bug reported by Ray Chiang and fixed by Ray Chiang
  Fix PROCFS_USE_SMAPS_BASED_RSS_ENABLED property in YarnConfiguration
  

• YARN-2432. Major bug reported by Varun Saxena and fixed by Varun Saxena (resourcemanager)
  RMStateStore should process the pending events before close
  

• YARN-2427. Major new feature reported by Varun Vasudev and fixed by Varun Vasudev (resourcemanager)
  Add support for moving apps between queues in RM web services
  

• YARN-2414. Major bug reported by Zhijie Shen and fixed by Wangda Tan (webapp)
  RM web UI: app page will crash if app is failed before any attempt has been created
  

• YARN-2404. Major sub-task reported by Jian He and fixed by Tsuyoshi Ozawa
  Remove ApplicationAttemptState and ApplicationState class in RMStateStore class
  

• YARN-2375. Major sub-task reported by Jonathan Eagles and fixed by Mit Desai
  Allow enabling/disabling timeline server per framework
  

• YARN-2360. Major new feature reported by Ashwin Shankar and fixed by Ashwin Shankar (fairscheduler)
  Fair Scheduler: Display dynamic fair share for queues on the scheduler page
  

• YARN-2356. Minor bug reported by Sunil G and fixed by Sunil G (client)
  yarn status command for non-existent application/application attempt/container is too verbose
  

• YARN-2340. Critical bug reported by Nishan Shetty and fixed by Rohith (resourcemanager , scheduler)
  NPE thrown when RM restart after queue is STOPPED. There after RM can not recovery application's and remain in standby
  

• YARN-2315. Major bug reported by zhihai xu and fixed by zhihai xu
  FairScheduler: Set current capacity in addition to capacity
  

• YARN-2301. Major improvement reported by Jian He and fixed by Naganarasimha G R
  Improve yarn container command
  

• YARN-2254. Minor improvement reported by zhihai xu and fixed by zhihai xu
  TestRMWebServicesAppsModification should run against both CS and FS
  

• YARN-2246. Major bug reported by Devaraj K and fixed by Devaraj K (webapp)
  Job History Link in RM UI is redirecting to the URL which contains Job Id twice
  

• YARN-2243. Minor bug reported by Ted Yu and fixed by Devaraj K
  Order of arguments for Preconditions.checkNotNull() is wrong in SchedulerApplicationAttempt ctor
  

• YARN-2236. Major sub-task reported by Chris Trezzo and fixed by Chris Trezzo
  Shared Cache uploader service on the Node Manager
  

• YARN-2230. Minor bug reported by Adam Kawa and fixed by Vijay Bhat (client , documentation , scheduler)
  Fix description of yarn.scheduler.maximum-allocation-vcores in yarn-default.xml (or code)
  

  I have modified the description of the yarn.scheduler.maximum-allocation-vcores setting in yarn-default.xml to be reflective of the actual behavior (throw InvalidRequestException when the limit is crossed). Since this is a documentation change, I have not added any test cases. Please review the patch, thanks!

• YARN-2217. Major sub-task reported by Chris Trezzo and fixed by Chris Trezzo
  Shared cache client side changes
  

• YARN-2213. Minor task reported by Ted Yu and fixed by Varun Saxena
  Change proxy-user cookie log in AmIpFilter to DEBUG
  

• YARN-2203. Major sub-task reported by Chris Trezzo and fixed by Chris Trezzo
  Web UI for cache manager
  

• YARN-2190. Major new feature reported by Chuan Liu and fixed by Chuan Liu (nodemanager)
  Add CPU and memory limit options to the default container executor for Windows containers
  

• YARN-2189. Major sub-task reported by Chris Trezzo and fixed by Chris Trezzo
  Admin service for cache manager
  

• YARN-2188. Major sub-task reported by Chris Trezzo and fixed by Chris Trezzo
  Client service for cache manager
  

• YARN-2186. Major sub-task reported by Chris Trezzo and fixed by Chris Trezzo
  Node Manager uploader service for cache manager
  

• YARN-2183. Major sub-task reported by Chris Trezzo and fixed by Chris Trezzo
  Cleaner service for cache manager
  

• YARN-2180. Major sub-task reported by Chris Trezzo and fixed by Chris Trezzo
  In-memory backing store for cache manager
  

• YARN-2179. Major sub-task reported by Chris Trezzo and fixed by Chris Trezzo
  Initial cache manager structure and context
  

• YARN-2165. Major sub-task reported by Karam Singh and fixed by Vasanth kumar RJ (timelineserver)
  Timeline server should validate the numeric configuration values
  

• YARN-2157. Major improvement reported by Akira AJISAKA and fixed by Akira AJISAKA (documentation)
  Document YARN metrics
  

• YARN-2136. Major bug reported by Jian He and fixed by Varun Saxena
  RMStateStore can explicitly handle store/update events when fenced
  

• YARN-2079. Major sub-task reported by Jason Lowe and fixed by Jason Lowe (nodemanager)
  Recover NonAggregatingLogHandler state upon nodemanager restart
  

• YARN-2056. Major sub-task reported by Mayank Bansal and fixed by Eric Payne (resourcemanager)
  Disable preemption at Queue level
  

• YARN-1984. Major sub-task reported by Jason Lowe and fixed by Varun Saxena
  LeveldbTimelineStore does not handle db exceptions properly
  

• YARN-1979. Major test reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli
  TestDirectoryCollection fails when the umask is unusual
  

• YARN-1904. Major sub-task reported by Zhijie Shen and fixed by Zhijie Shen
  Uniform the XXXXNotFound messages from ClientRMService and ApplicationHistoryClientService
  

  I just committed this. Thanks Zhijie!

• YARN-1884. Major sub-task reported by Zhijie Shen and fixed by Xuan Gong
  ContainerReport should have nodeHttpAddress
  

• YARN-1809. Major sub-task reported by Zhijie Shen and fixed by Xuan Gong
  Synchronize RM and Generic History Service Web-UIs
  

• YARN-1723. Major sub-task reported by Bikas Saha and fixed by Bartosz Ługowski
  AMRMClientAsync missing blacklist addition and removal functionality
  

• YARN-1703. Critical bug reported by Rohith and fixed by Rohith
  Too many connections are opened for proxy server when applicationMaster UI is accessed.
  

• YARN-1615. Trivial bug reported by Akira AJISAKA and fixed by Akira AJISAKA (documentation , scheduler)
  Fix typos in description about delay scheduling
  

• YARN-1582. Major improvement reported by Thomas Graves and fixed by Thomas Graves (capacityscheduler)
  Capacity Scheduler: add a maximum-allocation-mb setting per queue
  

• YARN-1580. Trivial bug reported by German Florez-Larrahondo and fixed by Brahma Reddy Battula (documentation)
  Documentation error regarding "container-allocation.expiry-interval-ms"
  

• YARN-1537. Major test reported by Hong Shen and fixed by Xuan Gong (nodemanager)
  TestLocalResourcesTrackerImpl.testLocalResourceCache often failed
  

• YARN-1514. Major sub-task reported by Tsuyoshi Ozawa and fixed by Tsuyoshi Ozawa
  Utility to benchmark ZKRMStateStore#loadState for ResourceManager-HA
  

• YARN-1453. Minor bug reported by Andrew Purtell and fixed by Akira AJISAKA
  [JDK8] Fix Javadoc errors caused by incorrect or illegal tags in doc comments
  

• YARN-1393. Major improvement reported by Wei Yan and fixed by Wei Yan
  SLS: Add how-to-use instructions
  

• YARN-1299. Major improvement reported by Devaraj K and fixed by (resourcemanager)
  Improve a log message in AppSchedulingInfo by adding application id
  

• YARN-1237. Minor bug reported by Hitesh Shah and fixed by Brahma Reddy Battula (documentation)
  Description for yarn.nodemanager.aux-services in yarn-default.xml is misleading
  

• YARN-1156. Major improvement reported by Akira AJISAKA and fixed by Tsuyoshi Ozawa
  Enhance NodeManager AllocatedGB and AvailableGB metrics for aggregation of decimal values
  

• YARN-933. Major bug reported by J.Andreina and fixed by Rohith (resourcemanager)
  Potential InvalidStateTransitonException: Invalid event: LAUNCHED at FINAL_SAVING
  

• YARN-570. Major bug reported by Peng Zhang and fixed by Akira AJISAKA (webapp)
  Time strings are formated in different timezone
  

• MAPREDUCE-6303. Blocker bug reported by Jason Lowe and fixed by Jason Lowe
  Read timeout when retrying a fetch error can be fatal to a reducer
  

• MAPREDUCE-6292. Minor improvement reported by Akira AJISAKA and fixed by Akira AJISAKA (test)
  Use org.junit package instead of junit.framework in TestCombineFileInputFormat
  

• MAPREDUCE-6285. Major bug reported by Jonathan Eagles and fixed by Jonathan Eagles
  ClientServiceDelegate should not retry upon AuthenticationException
  

• MAPREDUCE-6277. Major bug reported by Chang Li and fixed by Chang Li (mr-am)
  Job can post multiple history files if attempt loses connection to the RM
  

• MAPREDUCE-6275. Critical bug reported by Siqi Li and fixed by Gera Shegalov
  Race condition in FileOutputCommitter v2 for user-specified task output subdirs
  

• MAPREDUCE-6268. Minor bug reported by Ryu Kobayashi and fixed by Ryu Kobayashi
  Fix typo in Task Attempt API's URL
  

• MAPREDUCE-6267. Minor improvement reported by Chris Trezzo and fixed by Chris Trezzo
  Refactor JobSubmitter#copyAndConfigureFiles into it's own class
  

• MAPREDUCE-6265. Major improvement reported by zhihai xu and fixed by zhihai xu (mrv2)
  Make ContainerLauncherImpl.INITIAL_POOL_SIZE configurable to better control to launch/kill containers
  

• MAPREDUCE-6264. Major task reported by Akira AJISAKA and fixed by Brahma Reddy Battula
  Remove httpclient dependency from hadoop-mapreduce-client
  

• MAPREDUCE-6263. Major bug reported by Jason Lowe and fixed by Eric Payne (client)
  Configurable timeout between YARNRunner terminate the application and forcefully kill.
  

• MAPREDUCE-6261. Major bug reported by Jason Lowe and fixed by Tsuyoshi Ozawa (mrv2)
  NullPointerException if MapOutputBuffer.flush invoked twice
  

• MAPREDUCE-6256. Minor improvement reported by Devaraj K and fixed by Naganarasimha G R
  Removed unused private methods in o.a.h.mapreduce.Job.java
  

• MAPREDUCE-6255. Minor improvement reported by Ryu Kobayashi and fixed by Ryu Kobayashi (client)
  Fix JobCounter's format to use grouping separator
  

• MAPREDUCE-6253. Minor improvement reported by Ray Chiang and fixed by Ray Chiang
  Update use of Iterator to Iterable
  

• MAPREDUCE-6248. Major improvement reported by Jing Zhao and fixed by Jing Zhao (distcp)
  Allow users to get the MR job information for distcp
  

• MAPREDUCE-6243. Minor bug reported by Akira AJISAKA and fixed by Masatake Iwasaki (tools/rumen)
  Fix findbugs warnings in hadoop-rumen
  

• MAPREDUCE-6233. Major bug reported by Yongjun Zhang and fixed by zhihai xu (test)
  org.apache.hadoop.mapreduce.TestLargeSort.testLargeSort failed in trunk
  

• MAPREDUCE-6231. Major bug reported by Akira AJISAKA and fixed by Akira AJISAKA (examples)
  Grep example job is not working on a fully-distributed cluster
  

• MAPREDUCE-6230. Blocker bug reported by Jason Lowe and fixed by Jason Lowe (mr-am)
  MR AM does not survive RM restart if RM activated a new AMRM secret key
  

• MAPREDUCE-6228. Major new feature reported by Konstantin Shvachko and fixed by Plamen Jeliazkov (benchmarks , test)
  Add truncate operation to SLive
  

• MAPREDUCE-6227. Major new feature reported by Konstantin Shvachko and fixed by Konstantin Shvachko (benchmarks , test)
  DFSIO for truncate
  

• MAPREDUCE-6225. Major bug reported by Jason Lowe and fixed by Varun Saxena
  Fix new findbug warnings in hadoop-mapreduce-client-core
  

• MAPREDUCE-6221. Minor bug reported by Ted Yu and fixed by Ted Yu
  Stringifier is left unclosed in Chain#getChainElementConf()
  

• MAPREDUCE-6210. Minor bug reported by Leitao Guo and fixed by Leitao Guo (applicationmaster)
  Use getApplicationAttemptId() instead of getApplicationID() for logging AttemptId in RMContainerAllocator.java
  

• MAPREDUCE-6206. Critical bug reported by Jens Rabe and fixed by Jens Rabe
  TestAggregatedTransferRate fails on non-US systems
  

• MAPREDUCE-6194. Minor improvement reported by Harsh J and fixed by Varun Saxena (task)
  Bubble up final exception in failures during creation of output collectors
  

• MAPREDUCE-6186. Minor bug reported by Rohit Agarwal and fixed by Rohit Agarwal (jobhistoryserver)
  Redundant call to requireJob() while displaying the conf page
  

• MAPREDUCE-6177. Trivial bug reported by yangping wu and fixed by yangping wu (documentation)
  Minor typo in the EncryptedShuffle document about ssl-client.xml
  

• MAPREDUCE-6173. Major improvement reported by Junping Du and fixed by Junping Du (distributed-cache , documentation)
  Document the configuration of deploying MR over distributed cache with enabling wired encryption at the same time
  

• MAPREDUCE-6172. Minor bug reported by Jason Lowe and fixed by Varun Saxena (test)
  TestDbClasses timeouts are too aggressive
  

• MAPREDUCE-6169. Major improvement reported by zhihai xu and fixed by zhihai xu (mrv2)
  MergeQueue should release reference to the current item from key and value at the end of the iteration to save memory.
  

• MAPREDUCE-6166. Major bug reported by Eric Payne and fixed by Eric Payne (mrv2)
  Reducers do not validate checksum of map outputs when fetching directly to disk
  

• MAPREDUCE-6162. Blocker bug reported by Jason Lowe and fixed by Jason Lowe (jobhistoryserver)
  mapred hsadmin fails on a secure cluster
  

• MAPREDUCE-6160. Major bug reported by Rohith and fixed by Rohith
  Potential NullPointerException in MRClientProtocol interface implementation.
  

• MAPREDUCE-6151. Minor improvement reported by Masatake Iwasaki and fixed by Masatake Iwasaki (documentation)
  Update document of GridMix
  

• MAPREDUCE-6150. Minor improvement reported by Masatake Iwasaki and fixed by Masatake Iwasaki (documentation)
  Update document of Rumen
  

• MAPREDUCE-6149. Major improvement reported by Junping Du and fixed by Junping Du (documentation)
  Document override log4j.properties in MR job
  

• MAPREDUCE-6143. Major improvement reported by zhihai xu and fixed by zhihai xu (mrv2)
  add configuration for mapreduce speculative execution in MR2
  

• MAPREDUCE-6141. Major improvement reported by Jason Lowe and fixed by Jason Lowe (jobhistoryserver)
  History server leveldb recovery store
  

• MAPREDUCE-6136. Major bug reported by Noah Watkins and fixed by Brahma Reddy Battula (applicationmaster)
  MRAppMaster doesn't shutdown file systems
  

• MAPREDUCE-6059. Major improvement reported by Siqi Li and fixed by Siqi Li
  Speed up history server startup time
  

• MAPREDUCE-6049. Major bug reported by Nishan Shetty and fixed by Rohith (applicationmaster , resourcemanager)
  AM JVM does not exit if MRClientService gracefull shutdown fails
  

• MAPREDUCE-6046. Minor improvement reported by Devaraj K and fixed by Sahil Takiar (mr-am)
  Change the class name for logs in RMCommunicator.java
  

• MAPREDUCE-6045. Minor bug reported by zhihai xu and fixed by zhihai xu (test)
  need close the DataInputStream after open it in TestMapReduce.java
  

• MAPREDUCE-5988. Minor bug reported by Akira AJISAKA and fixed by Akira AJISAKA (documentation)
  Fix dead links to the javadocs in mapreduce project
  

• MAPREDUCE-5932. Major improvement reported by Gera Shegalov and fixed by Gera Shegalov (mrv2)
  Provide an option to use a dedicated reduce-side shuffle log
  

• MAPREDUCE-5918. Major bug reported by Sergey Murylev and fixed by Sergey Murylev
  LineRecordReader can return the same decompressor to CodecPool multiple times
  

• MAPREDUCE-5800. Minor improvement reported by Akira AJISAKA and fixed by Akira AJISAKA
  Use Job#getInstance instead of deprecated constructors
  

• MAPREDUCE-5657. Minor bug reported by Andrew Purtell and fixed by Akira AJISAKA (documentation)
  [JDK8] Fix Javadoc errors caused by incorrect or illegal tags in doc comments
  

• MAPREDUCE-5612. Minor improvement reported by Sandy Ryza and fixed by Chris Palmer (documentation)
  Add javadoc for TaskCompletionEvent.Status
  

• MAPREDUCE-5583. Major improvement reported by Jason Lowe and fixed by Jason Lowe (mr-am , mrv2)
  Ability to limit running map and reduce tasks
  

• MAPREDUCE-5568. Major bug reported by Jian He and fixed by MinJi Kim
  JHS returns invalid string for reducer completion percentage if AM restarts with 0 reducer.
  

• MAPREDUCE-5420. Major task reported by Sandy Ryza and fixed by James Carman
  Remove mapreduce.task.tmp.dir from mapred-default.xml
  

• MAPREDUCE-5335. Major improvement reported by Devaraj K and fixed by Devaraj K (applicationmaster)
  Rename Job Tracker terminology in ShuffleSchedulerImpl
  

• MAPREDUCE-4879. Major bug reported by Gera Shegalov and fixed by Gera Shegalov (examples)
  TeraOutputFormat may overwrite an existing output directory
  

• MAPREDUCE-4815. Major improvement reported by Jason Lowe and fixed by Siqi Li (mrv2)
  Speed up FileOutputCommitter#commitJob for many output files
  

• MAPREDUCE-4742. Trivial bug reported by Liang Xie and fixed by Liang Xie (test)
  Fix typo in nnbench#displayUsage
  

• MAPREDUCE-4431. Minor improvement reported by Nishan Shetty and fixed by Devaraj K (mrv2)
  mapred command should print the reason on killing already completed jobs
  

• MAPREDUCE-4286. Major bug reported by Devaraj K and fixed by Devaraj K
  TestClientProtocolProviderImpls passes on failure conditions
  

• MAPREDUCE-3283. Minor bug reported by Ramya Sunil and fixed by Varun Saxena (scripts)
  mapred classpath CLI does not display the complete classpath
  

• MAPREDUCE-2815. Minor bug reported by Shane Butler and fixed by Chris Palmer (documentation)
  JavaDoc does not generate correctly for MultithreadedMapRunner
  

• HDFS-8072. Major bug reported by Arpit Agarwal and fixed by Arpit Agarwal (datanode)
  Reserved RBW space is not released if client terminates while writing block
  

• HDFS-8071. Major improvement reported by Konstantin Shvachko and fixed by Konstantin Shvachko
  Redundant checkFileProgress() in PART II of getAdditionalBlock()
  

• HDFS-8063. Minor bug reported by Masatake Iwasaki and fixed by Masatake Iwasaki (test)
  Fix intermittent test failures in TestTracing
  

• HDFS-8051. Major bug reported by Lei (Eddy) Xu and fixed by Lei (Eddy) Xu (datanode)
  FsVolumeList#addVolume should release volume reference if not put it into BlockScanner.
  

• HDFS-8039. Minor sub-task reported by Xiaoyu Yao and fixed by Xiaoyu Yao (test)
  Fix TestDebugAdmin#testRecoverLease and testVerfiyBlockChecksumCommand on Windows
  

• HDFS-8038. Minor sub-task reported by Xiaoyu Yao and fixed by Xiaoyu Yao (tools)
  PBImageDelimitedTextWriter#getEntry output HDFS path in platform-specific format.
  

• HDFS-8036. Major bug reported by Jing Zhao and fixed by Jing Zhao (distcp)
  Use snapshot path as source when using snapshot diff report in DistCp
  

• HDFS-8025. Major bug reported by Ming Ma and fixed by Ming Ma
  Addendum fix for HDFS-3087 Decomissioning on NN restart can complete without blocks being replicated
  

• HDFS-8001. Trivial bug reported by Remi Catherinot and fixed by Remi Catherinot (nfs)
  RpcProgramNfs3 : wrong parsing of dfs.blocksize
  

  patch is fully backward compatible.

• HDFS-7999. Major bug reported by zhouyingchao and fixed by zhouyingchao
  FsDatasetImpl#createTemporary sometimes holds the FSDatasetImpl lock for a very long time
  

• HDFS-7996. Critical bug reported by Lei (Eddy) Xu and fixed by Lei (Eddy) Xu (datanode)
  After swapping a volume, BlockReceiver reports ReplicaNotFoundException
  

• HDFS-7977. Major bug reported by Brandon Li and fixed by Brandon Li (nfs)
  NFS couldn't take percentile intervals
  

• HDFS-7976. Major improvement reported by Brandon Li and fixed by Brandon Li (documentation , nfs)
  Update NFS user guide for mount option "sync" to minimize or avoid reordered writes
  

• HDFS-7963. Critical bug reported by Masatake Iwasaki and fixed by Masatake Iwasaki (test)
  Fix expected tracing spans in TestTracing along with HDFS-7054
  

• HDFS-7962. Minor improvement reported by Yi Liu and fixed by Yi Liu
  Remove duplicated logs in BlockManager
  

• HDFS-7961. Major bug reported by Lei (Eddy) Xu and fixed by Lei (Eddy) Xu
  Trigger full block report after hot swapping disk
  

• HDFS-7960. Critical bug reported by Lei (Eddy) Xu and fixed by Colin Patrick McCabe
  The full block report should prune zombie storages even if they're not empty
  

• HDFS-7957. Critical bug reported by Jing Zhao and fixed by Jing Zhao (namenode)
  Truncate should verify quota before making changes
  

• HDFS-7956. Major bug reported by Konstantin Shvachko and fixed by Plamen Jeliazkov (namenode)
  Improve logging for DatanodeRegistration.
  

• HDFS-7954. Minor sub-task reported by Xiaoyu Yao and fixed by Xiaoyu Yao (test)
  TestBalancer#testBalancerWithPinnedBlocks failed on Windows
  

• HDFS-7953. Minor bug reported by kanaka kumar avvaru and fixed by kanaka kumar avvaru (namenode)
  NN Web UI fails to navigate to paths that contain #
  

• HDFS-7951. Major sub-task reported by Xiaoyu Yao and fixed by Xiaoyu Yao (test)
  Fix NPE for TestFsDatasetImpl#testAddVolumeFailureReleasesInUseLock on Linux
  

• HDFS-7950. Major sub-task reported by Xiaoyu Yao and fixed by Xiaoyu Yao (test)
  Fix TestFsDatasetImpl#testAddVolumes failure on Windows
  

• HDFS-7948. Major sub-task reported by Xiaoyu Yao and fixed by Xiaoyu Yao (test)
  TestDataNodeHotSwapVolumes#testAddVolumeFailures failed on Windows
  

• HDFS-7946. Major sub-task reported by Xiaoyu Yao and fixed by Xiaoyu Yao (test)
  TestDataNodeVolumeFailureReporting NPE on Windows
  

• HDFS-7945. Blocker bug reported by Haohui Mai and fixed by Haohui Mai
  The WebHdfs system on DN does not honor the length parameter
  

• HDFS-7943. Blocker bug reported by Jing Zhao and fixed by Jing Zhao
  Append cannot handle the last block with length greater than the preferred block size
  

• HDFS-7942. Major bug reported by Brandon Li and fixed by Brandon Li (nfs)
  NFS: support regexp grouping in nfs.exports.allowed.hosts
  

• HDFS-7940. Major sub-task reported by Rakesh R and fixed by Rakesh R (hdfs-client)
  Add tracing to DFSClient#setQuotaByStorageType
  

• HDFS-7932. Major bug reported by Kihwal Lee and fixed by Kihwal Lee
  Speed up the shutdown of datanode during rolling upgrade
  

• HDFS-7930. Blocker bug reported by Konstantin Shvachko and fixed by Yi Liu (namenode)
  commitBlockSynchronization() does not remove locations
  

• HDFS-7929. Major bug reported by Zhe Zhang and fixed by Zhe Zhang
  inotify unable fetch pre-upgrade edit log segments once upgrade starts
  

• HDFS-7926. Major bug reported by Tsz Wo Nicholas Sze and fixed by Tsz Wo Nicholas Sze (namenode)
  NameNode implementation of ClientProtocol.truncate(..) is not idempotent
  

• HDFS-7917. Minor improvement reported by Lei (Eddy) Xu and fixed by Lei (Eddy) Xu (test)
  Use file to replace data dirs in test to simulate a disk failure.
  

• HDFS-7915. Major bug reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe
  The DataNode can sometimes allocate a ShortCircuitShm slot and fail to tell the DFSClient about it because of a network error
  

• HDFS-7914. Major test reported by Xiaoyu Yao and fixed by Xiaoyu Yao
  TestJournalNode#testFailToStartWithBadConfig fails when the default dfs.journalnode.http-address port 8480 is in use
  

• HDFS-7903. Blocker sub-task reported by Tsz Wo Nicholas Sze and fixed by Plamen Jeliazkov (datanode , namenode)
  Cannot recover block after truncate and delete snapshot
  

• HDFS-7898. Minor improvement reported by Tsz Wo Nicholas Sze and fixed by Tsz Wo Nicholas Sze (test)
  Change TestAppendSnapshotTruncate to fail-fast
  

• HDFS-7886. Minor bug reported by Yi Liu and fixed by Plamen Jeliazkov (test)
  TestFileTruncate#testTruncateWithDataNodesRestart runs timeout sometimes
  

• HDFS-7885. Critical bug reported by vitthal (Suhas) Gogate and fixed by Tsz Wo Nicholas Sze (datanode)
  Datanode should not trust the generation stamp provided by client
  

• HDFS-7884. Blocker bug reported by Tsz Wo Nicholas Sze and fixed by Brahma Reddy Battula (datanode)
  NullPointerException in BlockSender
  

• HDFS-7881. Blocker bug reported by Akira AJISAKA and fixed by Brahma Reddy Battula
  TestHftpFileSystem#testSeek fails in branch-2
  

• HDFS-7880. Blocker bug reported by Akira AJISAKA and fixed by Brahma Reddy Battula (test)
  Remove the tests for legacy Web UI in branch-2
  

• HDFS-7879. Major bug reported by Chris Nauroth and fixed by Chris Nauroth (build , libhdfs)
  hdfs.dll does not export functions of the public libhdfs API
  

• HDFS-7871. Critical bug reported by Jing Zhao and fixed by Jing Zhao
  NameNodeEditLogRoller can keep printing "Swallowing exception" message
  

• HDFS-7869. Major bug reported by J.Andreina and fixed by J.Andreina
  Inconsistency in the return information while performing rolling upgrade
  

• HDFS-7855. Major sub-task reported by Li Bo and fixed by Li Bo (hdfs-client)
  Separate class Packet from DFSOutputStream
  

• HDFS-7849. Minor improvement reported by Tsz Wo Nicholas Sze and fixed by J.Andreina (documentation)
  Update documentation for enabling a new feature in rolling upgrade
  

• HDFS-7843. Blocker sub-task reported by Tsz Wo Nicholas Sze and fixed by Tsz Wo Nicholas Sze (namenode)
  A truncated file is corrupted after rollback from a rolling upgrade
  

• HDFS-7838. Major sub-task reported by Yi Liu and fixed by Yi Liu (datanode , namenode)
  Expose truncate API for libhdfs
  

• HDFS-7832. Major improvement reported by Vinayakumar B and fixed by Vinayakumar B (namenode)
  Show 'Last Modified' in Namenode's 'Browse Filesystem'
  

• HDFS-7831. Major bug reported by Konstantin Shvachko and fixed by Konstantin Shvachko
  Fix the starting index and end condition of the loop in FileDiffList.findEarlierSnapshotBlocks()
  

• HDFS-7830. Major bug reported by Lei (Eddy) Xu and fixed by Lei (Eddy) Xu (datanode)
  DataNode does not release the volume lock when adding a volume fails.
  

• HDFS-7824. Major sub-task reported by Xiaoyu Yao and fixed by Xiaoyu Yao (datanode , namenode)
  GetContentSummary API and its namenode implementation for Storage Type Quota/Usage
  

• HDFS-7819. Major sub-task reported by Rakesh R and fixed by Rakesh R (datanode)
  Log WARN message for the blocks which are not in Block ID based layout
  

• HDFS-7818. Blocker bug reported by Eric Payne and fixed by Eric Payne (webhdfs)
  OffsetParam should return the default value instead of throwing NPE when the value is unspecified
  

• HDFS-7816. Blocker bug reported by Jason Lowe and fixed by Haohui Mai (webhdfs)
  Unable to open webhdfs paths with "+"
  

• HDFS-7814. Minor sub-task reported by Xiaoyu Yao and fixed by Xiaoyu Yao (datanode , namenode)
  Fix usage string of storageType parameter for "dfsadmin -setSpaceQuota/clrSpaceQuota"
  

• HDFS-7813. Major bug reported by Rakesh R and fixed by Rakesh R (ha , test)
  TestDFSHAAdminMiniCluster#testFencer testcase is failing frequently
  

• HDFS-7811. Minor sub-task reported by Xiaoyu Yao and fixed by Xiaoyu Yao (namenode)
  Avoid recursive call getStoragePolicyID in INodeFile#computeQuotaUsage
  

• HDFS-7807. Major bug reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (native)
  libhdfs htable.c: fix htable resizing, add unit test
  

• HDFS-7806. Minor sub-task reported by Xiaoyu Yao and fixed by Xiaoyu Yao (datanode , namenode)
  Refactor: move StorageType from hadoop-hdfs to hadoop-common
  

  This fix moves the public class StorageType from the package org.apache.hadoop.hdfs to org.apache.hadoop.fs.

• HDFS-7805. Major bug reported by surendra singh lilhore and fixed by surendra singh lilhore (namenode)
  NameNode recovery prompt should be printed on console
  

• HDFS-7798. Critical bug reported by Chengbing Liu and fixed by Chengbing Liu (security)
  Checkpointing failure caused by shared KerberosAuthenticator
  

• HDFS-7797. Major improvement reported by Rakesh R and fixed by Rakesh R (namenode)
  Add audit log for setQuota operation
  

• HDFS-7795. Minor improvement reported by Kihwal Lee and fixed by Kihwal Lee
  Show warning if not all favored nodes were chosen by namenode
  

• HDFS-7790. Minor improvement reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (hdfs-client)
  Do not create optional fields in DFSInputStream unless they are needed
  

• HDFS-7789. Major improvement reported by Gera Shegalov and fixed by Gera Shegalov (tools)
  DFSck should resolve the path to support cross-FS symlinks
  

• HDFS-7788. Blocker bug reported by Kihwal Lee and fixed by Rushabh S Shah
  Post-2.6 namenode may not start up with an image containing inodes created with an old release.
  

• HDFS-7785. Major bug reported by Chengbing Liu and fixed by Chengbing Liu (namenode)
  Improve diagnostics information for HttpPutFailedException
  

• HDFS-7780. Minor improvement reported by Ray Chiang and fixed by Ray Chiang
  Update use of Iterator to Iterable in DataXceiverServer and SnapshotDiffInfo
  

• HDFS-7778. Major bug reported by Lei (Eddy) Xu and fixed by Lei (Eddy) Xu (datanode , test)
  Rename FsVolumeListTest to TestFsVolumeList and commit it to branch-2
  

• HDFS-7776. Major sub-task reported by Xiaoyu Yao and fixed by Xiaoyu Yao (test)
  Adding additional unit tests for Quota By Storage Type
  

• HDFS-7775. Minor sub-task reported by Xiaoyu Yao and fixed by Xiaoyu Yao (datanode , namenode)
  Use consistent naming for NN-internal quota related types and functions
  

• HDFS-7774. Critical bug reported by Venkatasubramaniam Ramakrishnan and fixed by Kiran Kumar M R (build , native)
  Unresolved symbols error while compiling HDFS on Windows 7/32 bit
  

  LibHDFS now supports 32-bit build targets on Windows.

• HDFS-7773. Major improvement reported by Anu Engineer and fixed by Anu Engineer (datanode , namenode)
  Additional metrics in HDFS to be accessed via jmx.
  

• HDFS-7772. Trivial improvement reported by Xiaoyu Yao and fixed by Xiaoyu Yao (documentation)
  Document hdfs balancer -exclude/-include option in HDFSCommands.html
  

• HDFS-7771. Major improvement reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe
  fuse_dfs should permit FILE: on the front of KRB5CCNAME
  

• HDFS-7769. Trivial bug reported by Tsz Wo Nicholas Sze and fixed by (test)
  TestHDFSCLI create files in hdfs project root dir
  

• HDFS-7763. Major bug reported by Liang Xie and fixed by Liang Xie (ha)
  fix zkfc hung issue due to not catching exception in a corner case
  

• HDFS-7761. Minor improvement reported by Yi Liu and fixed by Yi Liu
  cleanup unnecssary code logic in LocatedBlock
  

• HDFS-7760. Minor sub-task reported by Yi Liu and fixed by Konstantin Shvachko (documentation)
  Document truncate for WebHDFS.
  

• HDFS-7757. Major improvement reported by Arpit Agarwal and fixed by Brahma Reddy Battula (namenode)
  Misleading error messages in FSImage.java
  

• HDFS-7756. Major bug reported by Ted Yu and fixed by Ted Yu
  Restore method signature for LocatedBlock#getLocations()
  

• HDFS-7753. Major bug reported by Rakesh R and fixed by Konstantin Shvachko
  Fix Multithreaded correctness Warnings in BackupImage.java
  

• HDFS-7752. Minor improvement reported by Wellington Chevreuil and fixed by Wellington Chevreuil (documentation)
  Improve description for "dfs.namenode.num.extra.edits.retained" and "dfs.namenode.num.checkpoints.retained" properties on hdfs-default.xml
  

• HDFS-7748. Blocker bug reported by Haohui Mai and fixed by Anu Engineer
  Separate ECN flags from the Status in the DataTransferPipelineAck
  

• HDFS-7746. Minor sub-task reported by Tsz Wo Nicholas Sze and fixed by Tsz Wo Nicholas Sze (test)
  Add a test randomly mixing append, truncate and snapshot
  

• HDFS-7744. Major bug reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (hdfs-client)
  Fix potential NPE in DFSInputStream after setDropBehind or setReadahead is called
  

• HDFS-7743. Minor improvement reported by Jing Zhao and fixed by Jing Zhao (namenode)
  Code cleanup of BlockInfo and rename BlockInfo to BlockInfoContiguous
  

• HDFS-7742. Major bug reported by Nathan Roberts and fixed by Nathan Roberts (namenode)
  favoring decommissioning node for replication can cause a block to stay underreplicated for long periods
  

• HDFS-7741. Minor bug reported by Yi Liu and fixed by Yi Liu
  Remove unnecessary synchronized in FSDataInputStream and HdfsDataInputStream
  

• HDFS-7740. Major sub-task reported by Konstantin Shvachko and fixed by Yi Liu (test)
  Test truncate with DataNodes restarting
  

• HDFS-7738. Minor sub-task reported by Tsz Wo Nicholas Sze and fixed by Tsz Wo Nicholas Sze (test)
  Add more tests for truncate
  

• HDFS-7734. Blocker bug reported by Arpit Agarwal and fixed by Yi Liu (namenode)
  Class cast exception in NameNode#main
  

• HDFS-7732. Trivial improvement reported by Akira AJISAKA and fixed by Brahma Reddy Battula
  Fix the order of the parameters in DFSConfigKeys
  

• HDFS-7723. Major sub-task reported by Xiaoyu Yao and fixed by Xiaoyu Yao (namenode)
  Quota By Storage Type namenode implemenation
  

• HDFS-7722. Major bug reported by Lei (Eddy) Xu and fixed by Lei (Eddy) Xu (datanode)
  DataNode#checkDiskError should also remove Storage when error is found.
  

• HDFS-7721. Major bug reported by Tsz Wo Nicholas Sze and fixed by Colin Patrick McCabe (datanode)
  The HDFS BlockScanner may run fast during the first hour
  

• HDFS-7720. Major sub-task reported by Xiaoyu Yao and fixed by Xiaoyu Yao (datanode , namenode)
  Quota by Storage Type API, tools and ClientNameNode Protocol changes
  

• HDFS-7719. Major bug reported by Lei (Eddy) Xu and fixed by Lei (Eddy) Xu
  BlockPoolSliceStorage#removeVolumes fails to remove some in-memory state associated with volumes
  

• HDFS-7718. Major bug reported by Arun Suresh and fixed by Arun Suresh
  Store KeyProvider in ClientContext to avoid leaking key provider threads when using FileContext
  

• HDFS-7714. Major bug reported by Chris Nauroth and fixed by Vinayakumar B (datanode)
  Simultaneous restart of HA NameNodes and DataNode can cause DataNode to register successfully with only one NameNode.
  

• HDFS-7710. Minor improvement reported by Xiaoyu Yao and fixed by Xiaoyu Yao
  Remove dead code in BackupImage.java
  

• HDFS-7709. Major bug reported by Rakesh R and fixed by Rakesh R
  Fix findbug warnings in httpfs
  

• HDFS-7707. Major bug reported by Yongjun Zhang and fixed by Yongjun Zhang (namenode)
  Edit log corruption due to delayed block removal again
  

• HDFS-7706. Minor improvement reported by Andrew Wang and fixed by Andrew Wang (namenode)
  Switch BlockManager logging to use slf4j
  

• HDFS-7704. Major bug reported by Rushabh S Shah and fixed by Rushabh S Shah (datanode , namenode)
  DN heartbeat to Active NN may be blocked and expire if connection to Standby NN continues to time out.
  

• HDFS-7703. Major improvement reported by Vinayakumar B and fixed by Vinayakumar B
  Support favouredNodes for the append for new blocks
  

• HDFS-7700. Major sub-task reported by Xiaoyu Yao and fixed by Xiaoyu Yao (documentation)
  Document quota support for storage types
  

• HDFS-7698. Major bug reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe
  Fix locking on HDFS read statistics and add a method for clearing them.
  

• HDFS-7697. Major bug reported by Haohui Mai and fixed by Lei (Eddy) Xu
  Mark the PB OIV tool as experimental
  

• HDFS-7696. Minor bug reported by Tsz Wo Nicholas Sze and fixed by Tsz Wo Nicholas Sze (datanode)
  FsDatasetImpl.getTmpInputStreams(..) may leak file descriptors
  

• HDFS-7694. Major improvement reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe
  FSDataInputStream should support "unbuffer"
  

• HDFS-7686. Blocker bug reported by Rushabh S Shah and fixed by Colin Patrick McCabe
  Re-add rapid rescan of possibly corrupt block feature to the block scanner
  

• HDFS-7685. Minor improvement reported by Frank Lanitz and fixed by Kai Sasaki (documentation)
  Document dfs.namenode.heartbeat.recheck-interval in hdfs-default.xml
  

• HDFS-7684. Major improvement reported by Tianyin Xu and fixed by Anu Engineer
  The host:port settings of the daemons should be trimmed before use
  

• HDFS-7683. Minor improvement reported by Vinayakumar B and fixed by Vinayakumar B (namenode)
  Combine usages and percent stats in NameNode UI
  

• HDFS-7682. Major bug reported by Charles Lamb and fixed by Charles Lamb
  {{DistributedFileSystem#getFileChecksum}} of a snapshotted file includes non-snapshotted content
  

• HDFS-7681. Major sub-task reported by Joe Pallas and fixed by Joe Pallas (datanode)
  Fix ReplicaInputStream constructor to take InputStreams
  

• HDFS-7677. Major sub-task reported by Yi Liu and fixed by Yi Liu (datanode , namenode)
  DistributedFileSystem#truncate should resolve symlinks
  

• HDFS-7676. Major sub-task reported by Konstantin Shvachko and fixed by Konstantin Shvachko (test)
  Fix TestFileTruncate to avoid bug of HDFS-7611
  

• HDFS-7675. Trivial improvement reported by Konstantin Shvachko and fixed by Colin Patrick McCabe (hdfs-client)
  Remove unused member DFSClient#spanReceiverHost
  

• HDFS-7668. Major improvement reported by Allen Wittenauer and fixed by Masatake Iwasaki (documentation)
  Convert site documentation from apt to markdown
  

• HDFS-7660. Minor bug reported by Lei (Eddy) Xu and fixed by Lei (Eddy) Xu
  BlockReceiver#close() might be called multiple times, which causes the fsvolume reference being released incorrectly.
  

• HDFS-7659. Major sub-task reported by Yi Liu and fixed by Yi Liu (datanode , namenode)
  We should check the new length of truncate can't be a negative value.
  

• HDFS-7656. Major sub-task reported by Yi Liu and fixed by Yi Liu (datanode , namenode)
  Expose truncate API for HDFS httpfs
  

• HDFS-7655. Major sub-task reported by Yi Liu and fixed by Yi Liu (datanode , namenode)
  Expose truncate API for Web HDFS
  

• HDFS-7647. Major bug reported by Milan Desai and fixed by Milan Desai
  DatanodeManager.sortLocatedBlocks sorts DatanodeInfos but not StorageIDs
  

• HDFS-7644. Trivial bug reported by Charles Lamb and fixed by Charles Lamb (documentation)
  minor typo in HttpFS doc
  

• HDFS-7643. Major sub-task reported by Arpit Agarwal and fixed by Yi Liu (test)
  Test case to ensure lazy persist files cannot be truncated
  

• HDFS-7641. Minor bug reported by Yi Liu and fixed by Yi Liu (documentation)
  Update archival storage user doc for list/set/get block storage policies
  

• HDFS-7640. Trivial improvement reported by Brandon Li and fixed by Brandon Li (nfs)
  print NFS Client in the NFS log
  

• HDFS-7638. Major sub-task reported by Yi Liu and fixed by Yi Liu (datanode , namenode)
  Small fix and few refinements for FSN#truncate
  

• HDFS-7637. Minor bug reported by Yi Liu and fixed by Yi Liu
  Fix the check condition for reserved path
  

• HDFS-7635. Minor bug reported by Chris Nauroth and fixed by Chris Nauroth (test)
  Remove TestCorruptFilesJsp from branch-2.
  

• HDFS-7634. Major sub-task reported by Yi Liu and fixed by Yi Liu (datanode , namenode)
  Disallow truncation of Lazy persist files
  

• HDFS-7632. Major bug reported by Chris Nauroth and fixed by Chris Nauroth (test)
  MiniDFSCluster configures DataNode data directories incorrectly if using more than 1 DataNode and more than 2 storage locations per DataNode.
  

• HDFS-7623. Major sub-task reported by Yi Liu and fixed by Yi Liu
  Add htrace configuration properties to core-default.xml and update user doc about how to enable htrace
  

• HDFS-7615. Major bug reported by Kihwal Lee and fixed by Kihwal Lee
  Remove longReadLock
  

• HDFS-7611. Critical bug reported by Konstantin Shvachko and fixed by Jing Zhao (namenode)
  deleteSnapshot and delete of a file can leave orphaned blocks in the blocksMap on NameNode restart.
  

• HDFS-7610. Major bug reported by Lei (Eddy) Xu and fixed by Lei (Eddy) Xu (datanode)
  Fix removal of dynamically added DN volumes
  

• HDFS-7606. Minor bug reported by Ted Yu and fixed by Byron Wong
  Missing null check in INodeFile#getBlocks()
  

• HDFS-7604. Major improvement reported by Chris Nauroth and fixed by Chris Nauroth (datanode , namenode)
  Track and display failed DataNode storage locations in NameNode.
  

• HDFS-7603. Critical bug reported by Kihwal Lee and fixed by Kihwal Lee (rolling upgrades)
  The background replication queue initialization may not let others run
  

• HDFS-7600. Major improvement reported by Yi Liu and fixed by Jing Zhao (tools)
  Refine hdfs admin classes to reuse common code
  

• HDFS-7598. Minor improvement reported by Sangjin Lee and fixed by Sangjin Lee (test)
  Remove dependency on old version of Guava in TestDFSClientCache#testEviction
  

• HDFS-7596. Major bug reported by Arpit Agarwal and fixed by Arpit Agarwal (namenode)
  NameNode should prune dead storages from storageMap
  

• HDFS-7591. Minor improvement reported by Chris Nauroth and fixed by Varun Saxena (scripts)
  hdfs classpath command should support same options as hadoop classpath.
  

• HDFS-7589. Major sub-task reported by Zhanwei Wang and fixed by Zhanwei Wang (libhdfs)
  Break the dependency between libnative_mini_dfs and libhdfs
  

• HDFS-7587. Blocker bug reported by Kihwal Lee and fixed by Jing Zhao (namenode)
  Edit log corruption can happen if append fails with a quota violation
  

• HDFS-7585. Major test reported by sam liu and fixed by sam liu (test)
  Get TestEnhancedByteBufferAccess working on CPU architectures with page sizes other than 4096
  

• HDFS-7584. Major new feature reported by Xiaoyu Yao and fixed by Xiaoyu Yao (datanode , namenode)
  Enable Quota Support for Storage Types
  

  1. Introduced quota by storage type as a hard limit on the amount of space usage allowed for different storage types (SSD, DISK, ARCHIVE) under the target directory. 2. Added {{SetQuotaByStorageType}} API and {{-storagetype}} option for {{hdfs dfsadmin -setSpaceQuota/-clrSpaceQuota}} commands to allow set/clear quota by storage type under the target directory.

• HDFS-7583. Minor bug reported by Vinayakumar B and fixed by Vinayakumar B (namenode)
  Fix findbug in TransferFsImage.java
  

• HDFS-7579. Minor improvement reported by Charles Lamb and fixed by Charles Lamb (datanode)
  Improve log reporting during block report rpc failure
  

• HDFS-7575. Critical bug reported by Lars Francke and fixed by Arpit Agarwal
  Upgrade should generate a unique storage ID for each volume
  

• HDFS-7573. Major sub-task reported by Haohui Mai and fixed by Haohui Mai
  Consolidate the implementation of delete() into a single class
  

• HDFS-7572. Major bug reported by Arpit Agarwal and fixed by Arpit Agarwal (test)
  TestLazyPersistFiles#testDnRestartWithSavedReplicas is flaky on Windows
  

• HDFS-7566. Major bug reported by Ray Chiang and fixed by Ray Chiang
  Remove obsolete entries from hdfs-default.xml
  

• HDFS-7564. Minor improvement reported by Hari Sekhon and fixed by Yongjun Zhang (nfs)
  NFS gateway dynamically reload UID/GID mapping file /etc/nfs.map
  

• HDFS-7563. Major bug reported by Hari Sekhon and fixed by Yongjun Zhang (nfs)
  NFS gateway parseStaticMap NumberFormatException
  

• HDFS-7561. Major bug reported by Konstantin Shvachko and fixed by Liang Xie
  TestFetchImage should write fetched-image-dir under target.
  

• HDFS-7560. Critical bug reported by Vinayakumar B and fixed by Vinayakumar B (namenode)
  ACLs removed by removeDefaultAcl() will be back after NameNode restart/failover
  

• HDFS-7557. Minor improvement reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe
  Fix spacing for a few keys in DFSConfigKeys.java
  

• HDFS-7555. Major improvement reported by Haohui Mai and fixed by Haohui Mai
  Remove the support of unmanaged connectors in HttpServer2
  

• HDFS-7552. Major bug reported by Liang Xie and fixed by Liang Xie (datanode , test)
  change FsVolumeList toString() to fix TestDataNodeVolumeFailureToleration
  

• HDFS-7548. Major bug reported by Rushabh S Shah and fixed by Rushabh S Shah
  Corrupt block reporting delayed until datablock scanner thread detects it
  

• HDFS-7543. Major sub-task reported by Haohui Mai and fixed by Haohui Mai
  Avoid path resolution when getting FileStatus for audit logs
  

• HDFS-7537. Major improvement reported by Allen Wittenauer and fixed by GAO Rui (namenode)
  fsck is confusing when dfs.namenode.replication.min > 1 && missing replicas && NN restart
  

• HDFS-7536. Minor bug reported by Yi Liu and fixed by Yi Liu (security)
  Remove unused CryptoCodec in org.apache.hadoop.fs.Hdfs
  

• HDFS-7535. Major improvement reported by Jing Zhao and fixed by Jing Zhao (distcp , snapshots)
  Utilize Snapshot diff report for distcp
  

• HDFS-7533. Major bug reported by Kihwal Lee and fixed by Eric Payne
  Datanode sometimes does not shutdown on receiving upgrade shutdown command
  

• HDFS-7531. Major improvement reported by Lei (Eddy) Xu and fixed by Lei (Eddy) Xu (datanode)
  Improve the concurrent access on FsVolumeList
  

• HDFS-7530. Minor bug reported by Charles Lamb and fixed by Charles Lamb (namenode)
  Allow renaming of encryption zone roots
  

• HDFS-7528. Major sub-task reported by Haohui Mai and fixed by Haohui Mai
  Consolidate symlink-related implementation into a single class
  

• HDFS-7517. Major bug reported by Haohui Mai and fixed by Haohui Mai
  Remove redundant non-null checks in FSNamesystem#getBlockLocations
  

• HDFS-7516. Major bug reported by Brandon Li and fixed by Brandon Li (nfs)
  Fix findbugs warnings in hadoop-nfs project
  

• HDFS-7515. Major bug reported by Haohui Mai and fixed by Haohui Mai
  Fix new findbugs warnings in hadoop-hdfs
  

• HDFS-7514. Major bug reported by Arpit Agarwal and fixed by Arpit Agarwal (test)
  TestTextCommand fails on Windows
  

• HDFS-7513. Major improvement reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (namenode)
  HDFS inotify: add defaultBlockSize to CreateEvent
  

• HDFS-7509. Major sub-task reported by Jing Zhao and fixed by Jing Zhao
  Avoid resolving path multiple times
  

• HDFS-7506. Major sub-task reported by Haohui Mai and fixed by Haohui Mai
  Consolidate implementation of setting inode attributes into a single class
  

• HDFS-7502. Major bug reported by Brandon Li and fixed by Brandon Li (nfs)
  Fix findbugs warning in hdfs-nfs project
  

• HDFS-7498. Major sub-task reported by Jing Zhao and fixed by Jing Zhao (namenode)
  Simplify the logic in INodesInPath
  

• HDFS-7497. Major bug reported by Yongjun Zhang and fixed by Yongjun Zhang (datanode , namenode)
  Inconsistent report of decommissioning DataNodes between dfsadmin and NameNode webui
  

• HDFS-7496. Major bug reported by Colin Patrick McCabe and fixed by Lei (Eddy) Xu
  Fix FsVolume removal race conditions on the DataNode by reference-counting the volume instances
  

• HDFS-7495. Minor bug reported by Ted Yu and fixed by Colin Patrick McCabe
  Remove updatePosition argument from DFSInputStream#getBlockAt()
  

• HDFS-7494. Minor bug reported by Ted Yu and fixed by Ted Yu
  Checking of closed in DFSInputStream#pread() should be protected by synchronization
  

• HDFS-7491. Minor improvement reported by Ming Ma and fixed by Ming Ma (datanode)
  Add incremental blockreport latency to DN metrics
  

• HDFS-7490. Major bug reported by Steve Loughran and fixed by Steve Loughran (build , test)
  HDFS tests OOM on Java7+
  

• HDFS-7486. Major sub-task reported by Haohui Mai and fixed by Haohui Mai
  Consolidate XAttr-related implementation into a single class
  

• HDFS-7484. Major improvement reported by Haohui Mai and fixed by Jing Zhao
  Make FSDirectory#addINode take existing INodes as its parameter
  

• HDFS-7481. Minor bug reported by Vinayakumar B and fixed by Vinayakumar B (namenode)
  Add ACL indicator to the "Permission Denied" exception.
  

• HDFS-7478. Major improvement reported by Li Lu and fixed by Li Lu
  Move org.apache.hadoop.hdfs.server.namenode.NNConf to FSNamesystem
  

• HDFS-7476. Major sub-task reported by Haohui Mai and fixed by Haohui Mai (namenode)
  Consolidate ACL-related operations to a single class
  

• HDFS-7475. Major test reported by Xiaoyu Yao and fixed by Xiaoyu Yao (test)
  Make TestLazyPersistFiles#testLazyPersistBlocksAreSaved deterministic
  

• HDFS-7474. Major sub-task reported by Jing Zhao and fixed by Jing Zhao (namenode)
  Avoid resolving path in FSPermissionChecker
  

• HDFS-7473. Major bug reported by Jason Keller and fixed by Akira AJISAKA (documentation)
  Document setting dfs.namenode.fs-limits.max-directory-items to 0 is invalid
  

• HDFS-7472. Trivial bug reported by Masatake Iwasaki and fixed by Masatake Iwasaki
  Fix typo in message of ReplicaNotFoundException
  

• HDFS-7470. Major bug reported by zhaoyunjiong and fixed by zhaoyunjiong (namenode)
  SecondaryNameNode need twice memory when calling reloadFromImageFile
  

• HDFS-7468. Major sub-task reported by Li Lu and fixed by Li Lu
  Moving verify* functions to corresponding classes
  

• HDFS-7467. Major sub-task reported by Benoy Antony and fixed by Benoy Antony (balancer & mover)
  Provide storage tier information for a directory via fsck
  

• HDFS-7463. Major improvement reported by Haohui Mai and fixed by Haohui Mai
  Simplify FSNamesystem#getBlockLocationsUpdateTimes
  

• HDFS-7462. Major sub-task reported by Haohui Mai and fixed by Haohui Mai
  Consolidate implementation of mkdirs() into a single class
  

• HDFS-7459. Major sub-task reported by Haohui Mai and fixed by Haohui Mai
  Consolidate cache-related implementation in FSNamesystem into a single class
  

• HDFS-7458. Minor improvement reported by Yongjun Zhang and fixed by Yongjun Zhang (nfs , test)
  Add description to the nfs ports in core-site.xml used by nfs test to avoid confusion
  

• HDFS-7457. Major bug reported by Daryn Sharp and fixed by Daryn Sharp (namenode)
  DatanodeID generates excessive garbage
  

  Thanks for the reviews, gentlemen. I've committed this to trunk and branch-2. Thanks for identifying and working on the issue, Daryn.

• HDFS-7456. Major bug reported by Vinayakumar B and fixed by Vinayakumar B (namenode)
  De-duplicate AclFeature instances with same AclEntries do reduce memory footprint of NameNode
  

• HDFS-7454. Major improvement reported by Vinayakumar B and fixed by Vinayakumar B (namenode)
  Reduce memory footprint for AclEntries in NameNode
  

• HDFS-7450. Major sub-task reported by Haohui Mai and fixed by Haohui Mai
  Consolidate the implementation of GetFileInfo(), GetListings() and GetContentSummary() into a single class
  

• HDFS-7449. Major new feature reported by Brandon Li and fixed by Brandon Li (nfs)
  Add metrics to NFS gateway
  

• HDFS-7448. Minor test reported by Ted Yu and fixed by Akira AJISAKA
  TestBookKeeperHACheckpoints fails in trunk build
  

• HDFS-7446. Major improvement reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (hdfs-client)
  HDFS inotify should have the ability to determine what txid it has read up to
  

• HDFS-7444. Major bug reported by Haohui Mai and fixed by Haohui Mai
  convertToBlockUnderConstruction should preserve BlockCollection
  

• HDFS-7440. Major sub-task reported by Haohui Mai and fixed by Haohui Mai
  Consolidate snapshot related operations in a single class
  

• HDFS-7439. Minor improvement reported by Ming Ma and fixed by Takanobu Asanuma (balancer & mover , datanode , hdfs-client)
  Add BlockOpResponseProto's message to DFSClient's exception message
  

• HDFS-7438. Major sub-task reported by Haohui Mai and fixed by Haohui Mai
  Consolidate the implementation of rename() into a single class
  

• HDFS-7435. Critical improvement reported by Daryn Sharp and fixed by Daryn Sharp (datanode , namenode)
  PB encoding of block reports is very inefficient
  

• HDFS-7434. Major improvement reported by Daryn Sharp and fixed by Daryn Sharp (namenode)
  DatanodeID hashCode should not be mutable
  

• HDFS-7431. Major bug reported by Yi Liu and fixed by Yi Liu (security)
  log message for InvalidMagicNumberException may be incorrect
  

• HDFS-7430. Major improvement reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe
  Rewrite the BlockScanner to use O(1) memory and use multiple threads
  

• HDFS-7426. Major improvement reported by Andrew Wang and fixed by Andrew Wang (namenode)
  Change nntop JMX format to be a JSON blob
  

• HDFS-7424. Major new feature reported by Brandon Li and fixed by Brandon Li (nfs)
  Add web UI for NFS gateway
  

• HDFS-7423. Trivial bug reported by Charles Lamb and fixed by Charles Lamb (nfs)
  various typos and message formatting fixes in nfs daemon and doc
  

• HDFS-7420. Major sub-task reported by Haohui Mai and fixed by Haohui Mai
  Delegate permission checks to FSDirectory
  

• HDFS-7419. Major improvement reported by Lei (Eddy) Xu and fixed by Lei (Eddy) Xu (datanode)
  Improve error messages for DataNode hot swap drive feature
  

• HDFS-7415. Major sub-task reported by Haohui Mai and fixed by Haohui Mai
  Move FSNameSystem.resolvePath() to FSDirectory
  

• HDFS-7413. Major sub-task reported by Haohui Mai and fixed by Haohui Mai (test)
  Some unit tests should use NameNodeProtocols instead of FSNameSystem
  

• HDFS-7412. Major sub-task reported by Haohui Mai and fixed by Haohui Mai
  Move RetryCache to NameNodeRpcServer
  

• HDFS-7411. Major improvement reported by Andrew Wang and fixed by Andrew Wang
  Refactor and improve decommissioning logic into DecommissionManager
  

  This change introduces a new configuration key used to throttle decommissioning work, "dfs.namenode.decommission.blocks.per.interval". This new key overrides and deprecates the previous related configuration key "dfs.namenode.decommission.nodes.per.interval". The new key is intended to result in more predictable pause times while scanning decommissioning nodes.

• HDFS-7410. Major improvement reported by Vinayakumar B and fixed by Vinayakumar B (hdfs-client)
  Support CreateFlags with append() to support hsync() for appending streams
  

• HDFS-7409. Minor improvement reported by Andrew Wang and fixed by Andrew Wang
  Allow dead nodes to finish decommissioning if all files are fully replicated
  

• HDFS-7406. Major bug reported by Haohui Mai and fixed by Haohui Mai
  SimpleHttpProxyHandler puts incorrect "Connection: Close" header
  

• HDFS-7404. Major improvement reported by Haohui Mai and fixed by Li Lu
  Remove o.a.h.hdfs.server.datanode.web.resources
  

• HDFS-7403. Trivial bug reported by Yongjun Zhang and fixed by Yongjun Zhang (namenode)
  Inaccurate javadoc of BlockUCState#COMPLETE state
  

• HDFS-7399. Minor bug reported by Ted Yu and fixed by Vinayakumar B
  Lack of synchronization in DFSOutputStream#Packet#getLastByteOffsetBlock()
  

• HDFS-7398. Major improvement reported by Gera Shegalov and fixed by Gera Shegalov (namenode)
  Reset cached thread-local FSEditLogOp's on every FSEditLog#logEdit
  

• HDFS-7395. Major bug reported by Yongjun Zhang and fixed by Haohui Mai (namenode)
  BlockIdManager#clear() bails out when resetting the GenerationStampV1Limit
  

• HDFS-7394. Minor bug reported by Kihwal Lee and fixed by Keith Pak
  Log at INFO level, not WARN level, when InvalidToken is seen in ShortCircuitCache
  

• HDFS-7389. Major bug reported by Chunjun Xiao and fixed by Vinayakumar B (namenode)
  Named user ACL cannot stop the user from accessing the FS entity.
  

• HDFS-7386. Trivial improvement reported by Yongjun Zhang and fixed by Yongjun Zhang (datanode , security)
  Replace check "port number < 1024" with shared isPrivilegedPort method
  

• HDFS-7384. Major improvement reported by Vinayakumar B and fixed by Vinayakumar B (namenode)
  'getfacl' command and 'getAclStatus' output should be in sync
  

• HDFS-7381. Major improvement reported by Haohui Mai and fixed by Haohui Mai
  Decouple the management of block id and gen stamps from FSNamesystem
  

• HDFS-7375. Major improvement reported by Haohui Mai and fixed by Haohui Mai
  Move FSClusterStats to o.a.h.h.hdfs.server.blockmanagement
  

• HDFS-7374. Major bug reported by Zhe Zhang and fixed by Zhe Zhang
  Allow decommissioning of dead DataNodes
  

• HDFS-7373. Major bug reported by Kihwal Lee and fixed by Kihwal Lee
  Clean up temporary files after fsimage transfer failures
  

• HDFS-7366. Minor bug reported by Haohui Mai and fixed by Li Lu
  BlockInfo should take replication as an short in the constructor
  

• HDFS-7365. Minor improvement reported by Li Lu and fixed by Li Lu
  Remove hdfs.server.blockmanagement.MutableBlockCollection
  

• HDFS-7361. Minor bug reported by Chris Nauroth and fixed by Konstantin Shvachko (datanode , namenode , test)
  TestCheckpoint#testStorageAlreadyLockedErrorMessage fails after change of log message related to locking violation.
  

• HDFS-7358. Major bug reported by Tsz Wo Nicholas Sze and fixed by Tsz Wo Nicholas Sze (hdfs-client)
  Clients may get stuck waiting when using ByteArrayManager
  

• HDFS-7357. Minor improvement reported by Tsz Wo Nicholas Sze and fixed by Tsz Wo Nicholas Sze (namenode)
  FSNamesystem.checkFileProgress should log file path
  

• HDFS-7356. Minor improvement reported by Haohui Mai and fixed by Li Lu (nfs)
  Use DirectoryListing.hasMore() directly in nfs
  

• HDFS-7336. Major improvement reported by Konstantin Shvachko and fixed by Milan Desai (hdfs-client)
  Unused member DFSInputStream.buffersize
  

• HDFS-7335. Major improvement reported by Konstantin Shvachko and fixed by Milan Desai (namenode)
  Redundant checkOperation() in FSN.analyzeFileState()
  

• HDFS-7333. Major improvement reported by Konstantin Shvachko and fixed by Konstantin Shvachko (datanode , namenode)
  Improve log message in Storage.tryLock()
  

• HDFS-7331. Minor improvement reported by Charles Lamb and fixed by Charles Lamb (datanode)
  Add Datanode network counts to datanode jmx page
  

• HDFS-7329. Major improvement reported by Konstantin Shvachko and fixed by Byron Wong (test)
  MiniDFSCluster should log the exception when createNameNodesAndSetConf() fails.
  

• HDFS-7326. Minor improvement reported by Colin Patrick McCabe and fixed by Vijay Bhat (documentation)
  Add documentation for hdfs debug commands
  

  Added documentation for the hdfs debug commands to the following URL in the documentation website. hadoop-project-dist/hadoop-hdfs/HDFSCommands.html In order to view the new documentation, build the website in a staging area: $ mvn clean site; mvn site:stage -DstagingDirectory=/tmp/hadoop-site Point your browser to file:///tmp/hadoop-site/hadoop-project/hadoop-project-dist/hadoop-hdfs/HDFSCommands.html

• HDFS-7323. Major improvement reported by Tsz Wo Nicholas Sze and fixed by Jing Zhao (hdfs-client)
  Move the get/setStoragePolicy commands out from dfsadmin
  

• HDFS-7315. Trivial bug reported by Plamen Jeliazkov and fixed by Plamen Jeliazkov
  DFSTestUtil.readFileBuffer opens extra FSDataInputStream
  

• HDFS-7310. Major improvement reported by Uma Maheswara Rao G and fixed by Vinayakumar B (balancer & mover)
  Mover can give first priority to local DN if it has target storage type available in local DN
  

• HDFS-7308. Minor improvement reported by Tsz Wo Nicholas Sze and fixed by Takuya Fukudome (hdfs-client)
  DFSClient write packet size may > 64kB
  

• HDFS-7303. Minor bug reported by Benoy Antony and fixed by Benoy Antony
  NN UI fails to distinguish datanodes on the same host
  

• HDFS-7301. Minor bug reported by Zhe Zhang and fixed by Zhe Zhang
  TestMissingBlocksAlert should use MXBeans instead of old web UI
  

• HDFS-7283. Trivial improvement reported by Stephen Chu and fixed by Stephen Chu (datanode)
  Bump DataNode OOM log from WARN to ERROR
  

• HDFS-7282. Major bug reported by Jinghui Wang and fixed by Jinghui Wang (test)
  Fix intermittent TestShortCircuitCache and TestBlockReaderFactory failures resulting from TemporarySocketDirectory GC
  

• HDFS-7280. Major improvement reported by Haohui Mai and fixed by Haohui Mai
  Use netty 4 in WebImageViewer
  

• HDFS-7279. Major improvement reported by Haohui Mai and fixed by Haohui Mai (datanode , webhdfs)
  Use netty to implement DatanodeWebHdfsMethods
  

• HDFS-7278. Major improvement reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (datanode)
  Add a command that allows sysadmins to manually trigger full block reports from a DN
  

• HDFS-7277. Minor bug reported by Haohui Mai and fixed by Haohui Mai (build)
  Remove explicit dependency on netty 3.2 in BKJournal
  

• HDFS-7270. Major improvement reported by Haohui Mai and fixed by Haohui Mai (datanode)
  Add congestion signaling capability to DataNode write protocol
  

• HDFS-7266. Minor improvement reported by Gopal V and fixed by Andrew Wang (hdfs-client)
  HDFS Peercache enabled check should not lock on object
  

• HDFS-7263. Major bug reported by Konstantin Shvachko and fixed by Tao Luo (hdfs-client)
  Snapshot read can reveal future bytes for appended files.
  

• HDFS-7258. Minor bug reported by Xiaoyu Yao and fixed by Xiaoyu Yao (namenode)
  CacheReplicationMonitor rescan schedule log should use DEBUG level instead of INFO level
  

• HDFS-7257. Minor improvement reported by Charles Lamb and fixed by Charles Lamb (namenode)
  Add the time of last HA state transition to NN's /jmx page
  

• HDFS-7254. Major sub-task reported by Lei (Eddy) Xu and fixed by Lei (Eddy) Xu (datanode)
  Add documentation for hot swaping DataNode drives
  

• HDFS-7252. Trivial improvement reported by Yi Liu and fixed by Yi Liu
  small refinement to the use of isInAnEZ in FSNamesystem
  

• HDFS-7242. Minor improvement reported by Yi Liu and fixed by Yi Liu (namenode)
  Code improvement for FSN#checkUnreadableBySuperuser
  

• HDFS-7235. Major bug reported by Yongjun Zhang and fixed by Yongjun Zhang (datanode , namenode)
  DataNode#transferBlock should report blocks that don't exist using reportBadBlock
  

• HDFS-7232. Trivial bug reported by Zoran Dimitrijevic and fixed by Zoran Dimitrijevic
  Populate hostname in httpfs audit log
  

• HDFS-7227. Minor bug reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe
  Fix findbugs warning about NP_DEREFERENCE_OF_READLINE_VALUE in SpanReceiverHost
  

• HDFS-7225. Major bug reported by Zhe Zhang and fixed by Zhe Zhang (namenode)
  Remove stale block invalidation work when DN re-registers with different UUID
  

• HDFS-7224. Major bug reported by Eric Payne and fixed by Eric Payne (webhdfs)
  Allow reuse of NN connections via webhdfs
  

• HDFS-7223. Minor improvement reported by Masatake Iwasaki and fixed by Masatake Iwasaki
  Tracing span description of IPC client is too long
  

• HDFS-7222. Minor new feature reported by Charles Lamb and fixed by Charles Lamb (datanode)
  Expose DataNode network errors as a metric
  

• HDFS-7213. Critical bug reported by Daryn Sharp and fixed by Eric Payne (namenode)
  processIncrementalBlockReport performance degradation
  

• HDFS-7210. Major improvement reported by Vinayakumar B and fixed by Vinayakumar B (hdfs-client , namenode)
  Avoid two separate RPC's namenode.append() and namenode.getFileInfo() for an append call from DFSClient
  

• HDFS-7209. Major sub-task reported by Yi Liu and fixed by Yi Liu (encryption , performance)
  Populate EDEK cache when creating encryption zone
  

• HDFS-7202. Minor bug reported by Masatake Iwasaki and fixed by Masatake Iwasaki
  Should be able to omit package name of SpanReceiver on "hadoop trace -add"
  

• HDFS-7201. Major bug reported by Konstantin Shvachko and fixed by Dawson Choong
  Fix typos in hdfs-default.xml
  

• HDFS-7198. Trivial bug reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe
  Fix findbugs "unchecked conversion" warning in DFSClient#getPathTraceScope
  

• HDFS-7194. Trivial bug reported by Yongjun Zhang and fixed by Yongjun Zhang
  Fix findbugs "inefficient new String constructor" warning in DFSClient#PATH
  

• HDFS-7190. Major improvement reported by Konstantin Shvachko and fixed by Dawson Choong (namenode)
  Bad use of Preconditions in startFileInternal()
  

• HDFS-7189. Major sub-task reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (datanode , namenode)
  Add trace spans for DFSClient metadata operations
  

• HDFS-7186. Minor improvement reported by Masatake Iwasaki and fixed by Masatake Iwasaki (documentation)
  Document the "hadoop trace" command.
  

• HDFS-7182. Major improvement reported by Ming Ma and fixed by Ming Ma
  JMX metrics aren't accessible when NN is busy
  

• HDFS-7165. Major improvement reported by Andrew Wang and fixed by Zhe Zhang (namenode)
  Separate block metrics for files with replication count 1
  

• HDFS-7146. Major bug reported by Yongjun Zhang and fixed by Yongjun Zhang (nfs)
  NFS ID/Group lookup requires SSSD enumeration on the server
  

• HDFS-7097. Critical bug reported by Kihwal Lee and fixed by Kihwal Lee
  Allow block reports to be processed during checkpointing on standby name node
  

• HDFS-7058. Major sub-task reported by Konstantin Shvachko and fixed by Dasha Boudnik (test)
  Tests for truncate CLI
  

• HDFS-7056. Major sub-task reported by Konstantin Shvachko and fixed by Plamen Jeliazkov (namenode)
  Snapshot support for truncate
  

• HDFS-7055. Major sub-task reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (datanode , namenode)
  Add tracing to DFSInputStream
  

• HDFS-7054. Major sub-task reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (datanode , namenode)
  Make DFSOutputStream tracing more fine-grained
  

• HDFS-7035. Major sub-task reported by Lei (Eddy) Xu and fixed by Lei (Eddy) Xu (datanode)
  Make adding a new data directory to the DataNode an atomic operation and improve error handling
  

• HDFS-7026. Trivial improvement reported by Yongjun Zhang and fixed by Yongjun Zhang
  Introduce a string constant for "Failed to obtain user group info..."
  

• HDFS-7009. Major bug reported by Ming Ma and fixed by Ming Ma (datanode)
  Active NN and standby NN have different live nodes
  

• HDFS-7008. Minor bug reported by Ted Yu and fixed by Tsuyoshi Ozawa
  xlator should be closed upon exit from DFSAdmin#genericRefresh()
  

• HDFS-6982. Major new feature reported by Maysam Yabandeh and fixed by Maysam Yabandeh
  nntop: top­-like tool for name node users
  

• HDFS-6938. Trivial bug reported by Charles Lamb and fixed by Charles Lamb (namenode)
  Cleanup javac warnings in FSNamesystem
  

• HDFS-6917. Major bug reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (hdfs-client)
  Add an hdfs debug command to validate blocks, call recoverlease, etc.
  

• HDFS-6877. Major sub-task reported by Lei (Eddy) Xu and fixed by Lei (Eddy) Xu (datanode)
  Avoid calling checkDisk when an HDFS volume is removed during a write.
  

• HDFS-6841. Major bug reported by Vinayakumar B and fixed by Vinayakumar B
  Use Time.monotonicNow() wherever applicable instead of Time.now()
  

• HDFS-6833. Critical bug reported by Shinichi Yamashita and fixed by Shinichi Yamashita (datanode)
  DirectoryScanner should not register a deleting block with memory of DataNode
  

• HDFS-6826. Major new feature reported by Alejandro Abdelnur and fixed by Arun Suresh (security)
  Plugin interface to enable delegation of HDFS authorization assertions
  

• HDFS-6824. Minor sub-task reported by Andrew Wang and fixed by Andrew Wang (documentation)
  Additional user documentation for HDFS encryption.
  

• HDFS-6806. Minor improvement reported by Akira AJISAKA and fixed by J.Andreina (documentation)
  HDFS Rolling upgrade document should mention the versions available
  

• HDFS-6803. Major sub-task reported by stack and fixed by stack (hdfs-client)
  Documenting DFSClient#DFSInputStream expectations reading and preading in concurrent context
  

• HDFS-6753. Major bug reported by J.Andreina and fixed by J.Andreina
  Initialize checkDisk when DirectoryScanner not able to get files list for scanning
  

• HDFS-6741. Trivial improvement reported by Stephen Chu and fixed by Harsh J
  Improve permission denied message when FSPermissionChecker#checkOwner fails
  

• HDFS-6735. Major improvement reported by Liang Xie and fixed by Lars Hofhansl (hdfs-client)
  A minor optimization to avoid pread() be blocked by read() inside the same DFSInputStream
  

• HDFS-6673. Minor sub-task reported by Lei (Eddy) Xu and fixed by Lei (Eddy) Xu
  Add delimited format support to PB OIV tool
  

• HDFS-6663. Major new feature reported by Kihwal Lee and fixed by Chen He
  Admin command to track file and locations from block id
  

• HDFS-6662. Critical bug reported by Brahma Reddy Battula and fixed by Gerson Carlos (namenode)
  WebHDFS cannot open a file if its path contains "%"
  

• HDFS-6657. Minor bug reported by Vinayakumar B and fixed by Vinayakumar B
  Remove link to 'Legacy UI' in Namenode UI
  

• HDFS-6651. Critical bug reported by Kihwal Lee and fixed by Jing Zhao
  Deletion failure can leak inodes permanently
  

• HDFS-6565. Major improvement reported by Haohui Mai and fixed by Akira AJISAKA
  Use jackson instead jetty json in hdfs-client
  

• HDFS-6538. Trivial bug reported by debugging and fixed by David Luo (datanode)
  Comment format error in ShortCircuitRegistry javadoc
  

• HDFS-6488. Major new feature reported by Stephen Chu and fixed by Brandon Li (nfs)
  Support HDFS superuser in NFSv3 gateway
  

• HDFS-6425. Major bug reported by Ming Ma and fixed by Ming Ma
  Large postponedMisreplicatedBlocks has impact on blockReport latency
  

• HDFS-6252. Minor improvement reported by Fengdong Yu and fixed by Haohui Mai (namenode)
  Phase out the old web UI in HDFS
  

• HDFS-6133. Major improvement reported by zhaoyunjiong and fixed by zhaoyunjiong (balancer & mover , datanode)
  Make Balancer support exclude specified path
  

  Add a feature for replica pinning so that when a replica is pinned in a datanode, it will not be moved by Balancer/Mover. The replica pinning feature can be enabled/disabled by "dfs.datanode.block-pinning.enabled", where the default is false.

• HDFS-5928. Major sub-task reported by Siqi Li and fixed by Siqi Li
  show namespace and namenode ID on NN dfshealth page
  

• HDFS-5853. Minor improvement reported by Akira AJISAKA and fixed by Akira AJISAKA (documentation , namenode)
  Add "hadoop.user.group.metrics.percentiles.intervals" to hdfs-default.xml
  

• HDFS-5782. Minor sub-task reported by David Powell and fixed by Joe Pallas (datanode)
  BlockListAsLongs should take lists of Replicas rather than concrete classes
  

• HDFS-5631. Minor sub-task reported by David Powell and fixed by Joe Pallas (datanode)
  Expose interfaces required by FsDatasetSpi implementations
  

• HDFS-5578. Minor bug reported by Andrew Purtell and fixed by Andrew Purtell
  [JDK8] Fix Javadoc errors caused by incorrect or illegal tags in doc comments
  

• HDFS-5445. Minor bug reported by Jonathan Mace and fixed by Jonathan Mace (datanode)
  PacketReceiver populates the packetLen field in PacketHeader incorrectly
  

• HDFS-4266. Major sub-task reported by Ivan Kelly and fixed by Rakesh R (ha)
  BKJM: Separate write and ack quorum
  

• HDFS-4265. Major sub-task reported by Ivan Kelly and fixed by Rakesh R (ha)
  BKJM doesn't take advantage of speculative reads
  

• HDFS-3689. Major new feature reported by Suresh Srinivas and fixed by Jing Zhao (datanode , hdfs-client , namenode)
  Add support for variable length block
  

  1. HDFS now can choose to append data to a new block instead of end of the last partial block. Users can pass {{CreateFlag.APPEND}} and {{CreateFlag.NEW_BLOCK}} to the {{append}} API to indicate this requirement. 2. HDFS now allows users to pass {{SyncFlag.END_BLOCK}} to the {{hsync}} API to finish the current block and write remaining data to a new block.

• HDFS-3519. Critical bug reported by Todd Lipcon and fixed by Ming Ma (namenode)
  Checkpoint upload may interfere with a concurrent saveNamespace
  

• HDFS-3342. Minor improvement reported by Todd Lipcon and fixed by Yongjun Zhang (datanode)
  SocketTimeoutException in BlockSender.sendChunks could have a better error message
  

• HDFS-3107. Major new feature reported by Lei Chang and fixed by Plamen Jeliazkov (datanode , namenode)
  HDFS truncate
  

• HDFS-2605. Major bug reported by Konstantin Shvachko and fixed by Allen Wittenauer (documentation)
  CHANGES.txt has two "Release 0.21.1" sections
  

• HDFS-2486. Minor task reported by Steve Loughran and fixed by Uma Maheswara Rao G (namenode)
  Review issues with UnderReplicatedBlocks
  

• HDFS-2219. Minor improvement reported by Jitendra Nath Pandey and fixed by Tsz Wo Nicholas Sze (tools)
  Fsck should work with fully qualified file paths.
  

• HDFS-1522. Major bug reported by Konstantin Shvachko and fixed by Dongming Liang (datanode)
  Merge Block.BLOCK_FILE_PREFIX and DataStorage.BLOCK_FILE_PREFIX into one constant
  

  This merges Block.BLOCK_FILE_PREFIX and DataStorage.BLOCK_FILE_PREFIX into one constant. Hard-coded literals of "blk_" in various files are also updated to use the same constant.

• HDFS-1362. Major new feature reported by Wang Xu and fixed by Wang Xu (datanode)
  Provide volume management functionality for DataNode
  

  Based on the reconfiguration framework provided by HADOOP-7001, enable reconfigure the dfs.datanode.data.dir and add new volumes into service.

• HDFS-316. Minor improvement reported by Brian Bockelman and fixed by Xiaoyu Yao (balancer & mover)
  Balancer should run for a configurable # of iterations
  

• HDFS-49. Minor bug reported by Steve Loughran and fixed by Steve Loughran (test)
  MiniDFSCluster.stopDataNode will always shut down a node in the cluster if a matching name is not found
  

• HADOOP-11815. Blocker bug reported by Rohith and fixed by Rohith
  HttpServer2 should destroy SignerSecretProvider when it stops
  

• HADOOP-11801. Minor improvement reported by Gabor Liptak and fixed by Gabor Liptak (documentation)
  Update BUILDING.txt for Ubuntu
  

  ProtocolBuffer is packaged in Ubuntu

• HADOOP-11796. Minor sub-task reported by Xiaoyu Yao and fixed by Xiaoyu Yao (test)
  Skip TestShellBasedIdMapping.testStaticMapUpdate on Windows
  

• HADOOP-11787. Critical bug reported by Colin Patrick McCabe and fixed by Kiran Kumar M R (native)
  OpensslSecureRandom.c pthread_threadid_np usage signature is wrong on 32-bit Mac
  

• HADOOP-11776. Blocker bug reported by Li Lu and fixed by Li Lu
  jdiff is broken in Hadoop 2
  

• HADOOP-11761. Minor bug reported by Li Lu and fixed by Li Lu
  Fix findbugs warnings in org.apache.hadoop.security.authentication
  

• HADOOP-11757. Major bug reported by Brandon Li and fixed by Brandon Li (nfs)
  NFS gateway should shutdown when it can't start UDP or TCP server
  

• HADOOP-11754. Blocker bug reported by Sangjin Lee and fixed by Haohui Mai
  RM fails to start in non-secure mode due to authentication filter failure
  

• HADOOP-11748. Critical bug reported by Haohui Mai and fixed by Li Lu
  The secrets of auth cookies should not be specified in configuration in clear text
  

• HADOOP-11738. Major bug reported by Tsuyoshi Ozawa and fixed by Tsuyoshi Ozawa
  Fix a link of Protocol Buffers 2.5 for download in BUILDING.txt
  

• HADOOP-11729. Minor bug reported by Masatake Iwasaki and fixed by Masatake Iwasaki (documentation)
  Fix link to cgroups doc in site.xml
  

  Committed this to trunk, branch-2, and branch-2.7. Thanks Masatake for your contribution!

• HADOOP-11722. Major bug reported by Arun Suresh and fixed by Arun Suresh
  Some Instances of Services using ZKDelegationTokenSecretManager go down when old token cannot be deleted
  

• HADOOP-11720. Major bug reported by Akira AJISAKA and fixed by Akira AJISAKA (documentation)
  [JDK8] Fix javadoc errors caused by incorrect or illegal tags in hadoop-tools
  

• HADOOP-11714. Minor improvement reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (tracing)
  Add more trace log4j messages to SpanReceiverHost
  

• HADOOP-11710. Critical sub-task reported by Sean Busbey and fixed by Sean Busbey (fs)
  Make CryptoOutputStream behave like DFSOutputStream wrt synchronization
  

• HADOOP-11693. Major bug reported by Duo Xu and fixed by Duo Xu (tools)
  Azure Storage FileSystem rename operations are throttled too aggressively to complete HBase WAL archiving.
  

• HADOOP-11691. Critical bug reported by Remus Rusanu and fixed by Kiran Kumar M R (build , native)
  X86 build of libwinutils is broken
  

• HADOOP-11686. Major bug reported by zhangduo and fixed by zhangduo (security , test)
  MiniKDC cannot change ORG_NAME or ORG_DOMAIN
  

• HADOOP-11674. Critical bug reported by Sean Busbey and fixed by Sean Busbey (io)
  oneByteBuf in CryptoInputStream and CryptoOutputStream should be non static
  

• HADOOP-11670. Blocker sub-task reported by Adam Budde and fixed by Adam Budde (fs/s3)
  Regression: s3a auth setup broken
  

• HADOOP-11666. Major bug reported by Akira AJISAKA and fixed by Byron Wong
  Revert the format change of du output introduced by HADOOP-6857
  

• HADOOP-11658. Minor improvement reported by Kai Zheng and fixed by Kai Zheng
  Externalize io.compression.codecs property
  

• HADOOP-11648. Major improvement reported by Liang Xie and fixed by Liang Xie (net)
  Set DomainSocketWatcher thread name explicitly
  

• HADOOP-11642. Major improvement reported by shashank and fixed by shashank (tools)
  Upgrade azure sdk version from 0.6.0 to 2.0.0
  

• HADOOP-11639. Major bug reported by Chris Nauroth and fixed by Remus Rusanu (native)
  Clean up Windows native code compilation warnings related to Windows Secure Container Executor.
  

• HADOOP-11638. Major bug reported by Dmitry Sivachenko and fixed by Kiran Kumar M R (native)
  OpensslSecureRandom.c pthreads_thread_id should support FreeBSD and Solaris in addition to Linux
  

• HADOOP-11634. Major bug reported by Brahma Reddy Battula and fixed by Brahma Reddy Battula (documentation)
  Description of webhdfs' principal/keytab should switch places each other
  

• HADOOP-11633. Major improvement reported by Colin Patrick McCabe and fixed by Masatake Iwasaki (documentation)
  Convert remaining branch-2 .apt.vm files to markdown
  

• HADOOP-11632. Minor improvement reported by Akira AJISAKA and fixed by Akira AJISAKA
  Cleanup Find.java to remove SupressWarnings annotations
  

• HADOOP-11629. Major bug reported by shanyu zhao and fixed by shanyu zhao (tools)
  WASB filesystem should not start BandwidthGaugeUpdater if fs.azure.skip.metrics set to true
  

• HADOOP-11620. Major improvement reported by Arun Suresh and fixed by Arun Suresh (kms)
  Add support for load balancing across a group of KMS for HA
  

• HADOOP-11619. Major bug reported by Gera Shegalov and fixed by Brahma Reddy Battula (fs)
  FTPFileSystem should override getDefaultPort
  

• HADOOP-11618. Major bug reported by Gera Shegalov and fixed by Brahma Reddy Battula (fs)
  DelegateToFileSystem erroneously uses default FS's port in constructor
  

• HADOOP-11615. Minor bug reported by Akira AJISAKA and fixed by Brahma Reddy Battula (documentation)
  Update ServiceLevelAuth.md for YARN
  

• HADOOP-11612. Major task reported by Robert Kanter and fixed by Robert Kanter
  Workaround for Curator's ChildReaper requiring Guava 15+
  

• HADOOP-11609. Major bug reported by Brahma Reddy Battula and fixed by Varun Saxena (documentation , security)
  Correct credential commands info in CommandsManual.html#credential
  

• HADOOP-11607. Trivial improvement reported by Lei (Eddy) Xu and fixed by Lei (Eddy) Xu (fs/s3)
  Reduce log spew in S3AFileSystem
  

• HADOOP-11605. Minor bug reported by Gera Shegalov and fixed by Gera Shegalov (fs)
  FilterFileSystem#create with ChecksumOpt should propagate it to wrapped FS
  

• HADOOP-11604. Critical bug reported by Liang Xie and fixed by Chris Nauroth (net)
  Prevent ConcurrentModificationException while closing domain sockets during shutdown of DomainSocketWatcher thread.
  

• HADOOP-11602. Blocker bug reported by Tsuyoshi Ozawa and fixed by Tsuyoshi Ozawa
  Fix toUpperCase/toLowerCase to use Locale.ENGLISH
  

• HADOOP-11600. Major sub-task reported by Tsuyoshi Ozawa and fixed by Tsuyoshi Ozawa
  Fix up source codes to be compiled with Guava 17.0
  

• HADOOP-11599. Minor improvement reported by zhihai xu and fixed by zhihai xu (ipc)
  Client#getTimeout should use IPC_CLIENT_PING_DEFAULT when IPC_CLIENT_PING_KEY is not configured.
  

• HADOOP-11595. Major bug reported by Yi Liu and fixed by Yi Liu (fs)
  Add default implementation for AbstractFileSystem#truncate
  

• HADOOP-11589. Minor improvement reported by Akira AJISAKA and fixed by Rakesh R (net)
  NetUtils.createSocketAddr should trim the input URI
  

• HADOOP-11587. Trivial bug reported by Xiaoyu Yao and fixed by Xiaoyu Yao (test)
  TestMapFile#testMainMethodMapFile creates test files in hadoop-common project root
  

• HADOOP-11586. Minor improvement reported by Ray Chiang and fixed by Ray Chiang (metrics)
  Update use of Iterator to Iterable in AbstractMetricsContext.java
  

• HADOOP-11584. Blocker sub-task reported by Dan Hecht and fixed by Brahma Reddy Battula (fs/s3)
  s3a file block size set to 0 in getFileStatus
  

• HADOOP-11579. Major improvement reported by Steve Loughran and fixed by Konstantin Shvachko (documentation)
  Documentation for truncate
  

• HADOOP-11571. Blocker bug reported by Steve Loughran and fixed by Steve Loughran (fs/s3)
  Über-jira: S3a stabilisation phase I
  

• HADOOP-11570. Major sub-task reported by Dan Hecht and fixed by Dan Hecht (fs/s3)
  S3AInputStream.close() downloads the remaining bytes of the object from S3
  

• HADOOP-11569. Major improvement reported by Vinayakumar B and fixed by Vinayakumar B
  Provide Merge API for MapFile to merge multiple similar MapFiles to one MapFile
  

• HADOOP-11558. Minor bug reported by Masatake Iwasaki and fixed by Jean-Pierre Matsumoto (documentation)
  Fix dead links to doc of hadoop-tools
  

• HADOOP-11549. Major bug reported by Yongjun Zhang and fixed by Yongjun Zhang (tools)
  flaky test detection tool failed to handle special control characters in test result
  

• HADOOP-11548. Major bug reported by Colin Patrick McCabe and fixed by Anu Engineer (build , native)
  checknative should display a nicer error message when openssl support is not compiled in
  

• HADOOP-11547. Major bug reported by Chris Nauroth and fixed by Chris Nauroth (native)
  hadoop-common native compilation fails on Windows due to missing support for __attribute__ declaration.
  

• HADOOP-11546. Major bug reported by Steve Loughran and fixed by Tsuyoshi Ozawa (build)
  Checkstyle failing: Unable to instantiate DoubleCheckedLockingCheck
  

• HADOOP-11545. Minor bug reported by Brahma Reddy Battula and fixed by Brahma Reddy Battula (security)
  ArrayIndexOutOfBoundsException is thrown with "hadoop credential list -provider"
  

• HADOOP-11544. Trivial improvement reported by Masatake Iwasaki and fixed by Masatake Iwasaki
  Remove unused configuration keys for tracing
  

• HADOOP-11543. Trivial improvement reported by Jagadesh Kiran N and fixed by Brahma Reddy Battula (scripts)
  Improve help message for hadoop/yarn command
  

• HADOOP-11535. Minor test reported by Kai Zheng and fixed by Kai Zheng
  TableMapping related tests failed due to 'successful' resolving of invalid test hostname
  

• HADOOP-11529. Minor bug reported by Masatake Iwasaki and fixed by Masatake Iwasaki (tools)
  Fix findbugs warnings in hadoop-archives
  

• HADOOP-11526. Major bug reported by Ian Rogers and fixed by Anu Engineer (io , native)
  Memory leak in Bzip2Compressor and Bzip2Decompressor
  

• HADOOP-11523. Major bug reported by Duo Xu and fixed by Duo Xu (tools)
  StorageException complaining " no lease ID" when updating FolderLastModifiedTime in WASB
  

• HADOOP-11522. Major sub-task reported by Thomas Demoor and fixed by Thomas Demoor (fs/s3)
  Update S3A Documentation
  

• HADOOP-11521. Minor sub-task reported by Thomas Demoor and fixed by Thomas Demoor (fs/s3)
  Make connection timeout configurable in s3a
  

• HADOOP-11520. Minor improvement reported by Thomas Demoor and fixed by Thomas Demoor (fs/s3)
  Clean incomplete multi-part uploads in S3A tests
  

• HADOOP-11512. Minor bug reported by Harsh J and fixed by Ryan P (io)
  Use getTrimmedStrings when reading serialization keys
  

• HADOOP-11510. Major new feature reported by Yi Liu and fixed by Yi Liu (fs)
  Expose truncate API via FileContext
  

• HADOOP-11509. Major bug reported by Xuan Gong and fixed by Xuan Gong
  change parsing sequence in GenericOptionsParser to parse -D parameters first
  

• HADOOP-11507. Minor bug reported by Talat UYARER and fixed by Talat UYARER
  Hadoop RPC Authentication problem with different user locale
  

• HADOOP-11506. Major improvement reported by Dmitriy V. Ryaboy and fixed by Gera Shegalov (conf)
  Configuration variable expansion regex expensive for long values
  

• HADOOP-11500. Major bug reported by Ted Yu and fixed by Ted Yu
  InputStream is left unclosed in ApplicationClassLoader
  

• HADOOP-11499. Minor bug reported by Ted Yu and fixed by Ted Yu
  Check of executorThreadsStarted in ValueQueue#submitRefillTask() evades lock acquisition
  

• HADOOP-11498. Major improvement reported by Masatake Iwasaki and fixed by Masatake Iwasaki
  Bump the version of HTrace to 3.1.0-incubating
  

• HADOOP-11497. Major bug reported by Christian Winkler and fixed by Christian Winkler (documentation)
  Fix typo in ClusterSetup.html#Hadoop_Startup
  

  Correct startup command for cluster data nodes

• HADOOP-11495. Major improvement reported by Allen Wittenauer and fixed by Masatake Iwasaki (documentation)
  Convert site documentation from apt to markdown
  

• HADOOP-11494. Minor bug reported by Ted Yu and fixed by Ted Yu
  Lock acquisition on WrappedInputStream#unwrappedRpcBuffer may race with another thread
  

• HADOOP-11493. Trivial bug reported by Charles Lamb and fixed by Charles Lamb (kms)
  Fix some typos in kms-acls.xml description
  

• HADOOP-11492. Major task reported by Karthik Kambatla and fixed by Arun Suresh
  Bump up curator version to 2.7.1
  

  Apache Curator version change: Apache Hadoop has updated the version of Apache Curator used from 2.6.0 to 2.7.1. This change should be binary and source compatible for the majority of downstream users. Notable exceptions: # Binary incompatible change: org.apache.curator.utils.PathUtils.validatePath(String) changed return types. Downstream users of this method will need to recompile. # Source incompatible change: org.apache.curator.framework.recipes.shared.SharedCountReader added a method to its interface definition. Downstream users with custom implementations of this interface can continue without binary compatibility problems but will need to modify their source code to recompile. # Source incompatible change: org.apache.curator.framework.recipes.shared.SharedValueReader added a method to its interface definition. Downstream users with custom implementations of this interface can continue without binary compatibility problems but will need to modify their source code to recompile. Downstream users are reminded that while the Hadoop community will attempt to avoid egregious incompatible dependency changes, there is currently no policy around when Hadoop's exposed dependencies will change across versions (ref https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#Java_Classpath).

• HADOOP-11490. Major new feature reported by Konstantin Shvachko and fixed by Milan Desai (fs)
  Expose truncate API via FileSystem and shell command
  

• HADOOP-11489. Minor task reported by Ted Yu and fixed by Ted Yu (nfs)
  Dropping dependency on io.netty from hadoop-nfs' pom.xml
  

• HADOOP-11488. Minor bug reported by Harsh J and fixed by Daisuke Kobayashi (fs/s3)
  Difference in default connection timeout for S3A FS
  

• HADOOP-11483. Major improvement reported by Colin Patrick McCabe and fixed by Akira AJISAKA
  HardLink.java should use the jdk7 createLink method
  

• HADOOP-11482. Major bug reported by Arun Suresh and fixed by Arun Suresh
  Use correct UGI when KMSClientProvider is called by a proxy user
  

• HADOOP-11481. Minor improvement reported by Yi Yao and fixed by Charles Lamb
  ClassCastException while using a key created by keytool to create encryption zone.
  

• HADOOP-11480. Minor bug reported by Ted Yu and fixed by Ted Yu (documentation)
  Typo in hadoop-aws/index.md uses wrong scheme for test.fs.s3.name
  

• HADOOP-11470. Major bug reported by Sangjin Lee and fixed by Sangjin Lee
  Remove some uses of obsolete guava APIs from the hadoop codebase
  

• HADOOP-11469. Minor bug reported by Dian Fu and fixed by Dian Fu (kms)
  KMS should skip default.key.acl and whitelist.key.acl when loading key acl
  

• HADOOP-11467. Critical bug reported by Robert Kanter and fixed by Yongjun Zhang (security)
  KerberosAuthenticator can connect to a non-secure cluster
  

• HADOOP-11465. Major sub-task reported by Varun Saxena and fixed by Varun Saxena
  Fix findbugs warnings in hadoop-gridmix
  

• HADOOP-11464. Major improvement reported by Chris Nauroth and fixed by Chris Nauroth (bin)
  Reinstate support for launching Hadoop processes on Windows using Cygwin.
  

  We have reinstated support for launching Hadoop processes on Windows by using Cygwin to run the shell scripts. All processes still must have access to the native components: hadoop.dll and winutils.exe.

• HADOOP-11463. Major task reported by Ted Yu and fixed by Ted Yu (fs/s3)
  Replace method-local TransferManager object with S3AFileSystem#transfers
  

• HADOOP-11462. Major bug reported by Ayappan and fixed by Ayappan (test)
  TestSocketIOWithTimeout needs change for PowerPC platform
  

• HADOOP-11459. Minor bug reported by Vinayakumar B and fixed by Vinayakumar B
  Fix recent findbugs in ActiveStandbyElector, NetUtils and ShellBasedIdMapping
  

• HADOOP-11455. Minor improvement reported by Charles Lamb and fixed by Charles Lamb (security)
  KMS and Credential CLI should request confirmation for deletion by default
  

• HADOOP-11450. Minor bug reported by Tsuyoshi Ozawa and fixed by Varun Saxena (tools/distcp)
  Cleanup DistCpV1 not to use deprecated methods and fix javadocs
  

• HADOOP-11449. Major bug reported by Alec Taylor and fixed by Chris Nauroth (build)
  [JDK8] Cannot build on Windows: error: unexpected end tag: </ul>
  

• HADOOP-11448. Minor improvement reported by Akira AJISAKA and fixed by Tsuyoshi Ozawa
  Fix findbugs warnings in FileBasedIPList
  

• HADOOP-11446. Major bug reported by Ted Yu and fixed by Ted Yu (fs/s3)
  S3AOutputStream should use shared thread pool to avoid OutOfMemoryError
  

  The following parameters are introduced in this JIRA: fs.s3a.threads.max: the maximum number of threads to allow in the pool used by TransferManager fs.s3a.threads.core: the number of threads to keep in the pool used by TransferManager fs.s3a.threads.keepalivetime: when the number of threads is greater than the core, this is the maximum time that excess idle threads will wait for new tasks before terminating fs.s3a.max.total.tasks: the maximum number of tasks that the LinkedBlockingQueue can hold

• HADOOP-11445. Critical bug reported by Ankit Kamboj and fixed by Ankit Kamboj
  Bzip2Codec: Data block is skipped when position of newly created stream is equal to start of split
  

• HADOOP-11442. Major improvement reported by shashank and fixed by shashank (tools)
  hadoop-azure: Create test jar
  

• HADOOP-11441. Minor improvement reported by shashank and fixed by shashank (tools)
  Hadoop-azure: Change few methods scope to public
  

• HADOOP-11440. Minor improvement reported by Akira AJISAKA and fixed by Kengo Seki
  Use "test.build.data" instead of "build.test.dir" for testing in ClientBaseWithFixes
  

• HADOOP-11432. Major test reported by Liang Xie and fixed by Liang Xie (fs)
  Fix SymlinkBaseTest#testCreateLinkUsingPartQualPath2
  

• HADOOP-11431. Major bug reported by Hervé Boutemy and fixed by Hervé Boutemy
  clean up redundant maven-site-plugin configuration
  

• HADOOP-11430. Major improvement reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe
  Add GenericTestUtils#disableLog, GenericTestUtils#setLogLevel
  

• HADOOP-11429. Major sub-task reported by Varun Saxena and fixed by Varun Saxena
  Findbugs warnings in hadoop extras
  

• HADOOP-11428. Major bug reported by Arpit Agarwal and fixed by Arpit Agarwal (documentation)
  Remove obsolete reference to Cygwin in BUILDING.txt
  

• HADOOP-11427. Major improvement reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe
  ChunkedArrayList: fix removal via iterator and implement get
  

• HADOOP-11422. Minor improvement reported by Yi Liu and fixed by Yi Liu
  Check CryptoCodec is AES-CTR for Crypto input/output stream
  

• HADOOP-11421. Major improvement reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe
  Add IOUtils#listDirectory
  

• HADOOP-11420. Trivial bug reported by Hervé Boutemy and fixed by Hervé Boutemy (site)
  Use latest maven-site-plugin and replace link to svn with link to git
  

• HADOOP-11419. Minor improvement reported by Hervé Boutemy and fixed by Hervé Boutemy (build)
  improve hadoop-maven-plugins
  

• HADOOP-11416. Major improvement reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe
  Move ChunkedArrayList into hadoop-common
  

• HADOOP-11414. Minor bug reported by Ted Yu and fixed by Tsuyoshi Ozawa
  FileBasedIPList#readLines() can leak file descriptors
  

• HADOOP-11412. Trivial bug reported by Hervé Boutemy and fixed by Hervé Boutemy
  POMs mention "The Apache Software License" rather than "Apache License"
  

• HADOOP-11411. Major bug reported by Jason Dere and fixed by
  Hive build failure on hadoop-2.7 due to HADOOP-11356
  

• HADOOP-11410. Major improvement reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe
  make the rpath of libhadoop.so configurable
  

• HADOOP-11409. Major bug reported by Jason Lowe and fixed by Gera Shegalov
  FileContext.getFileContext can stack overflow if default fs misconfigured
  

• HADOOP-11403. Major bug reported by Malcolm Kavalsky and fixed by Malcolm Kavalsky
  Avoid using sys_errlist on Solaris, which lacks support for it
  

• HADOOP-11402. Major bug reported by Colin Patrick McCabe and fixed by Varun Saxena
  Negative user-to-group cache entries are never cleared for never-again-accessed users
  

• HADOOP-11400. Major bug reported by Kamil Gorlo and fixed by Kamil Gorlo (metrics)
  GraphiteSink does not reconnect to Graphite after 'broken pipe'
  

• HADOOP-11399. Minor improvement reported by Ray Chiang and fixed by Ray Chiang
  Java Configuration file and .xml files should be automatically cross-compared
  

• HADOOP-11396. Major improvement reported by Chris Nauroth and fixed by Chris Nauroth (documentation)
  Provide navigation in the site documentation linking to the Hadoop Compatible File Systems.
  

• HADOOP-11395. Major improvement reported by Chris Nauroth and fixed by Chris Nauroth (documentation)
  Add site documentation for Azure Storage FileSystem integration.
  

• HADOOP-11394. Major bug reported by Chris Nauroth and fixed by Chris Nauroth (documentation)
  hadoop-aws documentation missing.
  

• HADOOP-11390. Minor improvement reported by Steve Loughran and fixed by Varun Saxena (metrics)
  Metrics 2 ganglia provider to include hostname in unresolved address problems
  

• HADOOP-11389. Major sub-task reported by Haohui Mai and fixed by Haohui Mai
  Clean up byte to string encoding issues in hadoop-common
  

• HADOOP-11388. Minor bug reported by Haohui Mai and fixed by Li Lu
  Remove deprecated o.a.h.metrics.file.FileContext
  

• HADOOP-11386. Major bug reported by Li Lu and fixed by Li Lu
  Replace \n by %n in format hadoop-common format strings
  

• HADOOP-11385. Critical bug reported by Haohui Mai and fixed by Haohui Mai
  Prevent cross site scripting attack on JMXJSONServlet
  

• HADOOP-11381. Major sub-task reported by Li Lu and fixed by Li Lu
  Fix findbugs warnings in hadoop-distcp, hadoop-aws, hadoop-azure, and hadoop-openstack
  

• HADOOP-11379. Major sub-task reported by Li Lu and fixed by Li Lu
  Fix new findbugs warnings in hadoop-auth*
  

• HADOOP-11378. Major sub-task reported by Li Lu and fixed by Li Lu
  Fix new findbugs warnings in hadoop-kms
  

• HADOOP-11377. Major sub-task reported by Steve Loughran and fixed by Tsuyoshi Ozawa (build)
  jdiff failing on java 7 and java 8, "Null.java" not found
  

• HADOOP-11372. Major sub-task reported by Li Lu and fixed by Li Lu
  Fix new findbugs warnings in mapreduce-examples
  

• HADOOP-11370. Major sub-task reported by Zhijie Shen and fixed by
  Fix new findbug warnings hadoop-yarn
  

• HADOOP-11369. Major sub-task reported by Li Lu and fixed by Li Lu
  Fix new findbugs warnings in hadoop-mapreduce-client, non-core directories
  

• HADOOP-11368. Major bug reported by Arun Suresh and fixed by Arun Suresh (kms)
  Fix SSLFactory truststore reloader thread leak in KMSClientProvider
  

• HADOOP-11367. Major sub-task reported by Li Lu and fixed by Li Lu
  Fix warnings from findbugs 3.0 in hadoop-streaming
  

• HADOOP-11363. Major bug reported by Steve Loughran and fixed by Steve Loughran (build)
  Hadoop maven surefire-plugin uses must set heap size
  

• HADOOP-11358. Major test reported by Yi Liu and fixed by Yi Liu (security , test)
  Tests for encryption/decryption with IV calculation overflow
  

• HADOOP-11355. Minor bug reported by Arun Suresh and fixed by Arun Suresh
  When accessing data in HDFS and the key has been deleted, a Null Pointer Exception is shown.
  

• HADOOP-11354. Major bug reported by Ted Yu and fixed by Ted Yu
  ThrottledInputStream doesn't perform effective throttling
  

• HADOOP-11350. Major bug reported by Benoy Antony and fixed by Benoy Antony (security)
  The size of header buffer of HttpServer is too small when HTTPS is enabled
  

• HADOOP-11349. Minor bug reported by Chris Nauroth and fixed by Varun Saxena (fs)
  RawLocalFileSystem leaks file descriptor while creating a file if creat succeeds but chmod fails.
  

• HADOOP-11348. Minor bug reported by Dian Fu and fixed by Dian Fu
  Remove unused variable from CMake error message for finding openssl
  

  Test failure is unrelated. Committed to 2.7. Thanks, Dian.

• HADOOP-11344. Major bug reported by Arun Suresh and fixed by Arun Suresh
  KMS kms-config.sh sets a default value for the keystore password even in non-ssl setup
  

• HADOOP-11343. Blocker bug reported by Jerry Chen and fixed by Jerry Chen (security)
  Overflow is not properly handled in caclulating final iv for AES CTR
  

• HADOOP-11342. Major bug reported by Dian Fu and fixed by Dian Fu (kms , security)
  KMS key ACL should ignore ALL operation for default key ACL and whitelist key ACL
  

• HADOOP-11341. Major new feature reported by Arun Suresh and fixed by Arun Suresh (kms , security)
  KMS support for whitelist key ACLs
  

• HADOOP-11337. Major bug reported by Dian Fu and fixed by Dian Fu
  KeyAuthorizationKeyProvider access checks need to be done atomically
  

• HADOOP-11333. Major bug reported by zhaoyunjiong and fixed by zhaoyunjiong
  Fix deadlock in DomainSocketWatcher when the notification pipe is full
  

• HADOOP-11332. Major bug reported by Dian Fu and fixed by Dian Fu (security)
  KerberosAuthenticator#doSpnegoSequence should check if kerberos TGT is available in the subject
  

• HADOOP-11329. Major bug reported by Dian Fu and fixed by Arun Suresh (kms , security)
  Add JAVA_LIBRARY_PATH to KMS startup options
  

• HADOOP-11327. Minor bug reported by Tim Luo and fixed by Eric Payne (util)
  BloomFilter#not() omits the last bit, resulting in an incorrect filter
  

• HADOOP-11323. Major improvement reported by Wilfred Spiegelenburg and fixed by Wilfred Spiegelenburg (performance)
  WritableComparator#compare keeps reference to byte array
  

• HADOOP-11322. Major bug reported by Dian Fu and fixed by Dian Fu (security)
  key based ACL check in KMS always check KeyOpType.MANAGEMENT even actual KeyOpType is not MANAGEMENT
  

• HADOOP-11321. Major bug reported by Chris Nauroth and fixed by Chris Nauroth (fs)
  copyToLocal cannot save a file to an SMB share unless the user has Full Control permissions.
  

• HADOOP-11318. Major bug reported by Akira AJISAKA and fixed by Akira AJISAKA (documentation)
  Update the document for hadoop fs -stat
  

• HADOOP-11317. Major sub-task reported by Steve Loughran and fixed by Tim Robertson (build)
  Increment SLF4J version to 1.7.10
  

• HADOOP-11316. Blocker bug reported by Tsuyoshi Ozawa and fixed by Tsuyoshi Ozawa
  "mvn package -Pdist,docs -DskipTests -Dtar" fails because of non-ascii characters
  

• HADOOP-11313. Major improvement reported by Tsuyoshi Ozawa and fixed by Tsuyoshi Ozawa (documentation)
  Adding a document about NativeLibraryChecker
  

• HADOOP-11312. Major bug reported by Andrew Wang and fixed by Andrew Wang (security)
  Fix unit tests to not use uppercase key names
  

• HADOOP-11311. Major bug reported by Andrew Wang and fixed by Andrew Wang (security)
  Restrict uppercase key names from being created with JCEKS
  

  Keys with uppercase names can no longer be created when using the JavaKeyStoreProvider to resolve ambiguity about case-sensitivity in the KeyStore spec.

• HADOOP-11309. Blocker bug reported by Gera Shegalov and fixed by Gera Shegalov
  System class pattern package.Foo should match package.Foo$Bar, too
  

• HADOOP-11301. Major improvement reported by Maysam Yabandeh and fixed by Maysam Yabandeh
  [optionally] update jmx cache to drop old metrics
  

• HADOOP-11300. Major bug reported by Arun Suresh and fixed by Arun Suresh (kms)
  KMS startup scripts must not display the keystore / truststore passwords
  

• HADOOP-11295. Major bug reported by Ming Ma and fixed by Ming Ma
  RPC Server Reader thread can't shutdown if RPCCallQueue is full
  

• HADOOP-11294. Minor bug reported by Brandon Li and fixed by Brandon Li (nfs)
  Nfs3FileAttributes should not change the values of rdev, nlink and size in the constructor
  

• HADOOP-11291. Minor improvement reported by Stephen Chu and fixed by Stephen Chu (security)
  Log the cause of SASL connection failures
  

• HADOOP-11289. Trivial bug reported by Charles Lamb and fixed by Charles Lamb (net)
  Fix typo in RpcUtil log message
  

• HADOOP-11287. Major bug reported by Haohui Mai and fixed by Li Lu
  Simplify UGI#reloginFromKeytab for Java 7+
  

• HADOOP-11283. Minor bug reported by Ted Yu and fixed by Varun Saxena
  Potentially unclosed SequenceFile.Writer in DistCpV1#setup()
  

• HADOOP-11273. Major bug reported by Gao Zhong Liang and fixed by Gao Zhong Liang (test)
  TestMiniKdc failure: login options not compatible with IBM JDK
  

• HADOOP-11272. Major bug reported by Arun Suresh and fixed by Arun Suresh
  Allow ZKSignerSecretProvider and ZKDelegationTokenSecretManager to use the same curator client
  

• HADOOP-11271. Minor bug reported by Vinayakumar B and fixed by Vinayakumar B
  Use Time.monotonicNow() in Shell.java instead of Time.now()
  

• HADOOP-11269. Major bug reported by Haohui Mai and fixed by Li Lu
  Add java 8 profile for hadoop-annotations
  

• HADOOP-11268. Minor bug reported by Haohui Mai and fixed by Li Lu
  Update BUILDING.txt to remove the workaround for tools.jar
  

• HADOOP-11267. Minor bug reported by Stephen Chu and fixed by Stephen Chu (security , test)
  TestSecurityUtil fails when run with JDK8 because of empty principal names
  

• HADOOP-11266. Trivial bug reported by Masatake Iwasaki and fixed by Masatake Iwasaki (build)
  Remove no longer supported activation properties for packaging from pom
  

• HADOOP-11261. Major improvement reported by Thomas Demoor and fixed by Thomas Demoor (fs/s3)
  Set custom endpoint for S3A
  

• HADOOP-11257. Blocker improvement reported by Allen Wittenauer and fixed by Masatake Iwasaki
  Update "hadoop jar" documentation to warn against using it for launching yarn jars
  

• HADOOP-11256. Minor bug reported by Masatake Iwasaki and fixed by Masatake Iwasaki (documentation)
  Some site docs have inconsistent appearance
  

• HADOOP-11248. Major bug reported by shanyu zhao and fixed by shanyu zhao (fs)
  Add hadoop configuration to disable Azure Filesystem metrics collection
  

• HADOOP-11246. Major bug reported by Haohui Mai and fixed by Steve Loughran
  Move jenkins to Java 7
  

• HADOOP-11238. Minor bug reported by Chris Li and fixed by Chris Li
  Update the NameNode's Group Cache in the background when possible
  

• HADOOP-11236. Trivial bug reported by Abhiraj Butala and fixed by Abhiraj Butala (documentation)
  NFS: Fix javadoc warning in RpcProgram.java
  

• HADOOP-11231. Minor improvement reported by Haohui Mai and fixed by Li Lu
  Remove dead code in ServletUtil
  

• HADOOP-11230. Major bug reported by Robert Kanter and fixed by Robert Kanter (test)
  Add missing dependency of bouncycastle for kms, httpfs, hdfs, MR and YARN
  

• HADOOP-11213. Minor bug reported by Wei Yan and fixed by Wei Yan
  Typos in html pages: SecureMode and EncryptedShuffle
  

• HADOOP-11211. Major bug reported by Yitong Zhou and fixed by Yitong Zhou
  mapreduce.job.classloader.system.classes semantics should be order-independent
  

• HADOOP-11209. Major bug reported by Josh Rosen and fixed by Varun Saxena (conf)
  Configuration#updatingResource/finalParameters are not thread-safe
  

• HADOOP-11201. Blocker bug reported by Gera Shegalov and fixed by Gera Shegalov (tools)
  Hadoop Archives should support globs resolving to files
  

• HADOOP-11188. Major improvement reported by Eric Hanson and fixed by Eric Hanson (fs)
  hadoop-azure: automatically expand page blobs when they become full
  

• HADOOP-11187. Major bug reported by Arun Suresh and fixed by Arun Suresh
  NameNode - KMS communication fails after a long period of inactivity
  

• HADOOP-11186. Minor bug reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe
  documentation should talk about hadoop.htrace.spanreceiver.classes, not hadoop.trace.spanreceiver.classes
  

• HADOOP-11183. Major sub-task reported by Thomas Demoor and fixed by Thomas Demoor (fs/s3)
  Memory-based S3AOutputstream
  

• HADOOP-11173. Minor improvement reported by Andrew Wang and fixed by Andrew Wang
  Improve error messages for some KeyShell commands
  

• HADOOP-11172. Major improvement reported by Yongjun Zhang and fixed by Yongjun Zhang
  Improve error message in Shell#runCommand on OutOfMemoryError
  

• HADOOP-11171. Major improvement reported by Thomas Demoor and fixed by Thomas Demoor (fs/s3)
  Enable using a proxy server to connect to S3a.
  

• HADOOP-11166. Major bug reported by Andrew Wang and fixed by Andrew Wang
  Remove ulimit from test-patch.sh
  

• HADOOP-11165. Minor test reported by Ted Yu and fixed by Stephen Chu (test)
  TestUTF8 fails when run against java 8
  

• HADOOP-11157. Major bug reported by Gregory Chanan and fixed by Arun Suresh (security)
  ZKDelegationTokenSecretManager never shuts down listenerThreadPool
  

• HADOOP-11156. Major bug reported by zhihai xu and fixed by zhihai xu (fs)
  DelegateToFileSystem should implement getFsStatus(final Path f).
  

• HADOOP-11125. Major test reported by Ted Yu and fixed by Masanori Oyama
  Remove redundant tests in TestOsSecureRandom
  

• HADOOP-11045. Major improvement reported by Yongjun Zhang and fixed by Yongjun Zhang (build , tools)
  Introducing a tool to detect flaky tests of hadoop jenkins test job
  

• HADOOP-11039. Minor bug reported by Yi Liu and fixed by Yi Liu (documentation)
  ByteBufferReadable API doc is inconsistent with the implementations.
  

• HADOOP-11032. Major improvement reported by Gary Steelman and fixed by Tsuyoshi Ozawa
  Replace use of Guava's Stopwatch with Hadoop's StopWatch
  

• HADOOP-11014. Minor bug reported by Ted Yu and fixed by Tsuyoshi Ozawa (security)
  Potential resource leak in JavaKeyStoreProvider due to unclosed stream
  

• HADOOP-11008. Minor bug reported by Masatake Iwasaki and fixed by Masatake Iwasaki (documentation)
  Remove duplicated description about proxy-user in site documents
  

• HADOOP-11000. Major bug reported by Ming Ma and fixed by Ming Ma
  HAServiceProtocol's health state is incorrectly transitioned to SERVICE_NOT_RESPONDING
  

• HADOOP-10987. Major improvement reported by Kihwal Lee and fixed by Kihwal Lee
  Provide an iterator-based listing API for FileSystem
  

• HADOOP-10976. Minor improvement reported by Masatake Iwasaki and fixed by Masatake Iwasaki (documentation)
  moving the source code of hadoop-tools docs to the directory under hadoop-tools
  

• HADOOP-10953. Minor bug reported by Liang Xie and fixed by Liang Xie (net)
  NetworkTopology#add calls NetworkTopology#toString without holding the netlock
  

• HADOOP-10852. Major bug reported by Benoy Antony and fixed by Benoy Antony (security)
  NetgroupCache is not thread-safe
  

• HADOOP-10847. Minor improvement reported by Kai Zheng and fixed by pascal oliva (security)
  Remove the usage of sun.security.x509.* in testing code
  

• HADOOP-10840. Major bug reported by shanyu zhao and fixed by shanyu zhao (metrics)
  Fix OutOfMemoryError caused by metrics system in Azure File System
  

• HADOOP-10809. Major improvement reported by Mike Liddell and fixed by Eric Hanson (tools)
  hadoop-azure: page blob support
  

• HADOOP-10786. Major improvement reported by Tobi Vollebregt and fixed by Stephen Chu (security)
  Fix UGI#reloginFromKeytab on Java 8
  

• HADOOP-10748. Major bug reported by Haohui Mai and fixed by Haohui Mai
  HttpServer2 should not load JspServlet
  

• HADOOP-10728. Major new feature reported by Mike Liddell and fixed by Mike Liddell (tools)
  Metrics system for Windows Azure Storage Filesystem
  

• HADOOP-10717. Blocker bug reported by Dapeng Sun and fixed by Dapeng Sun
  HttpServer2 should load jsp DTD from local jars instead of going remote
  

• HADOOP-10714. Critical bug reported by David S. Wang and fixed by Juan Yu (fs/s3)
  AmazonS3Client.deleteObjects() need to be limited to 1000 entries per call
  

• HADOOP-10703. Major bug reported by Benoy Antony and fixed by Benoy Antony (security)
  HttpServer2 creates multiple authentication filters
  

• HADOOP-10690. Minor bug reported by Ted Yu and fixed by Chen He (tools)
  Lack of synchronization on access to InputStream in NativeAzureFileSystem#NativeAzureFsInputStream#close()
  

• HADOOP-10689. Minor bug reported by Ted Yu and fixed by Chen He (tools)
  InputStream is not closed in AzureNativeFileSystemStore#retrieve()
  

• HADOOP-10670. Minor improvement reported by Kai Zheng and fixed by Kai Zheng (security)
  Allow AuthenticationFilters to load secret from signature secret files
  

• HADOOP-10668. Major test reported by Ted Yu and fixed by Ming Ma (test)
  TestZKFailoverControllerStress#testExpireBackAndForth occasionally fails
  

• HADOOP-10651. Major sub-task reported by Benoy Antony and fixed by Benoy Antony (security)
  Add ability to restrict service access using IP addresses and hostnames
  

• HADOOP-10626. Major improvement reported by Jason Hubbard and fixed by Jason Hubbard (security)
  Limit Returning Attributes for LDAP search
  

• HADOOP-10574. Major sub-task reported by Steve Loughran and fixed by Akira AJISAKA (build)
  Bump the maven plugin versions too -moving the numbers into properties
  

• HADOOP-10563. Major improvement reported by Haohui Mai and fixed by Haohui Mai
  Remove the dependency of jsp in trunk
  

• HADOOP-10542. Minor bug reported by Ted Yu and fixed by Ted Yu (fs/s3)
  Potential null pointer dereference in Jets3tFileSystemStore#retrieveBlock()
  

• HADOOP-10530. Blocker improvement reported by Steve Loughran and fixed by Steve Loughran (build)
  Make hadoop trunk build on Java7+ only
  

• HADOOP-10525. Minor improvement reported by Akira AJISAKA and fixed by Akira AJISAKA
  Remove DRFA.MaxBackupIndex config from log4j.properties
  

• HADOOP-10482. Major sub-task reported by Haohui Mai and fixed by Haohui Mai
  Fix various findbugs warnings in hadoop-common
  

• HADOOP-10478. Major sub-task reported by Haohui Mai and fixed by Li Lu
  Fix new findbugs warnings in hadoop-maven-plugins
  

• HADOOP-10476. Major sub-task reported by Haohui Mai and fixed by Haohui Mai (build)
  Bumping the findbugs version to 3.0.0
  

• HADOOP-10181. Minor bug reported by Andrew Otto and fixed by Andrew Johnson (metrics)
  GangliaContext does not work with multicast ganglia setup
  

  Hadoop metrics sent to Ganglia over multicast now support optional configuration of socket TTL. The default TTL is 1, which preserves the behavior of prior Hadoop versions. Clusters that span multiple subnets/VLANs will likely want to increase this.

• HADOOP-10140. Minor improvement reported by Ian Jackson and fixed by Kiran Kumar M R (scripts)
  Specification of HADOOP_CONF_DIR via the environment in hadoop_config.cmd
  

• HADOOP-10134. Minor bug reported by Andrew Purtell and fixed by Andrew Purtell
  [JDK8] Fix Javadoc errors caused by incorrect or illegal tags in doc comments
  

• HADOOP-10062. Major bug reported by Shinichi Yamashita and fixed by Sangjin Lee (metrics)
  race condition in MetricsSystemImpl#publishMetricsNow that causes incorrect results
  

• HADOOP-9992. Major improvement reported by Akshay Radia and fixed by Akshay Radia (test)
  Modify the NN loadGenerator to optionally run as a MapReduce job
  

• HADOOP-9922. Major bug reported by Vinayakumar B and fixed by Kiran Kumar M R (build , native)
  hadoop windows native build will fail in 32 bit machine
  

  The Hadoop Common native components now support 32-bit build targets on Windows.

• HADOOP-9907. Critical bug reported by Jian He and fixed by Akira AJISAKA
  Webapp http://hostname:port/metrics link is not working
  

• HADOOP-9869. Minor improvement reported by Steve Loughran and fixed by Tsuyoshi Ozawa (conf)
  Configuration.getSocketAddr()/getEnum() should use getTrimmed()
  

• HADOOP-9629. Major new feature reported by Mostafa Elhemali and fixed by Chris Nauroth (tools)
  Support Windows Azure Storage - Blob as a file system in Hadoop
  

  Hadoop now supports integration with Azure Storage as an alternative Hadoop Compatible File System.

• HADOOP-9329. Trivial task reported by Colin Patrick McCabe and fixed by Vijay Bhat (documentation)
  document native build dependencies in BUILDING.txt
  

  Added a section to BUILDING.txt on how to install required / optional packages on a clean install of Ubuntu 14.04 LTS Desktop. Went through the CMakeLists.txt files in the repo and added the following optional library dependencies - Snappy, Bzip2, Linux FUSE and Jansson. Updated the required packages / version numbers from the trunk branch version of BUILDING.txt.

• HADOOP-9137. Major bug reported by Sanjay Radia and fixed by Kihwal Lee
  Support connection limiting in IPC server
  

• HADOOP-9087. Minor bug reported by Mostafa Elhemali and fixed by Akira AJISAKA (metrics)
  Queue size metric for metric sinks isn't actually maintained
  

• HADOOP-8989. Major new feature reported by Marco Nicosia and fixed by Jonathan Allen
  hadoop fs -find feature
  

  New fs -find command

• HADOOP-8757. Minor improvement reported by Todd Lipcon and fixed by Ray Chiang (metrics)
  Metrics should disallow names with invalid characters
  

• HADOOP-8642. Major bug reported by Eli Collins and fixed by Akira AJISAKA (documentation , native)
  Document that io.native.lib.available only controls native bz2 and zlib compression codecs
  

• HADOOP-8059. Major improvement reported by Suresh Srinivas and fixed by Brandon Li (documentation)
  Add javadoc to InterfaceAudience and InterfaceStability
  

• HADOOP-7984. Minor new feature reported by Eli Collins and fixed by Akira AJISAKA (scripts)
  Add hadoop --loglevel option to change log level
  

• HADOOP-6221. Minor bug reported by Steve Loughran and fixed by Steve Loughran (ipc)
  RPC Client operations cannot be interrupted
  

• HADOOP-4297. Major improvement reported by Yoram Kulbak and fixed by Tsz Wo Nicholas Sze (build)
  Enable Java assertions when running tests