These release notes cover new developer and user-facing incompatibilities, important issues, features, and major improvements.
FsShell mkdir now accepts a -p flag. Like unix, mkdir -p will not fail if the directory already exists. Unlike unix, intermediate directories are always created, regardless of the flag, to avoid incompatibilities at this time.
This jira only allows providing paths using back slash as separator on Windows. The back slash on *nix system will be used as escape character. The support for paths using back slash as path separator will be removed in HADOOP-8139 in release 23.3.
WARNING: No release note provided for this incompatible change.
The configuration dfs.secondary.https.port has been renamed to dfs.namenode.secondary.https-port for consistency. The old configuration is still supported via a deprecation path.
Add a private conf property dfs.datanode.fsdataset.factory to make FSDataset in Datanode pluggable.
FSVolume, is a part of FSDatasetInterface implementation, should not be referred outside FSDataset. A new FSVolumeInterface is defined. The BlockVolumeChoosingPolicy.chooseVolume(..) method signature is also updated.
Fixed FileOutputCommitter to not err out for an ‘empty-job’ whose tasks don’t write any outputs.
Exporting mapreduce.job.local.dir for mapreduce tasks to use as job-level shared scratch space.
Added new envs to separate heap size for different daemons started via bin scripts.
Changed PB implementation of LocalResource to take locks so that race conditions don’t fail tasks by inadvertantly changing the timestamps.
Fixed build to not compile 32bit container-executor binary by default on all platforms.
port number changes for resourcemanager and nodemanager
Fixed a bug in CapacityScheduler LeafQueue which was causing app-submission to fail.
Modified JobHistory records in YARN to lazily load job and task reports so as to improve UI response times.
Fixed the bin/yarn script to not print the command line unnecessarily.
Fixed failing TestJobCleanup.testCusomCleanup() and moved it to the maven build.
Fix “bin/mapred job -list” to display all jobs instead of only the jobs owned by the user.
Committed to trunk and branch-0.23. Thanks Jason.
Modified RM to allow restarted NMs to be able to join the cluster without waiting for expiry.
Fixed two bugs in Counters because of which web app displays zero counter values for framework counters.
Fixed all daemons to crash instead of hanging around when their EventHandlers get exceptions.
Fixed MR AM to close history file quickly and send a correct final state to the RM when it is killed.
Fixed node link on JobHistory webapp.
Passing a cached class-loader to ResourceBundle creator to minimize counter names lookup time.
Corrected AppIDs, JobIDs, TaskAttemptIDs to be of correct format on the web pages.