These release notes cover new developer and user-facing incompatibilities, important issues, features, and major improvements.
This JIRA makes following change: Change Router metrics context from ‘router’ to ‘dfs’.
Mount tables support ACL, The users won’t be able to modify their own entries (we are assuming these old (no-permissions before) mount table with owner:superuser, group:supergroup, permission:755 as the default permissions). The fix way is login as superuser to modify these mount table entries.
Support multi-thread pre-read in AliyunOSSInputStream to improve the sequential read performance from Hadoop to Aliyun OSS.
MapReduce jobs that output to filesystems without direct support for recursive delete can set mapreduce.fileoutputcommitter.task.cleanup.enabled=true to have each task delete their intermediate work directory rather than waiting for the ApplicationMaster to clean up at the end of the job. This can significantly speed up the cleanup phase for large jobs on such filesystems.
Added an option to not disables short-circuit reads on failures, by setting dfs.domain.socket.disable.interval.seconds to 0.
Fix the document error of setting up HFDS Router Federation
Change default State Store from local file to ZooKeeper. This will require additional zk address to be configured.
HBase integration module was mixed up with for hbase-server and hbase-client dependencies. This JIRA split into sub modules such that hbase-client dependent modules and hbase-server dependent modules are separated. This allows to make conditional compilation with different version of Hbase.
Use environment variable HTTPFS_HTTP_HOSTNAME to limit the IP addresses httpfs server binds to. Default: httpfs server binds to all IP addresses on the host.
WASB: Bug fix to support non-sequential page blob reads. Required for HBASE replication.
WASB: Bug fix for recent regression in hflush() and hsync().
WASB: Fix Spark process hang at shutdown due to use of non-daemon threads by updating Azure Storage Java SDK to 7.0
Federation supports and controls global quota at mount table level.
In a federated environment, a folder can be spread across multiple subclusters. Router aggregates quota that queried from these subclusters and uses that for the quota-verification.