This document explains the upcoming work for upgrading S3A to AWS SDK V2. This work is tracked in HADOOP-18073.
The SDK V2 for S3 is very different from SDK V1, and brings breaking changes for S3A. A complete list of the changes can be found in the Changelog.
The major changes and how this affects S3A are listed below.
Package names have changed, all classes in SDK V2 are under software.amazon.awssdk, SDK V1 classes were under com.amazonaws.
The change in interface will mean that custom credential providers will need to be updated to now implement AwsCredentialsProvider instead of AWSCredentialProvider.
Due to change in class names, references to SDK V1 credential providers in fs.s3a.aws.credentials.provider will need to be updated to reference V2 providers.
Custom credential providers used in delegation token binding classes will also need to be updated.
The s3 client is an instance of S3Client in V2 rather than AmazonS3.
For this reason, the S3ClientFactory will be deprecated and replaced by one that creates a V2 S3Client.
The getAmazonS3ClientForTesting() method will also be updated to return the S3Client.
Interface change: com.amazonaws.auth.Signer has been replaced by software.amazon.awssdk.core.signer.Signer.
The change in signers will mean the custom signers will need to be updated to implement the new interface.