org.apache.hadoop.fs

Class FSDataOutputStreamBuilder<S extends FSDataOutputStream,B extends FSDataOutputStreamBuilder<S,B>>

java.lang.Object

org.apache.hadoop.fs.FSDataOutputStreamBuilder<S,B>

@InterfaceAudience.Public
 @InterfaceStability.Evolving
public abstract class FSDataOutputStreamBuilder<S extends FSDataOutputStream,B extends FSDataOutputStreamBuilder<S,B>>
extends Object

Builder for FSDataOutputStream and its subclasses. It is used to create FSDataOutputStream when creating a new file or appending an existing file on FileSystem. By default, it does not create parent directory that do not exist. FileSystem.createNonRecursive(Path, boolean, int, short, long, Progressable). To create missing parent directory, use recursive(). To be more generic, opt(String, int) and must(String, int) variants provide implementation-agnostic way to customize the builder. Each FS-specific builder implementation can interpret the FS-specific options accordingly, for example: // Don't if (fs instanceof FooFileSystem) { FooFileSystem fs = (FooFileSystem) fs; OutputStream out = dfs.createFile(path) .optionA() .optionB("value") .cache() .build() } else if (fs instanceof BarFileSystem) { ... } // Do OutputStream out = fs.createFile(path) .permission(perm) .bufferSize(bufSize) .opt("foofs:option.a", true) .opt("foofs:option.b", "value") .opt("barfs:cache", true) .must("foofs:cache", true) .must("barfs:cache-size", 256 * 1024 * 1024) .build(); If the option is not related to the file system, the option will be ignored. If the option is must, but not supported by the file system, a IllegalArgumentException will be thrown.

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	FSDataOutputStreamBuilder(FileSystem fileSystem, Path p) Constructor.

Method Summary

Modifier and Type	Method and Description
B	append() Append to an existing file (optional operation).
B	blockSize(long blkSize) Set block size.
B	bufferSize(int bufSize) Set the size of the buffer to be used.
abstract S	build() Create the FSDataOutputStream to write on the file system.
B	checksumOpt(org.apache.hadoop.fs.Options.ChecksumOpt chksumOpt) Set checksum opt.
B	create() Create an FSDataOutputStream at the specified path.
protected long	getBlockSize()
protected int	getBufferSize()
protected org.apache.hadoop.fs.Options.ChecksumOpt	getChecksumOpt()
protected EnumSet<CreateFlag>	getFlags()
protected FileSystem	getFS()
protected Set<String>	getMandatoryKeys() Get all the keys that are set as mandatory keys.
protected Configuration	getOptions()
protected Path	getPath()
protected FsPermission	getPermission()
protected Progressable	getProgress()
protected short	getReplication()
protected abstract B	getThisBuilder() Return the concrete implementation of the builder instance.
protected boolean	isRecursive() Return true to create the parent directories if they do not exist.
B	must(String key, boolean value) Set mandatory boolean option.
B	must(String key, double value) Set mandatory double option.
B	must(String key, float value) Set mandatory float option.
B	must(String key, int value) Set mandatory int option.
B	must(String key, String... values) Set a string array as mandatory option.
B	must(String key, String value) Set mandatory option to the Builder.
B	opt(String key, boolean value) Set optional boolean parameter for the Builder.
B	opt(String key, double value) Set optional double parameter for the Builder.
B	opt(String key, float value) Set optional float parameter for the Builder.
B	opt(String key, int value) Set optional int parameter for the Builder.
B	opt(String key, String... values) Set an array of string values as optional parameter for the Builder.
B	opt(String key, String value) Set optional Builder parameter.
B	overwrite(boolean overwrite) Set to true to overwrite the existing file.
B	permission(FsPermission perm) Set permission for the file.
B	progress(Progressable prog) Set the facility of reporting progress.
B	recursive() Create the parent directory if they do not exist.
B	replication(short replica) Set replication factor.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FSDataOutputStreamBuilder

protected FSDataOutputStreamBuilder(@Nonnull
                                    FileSystem fileSystem,
                                    @Nonnull
                                    Path p)

Constructor.

Method Detail

getThisBuilder

protected abstract B getThisBuilder()

Return the concrete implementation of the builder instance.

getFS

protected FileSystem getFS()

getPath

protected Path getPath()

getPermission

protected FsPermission getPermission()

permission

public B permission(@Nonnull
                    FsPermission perm)

Set permission for the file.

getBufferSize

protected int getBufferSize()

bufferSize

public B bufferSize(int bufSize)

Set the size of the buffer to be used.

getReplication

protected short getReplication()

replication

public B replication(short replica)

Set replication factor.

getBlockSize

protected long getBlockSize()

blockSize

public B blockSize(long blkSize)

Set block size.

isRecursive

protected boolean isRecursive()

Return true to create the parent directories if they do not exist.

recursive

public B recursive()

Create the parent directory if they do not exist.

getProgress

protected Progressable getProgress()

progress

public B progress(@Nonnull
                  Progressable prog)

Set the facility of reporting progress.

getFlags

protected EnumSet<CreateFlag> getFlags()

create

public B create()

Create an FSDataOutputStream at the specified path.

overwrite

public B overwrite(boolean overwrite)

Set to true to overwrite the existing file. Set it to false, an exception will be thrown when calling build() if the file exists.

append

public B append()

Append to an existing file (optional operation).

getChecksumOpt

protected org.apache.hadoop.fs.Options.ChecksumOpt getChecksumOpt()

checksumOpt

public B checksumOpt(@Nonnull
                     org.apache.hadoop.fs.Options.ChecksumOpt chksumOpt)

Set checksum opt.

opt

public B opt(@Nonnull
             String key,
             @Nonnull
             String value)

Set optional Builder parameter.

opt

public B opt(@Nonnull
             String key,
             boolean value)

Set optional boolean parameter for the Builder.

See Also:

opt(String, String)

opt

public B opt(@Nonnull
             String key,
             int value)

Set optional int parameter for the Builder.

See Also:

opt(String, String)

opt

public B opt(@Nonnull
             String key,
             float value)

Set optional float parameter for the Builder.

See Also:

opt(String, String)

opt

public B opt(@Nonnull
             String key,
             double value)

Set optional double parameter for the Builder.

See Also:

opt(String, String)

opt

public B opt(@Nonnull
             String key,
             @Nonnull
             String... values)

Set an array of string values as optional parameter for the Builder.

See Also:

opt(String, String)

must

public B must(@Nonnull
              String key,
              @Nonnull
              String value)

Set mandatory option to the Builder. If the option is not supported or unavailable on the FileSystem, the client should expect build() throws IllegalArgumentException.

must

public B must(@Nonnull
              String key,
              boolean value)

Set mandatory boolean option.

See Also:

must(String, String)

must

public B must(@Nonnull
              String key,
              int value)

Set mandatory int option.

See Also:

must(String, String)

must

public B must(@Nonnull
              String key,
              float value)

Set mandatory float option.

See Also:

must(String, String)

must

public B must(@Nonnull
              String key,
              double value)

Set mandatory double option.

See Also:

must(String, String)

must

public B must(@Nonnull
              String key,
              @Nonnull
              String... values)

Set a string array as mandatory option.

See Also:

must(String, String)

getOptions

protected Configuration getOptions()

getMandatoryKeys

protected Set<String> getMandatoryKeys()

Get all the keys that are set as mandatory keys.

build

public abstract S build()
                 throws IllegalArgumentException,
                        IOException

Create the FSDataOutputStream to write on the file system.

Throws:

IllegalArgumentException - if the parameters are not valid.

IOException - on errors when file system creates or appends the file.