Uses of Class
org.apache.hadoop.fs.FileSystem

Packages that use FileSystem
org.apache.hadoop.contrib.index.lucene   
org.apache.hadoop.contrib.index.mapred   
org.apache.hadoop.examples.terasort This package consists of 3 map/reduce applications for Hadoop to compete in the annual terabyte sort competition. 
org.apache.hadoop.filecache   
org.apache.hadoop.fs An abstract file system API. 
org.apache.hadoop.fs.ftp   
org.apache.hadoop.fs.kfs A client for the Kosmos filesystem (KFS) 
org.apache.hadoop.fs.s3 A distributed, block-based implementation of FileSystem that uses Amazon S3 as a backing store. 
org.apache.hadoop.fs.s3native A distributed implementation of FileSystem for reading and writing files on Amazon S3
org.apache.hadoop.io Generic i/o code for use when reading and writing data to the network, to databases, and to files. 
org.apache.hadoop.mapred A software framework for easily writing applications which process vast amounts of data (multi-terabyte data-sets) parallelly on large clusters (thousands of nodes) built of commodity hardware in a reliable, fault-tolerant manner. 
org.apache.hadoop.mapred.lib Library of generally useful mappers, reducers, and partitioners. 
org.apache.hadoop.mapred.lib.db org.apache.hadoop.mapred.lib.db Package 
org.apache.hadoop.mapreduce.lib.input   
org.apache.hadoop.mapreduce.server.tasktracker   
org.apache.hadoop.mapreduce.split   
org.apache.hadoop.streaming Hadoop Streaming is a utility which allows users to create and run Map-Reduce jobs with any executables (e.g. 
 

Uses of FileSystem in org.apache.hadoop.contrib.index.lucene
 

Constructors in org.apache.hadoop.contrib.index.lucene with parameters of type FileSystem
FileSystemDirectory(FileSystem fs, Path directory, boolean create, Configuration conf)
          Constructor
ShardWriter(FileSystem fs, Shard shard, String tempDir, IndexUpdateConfiguration iconf)
          Constructor
 

Uses of FileSystem in org.apache.hadoop.contrib.index.mapred
 

Methods in org.apache.hadoop.contrib.index.mapred with parameters of type FileSystem
 RecordWriter<Shard,Text> IndexUpdateOutputFormat.getRecordWriter(FileSystem fs, JobConf job, String name, Progressable progress)
           
 

Uses of FileSystem in org.apache.hadoop.examples.terasort
 

Methods in org.apache.hadoop.examples.terasort with parameters of type FileSystem
 RecordWriter<Text,Text> TeraOutputFormat.getRecordWriter(FileSystem ignored, JobConf job, String name, Progressable progress)
           
 

Uses of FileSystem in org.apache.hadoop.filecache
 

Methods in org.apache.hadoop.filecache with parameters of type FileSystem
static void DistributedCache.addArchiveToClassPath(Path archive, Configuration conf, FileSystem fs)
          Add an archive path to the current set of classpath entries.
static void DistributedCache.addFileToClassPath(Path file, Configuration conf, FileSystem fs)
          Add a file path to the current set of classpath entries.
 

Uses of FileSystem in org.apache.hadoop.fs
 

Subclasses of FileSystem in org.apache.hadoop.fs
 class ChecksumFileSystem
          Abstract Checksumed FileSystem.
 class FilterFileSystem
          A FilterFileSystem contains some other file system, which it uses as its basic file system, possibly transforming the data along the way or providing additional functionality.
 class HarFileSystem
          This is an implementation of the Hadoop Archive Filesystem.
 class InMemoryFileSystem
          Deprecated. 
 class LocalFileSystem
          Implement the FileSystem API for the checksumed local filesystem.
 class RawLocalFileSystem
          Implement the FileSystem API for the raw local filesystem.
 

Fields in org.apache.hadoop.fs declared as FileSystem
protected  FileSystem FilterFileSystem.fs
           
 

Methods in org.apache.hadoop.fs that return FileSystem
static FileSystem FileSystem.get(Configuration conf)
          Returns the configured filesystem implementation.
static FileSystem FileSystem.get(URI uri, Configuration conf)
          Returns the FileSystem for this URI's scheme and authority.
static FileSystem FileSystem.get(URI uri, Configuration conf, String user)
           
 FileSystem Path.getFileSystem(Configuration conf)
          Return the FileSystem that owns this Path.
protected  FileSystem FsShell.getFS()
           
static FileSystem FileSystem.getNamed(String name, Configuration conf)
          Deprecated. call #get(URI,Configuration) instead.
 FileSystem LocalFileSystem.getRaw()
           
 FileSystem ChecksumFileSystem.getRawFileSystem()
          get the raw file system
 

Methods in org.apache.hadoop.fs with parameters of type FileSystem
static void FileSystem.addFileSystemForTesting(URI uri, Configuration conf, FileSystem fs)
          This method adds a file system for testing so that we can find it later.
static boolean FileUtil.copy(File src, FileSystem dstFS, Path dst, boolean deleteSource, Configuration conf)
          Copy local files to a FileSystem.
static boolean FileUtil.copy(FileSystem srcFS, Path[] srcs, FileSystem dstFS, Path dst, boolean deleteSource, boolean overwrite, Configuration conf)
           
static boolean FileUtil.copy(FileSystem srcFS, Path src, File dst, boolean deleteSource, Configuration conf)
          Copy FileSystem files to local files.
static boolean FileUtil.copy(FileSystem srcFS, Path src, FileSystem dstFS, Path dst, boolean deleteSource, boolean overwrite, Configuration conf)
          Copy files between FileSystems.
static boolean FileUtil.copy(FileSystem srcFS, Path src, FileSystem dstFS, Path dst, boolean deleteSource, Configuration conf)
          Copy files between FileSystems.
static boolean FileUtil.copyMerge(FileSystem srcFS, Path srcDir, FileSystem dstFS, Path dstFile, boolean deleteSource, Configuration conf, String addString)
          Copy all files in a directory to one output file (merge).
static FSDataOutputStream FileSystem.create(FileSystem fs, Path file, FsPermission permission)
          create a file with the provided permission The permission of the file is set to be the provided permission as in setPermission, not permission&~umask It is implemented using two RPCs.
static void FileUtil.fullyDelete(FileSystem fs, Path dir)
          Deprecated. Use delete(Path, boolean)
 Path Path.makeQualified(FileSystem fs)
          Returns a qualified path object.
static boolean FileSystem.mkdirs(FileSystem fs, Path dir, FsPermission permission)
          create a directory with the provided permission The permission of the directory is set to be the provided permission as in setPermission, not permission&~umask
 

Method parameters in org.apache.hadoop.fs with type arguments of type FileSystem
static FileSystem.Statistics FileSystem.getStatistics(String scheme, Class<? extends FileSystem> cls)
          Get the statistics for a particular file system
 

Constructors in org.apache.hadoop.fs with parameters of type FileSystem
ChecksumFileSystem(FileSystem fs)
           
FilterFileSystem(FileSystem fs)
           
HarFileSystem(FileSystem fs)
          Constructor to create a HarFileSystem with an underlying filesystem.
LocalFileSystem(FileSystem rawLocalFileSystem)
           
Trash(FileSystem fs, Configuration conf)
          Construct a trash can accessor for the FileSystem provided.
 

Uses of FileSystem in org.apache.hadoop.fs.ftp
 

Subclasses of FileSystem in org.apache.hadoop.fs.ftp
 class FTPFileSystem
           A FileSystem backed by an FTP client provided by Apache Commons Net.
 

Uses of FileSystem in org.apache.hadoop.fs.kfs
 

Subclasses of FileSystem in org.apache.hadoop.fs.kfs
 class KosmosFileSystem
          A FileSystem backed by KFS.
 

Uses of FileSystem in org.apache.hadoop.fs.s3
 

Subclasses of FileSystem in org.apache.hadoop.fs.s3
 class S3FileSystem
           A block-based FileSystem backed by Amazon S3.
 

Uses of FileSystem in org.apache.hadoop.fs.s3native
 

Subclasses of FileSystem in org.apache.hadoop.fs.s3native
 class NativeS3FileSystem
           A FileSystem for reading and writing files stored on Amazon S3.
 

Uses of FileSystem in org.apache.hadoop.io
 

Methods in org.apache.hadoop.io with parameters of type FileSystem
protected  SequenceFile.Reader MapFile.Reader.createDataFileReader(FileSystem fs, Path dataFile, Configuration conf)
          Override this method to specialize the type of SequenceFile.Reader returned.
static SequenceFile.Writer SequenceFile.createWriter(FileSystem fs, Configuration conf, Path name, Class keyClass, Class valClass)
          Construct the preferred type of SequenceFile Writer.
static SequenceFile.Writer SequenceFile.createWriter(FileSystem fs, Configuration conf, Path name, Class keyClass, Class valClass, int bufferSize, short replication, long blockSize, boolean createParent, SequenceFile.CompressionType compressionType, CompressionCodec codec, SequenceFile.Metadata metadata)
          Construct the preferred type of SequenceFile Writer.
static SequenceFile.Writer SequenceFile.createWriter(FileSystem fs, Configuration conf, Path name, Class keyClass, Class valClass, int bufferSize, short replication, long blockSize, SequenceFile.CompressionType compressionType, CompressionCodec codec, Progressable progress, SequenceFile.Metadata metadata)
          Construct the preferred type of SequenceFile Writer.
static SequenceFile.Writer SequenceFile.createWriter(FileSystem fs, Configuration conf, Path name, Class keyClass, Class valClass, SequenceFile.CompressionType compressionType)
          Construct the preferred type of SequenceFile Writer.
static SequenceFile.Writer SequenceFile.createWriter(FileSystem fs, Configuration conf, Path name, Class keyClass, Class valClass, SequenceFile.CompressionType compressionType, CompressionCodec codec)
          Construct the preferred type of SequenceFile Writer.
static SequenceFile.Writer SequenceFile.createWriter(FileSystem fs, Configuration conf, Path name, Class keyClass, Class valClass, SequenceFile.CompressionType compressionType, CompressionCodec codec, Progressable progress)
          Construct the preferred type of SequenceFile Writer.
static SequenceFile.Writer SequenceFile.createWriter(FileSystem fs, Configuration conf, Path name, Class keyClass, Class valClass, SequenceFile.CompressionType compressionType, CompressionCodec codec, Progressable progress, SequenceFile.Metadata metadata)
          Construct the preferred type of SequenceFile Writer.
static SequenceFile.Writer SequenceFile.createWriter(FileSystem fs, Configuration conf, Path name, Class keyClass, Class valClass, SequenceFile.CompressionType compressionType, Progressable progress)
          Construct the preferred type of SequenceFile Writer.
static void MapFile.delete(FileSystem fs, String name)
          Deletes the named map file.
static void BloomMapFile.delete(FileSystem fs, String name)
           
static long MapFile.fix(FileSystem fs, Path dir, Class<? extends Writable> keyClass, Class<? extends Writable> valueClass, boolean dryrun, Configuration conf)
          This method attempts to fix a corrupt MapFile by re-creating its index.
protected  void MapFile.Reader.open(FileSystem fs, String dirName, WritableComparator comparator, Configuration conf)
           
protected  FSDataInputStream SequenceFile.Reader.openFile(FileSystem fs, Path file, int bufferSize, long length)
          Override this method to specialize the type of FSDataInputStream returned.
static void MapFile.rename(FileSystem fs, String oldName, String newName)
          Renames an existing map directory.
 

Constructors in org.apache.hadoop.io with parameters of type FileSystem
ArrayFile.Reader(FileSystem fs, String file, Configuration conf)
          Construct an array reader for the named file.
ArrayFile.Writer(Configuration conf, FileSystem fs, String file, Class<? extends Writable> valClass)
          Create the named file for values of the named class.
ArrayFile.Writer(Configuration conf, FileSystem fs, String file, Class<? extends Writable> valClass, SequenceFile.CompressionType compress, Progressable progress)
          Create the named file for values of the named class.
BloomMapFile.Reader(FileSystem fs, String dirName, Configuration conf)
           
BloomMapFile.Reader(FileSystem fs, String dirName, WritableComparator comparator, Configuration conf)
           
BloomMapFile.Reader(FileSystem fs, String dirName, WritableComparator comparator, Configuration conf, boolean open)
           
BloomMapFile.Writer(Configuration conf, FileSystem fs, String dirName, Class<? extends WritableComparable> keyClass, Class valClass)
           
BloomMapFile.Writer(Configuration conf, FileSystem fs, String dirName, Class<? extends WritableComparable> keyClass, Class<? extends Writable> valClass, SequenceFile.CompressionType compress, CompressionCodec codec, Progressable progress)
           
BloomMapFile.Writer(Configuration conf, FileSystem fs, String dirName, Class<? extends WritableComparable> keyClass, Class valClass, SequenceFile.CompressionType compress)
           
BloomMapFile.Writer(Configuration conf, FileSystem fs, String dirName, Class<? extends WritableComparable> keyClass, Class valClass, SequenceFile.CompressionType compress, Progressable progress)
           
BloomMapFile.Writer(Configuration conf, FileSystem fs, String dirName, WritableComparator comparator, Class valClass)
           
BloomMapFile.Writer(Configuration conf, FileSystem fs, String dirName, WritableComparator comparator, Class valClass, SequenceFile.CompressionType compress)
           
BloomMapFile.Writer(Configuration conf, FileSystem fs, String dirName, WritableComparator comparator, Class valClass, SequenceFile.CompressionType compress, CompressionCodec codec, Progressable progress)
           
BloomMapFile.Writer(Configuration conf, FileSystem fs, String dirName, WritableComparator comparator, Class valClass, SequenceFile.CompressionType compress, Progressable progress)
           
MapFile.Reader(FileSystem fs, String dirName, Configuration conf)
          Construct a map reader for the named map.
MapFile.Reader(FileSystem fs, String dirName, WritableComparator comparator, Configuration conf)
          Construct a map reader for the named map using the named comparator.
MapFile.Reader(FileSystem fs, String dirName, WritableComparator comparator, Configuration conf, boolean open)
          Hook to allow subclasses to defer opening streams until further initialization is complete.
MapFile.Writer(Configuration conf, FileSystem fs, String dirName, Class<? extends WritableComparable> keyClass, Class valClass)
          Create the named map for keys of the named class.
MapFile.Writer(Configuration conf, FileSystem fs, String dirName, Class<? extends WritableComparable> keyClass, Class valClass, SequenceFile.CompressionType compress)
          Create the named map for keys of the named class.
MapFile.Writer(Configuration conf, FileSystem fs, String dirName, Class<? extends WritableComparable> keyClass, Class valClass, SequenceFile.CompressionType compress, CompressionCodec codec, Progressable progress)
          Create the named map for keys of the named class.
MapFile.Writer(Configuration conf, FileSystem fs, String dirName, Class<? extends WritableComparable> keyClass, Class valClass, SequenceFile.CompressionType compress, Progressable progress)
          Create the named map for keys of the named class.
MapFile.Writer(Configuration conf, FileSystem fs, String dirName, WritableComparator comparator, Class valClass)
          Create the named map using the named key comparator.
MapFile.Writer(Configuration conf, FileSystem fs, String dirName, WritableComparator comparator, Class valClass, SequenceFile.CompressionType compress)
          Create the named map using the named key comparator.
MapFile.Writer(Configuration conf, FileSystem fs, String dirName, WritableComparator comparator, Class valClass, SequenceFile.CompressionType compress, CompressionCodec codec, Progressable progress)
          Create the named map using the named key comparator.
MapFile.Writer(Configuration conf, FileSystem fs, String dirName, WritableComparator comparator, Class valClass, SequenceFile.CompressionType compress, Progressable progress)
          Create the named map using the named key comparator.
SequenceFile.Reader(FileSystem fs, Path file, Configuration conf)
          Open the named file.
SequenceFile.Sorter(FileSystem fs, Class<? extends WritableComparable> keyClass, Class valClass, Configuration conf)
          Sort and merge files containing the named classes.
SequenceFile.Sorter(FileSystem fs, RawComparator comparator, Class keyClass, Class valClass, Configuration conf)
          Sort and merge using an arbitrary RawComparator.
SequenceFile.Writer(FileSystem fs, Configuration conf, Path name, Class keyClass, Class valClass)
          Create the named file.
SequenceFile.Writer(FileSystem fs, Configuration conf, Path name, Class keyClass, Class valClass, int bufferSize, short replication, long blockSize, Progressable progress, SequenceFile.Metadata metadata)
          Create the named file with write-progress reporter.
SequenceFile.Writer(FileSystem fs, Configuration conf, Path name, Class keyClass, Class valClass, Progressable progress, SequenceFile.Metadata metadata)
          Create the named file with write-progress reporter.
SetFile.Reader(FileSystem fs, String dirName, Configuration conf)
          Construct a set reader for the named set.
SetFile.Reader(FileSystem fs, String dirName, WritableComparator comparator, Configuration conf)
          Construct a set reader for the named set using the named comparator.
SetFile.Writer(Configuration conf, FileSystem fs, String dirName, Class<? extends WritableComparable> keyClass, SequenceFile.CompressionType compress)
          Create a set naming the element class and compression type.
SetFile.Writer(Configuration conf, FileSystem fs, String dirName, WritableComparator comparator, SequenceFile.CompressionType compress)
          Create a set naming the element comparator and compression type.
SetFile.Writer(FileSystem fs, String dirName, Class<? extends WritableComparable> keyClass)
          Deprecated. pass a Configuration too
 

Uses of FileSystem in org.apache.hadoop.mapred
 

Fields in org.apache.hadoop.mapred declared as FileSystem
protected static FileSystem JobHistory.DONEDIR_FS
           
 

Methods in org.apache.hadoop.mapred that return FileSystem
 FileSystem JobClient.getFs()
          Get a filesystem handle.
 

Methods in org.apache.hadoop.mapred with parameters of type FileSystem
 void FileOutputFormat.checkOutputSpecs(FileSystem ignored, JobConf job)
           
 void SequenceFileAsBinaryOutputFormat.checkOutputSpecs(FileSystem ignored, JobConf job)
           
 void OutputFormat.checkOutputSpecs(FileSystem ignored, JobConf job)
          Check for validity of the output-specification for the job.
static MapFile.Reader[] MapFileOutputFormat.getReaders(FileSystem ignored, Path dir, Configuration conf)
          Open the output generated by this format.
 RecordWriter<WritableComparable,Writable> MapFileOutputFormat.getRecordWriter(FileSystem ignored, JobConf job, String name, Progressable progress)
           
abstract  RecordWriter<K,V> FileOutputFormat.getRecordWriter(FileSystem ignored, JobConf job, String name, Progressable progress)
           
 RecordWriter<BytesWritable,BytesWritable> SequenceFileAsBinaryOutputFormat.getRecordWriter(FileSystem ignored, JobConf job, String name, Progressable progress)
           
 RecordWriter<K,V> OutputFormat.getRecordWriter(FileSystem ignored, JobConf job, String name, Progressable progress)
          Get the RecordWriter for the given job.
 RecordWriter<K,V> SequenceFileOutputFormat.getRecordWriter(FileSystem ignored, JobConf job, String name, Progressable progress)
           
 RecordWriter<K,V> TextOutputFormat.getRecordWriter(FileSystem ignored, JobConf job, String name, Progressable progress)
           
static boolean JobClient.isJobDirValid(Path jobDirPath, FileSystem fs)
          Checks if the job directory is clean and has all the required components for (re) starting the job
protected  boolean TextInputFormat.isSplitable(FileSystem fs, Path file)
           
protected  boolean FileInputFormat.isSplitable(FileSystem fs, Path filename)
          Is the given filename splitable? Usually, true, but if the file is stream compressed, it will not be.
protected  boolean KeyValueTextInputFormat.isSplitable(FileSystem fs, Path file)
           
static void JobHistory.parseHistoryFromFS(String path, JobHistory.Listener l, FileSystem fs)
          Parses history file and invokes Listener.handle() for each line of history.
static void DefaultJobHistoryParser.parseJobTasks(String jobHistoryFile, JobHistory.JobInfo job, FileSystem fs)
          Populates a JobInfo object from the job's history log file.
protected static String TaskController.writeCommand(String cmdLine, FileSystem fs, Path commandFile)
           
 

Constructors in org.apache.hadoop.mapred with parameters of type FileSystem
HDFSMonitorThread(Configuration conf, JobTracker jt, FileSystem fs)
           
 

Uses of FileSystem in org.apache.hadoop.mapred.lib
 

Fields in org.apache.hadoop.mapred.lib declared as FileSystem
protected  FileSystem CombineFileRecordReader.fs
           
 

Methods in org.apache.hadoop.mapred.lib with parameters of type FileSystem
 void NullOutputFormat.checkOutputSpecs(FileSystem ignored, JobConf job)
           
protected  RecordWriter<K,V> MultipleTextOutputFormat.getBaseRecordWriter(FileSystem fs, JobConf job, String name, Progressable arg3)
           
protected  RecordWriter<K,V> MultipleSequenceFileOutputFormat.getBaseRecordWriter(FileSystem fs, JobConf job, String name, Progressable arg3)
           
protected abstract  RecordWriter<K,V> MultipleOutputFormat.getBaseRecordWriter(FileSystem fs, JobConf job, String name, Progressable arg3)
           
 RecordWriter<K,V> MultipleOutputFormat.getRecordWriter(FileSystem fs, JobConf job, String name, Progressable arg3)
          Create a composite record writer that can write key/value data to different output files
 RecordWriter<K,V> NullOutputFormat.getRecordWriter(FileSystem ignored, JobConf job, String name, Progressable progress)
           
 

Uses of FileSystem in org.apache.hadoop.mapred.lib.db
 

Methods in org.apache.hadoop.mapred.lib.db with parameters of type FileSystem
 void DBOutputFormat.checkOutputSpecs(FileSystem filesystem, JobConf job)
          Check for validity of the output-specification for the job.
 RecordWriter<K,V> DBOutputFormat.getRecordWriter(FileSystem filesystem, JobConf job, String name, Progressable progress)
          Get the RecordWriter for the given job.
 

Uses of FileSystem in org.apache.hadoop.mapreduce.lib.input
 

Fields in org.apache.hadoop.mapreduce.lib.input declared as FileSystem
protected  FileSystem CombineFileRecordReader.fs
           
 

Methods in org.apache.hadoop.mapreduce.lib.input with parameters of type FileSystem
protected  BlockLocation[] CombineFileInputFormat.getFileBlockLocations(FileSystem fs, FileStatus stat)
           
 

Uses of FileSystem in org.apache.hadoop.mapreduce.server.tasktracker
 

Constructors in org.apache.hadoop.mapreduce.server.tasktracker with parameters of type FileSystem
Localizer(FileSystem fileSys, String[] lDirs)
          Create a Localizer instance
 

Uses of FileSystem in org.apache.hadoop.mapreduce.split
 

Methods in org.apache.hadoop.mapreduce.split with parameters of type FileSystem
static void JobSplitWriter.createSplitFiles(Path jobSubmitDir, Configuration conf, FileSystem fs, InputSplit[] splits)
           
static
<T extends InputSplit>
void
JobSplitWriter.createSplitFiles(Path jobSubmitDir, Configuration conf, FileSystem fs, List<InputSplit> splits)
           
static
<T extends InputSplit>
void
JobSplitWriter.createSplitFiles(Path jobSubmitDir, Configuration conf, FileSystem fs, T[] splits)
           
static JobSplit.TaskSplitMetaInfo[] SplitMetaInfoReader.readSplitMetaInfo(JobID jobId, FileSystem fs, Configuration conf, Path jobSubmitDir)
           
 

Uses of FileSystem in org.apache.hadoop.streaming
 

Constructors in org.apache.hadoop.streaming with parameters of type FileSystem
StreamBaseRecordReader(FSDataInputStream in, FileSplit split, Reporter reporter, JobConf job, FileSystem fs)
           
StreamXmlRecordReader(FSDataInputStream in, FileSplit split, Reporter reporter, JobConf job, FileSystem fs)
           
 



Copyright © 2009 The Apache Software Foundation