org.apache.hadoop.hdfs.web

Class WebHdfsFileSystem.ReadRunner

java.lang.Object

org.apache.hadoop.hdfs.web.WebHdfsFileSystem.ReadRunner

Enclosing class:

WebHdfsFileSystem

protected class WebHdfsFileSystem.ReadRunner
extends Object

This class will allow retries to occur for both open and read operations. The first WebHdfsFileSystem#open creates a new WebHdfsInputStream object, which creates a new ReadRunner object that will be used to open a connection and read or seek into the input stream. ReadRunner is a subclass of the AbstractRunner class, which will run the ReadRunner#getUrl(), ReadRunner#connect(URL), and ReadRunner#getResponse methods within a retry loop, based on the configured retry policy. ReadRunner#connect will create a connection if one has not already been created. Otherwise, it will return the previously created connection object. This is necessary because a new connection should not be created for every read. Likewise, ReadRunner#getUrl will construct a new URL object only if the connection has not previously been established. Otherwise, it will return the previously created URL object. ReadRunner#getResponse will initialize the input stream if it has not already been initialized and read the requested data from the specified input stream.

Field Summary

Fields

Modifier and Type	Field and Description
protected ExcludeDatanodesParam	excludeDatanodes
protected HttpOpParam.Op	op

Method Summary

Methods

Modifier and Type	Method and Description
void	close()
protected HttpURLConnection	connect(URL url) Two-step requests redirected to a DN Create/Append: Step 1) Submit a Http request with neither auto-redirect nor data.
protected org.apache.hadoop.fs.FileEncryptionInfo	getFileEncryptionInfo()
protected InputStream	getInputStream()
protected URL	getUrl()
protected void	setInputStream(InputStream inStream)
protected void	updateURLParameters(Param<?,?>... p)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

op

protected final HttpOpParam.Op op

excludeDatanodes

protected ExcludeDatanodesParam excludeDatanodes

Method Detail

close

public void close()
           throws IOException

Throws:

IOException

getUrl

protected URL getUrl()
              throws IOException

Throws:

IOException

connect

protected HttpURLConnection connect(URL url)
                             throws IOException

Two-step requests redirected to a DN Create/Append: Step 1) Submit a Http request with neither auto-redirect nor data. Step 2) Submit another Http request with the URL from the Location header with data. The reason of having two-step create/append is for preventing clients to send out the data before the redirect. This issue is addressed by the "Expect: 100-continue" header in HTTP/1.1; see RFC 2616, Section 8.2.3. Unfortunately, there are software library bugs (e.g. Jetty 6 http server and Java 6 http client), which do not correctly implement "Expect: 100-continue". The two-step create/append is a temporary workaround for the software library bugs. Open/Checksum Also implements two-step connects for other operations redirected to a DN such as open and checksum

Throws:

IOException

getInputStream

protected InputStream getInputStream()

setInputStream

protected void setInputStream(InputStream inStream)

getFileEncryptionInfo

protected org.apache.hadoop.fs.FileEncryptionInfo getFileEncryptionInfo()

updateURLParameters

protected void updateURLParameters(Param<?,?>... p)