@InterfaceAudience.Public @InterfaceStability.Stable public class Configuration extends Object implements Iterable<Map.Entry<String,String>>, Writable
Configurations are specified by resources. A resource contains a set of
name/value pairs as XML data. Each resource is named by either a
String
or by a Path
. If named by a String
,
then the classpath is examined for a file with that name. If named by a
Path
, then the local filesystem is examined directly, without
referring to the classpath.
Unless explicitly turned off, Hadoop by default specifies two resources, loaded in-order from the classpath:
Configuration parameters may be declared final.
Once a resource declares a value final, no subsequently-loaded
resource can alter that value.
For example, one might define a final parameter with:
<property>
<name>dfs.client.buffer.dir</name>
<value>/tmp/hadoop/dfs/client</value>
<final>true</final>
</property>
Administrators typically define parameters as final in
core-site.xml for values that user applications may not alter.
Value strings are first processed for variable expansion. The available properties are:
System.getProperties()
.For example, if a configuration resource contains the following property
definitions:
<property>
<name>basedir</name>
<value>/user/${user.name}</value>
</property>
<property>
<name>tempdir</name>
<value>${basedir}/tmp</value>
</property>
When conf.get("tempdir") is called, then ${basedir}
will be resolved to another property in this Configuration, while
${user.name} would then ordinarily be resolved to the value
of the System property with that name.
Constructor and Description |
---|
Configuration()
A new configuration.
|
Configuration(boolean loadDefaults)
A new configuration where the behavior of reading from the default
resources can be turned off.
|
Configuration(Configuration other)
A new configuration with the same settings cloned from another.
|
Modifier and Type | Method and Description |
---|---|
static void |
addDefaultResource(String name)
Add a default resource.
|
static void |
addDeprecation(String key,
String newKey)
Adds the deprecated key to the deprecation map when no custom message
is provided.
|
static void |
addDeprecation(String key,
String[] newKeys)
Deprecated.
use
addDeprecation(String key, String newKey) instead |
static void |
addDeprecation(String key,
String[] newKeys,
String customMessage)
Deprecated.
|
static void |
addDeprecation(String key,
String newKey,
String customMessage)
Adds the deprecated key to the deprecation map.
|
void |
addResource(InputStream in)
Add a configuration resource.
|
void |
addResource(InputStream in,
String name)
Add a configuration resource.
|
void |
addResource(Path file)
Add a configuration resource.
|
void |
addResource(String name)
Add a configuration resource.
|
void |
addResource(URL url)
Add a configuration resource.
|
void |
clear()
Clears all keys from the configuration.
|
static void |
dumpConfiguration(Configuration config,
Writer out)
Writes out all the parameters and their properties (final and resource) to
the given
Writer
The format of the output would be
{ "properties" : [ {key1,value1,key1.isFinal,key1.resource}, {key2,value2,
key2.isFinal,key2.resource}... |
String |
get(String name)
Get the value of the
name property, null if
no such property exists. |
String |
get(String name,
String defaultValue)
Get the value of the
name . |
boolean |
getBoolean(String name,
boolean defaultValue)
Get the value of the
name property as a boolean . |
Class<?> |
getClass(String name,
Class<?> defaultValue)
Get the value of the
name property as a Class . |
<U> Class<? extends U> |
getClass(String name,
Class<? extends U> defaultValue,
Class<U> xface)
Get the value of the
name property as a Class
implementing the interface specified by xface . |
Class<?> |
getClassByName(String name)
Load a class by name.
|
Class<?> |
getClassByNameOrNull(String name)
Load a class by name, returning null rather than throwing an exception
if it couldn't be loaded.
|
Class<?>[] |
getClasses(String name,
Class<?>... defaultValue)
Get the value of the
name property
as an array of Class . |
ClassLoader |
getClassLoader()
Get the
ClassLoader for this job. |
InputStream |
getConfResourceAsInputStream(String name)
Get an input stream attached to the configuration resource with the
given
name . |
Reader |
getConfResourceAsReader(String name)
Get a
Reader attached to the configuration resource with the
given name . |
<T extends Enum<T>> |
getEnum(String name,
T defaultValue)
Return value matching this enumerated type.
|
File |
getFile(String dirsProp,
String path)
Get a local file name under a directory named in dirsProp with
the given path.
|
Set<String> |
getFinalParameters()
Get the set of parameters marked final.
|
float |
getFloat(String name,
float defaultValue)
Get the value of the
name property as a float . |
<U> List<U> |
getInstances(String name,
Class<U> xface)
Get the value of the
name property as a List
of objects implementing the interface specified by xface . |
int |
getInt(String name,
int defaultValue)
Get the value of the
name property as an int . |
Path |
getLocalPath(String dirsProp,
String path)
Get a local file under a directory named by dirsProp with
the given path.
|
long |
getLong(String name,
long defaultValue)
Get the value of the
name property as a long . |
long |
getLongBytes(String name,
long defaultValue)
Get the value of the
name property as a long or
human readable format. |
Pattern |
getPattern(String name,
Pattern defaultValue)
Get the value of the
name property as a Pattern . |
String[] |
getPropertySources(String name)
Gets information about why a property was set.
|
protected Properties |
getProps() |
org.apache.hadoop.conf.Configuration.IntegerRanges |
getRange(String name,
String defaultValue)
Parse the given attribute as a set of integer ranges
|
String |
getRaw(String name)
Get the value of the
name property, without doing
variable expansion.If the key is
deprecated, it returns the value of the first key which replaces
the deprecated key and is not null. |
URL |
getResource(String name)
Get the
URL for the named resource. |
InetSocketAddress |
getSocketAddr(String name,
String defaultAddress,
int defaultPort)
Get the socket address for
name property as a
InetSocketAddress . |
Collection<String> |
getStringCollection(String name)
Get the comma delimited values of the
name property as
a collection of String s. |
String[] |
getStrings(String name)
Get the comma delimited values of the
name property as
an array of String s. |
String[] |
getStrings(String name,
String... defaultValue)
Get the comma delimited values of the
name property as
an array of String s. |
String |
getTrimmed(String name)
Get the value of the
name property as a trimmed String ,
null if no such property exists. |
Collection<String> |
getTrimmedStringCollection(String name)
Get the comma delimited values of the
name property as
a collection of String s, trimmed of the leading and trailing whitespace. |
String[] |
getTrimmedStrings(String name)
Get the comma delimited values of the
name property as
an array of String s, trimmed of the leading and trailing whitespace. |
String[] |
getTrimmedStrings(String name,
String... defaultValue)
Get the comma delimited values of the
name property as
an array of String s, trimmed of the leading and trailing whitespace. |
Map<String,String> |
getValByRegex(String regex)
get keys matching the the regex
|
static boolean |
isDeprecated(String key)
checks whether the given
key is deprecated. |
Iterator<Map.Entry<String,String>> |
iterator()
Get an
Iterator to go through the list of String
key-value pairs in the configuration. |
static void |
main(String[] args)
For debugging.
|
void |
readFields(DataInput in)
Deserialize the fields of this object from
in . |
void |
reloadConfiguration()
Reload configuration from previously added resources.
|
void |
set(String name,
String value)
Set the
value of the name property. |
void |
set(String name,
String value,
String source)
Set the
value of the name property. |
void |
setBoolean(String name,
boolean value)
Set the value of the
name property to a boolean . |
void |
setBooleanIfUnset(String name,
boolean value)
Set the given property, if it is currently unset.
|
void |
setClass(String name,
Class<?> theClass,
Class<?> xface)
Set the value of the
name property to the name of a
theClass implementing the given interface xface . |
void |
setClassLoader(ClassLoader classLoader)
Set the class loader that will be used to load the various objects.
|
<T extends Enum<T>> |
setEnum(String name,
T value)
Set the value of the
name property to the given type. |
void |
setFloat(String name,
float value)
Set the value of the
name property to a float . |
void |
setIfUnset(String name,
String value)
Sets a property if it is currently unset.
|
void |
setInt(String name,
int value)
Set the value of the
name property to an int . |
void |
setLong(String name,
long value)
Set the value of the
name property to a long . |
void |
setPattern(String name,
Pattern pattern)
Set the given property to
Pattern . |
void |
setQuietMode(boolean quietmode)
Set the quietness-mode.
|
void |
setSocketAddr(String name,
InetSocketAddress addr)
Set the socket address for the
name property as
a host:port . |
void |
setStrings(String name,
String... values)
Set the array of string values for the
name property as
as comma delimited values. |
int |
size()
Return the number of keys in the configuration.
|
String |
toString() |
void |
unset(String name)
Unset a previously set property.
|
InetSocketAddress |
updateConnectAddr(String name,
InetSocketAddress addr)
Set the socket address a client can use to connect for the
name property as a host:port . |
void |
write(DataOutput out)
Serialize the fields of this object to
out . |
void |
writeXml(OutputStream out)
Write out the non-default properties in this configuration to the given
OutputStream . |
void |
writeXml(Writer out)
Write out the non-default properties in this configuration to the given
Writer . |
public Configuration()
public Configuration(boolean loadDefaults)
loadDefaults
is false, the new instance
will not load resources from the default files.loadDefaults
- specifies whether to load from the default filespublic Configuration(Configuration other)
other
- the configuration from which to clone settings.@Deprecated public static void addDeprecation(String key, String[] newKeys, String customMessage)
addDeprecation(String key, String newKey,
String customMessage)
insteadkey
- newKeys
- customMessage
- public static void addDeprecation(String key, String newKey, String customMessage)
key
- newKey
- customMessage
- @Deprecated public static void addDeprecation(String key, String[] newKeys)
addDeprecation(String key, String newKey)
insteadkey
- Key that is to be deprecatednewKeys
- list of keys that take up the values of deprecated keypublic static void addDeprecation(String key, String newKey)
key
- Key that is to be deprecatednewKey
- key that takes up the value of deprecated keypublic static boolean isDeprecated(String key)
key
is deprecated.key
- the parameter which is to be checked for deprecationtrue
if the key is deprecated and
false
otherwise.public static void addDefaultResource(String name)
name
- file name. File should be present in the classpath.public void addResource(String name)
name
- resource to be added, the classpath is examined for a file
with that name.public void addResource(URL url)
url
- url of the resource to be added, the local filesystem is
examined directly to find the resource, without referring to
the classpath.public void addResource(Path file)
file
- file-path of resource to be added, the local filesystem is
examined directly to find the resource, without referring to
the classpath.public void addResource(InputStream in)
in
- InputStream to deserialize the object from. In will be read from
when a get or set is called next. After it is read the stream will be
closed.public void addResource(InputStream in, String name)
in
- InputStream to deserialize the object from.name
- the name of the resource because InputStream.toString is not
very descriptive some times.public void reloadConfiguration()
public String get(String name)
name
property, null
if
no such property exists. If the key is deprecated, it returns the value of
the first key which replaces the deprecated key and is not null
Values are processed for variable expansion
before being returned.name
- the property name.name
or its replacing property,
or null if no such property exists.public String getTrimmed(String name)
name
property as a trimmed String
,
null
if no such property exists.
If the key is deprecated, it returns the value of
the first key which replaces the deprecated key and is not null
Values are processed for variable expansion
before being returned.name
- the property name.name
or its replacing property,
or null if no such property exists.public String getRaw(String name)
name
property, without doing
variable expansion.If the key is
deprecated, it returns the value of the first key which replaces
the deprecated key and is not null.name
- the property name.name
property or
its replacing property and null if no such property exists.public void set(String name, String value)
value
of the name
property. If
name
is deprecated or there is a deprecated name associated to it,
it sets the value to both names.name
- property name.value
- property value.public void set(String name, String value, String source)
value
of the name
property. If
name
is deprecated or there is a deprecated name associated to it,
it sets the value to both names.name
- property name.value
- property value.source
- the place that this configuration value came from
(For debugging).public void setIfUnset(String name, String value)
name
- the property namevalue
- the new valuepublic String get(String name, String defaultValue)
name
. If the key is deprecated,
it returns the value of the first key which replaces the deprecated key
and is not null.
If no such property exists,
then defaultValue
is returned.name
- property name.defaultValue
- default value.defaultValue
if the property
doesn't exist.public int getInt(String name, int defaultValue)
name
property as an int
.
If no such property exists, the provided default value is returned,
or if the specified value is not a valid int
,
then an error is thrown.name
- property name.defaultValue
- default value.int
,
or defaultValue
.NumberFormatException
- when the value is invalidpublic void setInt(String name, int value)
name
property to an int
.name
- property name.value
- int
value of the property.public long getLong(String name, long defaultValue)
name
property as a long
.
If no such property exists, the provided default value is returned,
or if the specified value is not a valid long
,
then an error is thrown.name
- property name.defaultValue
- default value.long
,
or defaultValue
.NumberFormatException
- when the value is invalidpublic long getLongBytes(String name, long defaultValue)
name
property as a long
or
human readable format. If no such property exists, the provided default
value is returned, or if the specified value is not a valid
long
or human readable format, then an error is thrown. You
can use the following suffix (case insensitive): k(kilo), m(mega), g(giga),
t(tera), p(peta), e(exa)name
- property name.defaultValue
- default value.long
,
or defaultValue
.NumberFormatException
- when the value is invalidpublic void setLong(String name, long value)
name
property to a long
.name
- property name.value
- long
value of the property.public float getFloat(String name, float defaultValue)
name
property as a float
.
If no such property exists, the provided default value is returned,
or if the specified value is not a valid float
,
then an error is thrown.name
- property name.defaultValue
- default value.float
,
or defaultValue
.NumberFormatException
- when the value is invalidpublic void setFloat(String name, float value)
name
property to a float
.name
- property name.value
- property value.public boolean getBoolean(String name, boolean defaultValue)
name
property as a boolean
.
If no such property is specified, or if the specified value is not a valid
boolean
, then defaultValue
is returned.name
- property name.defaultValue
- default value.boolean
,
or defaultValue
.public void setBoolean(String name, boolean value)
name
property to a boolean
.name
- property name.value
- boolean
value of the property.public void setBooleanIfUnset(String name, boolean value)
name
- property namevalue
- new valuepublic <T extends Enum<T>> void setEnum(String name, T value)
name
property to the given type. This
is equivalent to set(<name>, value.toString())
.name
- property namevalue
- new valuepublic <T extends Enum<T>> T getEnum(String name, T defaultValue)
name
- Property namedefaultValue
- Value returned if no mapping existsIllegalArgumentException
- If mapping is illegal for the type
providedpublic Pattern getPattern(String name, Pattern defaultValue)
name
property as a Pattern
.
If no such property is specified, or if the specified value is not a valid
Pattern
, then DefaultValue
is returned.name
- property namedefaultValue
- default valuepublic void setPattern(String name, Pattern pattern)
Pattern
.
If the pattern is passed as null, sets the empty pattern which results in
further calls to getPattern(...) returning the default value.name
- property namepattern
- new value@InterfaceStability.Unstable public String[] getPropertySources(String name)
name
- - The property name to get the source of.public org.apache.hadoop.conf.Configuration.IntegerRanges getRange(String name, String defaultValue)
name
- the attribute namedefaultValue
- the default value if it is not setpublic Collection<String> getStringCollection(String name)
name
property as
a collection of String
s.
If no such property is specified then empty collection is returned.
This is an optimized version of getStrings(String)
name
- property name.String
s.public String[] getStrings(String name)
name
property as
an array of String
s.
If no such property is specified then null
is returned.name
- property name.String
s,
or null
.public String[] getStrings(String name, String... defaultValue)
name
property as
an array of String
s.
If no such property is specified then default value is returned.name
- property name.defaultValue
- The default valueString
s,
or default value.public Collection<String> getTrimmedStringCollection(String name)
name
property as
a collection of String
s, trimmed of the leading and trailing whitespace.
If no such property is specified then empty Collection
is returned.name
- property name.String
s, or empty Collection
public String[] getTrimmedStrings(String name)
name
property as
an array of String
s, trimmed of the leading and trailing whitespace.
If no such property is specified then an empty array is returned.name
- property name.String
s,
or empty array.public String[] getTrimmedStrings(String name, String... defaultValue)
name
property as
an array of String
s, trimmed of the leading and trailing whitespace.
If no such property is specified then default value is returned.name
- property name.defaultValue
- The default valueString
s,
or default value.public void setStrings(String name, String... values)
name
property as
as comma delimited values.name
- property name.values
- The valuespublic InetSocketAddress getSocketAddr(String name, String defaultAddress, int defaultPort)
name
property as a
InetSocketAddress
.name
- property name.defaultAddress
- the default valuedefaultPort
- the default portpublic void setSocketAddr(String name, InetSocketAddress addr)
name
property as
a host:port
.public InetSocketAddress updateConnectAddr(String name, InetSocketAddress addr)
name
property as a host:port
. The wildcard
address is replaced with the local host's address.name
- property name.addr
- InetSocketAddress of a listener to store in the given propertypublic Class<?> getClassByName(String name) throws ClassNotFoundException
name
- the class name.ClassNotFoundException
- if the class is not found.public Class<?> getClassByNameOrNull(String name)
name
- the class namepublic Class<?>[] getClasses(String name, Class<?>... defaultValue)
name
property
as an array of Class
.
The value of the property specifies a list of comma separated class names.
If no such property is specified, then defaultValue
is
returned.name
- the property name.defaultValue
- default value.Class[]
,
or defaultValue
.public Class<?> getClass(String name, Class<?> defaultValue)
name
property as a Class
.
If no such property is specified, then defaultValue
is
returned.name
- the class name.defaultValue
- default value.Class
,
or defaultValue
.public <U> Class<? extends U> getClass(String name, Class<? extends U> defaultValue, Class<U> xface)
name
property as a Class
implementing the interface specified by xface
.
If no such property is specified, then defaultValue
is
returned.
An exception is thrown if the returned class does not implement the named
interface.name
- the class name.defaultValue
- default value.xface
- the interface implemented by the named class.Class
,
or defaultValue
.public <U> List<U> getInstances(String name, Class<U> xface)
name
property as a List
of objects implementing the interface specified by xface
.
An exception is thrown if any of the classes does not exist, or if it does
not implement the named interface.name
- the property name.xface
- the interface implemented by the classes named by
name
.List
of objects implementing xface
.public void setClass(String name, Class<?> theClass, Class<?> xface)
name
property to the name of a
theClass
implementing the given interface xface
.
An exception is thrown if theClass
does not implement the
interface xface
.name
- property name.theClass
- property value.xface
- the interface implemented by the named class.public Path getLocalPath(String dirsProp, String path) throws IOException
dirsProp
- directory in which to locate the file.path
- file-path.IOException
public File getFile(String dirsProp, String path) throws IOException
dirsProp
- directory in which to locate the file.path
- file-path.IOException
public URL getResource(String name)
URL
for the named resource.name
- resource name.public InputStream getConfResourceAsInputStream(String name)
name
.name
- configuration resource name.public Reader getConfResourceAsReader(String name)
Reader
attached to the configuration resource with the
given name
.name
- configuration resource name.public Set<String> getFinalParameters()
protected Properties getProps()
public int size()
public void clear()
public Iterator<Map.Entry<String,String>> iterator()
Iterator
to go through the list of String
key-value pairs in the configuration.public void writeXml(OutputStream out) throws IOException
OutputStream
.out
- the output stream to write to.IOException
public void writeXml(Writer out) throws IOException
Writer
.out
- the writer to write to.IOException
public static void dumpConfiguration(Configuration config, Writer out) throws IOException
Writer
The format of the output would be
{ "properties" : [ {key1,value1,key1.isFinal,key1.resource}, {key2,value2,
key2.isFinal,key2.resource}... ] }
It does not output the parameters of the configuration object which is
loaded from an input stream.out
- the Writer to write toIOException
public ClassLoader getClassLoader()
ClassLoader
for this job.public void setClassLoader(ClassLoader classLoader)
classLoader
- the new class loader.public void setQuietMode(boolean quietmode)
quietmode
- true
to set quiet-mode on, false
to turn it off.public static void main(String[] args) throws Exception
Exception
public void readFields(DataInput in) throws IOException
Writable
in
.
For efficiency, implementations should attempt to re-use storage in the existing object where possible.
readFields
in interface Writable
in
- DataInput
to deseriablize this object from.IOException
public void write(DataOutput out) throws IOException
Writable
out
.write
in interface Writable
out
- DataOuput
to serialize this object into.IOException
public Map<String,String> getValByRegex(String regex)
regex
- Copyright © 2014 Apache Software Foundation. All Rights Reserved.