org.apache.hadoop.yarn.logaggregation

Class LogAggregationUtils

java.lang.Object

org.apache.hadoop.yarn.logaggregation.LogAggregationUtils

@InterfaceAudience.Private
public class LogAggregationUtils
extends Object

Field Summary

Fields

Modifier and Type	Field and Description
static String	TMP_FILE_SUFFIX

Constructor Summary

Constructors

Constructor and Description
LogAggregationUtils()

Method Summary

Modifier and Type	Method and Description
static String	getNodeString(org.apache.hadoop.yarn.api.records.NodeId nodeId) Converts a nodeId to a form used in the app log file name.
static String	getNodeString(String nodeId)
static org.apache.hadoop.fs.Path	getRemoteAppLogDir(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.yarn.api.records.ApplicationId appId, String appOwner) Return the remote application log directory.
static org.apache.hadoop.fs.Path	getRemoteAppLogDir(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.yarn.api.records.ApplicationId appId, String appOwner, org.apache.hadoop.fs.Path remoteRootLogDir, String suffix) Return the remote application log directory.
static org.apache.hadoop.fs.Path	getRemoteAppLogDir(org.apache.hadoop.fs.Path remoteRootLogDir, org.apache.hadoop.yarn.api.records.ApplicationId appId, String user, String suffix) Gets the remote app log dir.
static org.apache.hadoop.fs.Path	getRemoteLogSuffixedDir(org.apache.hadoop.fs.Path remoteRootLogDir, String user, String suffix) Gets the remote suffixed log dir for the user.
static org.apache.hadoop.fs.Path	getRemoteLogUserDir(org.apache.hadoop.fs.Path remoteRootLogDir, String user) Gets the remote log user dir.
static org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.FileStatus>	getRemoteNodeFileDir(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.yarn.api.records.ApplicationId appId, String appOwner) Get all available log files under remote app log directory.
static org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.FileStatus>	getRemoteNodeFileDir(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.yarn.api.records.ApplicationId appId, String appOwner, org.apache.hadoop.fs.Path remoteRootLogDir, String suffix) Get all available log files under remote app log directory.
static List<org.apache.hadoop.fs.FileStatus>	getRemoteNodeFileList(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.yarn.api.records.ApplicationId appId, String appOwner, org.apache.hadoop.fs.Path remoteRootLogDir, String suffix) Get all available log files under remote app log directory.
static String	getRemoteNodeLogDirSuffix(org.apache.hadoop.conf.Configuration conf) Returns the suffix component of the log dir.
static org.apache.hadoop.fs.Path	getRemoteNodeLogFileForApp(org.apache.hadoop.fs.Path remoteRootLogDir, org.apache.hadoop.yarn.api.records.ApplicationId appId, String user, org.apache.hadoop.yarn.api.records.NodeId nodeId, String suffix) Constructs the full filename for an application's log file per node.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

TMP_FILE_SUFFIX

public static final String TMP_FILE_SUFFIX

See Also:

Constant Field Values

Constructor Detail

LogAggregationUtils

public LogAggregationUtils()

Method Detail

getRemoteNodeLogFileForApp

public static org.apache.hadoop.fs.Path getRemoteNodeLogFileForApp(org.apache.hadoop.fs.Path remoteRootLogDir,
                                                                   org.apache.hadoop.yarn.api.records.ApplicationId appId,
                                                                   String user,
                                                                   org.apache.hadoop.yarn.api.records.NodeId nodeId,
                                                                   String suffix)

Constructs the full filename for an application's log file per node.

Parameters:

remoteRootLogDir - the aggregated remote root log dir

appId - the application Id

user - the application owner

nodeId - the node id

suffix - the log dir suffix

Returns:

the remote log file.

getRemoteAppLogDir

public static org.apache.hadoop.fs.Path getRemoteAppLogDir(org.apache.hadoop.fs.Path remoteRootLogDir,
                                                           org.apache.hadoop.yarn.api.records.ApplicationId appId,
                                                           String user,
                                                           String suffix)

Gets the remote app log dir.

Parameters:

remoteRootLogDir - the aggregated log remote root log dir

appId - the application id

user - the application owner

suffix - the log directory suffix

Returns:

the remote application specific log dir.

getRemoteLogSuffixedDir

public static org.apache.hadoop.fs.Path getRemoteLogSuffixedDir(org.apache.hadoop.fs.Path remoteRootLogDir,
                                                                String user,
                                                                String suffix)

Gets the remote suffixed log dir for the user.

Parameters:

remoteRootLogDir - the aggregated log remote root log dir

user - the application owner

suffix - the log dir suffix

Returns:

the remote suffixed log dir.

getRemoteLogUserDir

public static org.apache.hadoop.fs.Path getRemoteLogUserDir(org.apache.hadoop.fs.Path remoteRootLogDir,
                                                            String user)

Gets the remote log user dir.

Parameters:

remoteRootLogDir - the aggregated log remote root log dir

user - the application owner

Returns:

the remote per user log dir.

getRemoteNodeLogDirSuffix

public static String getRemoteNodeLogDirSuffix(org.apache.hadoop.conf.Configuration conf)

Returns the suffix component of the log dir.

Parameters:

conf - the configuration

Returns:

the suffix which will be appended to the user log dir.

getNodeString

public static String getNodeString(org.apache.hadoop.yarn.api.records.NodeId nodeId)

Converts a nodeId to a form used in the app log file name.

Parameters:

nodeId - the nodeId

Returns:

the node string to be used to construct the file name.

getNodeString

public static String getNodeString(String nodeId)

getRemoteAppLogDir

public static org.apache.hadoop.fs.Path getRemoteAppLogDir(org.apache.hadoop.conf.Configuration conf,
                                                           org.apache.hadoop.yarn.api.records.ApplicationId appId,
                                                           String appOwner)
                                                    throws IOException

Return the remote application log directory.

Parameters:

conf - the configuration

appId - the application

appOwner - the application owner

Returns:

the remote application log directory path

Throws:

IOException - if we can not find remote application log directory

getRemoteAppLogDir

public static org.apache.hadoop.fs.Path getRemoteAppLogDir(org.apache.hadoop.conf.Configuration conf,
                                                           org.apache.hadoop.yarn.api.records.ApplicationId appId,
                                                           String appOwner,
                                                           org.apache.hadoop.fs.Path remoteRootLogDir,
                                                           String suffix)
                                                    throws IOException

Return the remote application log directory.

Parameters:

conf - the configuration

appId - the application

appOwner - the application owner

remoteRootLogDir - the remote root log directory

suffix - the log directory suffix

Returns:

the remote application log directory path

Throws:

IOException - if we can not find remote application log directory

getRemoteNodeFileDir

public static org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.FileStatus> getRemoteNodeFileDir(org.apache.hadoop.conf.Configuration conf,
                                                                                                        org.apache.hadoop.yarn.api.records.ApplicationId appId,
                                                                                                        String appOwner,
                                                                                                        org.apache.hadoop.fs.Path remoteRootLogDir,
                                                                                                        String suffix)
                                                                                                 throws IOException

Get all available log files under remote app log directory.

Parameters:

conf - the configuration

appId - the applicationId

appOwner - the application owner

remoteRootLogDir - the remote root log directory

suffix - the log directory suffix

Returns:

the iterator of available log files

Throws:

IOException - if there is no log file available

getRemoteNodeFileList

public static List<org.apache.hadoop.fs.FileStatus> getRemoteNodeFileList(org.apache.hadoop.conf.Configuration conf,
                                                                          org.apache.hadoop.yarn.api.records.ApplicationId appId,
                                                                          String appOwner,
                                                                          org.apache.hadoop.fs.Path remoteRootLogDir,
                                                                          String suffix)
                                                                   throws IOException

Get all available log files under remote app log directory.

Parameters:

conf - the configuration

appId - the applicationId

appOwner - the application owner

remoteRootLogDir - the remote root log directory

suffix - the log directory suffix

Returns:

the list of available log files

Throws:

IOException - if there is no log file available

getRemoteNodeFileDir

public static org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.FileStatus> getRemoteNodeFileDir(org.apache.hadoop.conf.Configuration conf,
                                                                                                        org.apache.hadoop.yarn.api.records.ApplicationId appId,
                                                                                                        String appOwner)
                                                                                                 throws IOException

Get all available log files under remote app log directory.

Parameters:

conf - the configuration

appId - the applicationId

appOwner - the application owner

Returns:

the iterator of available log files

Throws:

IOException - if there is no log file available