Package org.apache.hadoop.yarn.logaggregation.filecontroller.tfile

Class LogAggregationTFileController

java.lang.Object

org.apache.hadoop.yarn.logaggregation.filecontroller.LogAggregationFileController

org.apache.hadoop.yarn.logaggregation.filecontroller.tfile.LogAggregationTFileController

@Private
@Unstable
public class LogAggregationTFileController
extends LogAggregationFileController

The TFile log aggregation file Controller implementation.

Field Summary

Fields inherited from class org.apache.hadoop.yarn.logaggregation.filecontroller.LogAggregationFileController

APP_DIR_PERMISSIONS, APP_LOG_FILE_UMASK, conf, fileControllerName, fsSupportsChmod, remoteRootLogDir, remoteRootLogDirSuffix, retentionSize, TLDIR_PERMISSIONS

Constructor Summary

Constructors

Constructor	Description
LogAggregationTFileController()

Method Summary

Modifier and Type	Method	Description
void	closeWriter()	Close the writer.
Map<org.apache.hadoop.yarn.api.records.ApplicationAccessType,String>	getApplicationAcls(org.apache.hadoop.fs.Path aggregatedLog, org.apache.hadoop.yarn.api.records.ApplicationId appId)	Returns ACLs for the application.
String	getApplicationOwner(org.apache.hadoop.fs.Path aggregatedLog, org.apache.hadoop.yarn.api.records.ApplicationId appId)	Returns the owner of the application.
Map<String,List<ContainerLogFileInfo>>	getLogMetaFilesOfNode(ExtendedLogMetaRequest logRequest, org.apache.hadoop.fs.FileStatus currentNodeFile, org.apache.hadoop.yarn.api.records.ApplicationId appId)	Returns log file metadata for a node grouped by containers.
void	initializeWriter(LogAggregationFileControllerContext context)	Initialize the writer.
void	initInternal(org.apache.hadoop.conf.Configuration conf)	Derived classes initialize themselves using this method.
void	postWrite(LogAggregationFileControllerContext record)	Operations needed after write the log content.
boolean	readAggregatedLogs(ContainerLogsRequest logRequest, OutputStream os)	Output container log.
List<ContainerLogMeta>	readAggregatedLogsMeta(ContainerLogsRequest logRequest)	Return a list of ContainerLogMeta for an application from Remote FileSystem.
void	renderAggregatedLogsBlock(HtmlBlock.Block html, View.ViewContext context)	Render Aggregated Logs block.
void	write(AggregatedLogFormat.LogKey logKey, AggregatedLogFormat.LogValue logValue)	Write the log content.

Methods inherited from class org.apache.hadoop.yarn.logaggregation.filecontroller.LogAggregationFileController

aggregatedLogSuffix, belongsToAppAttempt, checkExists, cleanOldLogs, closePrintStream, createAppDir, createDir, getApplicationDirectoriesOfUser, getFileControllerName, getFileSystem, getNodeFilesOfApplicationDirectory, getOlderRemoteAppLogDir, getRemoteAppLogDir, getRemoteNodeLogFileForApp, getRemoteRootLogDir, getRemoteRootLogDirSuffix, initialize, isFsSupportsChmod, verifyAndCreateRemoteLogDir

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

LogAggregationTFileController

public LogAggregationTFileController()

Method Details

initInternal

public void initInternal(org.apache.hadoop.conf.Configuration conf)

Description copied from class: LogAggregationFileController

Derived classes initialize themselves using this method.

Specified by:

initInternal in class LogAggregationFileController

Parameters:

conf - the Configuration

initializeWriter

public void initializeWriter(LogAggregationFileControllerContext context)
                      throws IOException

Description copied from class: LogAggregationFileController

Initialize the writer.

Specified by:

initializeWriter in class LogAggregationFileController

Parameters:

context - the LogAggregationFileControllerContext

Throws:

IOException - if fails to initialize the writer

closeWriter

public void closeWriter()
                 throws LogAggregationDFSException

Description copied from class: LogAggregationFileController

Close the writer.

Specified by:

closeWriter in class LogAggregationFileController

Throws:

LogAggregationDFSException - if the closing of the writer fails (for example due to HDFS quota being exceeded)

write

public void write(AggregatedLogFormat.LogKey logKey,
 AggregatedLogFormat.LogValue logValue)
           throws IOException

Description copied from class: LogAggregationFileController

Write the log content.

Specified by:

write in class LogAggregationFileController

Parameters:

logKey - the log key

logValue - the log content

Throws:

IOException - if fails to write the logs

postWrite

public void postWrite(LogAggregationFileControllerContext record)
               throws Exception

Description copied from class: LogAggregationFileController

Operations needed after write the log content.

Specified by:

postWrite in class LogAggregationFileController

Parameters:

record - the LogAggregationFileControllerContext

Throws:

Exception - if anything fails

readAggregatedLogs

public boolean readAggregatedLogs(ContainerLogsRequest logRequest,
 OutputStream os)
                           throws IOException

Description copied from class: LogAggregationFileController

Output container log.

Specified by:

readAggregatedLogs in class LogAggregationFileController

Parameters:

logRequest - ContainerLogsRequest

os - the output stream

Returns:

true if we can read the aggregated logs successfully

Throws:

IOException - if we can not access the log file.

getLogMetaFilesOfNode

public Map<String,List<ContainerLogFileInfo>> getLogMetaFilesOfNode(ExtendedLogMetaRequest logRequest,
 org.apache.hadoop.fs.FileStatus currentNodeFile,
 org.apache.hadoop.yarn.api.records.ApplicationId appId)
                                                             throws IOException

Description copied from class: LogAggregationFileController

Returns log file metadata for a node grouped by containers.

Overrides:

getLogMetaFilesOfNode in class LogAggregationFileController

Parameters:

logRequest - extended query information holder

currentNodeFile - file status of a node in an application directory

appId - id of the application, which is the same as in node path

Returns:

log file metadata

Throws:

IOException - if there is no node file

readAggregatedLogsMeta

public List<ContainerLogMeta> readAggregatedLogsMeta(ContainerLogsRequest logRequest)
                                              throws IOException

Description copied from class: LogAggregationFileController

Return a list of ContainerLogMeta for an application from Remote FileSystem.

Specified by:

readAggregatedLogsMeta in class LogAggregationFileController

Parameters:

logRequest - ContainerLogsRequest

Returns:

a list of ContainerLogMeta

Throws:

IOException - if there is no available log file

renderAggregatedLogsBlock

public void renderAggregatedLogsBlock(HtmlBlock.Block html,
 View.ViewContext context)

Description copied from class: LogAggregationFileController

Render Aggregated Logs block.

Specified by:

renderAggregatedLogsBlock in class LogAggregationFileController

Parameters:

html - the html

context - the ViewContext

getApplicationOwner

public String getApplicationOwner(org.apache.hadoop.fs.Path aggregatedLog,
 org.apache.hadoop.yarn.api.records.ApplicationId appId)
                           throws IOException

Description copied from class: LogAggregationFileController

Returns the owner of the application.

Specified by:

getApplicationOwner in class LogAggregationFileController

Parameters:

aggregatedLog - the aggregatedLog path

appId - the ApplicationId

Returns:

the application owner

Throws:

IOException - if we can not get the application owner

getApplicationAcls

public Map<org.apache.hadoop.yarn.api.records.ApplicationAccessType,String> getApplicationAcls(org.apache.hadoop.fs.Path aggregatedLog,
 org.apache.hadoop.yarn.api.records.ApplicationId appId)
                                                                                        throws IOException

Description copied from class: LogAggregationFileController

Returns ACLs for the application. An empty map is returned if no ACLs are found.

Specified by:

getApplicationAcls in class LogAggregationFileController

Parameters:

aggregatedLog - the aggregatedLog path.

appId - the ApplicationId

Returns:

a map of the Application ACLs.

Throws:

IOException - if we can not get the application acls