Package org.apache.hadoop.fs.s3a.audit

Class S3LogParser

java.lang.Object

org.apache.hadoop.fs.s3a.audit.S3LogParser

@Public
@Unstable
public final class S3LogParser
extends Object

Class to help parse AWS S3 Logs. see https://docs.aws.amazon.com/AmazonS3/latest/userguide/LogFormat.html Getting the regexp right is surprisingly hard; this class does it explicitly and names each group in the process. All group names are included in AWS_LOG_REGEXP_GROUPS in the order within the log entries.

Field Summary

Fields

Modifier and Type	Field	Description
static final String	AUTH_GROUP	Log group "auth".
static final List<String>	AWS_LOG_REGEXP_GROUPS	Ordered list of regular expression group names.
static final String	AWSERRORCODE_GROUP	Log group "awserrorcode".
static final String	BUCKET_GROUP	Log group "bucket".
static final String	BYTESSENT_GROUP	Log group "bytessent".
static final String	CYPHER_GROUP	Log group "cypher".
static final String	ENDPOINT_GROUP	Log group "endpoint".
static final String	HOSTID_GROUP	Log group "hostid".
static final String	HTTP_GROUP	Log group "http".
static final String	KEY_GROUP	Log group "key".
static final Pattern	LOG_ENTRY_PATTERN	And the actual compiled pattern.
static final String	LOG_ENTRY_REGEXP	Construct the log entry pattern.
static final String	OBJECTSIZE_GROUP	Log group "objectsize".
static final String	OWNER_GROUP	Log group "owner".
static final String	REFERRER_GROUP	Log group "referrer".
static final String	REMOTEIP_GROUP	Log group "remoteip".
static final String	REQUESTER_GROUP	Log group "requester".
static final String	REQUESTID_GROUP	Log group "requestid".
static final String	REQUESTURI_GROUP	Log group "requesturi".
static final String	SIGV_GROUP	Log group "sigv".
static final String	TAIL_GROUP	This is where anything at the tail of a log entry ends up; it is null unless/until the AWS logs are enhanced in future.
static final String	TIMESTAMP_GROUP	Log group "timestamp".
static final String	TLS_GROUP	Log group "tls".
static final String	TOTALTIME_GROUP	Log group "totaltime".
static final String	TURNAROUNDTIME_GROUP	Log group "turnaroundtime".
static final String	USERAGENT_GROUP	Log group "useragent".
static final String	VERB_GROUP	Log group "verb".
static final String	VERSION_GROUP	Log group "version".

Method Summary

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

OWNER_GROUP

public static final String OWNER_GROUP

Log group "owner".

See Also:

• Constant Field Values

BUCKET_GROUP

public static final String BUCKET_GROUP

Log group "bucket".

See Also:

• Constant Field Values

TIMESTAMP_GROUP

public static final String TIMESTAMP_GROUP

Log group "timestamp".

See Also:

• Constant Field Values

REMOTEIP_GROUP

public static final String REMOTEIP_GROUP

Log group "remoteip".

See Also:

• Constant Field Values

REQUESTER_GROUP

public static final String REQUESTER_GROUP

Log group "requester".

See Also:

• Constant Field Values

REQUESTID_GROUP

public static final String REQUESTID_GROUP

Log group "requestid".

See Also:

• Constant Field Values

VERB_GROUP

public static final String VERB_GROUP

Log group "verb".

See Also:

• Constant Field Values

KEY_GROUP

public static final String KEY_GROUP

Log group "key".

See Also:

• Constant Field Values

REQUESTURI_GROUP

public static final String REQUESTURI_GROUP

Log group "requesturi".

See Also:

• Constant Field Values

HTTP_GROUP

public static final String HTTP_GROUP

Log group "http".

See Also:

• Constant Field Values

AWSERRORCODE_GROUP

public static final String AWSERRORCODE_GROUP

Log group "awserrorcode".

See Also:

• Constant Field Values

BYTESSENT_GROUP

public static final String BYTESSENT_GROUP

Log group "bytessent".

See Also:

• Constant Field Values

OBJECTSIZE_GROUP

public static final String OBJECTSIZE_GROUP

Log group "objectsize".

See Also:

• Constant Field Values

TOTALTIME_GROUP

public static final String TOTALTIME_GROUP

Log group "totaltime".

See Also:

• Constant Field Values

TURNAROUNDTIME_GROUP

public static final String TURNAROUNDTIME_GROUP

Log group "turnaroundtime".

See Also:

• Constant Field Values

REFERRER_GROUP

public static final String REFERRER_GROUP

Log group "referrer".

See Also:

• Constant Field Values

USERAGENT_GROUP

public static final String USERAGENT_GROUP

Log group "useragent".

See Also:

• Constant Field Values

VERSION_GROUP

public static final String VERSION_GROUP

Log group "version".

See Also:

• Constant Field Values

HOSTID_GROUP

public static final String HOSTID_GROUP

Log group "hostid".

See Also:

• Constant Field Values

SIGV_GROUP

public static final String SIGV_GROUP

Log group "sigv".

See Also:

• Constant Field Values

CYPHER_GROUP

public static final String CYPHER_GROUP

Log group "cypher".

See Also:

• Constant Field Values

AUTH_GROUP

public static final String AUTH_GROUP

Log group "auth".

See Also:

• Constant Field Values

ENDPOINT_GROUP

public static final String ENDPOINT_GROUP

Log group "endpoint".

See Also:

• Constant Field Values

TLS_GROUP

public static final String TLS_GROUP

Log group "tls".

See Also:

• Constant Field Values

TAIL_GROUP

public static final String TAIL_GROUP

This is where anything at the tail of a log entry ends up; it is null unless/until the AWS logs are enhanced in future. Value "tail".

See Also:

• Constant Field Values

LOG_ENTRY_REGEXP

public static final String LOG_ENTRY_REGEXP

Construct the log entry pattern.

AWS_LOG_REGEXP_GROUPS

public static final List<String> AWS_LOG_REGEXP_GROUPS

Ordered list of regular expression group names.

LOG_ENTRY_PATTERN

public static final Pattern LOG_ENTRY_PATTERN

And the actual compiled pattern.