wodle name="aws-s3"

New in version 3.2.0.

XML section name

<wodle name="aws-s3">
</wodle>

Configuration options of the AWS-S3 wodle.

Options

Main options

Main options

Allowed values

Mandatory/Optional

disabled

yes, no

Mandatory

bucket

Any valid bucket name

Deprecated

access_key

Alphanumerical key

Deprecated

secret_key

Alphanumerical key

Deprecated

remove_from_bucket

yes, no

Optional

skip_on_error

yes, no

Optional

bucket type

N/A

Mandatory

Scheduling options

disabled

Disables the AWS-S3 wodle.

Default value

no

Allowed values

yes, no

bucket

Name of the S3 bucket from where logs are read.

Default value

N/A

Allowed values

Any valid bucket name

access_key

The access key ID for the IAM user with the permission to read logs from the bucket.

Default value

N/A

Allowed values

Any alphanumerical key.

secret_key

The secret key created for the IAM user with the permission to read logs from the bucket.

Default value

N/A

Allowed values

Any alphanumerical key.

remove_from_bucket

Define if you want to remove logs from your S3 bucket after they are read by the wodle.

Default value

no

Allowed values

yes, no

skip_on_error

When unable to process and parse a CloudTrail log, skip the log and continue processing

Default value

yes

Allowed values

yes, no

bucket type

Defines a bucket to process. Must have its attribute type defined. (Supports multiple instances of this option).

Bucket options

Options

Allowed values

Mandatory/Optional

type

cloudtrail, guardduty, vpcflow, config, custom

Mandatory

bucket\name

Any valid bucket name

Mandatory

bucket\aws_account_id

Comma list of AWS Accounts

Optional (only works with CloudTrail buckets)

bucket\aws_account_alias

Any string

Optional

bucket\access_key

Alphanumerical key

Optional

bucket\secret_key

Alphanumerical key

Optional

bucket\aws_profile

Any string

Optional

bucket\iam_role_arn

IAM role ARN

Optional

bucket\path

Prefix for S3 bucket key

Optional

bucket\only_logs_after

Date (YYYY-MMM-DDD, for example 2018-AUG-21)

Optional

bucket\regions

Comma list of AWS regions

Optional (only works with CloudTrail buckets)

bucket\aws_organization_id

Name of AWS organization

Optional (only works with CloudTrail buckets)

type

Specifies type of bucket. Is an attribute of the bucket tag.

Default value

N/A

Allowed values

cloudtrail, guardduty, vpcflow, config, custom

Note

Different configurations as macie has custom type.

bucket\name

Name of the S3 bucket from where logs are read.

Default value

N/A

Allowed values

Any valid bucket name

bucket\aws_account_id

The AWS Account ID for the bucket logs. Only works with CloudTrail buckets.

Default value

All accounts.

Allowed values

Comma list of 12 digit AWS Account ID

bucket\aws_account_alias

A user-friendly name for the AWS account.

Default value

N/A

Allowed values

Any string

bucket\access_key

The access key ID for the IAM user with the permission to read logs from the bucket.

Default value

N/A

Allowed values

Any alphanumerical key.

bucket\secret_key

The secret key created for the IAM user with the permission to read logs from the bucket.

Default value

N/A

Allowed values

Any alphanumerical key.

bucket\aws_profile

A valid profile name from a Shared Credential File or AWS Config File with the permission to read logs from the bucket.

Default value

N/A

Allowed values

Valid profile name

bucket\iam_role_arn

A valid role arn with permission to read logs from the bucket.

Default value

N/A

Allowed values

Valid role arn

bucket\path

If defined, the path or prefix for the bucket.

Default value

N/A

Allowed values

Valid path

bucket\only_logs_after

A valid date, in YYYY-MMM-DD format, that only logs from after that date will be parsed. All logs from before that date will be skipped.

Default value

1970-JAN-01

Allowed values

Valid date

bucket\regions

A comma-delimited list of regions to limit parsing of logs. Only works with CloudTrail buckets.

Default value

All regions

Allowed values

Comma-delimited list of valid regions

bucket\aws_organization_id

Name of AWS organization. Only works with CloudTrail buckets.

Default value

N/A

Allowed values

Valid AWS organization name

run_on_start

Run evaluation immediately when service is started.

Default value

yes

Allowed values

yes, no

interval

Frequency for reading from the S3 bucket.

Default value

10m

Allowed values

A positive number that should contain a suffix character indicating a time unit, such as, s (seconds), m (minutes), h (hours), d (days), M (months).

day

Day of the month to run the scan.

Default value

n/a

Allowed values

Day of the month [1..31]

Note

When the day option is set, the interval value must be a multiple of months. By default, the interval is set to a month.

wday

Day of the week to run the scan. This option is not compatible with the day option.

Default value

n/a

Allowed values
Day of the week:

  • sunday/sun

  • monday/mon

  • tuesday/tue

  • wednesday/wed

  • thursday/thu

  • friday/fri

  • saturday/sat

Note

When the wday option is set, the interval value must be a multiple of weeks. By default, the interval is set to a week.

time

Time of the day to run the scan. It has to be represented in the format hh:mm.

Default value

n/a

Allowed values

Time of day [hh:mm]

Note

When only the time option is set, the interval value must be a multiple of days or weeks. By default, the interval is set to a day.

service type

Define a service to process. Must have the attribute type defined. (Supports multiple instances of this option).

Service\aws_account_id

The AWS Account ID for accessing the service.

Default value

All accounts.

Allowed values

Comma-delimited list of 12 digit AWS Account ID

Service\aws_account_alias

A user-friendly name for the AWS account.

Default value

N/A

Allowed values

Any string

Service\access_key

The access key ID for the IAM user with the permission to access the service.

Default value

N/A

Allowed values

Any alphanumerical key.

Service\aws_log_groups

New in version 4.0.0.

A comma-delimited list of log group names from where the logs should be extracted. Only works for CloudWatch Logs service.

Default value

All regions

Allowed values

Comma-delimited list of valid log group names

Service\secret_key

The secret key created for the IAM user with the permission to access the service.

Default value

N/A

Allowed values

Any alphanumerical key.

Service\aws_profile

A valid profile name from a Shared Credential File or AWS Config File with the permission to access the service.

Default value

N/A

Allowed values

Valid profile name

Service\iam_role_arn

A valid role arn with permission to access the service.

Default value

N/A

Allowed values

Valid role arn

Service\only_logs_after

New in version 4.0.0.

A valid date, in YYYY-MMM-DD format. Only those logs from after that date will be parsed, the logs from before that date will be skipped. Only works for CloudWatch Logs service.

Default value

1970-JAN-01

Allowed values

Valid date

Service\regions

New in version 4.0.0.

A comma-delimited list of regions to limit parsing of logs. Only works for CloudWatch Logs service.

Default value

All regions

Allowed values

Comma-delimited list of valid regions

Service\remove_log_streams

New in version 4.0.0.

Define whether or not to remove the log streams from the log groups after they are read by the module. Only works for CloudWatch Logs service.

Default value

no

Allowed values

yes, no

Example of configuration 

<wodle name="aws-s3">
    <disabled>no</disabled>
    <remove_from_bucket>no</remove_from_bucket>
    <interval>10m</interval>
    <run_on_start>no</run_on_start>
    <skip_on_error>no</skip_on_error>
    <bucket type="cloudtrail">
        <name>s3-dev-bucket</name>
        <access_key>insert_access_key</access_key>
        <secret_key>insert_secret_key</secret_key>
        <only_logs_after>2018-JUN-01</only_logs_after>
        <regions>us-east-1,us-west-1,eu-central-1</regions>
        <path>/dev1/</path>
        <aws_account_id>123456789012</aws_account_id>
        <aws_account_alias>dev1-account</aws_account_alias>
    </bucket>
    <bucket type="cloudtrail">
        <name>s3-dev-bucket</name>
        <access_key>insert_access_key</access_key>
        <secret_key>insert_secret_key</secret_key>
        <only_logs_after>2018-JUN-01</only_logs_after>
        <regions>us-east-1,us-west-1,eu-central-1</regions>
        <path>/dev2/</path>
        <aws_account_id>112233445566</aws_account_id>
        <aws_account_alias>dev2-account</aws_account_alias>
    </bucket>
    <bucket type="custom">
        <name>s3-stage-bucket</name>
        <aws_profile>stage-creds</aws_profile>
        <aws_account_id>111222333444</aws_account_id>
        <aws_account_alias>stage-account</aws_account_alias>
    </bucket>
    <bucket type="custom">
        <name>s3-prod-bucket</name>
        <iam_role_arn>arn:aws:iam::010203040506:role/ROLE_SVC_Log-Parser</iam_role_arn>
        <aws_account_id>11112222333</aws_account_id>
        <aws_account_alias>prod-account</aws_account_alias>
    </bucket>
    <service type="cloudwatchlogs">
        <access_key>insert_access_key</access_key>
        <secret_key>insert_secret_key</secret_key>
        <aws_log_groups>log_group1,log_group2</aws_log_groups>
        <only_logs_after>2018-JUN-01</only_logs_after>
        <regions>us-east-1,us-west-1,eu-central-1</regions>
    </service>
</wodle>