wodle name="aws-s3"
New in version 3.2.0.
Configuration options of the AWS-S3 wodle.
Options
Main options
Main options |
Allowed values |
Mandatory/Optional |
---|---|---|
yes, no |
Mandatory |
|
Any valid bucket name |
Deprecated |
|
Alphanumerical key |
Deprecated |
|
Alphanumerical key |
Deprecated |
|
yes, no |
Optional |
|
yes, no |
Optional |
|
N/A |
Mandatory |
Scheduling options
disabled
Disables the AWS-S3 wodle.
Default value |
no |
Allowed values |
yes, no |
bucket
Name of the S3 bucket from where logs are read.
Default value |
N/A |
Allowed values |
Any valid bucket name |
access_key
The access key ID for the IAM user with the permission to read logs from the bucket.
Default value |
N/A |
Allowed values |
Any alphanumerical key. |
secret_key
The secret key created for the IAM user with the permission to read logs from the bucket.
Default value |
N/A |
Allowed values |
Any alphanumerical key. |
remove_from_bucket
Define if you want to remove logs from your S3 bucket after they are read by the wodle.
Default value |
no |
Allowed values |
yes, no |
skip_on_error
When unable to process and parse a CloudTrail log, skip the log and continue processing
Default value |
yes |
Allowed values |
yes, no |
bucket type
Defines a bucket to process. Must have its attribute type
defined. (Supports multiple instances of this option).
Bucket options
Options |
Allowed values |
Mandatory/Optional |
---|---|---|
cloudtrail, guardduty, vpcflow, config, custom |
Mandatory |
|
Any valid bucket name |
Mandatory |
|
Comma list of AWS Accounts |
Optional (only works with CloudTrail buckets) |
|
Any string |
Optional |
|
Alphanumerical key |
Optional |
|
Alphanumerical key |
Optional |
|
Any string |
Optional |
|
IAM role ARN |
Optional |
|
Prefix for S3 bucket key |
Optional |
|
Date (YYYY-MMM-DDD, for example 2018-AUG-21) |
Optional |
|
Comma list of AWS regions |
Optional (only works with CloudTrail buckets) |
|
Name of AWS organization |
Optional (only works with CloudTrail buckets) |
type
Specifies type of bucket. Is an attribute of the bucket
tag.
Default value |
N/A |
Allowed values |
cloudtrail, guardduty, vpcflow, config, custom |
Note
Different configurations as macie
has custom
type.
bucket\name
Name of the S3 bucket from where logs are read.
Default value |
N/A |
Allowed values |
Any valid bucket name |
bucket\aws_account_id
The AWS Account ID for the bucket logs. Only works with CloudTrail buckets.
Default value |
All accounts. |
Allowed values |
Comma list of 12 digit AWS Account ID |
bucket\aws_account_alias
A user-friendly name for the AWS account.
Default value |
N/A |
Allowed values |
Any string |
bucket\access_key
The access key ID for the IAM user with the permission to read logs from the bucket.
Default value |
N/A |
Allowed values |
Any alphanumerical key. |
bucket\secret_key
The secret key created for the IAM user with the permission to read logs from the bucket.
Default value |
N/A |
Allowed values |
Any alphanumerical key. |
bucket\aws_profile
A valid profile name from a Shared Credential File or AWS Config File with the permission to read logs from the bucket.
Default value |
N/A |
Allowed values |
Valid profile name |
bucket\iam_role_arn
A valid role arn with permission to read logs from the bucket.
Default value |
N/A |
Allowed values |
Valid role arn |
bucket\path
If defined, the path or prefix for the bucket.
Default value |
N/A |
Allowed values |
Valid path |
bucket\only_logs_after
A valid date, in YYYY-MMM-DD format, that only logs from after that date will be parsed. All logs from before that date will be skipped.
Default value |
1970-JAN-01 |
Allowed values |
Valid date |
bucket\regions
A comma-delimited list of regions to limit parsing of logs. Only works with CloudTrail buckets.
Default value |
All regions |
Allowed values |
Comma-delimited list of valid regions |
bucket\aws_organization_id
Name of AWS organization. Only works with CloudTrail buckets.
Default value |
N/A |
Allowed values |
Valid AWS organization name |
run_on_start
Run evaluation immediately when service is started.
Default value |
yes |
Allowed values |
yes, no |
interval
Frequency for reading from the S3 bucket.
Default value |
10m |
Allowed values |
A positive number that should contain a suffix character indicating a time unit, such as, s (seconds), m (minutes), h (hours), d (days), M (months). |
day
Day of the month to run the scan.
Default value |
n/a |
Allowed values |
Day of the month [1..31] |
Note
When the day
option is set, the interval value must be a multiple of months. By default, the interval is set to a month.
wday
Day of the week to run the scan. This option is not compatible with the day
option.
Default value |
n/a |
Allowed values |
|
Note
When the wday
option is set, the interval value must be a multiple of weeks. By default, the interval is set to a week.
time
Time of the day to run the scan. It has to be represented in the format hh:mm.
Default value |
n/a |
Allowed values |
Time of day [hh:mm] |
Note
When only the time
option is set, the interval value must be a multiple of days or weeks. By default, the interval is set to a day.
service type
Define a service to process. Must have the attribute type
defined. (Supports multiple instances of this option).
Service options
Service\aws_account_id
The AWS Account ID for accessing the service.
Default value |
All accounts. |
Allowed values |
Comma-delimited list of 12 digit AWS Account ID |
Service\aws_account_alias
A user-friendly name for the AWS account.
Default value |
N/A |
Allowed values |
Any string |
Service\access_key
The access key ID for the IAM user with the permission to access the service.
Default value |
N/A |
Allowed values |
Any alphanumerical key. |
Service\aws_log_groups
New in version 4.0.0.
A comma-delimited list of log group names from where the logs should be extracted. Only works for CloudWatch Logs service.
Default value |
All regions |
Allowed values |
Comma-delimited list of valid log group names |
Service\secret_key
The secret key created for the IAM user with the permission to access the service.
Default value |
N/A |
Allowed values |
Any alphanumerical key. |
Service\aws_profile
A valid profile name from a Shared Credential File or AWS Config File with the permission to access the service.
Default value |
N/A |
Allowed values |
Valid profile name |
Service\iam_role_arn
A valid role arn with permission to access the service.
Default value |
N/A |
Allowed values |
Valid role arn |
Service\only_logs_after
New in version 4.0.0.
A valid date, in YYYY-MMM-DD format. Only those logs from after that date will be parsed, the logs from before that date will be skipped. Only works for CloudWatch Logs service.
Default value |
1970-JAN-01 |
Allowed values |
Valid date |
Service\regions
New in version 4.0.0.
A comma-delimited list of regions to limit parsing of logs. Only works for CloudWatch Logs service.
Default value |
All regions |
Allowed values |
Comma-delimited list of valid regions |
Service\remove_log_streams
New in version 4.0.0.
Define whether or not to remove the log streams from the log groups after they are read by the module. Only works for CloudWatch Logs service.
Default value |
no |
Allowed values |
yes, no |
Example of configuration
<wodle name="aws-s3">
<disabled>no</disabled>
<remove_from_bucket>no</remove_from_bucket>
<interval>10m</interval>
<run_on_start>no</run_on_start>
<skip_on_error>no</skip_on_error>
<bucket type="cloudtrail">
<name>s3-dev-bucket</name>
<access_key>insert_access_key</access_key>
<secret_key>insert_secret_key</secret_key>
<only_logs_after>2018-JUN-01</only_logs_after>
<regions>us-east-1,us-west-1,eu-central-1</regions>
<path>/dev1/</path>
<aws_account_id>123456789012</aws_account_id>
<aws_account_alias>dev1-account</aws_account_alias>
</bucket>
<bucket type="cloudtrail">
<name>s3-dev-bucket</name>
<access_key>insert_access_key</access_key>
<secret_key>insert_secret_key</secret_key>
<only_logs_after>2018-JUN-01</only_logs_after>
<regions>us-east-1,us-west-1,eu-central-1</regions>
<path>/dev2/</path>
<aws_account_id>112233445566</aws_account_id>
<aws_account_alias>dev2-account</aws_account_alias>
</bucket>
<bucket type="custom">
<name>s3-stage-bucket</name>
<aws_profile>stage-creds</aws_profile>
<aws_account_id>111222333444</aws_account_id>
<aws_account_alias>stage-account</aws_account_alias>
</bucket>
<bucket type="custom">
<name>s3-prod-bucket</name>
<iam_role_arn>arn:aws:iam::010203040506:role/ROLE_SVC_Log-Parser</iam_role_arn>
<aws_account_id>11112222333</aws_account_id>
<aws_account_alias>prod-account</aws_account_alias>
</bucket>
<service type="cloudwatchlogs">
<access_key>insert_access_key</access_key>
<secret_key>insert_secret_key</secret_key>
<aws_log_groups>log_group1,log_group2</aws_log_groups>
<only_logs_after>2018-JUN-01</only_logs_after>
<regions>us-east-1,us-west-1,eu-central-1</regions>
</service>
</wodle>