Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug fixes and speed optimizations in AWS Collection Connector #363

Merged
merged 8 commits into from
Dec 5, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions migrations/v1_9_0-v1_9_1.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,49 @@ AS '
'
;
~~~

## Update AWS Collect landing tables

Note: the below commands refer to default table names, please add the custom connection name if you set it.

There were more fixes to this connector than we anticipated, so we recommend simply re-creating these connections.
Please reach out if you're not sure how to go about this. If you absolutely must retain the data that was collected
earlier, the following migration should help:

~~~
ALTER TABLE data.aws_collect_grants
RENAME TO aws_collect_iam_get_credential_report
;
ALTER TABLE data.aws_collect_ec2_describe_instances
ADD COLUMN requester_id STRING
;
ALTER TABLE data.aws_collect_iam_list_users
ADD COLUMN tags VARIANT
;
ALTER TABLE data.aws_collect_iam_list_users
ADD COLUMN permissions_boundary VARIANT
;
ALTER TABLE data.aws_collect_iam_list_policies
ADD COLUMN description STRING
;
ALTER TABLE data.aws_collect_s3_get_bucket_acl
DROP COLUMN owner
;
ALTER TABLE data.aws_collect_iam_list_virtual_mfa_devices
ADD COLUMN enable_date TIMESTAMP_NTZ
;
ALTER TABLE data.aws_collect_cloudtrail_describe_trails
ADD COLUMN has_insight_selectors BOOLEAN
;
ALTER TABLE data.aws_collect_cloudtrail_get_event_selectors
ADD COLUMN exclude_management_event_sources VARIANT
;
ALTER TABLE data.aws_collect_kms_get_key_rotation_status
ADD COLUMN key_arn STRING
;
ALTER TABLE data.aws_collect_s3_get_bucket_policy
ADD COLUMN policy_json_parsed VARIANT
;
~~~

This does not migrate some of the table column types, but should get you to a working state.
Loading