Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use step functions to run event jobs #744

Closed
lorenyu opened this issue Aug 28, 2024 · 0 comments · Fixed by #757
Closed

Use step functions to run event jobs #744

lorenyu opened this issue Aug 28, 2024 · 0 comments · Fixed by #757
Assignees
@coilysiren
Labels
scope: service Service layer / component type: feature New feature or new functionality to existing feature
Milestone
Background jobs

Comments

@lorenyu
Copy link
Contributor

lorenyu commented Aug 28, 2024 •
edited
Loading

We are using step functions to run scheduled jobs, so we should use the same workflow orchestrator to run event jobs
@lorenyu lorenyu added this to the Background jobs milestone Aug 28, 2024
@lorenyu lorenyu added type: feature New feature or new functionality to existing feature scope: service Service layer / component labels Aug 28, 2024
@coilysiren coilysiren self-assigned this Aug 30, 2024
@coilysiren coilysiren mentioned this issue Sep 13, 2024
Closed
@coilysiren coilysiren mentioned this issue Sep 27, 2024
Merged
@coilysiren coilysiren mentioned this issue Oct 17, 2024
Open
coilysiren added a commit that referenced this issue Oct 18, 2024
@coilysiren
Perform ETL events via Step Functions (#757)
077a980 
## Ticket

Resolves #744

## Changes

- Changes the ETL job from invoking ECS directly, to instead invoking
Step Functions which then invokes ECS
- Makes some changes to the scheduled changes to increase their
distinction from the events jobs, and the readability of their IAM roles

## Context for reviewers

This is a similar PR to
#745. In-fact they use
many of the same resources. To goal of this PR is to run ETL events via
Step Functions. Step Functions create a tracking layer that's not
available when invoking ECS directly. This tracking layer allows you to
see the success and failure status of your jobs.

## Testing

navapbc/platform-test#136
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@coilysiren coilysiren

Labels
scope: service Service layer / component type: feature New feature or new functionality to existing feature
Projects
None yet
Milestone
Background jobs
2 participants
@lorenyu @coilysiren