Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(docker): add archiver service to compose file #15

Merged
merged 2 commits into from
Apr 18, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 16 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ It should be able to scale based on the SQS Queue's number of message.

### Local

Localstack is available through the docker-compose manifest at the top-level directory of this project.
Localstack and archiver services are available through the docker-compose manifest at the top-level directory of this project.

It is currently setup to:

Expand All @@ -78,19 +78,31 @@ automatically loads `.envrc` files and injects the found variables in the curren

[direnv]: https://direnv.net/

In order to start the localstack container, you can run the following commands:
In order to start containers, you can run the following commands:

```console
$ cp .envrc.dist .envrc
$ docker compose up -d
$ poetry install
$ poetry run archiver
```

You can now rely on the http://localhost:4566/ endpoint with the fake credentials to
send messages in the localstack queues.

### Service usage example
Every time you want to run aws commands, ensure that endpoint_url is added for any localstack calls

#### Import file to S3 buckets
```bash
aws s3 cp ~/${path_to_your_file} s3://source-images \
Sebastien-jo marked this conversation as resolved.
Show resolved Hide resolved
--endpoint-url http://localhost:4566
```

#### Ensure file is properly imported
```bash
aws s3 ls --recursive s3://source-images
```

#### Send message to SQS queue
```bash
aws sqs send-message --queue-url http://localhost:4566/000000000000/input-queue \
--endpoint-url http://localhost:4566 --message-body \
Expand Down
24 changes: 24 additions & 0 deletions docker-compose.yaml
Original file line number Diff line number Diff line change
@@ -1,8 +1,12 @@
---
version: "3.8"
networks:
main:

services:
localstack:
networks:
- main
image: localstack/localstack
ports:
- "127.0.0.1:4566:4566" # LocalStack Gateway
Expand All @@ -11,3 +15,23 @@ services:
- "./localstack_data:/var/lib/localstack"
- "./localstack.d/init-aws.sh:/etc/localstack/init/ready.d/init-aws.sh" # ready hook
- "./source.jpg:/tmp/source.jpg"

archiver:
networks:
- main
build:
context: .
depends_on:
localstack:
condition: service_healthy
ports:
- "127.0.0.1:8080:8080"
environment:
- S3_SOURCE_BUCKET=source-images
- S3_DESTINATION_BUCKET=zip-storage
- SQS_SOURCE_QUEUE=input-queue
- SQS_DESTINATION=output-queue
- AWS_ACCESS_KEY_ID=test
- AWS_SECRET_ACCESS_KEY=test
- AWS_ENDPOINT_URL=http://localstack:4566/
- AWS_DEFAULT_REGION=eu-west-1
4 changes: 0 additions & 4 deletions src/archiver/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -150,10 +150,6 @@ def process_archive(s3_client, sqs_client, sqs_response, logger):
QueueUrl=SQS_SOURCE,
ReceiptHandle=message['ReceiptHandle'],
)
logger.info(
"done processing request with template_id=%s",
content['sendgrid_template_id'],
)
return archive_s3_path


Expand Down
Loading