Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle exceptions from ingestion server's calls to Elasticsearch #3437

Closed
dhruvkb opened this issue Dec 1, 2023 · 3 comments
Closed

Handle exceptions from ingestion server's calls to Elasticsearch #3437

dhruvkb opened this issue Dec 1, 2023 · 3 comments
Labels
💻 aspect: code Concerns the software code in the repository 🛠 goal: fix Bug fix help wanted Open to participation from the community 🟩 priority: low Low priority and doesn't need to be rushed 🧱 stack: ingestion server Related to the ingestion/data refresh server 🐍 tech: python Involves Python 🐛 tooling: sentry Sentry issue

Comments

@dhruvkb
Copy link
Member

dhruvkb commented Dec 1, 2023

Description

Handle exceptions from ingestion server's calls to Elasticsearch.

Example

The following calls to Elasticsearch to create a new index can raise an exception, if that index already exists.

BadRequestError(400, 'resource_already_exists_exception', 'index [image-init-filtered/-jGDxWxbQS-_SdarNzk34g] already exists')

self.es.indices.create(
index=destination_index,
body=index_settings(model_name),
)

self.es.indices.create(
index=destination_index,
body=index_settings(model_name),
)

This exception is not handled in the code. It should be captured and should mark the appropriate task as 'succeeded' in this particular case because the task is idempotent and the index has been created.

Additional context

This exception was captured by Sentry.

@dhruvkb dhruvkb added help wanted Open to participation from the community 🟩 priority: low Low priority and doesn't need to be rushed 🛠 goal: fix Bug fix 💻 aspect: code Concerns the software code in the repository 🐍 tech: python Involves Python 🐛 tooling: sentry Sentry issue 🧱 stack: ingestion server Related to the ingestion/data refresh server labels Dec 1, 2023
Copy link

sentry-io bot commented Dec 1, 2023

Sentry issue: OPENVERSE-INGESTION_SERVER-12

Copy link

sentry-io bot commented Dec 1, 2023

Sentry issue: OPENVERSE-INGESTION_SERVER-10

@AetherUnbound AetherUnbound moved this from 📋 Backlog to 📅 To Do in Openverse Backlog Dec 5, 2023
@AetherUnbound
Copy link
Collaborator

After looking into this, it looks like all of the events related to this error are in the local environment. Additionally, I feel like that piece of the code should fail the way it's currently set up to do in production; the attempted recreation of an existing index, while idempotent at that particular step, would likely mean a failure elsewhere in our process. It'd be best for execution to stop here in that case I think!

I'm going to close this issue for now.

@AetherUnbound AetherUnbound closed this as not planned Won't fix, can't repro, duplicate, stale Dec 22, 2023
@openverse-bot openverse-bot moved this from 📅 To Do to 🗑 Discarded in Openverse Backlog Dec 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
💻 aspect: code Concerns the software code in the repository 🛠 goal: fix Bug fix help wanted Open to participation from the community 🟩 priority: low Low priority and doesn't need to be rushed 🧱 stack: ingestion server Related to the ingestion/data refresh server 🐍 tech: python Involves Python 🐛 tooling: sentry Sentry issue
Projects
Archived in project
Development

No branches or pull requests

2 participants