Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add timeout to NLP resolver #7

Open
joshua-stauffer opened this issue May 27, 2021 · 0 comments
Open

Add timeout to NLP resolver #7

joshua-stauffer opened this issue May 27, 2021 · 0 comments
Labels
enhancement New feature or request

Comments

@joshua-stauffer
Copy link
Owner

When the NLP service receives a query to transform text, it first runs the text through a ML model, and then sends subqueries out to the Geo and ReadModel services in parallel. The issues is that if one of those services are down, a result will never return, and the NLP service will keep a reference to that task in memory indefinitely. This isn't an issue for the end user, since the API service will automatically timeout requests that take too long (currently 5 seconds). I'd like to add similar logic to the NLP Resolver class, and automatically remove queries that exceed a constant timeout.

Another factor is that the NLP processing itself can be very time consuming. A nice solution may be to refactor the text processing call from the base NLPService class into the Resolver class, and initialize a TTL value on creation. Then, after each processing step the TTL could be checked and the process aborted if the timeout has been exceeded.

@joshua-stauffer joshua-stauffer added the enhancement New feature or request label Jun 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant