Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improvement: add new ARGILLA_DATABASE_SQLITE_TIMEOUT environment variable setting #5213

Merged
merged 5 commits into from
Jul 12, 2024

Conversation

jfcalvo
Copy link
Member

@jfcalvo jfcalvo commented Jul 11, 2024

Description

On the distribution task testing effort meeting we found some errors raised by SQLite when the database was locked because too many writes were executed concurrently.

Increasing the timeout for SQLite reduces this problem, specifically changing the connection timeout parameter from the default value of 5 seconds to 30 seconds make the problem totally disappear locally.

In this PR I have added a new ARGILLA_DATABASE_SQLITE_TIMEOUT environment variable that allow us to set a value for this. With a default value of 15 seconds.

The idea is to test this change on Spaces using a value of 30 seconds.

Refs #5000

Type of change

  • Improvement (change adding some improvement to an existing functionality)

How Has This Been Tested

  • Tested locally using SQLite and running 20 parallel response creations in bulk.

Checklist

  • I added relevant documentation
  • I followed the style guidelines of this project
  • I did a self-review of my code
  • I made corresponding changes to the documentation
  • I confirm My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I have added relevant notes to the CHANGELOG.md file (See https://keepachangelog.com/)

@jfcalvo jfcalvo requested a review from frascuchon July 11, 2024 15:25
Copy link

codecov bot commented Jul 11, 2024

Codecov Report

Attention: Patch coverage is 86.66667% with 2 lines in your changes missing coverage. Please review.

Project coverage is 91.94%. Comparing base (cce4403) to head (f6b7972).

Files Patch % Lines
argilla-server/src/argilla_server/settings.py 83.33% 2 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #5213      +/-   ##
===========================================
- Coverage    91.96%   91.94%   -0.02%     
===========================================
  Files          135      135              
  Lines         5822     5835      +13     
===========================================
+ Hits          5354     5365      +11     
- Misses         468      470       +2     
Flag Coverage Δ
argilla-server 91.94% <86.66%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@jfcalvo jfcalvo changed the title improvement: add new ARGILLA_DATABASE_TIMEOUT environment variable setting improvement: add new ARGILLA_DATABASE_SQLITE_TIMEOUT environment variable setting Jul 12, 2024
@jfcalvo jfcalvo merged commit 5cdd7f5 into develop Jul 12, 2024
5 of 6 checks passed
@jfcalvo jfcalvo deleted the improvement/add-database-timeout-setting branch July 12, 2024 10:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants