Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change how initial demo languages are loaded #534

Closed
20 tasks done
jzohrab opened this issue Dec 7, 2024 · 4 comments
Closed
20 tasks done

Change how initial demo languages are loaded #534

jzohrab opened this issue Dec 7, 2024 · 4 comments
Assignees
Labels
code-improvement Make code/structure better fixed Fixed in develop or master, to be launched.

Comments

@jzohrab
Copy link
Collaborator

jzohrab commented Dec 7, 2024

Currently, languages are pre-loaded into the startup database. This is a hassle sometimes, as changing a language's stories (e.g the Tutorial) can cause the entire DB file to change, even though it's only data that's changed. The data exists in the language def files, there's no need for that thrash to impact the db diffs as well. This will also simplify database versioning, I think.

To do:

  • add a system flag, something like 'shouldLoadDemoData', which is only set for the new database
  • on app startup, check that flag, and load from the appropriate def files
  • once app is started, remove 'shouldLoadDemoData' flag
  • if any data exists, nothing should happen even if 'shouldLoadDemoData' flag is present - can't write over existing stuff
  • get all tests passing again
  • look into speeding up the language defs load -- can probably just load all of the yaml only and cache it in a dict
  • move all existing tests from stub file to the test_demo.py
  • add any remaining tests
  • remove stub test file test_issue_534 ... for db demo redesign
  • change the LoadDemoData flag, should only be present or deleted
  • add message, "loading sample data"
  • test release with new kind of empty db, try install and see if works
  • delete install, remove mecab, do install, see if a lang is marked as a preload but its parser isn't supported then we shouldn't do that
  • try running install with a copy of my own prod data
  • change db.reset to use new baseline stuff, remove old baseline stuff
  • remove db.reset inv task?
  • fix docs, have to re-baseline
  • change the db.demo functions to a service object, simplifies importing
  • remove empty.db, db_export_empty(c), def db_wipe(c), test for db wipe?, check for "wipe", "empty"
  • final beta release and fresh install?
@jzohrab jzohrab added the code-improvement Make code/structure better label Dec 7, 2024
@jzohrab jzohrab added this to Lute-v3 Dec 7, 2024
@jzohrab jzohrab moved this to In Progress in Lute-v3 Dec 8, 2024
@jzohrab
Copy link
Collaborator Author

jzohrab commented Dec 8, 2024

Started work in issue_534_change_initial_demo_data_load branch. Currently 127 pytest tests fail, as the baseline db no longer has the demo languages and books. pytest -xand fix the rest.

@jzohrab jzohrab self-assigned this Dec 8, 2024
@jzohrab
Copy link
Collaborator Author

jzohrab commented Dec 8, 2024

re speeding up the tests: test perf chars:

this branch
unit: 393 passed in 115.20s (0:01:55)
accept: 45 passed in 209.42s (0:03:29)

develop
unit: 385 passed in 46.24s
accept: 45 passed in 210.53s (0:03:30)

so the unit tests are quite a bit slower --- update, caching worked:

unit 393 passed in 51.56s
accept 45 passed in 201.66s (0:03:21)

@jzohrab
Copy link
Collaborator Author

jzohrab commented Dec 10, 2024

All seems good, opening PR for CI check.

@jzohrab jzohrab added the fixed Fixed in develop or master, to be launched. label Dec 13, 2024
@jzohrab jzohrab moved this from In Progress to Done in Lute-v3 Dec 13, 2024
@jzohrab
Copy link
Collaborator Author

jzohrab commented Dec 13, 2024

Released in 3.6.1.

@jzohrab jzohrab closed this as completed Dec 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
code-improvement Make code/structure better fixed Fixed in develop or master, to be launched.
Projects
Archived in project
Development

No branches or pull requests

1 participant