Delphi train after create model #489
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Collaborator
manujinda
commented
Mar 26, 2021
manujinda
commented
- Train model just after create model
- Train and run experiment asynchronously in separate processes. (One problem is that no other process waits for those processes and hence the process IDs associated with each such process ends up allocated till the Flask server is terminated)
added 12 commits
March 13, 2021 15:17
1. Made training occur asynchronously on a separate process. Since this process is independent of Flask, it does not have access to SQL Alhemy. So, added a new source file, database.cpp, to handle database access and added a new method, AnalysisGraph::write_model_to_db() to write the trained model back to the database.
Earlier test_rest_api.py test script created a temporary database /tmp/test.db for testing and dropped all the tables t the end of the test. Noew it uses the main delphi.db and at the end of the test deletes gthe test related rows added to the database. Added some error detection code to the test script.
On 2021/03/15 version1.4 had been released. It creates an error. pallets-eco/flask-sqlalchemy#885 https://stackoverflow.com/questions/66648383/sqlalchemy-1-4-0-greenlet-greenlet-object-error 2. Add lines to remove test rows added to experimentresults table in test_rest_api.py 3. Reduced the number of samples burned and retained in test_cpp_evauation.py
Version I have locally
Variable expansion does not seem to work with github yml script.
This is needed to send the sampled beta distributions for each edge to CauseMos. Thetas are converted to betas before sending by taking tan(theta).
1. How a new process should be created (spawn, fork etc.) should only be specified once per script. The way I had included this call makes that line executed multiple times. Fixed it. 2. Figured out how to pass the Flask app context to a user created process and updated the code to do so. Now we do not have to rely on C++ to update the database. We can use SQLAlchemy and do it within the user created process. 3. Made create experiment utilize a user created process to make that process asynchronous and to quickly send the create model response to the user. 4. TO TEST THE CREATE MODEL ASYNCHRONOUS BEHAVIOR, ARTIFICIALLY SLOWED DOWN THE AnalysisGraph::generate_latent_state_sequences() method in prediction.cpp BY INTRODUCIG A SLEEP(). *** REMOVE THAT BEFORE MERGING TO MASTER ***.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.