Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add sync endpoint benchmarks #661

Draft
wants to merge 4 commits into
base: santiagopittella-stress-testing-copy
Choose a base branch
from

Conversation

SantiagoPittella
Copy link
Collaborator

Part of #609

I used @TomasArrachea approach to test the SyncState endpoint and copied some of his implementation.

It adds the tracing used in the node, to ease debugging.

For 10k request, this is the current output:

Total sync request took: 16.134835583s
Average sync request took: 1.612588ms

I also added some methods to the store to initialize an existing DB because Store::init always tries to run the migrations and that was causing runtime errores. The node binary uses Store::init without any errors when using the miden-node start command; I could not understand why it wasn't failing there. Maybe my new methods (and I'm pretty sure) were not needed. Maybe @Mirko-von-Leipzig @polydez can help me understand why.

@Mirko-von-Leipzig
Copy link
Contributor

I also added some methods to the store to initialize an existing DB because Store::init always tries to run the migrations and that was causing runtime errores. The node binary uses Store::init without any errors when using the miden-node start command; I could not understand why it wasn't failing there. Maybe my new methods (and I'm pretty sure) were not needed. Maybe @Mirko-von-Leipzig @polydez can help me understand why.

Could you add the errors you're getting?

In general the database setup should only apply new migrations. So an existing database won't have any new ones - and we only have a single migration at the moment.

It does always log that its doing migrations though so maybe that's just confusing?

INFO apply_migrations: miden-store: Running database migrations, version_before: 1 (inside)
INFO apply_migrations: miden-store: Finished database migrations, version_after: 1 (inside)

@SantiagoPittella
Copy link
Collaborator Author

@Mirko-von-Leipzig this is the error:

2025-02-03T13:16:44.750591Z  INFO apply_migrations: miden-store: crates/store/src/db/migrations.rs:27: new
2025-02-03T13:16:44.750632Z  INFO apply_migrations: miden-store: crates/store/src/db/migrations.rs:31: Running database migrations, version_before: 1 (inside)
2025-02-03T13:16:44.750741Z ERROR apply_migrations: miden-store: crates/store/src/db/migrations.rs:47: Schema version mismatch, last_schema_version: 13, current_schema_version: 14
2025-02-03T13:16:44.750746Z ERROR apply_migrations: miden-store: crates/store/src/db/migrations.rs:27: error: unsupported database version. There is no migration chain from/to this version. Remove all database files and try again.
2025-02-03T13:16:44.750749Z  INFO apply_migrations: miden-store: crates/store/src/db/migrations.rs:27: close, time.busy: 146µs, time.idle: 13.1µs
2025-02-03T13:16:44.750793Z  INFO setup: miden-store: crates/store/src/db/mod.rs:181: close, time.busy: 216µs, time.idle: 3.22ms

@Mirko-von-Leipzig
Copy link
Contributor

@Mirko-von-Leipzig this is the error:

...

Hmm. I don't know this code particularly well. My local node's schema version is 13, but I'm unsure why your's would jump to 14. @polydez should be more familiar with this code

@SantiagoPittella
Copy link
Collaborator Author

Putting this as draft until block and batch production changes.

@SantiagoPittella SantiagoPittella marked this pull request as draft February 4, 2025 13:27
@polydez
Copy link
Contributor

polydez commented Feb 5, 2025

2025-02-03T13:16:44.750741Z ERROR apply_migrations: miden-store: crates/store/src/db/migrations.rs:47: Schema version mismatch, last_schema_version: 13, current_schema_version: 14

According to the error message in logs, the database structure was changed outside of migrations workflow. If you need to update DB schema, you should update 000-init.sql. Another way is to add new migration from the initial schema, but we haven't used it yet (we're going to start using it later).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants