Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better change detection benchmarking #11217

Open
alice-i-cecile opened this issue Jan 4, 2024 · 0 comments
Open

Better change detection benchmarking #11217

alice-i-cecile opened this issue Jan 4, 2024 · 0 comments
Labels
A-ECS Entities, components, systems, and events C-Benchmarks Stress tests and benchmarks used to measure how fast things are

Comments

@alice-i-cecile
Copy link
Member

What problem does this solve or what need does it fill?

The change detection benchmarks we currently have (#4883) don't accurately cover all of the realistic use cases we care about.\

What solution would you like?

We should cover:

  1. No entity has changed.
  2. Only one entity has changed across many entities.
  3. Only entities within a single archetype has changed, across many archetypes and entities.
  4. Half of the entities have changed, completely at random.
  5. Almost all entities have changed.
  6. Almost all of the archetypes have changed.
  7. All entities have changed.

We should independently measure both the speed of making the mutations, and the speed of detecting the changes.

Additional context

#11173 improves the reliability of the existing benchmarks, and should be merged first.

This is needed by #4882 and #5097, to evaluate the impacts of changes made to address them.

@alice-i-cecile alice-i-cecile added A-ECS Entities, components, systems, and events S-Needs-Triage This issue needs to be labelled C-Benchmarks Stress tests and benchmarks used to measure how fast things are and removed S-Needs-Triage This issue needs to be labelled labels Jan 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-ECS Entities, components, systems, and events C-Benchmarks Stress tests and benchmarks used to measure how fast things are
Projects
None yet
Development

No branches or pull requests

1 participant