Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

H-3316: Flatten apps/hash-graph workspace hierarchy #5033

Merged

Conversation

TimDiekmann
Copy link
Member

@TimDiekmann TimDiekmann commented Sep 9, 2024

🌟 What is the purpose of this PR?

turbo does not properly detect dependencies and it turned out this is because of nested workspaces. This PR re-organizes the apps/hash-graph folder to remove the workspace nesting.

🔍 What does this change?

  • Move Postgres-migrations to graph crate
  • Move OpenAPI generation to api-crate
  • Move status type definitions to own crate type-defs
  • Move http-tests to tests/ folder
  • Rename apps/hash-graph/bins/cli to @apps/hash-graph
  • Allow package name to be set in sync-turborepo script

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

  • does not modify any publishable blocks or libraries, or modifications do not need publishing

📜 Does this require a change to the docs?

The changes in this PR:

  • are internal and do not require a docs change

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

  • affected the execution graph, and the turbo.json's have been updated to reflect this

@github-actions github-actions bot added area/deps Relates to third-party dependencies (area) area/apps > hash* Affects HASH (a `hash-*` app) area/infra Relates to version control, CI, CD or IaC (area) type/eng > backend Owned by the @backend team area/tests New or updated tests area/apps area/apps > hash-graph type/legal Owned by the @legal team labels Sep 9, 2024
…-not-detected-by-turborepo

# Conflicts:
#	apps/hash-graph/libs/api/openapi/openapi.json
@github-actions github-actions bot added the area/libs Relates to first-party libraries/crates/packages (area) label Sep 9, 2024
@github-advanced-security
Copy link

This pull request sets up GitHub code scanning for this repository. Once the scans have completed and the checks have passed, the analysis results for this pull request branch will appear on this overview. Once you merge this pull request, the 'Security' tab will show more code scanning analysis results (for example, for the default branch). Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results. For more information about GitHub code scanning, check out the documentation.

vilkinsons
vilkinsons previously approved these changes Sep 10, 2024
Copy link

codecov bot commented Sep 10, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 17.84%. Comparing base (b266275) to head (dfb8108).
Report is 126 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5033      +/-   ##
==========================================
+ Coverage   17.79%   17.84%   +0.05%     
==========================================
  Files         506      506              
  Lines       16569    16569              
  Branches     2546     2546              
==========================================
+ Hits         2948     2957       +9     
  Misses      13600    13600              
+ Partials       21       12       -9     
Flag Coverage Δ
apps.hash-ai-worker-ts 1.58% <ø> (ø)
apps.hash-api 1.18% <ø> (ø)
blockprotocol.type-system 55.38% <ø> (+6.92%) ⬆️
local.hash-backend-utils 9.38% <ø> (ø)
local.hash-isomorphic-utils 1.06% <ø> (ø)
local.hash-subgraph 25.22% <ø> (ø)
rust.deer 5.88% <ø> (ø)
rust.error-stack 72.68% <ø> (ø)
rust.sarif 87.83% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

Benchmark results

@rust/graph-benches – Integrations

scaling_read_entity_complete_one_depth

Function Value Mean Flame graphs
entity_by_id 10 entities $$51.6 \mathrm{ms} \pm 230 \mathrm{μs}\left({\color{gray}0.135 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 50 entities $$1.55 \mathrm{s} \pm 3.20 \mathrm{ms}\left({\color{red}290 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1 entities $$19.8 \mathrm{ms} \pm 75.0 \mathrm{μs}\left({\color{gray}-0.505 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 25 entities $$70.2 \mathrm{ms} \pm 343 \mathrm{μs}\left({\color{gray}-4.146 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 5 entities $$25.4 \mathrm{ms} \pm 267 \mathrm{μs}\left({\color{gray}1.55 \mathrm{\%}}\right) $$ Flame Graph

scaling_read_entity_complete_zero_depth

Function Value Mean Flame graphs
entity_by_id 10 entities $$2.04 \mathrm{ms} \pm 9.63 \mathrm{μs}\left({\color{gray}-0.878 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 50 entities $$3.90 \mathrm{ms} \pm 20.2 \mathrm{μs}\left({\color{gray}-3.515 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1 entities $$1.87 \mathrm{ms} \pm 11.5 \mathrm{μs}\left({\color{gray}0.645 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 25 entities $$2.59 \mathrm{ms} \pm 14.1 \mathrm{μs}\left({\color{lightgreen}-17.041 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 5 entities $$1.91 \mathrm{ms} \pm 14.5 \mathrm{μs}\left({\color{gray}0.305 \mathrm{\%}}\right) $$ Flame Graph

scaling_read_entity_linkless

Function Value Mean Flame graphs
entity_by_id 10 entities $$1.88 \mathrm{ms} \pm 9.25 \mathrm{μs}\left({\color{gray}1.38 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 100 entities $$2.02 \mathrm{ms} \pm 9.15 \mathrm{μs}\left({\color{gray}1.87 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1000 entities $$2.72 \mathrm{ms} \pm 12.8 \mathrm{μs}\left({\color{gray}-0.935 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1 entities $$1.86 \mathrm{ms} \pm 9.59 \mathrm{μs}\left({\color{gray}0.378 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10000 entities $$12.8 \mathrm{ms} \pm 125 \mathrm{μs}\left({\color{gray}-0.200 \mathrm{\%}}\right) $$ Flame Graph

representative_read_entity_type

Function Value Mean Flame graphs
get_entity_type_by_id Account ID: d4e16033-c281-4cde-aa35-9085bf2e7579 $$1.41 \mathrm{ms} \pm 5.09 \mathrm{μs}\left({\color{gray}0.141 \mathrm{\%}}\right) $$ Flame Graph

representative_read_entity

Function Value Mean Flame graphs
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/organization/v/1 $$16.3 \mathrm{ms} \pm 161 \mathrm{μs}\left({\color{gray}0.517 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/book/v/1 $$15.5 \mathrm{ms} \pm 191 \mathrm{μs}\left({\color{lightgreen}-5.750 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/playlist/v/1 $$15.7 \mathrm{ms} \pm 175 \mathrm{μs}\left({\color{gray}-2.938 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/building/v/1 $$16.2 \mathrm{ms} \pm 156 \mathrm{μs}\left({\color{gray}4.78 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/song/v/1 $$15.8 \mathrm{ms} \pm 173 \mathrm{μs}\left({\color{gray}1.78 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1 $$16.8 \mathrm{ms} \pm 202 \mathrm{μs}\left({\color{red}5.17 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/person/v/1 $$16.0 \mathrm{ms} \pm 177 \mathrm{μs}\left({\color{gray}-0.050 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/block/v/1 $$16.1 \mathrm{ms} \pm 163 \mathrm{μs}\left({\color{lightgreen}-33.771 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/page/v/2 $$15.9 \mathrm{ms} \pm 157 \mathrm{μs}\left({\color{gray}-3.212 \mathrm{\%}}\right) $$ Flame Graph

representative_read_multiple_entities

Function Value Mean Flame graphs
link_by_source_by_property depths: DT=2, PT=2, ET=2, E=2 $$96.4 \mathrm{ms} \pm 463 \mathrm{μs}\left({\color{gray}-0.573 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=0, PT=0, ET=0, E=2 $$77.4 \mathrm{ms} \pm 404 \mathrm{μs}\left({\color{gray}-0.976 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=0, PT=0, ET=0, E=0 $$40.5 \mathrm{ms} \pm 199 \mathrm{μs}\left({\color{gray}-1.077 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=0, PT=0, ET=2, E=2 $$87.8 \mathrm{ms} \pm 529 \mathrm{μs}\left({\color{gray}-0.626 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=0, PT=2, ET=2, E=2 $$93.1 \mathrm{ms} \pm 424 \mathrm{μs}\left({\color{gray}0.653 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=255, PT=255, ET=255, E=255 $$103 \mathrm{ms} \pm 464 \mathrm{μs}\left({\color{gray}-1.105 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=2, PT=2, ET=2, E=2 $$56.3 \mathrm{ms} \pm 357 \mathrm{μs}\left({\color{gray}0.067 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=0, PT=0, ET=0, E=2 $$41.2 \mathrm{ms} \pm 214 \mathrm{μs}\left({\color{gray}-1.117 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=0, PT=0, ET=0, E=0 $$37.2 \mathrm{ms} \pm 170 \mathrm{μs}\left({\color{gray}-0.535 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=0, PT=0, ET=2, E=2 $$47.8 \mathrm{ms} \pm 217 \mathrm{μs}\left({\color{gray}-1.009 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=0, PT=2, ET=2, E=2 $$52.2 \mathrm{ms} \pm 286 \mathrm{μs}\left({\color{gray}-0.562 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=255, PT=255, ET=255, E=255 $$64.3 \mathrm{ms} \pm 210 \mathrm{μs}\left({\color{gray}-1.085 \mathrm{\%}}\right) $$ Flame Graph

Merged via the queue into main with commit 857f89a Sep 10, 2024
156 of 157 checks passed
@TimDiekmann TimDiekmann deleted the t/h-3316-figure-out-why-tests-are-not-detected-by-turborepo branch September 10, 2024 12:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/apps > hash* Affects HASH (a `hash-*` app) area/apps > hash-graph area/apps area/deps Relates to third-party dependencies (area) area/infra Relates to version control, CI, CD or IaC (area) area/libs Relates to first-party libraries/crates/packages (area) area/tests > integration New or updated integration tests area/tests New or updated tests type/eng > backend Owned by the @backend team type/legal Owned by the @legal team
Development

Successfully merging this pull request may close these issues.

3 participants