perf: Allow covering relation queries with minimal index #6581
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
My actual changes were quite difficult to find when buried in this sea of style changes, which were getting automatically applied during a pre-commit hook. Here I just run the hooks against the files I'm going to be touching in the following commit, so that a reviewer can ignore these automatically generated diffs and just view the meaningful commit.
When finding objects through a relation, we're sending Mongo queries
that look like this:
```
db.getCollection('_Join:foo:bar').find({ relatedId: { $in: [...] } });
```
From the result of that query, we're only reading the `owningId` field,
so we can start by adding it as a projection:
```
db.getCollection('_Join:foo:bar')
.find({ relatedId: { $in: [...] } })
.project({ owningId: 1 });
```
This seems like the perfect example of a query that could be satisfied
with an index scan: we are querying on one field, and only need one
field from the matching document.
For example, this can allow users to speed up the fetching of user roles
in authentication, because they query a `roles` relation on the `_Role`
collection. To add a covering index on that, you could now add an index
like the following:
```
db.getCollection('_Join:roles:_Role').createIndex(
{ relatedId: 1, owningId: 1 },
{ background: true }
);
```
One caveat there is that the index I propose above doesn't include the
`_id` column. For the query in question, we don't actually care about
the ID of the row in the join table, just the `owningId` field, so we
can avoid some overhead of putting the `_id` column into the index if we
can also drop it from the projection. This requires adding a small
special case to the MongoStorageAdapter, because the `_id` field is
special: you have to opt-out of using it by projecting `{ _id: 0 }`.
Codecov Report
@@ Coverage Diff @@
## master #6581 +/- ##
==========================================
- Coverage 93.89% 93.80% -0.10%
==========================================
Files 169 169
Lines 11968 11970 +2
==========================================
- Hits 11237 11228 -9
- Misses 731 742 +11
Continue to review full report at Codecov.
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When finding objects through a relation, we're sending Mongo queries that look like this:
From the result of that query, we're only reading the
owningIdfield, so we can start by adding it as a projection:This seems like the perfect example of a query that could be satisfied with an index scan: we are querying on one field, and only need one field from the matching document.
For example, this can allow users to speed up the fetching of user roles in authentication, because they query a
rolesrelation on the_Rolecollection. To add a covering index on that, you could now add an index like the following:One caveat there is that the index I propose above doesn't include the
_idcolumn. For the query in question, we don't actually care about the ID of the row in the join table, just theowningIdfield, so we can avoid some overhead of putting the_idcolumn into the index if we can also drop it from the projection. This requires adding a small special case to the MongoStorageAdapter, because the_idfield is special: you have to opt-out of using it by projecting{ _id: 0 }.It appears that there is also now an automatic commit hook that applies some linting and styling rules, so I also broke this PR up into two commits: one that just applies those automatic fixes, and one that then makes the actual change. Hopefully this makes. it easier to review the diff!