Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DOC Document manipulating eager loading queries #456

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 59 additions & 3 deletions en/02_Developer_Guides/00_Model/02_Relations.md
Original file line number Diff line number Diff line change
Expand Up @@ -784,6 +784,11 @@ The N + 1 query problem can be alleviated using eager loading which in this exam
$teams = Team::get()->eagerLoad('Players');
```

> [!CAUTION]
> Manipulating the eager loading query is significantly (up to ~600%) faster than Filtering an `EagerLoadedlist` after the query has been made.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This percentage is based on the middle test in the issue - i.e. used to be 0.9002s, now 0.1264s. Rounded to the nearest 100

>
> See [manipulating eagerloading queries](#manipulating-eager-loading-queries).

With eager loading now only two queries will be executed:

```sql
Expand Down Expand Up @@ -891,10 +896,61 @@ Eager loading supports all relationship types.

> [!WARNING]
> Eager loading is only intended to be used in read-only scenarios such as when outputting data on the front-end of a website. When using default lazy-loading, relationship methods will return a subclass of [`DataList`](api:SilverStripe\ORM\DataList) such as [`HasManyList`](api:SilverStripe\ORM\HasManyList). However when using eager-loading, an [`EagerLoadedList`](api:SilverStripe\ORM\EagerLoadedList) will be returned instead. `EagerLoadedList` has common methods such as `filter()`, `sort()`, `limit()` and `reverse()` available to manipulate its data, as well as some methods you'd expect on the various relation list implementations such as [`getExtraData()`](api:SilverStripe\ORM\EagerLoadedList::getExtraData()).

### Manipulating eager loading queries

There are some limitations to manipulating an `EagerLoadedList` (i.e. after the query has been executed).

The main limitation is that filtering or sorting an `EagerLoadedList` will be done in PHP rather than as part of the database query, since we have already loaded all its relevant data into memory.

> [!WARNING]
> `EagerLoadedList` can't filter or sort by fields on relations using dot notation (e.g. `sort('MySubRelation.Title')` won't work).
>
> Note that filtering or sorting an `EagerLoadedList` will be done in PHP rather than as part of the database query, since we have already loaded all its relevant data into memory.
>
> Note also that `EagerLoadedList` can't filter or sort by fields on relations using dot notation (e.g. `sort('MySubRelation.Title')` won't work).
> Manipulating the eager loading query is significantly (up to ~600%) faster than Filtering an `EagerLoadedlist` after the query has been made.

You can avoid those problems by applying manipulations such as filtering and sorting to the eager loading query as part of your call to `eagerLoad()`.

You can pass an associative array into the `eagerLoad()` method, with relation chains as the keys and callbacks as the values. The callback accepts a `DataList` argument, and must return a `DataList`.

> [!CAUTION]
> You can't manipulate the lists of `has_one` or `belongs_to` relations. This functionality is intended primarily as a way to pre-filter or pre-sort `has_many` and `many_many` relations.

```php
use SilverStripe\ORM\DataList;

$teams = Team::get()->eagerLoad([
'Players' => fn (DataList $list) => $list->filter(['Age:GreaterThan' => 18]),
]);
```

The list passed into your callback represents the query for that relation on *all* of the records you're fetching. For example, the `$list` variable above is a `DataList` that will fetch all `Player` records in the `Players` relation for all `Team` records (so long as they match the filter applied in the callback).

Note that each relation in the relation chain (e.g. `Players`, `Players.Fans`, `Players.Fans.Events`) can have their own callback:

```php
use SilverStripe\ORM\DataList;

$teams = Team::get()->eagerLoad([
'Players' => fn (DataList $list) => $list->filter(['Age:GreaterThan' => 18]),
'Players.Fans' => fn (DataList $list) => $list->filter(['Name:PartialMatch:nocase' => 'Sam']),
'Players.Fans.Events' => fn (DataList $list) => $list->filter(['DollarCost:LessThan' => 200]),
]);
```

If you have complex branching logic, and in some branches you want to avoid pre-filtering or perform different filtering, you can declare a different callback (or `null`) for that relation chain by calling `eagerLoad()` again. Note that subsequent callbacks *replace* previous callbacks.

```php
use SilverStripe\ORM\DataList;

$teams = Team::get()->eagerLoad([
// Remove any callback that was previously defined for this relation chain
'Players' => fn (DataList $list) => null,
// Replace any callback that was previously defined for this relation chain.
// If you want to apply *additional* filters rather than replacing existing ones, you will
// need to declare the original filter again here.
'Players.Fans.Events' => fn (DataList $list) => $list->filter(['DollarCost:GreaterThan' => 100]),
]);
```

## Cascading deletions

Expand Down
29 changes: 29 additions & 0 deletions en/08_Changelogs/5.2.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,35 @@ Check out [the linkfield documentation](/optional_features/linkfield/) which inc

This release comes jampacked with new ORM features, granting you access to some new abstractions for more powerful and efficient queries.

#### Manipulating eager loaded relation queries {#eager-loading}

Filtering or sorting an [`EagerLoadedList`](api:SilverStripe\ORM\EagerLoadedList) (i.e. after the eager loading query has been executed) requires filtering in PHP, which is less powerful and significantly slower than performing those manipulations on `DataList` before executing the query. For example, you can't filter or sort `EagerLoadedList` by fields on relations using dot notation (e.g. `sort('MySubRelation.Title')` won't work).

To alleviate this problem, we've introduced a new syntax for eager loading relations that lets you manipulate the eager loading queries.

The old syntax is still supported, because it can be used in templates for simple scenarios.

In a test setup with looping through 100 records each with 100 related records (for a total of 10,000 records per test run), the following performance improvements were observed for different types of relations (early manipulations in the database vs manipulating results in PHP):

- HasMany - ~581% faster (0.1080s vs 0.7358s)
- ManyMany - ~612% faster (0.1264s vs 0.9002s)
- ManyManyThrough - ~327% faster (0.2511s vs 1.0719s)

You can pass an associative array into the [`DataList::eagerLoad()`](api:SilverStripe\ORM\DataList::eagerLoad()) method, with relation chains as the keys and callbacks as the values. The callback accepts a `DataList` argument, and must return a `DataList`.

```php
use SilverStripe\ORM\DataList;

$teams = Team::get()->eagerLoad([
'Players' => fn (DataList $list) => $list->filter(['Age:GreaterThan' => 18]),
]);
```

> [!WARNING]
> It is very important to remember to return the list in your callback function.

There are a few edge cases to be aware of with this new feature. To learn more, see [eager loading](/developer_guides/model/relations/#eager-loading).

#### Multi-relational `has_one` relations

Traditionally, if you wanted to have multiple `has_many` relations for the same class, you would have to include a separate `has_one` relation for *each* `has_many` relation.
Expand Down
Loading