Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Subgraph service: Don't instantiate a new pool for each request #3435

Closed
o0Ignition0o opened this issue Jul 13, 2023 · 3 comments · Fixed by #3470
Closed

Subgraph service: Don't instantiate a new pool for each request #3435

o0Ignition0o opened this issue Jul 13, 2023 · 3 comments · Fixed by #3470
Assignees

Comments

@o0Ignition0o
Copy link
Contributor

We're currently recreating a connector and a pool for each subgraph_service instantiation.

We shouldn't.
While we refactor it, let's make sure we don't forget to set a pool idle timeout duration.

@garypen garypen self-assigned this Jul 19, 2023
@garypen
Copy link
Contributor

garypen commented Jul 19, 2023

We only build a new SubgraphService at startup and during re-configuration. This is done once for each subgraph, so each subgraph has its own http client (and associated pool). We don't build a SubgraphService for each request.

We may want to modify the code so that we only have one connector/pool to share across all subgraphs, but I think it's fine to have a separate connector/pool per subgraph, since we may want to vary behaviour between subgraphs at some point.

I'll add a pool idle timeout.

garypen added a commit that referenced this issue Jul 19, 2023
Having a high idle pool timeout duration can sometimes trigger situations in which an HTTP request cannot complete (see [this comment](hyperium/hyper#2136 (comment)) for more information).

This changeset sets a default timeout duration of 5 seconds, which we may make configurable eventually.

fixes: #3435
@bnjjj
Copy link
Contributor

bnjjj commented Jul 19, 2023

@garypen I think for every request we build a new subgraph_service, look here

@bnjjj
Copy link
Contributor

bnjjj commented Jul 19, 2023

Ok after some investigations we don't call SubgraphService::new on every request we're calling the SubgraphServiceFactory which clone the subgraph service. And cloning hyper::Client won't create a new pool, it's an Arc<Mutex<>> under the hood and so it doesn't create a new connection pool. We're good !

@garypen garypen changed the title Subgraph service: Don't instanciate a new pool for each request Subgraph service: Don't instantiate a new pool for each request Jul 19, 2023
garypen added a commit that referenced this issue Jul 20, 2023
Having a high idle pool timeout duration can sometimes trigger
situations in which an HTTP request cannot complete (see [this
comment](hyperium/hyper#2136 (comment))
for more information).

This changeset sets a default timeout duration of 5 seconds, which we
may make configurable eventually.

fixes: #3435

<!-- start metadata -->

**Checklist**

Complete the checklist (and note appropriate exceptions) before a final
PR is raised.

- [x] Changes are compatible[^1]
- [x] Documentation[^2] completed
- [x] Performance impact assessed and acceptable
- Tests added and passing[^3]
    - [ ] Unit Tests
    - [ ] Integration Tests
    - [ ] Manual Tests

**Exceptions**

*Note any exceptions here*

**Notes**

[^1]. It may be appropriate to bring upcoming changes to the attention
of other (impacted) groups. Please endeavour to do this before seeking
PR approval. The mechanism for doing this will vary considerably, so use
your judgement as to how and when to do this.
[^2]. Configuration is an important part of many changes. Where
applicable please try to document configuration examples.
[^3]. Tick whichever testing boxes are applicable. If you are adding
Manual Tests:
- please document the manual testing (extensively) in the Exceptions.
- please raise a separate issue to automate the test and label it (or
ask for it to be labeled) as `manual test`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants