benchmark async vs. sync http providers #2002
Conversation
wolovim
force-pushed
the
benchmark-async
branch
4 times, most recently
from
5ff63a6 to
3bbc870
Compare
wolovim
changed the title
| loop = asyncio.get_event_loop() | ||
| async_w3_http = loop.run_until_complete(build_async_w3_http(fixture.endpoint_uri)) | ||
|
|
||
| methods = [ |
There was a problem hiding this comment.
Maybe worth pulling this out into a constant?
There was a problem hiding this comment.
There's probably a good way to clean this up a bit, but the methods rely on the providers built within this context. Will revisit next iteration.
| basepython=python | ||
| commands= | ||
| python {toxinidir}/web3/tools/benchmark/main.py --num-calls 5 | ||
| python {toxinidir}/web3/tools/benchmark/main.py --num-calls 50 |
There was a problem hiding this comment.
Just curious - what's the reasoning behind testing 5, 50, and 100 calls here and not just checking one?
There was a problem hiding this comment.
Once the suite is more mature, we should have vectors for benchmarking (the average of) 1 call vs. 1 call; this would be useful for comparing the HTTPProvider vs. IPCProvider, for example. I think making many calls is the best way to showcase the difference between AsyncHTTPProvider vs. HTTPProvider performance.
What was wrong?
Async http provider is under way; time to benchmark vs. the sync provider.
Update: turns out not much in the case of our CI resources and a locally running Geth node. Local tests with a remote node provider produced more dramatic results.
I think this is a reasonable starting point; the scope of benchmarks can grow from here.
To run the benchmarks locally:
make benchmarkTodo:
Cute Animal Picture