Evaluate Profile-Guided Optimization (PGO) and LLVM BOLT #244
Replies: 5 comments
-
Hi @zamazan4ik, Thanks for the effort and the adjusted tests. I need to evaluate whether or not this might be potentially beneficial for the project. |
Beta Was this translation helpful? Give feedback.
-
BOLT has caused many problems when combined with other opts such as PGO in the upstream Rust toolchain. I recommend applying caution about layering PGO, BOLT, strip... we get away with it partly by keeping close in touch with a gaggle of assorted LLVM and compiler experts, not something I'd wish on a smaller project. PGO should be fine to apply on its own. |
Beta Was this translation helpful? Give feedback.
-
Thanks a lot for the wonderful feedback/guidance @workingjubilee ❤️ ! |
Beta Was this translation helpful? Give feedback.
-
Update: I've just refactored / updated the CI workflows (#240) and will try to find some time to tackle this issue. |
Beta Was this translation helpful? Give feedback.
-
After giving it some more thoughts, the main issue at the moment is on the cross-compilation side since the project uses
TL;DR: I'll keep this thread as a discussion but won't do anything yet. |
Beta Was this translation helpful? Give feedback.
-
Hi!
Recently I did many Profile-Guided Optimization (PGO) benchmarks on multiple projects - the results are available here. So that's why I think it's worth trying to apply PGO to
jql
. I already performed some benchmarks and want to share my results here.Test environment
jql
version: the latest for now from themain
branch on commit7729cbafaea0367c2f86227234fb3f8e9a8fd905
Benchmark setup
For benchmarking purposes, I use the scenario from https://github.com/yamafaktory/jql/blob/main/performance.sh , just a bit tweaked (removed vanilla
jq
invocations, add multiplejql
versions to the script) - edited version is available here. Release build is done withcargo build --release
, PGO optimized build is done with cargo-pgo.I tested 3 configurations:
jql_z
opt-level = 3
), binary calledjql_opt_3
jql_optimized
PGO profiles were collected from the same workload in
performance.sh
and merged viallvm-profdata
during the PGO optimization phase.All benchmarks are done multiple times, on the same hardware/software setup, with the same background "noise" (as much I can guarantee ofc).
Results
I got the following results from running
performance.sh
:The same results in
performance.md
format:According to the tests, sometimes
user
time is significantly decreased with the PGO optimization.Further steps
I can suggest the following things to do:
jql
in more scenarios.Here are some examples of how PGO is already integrated into other projects' build scripts:
configure
scriptAfter PGO, I can suggest evaluating LLVM BOLT as an additional optimization step after PGO.
Beta Was this translation helpful? Give feedback.
All reactions