Add parameter to execute inference requests at a fixed frequency #26820
Conversation
boonsino
requested review from
ilya-lavrenov
and removed request for
a team
boonsino
force-pushed
the
add-benchmark-test-fps-arg
branch
from
c962946
to
e22a77d
Compare
| @@ -65,6 +65,11 @@ static const char cache_dir_message[] = "Optional. Enables caching of loaded mod | |||
| static const char load_from_file_message[] = "Optional. Loads model from file directly without read_model." | |||
| " All CNNNetwork options (like re-shape) will be ignored"; | |||
|
|
|||
| /// @brief message for run frequency | |||
| static const char run_frequency_message[] = | |||
There was a problem hiding this comment.
in serving it's typically named "requests rate"
| /// @brief message for run frequency | ||
| static const char run_frequency_message[] = | ||
| "Execute at a fixed frequency. Note if the targeted rate per second cannot be reached, " | ||
| "the benchmark would start the next run immediately, trying its best to catch up."; |
There was a problem hiding this comment.
Every iteration of the main loop starts with inferRequest = inferRequestsQueue.get_idle_request(); which blocks. That means the benchmark would not start the next run immediately.
samples/cpp/benchmark_app/main.cpp
Outdated
There was a problem hiding this comment.
Please, keep C++ and Python implementations aligned
samples/cpp/benchmark_app/main.cpp
Outdated
There was a problem hiding this comment.
Add a test to test_benchmark_app.py https://github.com/openvinotoolkit/openvino/tree/master/tests/samples_tests/smoke_tests
samples/cpp/benchmark_app/main.cpp
Outdated
There was a problem hiding this comment.
| @@ -307,6 +312,9 @@ DEFINE_string(api, "async", api_message); | |||
| /// @brief Number of infer requests in parallel | |||
| DEFINE_uint64(nireq, 0, infer_requests_count_message); | |||
|
|
|||
| /// @brief Execute infer requests at a fixed frequency | |||
| DEFINE_double(rfreq, 0, run_frequency_message); | |||
There was a problem hiding this comment.
The default could be infinity. That way you don't need to handle 0 separately.
There was a problem hiding this comment.
If set to INFINITY, then the function if (FLAGS_rfreq > 0) will still get executed which is redundant right? Having default as 0 would simply skip that and runs without delay. What do you think?
There was a problem hiding this comment.
I don't know for sure. You should try. I'd expect sleep_for to have the similar logic inside: if <=0 then return.
samples/cpp/benchmark_app/main.cpp
Outdated
There was a problem hiding this comment.
@nkogteva, what were your reasons against limiting infer requests rate introduction?
|
build_jenkins |
boonsino
force-pushed
the
add-benchmark-test-fps-arg
branch
from
1762638
to
551caa3
Compare
github-actions
bot
added
category: docs
boonsino
force-pushed
the
add-benchmark-test-fps-arg
branch
2 times, most recently
from
fa49ebc
to
91982ac
Compare
Signed-off-by: Maciej Falkowski <maciej.falkowski@intel.com> Signed-off-by: Ooi, Boon Sin <boon.sin.ooi@intel.com>
boonsino
force-pushed
the
add-benchmark-test-fps-arg
branch
from
91982ac
to
ab68729
Compare
Signed-off-by: Ooi, Boon Sin <boon.sin.ooi@intel.com>
boonsino
force-pushed
the
add-benchmark-test-fps-arg
branch
from
ab68729
to
7040c62
Compare
Details:
Tickets: