[1292] Fix detailed results statistics in csv reports

[alien11689]

No description provided.

[codecov]

Codecov Report

Merging #1297 into master will increase coverage by 0.07%.
The diff coverage is 90.90%.

@@            Coverage Diff             @@
##           master    #1297      +/-   ##
==========================================
+ Coverage   79.70%   79.77%   +0.07%     
==========================================
  Files          23       23              
  Lines        2074     2087      +13     
  Branches      321      325       +4     
==========================================
+ Hits         1653     1665      +12     
+ Misses        340      339       -1     
- Partials       81       83       +2     

Impacted Files	Coverage Δ
locust/stats.py	88.18% <90.90%> (-0.02%)	⬇️
locust/contrib/fasthttp.py	88.76% <0.00%> (+0.32%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 17d4ac4...220bced. Read the comment docs.

[cyberw]

Lgtm, but I dont see any tests? Did you maybe miss something in the tests-commit?

[heyman]

This PR does not make sense to me. Why would we be storing a dict of user counts on every StatsEntry (which corresponds to a (URL, HTTP method) pair in the case of HttpLocust)?

[heyman]

I assume (guessing since there is no PR description) that the purpose of this PR is to get a user_count column in the history CSV file? That functionality makes sense, but it would need a different implementation.

[alien11689]

@heyman The purpose of this change is described in this issue #1292

For me, the most important is to have detailed values for percentiles and additionally also information about current amount of spawned users in csv reports.

With my PR I added support for counting users in aggregated csv line. If there is better place then it would be great.

I also added percentiles information, because locust doesn't report them as described in issue. The reported percentiles in PR are accumulated - it would be nice to have them calculated from current time window (configurable).

I need to publish the collected metrics to other place than csv file (e. g. to influx). Do you think that publishing a list of dicts of calculated metrics from stats.py as an event would be a good idea?

[heyman]

For me, the most important is to have detailed values for percentiles

I've opened an issue for this: #1315

also information about current amount of spawned users in csv reports.

This should be a separate issue/PR, and the way to retrieve the current user count is through environment.runner.user_count. One would have to change so that locust.stats.stats_writer() gets a reference to the Environment instance instead of the RequestStats instance.

I need to publish the collected metrics to other place than csv file (e. g. to influx). Do you think that publishing a list of dicts of calculated metrics from stats.py as an event would be a good idea?

You could listen for the request_success and request_failure events and post info to some external DB there.

If you'd rather post the aggregate stats periodically (could make sense in order to not overload the external DB) you could spawn a Greenlet that does this every X second.

Something like this (note: haven't tested it, just wrote it up here, so I might have missed something, but hopefully you get the idea):

import gevent
from locust import events

stats_greenlet = None

@events.test_start.add_listener
def on_test_start(environment, **kw):
    def stats_reporter():
        while True:
            # grab stats from environment.runner.stats here and push to Influx or whatever
            gevent.sleep(1)
    stats_greenlet = gevent.spawn(stats_reporter)

@events.test_stop.add_listener
def on_test_stop(environment, **kw):
    if stats_greenlet:
       stats_greenlet.kill()