Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run automatic performance tests #3783

Merged

Conversation

leonardehrenfried
Copy link
Member

@leonardehrenfried leonardehrenfried commented Dec 14, 2021

Summary

This runs the performance test on a self-hosted Github Actions Runner.

It uses Micrometer to collect the measurements and then sends them to a InfluxDB instance and uses Grafana to visualise the metrics.

You can see what it looks like here: https://otp-performance.leonard.io/dashboard/snapshot/0VhIxrgdrSnTzmk5ZUWNQZlraGIhOUrD?orgId=1

Screenshot from 2021-12-14 11-07-51

Note: Before we merge this, we need to associate the runner with the OTP repo.

The runner, InfluxDB and Grafana is setup using ansible. The source code for this is here: https://github.com/leonardehrenfried/github-actions-runner-ansible

Do we want to move the code to the OTP org?

Issue

Relates to #3748

Unit tests

n/a

Code style

yes

Documentation

Some documentation added.

@leonardehrenfried leonardehrenfried requested a review from a team as a code owner December 14, 2021 10:10
@leonardehrenfried
Copy link
Member Author

@t2gran Right now the fixed Netex data set is hosted on my own server: https://leonard.io/otp/rb_norway-aggregated-netex-2021-12-11.zip

If you want to put that into Entur storage that would be lovely.

@t2gran t2gran added this to the 2.1 milestone Dec 14, 2021
@leonardehrenfried leonardehrenfried force-pushed the self-hosted-runner branch 5 times, most recently from f31e8cb to bcc0d52 Compare December 16, 2021 12:03
.gitignore Outdated Show resolved Hide resolved
@t2gran
Copy link
Member

t2gran commented Jan 5, 2022

FYI: I have done some changes to Raptor in #3798 witch might cause the SpeedTest to fail.

@hannesj hannesj self-requested a review January 6, 2022 14:43
@hannesj
Copy link
Contributor

hannesj commented Jan 6, 2022

  • Utilize three registries, one for warmup, one for actual test and third for reporting. The one in the test should push it to the reporting one.
  • Increase interval for pushing to influxDB
  • Use tags to separate test cases

@@ -3,27 +3,17 @@ testCaseId,description,departure,arrival,window,origin,fromPlace,fromLat,fromLon
2,NSB,11:00,15:00,1:00,Gol,NSR:StopPlace:324,60.69917,8.97123,Billingstad,NSR:StopPlace:577,59.874783,10.483057
3,Ruter,15:00,17:00,1:30,IKEA Slependen,NSR:StopPlace:4401,59.87577,10.500488,Oslo Bussterminal,NSR:StopPlace:6505,59.91177,10.757994
4,Østold,16:00,20:30,1:00,Sponvika,NSR:StopPlace:3009,59.09267,11.229988,Skjærhalden,NSR:StopPlace:1845,59.025497,11.036771
5,Østold,13:00,14:30,1:00,Ørje busstorg,NSR:StopPlace:911,59.480717,11.656314,Skjønhaug torv,NSR:StopPlace:1005,59.641224,11.315405
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Revert this file

@leonardehrenfried leonardehrenfried merged commit 346781a into opentripplanner:dev-2.x Jan 6, 2022
@leonardehrenfried leonardehrenfried deleted the self-hosted-runner branch January 6, 2022 14:53
t2gran pushed a commit that referenced this pull request Jan 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants