Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create a website where we display the latest code coverage results for our testsuite #546

Closed
fingolfin opened this issue Jan 27, 2016 · 12 comments
Assignees
Labels
topic: tests issues or PRs related to tests

Comments

@fingolfin
Copy link
Member

Building upon #545 there should be a website where the display the latest code coverage results. This website should be updated regularly (ideally on every commit, but more realistic is, say, every 24 hours).

I am not sure whether we can use Travis for that (probably not, due to runtime limitations), but perhaps a Jenkins task could be setup for that? The last step of that task could be to copy the created HTML files to a location where a webserver can access them. I.e. it is not even necessary to open Jenkins up to the public.

@fingolfin fingolfin added the topic: tests issues or PRs related to tests label Jan 27, 2016
@olexandr-konovalov olexandr-konovalov self-assigned this Jan 28, 2016
@olexandr-konovalov
Copy link
Member

Assigned to myself - Jenkins is certainly the right tool here. I will need help from John for the last step.

@olexandr-konovalov
Copy link
Member

For the record, by now I have dedicated Jenkins slave in the cloud, which produces the coverage report. It remains to sort out the publishing part.

@olexandr-konovalov
Copy link
Member

I will add some starting page with the description, but for now http://gap-coverage.northeurope.cloudapp.azure.com/ already reports coverage in testinstall.g and testall.g when GAP (master branch as on this evening) started with -r -A. Just click on links to directories to see coverage reports.

@hungaborhorvath
Copy link
Contributor

Thank you, this is really great! I have one question, though. What is the difference between the green and the red lines? (In particular, how do you get a red line? I only got green lines so far.) I did not seem to find a legend for this anywhere in the homepage of the profiling package.

@ChrisJefferson
Copy link
Contributor

A bright red green line is executed.

A bright red line is NOT executed.

Lines which are neither bright red or bright green are not code which can be executed (you might think those lines were executed, this is to do with how GAP internally represents code).

The important thing is, get rid of bright red lines by adding tests!

@olexandr-konovalov
Copy link
Member

For the first time in history - coverage report for the reference manual: http://gap-coverage.northeurope.cloudapp.azure.com/coverage-testmanuals/

@olexandr-konovalov
Copy link
Member

And now the next part: http://gap-coverage.northeurope.cloudapp.azure.com/coverage-total/ contains merged coverage report for teststandard and the reference manual

@ChrisJefferson
Copy link
Contributor

@alex-konovalov : Can you update http://gap-coverage.northeurope.cloudapp.azure.com/coverage-total/ ? (or send me the script you made it with)?

@olexandr-konovalov
Copy link
Member

@ChrisJefferson It's run via Jenkins and needs no my intervention to be updated!!! ;-)

As soon as the test passes successfully, it copies all to http://gap-coverage.northeurope.cloudapp.azure.com/ - the date of the creation of the directories that you may see there is 2016-02-23 19:48 so this is last night's build.

It's only when tests fail that the HTML will not be copied over.

Next update will be tomorrow, after nightly tests will start at 6pm.

@olexandr-konovalov
Copy link
Member

http://gap-coverage.northeurope.cloudapp.azure.com/ now has the fresh version, produced using the json update picked up this morning. Coloured cells indicating % of the coverage looks great!

@markuspf
Copy link
Member

markuspf commented Jul 2, 2016

This can now be closed, we have codecov.io coverage.

@markuspf
Copy link
Member

markuspf commented Jul 2, 2016

Here is a link: https://codecov.io/gh/gap-system/gap

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic: tests issues or PRs related to tests
Projects
None yet
Development

No branches or pull requests

5 participants