Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Possibility to merge different test project results into operncover format #196

Closed
balage90 opened this issue Sep 21, 2018 · 15 comments
Closed

Comments

@balage90
Copy link

Currently if i understand correctly the /p:mergeWith paramter only works in case of Json format.
I'd like to have results in opencover output due to sonarqube alanizis.

We have multiple test for different csproj,-s in our project.

e.g.: dotnet test someProject.csproj --configuration debug /p:CollectCoverage=true /p:CoverletOutput=/TestResultOutput/Coverlet/SomeProject_Merge.xml /p:MergeWith=/TestResultOutput/Coverlet/Merge_All.xml /p:CoverletOutputFormat=opencover

Or if it already possible please correct me.

Br

Balázs

@petli
Copy link
Collaborator

petli commented Sep 21, 2018

If you don't need a merged opencover file, but just a merged coverage report of some kind, you can use https://github.com/danielpalme/ReportGenerator to read multiple opencover.xml files and produce a single report.

@tonerdo
Copy link
Collaborator

tonerdo commented Sep 27, 2018

Hi @balage90, you can simply merge all your coverlet json result files and output them in OpenCover format

@balage90
Copy link
Author

hi @tonerdo exactly thats the solution what i would like to achieve. But maybe i am using it in a wrong way. I could create the merged Json file but what is questionable for me how to "output them into open cover format". I am using dotnet test and for the each test execution it working fine to generate the json and insert it into the merged one. But how can i export it afterwards into open cover? If i understand correctly i had to convert the merged json at the end of the tests execution. Or you mean separetly generate open cover format for each?

@tonerdo
Copy link
Collaborator

tonerdo commented Sep 27, 2018

Here's an example:

dotnet test /p:CollectCoverage=true /p:MergeWith="./other/result.json" /p:CoverletOutputFormat="opencover"

@pape77
Copy link
Contributor

pape77 commented Sep 28, 2018

I'm trying to run this but since i'm doing a dotnet test on the solution which detects what to test, it will say that the result.json file does not exist, as it doesn't create it the first time it runs the test for one of the projects in the solution :/
I'm forced to run tests for one test proj and then run the test solution with the merge param, which results in two steps and double testing a project.

Also see my issue(#203) referring to my attempt to do this with the mergeWith arg pls

@balage90
Copy link
Author

balage90 commented Oct 1, 2018

Yes i exactly wanted to do something similar, so for the first project test execution generates a merge.json and then i will try to use the MergeWith parameter at the second execution where I am referencing the merge.json from the previous run. And so on with the rest of the test projects. But sitll dont know how and wen should i able to generate opencover output since the previous repots are in Json format and not opencover.

@pape77
Copy link
Contributor

pape77 commented Oct 1, 2018

@balage90 you should be able to do it in the final dotnet test excecution. So you have:

1- The first one just doing a dotnet test and generating the first coverage.json file
2- Subsequent n-2 dotnet test using the mergeWith parameter referencing the just created coverage.json file
3- Last one should also use mergeWith BUT ALSO /p:CoverletOutputFormat=opencover in order to generate the final opencover.xml file that you want, based on the merged coverage.json.

You can also just do a dotnet test on the first one to generate the coverage.json and then do a dotnet test on the solution with the opencover output param, to generate all the subsequent merges and the final opencover.xml generation (it's all in the issue #203)

The problem is, however, the one I mentioned in the issue I opened. I made a pr for fixing that, but has to be reviewed by @tonerdo yet

Let me know your findings, and if you face the same problem i had (since you use more than one test project too :) )

@balage90
Copy link
Author

balage90 commented Oct 1, 2018

@pape77 Thanks a lot will try it out tomorrow and let you know the result.

@pape77
Copy link
Contributor

pape77 commented Oct 3, 2018

@balage90 And? And? :P

@8
Copy link

8 commented Oct 16, 2018

I'm trying to run this but since i'm doing a dotnet test on the solution which detects what to test, it will say that the result.json file does not exist, as it doesn't create it the first time it runs the test for one of the projects in the solution :/

I am struggling with the same problem.
I've worked around that by creating an (almost) empty .json file to start with:

# create the testresults folder
$TestResultFolder = (Get-Location).Path + '\TestResults\'
New-Item -Path $TestResultFolder -ItemType Directory -Force

# create the initial file that coverlet needs for merging
$CoverletOutput = $TestResultFolder + 'coverage.json'
Set-Content -Path $CoverletOutput -Value "{}"

and then I can create a combined file like so:

$cmd = "dotnet test --no-build -c Release /p:CollectCoverage=true /p:CoverletOutputFormat=json /p:MergeWith=$CoverletOutput /p:CoverletOutput=$CoverletOutput src\Solution.sln --filter FullyQualifiedName~Unit --logger=trx -r $TestResultFolder"
Invoke-Expression $cmd

Now it's only 1 run over the whole solution and I got a combined coverage.json - but I am stuck now converting that file to opencover without supplying a test project.

Anybody got an idea? I haven't found any straightforward way to test a solution and produce a single coverage report.

Take care,
Martin

@pape77
Copy link
Contributor

pape77 commented Oct 16, 2018

Hi @8

In #203 I create a opencover report.
Read my first comment for the commands.

You can replace the first run i make with your empty json file generation and it should work. Also i think the second step can be skipped and just run the 3rd one. Haven't tried it yet.

Problem comes with merging results (mentioned in the issue itself) . For what i created a pr #211 to fix this. You can read all about it in the issue and pr.

Cheers!

@8
Copy link

8 commented Oct 16, 2018

Hi @pape77,
thanks for the reply!

I finally solved my problem doing what @petli suggested:

  1. I switched back to generating an opencover report per project in the solution by pointing coverlet at the solution file.
  2. Then collect them using powershell and pass them as a single argument to reportgenerator which can handle multiple entries and combine them in a single report.

Take care,
-Martin

@pape77
Copy link
Contributor

pape77 commented Oct 16, 2018

@8 ah I don't use the report generator or poweshell. I want coverlet to generate the merged coverage. Then it can be used for whatever.
That's what I'm aiming for with the pr ;)

But good you found a way for your case

Cheers

@coverlet-coverage coverlet-coverage deleted a comment from petli Oct 17, 2018
@ghost
Copy link

ghost commented Oct 18, 2018

My pull request #220 is a quick fix that should allow for a way to accomplish this (see my comment), if someone's willing to test it.

@ghost
Copy link

ghost commented Feb 1, 2019

Since #220 has been released, this issue should be closed. It is now possible to merge in other formats than json. Exlained in more details in issue #220

@tonerdo tonerdo closed this as completed Feb 23, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants