codecov integration

[ricardogsilva]

Overview

This PR brings integration with the codecov service.

The general idea is that our CI service will run the tests with coverage.py and afterwards upload coverage results to codecov. This involves running parallel coverage and then combining everything in the end.
It will be using the new multi-stage capabilities in Travis (which are awesome by the way).

Related Issue / Discussion

See #511

Additional Information

Contributions and Licensing

(as per https://github.com/geopython/pycsw/blob/master/CONTRIBUTING.rst#contributions-and-licensing)

• I'd like to contribute [feature X|bugfix Y|docs|something else] to pycsw. I confirm that my contributions to pycsw will be compatible with the pycsw license guidelines at the time of contribution.
• I have already previously agreed to the pycsw Contributions and Licensing Guidelines

[ricardogsilva]

@tomkralidis
In order to complete this I'm going to need you, as github admin to enable pycsw integration with codecov. I'm not blocked for now, as I'm testing against my own fork. Let's meet on gitter sometime this week or next (at your convenience) and I can walk you through it if needed.

[tomkralidis]

@ricardogsilva I've enabled codecov is integration for pycsw. Do we need to close/reopen this PR?

[tomkralidis]

Reopening, travis ci running.

[ricardogsilva]

@tomkralidis the work is not finished yet.

I'll ask for review when it seems ready to merge

[codecov]

Codecov Report

❗ No coverage uploaded for pull request base (master@6ef74f6). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #525   +/-   ##
=========================================
  Coverage          ?   56.04%           
=========================================
  Files             ?       29           
  Lines             ?     6320           
  Branches          ?     1338           
=========================================
  Hits              ?     3542           
  Misses            ?     2400           
  Partials          ?      378

Flag	Coverage Δ
#integrationtests	54.69% <ø> (?)
#unittests	7.61% <ø> (?)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6ef74f6...2a117d7. Read the comment docs.

[ricardogsilva]

@tomkralidis @kalxas
I think this is done.

As you can see (from this PR) we will now have a codecov bot analyzing PRs and indicating code coverage status. In this case, since it is the first time that we are adding codecov it cannot compute the difference between the PR and the master branch, and that is why it is showing a failed check. Other codecov functionality is not working very well yet, due to the same problem. This will be fixed once we merge codecov integration into master.

Later on we can configure the codecov settings by adding a .codecov.yml file to the repo.

I think I won't be adding a coverage badge to the README yet because with the default codecov settings it would have a red color, since our global coverage is at 56% and the badge only turns green from 70% up. We could decide what a nice value for green would be, configure codecov appropriately and then add a badge to the README. What do you think?

Another thing implemented in this PR is usage of Travis' build stages. If you check Travis you'll see that we now have a unit-test and an integration-test stage. Stages run sequentially (first unit tests then integration tests). If one stage fails the following stages don't run. Inside each stage we have three 3 concurrent jobs, one for each python env (2.7, 3.4, 3.5). Each job of the same stage runs in parallel.

[tomkralidis]

cc @kalxas

Thanks @ricardogsilva. Should we keep with 70% code coverage with the goal to improve or lower our baseline? Thoughts?

I like the build stages and the early failure approach the provide.

+1 for merge pending decision on coverage threshold value.

[kalxas]

Thanks @ricardogsilva
+1 for the proposed changes
Lets not add the codecov badge yet.

[ricardogsilva]

Let's discuss the threshold for green coverage on the next dev meeting

[tomkralidis]

+1. Added to draft agenda.

[ricardogsilva]

FYI, now that is has been merged you can go to

https://codecov.io/gh/geopython/pycsw

and see a detailed view on the coverage of each file, with line by line visualization of code covered (green), missed (red) and partially covered (yellow)