CI run coverage on multiple builds #40394

MarcoGorelli · 2021-03-12T10:17:53Z

closes CI: remove the current coverage build, run coverage on multiple builds instead #39758

Moving the posix builds to GitHubActions + uploading coverage on every build, as they're merged by default by Codecov

Trying to get this in so that the codecov check in #40078 will pass

MarcoGorelli · 2021-03-13T20:24:24Z

pandas/plotting/_matplotlib/misc.py

-            coeffs.resize(int((coeffs.size + 1) / 2), 2)
+            coeffs = np.resize(coeffs, (int((coeffs.size + 1) / 2), 2))


This was throwing an error during CI:

____________________ TestDataFramePlots.test_andrews_curves ____________________ [gw0] linux -- Python 3.8.8 /usr/share/miniconda/envs/pandas-dev/bin/python self = <pandas.tests.plotting.test_misc.TestDataFramePlots object at 0x7f3ae76442b0> iris = SepalLength SepalWidth PetalLength PetalWidth Name 0 5.1 3.5 1.4 ... 2.3 Iris-virginica 149 5.9 3.0 5.1 1.8 Iris-virginica [150 rows x 5 columns] def test_andrews_curves(self, iris): from matplotlib import cm from pandas.plotting import andrews_curves df = iris # Ensure no UserWarning when making plot with tm.assert_produces_warning(None): > _check_plot_works(andrews_curves, frame=df, class_column="Name") pandas/tests/plotting/test_misc.py:158: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ pandas/tests/plotting/common.py:625: in _check_plot_works raise err pandas/tests/plotting/common.py:618: in _check_plot_works for ret in gen_plots(f, fig, **kwargs): pandas/tests/plotting/common.py:645: in _gen_two_subplots yield f(**kwargs) pandas/plotting/_misc.py:271: in andrews_curves return plot_backend.andrews_curves( pandas/plotting/_matplotlib/misc.py:282: in andrews_curves y = f(t) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ t = array([-3.14159265, -3.11001886, -3.07844506, -3.04687127, -3.01529747, -2.98372368, -2.95214988, -2.92057608, ...900229, 2.92057608, 2.95214988, 2.98372368, 3.01529747, 3.04687127, 3.07844506, 3.11001886, 3.14159265]) def f(t): x1 = amplitudes[0] result = x1 / np.sqrt(2.0) # Take the rest of the coefficients and resize them # appropriately. Take a copy of amplitudes as otherwise numpy # deletes the element from amplitudes itself. coeffs = np.delete(np.copy(amplitudes), 0) > coeffs.resize(int((coeffs.size + 1) / 2), 2) E ValueError: cannot resize an array that references or is referenced E by another array in this way. E Use the np.resize function or refcheck=False pandas/plotting/_matplotlib/misc.py:249: ValueError

MarcoGorelli · 2021-03-13T20:24:56Z

codecov.yml

+  notify:
+    after_n_builds: 10


this is so that codecov doesn't report on insufficient coverage prematurely

is this related to why codecov often gives an Access Denied?

Without this, then Codecov will report coverage as soon as one job finishes. But if that job isn't comprehensive, then it'll show e.g. "coverage 30% less than target of 80%, failed". Like this, it'll wait for multiple jobs to be done and will merge their reports

MarcoGorelli · 2021-03-13T20:25:42Z

ci/azure/posix.yml

-    - script: |
-        if [ "$(uname)" == "Linux" ]; then
-          sudo apt-get update
-          sudo apt-get install -y libc6-dev-i386 $EXTRA_APT
-        fi
-      displayName: 'Install extra packages'
-


these linux ones are all moved to .github/workflows/posix.yml

MarcoGorelli · 2021-03-13T20:25:52Z

azure-pipelines.yml

- template: ci/azure/posix.yml
-  parameters:
-    name: Linux
-    vmImage: ubuntu-16.04


moved to .github/workflows/posix.yml

MarcoGorelli · 2021-03-13T20:26:42Z

.github/workflows/posix.yml

+    - uses: conda-incubator/setup-miniconda@v2
+      with:
+        activate-environment: pandas-dev
+        channel-priority: flexible


I couldn't get the environment to resolve with strict

fangchenli · 2021-03-14T04:17:07Z

@jreback Would there be a capacity/credit issue if we run all Linux builds on GitHub Actions?

jreback · 2021-03-14T04:35:29Z

no we can run on github actions

w/o restrictions

jreback · 2021-03-14T23:38:54Z

this looks pretty good. cc @jbrockmendel as i know you look at coverage lot. cc @datapythonista if any comments here.

jbrockmendel · 2021-03-15T03:21:25Z

No objections. The CI config is pretty mysterious to me.

datapythonista

Looks nice. Do we have enough builds in actions to not slow the CI results with this?

datapythonista · 2021-03-15T15:28:42Z

.github/workflows/posix.yml

+    strategy:
+      matrix:
+        settings: [
+          [ci/deps/actions-37-minimum_versions.yaml, "not slow and not network and not clipboard", "", "", "", "", ""],


I think you can define this as before, something like:

minumum_versions: - ENV_FILE: ci/deps/actions-37-minimum_versions.yaml - PATTERN: "not slow and not network and not clipboard" ...

Do you find this better, or couldn't you implement it this way?

Also, very minor, but instead of repeating ci/deps/ or ci/deps/actions-{}.yaml you could move it below, when you set ENV_FILE.

Do you find this better, or couldn't you implement it this way?

Are you sure this can be done with GitHubActions without repeating all the steps? Do you have an example? (I couldn't find one, but I haven't done that much GitHubActions)

Also, very minor, but instead of repeating ci/deps/ or ci/deps/actions-{}.yaml you could move it below, when you set ENV_FILE.

Good point, thanks!

Are you sure this can be done with GitHubActions without repeating all the steps? Do you have an example? (I couldn't find one, but I haven't done that much GitHubActions)

I think the best way is to use include. You've got an example in the docs that should be similar:

https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#using-environment-variables-in-a-matrix

jreback · 2021-03-15T16:01:57Z

Looks nice. Do we have enough builds in actions to not slow the CI results with this?

do you know where this info is?

datapythonista · 2021-03-15T16:21:19Z

Looks nice. Do we have enough builds in actions to not slow the CI results with this?

do you know where this info is?

I couldn't find information online, not sure why. But checking the messages I sent to Azure and GitHub, Azure gives 10 parallel workers by default, but we had 30, and GitHub actions gives 20, and I asked them if they could increase it, but afaik that never happened.

I think we balanced things so both azure and actions had the same load, with the 30+20. If this is still the more or less the case, and we move lots of builds from azure to actions, feels like the 20 on actions will be a bottleneck, and we won't use much the 30 in azure.

But not sure if things changed since I had all these discussions.

jreback · 2021-03-17T21:09:44Z

thanks @MarcoGorelli

very nice!

@pandas-dev/pandas-core so this moves the linux builds to github actions, if anything looks awry pls open an issue

MarcoGorelli requested review from fangchenli and removed request for fangchenli March 12, 2021 10:18

MarcoGorelli force-pushed the upload-coverage branch from c3f5dfe to 4233bbd Compare March 13, 2021 10:54

MarcoGorelli added 27 commits March 13, 2021 16:47

upload coverage

ec0479b

add pytest-cov

339ffe4

debug coverage

e1e973e

cobertura

3669e85

indent

630722b

wip

8df6f80

use correct path in print_skipped.py

95623e4

full run

7ea05a0

try moving to ghactions

cc1f37f

wip

753c3ac

stage posix file

25629d6

coverage: true

21b0682

fixup yml error

36f96e9

PYTEST_WORKERS

213a06f

remove pytest-azurepipelines

d0f4569

valid yaml

fff0217

pin pytest-cov

d6eefae

install extra

e9ba4be

install extra

e5e0087

wip

231def7

wip

7c5d5d4

flexi

318e206

wip

ff9a9b7

wip

c938acf

cov in numpydev

0ba4e76

cov in numpydev

ff4a09f

cov in numpydev

9338780

MarcoGorelli added 5 commits March 13, 2021 16:47

cov file

3e0321b

put the rest

815fd04

remove linux job from azure-pipelines.yml

97d0dc0

remove linux job from azure-pipelines.yml

ba9f12d

remove linux job from azure-pipelines.yml

7f5052e

MarcoGorelli force-pushed the upload-coverage branch from 77e8433 to 7f5052e Compare March 13, 2021 16:47

MarcoGorelli added 2 commits March 13, 2021 17:28

dont fail fast

d804bcd

dont fail fast

dbfb5b5

MarcoGorelli requested review from fangchenli and jreback March 13, 2021 18:51

resize differently

66f46df

MarcoGorelli requested a review from afeld March 13, 2021 20:21

MarcoGorelli commented Mar 13, 2021

View reviewed changes

simonjayhawkins added the CI Continuous Integration label Mar 14, 2021

jreback added this to the 1.3 milestone Mar 14, 2021

datapythonista reviewed Mar 15, 2021

View reviewed changes

share ci/deps/

a56b0a8

jreback merged commit 0ff5b40 into pandas-dev:master Mar 17, 2021

MarcoGorelli mentioned this pull request Mar 17, 2021

CI: remove the current coverage build, run coverage on multiple builds instead #39758

Closed

JulianWgs pushed a commit to JulianWgs/pandas that referenced this pull request Jul 3, 2021

CI run coverage on multiple builds (pandas-dev#40394)

858df05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CI run coverage on multiple builds #40394

CI run coverage on multiple builds #40394

MarcoGorelli commented Mar 12, 2021 •

edited

Loading

MarcoGorelli Mar 13, 2021

MarcoGorelli Mar 13, 2021

jbrockmendel Mar 15, 2021

MarcoGorelli Mar 15, 2021

MarcoGorelli Mar 13, 2021

MarcoGorelli Mar 13, 2021

MarcoGorelli Mar 13, 2021

fangchenli commented Mar 14, 2021

jreback commented Mar 14, 2021

jreback commented Mar 14, 2021

jbrockmendel commented Mar 15, 2021

datapythonista left a comment

datapythonista Mar 15, 2021

MarcoGorelli Mar 15, 2021

datapythonista Mar 17, 2021

jreback commented Mar 15, 2021

datapythonista commented Mar 15, 2021

jreback commented Mar 17, 2021

		coeffs.resize(int((coeffs.size + 1) / 2), 2)
		coeffs = np.resize(coeffs, (int((coeffs.size + 1) / 2), 2))

CI run coverage on multiple builds #40394

CI run coverage on multiple builds #40394

Conversation

MarcoGorelli commented Mar 12, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fangchenli commented Mar 14, 2021

jreback commented Mar 14, 2021

jreback commented Mar 14, 2021

jbrockmendel commented Mar 15, 2021

datapythonista left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jreback commented Mar 15, 2021

datapythonista commented Mar 15, 2021

jreback commented Mar 17, 2021

MarcoGorelli commented Mar 12, 2021 •

edited

Loading