Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: try to speed up Github workflows #12089

Closed
wants to merge 6 commits into from
Closed

Conversation

ktmud
Copy link
Member

@ktmud ktmud commented Dec 17, 2020

SUMMARY

CI has been slow recently. This PR is the latest effort in trying to mitigate:

  1. Update the Cancel Previous Run job added in build: cancel previous github action runs #11940 to cancel future jobs as well.

  2. Skip dependabot branches when possible (e.g. don't run Python jobs for npm updates).

  3. Merge lighter actions (Prefer Typescript and License Check) into one workflow.

  4. Add a script to allow committers to manually cancel CI jobs in queue. This is useful on a busy day when the CI pipeline got clogged by continuous new commits to open PRs. Users have to configure GITHUB_TOKEN in their env variables in order to use this.

    (superset) ➜  incubator-superset git:(manage-github-workflow) ✗ ./scripts/cancel_github_workflows.py --help 
    Usage: cancel_github_workflows.py [OPTIONS] BRANCH_OR_PULL
    
      Cancel running or queued GitHub workflows by branch or pull request ID.
    
      By default jobs that are already running or are the latest of the branch
      or PR will not be cancelled.
    
    Options:
      --repo TEXT                     Default is apache/incubator-superset
      --event [pull_request|push|issue]
      --include-last / --no-include-last
                                      Whether to also cancel the lastest run.
      --include-running / --no-include-running
                                      Whether to also cancel running workflows
      --help                          Show this message and exit.
    

    Snip20201204_53

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

N/A

TEST PLAN

Test locally and CI

ADDITIONAL INFORMATION

  • Has associated issue:
  • Changes UI
  • Requires DB Migration.
  • Confirm DB Migration upgrade and downgrade tested.
  • Introduces new feature or API
  • Removes existing feature or API

@ktmud
Copy link
Member Author

ktmud commented Dec 17, 2020

#11944 got stuck at some required CI jobs not reporting. Recreating a new PR to see if it works.

@ktmud ktmud closed this Dec 17, 2020
@ktmud ktmud reopened this Dec 17, 2020
@ktmud ktmud closed this Dec 17, 2020
@ktmud ktmud reopened this Dec 17, 2020
@ktmud ktmud closed this Dec 17, 2020
@ktmud ktmud reopened this Dec 17, 2020
@ktmud ktmud force-pushed the manage-github-workflow branch from 8d2f297 to be32394 Compare December 17, 2020 02:42
@ktmud ktmud force-pushed the manage-github-workflow branch from be32394 to bb890be Compare December 17, 2020 02:44
@codecov-io
Copy link

codecov-io commented Dec 17, 2020

Codecov Report

Merging #12089 (bb890be) into master (08b3ebe) will decrease coverage by 3.27%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #12089      +/-   ##
==========================================
- Coverage   67.26%   63.98%   -3.28%     
==========================================
  Files         959      476     -483     
  Lines       47241    29386   -17855     
  Branches     4629        0    -4629     
==========================================
- Hits        31778    18804   -12974     
+ Misses      15345    10582    -4763     
+ Partials      118        0     -118     
Flag Coverage Δ
cypress ?
javascript ?
python 63.98% <ø> (-0.32%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
superset/sql_validators/postgres.py 50.00% <0.00%> (-50.00%) ⬇️
superset/db_engine_specs/mysql.py 79.59% <0.00%> (-12.25%) ⬇️
superset/db_engine_specs/sqlite.py 65.62% <0.00%> (-9.38%) ⬇️
superset/databases/commands/create.py 83.67% <0.00%> (-8.17%) ⬇️
superset/databases/commands/update.py 85.71% <0.00%> (-8.17%) ⬇️
superset/utils/celery.py 96.42% <0.00%> (-3.58%) ⬇️
superset/models/core.py 85.90% <0.00%> (-2.99%) ⬇️
superset/views/core.py 72.97% <0.00%> (-2.48%) ⬇️
superset/db_engine_specs/base.py 85.75% <0.00%> (-1.85%) ⬇️
superset/datasets/api.py 89.44% <0.00%> (-1.84%) ⬇️
... and 489 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 08b3ebe...bb890be. Read the comment docs.

@ktmud ktmud closed this Dec 17, 2020
@ktmud ktmud deleted the manage-github-workflow branch January 5, 2021 23:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants