Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: try to speed up Github workflows #12090

Merged
merged 6 commits into from
Jan 7, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions .github/workflows/cancel_duplicates.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
name: Cancel Duplicates
on:
workflow_run:
workflows: ["CI"]
types: ["requested"]

jobs:
cancel-duplicate-workflow-runs:
name: "Cancel duplicate workflow runs"
runs-on: ubuntu-latest
steps:
- uses: apache-superset/cancel-workflow-runs@953e057
name: "Cancel duplicate workflow runs"
with:
cancelMode: duplicates
cancelFutureDuplicates: true
token: ${{ secrets.GITHUB_TOKEN }}
sourceRunId: ${{ github.event.workflow_run.id }}
notifyPRCancel: true
skipEventTypes: '["push", "pull_request", "pull_request_target"]'
45 changes: 0 additions & 45 deletions .github/workflows/license-check.yml

This file was deleted.

91 changes: 91 additions & 0 deletions .github/workflows/misc.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
name: Miscellaneous

on:
push:
branches-ignore:
- "dependabot/**"
pull_request:

jobs:
license_check:
name: License Check
runs-on: ubuntu-18.04
steps:
- uses: actions/checkout@v2
- name: Setup Java
uses: actions/setup-java@v1
with:
java-version: 8
- name: Generate fossa report
env:
FOSSA_API_KEY: ${{ secrets.FOSSA_API_KEY }}
run: |
set -eo pipefail
if [[ "${{github.event_name}}" != "pull_request" ]]; then
./scripts/fossa.sh
exit 0
fi

URL="https://api.github.com/repos/${{ github.repository }}/pulls/${{ github.event.pull_request.number }}/files"
FILES=$(curl -s -X GET -G $URL | jq -r '.[] | .filename')

cat<<EOF
CHANGED FILES:
$FILES

EOF

if [[ "${FILES}" =~ (.*package*\.json|requirements\/[a-z_-]+\.txt|setup\.py) ]]; then
echo "Detected dependency changes... running fossa check"

./scripts/fossa.sh
else
echo "No dependency changes... skiping fossa check"
fi
shell: bash
- name: Run license check
run: ./scripts/check_license.sh

prefer_typescript:
if: github.ref == 'ref/heads/master' && github.event_name == 'pull_request'
name: Prefer Typescript
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Get changed files
id: changed
uses: ./.github/actions/file-changes-action
with:
githubToken: ${{ secrets.GITHUB_TOKEN }}

- name: Determine if a .js or .jsx file was added
id: check
run: |
js_files_added() {
jq -r '
map(
select(
endswith(".js") or endswith(".jsx")
)
) | join("\n")
' ${HOME}/files_added.json
}
echo ::set-output name=js_files_added::$(js_files_added)

- if: steps.check.outputs.js_files_added
name: Add Comment to PR
uses: ./.github/actions/comment-on-pr
continue-on-error: true
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This check is essentially useless since the comments will never work for PRs coming from forks (due to the GITHUB_TOKEN access). This would be a good candidate to port over to the workflow_run approach mentioned in https://securitylab.github.com/research/github-actions-preventing-pwn-requests. Obviously, out of scope for this PR

msg: |
### WARNING: Prefer TypeScript

Looks like your PR contains new `.js` or `.jsx` files:

```
${{steps.check.outputs.js_files_added}}
```

As decided in [SIP-36](https://github.com/apache/superset/issues/9101), all new frontend code should be written in TypeScript. Please convert above files to TypeScript then re-request review.
49 changes: 0 additions & 49 deletions .github/workflows/prefer-typescript.yml

This file was deleted.

8 changes: 7 additions & 1 deletion .github/workflows/superset-e2e.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,12 @@
name: E2E

on: [push, pull_request]
on:
push:
branches-ignore:
- "dependabot/**/docs/**"
paths-ignore:
- "docs/**"
pull_request:

jobs:
Cypress:
Expand Down
7 changes: 6 additions & 1 deletion .github/workflows/superset-frontend.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
name: Frontend

on: [push, pull_request]
on:
push:
branches-ignore:
- "dependabot/**/docs/**"
- "dependabot/**/cypress-base/**"
pull_request:

jobs:
build:
Expand Down
6 changes: 5 additions & 1 deletion .github/workflows/superset-python-misc.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,11 @@
# Python Misc unit tests
name: Python Misc

on: [push, pull_request]
on:
push:
branches-ignore:
- "dependabot/npm_and_yarn/**"
pull_request:

jobs:
lint:
Expand Down
51 changes: 0 additions & 51 deletions .github/workflows/superset-python-mysql.yml

This file was deleted.

52 changes: 0 additions & 52 deletions .github/workflows/superset-python-postgres.yml

This file was deleted.

6 changes: 5 additions & 1 deletion .github/workflows/superset-python-presto-hive.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,11 @@
# Python Presto/Hive unit tests
name: Python Presto/Hive

on: [push, pull_request]
on:
push:
branches-ignore:
- "dependabot/npm_and_yarn/**"
pull_request:

jobs:
test-postgres-presto:
Expand Down
Loading