Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed merge-publish and disabled circleci publish workflow #45

Merged
merged 10 commits into from
Dec 22, 2020
Merged

Fixed merge-publish and disabled circleci publish workflow #45

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
251aed2
addressed ronak's comments, fixed merge-publish and disabled circlec …
JBAhire Dec 18, 2020
c408a74
cleanup
JBAhire Dec 18, 2020
3ee1a1a
addressed workflow split discussion
JBAhire Dec 21, 2020
4be0b7f
renames workflow
JBAhire Dec 21, 2020
411a1e3
syntax error
JBAhire Dec 21, 2020
c5d0518
addressed Aaron's comments
JBAhire Dec 21, 2020
d765e95
cleanup
JBAhire Dec 21, 2020
84bca36
path error for reports
JBAhire Dec 21, 2020
3f4919c
path error for reports
JBAhire Dec 21, 2020
603275c
updates gradle plugin
JBAhire Dec 21, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 33 additions & 33 deletions .circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -202,36 +202,36 @@ workflows:
context:
- hypertrace-vulnerability-scanning
- dockerhub-read
- merge-publish:
context:
- hypertrace-publishing
- dockerhub-read
requires:
- build
- integration-test
- validate-charts
- snyk-scan
filters:
branches:
only:
- main
- release-publish:
context:
- hypertrace-publishing
- dockerhub-read
filters:
branches:
ignore: /.*/
tags:
only: /^release-.*/
- release-charts:
context:
- hypertrace-publishing
- dockerhub-read
requires:
- release-publish
filters:
branches:
ignore: /.*/
tags:
only: /^release-.*/
# - merge-publish:
# context:
# - hypertrace-publishing
# - dockerhub-read
# requires:
# - build
# - integration-test
# - validate-charts
# - snyk-scan
# filters:
# branches:
# only:
# - main
# - release-publish:
# context:
# - hypertrace-publishing
# - dockerhub-read
# filters:
# branches:
# ignore: /.*/
# tags:
# only: /^release-.*/
# - release-charts:
# context:
# - hypertrace-publishing
# - dockerhub-read
# requires:
# - release-publish
# filters:
# branches:
# ignore: /.*/
# tags:
# only: /^release-.*/
45 changes: 45 additions & 0 deletions .github/workflows/merge-publish.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
name: merge-publish
on:
push:
branches:
- main
workflow_dispatch:

jobs:
merge-publish:
runs-on: ubuntu-20.04
steps:
# Set fetch-depth: 0 to fetch commit history and tags for use in version calculation
- name: Check out code
uses: actions/checkout@v2.3.4
with:
ref: ${{github.event.pull_request.head.ref}}
repository: ${{github.event.pull_request.head.repo.full_name}}
fetch-depth: 0

- name: create checksum file
run: find . -type f -name "*.gradle*" -o -name "gradle-wrapper*" | xargs shasum | sort > checksum.txt && cat checksum.txt

- name: Cache packages
id: cache-packages
uses: actions/cache@v2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's use the same version v2.3.4 everywhere.

with:
path: ~/.gradle
key: gradle-packages-${{ runner.os }}-${{ github.job }}-${{ hashFiles('**/checksum.txt') }}
restore-keys: |
gradle-packages-${{ runner.os }}-${{ github.job }}
gradle-packages-${{ runner.os }}

- name: Login to Docker Hub
uses: docker/login-action@v1
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: branch name tag
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need this now as you moved the extraction of BRANCH_NAME in the Gradle plugin - https://github.com/hypertrace/hypertrace-gradle-docker-plugins/pull/21/files?

run: |
BRANCH="${GITHUB_REF#refs/heads/}"
aaron-steinfeld marked this conversation as resolved.
Show resolved Hide resolved
echo "BRANCH=${BRANCH}" >> $GITHUB_ENV

- name: push docker image
run: ./gradlew dockerPushImages
65 changes: 39 additions & 26 deletions .github/workflows/pr-build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,19 +1,21 @@
name: build and test
name: build and validate
on:
push:
branches:
- main
pull_request:
pull_request_target:


jobs:
build-and-test:
build:
runs-on: ubuntu-20.04
steps:
# Set fetch-depth: 0 to fetch commit history and tags for use in version calculation
- name: Check out code
uses: actions/checkout@v2.3.4
with:
ref: ${{github.event.pull_request.head.ref}}
repository: ${{github.event.pull_request.head.repo.full_name}}
fetch-depth: 0

- name: create checksum file
Expand All @@ -38,31 +40,42 @@ jobs:
- name: Build with Gradle
run: ./gradlew build dockerBuildImages

- name: Unit test
run: ./gradlew jacocoTestReport

- name: Upload coverage to Codecov
uses: codecov/codecov-action@v1
validate-helm-charts:
aaron-steinfeld marked this conversation as resolved.
Show resolved Hide resolved
runs-on: ubuntu-20.04
container:
image: hypertrace/helm-gcs-packager:0.3.0
credentials:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
# Set fetch-depth: 0 to fetch commit history and tags for use in version calculation
steps:
- name: Check out code
uses: actions/checkout@v2.3.4
with:
Copy link
Contributor

@kotharironak kotharironak Dec 18, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will need the below condition to take changes from the forked pull request otherwise it will checkout only the main branch code

with:
        ref: ${{github.event.pull_request.head.ref}}
        repository: ${{github.event.pull_request.head.repo.full_name}}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yup. forgot about this earlier

name: codecov-line-smoothing
fail_ci_if_error: true
verbose: true
flags: unit
ref: ${{github.event.pull_request.head.ref}}
repository: ${{github.event.pull_request.head.repo.full_name}}
fetch-depth: 0

- name: Integration test
run: ./gradlew jacocoIntegrationTestReport
- name: validate charts
run: |
helm dependency update ./helm/
helm lint --strict ./helm/
helm template ./helm/

- name: Upload coverage to Codecov
uses: codecov/codecov-action@v1
snyk-scan:
runs-on: ubuntu-20.04
steps:
# Set fetch-depth: 0 to fetch commit history and tags for use in version calculation
- name: Check out code
uses: actions/checkout@v2.3.4
with:
name: codecov-line-smoothing
fail_ci_if_error: true
verbose: true
flags: integration

- name: push docker image
if: github.event.pull_request.merged
run: ./gradlew dockerPushImages

ref: ${{github.event.pull_request.head.ref}}
repository: ${{github.event.pull_request.head.repo.full_name}}
aaron-steinfeld marked this conversation as resolved.
Show resolved Hide resolved
fetch-depth: 0


- name: Run Snyk to check for vulnerabilities
uses: snyk/actions/gradle-jdk11@0.3.0
env:
SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
with:
args: --all-sub-projects --policy-path=.snyk --configuration-matching='^runtimeClasspath$'
72 changes: 72 additions & 0 deletions .github/workflows/pr-test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
name: test
on:
push:
branches:
- main
pull_request:


jobs:
test:
runs-on: ubuntu-20.04
steps:
# Set fetch-depth: 0 to fetch commit history and tags for use in version calculation
- name: Check out code
uses: actions/checkout@v2.3.4
with:
fetch-depth: 0

- name: create checksum file
run: find . -type f -name "*.gradle*" -o -name "gradle-wrapper*" | xargs shasum | sort > checksum.txt && cat checksum.txt

- name: Cache packages
id: cache-packages
uses: actions/cache@v2
with:
path: ~/.gradle
key: gradle-packages-${{ runner.os }}-${{ github.job }}-${{ hashFiles('**/checksum.txt') }}
restore-keys: |
gradle-packages-${{ runner.os }}-${{ github.job }}
gradle-packages-${{ runner.os }}

- name: Unit test
run: ./gradlew jacocoTestReport

- name: Copy unit test reports
run: ./gradlew copyAllReports --output-dir=/tmp/unit-test-reports

- name: Upload coverage to Codecov
uses: codecov/codecov-action@v1
with:
name: codecov-line-smoothing
fail_ci_if_error: true
verbose: true
flags: unit

- name: Archive unit test report
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we have to archive?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#45 (comment)

uses: actions/upload-artifact@v1
with:
name: unit-test
path: /tmp/unit-test-reports
if: ${{ always() }}

- name: Integration test
run: ./gradlew jacocoIntegrationTestReport

- name: Copy integration test reports
run: ./gradlew copyAllReports --output-dir=/tmp/integration-test-reports
aaron-steinfeld marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The artifacts produced by this are the same as the artifacts reported by the previous step I believe since we're not cleaning out. We can either clean between the two test phases, or a single upload for both runs is probably fine too.

As a next step (can be a separate PR) it would make things much nicer to extract the results so they're visible in the GH UI (looks like it doesn't let viewing the artifacts, only downloading them). Something like this action:
https://github.com/marketplace/actions/publish-unit-test-results

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noted. Makes sense to me.


- name: Upload coverage to Codecov
uses: codecov/codecov-action@v1
with:
name: codecov-line-smoothing
fail_ci_if_error: true
verbose: true
flags: integration

- name: Archive integration test report
uses: actions/upload-artifact@v1
with:
name: integration-test
path: /tmp/integration-test-reports
if: ${{ always() }}
8 changes: 1 addition & 7 deletions .github/workflows/publish.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Publish Docker image
name: Publish artifacts
on:
# Will only run when release is published.
release:
Expand Down Expand Up @@ -51,12 +51,6 @@ jobs:
uses: actions/checkout@v2
with:
fetch-depth: 0

- name: Login to Docker Hub
uses: docker/login-action@v1
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: package and release charts
env:
Expand Down
25 changes: 0 additions & 25 deletions .github/workflows/snyk.yml
View file
Open in desktop

This file was deleted.

29 changes: 0 additions & 29 deletions .github/workflows/validate_charts.yml
View file
Open in desktop

This file was deleted.

4 changes: 2 additions & 2 deletions attribute-service/build.gradle.kts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ plugins {
java
application
jacoco
id("org.hypertrace.docker-java-application-plugin") version "0.8.0"
id("org.hypertrace.docker-publish-plugin") version "0.8.0"
id("org.hypertrace.docker-java-application-plugin") version "0.8.1"
id("org.hypertrace.docker-publish-plugin") version "0.8.1"
id("org.hypertrace.integration-test-plugin")
id("org.hypertrace.jacoco-report-plugin")
}
Expand Down