Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add only_changed input to run rubocop only against changed files #103

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 52 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@ name: CI
on: [pull_request]
jobs:
test-skip-install-and-use-bundler:
name: runner / rubocop
runs-on: ubuntu-latest
defaults:
run:
Expand All @@ -22,3 +21,55 @@ jobs:
skip_install: 'true'
use_bundler: 'true'
- run: test "$(bundle exec rubocop --version)" == "1.18.1"
test-only_changed:
runs-on: ubuntu-latest
defaults:
run:
shell: bash
env:
INPUT_ONLY_CHANGED: 'true'
steps:
- uses: actions/checkout@v4
- name: setup
run: |
git config user.email "workflow@github.com"
git config user.name "I am an automated workflow"
- name: Check when there are relevant files
run: |
git checkout ${{ github.sha }}
rm -f test/only_changed/reviewdog-was-called

cp test/only_changed/few_relevant/files/* .
git add *
git commit -m auto

export PATH=test/only_changed/few_relevant/mock_bins:test/only_changed/shared_mock_bins:$PATH
BASE_REF=$(git rev-parse HEAD~) HEAD_REF=$(git rev-parse HEAD) ./script.sh

[ -f test/only_changed/reviewdog-was-called ]
- name: Check when there are no relevant files
run: |
git checkout ${{ github.sha }}
rm -f test/only_changed/reviewdog-was-called

cp test/only_changed/nothing_relevant/files/* .
git add *
git commit -m auto

export PATH=test/only_changed/nothing_relevant/mock_bins:test/only_changed/shared_mock_bins:$PATH
BASE_REF=$(git rev-parse HEAD~) HEAD_REF=$(git rev-parse HEAD) ./script.sh

[ ! -f test/only_changed/reviewdog-was-called ]
- name: Check when there are too many relevant files
run: |
git checkout ${{ github.sha }}
rm -f test/only_changed/reviewdog-was-called

touch a{00..100}.rb
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we create 101 (more than 100) files here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The range is inclusive, so 101 files are generated, I can change to {01..101} if it will be clearer

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I accidentally tested that with exactly 100 it passes file names to rubocop: https://github.com/reviewdog/action-rubocop/actions/runs/9913233268/job/27389778004

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok.

I'm a bit confused, but why does the test result shows " No relevant files for rubocop, skipping"?
image
https://github.com/reviewdog/action-rubocop/actions/runs/9913690348/job/27391225472

Copy link
Contributor Author

@toy toy Jul 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great that you spotted it, I didn't notice that change from ruby to bash script for rubocop mock was producing files prefixed with ./. I reverted to ruby script for simplicity and hardened the test by explicitly validating that reviewdog mock was or was not called.

git add *
git commit -m auto

export PATH=test/only_changed/too_many_relevant/mock_bins:test/only_changed/shared_mock_bins:$PATH
BASE_REF=$(git rev-parse HEAD~) HEAD_REF=$(git rev-parse HEAD) ./script.sh

[ -f test/only_changed/reviewdog-was-called ]
8 changes: 8 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,14 @@ Default is `added`.
Optional. Report level for reviewdog [`info`, `warning`, `error`].
It's same as `-level` flag of reviewdog.

### `only_changed`

Optional. Run Rubocop only on changed (and added) files, for speedup [`true`, `false`].
Default: `false`.

Will fetch the tip of the base branch with depth 1 from remote origin if it is not available.
If you use different remote name or customize the checkout otherwise, make the tip of the base branch available before this action

### `reporter`

Optional. Reporter of reviewdog command [`github-pr-check`, `github-check`, `github-pr-review`].
Expand Down
9 changes: 9 additions & 0 deletions action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,12 @@ inputs:
level:
description: 'Report level for reviewdog [info,warning,error]'
default: 'error'
only_changed:
description: |
Run Rubocop only on changed (and added) files, for speedup [`true`, `false`].
Will fetch the tip of the base branch with depth 1 from remote origin if it is not available.
If you use different remote name or customize the checkout otherwise, make the tip of the base branch available before this action.
default: 'false'
reporter:
description: |
Reporter of reviewdog command [github-pr-check,github-check,github-pr-review].
Expand Down Expand Up @@ -61,6 +67,7 @@ runs:
INPUT_FILTER_MODE: ${{ inputs.filter_mode }}
INPUT_GITHUB_TOKEN: ${{ inputs.github_token }}
INPUT_LEVEL: ${{ inputs.level }}
INPUT_ONLY_CHANGED: ${{ inputs.only_changed }}
INPUT_REPORTER: ${{ inputs.reporter }}
INPUT_REVIEWDOG_FLAGS: ${{ inputs.reviewdog_flags }}
INPUT_RUBOCOP_EXTENSIONS: ${{ inputs.rubocop_extensions }}
Expand All @@ -70,6 +77,8 @@ runs:
INPUT_TOOL_NAME: ${{ inputs.tool_name }}
INPUT_USE_BUNDLER: ${{ inputs.use_bundler }}
INPUT_WORKDIR: ${{ inputs.workdir }}
BASE_REF: ${{ github.event.pull_request.base.sha }}
HEAD_REF: ${{ github.sha }}
Copy link

@thiemonipro thiemonipro Jun 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First off all thanks for this PR, it seems like a really useful addition that could speed up our github action by a lot as using rubocop takes about 5 minutes currently to run over the whole repo.

Should this look at github.event.pull_request.base.sha instead?

I tried out your branch in a github action workflow but the HEAD_REF commit seemed wrong and gave me the following error: fatal: Invalid revision range

After creating my own branch based on this but with the change to the HEAD_REF sha the action succeeded

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean using github.event.pull_request.head.sha for HEAD_REF?

The idea of this change is to give to rubocop only files introduced in the PR branch, but we don't want to also give to rubocop files that changed on the base branch especially if it diverged a lot.

This could be resolved in different ways, but in current implementation the idea is to also fetch bare minimum to not slowdown by fetching, so usage of actions/checkout with default depth: 1 is assumed.

Also github.sha is pointing at automatically created merge commit of base into head (docs), while github.event.pull_request.head.sha points at the PR branch head.

If we would have all commits available it would be simpler to do diff base...head (note three dots) where head can be both merge commit or the tip of the branch, but if we fetch only two tips of the branches, diff base...head is not possible, as not all needed commits are available.
Otherwise diff base..merge-head should give much smaller diff then diff base..head.

So most probable cause of fatal: Invalid revision range is because your workflow either doesn't fetch the tip of the base branch or fetches not the automatically created merge head (default using actions/checkout).

branding:
icon: 'check-circle'
color: 'red'
44 changes: 42 additions & 2 deletions script.sh
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
#!/bin/sh -e
#!/usr/bin/env bash

set -e
set -o pipefail

cd "${GITHUB_WORKSPACE}/${INPUT_WORKDIR}" || exit
export REVIEWDOG_GITHUB_API_TOKEN="${INPUT_GITHUB_TOKEN}"
Expand Down Expand Up @@ -85,9 +88,46 @@ else
BUNDLE_EXEC="bundle exec "
fi

if [ "${INPUT_ONLY_CHANGED}" = "true" ]; then
echo '::group:: Getting changed files list'

# check if commit is present in repository, otherwise fetch it
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about running `git fetch --depth 1 origin "${BASE_REF}" here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like a good idea, compared to adding a step to do it manually. Only concern is if remote is named differently or if checkout is very custom, so the command breaks.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doing it after checking commit availability should be the good way to make it simple by default and solvable if default doesn't work: eb0eda0

if ! git cat-file -e "${BASE_REF}"; then
git fetch --depth 1 origin "${BASE_REF}"
fi

# get intersection of changed files (excluding deleted) with target files for
# rubocop as an array
# shellcheck disable=SC2086
readarray -t CHANGED_FILES < <(
comm -12 \
<(git diff --diff-filter=d --name-only "${BASE_REF}..${HEAD_REF}" | sort || kill $$) \
<(${BUNDLE_EXEC}rubocop --list-target-files | sort || kill $$)
)

if (( ${#CHANGED_FILES[@]} == 0 )); then
echo "No relevant files for rubocop, skipping"
exit 0
fi

printf '%s\n' "${CHANGED_FILES[@]}"

if (( ${#CHANGED_FILES[@]} > 100 )); then
echo "More than 100 changed files (${#CHANGED_FILES[@]}), running rubocop on all files"
unset CHANGED_FILES
fi

echo '::endgroup::'
fi

echo '::group:: Running rubocop with reviewdog 🐶 ...'
# shellcheck disable=SC2086
${BUNDLE_EXEC}rubocop ${INPUT_RUBOCOP_FLAGS} --require ${GITHUB_ACTION_PATH}/rdjson_formatter/rdjson_formatter.rb --format RdjsonFormatter \
${BUNDLE_EXEC}rubocop \
${INPUT_RUBOCOP_FLAGS} \
--require ${GITHUB_ACTION_PATH}/rdjson_formatter/rdjson_formatter.rb \
--format RdjsonFormatter \
--fail-level error \
"${CHANGED_FILES[@]}" \
Copy link

@kehra kehra Aug 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@toy
Did you want to intentionally change(set) fail-level?
I think it seems a bug.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kehra It was intentional, due to turning on pipefail, which in turn is needed for pipes in getting the list of changed files.
Before this commit the exit code of rubocop … | reviewdog … was the exit code of reviewdog. But with pipefail and without changing fail-level, any rubocop violation would make the script have an error exit code, which will be confusing, as both violations will be reported and the action will fail.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@toy
Thank you for your response.
I think this change caused our CI to miss non-error level(e.g. warning) violations.
However, my colleague has made it possible to specify the fail-level in #120, so it is not a problem 😄

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kehra #120 looks good, it allows to "shoot in the foot" by changing formatter, but I think it is better to restrict less.

I'm still wondering about your CI - any violations of any level will create either annotation or comment to PR, so the rubocop outcome will be failing, but not the rubocop action itself, am I missing something?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@toy
Hmm.
I might be wrong, so I'll investigate this again 💦
Sorry for bothering you.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No worries, I'm just trying to understand if there is some case that allows action to not report violations. Please reply with your findings

| reviewdog -f=rdjson \
-name="${INPUT_TOOL_NAME}" \
-reporter="${INPUT_REPORTER}" \
Expand Down
1 change: 1 addition & 0 deletions test/only_changed/few_relevant/files/a.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
puts "Hello, " + "world!"

Check warning on line 1 in test/only_changed/few_relevant/files/a.rb

View workflow job for this annotation

GitHub Actions / test-skip-install-and-use-bundler

[rubocop] reported by reviewdog 🐶 Missing frozen string literal comment. Raw Output: test/only_changed/few_relevant/files/a.rb:1:1: C: Style/FrozenStringLiteralComment: Missing frozen string literal comment.

Check warning on line 1 in test/only_changed/few_relevant/files/a.rb

View workflow job for this annotation

GitHub Actions / test-skip-install-and-use-bundler

[rubocop] reported by reviewdog 🐶 Prefer string interpolation to string concatenation. Raw Output: test/only_changed/few_relevant/files/a.rb:1:6: C: Style/StringConcatenation: Prefer string interpolation to string concatenation.

Check warning on line 1 in test/only_changed/few_relevant/files/a.rb

View workflow job for this annotation

GitHub Actions / test-skip-install-and-use-bundler

[rubocop] reported by reviewdog 🐶 Prefer single-quoted strings when you don't need string interpolation or special symbols. Raw Output: test/only_changed/few_relevant/files/a.rb:1:6: C: Style/StringLiterals: Prefer single-quoted strings when you don't need string interpolation or special symbols.

Check warning on line 1 in test/only_changed/few_relevant/files/a.rb

View workflow job for this annotation

GitHub Actions / test-skip-install-and-use-bundler

[rubocop] reported by reviewdog 🐶 Prefer single-quoted strings when you don't need string interpolation or special symbols. Raw Output: test/only_changed/few_relevant/files/a.rb:1:18: C: Style/StringLiterals: Prefer single-quoted strings when you don't need string interpolation or special symbols.
1 change: 1 addition & 0 deletions test/only_changed/few_relevant/files/a.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
puts "Hello, " + "world!"
18 changes: 18 additions & 0 deletions test/only_changed/few_relevant/mock_bins/rubocop
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
#!/usr/bin/env ruby
# frozen_string_literal: true

case ARGV
when %w[
--list-target-files
]
puts Dir['**/*.rb']
when %W[
--require #{ENV['GITHUB_ACTION_PATH']}/rdjson_formatter/rdjson_formatter.rb
--format RdjsonFormatter
--fail-level error
a.rb
]
puts 'Mock message for reviewdog'
else
abort "rubocop mock called with unexpected arguments:\n#{ARGV.join("\n")}"
end
1 change: 1 addition & 0 deletions test/only_changed/nothing_relevant/files/a.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
puts "Hello, " + "world!"
11 changes: 11 additions & 0 deletions test/only_changed/nothing_relevant/mock_bins/rubocop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#!/usr/bin/env ruby
# frozen_string_literal: true

case ARGV
when %w[
--list-target-files
]
puts Dir['**/*.rb']
else
abort "rubocop mock called with unexpected arguments:\n#{ARGV.join("\n")}"
end
9 changes: 9 additions & 0 deletions test/only_changed/shared_mock_bins/bundle
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
#!/bin/bash

if [ "$1" = "exec" ]; then
shift
eval "$@"
else
echo "Only 'exec' command is supported"
exit 1
fi
8 changes: 8 additions & 0 deletions test/only_changed/shared_mock_bins/curl
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#!/bin/bash

arguments="$*"

if [ "$arguments" != "-sfL https://raw.githubusercontent.com/reviewdog/reviewdog/master/install.sh" ]; then
echo "curl mock got unexpected arguments: $arguments"
exit 1
fi
10 changes: 10 additions & 0 deletions test/only_changed/shared_mock_bins/reviewdog
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
#!/bin/bash

touch test/only_changed/reviewdog-was-called

read -r input

if [ "$input" != "Mock message for reviewdog" ]; then
echo "reviewdog mock got unexpected input: $input"
exit 1
fi
17 changes: 17 additions & 0 deletions test/only_changed/too_many_relevant/mock_bins/rubocop
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
#!/usr/bin/env ruby
# frozen_string_literal: true

case ARGV
when %w[
--list-target-files
]
puts Dir['**/*.rb']
when %W[
--require #{ENV['GITHUB_ACTION_PATH']}/rdjson_formatter/rdjson_formatter.rb
--format RdjsonFormatter
--fail-level error
]
puts 'Mock message for reviewdog'
else
abort "rubocop mock called with unexpected arguments:\n#{ARGV.join("\n")}"
end
Loading