Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update our ex_coveralls with forked master branch #2

Merged
merged 28 commits into from
Sep 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
8212269
Cobertura now handles defprotocol and defimpl definitions (#306)
gorghoa Mar 27, 2023
07a89a3
Bump version and update CHANGELOG
parroty Mar 27, 2023
354204a
Add Cobertura docs to README.md (#312)
solar05 Jul 2, 2023
509c526
Update Elixir requirement to 1.11+ (#316)
whatyouhide Aug 1, 2023
e41649c
Replace hackney with httpc (#311)
whatyouhide Aug 9, 2023
af7ed22
Bump version and update CHANGELOG
parroty Aug 9, 2023
b475ec9
Fix lcov 2.0 source file handling (#315)
warmwaffles Aug 9, 2023
68297d0
Update CHANGELOG
parroty Aug 9, 2023
36ac3ba
Remove erroneous line in ExCoveralls.poster (#318)
whatyouhide Aug 9, 2023
ead9d5a
Import `.coverdata` after test run and improve documentation (#309)
zachallaun Aug 9, 2023
9ab49e6
Update CHANGELOG
parroty Aug 9, 2023
206e246
Accept custom http options (#319)
rodrigues Aug 27, 2023
719d364
Bump version and update CHANGELOG
parroty Aug 27, 2023
c76d8ed
Always floor coverage instead of rounding (#310)
albertored Oct 10, 2023
97476ce
Bump version and update CHANGELOG
parroty Oct 10, 2023
55686be
Update README examples (#320)
solar05 Oct 11, 2023
e61a96e
Use explicit steps to remove 1.16 deprecation warning (#322)
gitneko Apr 1, 2024
d09ee95
Update CHANGELOG and bump version
parroty Apr 1, 2024
a530769
Detect and warn about incorrectly used ignore-comments (#325)
RKushnir May 30, 2024
f66fbc7
Fix Elixir 1.17 single-quoted string warning (#327)
hkrutzer Jul 23, 2024
91a8267
Bump version and update CHANGELOG
parroty Jul 23, 2024
00a96c4
add missing step for Cobertura's range (#329)
juanperi Sep 1, 2024
b4bf930
Update CHANGELOG and version
parroty Sep 1, 2024
46d66b2
Revert "add missing step for Cobertura's range (#329)" (#330)
parroty Sep 1, 2024
d0d8891
Revert version change
parroty Sep 1, 2024
26120fb
Update Range to use function syntax (#332)
juanperi Sep 4, 2024
7952661
Update CHANGELOG and version
parroty Sep 4, 2024
071a143
Merge branch 'master' of https://github.com/parroty/excoveralls into …
tonyrud Sep 26, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 11 additions & 6 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,22 +8,27 @@ jobs:
runs-on: ubuntu-20.04
strategy:
matrix:
otp: ['20.3']
elixir: ['1.4', '1.5', '1.6', '1.7']
include:
- otp: '21.3'
elixir: '1.11'
- otp: '26.0'
elixir: '1.15'
- otp: '27.0'
elixir: '1.17'
env:
MIX_ENV: test
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- uses: erlef/setup-beam@v1
with:
otp-version: ${{ matrix.otp }}
elixir-version: ${{ matrix.elixir }}
- uses: actions/cache@v2
- uses: actions/cache@v3
with:
path: deps
key: ${{ runner.os }}-mix-${{ hashFiles(format('{0}{1}', github.workspace, '/mix.lock')) }}
key: ${{ runner.os }}-${{ matrix.elixir }}-otp${{ matrix.otp }}-mix-${{ hashFiles(format('{0}{1}', github.workspace, '/mix.lock')) }}
restore-keys: |
${{ runner.os }}-mix-
${{ runner.os }}-${{ matrix.elixir }}-otp${{ matrix.otp }}-mix-
- run: mix deps.get
- run: mix coveralls.github
46 changes: 46 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,49 @@
0.18.3
------
#### Changes
- Avoid warning messages for Cobertura
- Update Range to use function syntax (#332)

0.18.2
------
#### Enhancements
- Print warnings about incorrectly used ignore-markers (#325), such as start-marker
without a corresponding stop-marker, or two start-markers without a stop-marker in-between etc.

#### Changes
- Fix Elixir 1.17 single-quoted string warning (#327)

0.18.1
------
#### Changes
- Use explicit steps to remove 1.16 deprecation warning in Cobertura (#322).

0.18.0
------
#### Changes
- Always floor coverage instead of rounding (#310).
- **Note:** If you want to keep the previous rounding behavior, please check the `floor_coverage` option.
- https://github.com/parroty/excoveralls#coverage-options

0.17.1
------
#### Enhancements
- Accept custom http options (#319).

0.17.0
------
#### Changes
- Replace hackney with httpc (#311).
- Update Elixir requirement to 1.11+ (#316).
- Fix lcov 2.0 source file handling (#315).
- Import .coverdata after test run and improve documentation (#309).
- Fixes around `--import-cover` option.

0.16.1
------
#### Changes
- Cobertura now handles defprotocol and defimpl definitions (#306).

0.16.0
------
#### Enhancements
Expand Down
98 changes: 89 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,8 @@ def project do
coveralls: :test,
"coveralls.detail": :test,
"coveralls.post": :test,
"coveralls.html": :test
"coveralls.html": :test,
"coveralls.cobertura": :test
]
# if you want to use espec,
# test_coverage: [tool: ExCoveralls, test_task: "espec"]
Expand All @@ -55,7 +56,7 @@ end

defp deps do
[
{:excoveralls, "~> 0.10", only: :test},
{:excoveralls, "~> 0.18", only: :test},
]
end
```
Expand Down Expand Up @@ -88,16 +89,20 @@ end
- [[mix coveralls.html] Show coverage as HTML report](#mix-coverallshtml-show-coverage-as-html-report)
- [[mix coveralls.json] Show coverage as JSON report](#mix-coverallsjson-show-coverage-as-json-report)
- [[mix coveralls.xml] Show coverage as XML report](#mix-coverallsxml-show-coverage-as-xml-report)
- [[mix coveralls.cobertura] Show coverage as Cobertura report](#mix-coverallscobertura-show-coverage-as-cobertura-report)
- [[mix coveralls.lcov] Show coverage as lcov report (Experimental)](#mix-coverallslcov-show-coverage-as-lcov-report-experimental)
- [coveralls.json](#coverallsjson)
- [Stop Words](#stop-words)
- [Exclude Files](#exclude-files)
- [Terminal Report Output](#terminal-report-output)
- [Coverage Options](#coverage-options)
- [Stop Words](#stop-words)
- [Exclude Files](#exclude-files)
- [Terminal Report Output](#terminal-report-output)
- [Coverage Options](#coverage-options)
- [Other Considerations](#other-considerations)
- [Ignore Lines](#ignore-lines)
- [Silence OTP Cover Warnings](#silence-otp-cover-warnings)
- [Merging Coverage Results](#merging-coverage-results)
- [Notes](#notes)
- [Todo](#todo)
- [License](#license)
- [License](#license)

### [mix coveralls] Show coverage
Run the `MIX_ENV=test mix coveralls` command to show coverage information on localhost.
Expand Down Expand Up @@ -145,7 +150,8 @@ Usage: mix coveralls <Options>
and your git repo resides in "app", then the root path should be: "/home/runs/app/" (from
coveralls.io)
--flagname Job flag name which will be shown in the Coveralls UI
--import_cover Directory from where '.coverdata' files should be imported and their results added to the report
--import-cover Directory from where '.coverdata' files should be imported and their results added to the report.
Coverdata is imported after tests are run.

Usage: mix coveralls.detail [--filter file-name-pattern]
Used to display coverage with detail
Expand Down Expand Up @@ -362,6 +368,13 @@ Output to the shell is the same as running the command `mix coveralls` (to suppr

Output reports are written to `cover/excoveralls.xml` by default, however, the path can be specified by overwriting the `"output_dir"` coverage option.

### [mix coveralls.cobertura] Show coverage as Cobertura report
This task displays coverage information at the source-code level formatted as a [Cobertura](https://cobertura.github.io/cobertura/) document.
The report follows a format supported by [Gitlab](https://docs.gitlab.com/ee/ci/testing/test_coverage_visualization.html) code coverage visualization.
Output to the shell is the same as running the command `mix coveralls` (to suppress this output, add `"print_summary": false` to your project's `coveralls.json` file). In a similar manner to `mix coveralls.detail`, reported source code can be filtered by specifying arguments using the `--filter` flag.

Output reports are written to `cover/cobertura.xml` by default, however, the path can be specified by overwriting the `"output_dir"` coverage option.

### [mix coveralls.lcov] Show coverage as lcov report (Experimental)
This task displays coverage information at the line level formatted as a lcov.
The report follows a format supported by several code coverage services like VSCode extension(`ryanluker.vscode-coverage-gutters`).
Expand Down Expand Up @@ -432,6 +445,9 @@ to `false`:
- When used in conjunction with `minimum_coverage`, overall project coverage is checked first before individual file coverages are checked.
- `html_filter_full_covered`
- A boolean, when `true` files with 100% coverage are not shown in the HTML report. Default to `false`.
- `floor_coverage`
- A boolean, when `false` coverage values are ceiled instead of floored, this means that a project with some lines
that are not covered can still have a total 100% coverage. Default to `true`.

Example configuration file:

Expand Down Expand Up @@ -461,6 +477,8 @@ Example configuration file:
}
```

## Other Considerations

### Ignore Lines

Use comments `coveralls-ignore-start` and `coveralls-ignore-stop` to ignore certain lines from code coverage calculation.
Expand Down Expand Up @@ -501,6 +519,68 @@ imported_info(_Text,_Module,_Imported) ->
ok.
```

### Merging Coverage Results

ExCoveralls can include `.coverdata` files in the result of the current test run through the `--import-cover` flag. This can be used to include coverage data from partitioned tests or integration tests that may run in a subprocess, for instance.

Coverage data is generated when running `mix test --cover`, optionally with the `--export-coverage` flag to specify an output name.

```shell
$ mix test --only integration --cover --export-coverage integration-coverage
Excluding tags: [:test]
Including tags: [:integration]
... test run omitted ...
# Coverage data written to cover/integration-coverage.coverdata

# Report coverage, do not run integration tests
$ mix coveralls --exclude integration
Excluding tags: [:integration]
... test run omitted ...

----------------
COV FILE LINES RELEVANT MISSED
...
[TOTAL] 80.2% # <-- This result does not include coverage from integration tests
----------------

# Report coverage, do not run integration tests, but include previously written coverdata
$ mix coveralls --exclude integration --import-cover cover
Excluding tags: [:integration]
... test run omitted ...

----------------
COV FILE LINES RELEVANT MISSED
...
[TOTAL] 95.3% # <-- This result now includes coverage from integration tests
----------------
```

Coverage data is imported after tests are run.

See the `mix test` [Coverage documentation](https://hexdocs.pm/mix/Mix.Tasks.Test.html#module-coverage) for more information on `.coverdata`.

### Configuring HTTP Options in ExCoveralls

You can customize the HTTP options used by [`:httpc`](https://www.erlang.org/doc/man/httpc.html) when posting results. The example below shows how to specify a custom `cacertfile`:

```elixir
config :excoveralls,
http_options: [
timeout: 10_000,
ssl: [
# Refer to the secure coding guide:
# https://erlef.github.io/security-wg/secure_coding_and_deployment_hardening/inets
verify: :verify_peer,
depth: 2,
customize_hostname_check: [
match_fun: :public_key.pkix_verify_hostname_match_fun(:https)
],
cacertfile: to_charlist(System.fetch_env!("TEST_COVERAGE_CACERTFILE"))
]
```

By default, ExCoveralls uses the `cacertfile` from [`castore`](https://hexdocs.pm/castore/api-reference.html) when the dependency is installed. If it's not available and you're running Erlang `25` or later, the system will attempt to use the OS certificates via [`:public_key.cacerts_load/0`](https://www.erlang.org/doc/man/public_key.html#cacerts_load-0).

### Notes
- If mock library is used, it will show some warnings during execution.
- https://github.com/eproxus/meck/pull/17
Expand All @@ -512,6 +592,6 @@ imported_info(_Text,_Module,_Imported) ->
- It might not work well on projects which handle multiple project (Mix.Project) files.
- Needs improvement on file-path handling.

## License
# License

This source code is licensed under the MIT license. Copyright (c) 2013-present, parroty.
3 changes: 2 additions & 1 deletion lib/conf/coveralls.json
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@
"coverage_options": {
"treat_no_relevant_lines_as_covered": false,
"output_dir": "cover/",
"minimum_coverage": 0
"minimum_coverage": 0,
"floor_coverage": true
},

"terminal_options": {
Expand Down
13 changes: 7 additions & 6 deletions lib/excoveralls.ex
Original file line number Diff line number Diff line change
Expand Up @@ -41,13 +41,14 @@ defmodule ExCoveralls do
def start(compile_path, opts) do
Cover.compile(compile_path)

options = ConfServer.get()
if options[:import_cover] do
Cover.import(options[:import_cover])
end

fn() ->
execute(ConfServer.get, compile_path, opts)
options = ConfServer.get()

if options[:import_cover] do
Cover.import(options[:import_cover])
end

execute(options, compile_path, opts)
end
end

Expand Down
3 changes: 2 additions & 1 deletion lib/excoveralls/circle.ex
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ defmodule ExCoveralls.Circle do
Handles circle-ci integration with coveralls.
"""
alias ExCoveralls.Poster
alias ExCoveralls.Stats

def execute(stats, options) do
json = generate_json(stats, Enum.into(options, %{}))
Expand All @@ -20,7 +21,7 @@ defmodule ExCoveralls.Circle do
service_number: get_number(),
service_job_id: get_job_id(),
service_pull_request: get_pull_request(),
source_files: stats,
source_files: Stats.serialize(stats),
git: generate_git_info(),
parallel: options[:parallel],
flag_name: options[:flagname]
Expand Down
24 changes: 15 additions & 9 deletions lib/excoveralls/cobertura.ex
Original file line number Diff line number Diff line change
Expand Up @@ -156,14 +156,12 @@ defmodule ExCoveralls.Cobertura do
end

defp module_name(source) do
case Regex.run(~r/^defmodule\s+(.*)\s+do$/m, source, capture: :all_but_first) do
[module] ->
module

_ ->
[module] = Regex.run(~r/^-module\((.*)\)\.$/m, source, capture: :all_but_first)
module
end
with nil <- Regex.run(~r/^def(?:module|protocol|impl)\s+(.*)\s+do$/m, source, capture: :all_but_first),
nil <- Regex.run(~r/^-module\((.*)\)\.$/m, source, capture: :all_but_first) do
"UNKNOWN_MODULE"
else
[module] -> module
end
end

defp package_name(path, c_paths) do
Expand All @@ -172,7 +170,7 @@ defmodule ExCoveralls.Cobertura do
c_paths
|> Enum.find_value(package_name, fn c_path ->
if String.starts_with?(package_name, c_path) do
String.slice(package_name, (String.length(c_path) + 1)..-1)
String.slice(package_name, get_slice_range_for_package_name(c_path))
else
false
end
Expand All @@ -182,6 +180,14 @@ defmodule ExCoveralls.Cobertura do
|> to_charlist()
end

# TODO: Remove when we require Elixir 1.12 as minimum and inline it with range syntax
if Version.match?(System.version(), ">= 1.12.0") do
# We use Range.new/3 because using x..y//step would give a syntax error on Elixir < 1.12
defp get_slice_range_for_package_name(c_path), do: Range.new(String.length(c_path) + 1, -1, 1)
else
defp get_slice_range_for_package_name(c_path), do: Range.new(String.length(c_path) + 1, -1)
end

defp rate(valid_lines) when length(valid_lines) == 0, do: 0.0

defp rate(valid_lines) do
Expand Down
10 changes: 3 additions & 7 deletions lib/excoveralls/cover.ex
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,9 @@ defmodule ExCoveralls.Cover do
{:ok, string_io} = StringIO.open("")
Process.group_leader(pid, string_io)

:cover.compile_beam_directory(compile_path |> string_to_charlist)
compile_path
|> String.to_charlist()
|> :cover.compile_beam_directory()
end

@doc """
Expand Down Expand Up @@ -59,12 +61,6 @@ defmodule ExCoveralls.Cover do
:cover.analyse(module, :calls, :line)
end

if Version.compare(System.version, "1.3.0") == :lt do
defp string_to_charlist(string), do: String.to_char_list(string)
else
defp string_to_charlist(string), do: String.to_charlist(string)
end

defp file_exist?(module, path) do
if File.exists?(path) do
true
Expand Down
3 changes: 2 additions & 1 deletion lib/excoveralls/drone.ex
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ defmodule ExCoveralls.Drone do
Handles drone-ci integration with coveralls.
"""
alias ExCoveralls.Poster
alias ExCoveralls.Stats

def execute(stats, options) do
json = generate_json(stats, Enum.into(options, %{}))
Expand All @@ -20,7 +21,7 @@ defmodule ExCoveralls.Drone do
service_number: get_build_num(),
service_job_id: get_build_num(),
service_pull_request: get_pull_request(),
source_files: stats,
source_files: Stats.serialize(stats),
git: generate_git_info(),
parallel: options[:parallel],
flag_name: options[:flagname]
Expand Down
3 changes: 2 additions & 1 deletion lib/excoveralls/github.ex
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ defmodule ExCoveralls.Github do
Handles GitHub Actions integration with coveralls.
"""
alias ExCoveralls.Poster
alias ExCoveralls.Stats

def execute(stats, options) do
json = generate_json(stats, Enum.into(options, %{}))
Expand All @@ -20,7 +21,7 @@ defmodule ExCoveralls.Github do
%{
repo_token: get_env("GITHUB_TOKEN"),
service_name: "github",
source_files: stats,
source_files: Stats.serialize(stats),
parallel: options[:parallel],
flag_name: options[:flagname],
git: git_info()
Expand Down
Loading