Replace valgrind by all sanitizers available. #476

tcojean · 2020-03-05T14:06:51Z

This tentatively removes valgrind and instead add leak sanitizer and memory sanitizer check. The hope is to gain quite a CI speedup without losing in quality checks (in fact, UBsan could find new important issues).

Add a new UBSAN configuration for undefined behavior.
Add a new leak sanitizer configuration.
Add a sanitizers test to ensure all sanitizers work.
Add a new ginkgo_create_thread_test helper for the sanitizers test.
Add CI steps to check for leaks and undefined behavior.

Also update containers with various fixes and additions:

Add Metis build option
Add MPI build option
Make CUDA 10.1 to be based on ubuntu 18.04.
Fix an issue with HIP and CUDA <= 9.1

codecov · 2020-03-06T05:32:48Z

Codecov Report

Merging #476 into develop will decrease coverage by 2.83%.
The diff coverage is 100%.

@@             Coverage Diff             @@
##           develop     #476      +/-   ##
===========================================
- Coverage    91.61%   88.78%   -2.84%     
===========================================
  Files          294      256      -38     
  Lines        19614    16457    -3157     
===========================================
- Hits         17969    14611    -3358     
- Misses        1645     1846     +201

Impacted Files	Coverage Δ
core/test/base/sanitizers.cpp	`100% <100%> (ø)`
core/devices/hip/executor.cpp	`33.33% <0%> (-66.67%)`	⬇️
core/devices/cuda/executor.cpp	`50% <0%> (-50%)`	⬇️
include/ginkgo/core/matrix/ell.hpp	`68.75% <0%> (-31.25%)`	⬇️
include/ginkgo/core/matrix/coo.hpp	`70.27% <0%> (-29.73%)`	⬇️
include/ginkgo/core/matrix/csr.hpp	`70.7% <0%> (-13.14%)`	⬇️
include/ginkgo/core/matrix/hybrid.hpp	`76.92% <0%> (-12.8%)`	⬇️
include/ginkgo/core/base/math.hpp	`93.93% <0%> (-3.04%)`	⬇️
include/ginkgo/core/preconditioner/jacobi.hpp	`92% <0%> (-2.74%)`	⬇️
reference/components/format_conversion.hpp	`100% <0%> (ø)`	⬆️
... and 51 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f0b82c8...e4b0e7a. Read the comment docs.

upsj

LGTM!

cmake/build_type_helpers.cmake

yhmtsai

LGTM

pratikvn

LGTM!

I guess now the outputs of ASAN and UBSAN are one ? There seems to be some problem with the CUDA and HIP checks, is that because CUDA and HIP are not supported ? Doesnt cuda-memcheck work ?

thoasm

LGTM except for case sensitivity of STREQUAL.

cmake/CTestScript.cmake

tcojean · 2020-03-10T12:45:26Z

@pratikvn The sanitizers do not work with CUDA or HIP code. In addition, I tried to make CTest interact with cuda-memcheck to create a custom step which would take reports from that tool, but was never successful in doing so. The feature is not available by default.

For the outputs of UBSAN, I actually now have all four separate sanitizers (ASAN, LSAN, UBSAN, TSAN), and a combined MSAN which does ASAN + LSAN + UBSAN. This allows to save build time and get all data at once, at least that's the plan.

thoasm

LGTM!

tcojean · 2020-03-13T17:19:04Z

I needed to update the CUDA containers for this PR. Looks like that created an issue with CUDA + Intel for everyone. I will try to fix this.

+ Add a new UBSAN configuration for undefined behavior. + Add a new leak sanitizer configuration. + Add a sanitizers test to ensure all sanitizers work. + Add a new ginkgo_create_thread_test helper for the sanitizers test. + Add CI steps to check for leaks and undefined behavior.

+ Add Metis build option + Add MPI build option + Make CUDA 10.1 and to be based on ubuntu 18.04. + Fix an issue with HIP and CUDA <= 9.1

tcojean · 2020-03-16T11:16:08Z

Hopefully everything is now finally working. Still awaiting normal CI confirmation. I had to change the containers settings because UBSAN works only with Ubuntu 18.04 but using Ubuntu 18.04 is extremely complex as it conflicts with some CUDA, Intel, etc versions.

The new sanitizer setup is in place and working. You can see what it looks like here:
https://my.cdash.org/index.php?project=Ginkgo+Project
Related pipeline: https://gitlab.com/ginkgo-project/ginkgo-public-ci/pipelines/126624153
We now probably have better information than with valgrind for less execution time. It now takes 30-40 minutes maximum compared to up to 250 minutes previously with Valgrind:
https://gitlab.com/ginkgo-project/ginkgo-public-ci/pipelines/125549867

I added four new tests which are designed to create an error for every of the new sanitizers. This is a way to test the sanitizers, so that we know they are properly setup and actually working. This proved to be important as UBSAN was clearly not working for most of the time and debugging this issue proved to be very time consuming. Here are the new added tests:

A wrong access (after free),
A memory leak,
A threading race condition,
An undefined integer operation.

tcojean · 2020-03-16T11:57:25Z

There is still a problem with one setup, clang+CUDA 10.1+HIP compilation for some reason. Please do not merge anything before I fix it:
https://gitlab.com/ginkgo-project/ginkgo-public-ci/-/jobs/472997356

tcojean · 2020-03-16T15:57:24Z

Everything is now fixed. For some reason, gtest is not compiled with -fPIC and that never was an issue before but it is for HIP + CUDA 10.1 + Ubuntu 18.04...

yhmtsai

LGTM. I have a question about hipblas.

cmake/CTestScript.cmake

dev_tools/containers/ginkgo-cuda-base.py

sonarqubecloud · 2020-03-17T12:08:42Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities (and 0 Security Hotspots to review)
0 Code Smells

0.0% Coverage
0.0% Duplication

pratikvn

LGTM!

The Ginkgo team is proud to announce the new minor release of Ginkgo version 1.2.0. This release brings full HIP support to Ginkgo, new preconditioners (ParILUT, ISAI), conversion between double and float for all LinOps, and many more features and fixes. Supported systems and requirements: + For all platforms, cmake 3.9+ + Linux and MacOS + gcc: 5.3+, 6.3+, 7.3+, all versions after 8.1+ + clang: 3.9+ + Intel compiler: 2017+ + Apple LLVM: 8.0+ + CUDA module: CUDA 9.0+ + HIP module: ROCm 2.8+ + Windows + MinGW and CygWin: gcc 5.3+, 6.3+, 7.3+, all versions after 8.1+ + Microsoft Visual Studio: VS 2017 15.7+ + CUDA module: CUDA 9.0+, Microsoft Visual Studio + OpenMP module: MinGW or CygWin. The current known issues can be found in the [known issues page](https://github.com/ginkgo-project/ginkgo/wiki/Known-Issues). # Additions Here are the main additions to the Ginkgo library. Other thematic additions are listed below. + Add full HIP support to Ginkgo [#344](#344), [#357](#357), [#384](#384), [#373](#373), [#391](#391), [#396](#396), [#395](#395), [#393](#393), [#404](#404), [#439](#439), [#443](#443), [#567](#567) + Add a new ISAI preconditioner [#489](#489), [#502](#502), [#512](#512), [#508](#508), [#520](#520) + Add support for ParILUT and ParICT factorization with ILU preconditioners [#400](#400) + Add a new BiCG solver [#438](#438) + Add a new permutation matrix format [#352](#352), [#469](#469) + Add CSR SpGEMM support [#386](#386), [#398](#398), [#418](#418), [#457](#457) + Add CSR SpGEAM support [#556](#556) + Make all solvers and preconditioners transposable [#535](#535) + Add CsrBuilder and CooBuilder for intrusive access to matrix arrays [#437](#437) + Add a standard-compliant allocator based on the Executors [#504](#504) + Support conversions for all LinOp between double and float [#521](#521) + Add a new boolean to the CUDA and HIP executors to control DeviceReset (default off) [#557](#557) + Add a relaxation factor to IR to represent Richardson Relaxation [#574](#574) + Add two new stopping criteria, for relative (to `norm(b)`) and absolute residual norm [#577](#577) ### Example additions + Templatize all examples to simplify changing the precision [#513](#513) + Add a new adaptive precision block-Jacobi example [#507](#507) + Add a new IR example [#522](#522) + Add a new Mixed Precision Iterative Refinement example [#525](#525) + Add a new example on iterative trisolves in ILU preconditioning [#526](#526), [#536](#536), [#550](#550) ### Compilation and library changes + Auto-detect compilation settings based on environment [#435](#435), [#537](#537) + Add SONAME to shared libraries [#524](#524) + Add clang-cuda support [#543](#543) ### Other additions + Add sorting, searching and merging kernels for GPUs [#403](#403), [#428](#428), [#417](#417), [#455](#455) + Add `gko::as` support for smart pointers [#493](#493) + Add setters and getters for criterion factories [#527](#527) + Add a new method to check whether a solver uses `x` as an initial guess [#531](#531) + Add contribution guidelines [#549](#549) # Fixes ### Algorithms + Improve the classical CSR strategy's performance [#401](#401) + Improve the CSR automatical strategy [#407](#407), [#559](#559) + Memory, speed improvements to the ELL kernel [#411](#411) + Multiple improvements and fixes to ParILU [#419](#419), [#427](#427), [#429](#429), [#456](#456), [#544](#544) + Fix multiple issues with GMRES [#481](#481), [#523](#523), [#575](#575) + Optimize OpenMP matrix conversions [#505](#505) + Ensure the linearity of the ILU preconditioner [#506](#506) + Fix IR's use of the advanced apply [#522](#522) + Fix empty matrices conversions and add tests [#560](#560) ### Other core functionalities + Fix complex number support in our math header [#410](#410) + Fix CUDA compatibility of the main ginkgo header [#450](#450) + Fix isfinite issues [#465](#465) + Fix the Array::view memory leak and the array/view copy/move [#485](#485) + Fix typos preventing use of some interface functions [#496](#496) + Fix the `gko::dim` to abide to the C++ standard [#498](#498) + Simplify the executor copy interface [#516](#516) + Optimize intermediate storage for Composition [#540](#540) + Provide an initial guess for relevant Compositions [#561](#561) + Better management of nullptr as criterion [#562](#562) + Fix the norm calculations for complex support [#564](#564) ### CUDA and HIP specific + Use the return value of the atomic operations in our wrappers [#405](#405) + Improve the portability of warp lane masks [#422](#422) + Extract thread ID computation into a separate function [#464](#464) + Reorder kernel parameters for consistency [#474](#474) + Fix the use of `pragma unroll` in HIP [#492](#492) ### Other + Fix the Ginkgo CMake installation files [#414](#414), [#553](#553) + Fix the Windows compilation [#415](#415) + Always use demangled types in error messages [#434](#434), [#486](#486) + Add CUDA header dependency to appropriate tests [#452](#452) + Fix several sonarqube or compilation warnings [#453](#453), [#463](#463), [#532](#532), [#569](#569) + Add shuffle tests [#460](#460) + Fix MSVC C2398 error [#490](#490) + Fix missing interface tests in test install [#558](#558) # Tools and ecosystem ### Benchmarks + Add better norm support in the benchmarks [#377](#377) + Add CUDA 10.1 generic SpMV support in benchmarks [#468](#468), [#473](#473) + Add sparse library ILU in benchmarks [#487](#487) + Add overhead benchmarking capacities [#501](#501) + Allow benchmarking from a matrix list file [#503](#503) + Fix benchmarking issue with JSON and non-finite numbers [#514](#514) + Fix benchmark logger crashers with OpenMP [#565](#565) ### CI related + Improvements to the CI setup with HIP compilation [#421](#421), [#466](#466) + Add MacOSX CI support [#470](#470), [#488](#488) + Add Windows CI support [#471](#471), [#488](#488), [#510](#510), [#566](#566) + Use sanitizers instead of valgrind [#476](#476) + Add automatic container generation and update facilities [#499](#499) + Fix the CI parallelism settings [#517](#517), [#538](#538), [#539](#539) + Make the codecov patch check informational [#519](#519) + Add support for LLVM sanitizers with improved thread sanitizer support [#578](#578) ### Test suite + Add an assertion for sparsity pattern equality [#416](#416) + Add core and reference multiprecision tests support [#448](#448) + Speed up GPU tests by avoiding device reset [#467](#467) + Change test matrix location string [#494](#494) ### Other + Add Ginkgo badges from our tools [#413](#413) + Update the `create_new_algorithm.sh` script [#420](#420) + Bump copyright and improve license management [#436](#436), [#433](#433) + Set clang-format minimum requirement [#441](#441), [#484](#484) + Update git-cmake-format [#446](#446), [#484](#484) + Disable the development tools by default [#442](#442) + Add a script for automatic header formatting [#447](#447) + Add GDB pretty printer for `gko::Array` [#509](#509) + Improve compilation speed [#533](#533) + Add editorconfig support [#546](#546) + Add a compile-time check for header self-sufficiency [#552](#552) # Related PR: #583

tcojean self-assigned this Mar 5, 2020

tcojean requested review from pratikvn, thoasm, upsj and yhmtsai March 5, 2020 14:07

tcojean added reg:build This is related to the build system. reg:ci-cd This is related to the continuous integration system. is:enhancement An improvement of an existing feature. 1:ST:WIP This PR is a work in progress. Not ready for review. labels Mar 5, 2020

tcojean force-pushed the disable_valgrind branch from cdbc7c3 to 471b3ab Compare March 5, 2020 21:16

tcojean added 1:ST:ready-for-review This PR is ready for review and removed 1:ST:WIP This PR is a work in progress. Not ready for review. labels Mar 6, 2020

tcojean force-pushed the disable_valgrind branch from f0a04d4 to 58c1df2 Compare March 9, 2020 10:52

upsj approved these changes Mar 9, 2020

View reviewed changes

cmake/build_type_helpers.cmake Outdated Show resolved Hide resolved

tcojean force-pushed the disable_valgrind branch from 58c1df2 to f4e8473 Compare March 9, 2020 11:10

yhmtsai approved these changes Mar 9, 2020

View reviewed changes

pratikvn approved these changes Mar 10, 2020

View reviewed changes

thoasm requested changes Mar 10, 2020

View reviewed changes

cmake/CTestScript.cmake Outdated Show resolved Hide resolved

tcojean force-pushed the disable_valgrind branch 3 times, most recently from db4fe8f to 34d880e Compare March 11, 2020 10:25

thoasm approved these changes Mar 11, 2020

View reviewed changes

tcojean force-pushed the disable_valgrind branch 6 times, most recently from 7f21870 to 1913ae6 Compare March 13, 2020 15:53

tcojean force-pushed the disable_valgrind branch from 1913ae6 to 2fcc4da Compare March 16, 2020 10:22

tcojean added 2 commits March 16, 2020 12:11

Update containers.

356da6c

+ Add Metis build option + Add MPI build option + Make CUDA 10.1 and to be based on ubuntu 18.04. + Fix an issue with HIP and CUDA <= 9.1

tcojean force-pushed the disable_valgrind branch from 2fcc4da to 356da6c Compare March 16, 2020 11:12

Try to add fPIC to gtest.

75fc51d

tcojean requested review from upsj, pratikvn, yhmtsai and thoasm March 16, 2020 15:57

tcojean force-pushed the disable_valgrind branch from be757de to 46f917c Compare March 17, 2020 09:08

yhmtsai approved these changes Mar 17, 2020

View reviewed changes

cmake/CTestScript.cmake Outdated Show resolved Hide resolved

dev_tools/containers/ginkgo-cuda-base.py Show resolved Hide resolved

Add some missing codecov calls and badge, fix coverage build.

e4b0e7a

tcojean force-pushed the disable_valgrind branch from 46f917c to e4b0e7a Compare March 17, 2020 11:48

pratikvn approved these changes Mar 17, 2020

View reviewed changes

tcojean merged commit fda35be into develop Mar 17, 2020

tcojean deleted the disable_valgrind branch March 17, 2020 15:16

tcojean added 1:ST:ready-to-merge This PR is ready to merge. and removed 1:ST:ready-for-review This PR is ready for review labels Jun 22, 2020

tcojean mentioned this pull request Jun 23, 2020

Release/1.2.0 #576

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Replace valgrind by all sanitizers available. #476

Replace valgrind by all sanitizers available. #476

tcojean commented Mar 5, 2020 •

edited

Loading

codecov bot commented Mar 6, 2020 •

edited

Loading

upsj left a comment

yhmtsai left a comment

pratikvn left a comment

thoasm left a comment

tcojean commented Mar 10, 2020 •

edited

Loading

thoasm left a comment

tcojean commented Mar 13, 2020

tcojean commented Mar 16, 2020 •

edited

Loading

tcojean commented Mar 16, 2020 •

edited

Loading

tcojean commented Mar 16, 2020

yhmtsai left a comment

sonarqubecloud bot commented Mar 17, 2020

pratikvn left a comment

Replace valgrind by all sanitizers available. #476

Replace valgrind by all sanitizers available. #476

Conversation

tcojean commented Mar 5, 2020 • edited Loading

codecov bot commented Mar 6, 2020 • edited Loading

Codecov Report

upsj left a comment

Choose a reason for hiding this comment

yhmtsai left a comment

Choose a reason for hiding this comment

pratikvn left a comment

Choose a reason for hiding this comment

thoasm left a comment

Choose a reason for hiding this comment

tcojean commented Mar 10, 2020 • edited Loading

thoasm left a comment

Choose a reason for hiding this comment

tcojean commented Mar 13, 2020

tcojean commented Mar 16, 2020 • edited Loading

tcojean commented Mar 16, 2020 • edited Loading

tcojean commented Mar 16, 2020

yhmtsai left a comment

Choose a reason for hiding this comment

sonarqubecloud bot commented Mar 17, 2020

pratikvn left a comment

Choose a reason for hiding this comment

tcojean commented Mar 5, 2020 •

edited

Loading

codecov bot commented Mar 6, 2020 •

edited

Loading

tcojean commented Mar 10, 2020 •

edited

Loading

tcojean commented Mar 16, 2020 •

edited

Loading

tcojean commented Mar 16, 2020 •

edited

Loading