Trilinos/Tribits: make dashboard submits to SNL regardless of config parameters.

[jjellio]

Bug Report

@bartlettroscoe (not sure what package this falls under)

Description

I've setup a CDash server and I've started trying to submit test builds to it.

I'm using:

CMake Setup

[jjellio@s1001943 build-stuff]$ cat ../test-dash.sh 
#!/bin/bash

module purge
module load  sems-env sems-git/2.10.1 sems-cmake/3.19.1 sems-ninja_fortran/1.10.0 sems-gcc/7.2.0 sems-openmpi/1.10.1

TRILINOS=$(realpath ..)
source $TRILINOS/cmake/std/atdm/load-env.sh sems-rhel7-gnu-release-static-serial

rm -rf ./*

# configure from scratch
cmake "$@" \
  -D Trilinos_ENABLE_ALL_PACKAGES=OFF \
  -D Trilinos_CONFIGURE_OPTIONS_FILE:STRING=cmake/std/atdm/ATDMDevEnv.cmake  \
  -D Trilinos_ENABLE_TESTS=ON \
  -D Trilinos_ENABLE_Tpetra=ON \
  "-DCTEST_DROP_SITE=s1001943.srn.sandia.gov:8080" \
  "-DCTEST_DROP_LOCATION=/submit.php?project=El-Cap-COE-Trilinos" \
  "-DCTEST_PROJECT_NAME=El-Cap-COE-Trilinos" \
  "-DCTEST_DROP_SITE_CDASH=TRUE" \
  "-DTRIBITS_2ND_CTEST_DROP_SITE=OFF" \
  "-DTRIBITS_2ND_CTEST_DROP_LOCATION=OFF" \
  "-DCTEST_BUILD_FLAGS=-j60" "-DCTEST_PARALLEL_LEVEL=12" \
  -G Ninja -D PYTHON_EXECUTABLE=$(which python3) \
$TRILINOS

Followed by make dashboard

After the thing finishes, I have output that looks like the following.
I've set the various I know of for sites. Yet, I still get http://testing-dev.sandia.gov/cdash/submit.php?project=Trilinos in the output. (I'm not sure if it's actually posting there)

Output

ile '/ssd_scratch/src/Trilinos-jje/build-stuff/Testing/Temporary/LastTestsFailed_20210430-1351.log' exists so there were non-passing tests!
info: using retry_args='RETRY_COUNT;5;RETRY_DELAY;3' for _ctest_submit call
	Add file: /ssd_scratch/src/Trilinos-jje/build-stuff/TrilinosRepoVersion.txt
	Add file: /ssd_scratch/src/Trilinos-jje/build-stuff/do-configure
Problem reading file: /ssd_scratch/src/Trilinos-jje/build-stuff/do-configure while creating notes
	Add file: /ssd_scratch/src/Trilinos-jje/build-stuff/CMakeCache.clean.txt
   Use HTTP Proxy: http://user:nopass@proxy.sandia.gov:80
Submit files
   Send to group: Experimental
   SubmitURL: http://s1001943.srn.sandia.gov:8080/submit.php?project=El-Cap-COE-Trilinos
   Uploaded: /home/jjellio/src/Trilinos-jje/build-stuff/Testing/20210430-1351/Test.xml
   Submission successful

Doing submit to second CDash site ...

info: using retry_args='RETRY_COUNT;5;RETRY_DELAY;3' for _ctest_submit call
	Add file: /ssd_scratch/src/Trilinos-jje/build-stuff/TrilinosRepoVersion.txt
	Add file: /ssd_scratch/src/Trilinos-jje/build-stuff/do-configure
Problem reading file: /ssd_scratch/src/Trilinos-jje/build-stuff/do-configure while creating notes
	Add file: /ssd_scratch/src/Trilinos-jje/build-stuff/CMakeCache.clean.txt
   Use HTTP Proxy: http://user:nopass@proxy.sandia.gov:80
Submit files
   Send to group: Experimental
   SubmitURL: http://testing-dev.sandia.gov/cdash/submit.php?project=Trilinos
   Uploaded: /home/jjellio/src/Trilinos-jje/build-stuff/Testing/20210430-1351/Test.xml
   Submission successful

Skipping converage tests because CTEST_DO_COVERAGE_TESTING='FALSE'!


Skipping memory testing because CTEST_DO_MEMORY_TESTING='FALSE'!


Done with the all-at-once configure, build, test, ans submit of Trilinos packages!


Final set of packages that had any failures: 'ALL_PACKAGES'
TRIBITS_CTEST_DRIVER_ERROR_QUEUE is empty. All is well.
CMake Error at /home/jjellio/src/Trilinos-jje/cmake/tribits/ctest_driver/TribitsCTestDriverCore.cmake:2337 (MESSAGE):
  

  See results for:

    Site: s1001943.srn.sandia.gov
    Build Name: Linux-build-stuff

  at:

    http://testing-dev.sandia.gov/cdash/index.php?project=Trilinos&display=project

  

  TRIBITS_CTEST_DRIVER: OVERALL: ALL FAILED

Call Stack (most recent call first):
  /home/jjellio/src/Trilinos-jje/cmake/tribits/ctest_driver/experimental_build_test.cmake:130 (TRIBITS_CTEST_DRIVER)

I suppose there are two questions:

1. Is that build actually reporting to the SNL site? Yes Here
2. If so, how do I stop it?

I pulled the variables used from
https://docs.trilinos.org/files/TrilinosBuildReference.html#id115
http://tribits.org/doc/TribitsBuildReference.html#dashboard-submissions

I got clever and set both sites equal to my dashboard, but still it posted to SNL's test site!

That's some dedication on the CI testing end ;)

Submit files
   Send to group: Experimental
   SubmitURL: http://testing-dev.sandia.gov/cdash/submit.php?project=Trilinos
   Uploaded: /home/jjellio/src/Trilinos-jje/build-stuff/Testing/20210430-1458/Test.xml
   Submission successful

[bartlettroscoe]

Yet, I still get http://testing-dev.sandia.gov/cdash/submit.php?project=Trilinos in the output. (I'm not sure if it's actually posting there)

That is coming from the second submit (see Specifying where the results go to CDash (TRIBITS_CTEST_DRIVER()):).

I think we need to move this discussion to an internal SNL Issue Tracker. I created:

• https://sems-atlassian-srn.sandia.gov/browse/ATDV-433

[bartlettroscoe]

@jjellio, FYI, PR #9139 should fix this problem and improve documentation to hopefully make it more clear how to use and manipulate the dashboard target. Note that PR #9139 also provides a good default for CTEST_BUILD_NAME when submitting to CDash with make dashboard.

[bartlettroscoe]

@jjellio,

As I mentioned in:

• https://sems-atlassian-srn.sandia.gov/browse/ATDV-433?focusedCommentId=81078&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-81078

the workaround before PR #9139 is merged is to run 'make dashboard' as:

$ env TRIBITS_2ND_CTEST_DROP_SITE=OFF TRIBITS_2ND_CTEST_DROP_LOCATION=OFF \
   make dashboard 

Merging PR #9139 allows you set:

   -DTRIBITS_2ND_CTEST_DROP_SITE=OFF \
   -DTRIBITS_2ND_CTEST_DROP_LOCATION=OFF \

in the CMake cache and now have to bother passing options in env to 'make dashboard'.

[jjellio]

This worked in a test - what really matters is that it isn't posting - and that seems to be working now. I did a few tests with CTEST_DO_SUBMIT=ON|OFF and checked the sems dashboard and all seems well. I also checked testing.sandia.gov and all seems well (no posts there)