Code

Feature 685 log updates#1992

Code

Merged

georgemccabe merged 59 commits intodevelopdtcenter/METplus:developfrom feature_685_log_updatesdtcenter/METplus:feature_685_log_updatesCopy head branch name to clipboardDec 19, 2022

+456-372

Commits

Commits on Dec 14, 2022

• clean up to reduce number of indents to make code more readable

  georgemccabe

  committed
  de77082

  Copy full SHA for de77082

• simplify logic for finding log filenames, remove unused function

  georgemccabe

  committed
  a945632

  Copy full SHA for a945632

• simplified logic to getting metplus log file and output appropriate log if logging to terminal only

  georgemccabe

  committed
  11fa333

  Copy full SHA for 11fa333

• remove print Running METplus version because the same info is logged in config setup

  georgemccabe

  committed
  b8fb37f

  Copy full SHA for b8fb37f

• change logs that are output to screen before log file is initialized to print statements so it is clear that they are not included in the logs. Removed extra info about config setup that is not rel…

  Show description for d6f2177
  georgemccabe

  committed
  d6f2177

  Copy full SHA for d6f2177

• added function to get useful message if log file is not set and call function when logging log file so it is clear that a log file is not used

  georgemccabe

  committed
  4fb2fa2

  Copy full SHA for 4fb2fa2

• skip log message with logfile path if no log file is set

  georgemccabe

  committed
  cd430ff

  Copy full SHA for cd430ff

• log all environment variable values at DEBUG level

  georgemccabe

  committed
  79c2f1d

  Copy full SHA for 79c2f1d

• added support for setting LOG_LEVEL_TERMINAL to set log level for screen output to a different value than the log file level

  georgemccabe

  committed
  26d8c36

  Copy full SHA for 26d8c36

• change terminal log level to warning to limit screen output when a pytest fails

  georgemccabe

  committed
  ea40378

  Copy full SHA for ea40378

• remove unnecessary extra variable, ci-run-all-cases

  georgemccabe

  committed
  9394f94

  Copy full SHA for 9394f94

• change log level for automated tests from DEBUG to INFO to see how much faster the tests run, ci-run-all-cases

  georgemccabe

  committed
  20e0728

  Copy full SHA for 20e0728

Commits on Dec 15, 2022

• run GHA tests with log file and terminal level DEBUG, ci-run-all-cases

  georgemccabe

  committed
  b31f69d

  Copy full SHA for b31f69d

• run GHA tests with log file and terminal level INFO, ci-run-all-cases

  georgemccabe

  committed
  4ddf6ae

  Copy full SHA for 4ddf6ae

• changed logs that list the length of time to run to INFO level, added 'took' to command timing log because that word can be searched to find all timing log info

  georgemccabe

  committed
  e7d5587

  Copy full SHA for e7d5587

• turn on use case to test

  georgemccabe

  committed
  b219ac5

  Copy full SHA for b219ac5

• separate out setup commands from use case commands, only add status variable checks if not running in Docker because Docker logic will be updated to run each use case in separate docker exec commands

  georgemccabe

  committed
  83a5ddb

  Copy full SHA for 83a5ddb

• create Docker container and run setup commands, then docker exec each use case command separately

  georgemccabe

  committed
  901e332

  Copy full SHA for 901e332

• add name identifier for container so exec and stop commands can find it

  georgemccabe

  committed
  e287614

  Copy full SHA for e287614

• docker run detached and run docker ps and images commands through run_docker_commands

  georgemccabe

  committed
  84d9d7d

  Copy full SHA for 84d9d7d

• start container in interactive mode detached, then exec commands to it, ci-skip-unit-tests

  georgemccabe

  committed
  e390f16

  Copy full SHA for e390f16

• do not exec detached because it will always return 0 error code, remove container at end of tests instead of stop, ci-skip-unit-tests

  georgemccabe

  committed
  e9b7377

  Copy full SHA for e9b7377

• run docker exec commands with bash login shell (-l), try adding export statement to bashrc so it will be read on each login shell, renamed constant variable to uppercase, ci-skip-unit-tests

  georgemccabe

  committed
  3815b0e

  Copy full SHA for 3815b0e

• run docker build command through docker command function

  georgemccabe

  committed
  9b1b4af

  Copy full SHA for 9b1b4af

• added missing semicolon

  georgemccabe

  committed
  d72f792

  Copy full SHA for d72f792

• force remove container at end of tests, ci-skip-unit-tests

  georgemccabe

  committed
  bd6cc1e

  Copy full SHA for bd6cc1e

• fix adding export to bashrc, ci-skip-unit-tests

  georgemccabe

  committed
  9a453b6

  Copy full SHA for 9a453b6

• another test, ci-skip-unit-tests

  georgemccabe

  committed
  9817a1e

  Copy full SHA for 9817a1e

• comment out broken function

  georgemccabe

  committed
  8af5c65

  Copy full SHA for 8af5c65

• try grouping GHA log output, ci-skip-unit-tests

  georgemccabe

  committed
  56c4999

  Copy full SHA for 56c4999

• add GHA log group for each docker command

  georgemccabe

  committed
  7674b27

  Copy full SHA for 7674b27

• another attempt at setting path, ci-skip-unit-tests

  georgemccabe

  committed
  3505446

  Copy full SHA for 3505446

• end group before error message so it is visible at top level, fix cat command to run inside docker, ci-skip-unit-tests

  georgemccabe

  committed
  ca5deff

  Copy full SHA for ca5deff

• add any env var assignment to bashrc, changed setup_env items into a list that is joined by semicolon at the end, ci-skip-unit-tests

  georgemccabe

  committed
  7e96fb3

  Copy full SHA for 7e96fb3

• remove source of bashrc file because running with login shell, added comment to note which values were added by test script

  georgemccabe

  committed
  3a1ab23

  Copy full SHA for 3a1ab23

Commits on Dec 16, 2022

• added GHA log groups for all commands run through time_command, moved TIMING info after ERROR info to match other command order

  georgemccabe

  committed
  806f768

  Copy full SHA for 806f768

• fix spacing between functions

  georgemccabe

  committed
  34e1abd

  Copy full SHA for 34e1abd

• clean up GHA log grouping, ci-skip-unit-tests

  georgemccabe

  committed
  5fa6c63

  Copy full SHA for 5fa6c63

• put TIMING inside GHA log group and output error message outside group, ci-skip-unit-tests

  georgemccabe

  committed
  39983fb

  Copy full SHA for 39983fb

• turn off use case group after tests, ci-run-all-diff

  georgemccabe

  committed
  5e6c4c5

  Copy full SHA for 5e6c4c5

• add GitHub Actions log group for full diff results so it is easier to read

  georgemccabe

  committed
  f774bff

  Copy full SHA for f774bff

• created python function to run commands with timing info, error reporting, and GitHub Actions log grouping

  georgemccabe

  committed
  39b8b8f

  Copy full SHA for 39b8b8f

• call run_commands function to consistently run processes and group logs, cleaned up logic to capitalize constants and created helper functions for better readability

  georgemccabe

  committed
  69da329

  Copy full SHA for 69da329

• changed calling of diff logic to start detached interactive container, run diff logic, and remove container to better organize log output in GitHub Actions

  georgemccabe

  committed
  1baa8c0

  Copy full SHA for 1baa8c0

• turn on use case to test, ci-skip-unit-tests, ci-run-diff

  georgemccabe

  committed
  741bf67

  Copy full SHA for 741bf67

• fixed typo in import, ci-skip-unit-tests, ci-run-diff

  georgemccabe

  committed
  3dd5b6a

  Copy full SHA for 3dd5b6a

• turn off use case after test

  georgemccabe

  committed
  c9a686c

  Copy full SHA for c9a686c

• turn on use case group and add typo to conf to create example of how GHA logs look when a use case fails, ci-skip-unit-tests, ci-run-diff

  georgemccabe

  committed
  22cb0ce

  Copy full SHA for 22cb0ce

• Revert "turn on use case group and add typo to conf to create example of how GHA logs look when a use case fails, ci-skip-unit-tests, ci-run-diff"

  Show description for f7bebac
  georgemccabe

  committed
  f7bebac

  Copy full SHA for f7bebac

• clean up function call because run_commands now accepts a string instead of requiring a list

  georgemccabe

  committed
  d80e8f4

  Copy full SHA for d80e8f4

• added documentation for LOG_LEVEL_TERMINAL and cleaned up info about LOG_LEVEL

  georgemccabe

  committed
  e1fe6f6

  Copy full SHA for e1fe6f6

• if LOG_LEVEL_TERMINAL is set so that INFO log message are not output to the screen, then print important log info to the screen like starting and stopping info

  georgemccabe

  committed
  84767d0

  Copy full SHA for 84767d0

• set file log level to DEBUG for automated tests so that the screen output is clean with INFO level and the log files that can be downloaded contain additional DEBUG logs

  georgemccabe

  committed
  3bad7ef

  Copy full SHA for 3bad7ef

• fixed bug in automated logic for getting Dockerfile to run tests

  georgemccabe

  committed
  64e82ad

  Copy full SHA for 64e82ad

Commits on Dec 19, 2022

• gather list of use cases that failed or could not be run to print summary at end of all use case runs

  georgemccabe

  committed
  205d460

  Copy full SHA for 205d460

• turn on use case group to test, ci-skip-unit-tests

  georgemccabe

  committed
  c679f3e

  Copy full SHA for c679f3e

• purposely cause a use case to fail, ci-skip-unit-tests

  georgemccabe

  committed
  b8722a0

  Copy full SHA for b8722a0

• Revert "purposely cause a use case to fail, ci-skip-unit-tests"

  Show description for c682a78
  georgemccabe

  committed
  c682a78

  Copy full SHA for c682a78

• Revert "turn on use case group to test, ci-skip-unit-tests"

  Show description for e39cd0d
  georgemccabe

  committed
  e39cd0d

  Copy full SHA for e39cd0d