Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor and add integration tests of invalid REST parameters. #350

Merged

Conversation

aaronweeden
Copy link
Contributor

@aaronweeden aaronweeden commented Sep 7, 2023

Description

This PR refactors some of the integration tests and adds new ones that test the validation of REST parameters that are obtained via the getParam() method of BaseControllerProvider.

Tests performed

I developed the tests incrementally while running in the tools-ext-01.ccr.xdmod.org/xdmod-job_performance-10.0.0:rockylinux8-0.1 Docker container upgraded to the branch of xdmod from ubccr/xdmod#1768.

Also, in a Docker container running tools-ext-01.ccr.xdmod.org/xdmod-job_performance-10.5.0:rockylinux8-0.1:

  1. Run the following commands:
    export COMPOSER_ALLOW_SUPERUSER=1
    export XDMOD_REALMS='jobs,storage,cloud,supremm,jobefficiency'
    export XDMOD_IS_CORE=yes
    export XDMOD_INSTALL_DIR=/xdmod
    export XDMOD_TEST_MODE=upgrade
    export XDMOD_SRC_DIR=./../xdmod
    git clone https://github.com/aaronweeden/xdmod-supremm -b php-error-log /xdmod-supremm
    dnf module -y reset nodejs
    dnf module -y install nodejs:16
    git clone https://github.com/ubccr/xdmod $XDMOD_SRC_DIR
    cd /xdmod-supremm
    ln -s `pwd` $XDMOD_SRC_DIR/open_xdmod/modules/supremm
    ./tests/ci/scripts/install_mongo.sh
    composer install --no-progress
    composer config -g cache-files-ttl 315360000
    composer install -d $XDMOD_SRC_DIR --no-progress
    mkdir -p ~/phpunit
    mkdir -p /tmp/screenshots
    export XDMOD_SOURCE_DIR=/xdmod
    export SHIPPABLE_BUILD_DIR=/xdmod-supremm
    pushd $XDMOD_SRC_DIR
    ~/bin/buildrpm xdmod supremm
    popd
    ./tests/integration/scripts/bootstrap.sh
    ./tests/integration/scripts/validate.sh
    composer install -d $XDMOD_SRC_DIR --no-progress
    composer install --no-progress
    ./tests/ci/scripts/qa-test-setup.sh
    export XDMOD_INSTALL_DIR=/usr/share/xdmod
    ./tests/ci/scripts/post-install-test.sh
    sed -i 's/\$phpunit /$phpunit --debug /' ./tests/component/runtests.sh
    ./tests/component/runtests.sh > /component-before.txt
    composer install -d $XDMOD_SRC_DIR --no-progress
    sed -i 's/\$phpunit /$phpunit --debug /' ./tests/integration/runtests.sh
    ./tests/integration/runtests.sh > /integration-before.txt
    sed -i 's/\$phpunit /$phpunit --debug /' ./tests/regression/runtests.sh
    ./tests/regression/runtests.sh > /regression-before.txt
    ./tests/artifacts/update-artifacts.sh
    cd tests/unit/
    ../../vendor/bin/phpunit --debug . > /unit-before.txt
    cd /xdmod-supremm
    git clone https://github.com/aaronweeden/xdmod-supremm -b fix-rest-array-param-validation /xdmod-supremm-new
    unalias rm
    rm -r /xdmod-supremm/tests
    cp -r /xdmod-supremm{-new,}/tests
    sed -i 's/\$phpunit /$phpunit --debug /' ./tests/component/runtests.sh
    ./tests/component/runtests.sh > /component-after.txt
    composer install -d $XDMOD_SRC_DIR --no-progress
    sed -i 's/\$phpunit /$phpunit --debug /' ./tests/integration/runtests.sh
    ./tests/integration/runtests.sh > /integration-after.txt
    sed -i 's/\$phpunit /$phpunit --debug /' ./tests/regression/runtests.sh
    ./tests/regression/runtests.sh > /regression-after.txt
    ./tests/artifacts/update-artifacts.sh
    cd tests/unit/
    ../../vendor/bin/phpunit --debug . > /unit-after.txt
    cd /xdmod-supremm
    sed 's/IntegrationTests\\REST\\Efficiency\\EfficiencyTest/IntegrationTests\\REST\\EfficiencyTest/g' /integration-before.txt  | sed 's/IntegrationTests\\REST\\SupremmDataflow\\QualityTest::testQuality/IntegrationTests\\REST\\SupremmDataflowControllerProviderTest::testGetQuality/g' > /integration-before-edited.txt
    
  2. Compare /unit-{before,after}.txt and make sure the only difference is the amount of time the tests ran.
  3. Compare /component-{before,after}.txt and make sure the only difference is the amount of time the tests ran.
  4. Compare /regression-{before,after}.txt and make sure the only differences are randomized parameters, amount of time the tests ran, and possibly transient "other notices" related to small differences in column values.
  5. Compare /integration-{before-edited,after}.txt and make sure the differences are correct:
    1. Different parameters and 14 additional tests for IntegrationTests\REST\EfficiencyTest::testCPUUsageScatterPlotEndpointMalformedRequest.
    2. 3 fewer tests for IntegrationTests\REST\EfficiencyTest::testCPUUsageDrillownEndpointMalformedRequest.
    3. 4 additional tests for IntegrationTests\REST\EfficiencyTest::testGetHistogramDataParamValidation.
    4. 8 additional tests for IntegrationTests\REST\SupremmDataflowControllerProviderTest::testGetDbst
    5. Different parameters and 3 additional tests for IntegrationTests\REST\SupremmDataflowControllerProviderTest::testGetQuality.
    6. 103 tests -> 129 tests, 475 assertions -> 616 assertions.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My code follows the code style of this project as found in the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

@aaronweeden aaronweeden added this to the 11.0.0 milestone Sep 7, 2023
@aaronweeden aaronweeden changed the base branch from xdmod10.5 to xdmod11.0 October 6, 2023 19:30
@aaronweeden aaronweeden force-pushed the fix-rest-array-param-validation branch 2 times, most recently from 87f82a4 to 8e335ca Compare October 6, 2023 19:38
@aaronweeden aaronweeden force-pushed the fix-rest-array-param-validation branch 3 times, most recently from a9de499 to 2b5d19b Compare December 5, 2023 16:19
@aaronweeden aaronweeden added the qa label Dec 5, 2023
@aaronweeden aaronweeden force-pushed the fix-rest-array-param-validation branch 2 times, most recently from ef54ffb to 4b0fb1f Compare December 7, 2023 20:18
@aaronweeden aaronweeden marked this pull request as ready for review December 7, 2023 21:25
@aaronweeden aaronweeden force-pushed the fix-rest-array-param-validation branch 2 times, most recently from 01d05c1 to 3f827bf Compare December 11, 2023 20:31
@aaronweeden aaronweeden force-pushed the fix-rest-array-param-validation branch 2 times, most recently from efbd33e to 2bdbc1e Compare December 14, 2023 20:14
@aaronweeden aaronweeden force-pushed the fix-rest-array-param-validation branch from 2bdbc1e to d0f152a Compare December 21, 2023 21:42
@aaronweeden aaronweeden force-pushed the fix-rest-array-param-validation branch from d0f152a to 7e87746 Compare January 2, 2024 21:06
@aaronweeden aaronweeden merged commit 700f9ec into ubccr:xdmod11.0 Jan 2, 2024
3 checks passed
@aaronweeden aaronweeden deleted the fix-rest-array-param-validation branch January 2, 2024 21:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants