Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Workload tests adapted to centralized result logging #1910

Merged
merged 1 commit into from
Mar 19, 2024

Conversation

martin-mat
Copy link
Collaborator

@martin-mat martin-mat commented Feb 28, 2024

Description

Workload tests adapted to cenralized result logging

Issues:

Refs: #1861

How has this been tested:

  • Covered by existing integration testing
  • Added integration testing to cover
  • Verified all A/C passes
    • develop
    • master
    • tag/other branch
  • Test environment
    • Shared Packet K8s cluster
    • New Packet K8s cluster
    • Kind cluster
  • Have not tested

Types of changes:

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update

Checklist:

Documentation

  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • No updates required.

Code Review

  • Does the test handle fatal exceptions, ie. rescue block

Issue

  • Tasks in issue are checked off

@agentpoyo agentpoyo changed the title WIP: Workload tests adapted to cenralized result logging WIP: Workload tests adapted to centralized result logging Feb 28, 2024
@martin-mat martin-mat changed the title WIP: Workload tests adapted to centralized result logging Workload tests adapted to centralized result logging Feb 29, 2024
@martin-mat
Copy link
Collaborator Author

spec test result the same as for main branch (still some failures but those don't not seem to be caused by this change).
Ready for review.

@agentpoyo
Copy link
Collaborator

spec test result the same as for main branch (still some failures but those don't not seem to be caused by this change). Ready for review.

We'll try to pull down and push up to verify github actions with no issues and manual testing prior to merging.

task_start_time = Time.utc
testsuite_task = "smf_upf_core_validator"
Log.for(testsuite_task).info { "Starting test" }
CNFManager::Task.task_runner(args, task: t) do |args, config|
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@martin-mat Are we losing the ability to track test start/end times with this change?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

start time of every test case moved to the task runner
https://github.com/cnti-testcatalog/testsuite/blob/main/src/tasks/utils/task.cr#L113

and reported along with test case result the same way as before the changes
https://github.com/cnti-testcatalog/testsuite/blob/main/src/tasks/utils/task.cr#L121

I'm not aware that test case duration/end time is recorded anywhere. But possible to introduce easily if needed.

@martin-mat
Copy link
Collaborator Author

rebased

@HashNuke
Copy link
Collaborator

Just tried running the build for this. There are some tests that are failing on repeated attempts.

https://github.com/cnti-testcatalog/testsuite/actions/runs/8232211049

Signed-off-by: Martin Matyas <martin.matyas@tietoevry.com>
@martin-mat
Copy link
Collaborator Author

fixed.
@HashNuke in the link provided I saw following failures of the spec tests:
curl, helm, operator_test, points , validate_config

I fixed reasons for: helm, operator_test, points , validate_config.
Failing "curl" specs: I don't know, I think it is not related to this change. For me it is failing even on main branch.

Can you please re-run github actions? Thanks.

@HashNuke HashNuke merged commit 0b34594 into lfn-cnti:main Mar 19, 2024
100 of 190 checks passed
@HashNuke
Copy link
Collaborator

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants