Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[OSASINFRA] Add multi test configs support #1980

Merged

Conversation

eurijon
Copy link
Contributor

@eurijon eurijon commented Jun 28, 2024

Support multiple test configs or job definitions from the calling zuul job, i.e:

  cifmw_run_test_shiftstack_testconfig:
    - 4.17_ovnkubernetes_ipi.yaml
    - 4.17_ovnkubernetes_upi.yaml
    - 4.17_ovnkubernetes_ipi-proxy.yaml

The shiftstack role will run the shiftstack-qa automation for each test config defined in cifmw_run_test_shiftstack_testconfig.

The artifacts directory includes now an additional directory for each test config, as multiple test configs can run now from a Zuul job.

This commit adds the cifmw_shiftstack_manifests_dir param as well for the role generated manifest files and the cifmw_shiftstack_ansible_command_logs_dir param for the ansible command module output logs.

As a pull request owner and reviewers, I checked that:

  • Appropriate testing is done and actually running
  • Appropriate documentation exists and/or is up-to-date:
  • README in the role
  • Content of the docs/source is reflecting the changes

Copy link
Contributor

openshift-ci bot commented Jun 28, 2024

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/223d6be507864fecaae83c44f29db6c7

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 39m 19s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 21m 39s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 14m 23s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 7m 21s
cifmw-pod-pre-commit FAILURE in 7m 34s
cifmw-molecule-shiftstack FAILURE in 14m 50s

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/3f72ec8cc5514096bbc78f3bef74709d

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 12m 41s
podified-multinode-edpm-deployment-crc FAILURE in 1h 41m 02s
cifmw-crc-podified-edpm-baremetal FAILURE in 1h 37m 31s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 37s
cifmw-pod-pre-commit FAILURE in 7m 06s
cifmw-molecule-shiftstack FAILURE in 12m 56s

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/c281ac2afa4d47c49ae965d1f0cb1901

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 34m 50s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 14m 32s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 17m 06s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 34s
cifmw-pod-pre-commit FAILURE in 7m 52s
cifmw-molecule-shiftstack FAILURE in 12m 39s

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/4e8e654f55264e2387df71174609ddab

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 34m 58s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 12m 50s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 20m 38s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 21s
cifmw-pod-pre-commit FAILURE in 7m 04s
cifmw-molecule-shiftstack FAILURE in 12m 59s

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/ac51c74494a345738aa11abd175f8868

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 32m 56s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 12m 15s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 18m 14s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 7m 50s
cifmw-pod-pre-commit FAILURE in 7m 11s
cifmw-molecule-shiftstack FAILURE in 13m 55s

Copy link

Merge Failed.

This change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset.
Warning:
Error merging github.com/openstack-k8s-operators/ci-framework for 1980,cdb3ed4cb9f7bd0b204499921f0154be87af5818

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/2c73f25a3ffa43c0bd84c2c44fb2bbab

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 52m 55s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 14m 18s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 13m 39s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 7m 39s
cifmw-pod-pre-commit FAILURE in 7m 11s
build-push-container-cifmw-client TIMED_OUT in 1h 30m 42s
cifmw-molecule-shiftstack FAILURE in 12m 41s

@eurijon eurijon force-pushed the multi-testconfig branch 3 times, most recently from 17282f8 to a22dc75 Compare September 23, 2024 14:00
Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/79bf3ab7f05041bfbfb335f1b66de41c

openstack-k8s-operators-content-provider FAILURE in 6m 24s
⚠️ podified-multinode-edpm-deployment-crc SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
⚠️ cifmw-crc-podified-edpm-baremetal SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 9m 01s
cifmw-pod-pre-commit FAILURE in 10m 05s
✔️ build-push-container-cifmw-client SUCCESS in 20m 43s
cifmw-molecule-shiftstack FAILURE in 13m 48s

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/f067a5a3ad634c319fae9a5693f4d864

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 15m 06s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 19m 24s
cifmw-crc-podified-edpm-baremetal POST_FAILURE in 26m 25s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 7m 32s
cifmw-pod-pre-commit FAILURE in 6m 53s
✔️ build-push-container-cifmw-client SUCCESS in 31m 01s
cifmw-molecule-shiftstack FAILURE in 12m 06s

@eurijon eurijon force-pushed the multi-testconfig branch 2 times, most recently from ceb179d to 59a7ee4 Compare September 27, 2024 10:31
Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/a8064be99ccc4980a77ddd0e73793d86

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 48m 01s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 18m 02s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 28m 08s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 22s
cifmw-pod-pre-commit FAILURE in 7m 38s
✔️ build-push-container-cifmw-client SUCCESS in 30m 25s
✔️ cifmw-molecule-shiftstack SUCCESS in 17m 21s

@eurijon eurijon force-pushed the multi-testconfig branch 2 times, most recently from eed7a4e to c7794cb Compare October 2, 2024 08:36
@eurijon eurijon marked this pull request as ready for review October 2, 2024 08:37
@eurijon
Copy link
Contributor Author

eurijon commented Oct 21, 2024

@lewisdenny thanks for the review, I'm sorry I just added a minor change (appended the loop index to the test config artifacts file name)

Do you think we can add @rlobillo as reviewer for the shiftstack role?
cc @pablintino

@lewisdenny
Copy link
Collaborator

@lewisdenny thanks for the review, I'm sorry I just added a minor change (appended the loop index to the test config artifacts file name)

Do you think we can add @rlobillo as reviewer for the shiftstack role? cc @pablintino

Yes this is something we would like to enable.
Let me purpose a patch

@lewisdenny
Copy link
Collaborator

@lewisdenny thanks for the review, I'm sorry I just added a minor change (appended the loop index to the test config artifacts file name)
Do you think we can add @rlobillo as reviewer for the shiftstack role? cc @pablintino

Yes this is something we would like to enable. Let me purpose a patch

#2468

@lewisdenny
Copy link
Collaborator

@lewisdenny thanks for the review, I'm sorry I just added a minor change (appended the loop index to the test config artifacts file name)

Do you think we can add @rlobillo as reviewer for the shiftstack role? cc @pablintino

@rlobillo has been added to the github org now :)

@lewisdenny lewisdenny requested a review from a team October 22, 2024 10:26
Copy link
Collaborator

@pablintino pablintino left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

roles/shiftstack/tasks/pre_test_shiftstack.yml Outdated Show resolved Hide resolved
@@ -21,13 +21,13 @@
PATH: "{{ cifmw_path }}"
KUBECONFIG: "{{ cifmw_openshift_kubeconfig }}"
ansible.builtin.command:
cmd: "oc rsh -n {{ namespace }} {{ pod }} bash -c '{{ command }}'"
register: command_result
cmd: "oc rsh -n {{ namespace }} {{ pod_name }} bash -c '{{ command }}'"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Command/Shell injection.... This is word a bug in Jira.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We decided to go with this as we need to run ansible from the pod (another inventory, hosts and so on) and we didn't want to change ci-framework every time we do some change in our automation repo (shiftstack-qa).
In any case I'm open to discuss alternatives to the command injection.

Support multiple test configs or job definitions from the calling
zuul job, i.e:

      cifmw_run_test_shiftstack_testconfig:
        - 4.17_ovnkubernetes_ipi.yaml
        - 4.17_ovnkubernetes_upi.yaml
        - 4.17_ovnkubernetes_ipi-proxy.yaml

The shiftstack role will run the shiftstack-qa automation for
each test config defined in `cifmw_run_test_shiftstack_testconfig`.

The artifacts directory includes now an additional directory for
each test config, as multiple test configs can run now from a
Zuul job.

This commit adds the `cifmw_shiftstack_manifests_dir` param as
well for the role generated manifest files and the
`cifmw_shiftstack_ansible_command_logs_dir` param for the
ansible command module output logs.
@lewisdenny
Copy link
Collaborator

Adding back @pablintino lgtm, thanks for changing to assert, much nicer.

/lgtm

@openshift-ci openshift-ci bot added the lgtm label Oct 23, 2024
@openshift-merge-bot openshift-merge-bot bot merged commit cf200c7 into openstack-k8s-operators:main Oct 24, 2024
4 of 5 checks passed
@eurijon eurijon deleted the multi-testconfig branch October 24, 2024 08:02
@eurijon
Copy link
Contributor Author

eurijon commented Oct 24, 2024

/cherry-pick 18.0-fr1

@openshift-cherrypick-robot

@eurijon: new pull request created: #2488

In response to this:

/cherry-pick 18.0-fr1

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants