Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update openstack-k8s-operators #690

Merged
merged 3 commits into from
Mar 8, 2024

Conversation

openstack-k8s-ci-robot
Copy link
Contributor

@openstack-k8s-ci-robot openstack-k8s-ci-robot commented Mar 2, 2024

This PR contains the following updates:

Package Type Update Change
github.com/openstack-k8s-operators/barbican-operator/api require digest cb4e05f -> 95cf5d9
github.com/openstack-k8s-operators/cinder-operator/api require digest f14a9a6 -> c2e2433
github.com/openstack-k8s-operators/dataplane-operator/api require digest 992e8b3 -> fdd88ea
github.com/openstack-k8s-operators/designate-operator/api require digest 6b52675 -> 073a542
github.com/openstack-k8s-operators/glance-operator/api require digest 4e4efb5 -> 6b5d600
github.com/openstack-k8s-operators/heat-operator/api require digest dc7919e -> f66e438
github.com/openstack-k8s-operators/horizon-operator/api require digest 8c50610 -> 946bc7d
github.com/openstack-k8s-operators/infra-operator/apis require digest 49edc0d -> 038a5ec
github.com/openstack-k8s-operators/keystone-operator/api require digest b1b853e -> 033a606
github.com/openstack-k8s-operators/lib-common/modules/certmanager require digest 169ced5 -> dc65ab4
github.com/openstack-k8s-operators/lib-common/modules/common require digest 169ced5 -> dc65ab4
github.com/openstack-k8s-operators/lib-common/modules/openstack indirect digest 169ced5 -> dc65ab4
github.com/openstack-k8s-operators/lib-common/modules/storage indirect digest 169ced5 -> dc65ab4
github.com/openstack-k8s-operators/lib-common/modules/storage require digest 169ced5 -> dc65ab4
github.com/openstack-k8s-operators/lib-common/modules/test require digest 169ced5 -> dc65ab4
github.com/openstack-k8s-operators/mariadb-operator/api require digest 76fef73 -> 81216f1
github.com/openstack-k8s-operators/neutron-operator/api require digest fe5c58a -> acb164b
github.com/openstack-k8s-operators/nova-operator/api require digest 17a8992 -> 2fc1351
github.com/openstack-k8s-operators/octavia-operator/api require digest aa54e85 -> 2dcd200
github.com/openstack-k8s-operators/openstack-ansibleee-operator/api require digest f9db481 -> 4221008
github.com/openstack-k8s-operators/openstack-baremetal-operator/api require digest b764323 -> 51d0de7
github.com/openstack-k8s-operators/ovn-operator/api require digest d42793e -> 826f326
github.com/openstack-k8s-operators/placement-operator/api require digest d2a5a5a -> ab60211
github.com/openstack-k8s-operators/swift-operator/api require digest 1394175 -> 1fa0278
github.com/openstack-k8s-operators/telemetry-operator/api require digest 444ba30 -> 25f01ea

Configuration

📅 Schedule: Branch creation - "every weekend" in timezone America/New_York, Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.


  • If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

@openshift-ci openshift-ci bot requested review from stuggi and viroel March 2, 2024 10:35
Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/4dad7e48f5c8449cac8037c7af11600f

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 32m 58s
podified-multinode-edpm-deployment-crc RETRY_LIMIT in 6m 15s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 17m 38s
✔️ cifmw-data-plane-adoption-osp-17-to-extracted-crc SUCCESS in 2h 12m 43s
✔️ openstack-operator-tempest-multinode SUCCESS in 1h 18m 10s

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/5b66c9b0b2104d4a899f5bfe3fc355d0

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 27m 45s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 07m 52s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 10m 12s
✔️ cifmw-data-plane-adoption-osp-17-to-extracted-crc SUCCESS in 2h 09m 07s
openstack-operator-tempest-multinode RETRY_LIMIT in 1h 23m 08s

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/50d1262d05cd4c1f9dd69f5a50a72ab9

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 28m 50s
podified-multinode-edpm-deployment-crc FAILURE in 1h 03m 05s
cifmw-crc-podified-edpm-baremetal FAILURE in 1h 03m 28s
cifmw-data-plane-adoption-osp-17-to-extracted-crc RETRY_LIMIT in 1h 08m 36s
openstack-operator-tempest-multinode FAILURE in 1h 04m 25s

@openstack-k8s-ci-robot openstack-k8s-ci-robot force-pushed the renovate/openstack-k8s-operators branch 2 times, most recently from 82b0831 to 0639560 Compare March 4, 2024 02:15
Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/e7fe228c50e545a99d455143c9aeb2b3

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 59m 26s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 06m 07s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 13m 37s
cifmw-data-plane-adoption-osp-17-to-extracted-crc FAILURE in 1h 41m 47s
✔️ openstack-operator-tempest-multinode SUCCESS in 1h 20m 05s

@stuggi
Copy link
Contributor

stuggi commented Mar 4, 2024

/test openstack-operator-build-deploy-kuttl

@stuggi
Copy link
Contributor

stuggi commented Mar 4, 2024

recheck

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/d27d4d022fd84f5e8322f9b695f5bdcf

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 31m 40s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 03m 42s
cifmw-crc-podified-edpm-baremetal RETRY_LIMIT in 2m 28s
✔️ cifmw-data-plane-adoption-osp-17-to-extracted-crc SUCCESS in 2h 13m 59s
✔️ openstack-operator-tempest-multinode SUCCESS in 1h 24m 02s

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/4029c8e962b1424b8028b660bdb35897

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 49m 29s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 02m 53s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 09m 17s
cifmw-data-plane-adoption-osp-17-to-extracted-crc FAILURE in 1h 30m 52s
✔️ openstack-operator-tempest-multinode SUCCESS in 1h 30m 13s

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/250014d39e4343bcbddbc55a7b7b2399

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 02m 57s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 06m 52s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 11m 14s
cifmw-data-plane-adoption-osp-17-to-extracted-crc FAILURE in 1h 43m 19s
✔️ openstack-operator-tempest-multinode SUCCESS in 1h 29m 29s

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/07051a9a66944f93a824cb951411c4e1

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 29m 23s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 05m 05s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 07m 55s
cifmw-data-plane-adoption-osp-17-to-extracted-crc FAILURE in 2h 10m 24s
✔️ openstack-operator-tempest-multinode SUCCESS in 1h 29m 30s

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/186118a89a4743cd98fea7d240c1428c

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 58m 37s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 04m 20s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 06m 33s
cifmw-data-plane-adoption-osp-17-to-extracted-crc FAILURE in 1h 40m 21s
✔️ openstack-operator-tempest-multinode SUCCESS in 1h 22m 29s

@gibizer
Copy link
Contributor

gibizer commented Mar 6, 2024

/hold we need to land #694 first then ask renovate to regenerate this PR

@fao89
Copy link
Contributor

fao89 commented Mar 6, 2024

/unhold

@openstack-k8s-ci-robot
Copy link
Contributor Author

Edited/Blocked Notification

Renovate will not automatically rebase this PR, because it does not recognize the last commit author and assumes somebody else may have edited the PR.

You can manually request rebase by checking the rebase/retry box above.

Warning: custom changes will be lost.

Akrog added a commit to Akrog/cinder-operator that referenced this pull request Mar 7, 2024
This patch updates the cinder-operator/api from 49edc0d to 038a5ec and
also updates the functional tests.

Tests need to be updated after adding MemcachedSpecCore
openstack-k8s-operators/infra-operator#191

Related-PR: openstack-k8s-operators/openstack-operator#690
Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/2b6a23261fcc49ad9b45e92a0d3ad28a

✔️ openstack-k8s-operators-content-provider SUCCESS in 3h 03m 14s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 08m 00s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 06m 40s
✔️ cifmw-data-plane-adoption-osp-17-to-extracted-crc SUCCESS in 2h 15m 34s
openstack-operator-tempest-multinode RETRY_LIMIT in 10m 24s

@stuggi
Copy link
Contributor

stuggi commented Mar 7, 2024

@zzzeek @gibizer

kuttl test failed in nova with:

  - lastTransitionTime: "2024-03-07T17:17:59Z"
    message: DB creation failed for cell0(accountName is empty),cell1(accountName
      is empty)
    reason: Error
    severity: Error
    status: "False"
    type: Ready

there is a mariadbaccount for nova-api, but not for cells:

https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/pr-logs/pull/openstack-k8s-operators_openstack-operator/690/pull-ci-openstack-k8s-operators-openstack-operator-main-openstack-operator-build-deploy-kuttl/1765779423744757760/artifacts/openstack-operator-build-deploy-kuttl/openstack-k8s-operators-gather/artifacts/must-gather/quay-io-openstack-k8s-operators-openstack-must-gather-sha256-25967b6a61adac75deffcaf718b8eb334ae5e8b663a475c9ba9b28567e781675/namespaces/openstack-kuttl-tests/crs/mariadbaccounts.mariadb.openstack.org/

from the nova-operator log at https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/pr-logs/pull/openstack-k8s-operators_openstack-operator/690/pull-ci-openstack-k8s-operators-openstack-operator-main-openstack-operator-build-deploy-kuttl/1765779423744757760/artifacts/openstack-operator-build-deploy-kuttl/openstack-k8s-operators-gather/artifacts/must-gather/quay-io-openstack-k8s-operators-openstack-must-gather-sha256-25967b6a61adac75deffcaf718b8eb334ae5e8b663a475c9ba9b28567e781675/namespaces/openstack-operators/pods/nova-operator-controller-manager-547df47b87-zkdpb/logs/manager.log .

first tests had the cell0/cell1 account , e.g.
2024-03-07T17:14:21.136Z INFO Controllers.Nova Successfully ensured MariaDBAccount nova-cell0 exists; database username is nova_cell0_eef9 {"controller": "nova", "controllerGroup": "nova.openstack.org", "controllerKind": "Nova", "Nova": {"name":"nova","namespace":"openstack-kuttl-tests"}, "namespace": "openstack-kuttl-tests", "name": "nova", "reconcileID": "16823c27-9d78-4e86-8cc1-1a4fd9b124b3", "ObjectType": "*v1beta1.MariaDBAccount", "ObjectNamespace": "openstack-kuttl-tests", "ObjectName": "nova-cell0"}

later logs only show waiting
2024-03-07T17:18:14.459Z INFO Controllers.Nova Skipping NovaCell as waiting for the cell DB to be created {"controller": "nova", "controllerGroup": "nova.openstack.org", "controllerKind": "Nova", "Nova": {"name":"nova","namespace":"openstack-kuttl-tests"}, "namespace": "openstack-kuttl-tests", "name": "nova", "reconcileID": "1937e58e-c7df-4f98-8504-8df23d44747c", "CellName": "cell0"}

@zzzeek
Copy link
Contributor

zzzeek commented Mar 7, 2024

@zzzeek @gibizer

kuttl test failed in nova with:

  - lastTransitionTime: "2024-03-07T17:17:59Z"
    message: DB creation failed for cell0(accountName is empty),cell1(accountName
      is empty)

I believe this means the crd (make manifests install when we work from a dev checkout) is still incorrect for nova. the databaseAccount fields have default values as well as that when nova creates cells, it populates account name the cell database account fields are required so the CR wouldn't go in if it didnt have them.

@zzzeek
Copy link
Contributor

zzzeek commented Mar 7, 2024

if i can deploy a control plane locally using "make openstack" with the latest nova operator, that would indicate nova works, right?

@zzzeek
Copy link
Contributor

zzzeek commented Mar 8, 2024

OK I can reproduce this with the current openstack-operator, I didnt try with this PR but it's likely the same thing. The nova operator has the correct templates, but the cellDatabaseAccount is not filled in becuase openstack-operator has to send it. So when doing a "make openstack_deploy", the nova CR looks like this:

  cellTemplates:
      cell0:
        cellDatabaseAccount: ""
        cellDatabaseInstance: openstack
        cellMessageBusInstance: rabbitmq

so that's wrong, that cellDatatabaseAccount should be filled in (and it's required, nova doesn't generate this, which I thought it did for a minute). I'm not sure what kubebuilder-required or whatever gets us here if it's happy to put a blank string there, I guess it just means "the key is present" :). well thanks :) ideally it would throw a big error for the blank string like that.

then over on the openstack-operator side inside the openstackcontrolplane CR we see this (note this is not with this PR), it wants to do the right thing (which means it can be fixed) but it's currently wrong:

  cellTemplates:
          cell0:
            cellDatabaseInstance: openstack
            cellDatabaseUser: nova_cell0
            cellMessageBusInstance: rabbitmq
            conductorServiceTemplate:

so that's also wrong. I think we earlier had said we would just remove these, but that's wrong. that databaseuser with underscore needs to become databaseaccount with a dash, like this:

  cellTemplates:
          cell0:
            cellDatabaseInstance: openstack
            cellDatabaseAccount: nova-cell0
            cellMessageBusInstance: rabbitmq
            conductorServiceTemplate:

right now, with the blank cellDatabaseAccount, my nova has the error we see:

message: NovaCell creation waits for DB creation for cell0,cell1

and I've got the other log message:

Skipping NovaCell as waiting for the cell DB to be created 

and there's no MariaDBAccount, because there can't be, there's no account name. so yes I can reproduce it, easy to fix.

What I do not see, which is also phew makes sense, are those messages regarding "Successfully ensured MariaDBAccount nova-cell0 exists;" on this end. that's not possible because there is no "nova-cell0" name indicated anywhere. So im not sure if a kuttl template is filling those in, maybe they conflict with the openstackoperator CR and they get deleted? i dont know this stuff that deeply yet to really know.

OK so I go and oc edit openstackcontrolplane openstack-galera-network-isolation and replace that username with an account name, making it identifiable as what I'm typing here to ensure these names arent flying in from somewhere else (making sure we are using dashes, not underscores):

    cellTemplates:
        cell0:
          cellDatabaseInstance: openstack
          cellDatabaseAccount: mikes-nova-cell0
          cellMessageBusInstance: rabbitmq
     ...

       cell1:
          cellDatabaseInstance: openstack-cell1
          cellDatabaseAccount: mikes-nova-cell1


I also blow away those passwordSelectors: database: NovaCell0DatabasePassword as those aren't in the CR anymore.

then...using the way I know how to do this at the moment, I delete the "nova" so that the openstackcontrolplane recreates it, oc delete nova nova....and that doesnt work yet because openstackcontrolplane does not know how to fill in cellDatabaseAccount yet, OK SO, let me populate those in the nova nova manually

$ oc edit nova nova

...

  cellTemplates:
    cell0:
      cellDatabaseAccount: mikes-nova-cell0
      cellDatabaseInstance: openstack


etc.

then we finally see those accounts:

[classic@localhost mariadb-operator]$ oc get mariadbaccount
NAME               AGE
barbican           17m
cinder             17m
glance             17m
keystone           18m
mikes-nova-cell0   7s
mikes-nova-cell1   7s
neutron            17m
nova-api           111s
placement          18m

the databases are there too, here's with cell0

 oc rsh openstack-galera-0 mysql -u root -p12345678 -e "show databases"
Defaulted container "galera" out of: galera, mysql-bootstrap (init)
+--------------------+
| Database           |
+--------------------+
| barbican           |
| cinder             |
| glance             |
| information_schema |
| keystone           |
| mysql              |
| neutron            |
| nova_api           |
| nova_cell0         |
| performance_schema |
| placement          |
| sys                |
+--------------------+

then after 5 minutes of panic I found cell1 which is of course in a separate galera

$ oc rsh openstack-cell1-galera-0 mysql -u root -p12345678 -e "show databases"
Defaulted container "galera" out of: galera, mysql-bootstrap (init)
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| nova_cell1         |
| performance_schema |
| sys                |
+--------------------+

OK so TL;DR; I can reproduce the problems above and we need to modify openstack-operator to fully specify and transfer to the nova CR it creates the cellDatabaseAccount: nova-cell0 keys, and we can also remove the passwordselectors

@gibizer
Copy link
Contributor

gibizer commented Mar 8, 2024

OK I can reproduce this with the current openstack-operator, I didnt try with this PR but it's likely the same thing. The nova operator has the correct templates, but the cellDatabaseAccount is not filled in becuase openstack-operator has to send it. So when doing a "make openstack_deploy", the nova CR looks like this:

  cellTemplates:
      cell0:
        cellDatabaseAccount: ""
        cellDatabaseInstance: openstack
        cellMessageBusInstance: rabbitmq

so that's wrong, that cellDatatabaseAccount should be filled in (and it's required, nova doesn't generate this, which I thought it did for a minute). I'm not sure what kubebuilder-required or whatever gets us here if it's happy to put a blank string there, I guess it just means "the key is present" :). well thanks :) ideally it would throw a big error for the blank string like that.

This look like a defaulting issue somewhere. The NovaSpec.cellTemplate is defaulted to have the account names defined https://github.com/openstack-k8s-operators/nova-operator/blob/2fc1351673afc28eb493b8d4d9feb2810197a199/api/v1beta1/nova_types.go#L52

Then in the CellTemplate the field is required https://github.com/openstack-k8s-operators/nova-operator/blob/2fc1351673afc28eb493b8d4d9feb2810197a199/api/v1beta1/novacell_types.go#L42-L44

then over on the openstack-operator side inside the openstackcontrolplane CR we see this (note this is not with this PR), it wants to do the right thing (which means it can be fixed) but it's currently wrong:

  cellTemplates:
          cell0:
            cellDatabaseInstance: openstack
            cellDatabaseUser: nova_cell0
            cellMessageBusInstance: rabbitmq
            conductorServiceTemplate:

If openstack-operator generates this then openstack-operator does not see the new Nova CRD where the cellDatabaseUser field is removed.

@gibizer
Copy link
Contributor

gibizer commented Mar 8, 2024

OK I can reproduce this with the current openstack-operator, I didnt try with this PR but it's likely the same thing. The nova operator has the correct templates, but the cellDatabaseAccount is not filled in becuase openstack-operator has to send it. So when doing a "make openstack_deploy", the nova CR looks like this:

  cellTemplates:
      cell0:
        cellDatabaseAccount: ""
        cellDatabaseInstance: openstack
        cellMessageBusInstance: rabbitmq

so that's wrong, that cellDatatabaseAccount should be filled in (and it's required, nova doesn't generate this, which I thought it did for a minute). I'm not sure what kubebuilder-required or whatever gets us here if it's happy to put a blank string there, I guess it just means "the key is present" :). well thanks :) ideally it would throw a big error for the blank string like that.

This look like a defaulting issue somewhere. The NovaSpec.cellTemplate is defaulted to have the account names defined https://github.com/openstack-k8s-operators/nova-operator/blob/2fc1351673afc28eb493b8d4d9feb2810197a199/api/v1beta1/nova_types.go#L52

Then in the CellTemplate the field is required https://github.com/openstack-k8s-operators/nova-operator/blob/2fc1351673afc28eb493b8d4d9feb2810197a199/api/v1beta1/novacell_types.go#L42-L44

It seems like this CRD definition does not contain our default

https://github.com/openstack-k8s-operators/openstack-operator/pull/690/files#diff-6a25e1ed1759e6ece5de4318bef75ae0d857f040ce08b6cf1feee411cca036c6R9003-R9011

but this has the proper defaults:
https://github.com/openstack-k8s-operators/openstack-operator/pull/690/files#diff-6a25e1ed1759e6ece5de4318bef75ae0d857f040ce08b6cf1feee411cca036c6R9379-R9387
So I'm not sure why those are not used.

@gibizer
Copy link
Contributor

gibizer commented Mar 8, 2024

Wait a sec. This needs to be something in the kuttl job as the zuul job https://review.rdoproject.org/zuul/build/cf9b91a2f939442c86ead8babc495082/logs managed to deploy the control plane and data plane and run a tempest smoke test successfully.

Copy link
Contributor

@gibizer gibizer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will push a fix

Comment on lines 80 to 86
cellDatabaseUser: nova_cell0
conductorServiceTemplate:
replicas: 0
hasAPIAccess: true
cell1:
cellDatabaseUser: nova_cell1
cellDatabaseInstance: openstack
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I see what is happening here. Originally we defined the cellDatabaseUser fields here explicitly as the defaulting only works properly if the celltemplate is not defined at all. If a field is defined in it then the rest is not defaulted according to the kubebuilder defaults. Which make sense as the defaults are defined in the template field level https://github.com/openstack-k8s-operators/nova-operator/blob/2fc1351673afc28eb493b8d4d9feb2810197a199/api/v1beta1/nova_types.go#L52
not on the individual field level within the template struct. (we tried to do that defaulting as well but that fails due to webhhooks ser/des the struct). So the solution here is to explicitly confiuger the database account name as well. In the other sample files we don't define the cell template at all and there the defaulting works.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually what I wrote here belongs to the collapsed cell sample file that is used as an input in the kuttl test. This is just the assert where we could drop the user and don't have to assert the account. But the sample file needs to explicitlly provide the account due to defaulting quirks

@openshift-ci openshift-ci bot removed the lgtm label Mar 8, 2024
Copy link
Contributor

openshift-ci bot commented Mar 8, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: openstack-k8s-ci-robot, stuggi

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

The cellDatabaseUser field is replaced with the cellDatabaseAccount.
Most of the sample and kuttl test assert files does not define the
cellTemplates and therefore they are defaulted by nova-operator. However
in the collapsed cell sample we need to finetune the cell template
fields and therefore some of the cellTemplate fields are explicitly
defined. However if at least one of the field is defined then the struct
level defaulting does not triggered, so the rest of the fields needs to
be define too, otherwise we get the golang default values.

This patch makes sure that cellDatabaseAccount field is defined in the
samples where we explicitly define cellTemplates.
@gibizer gibizer force-pushed the renovate/openstack-k8s-operators branch from 863148e to 33b7fa9 Compare March 8, 2024 09:01
@stuggi
Copy link
Contributor

stuggi commented Mar 8, 2024

keystone failed with :

  - lastTransitionTime: "2024-03-08T09:36:24Z"
    message: 'DB create job error occurred Error create or update DB object keystone
      *v1beta1.MariaDBDatabase openstack-kuttl-tests/keystone: MariaDBDatabase.mariadb.openstack.org
      "keystone" is invalid: metadata.finalizers: Forbidden: no new finalizers can
      be added if the object is being deleted, found new finalizers []string{"KeystoneAPI"}'

I had seen this in my env once or so where cleanup of mariadbdatabase failed. the keystone mariadbdatabase is still the one from previous test which has the deletionTimestamp set:

https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/pr-logs/pull/openstack-k8s-operators_openstack-operator/690/pull-ci-openstack-k8s-operators-openstack-operator-main-openstack-operator-build-deploy-kuttl/1766026521958420480/artifacts/openstack-operator-build-deploy-kuttl/openstack-k8s-operators-gather/artifacts/must-gather/quay-io-openstack-k8s-operators-openstack-must-gather-sha256-25967b6a61adac75deffcaf718b8eb334ae5e8b663a475c9ba9b28567e781675/namespaces/openstack-kuttl-tests/crs/mariadbdatabases.mariadb.openstack.org/keystone.yaml

lets rerun the kuttl and see if this is a permanent thing or we can fix as a follow up

@stuggi
Copy link
Contributor

stuggi commented Mar 8, 2024

/test openstack-operator-build-deploy-kuttl

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/0014205479fa4f649f527f2a0df5492b

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 57m 38s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 06m 39s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 09m 43s
cifmw-data-plane-adoption-osp-17-to-extracted-crc FAILURE in 1h 38m 59s
✔️ openstack-operator-tempest-multinode SUCCESS in 1h 22m 51s

@stuggi
Copy link
Contributor

stuggi commented Mar 8, 2024

for reference, we suppose that openstack-k8s-operators/mariadb-operator#206 might fix the issue.

@stuggi stuggi added the lgtm label Mar 8, 2024
Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/da6f39c6d11b49c4b07e7cfdae4476ff

openstack-k8s-operators-content-provider TIMED_OUT in 30m 57s
⚠️ podified-multinode-edpm-deployment-crc SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
⚠️ cifmw-crc-podified-edpm-baremetal SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
⚠️ cifmw-data-plane-adoption-osp-17-to-extracted-crc SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
⚠️ openstack-operator-tempest-multinode SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider

@olliewalsh
Copy link
Contributor

/retest

@olliewalsh
Copy link
Contributor

recheck

@openshift-merge-bot openshift-merge-bot bot merged commit c311bb0 into main Mar 8, 2024
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants