Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace greenwave message consumer with resultsdb and waiverdb message consumers #4224

Closed
AdamWill opened this issue May 21, 2021 · 2 comments · Fixed by #4230
Closed

Replace greenwave message consumer with resultsdb and waiverdb message consumers #4224

AdamWill opened this issue May 21, 2021 · 2 comments · Fixed by #4230

Comments

@AdamWill
Copy link
Contributor

AdamWill commented May 21, 2021

As detailed in #4219 (comment) , I feel like the current design where Greenwave publishes 'decision change' messages and Bodhi listens for them to update the gating status for updates is fundamentally problematic. It requires greenwave to duplicate logic and knowledge from greenwave clients (Bodhi, in this case) which it should not do.

I think a more fundamentally sensible design would be for Bodhi to replace its greenwave 'decision change' message consumer with a resultsdb message consumer which would listen out for results that might potentially change a decision, and query Greenwave for an updated decision when they see one. And, I guess, a waiverdb consumer that would look out for waiver changes similarly.

Bodhi already has a lot of knowledge about what kinds of results and waivers are significant and what decisions to ask Greenwave for; this consumer would only have to reuse that. And it would allow us to throw away all the duplication of that knowledge on the Greenwave side.

@hluk @mattiaverga for comment.

@AdamWill AdamWill changed the title Replace greenwave message consumer with a resultsdb message consumer Replace greenwave message consumer with resultsdb and waiverdb message consumers May 21, 2021
@AdamWill
Copy link
Contributor Author

Worth noting that I'm kind of filing this as an alternative to filing several issues for making the current design work properly with openQA test gating. It currently does not work properly for several reasons, which would require changes on both Bodhi and Greenwave sides to fix. If others agree that the current design is kind of not the best idea, I would rather work on replacing it than work on digging the hole even deeper. :D

@AdamWill
Copy link
Contributor Author

AdamWill commented Jun 3, 2021

So I decided to just go ahead and write the ResultsDB handler as a PoC. It didn't seem very hard. This isn't tested yet, but I think it should more or less work (typos, thinkos etc. excepted) and do the job:

# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along with
# this program; if not, write to the Free Software Foundation, Inc., 51
# Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
"""
The "resultsdb handler".

This module is responsible for listening for messages from resultsdb.
If a message seems like it might be a result change for an update or
build from an update, we re-check the gating decision for that update.
"""

import logging

import fedora_messaging

from bodhi.server.models import Build, TestGatingStatus, Update
from bodhi.server.util import transactional_session_maker

log = logging.getLogger(__name__)


class ResultsdbHandler:
    """
    The Bodhi ResultsDB Handler.

    A fedora-messaging listener waiting for messages from resultsdb that may
    affect gating status for an update.
    """

    def __init__(self):
        """Initialize the handler."""
        self.db_factory = transactional_session_maker()

    def __call__(self, message: fedora_messaging.api.Message):
        """Handle messages arriving with the configured topic."""
        msg = message.body
        if not msg:
            log.debug("Ignoring message without body.")
            return

        restype = msg.get("data", {}).get("type")

        if restype is None:
            log.debug("Couldn't find result type in ResultsDB message")
            return
        if restype not in ("koji_build", "bodhi_update"):
            log.debug(f"Irrelevant result type {restype}")
            return

        passed = False
        if msg["outcome"] in ("PASSED", "INFO"):
            passed = True

        with self.db_factory():
            # find the update
            if restype == "bodhi_update":
                updateid = msg["data"]["item"]
                update = Update.get(updateid)
            else:
                nvr = msg["data"]["nvr"]
                build = Build.get(nvr)
                if build is None:
                    log.debug(f"Couldn't find build {subject_identifier} in DB")
                    return
                update = build.update

            # update the gating status if there's a chance it changed
            status = update.test_gating_status
            if (
                passed and status in (TestGatingStatus.waiting, TestGatingStatus.failed) or
                not passed and status == TestGatingStatus.passed
            ):
                log.info(f"Updating the test_gating_status for: {update.alias}")
                update.update_test_gating_status()

I'll maybe polish it up into a full PR (and add a waiverdb handler) tomorrow.

AdamWill added a commit to AdamWill/bodhi that referenced this issue Jun 10, 2021
…a#4224)

This replaces the consumer that listens for Greenwave 'decision
change' messages with consumers that listen for ResultsDB and
WaiverDB "new result" / "new waiver" messages and update gating
decisions if the message may cause a change.

The goal here is to make it possible to remove the "decision
change" message code from Greenwave entirely, because in order
to publish those messages, Greenwave has to duplicate a lot of
Bodhi's knowledge about what kinds of decisions to ask for. This
being in Greenwave is really against the intended design of the
system, and means whenever anything changes about what kinds of
decision Bodhi might request, we need to change both Bodhi and
the Greenwave "decision change" publishing code.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
AdamWill added a commit to AdamWill/bodhi that referenced this issue Jun 11, 2021
…a#4224)

This replaces the consumer that listens for Greenwave 'decision
change' messages with consumers that listen for ResultsDB and
WaiverDB "new result" / "new waiver" messages and update gating
decisions if the message may cause a change.

The goal here is to make it possible to remove the "decision
change" message code from Greenwave entirely, because in order
to publish those messages, Greenwave has to duplicate a lot of
Bodhi's knowledge about what kinds of decisions to ask for. This
being in Greenwave is really against the intended design of the
system, and means whenever anything changes about what kinds of
decision Bodhi might request, we need to change both Bodhi and
the Greenwave "decision change" publishing code.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
AdamWill added a commit to AdamWill/bodhi that referenced this issue Jun 11, 2021
…a#4224)

This replaces the consumer that listens for Greenwave 'decision
change' messages with consumers that listen for ResultsDB and
WaiverDB "new result" / "new waiver" messages and update gating
decisions if the message may cause a change.

The goal here is to make it possible to remove the "decision
change" message code from Greenwave entirely, because in order
to publish those messages, Greenwave has to duplicate a lot of
Bodhi's knowledge about what kinds of decisions to ask for. This
being in Greenwave is really against the intended design of the
system, and means whenever anything changes about what kinds of
decision Bodhi might request, we need to change both Bodhi and
the Greenwave "decision change" publishing code.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
AdamWill added a commit to AdamWill/bodhi that referenced this issue Jun 12, 2021
…a#4224)

This replaces the consumer that listens for Greenwave 'decision
change' messages with consumers that listen for ResultsDB and
WaiverDB "new result" / "new waiver" messages and update gating
decisions if the message may cause a change.

The goal here is to make it possible to remove the "decision
change" message code from Greenwave entirely, because in order
to publish those messages, Greenwave has to duplicate a lot of
Bodhi's knowledge about what kinds of decisions to ask for. This
being in Greenwave is really against the intended design of the
system, and means whenever anything changes about what kinds of
decision Bodhi might request, we need to change both Bodhi and
the Greenwave "decision change" publishing code.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
AdamWill added a commit to AdamWill/bodhi that referenced this issue Jun 12, 2021
…a#4224)

This replaces the consumer that listens for Greenwave 'decision
change' messages with consumers that listen for ResultsDB and
WaiverDB "new result" / "new waiver" messages and update gating
decisions if the message may cause a change.

The goal here is to make it possible to remove the "decision
change" message code from Greenwave entirely, because in order
to publish those messages, Greenwave has to duplicate a lot of
Bodhi's knowledge about what kinds of decisions to ask for. This
being in Greenwave is really against the intended design of the
system, and means whenever anything changes about what kinds of
decision Bodhi might request, we need to change both Bodhi and
the Greenwave "decision change" publishing code.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
AdamSaleh pushed a commit to AdamSaleh/bodhi that referenced this issue Sep 10, 2021
…a#4224)

This replaces the consumer that listens for Greenwave 'decision
change' messages with consumers that listen for ResultsDB and
WaiverDB "new result" / "new waiver" messages and update gating
decisions if the message may cause a change.

The goal here is to make it possible to remove the "decision
change" message code from Greenwave entirely, because in order
to publish those messages, Greenwave has to duplicate a lot of
Bodhi's knowledge about what kinds of decisions to ask for. This
being in Greenwave is really against the intended design of the
system, and means whenever anything changes about what kinds of
decision Bodhi might request, we need to change both Bodhi and
the Greenwave "decision change" publishing code.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
mattiaverga pushed a commit to AdamWill/bodhi that referenced this issue Sep 12, 2021
…a#4224)

This replaces the consumer that listens for Greenwave 'decision
change' messages with consumers that listen for ResultsDB and
WaiverDB "new result" / "new waiver" messages and update gating
decisions if the message may cause a change.

The goal here is to make it possible to remove the "decision
change" message code from Greenwave entirely, because in order
to publish those messages, Greenwave has to duplicate a lot of
Bodhi's knowledge about what kinds of decisions to ask for. This
being in Greenwave is really against the intended design of the
system, and means whenever anything changes about what kinds of
decision Bodhi might request, we need to change both Bodhi and
the Greenwave "decision change" publishing code.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
AdamWill added a commit to AdamWill/bodhi that referenced this issue Sep 15, 2021
…a#4224)

This replaces the consumer that listens for Greenwave 'decision
change' messages with consumers that listen for ResultsDB and
WaiverDB "new result" / "new waiver" messages and update gating
decisions if the message may cause a change.

The goal here is to make it possible to remove the "decision
change" message code from Greenwave entirely, because in order
to publish those messages, Greenwave has to duplicate a lot of
Bodhi's knowledge about what kinds of decisions to ask for. This
being in Greenwave is really against the intended design of the
system, and means whenever anything changes about what kinds of
decision Bodhi might request, we need to change both Bodhi and
the Greenwave "decision change" publishing code.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
@mergify mergify bot closed this as completed in #4230 Sep 17, 2021
mergify bot pushed a commit that referenced this issue Sep 17, 2021
This replaces the consumer that listens for Greenwave 'decision
change' messages with consumers that listen for ResultsDB and
WaiverDB "new result" / "new waiver" messages and update gating
decisions if the message may cause a change.

The goal here is to make it possible to remove the "decision
change" message code from Greenwave entirely, because in order
to publish those messages, Greenwave has to duplicate a lot of
Bodhi's knowledge about what kinds of decisions to ask for. This
being in Greenwave is really against the intended design of the
system, and means whenever anything changes about what kinds of
decision Bodhi might request, we need to change both Bodhi and
the Greenwave "decision change" publishing code.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant