Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: [FC-0074] add docs about creating and consuming events #439

Open
wants to merge 32 commits into
base: MJG/event-design-suggestions-adr
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
187f16e
docs: refactor names for consistency in how-tos
mariajgrimaldi Dec 18, 2024
29c4c3a
docs: add docs about creating and consuming events
mariajgrimaldi Dec 19, 2024
f3e0d8e
docs: add assumptions for how-tos
mariajgrimaldi Dec 20, 2024
874f4f3
docs: add more considerations to keep in mind
mariajgrimaldi Dec 20, 2024
8f3da1c
docs: add consequences for ADR
mariajgrimaldi Dec 20, 2024
5809e22
docs: add note about consumption
mariajgrimaldi Dec 20, 2024
9029d47
docs: add note about considering support over time for the service
mariajgrimaldi Dec 20, 2024
c8ec268
docs: move identify triggering logic to previous step
mariajgrimaldi Dec 20, 2024
297069f
docs: use implement instead of write
mariajgrimaldi Dec 20, 2024
28845d0
docs: add note about data stability over time
mariajgrimaldi Dec 20, 2024
f8d1497
docs: split tests needed for the new event
mariajgrimaldi Dec 20, 2024
e30b79f
docs: add note about triggering only when facts happen
mariajgrimaldi Dec 20, 2024
d63020c
docs: add independent step for consuming the event
mariajgrimaldi Dec 20, 2024
dad6fcc
docs: add note about adding tests to the triggering logic
mariajgrimaldi Dec 20, 2024
b5d1cab
docs: add an example of testing integration
mariajgrimaldi Dec 20, 2024
7e66121
docs: add more details about example code
mariajgrimaldi Dec 25, 2024
287908a
fix: add plural for contribution process
mariajgrimaldi Dec 25, 2024
8206324
docs: clarify identify triggering logic
mariajgrimaldi Dec 25, 2024
6ad6bcf
docs: clarify where to place an event
mariajgrimaldi Dec 25, 2024
ae80dce
docs: add note about inspecting event content to implement custom logic
mariajgrimaldi Dec 25, 2024
992e7a2
docs: use example from openedx-events-2-zapier plugin
mariajgrimaldi Dec 25, 2024
4f259e6
docs: match description for new use case
mariajgrimaldi Dec 25, 2024
e540e29
docs: add notes about reviewing the event content for the use case
mariajgrimaldi Dec 25, 2024
84c4639
docs: add note about django plugins
mariajgrimaldi Dec 25, 2024
13e3d5f
docs: add note about OEP-49 handler patterns
mariajgrimaldi Dec 25, 2024
d124da8
docs: add note about identify event to consume
mariajgrimaldi Dec 25, 2024
f3f8141
refactor: fix typing error
mariajgrimaldi Dec 26, 2024
caba18f
docs: add notes about implementing receivers
mariajgrimaldi Dec 26, 2024
5c22b93
docs: add suggestion about avoiding suffixes in data field names
mariajgrimaldi Dec 26, 2024
37e3ae4
refactor: address PR review
mariajgrimaldi Jan 7, 2025
7980afc
docs: drop unnecessary assumption
mariajgrimaldi Jan 8, 2025
bdccd4c
docs: add reference to openedx-events-2-zapier
mariajgrimaldi Jan 8, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/concepts/event-bus.rst
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ How is the Open edX Event Bus Used?

The event bus is used to broadcast Open edX Events to multiple services, allowing them to react to changes or actions in the system.

We encourage you to review the :doc:`../reference/real-life-use-cases` page for examples of how the event bus is used in the Open edX ecosystem by the community. Also, see the :doc:`../how-tos/using-the-event-bus` guide to get start sending events to the event bus.
We encourage you to review the :doc:`../reference/real-life-use-cases` page for examples of how the event bus is used in the Open edX ecosystem by the community. Also, see the :doc:`../how-tos/use-the-event-bus-to-broadcast-and-consume-events` guide to get start sending events to the event bus.

.. _general_signal_handler: https://github.com/openedx/openedx-events/blob/main/openedx_events/apps.py#L16-L44
.. _EventProducer: https://github.com/openedx/openedx-events/blob/main/openedx_events/event_bus/__init__.py#L71-L91
Expand Down
2 changes: 1 addition & 1 deletion docs/concepts/openedx-events.rst
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ How are Open edX Events used?

As mentioned previously, developers can listen to Open edX Events by registering signal receivers from their Open edX Django plugins that respond to the emitted events or by using the :doc:`../concepts/event-bus` to send events to external services.

For more information on using Open edX Events, refer to the :doc:`../how-tos/using-events` how-to guide. We also encourage you to explore the :doc:`../reference/real-life-use-cases` section for real-life examples of how Open edX Events are used by the community.
For more information on using Open edX Events, refer to the :doc:`../how-tos/create-a-new-event` how-to guide. We also encourage you to explore the :doc:`../reference/real-life-use-cases` section for real-life examples of how Open edX Events are used by the community.

.. _Django Signals Documentation: https://docs.djangoproject.com/en/4.2/topics/signals/
.. _triggering the COURSE_ENROLLMENT_CREATED event: https://github.com/openedx/edx-platform/blob/master/common/djangoapps/student/models/course_enrollment.py#L777-L795
Expand Down
7 changes: 6 additions & 1 deletion docs/decisions/0016-event-design-practices.rst
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Event Purpose and Content
~~~~~~~~~~~~~~~~~~~~~~~~~

- An event should describe as accurately as possible what happened (what) and why it happened (why). It must contain enough information for consumers to understand the message. For instance, if an event is about a user enrollment, it should contain the user's data, the course data, and the enrollment status and the event should be named accordingly.
- Avoid immediately contacting the source service to retrieve additional information. Instead, consider adding the necessary information to the event payload by managing the granularity of the event. If the event requires additional information, consider adding a field to the event that contains the necessary information. This will reduce the number of dependencies between services and make the event more self-contained.
- Avoid immediately contacting the source service to retrieve additional information from the consumer-side. Instead, consider adding the necessary information to the event payload by managing the granularity of the event. If the event requires additional information, consider adding a field to the event that contains the necessary information. This will reduce the number of dependencies between services and make the event more self-contained.
- Keep the event size small. Avoid adding unnecessary information to the event. If the information is not necessary for consumers to react to the event, consider removing it.
- Avoid adding flow-control information or business logic to events. Events should be solely a representation of what took place. If a field is necessary to control the behavior of the consumer, consider moving it to the consumer side. If adding additional data to the event is absolutely necessary document the reasoning behind it and carefully study the use case and implications.

Expand All @@ -53,6 +53,11 @@ Some of these practices might not be applicable to all events, but they are a go

In addition to these practices, review the Architectural Decision Records (ADRs) related to events to understand the naming, versioning, payload, and other practices that are specific to Open edX events.

Consequences
------------

Following these practices will help ensure that the events are consistent, maintainable, and reusable. It will also help consumers understand the message and react to the event accordingly. However, it might require additional effort to design the event and ensure that it contains the necessary information for consumers to react to the event, although this effort will pay off in the long run. Having these standards in place will also make the decision process easier when designing new events.

.. _Event-Driven Microservices: https://www.oreilly.com/library/view/building-event-driven-microservices/9781492057888/
.. _Event-Driven article: https://martinfowler.com/articles/201701-event-driven.html
.. _Thin Events - The lean muscle of event-driven architecture: https://www.thoughtworks.com/insights/blog/architecture/thin-events-the-lean-muscle-of-event-driven-architecture
4 changes: 2 additions & 2 deletions docs/how-tos/add-new-event-bus-concrete-implementation.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
How to add a new concrete implementation of the event bus
=========================================================
Add a New Concrete Implementation of the Event Bus
==================================================

Context
-------
Expand Down
39 changes: 0 additions & 39 deletions docs/how-tos/adding-events-to-a-service.rst

This file was deleted.

132 changes: 132 additions & 0 deletions docs/how-tos/consume-an-event.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
Consume an Open edX Event
=========================

You have two ways of consuming an Open edX event, within the same service or in a different service. In this guide, we will show you how to consume an event within the same service. For consuming events across services, see :doc:`../how-tos/use-the-event-bus-to-broadcast-and-consume-events`.

.. note:: We encourage you to also consider the practices outlined in the :doc:`../decisions/0016-event-design-practices` ADR for event consumption.

Throughout this guide, we will implement the use case to send the enrollment data to a webhook when a user enrolls in a course to better illustrate the steps involved in creating a consumer for an event.

Assumptions
-----------

- You have a development environment set up using `Tutor`_.
- You have a basic understanding of Python and Django.
- You have a basic understanding of Django signals. If not, you can review the `Django Signals Documentation`_.
- You are familiar with the terminology used in the project, such as the terms :term:`Event Type` or :term:`Event Receiver`. If not, you can review the :doc:`../reference/glossary` documentation.

Steps
-----

To consume an event within the same service, follow these steps:

Step 1: Understand your Use Case and Identify the Event to Consume
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Before you start consuming an event, you should understand the use case and the event you want to consume, for this review the `list of events`_ available in the Open edX platform. In this case, we want to send the enrollment data to a webhook when a user enrolls in a course. You should review the event definition and payload to understand the data that is being passed to the event receiver and how you can use it to implement the custom logic.

In our example, we want to send the enrollment data to a webhook when a user enrolls in a course. We will consume the ``COURSE_ENROLLMENT_CREATED`` event, which is triggered every time a user enrolls in a course. You can review the event definition and payload to understand the data that is being passed to the event receiver and how you can use it to implement the request to the webhook.

Step 2: Install Open edX Events
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

First, add the ``openedx-events`` plugin into your dependencies so the library's environment recognizes the event you want to consume. You can install ``openedx-events`` by running:

.. code-block:: bash

pip install openedx-events

This will mainly make the events available for your CI/CD pipeline and local development environment. If you are using the Open edX platform, the library should be already be installed in the environment so no need to install it.

Step 3: Create a Event Receiver and Connect it to the Event
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

An :term:`Event Receiver` is simply a function that listens for a specific event and executes custom logic in response to the event being triggered. You can create an event receiver by using the Django signal receivers decorator. Here's an example of an event receiver that listens for the ``COURSE_ENROLLMENT_CREATED`` event and creates a notification preference for the user:

.. code-block:: python

from openedx_events import COURSE_ENROLLMENT_CREATED
from django.dispatch import receiver

@receiver(COURSE_ENROLLMENT_CREATED)
def send_enrollment_data_to_webhook(signal, sender, enrollment, metadata, **kwargs):
# Custom logic to send enrollment data to a webhook
pass

- The Django dispatcher will call the ``send_enrollment_data_to_webhook`` function when the ``COURSE_ENROLLMENT_CREATED`` event is triggered by using the ``receiver`` decorator. In this case, that would be every time a user enrolls in a course.
- Consider using asynchronous tasks to handle the event processing to avoid blocking the main thread and improve performance. Also, make sure to handle exceptions and errors gracefully to avoid silent failures and improve debugging. You should also consider not creating a tight coupling between receivers and other services, if doing so is necessary consider using the event bus to broadcast the event.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Consider using asynchronous tasks to handle the event processing to avoid blocking the main thread and improve performance. Also, make sure to handle exceptions and errors gracefully to avoid silent failures and improve debugging. You should also consider not creating a tight coupling between receivers and other services, if doing so is necessary consider using the event bus to broadcast the event.
- Consider using asynchronous tasks to handle the event processing to avoid blocking the main thread and improve performance. Also, make sure to handle exceptions and errors gracefully to avoid silent failures and improve debugging. It is recommended to not create a tight coupling between receivers and other services. If doing so is necessary consider using the event bus to broadcast the event.

Small language tweak for easier reading

- When implementing the receiver, inspect the event payload to understand the data that is being passed to the event receiver by reviewing the ``data.py`` file of the event you are consuming. For example, the ``COURSE_ENROLLMENT_CREATED`` event has the following payload:

.. code-block:: python

# Location openedx_events/learning/data.py
COURSE_ENROLLMENT_CREATED = OpenEdxPublicSignal(
event_type="org.openedx.learning.course.enrollment.created.v1",
data={
"enrollment": CourseEnrollmentData,
}
)

- This event has a single field called ``enrollment`` which is an instance of the ``CourseEnrollmentData`` class. You can review the ``CourseEnrollmentData`` class to understand the data that is available to you and how you can use it to implement the custom logic.
- The ``metadata`` parameter contains the Open edX-specific metadata for the event, such as the event version and timestamp when the event was sent. You can use this metadata to understand more about the event and its context.

These event receivers are usually implemented independently of the service in an `Open edX Django plugins`_ and are registered in the ``handlers.py`` (according to `OEP-49`_) file of the plugin. You can review the ``handlers.py`` file of the `openedx-events-2-zapier`_ plugin to understand how the event receivers are implemented and connected to the events.

.. TODO: change receivers.py in openedx-events-2-zapier to handlers.py
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be done before the PR merges?


Consider the following when implementing the event receiver:

- Limit each receiver to a single responsibility to make the code easier to maintain and test.
- Keep the receiver logic simple and focused on the specific task it needs to perform.
- Consider the performance implications of the receiver and avoid adding unnecessary complexity or overhead, considering that receivers will be executed each time the event is triggered. Consider using asynchronous tasks to handle the event processing to avoid blocking the main thread and improve performance.
- Implement error handling and logging in the pipeline step to handle exceptions and provide useful information for debugging, considering both development and production environments.

Step 4: Test the Event Receiver
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Given the design of Open edX Events, you can include the events definitions in your test suite to ensure that the event receiver is working as expected. You can use the ``send_event`` method to trigger the event and test the event receiver. Here's an example of how you can test the event receiver:

.. code-block:: python

from openedx_events import send_event, COURSE_ENROLLMENT_CREATED

def test_send_enrollment_data_to_webhook(self):
# Trigger the event
enrollment_data = CourseEnrollmentData(
user=UserData(
pii=UserPersonalData(
username=self.user.username,
email=self.user.email,
name=self.user.profile.name,
),
id=self.user.id,
is_active=self.user.is_active,
),
course=CourseData(
course_key=self.course.id,
display_name=self.course.display_name,
),
mode=self.course_enrollment.mode,
is_active=self.course_enrollment.is_active,
creation_date=self.course_enrollment.created,
)

COURSE_ENROLLMENT_CREATED.send_event(
enrollment=enrollment_data
)

# Assert that the request was sent to the webhook with the correct data
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you missing an assert statement here?


- In the test suite, you can use the ``send_event`` method to trigger the event and pass the necessary data to the event receiver. In this case, we are passing the user, course and enrollment data to the event receiver as the triggering logic would do.
- After triggering the event, you can assert that the event receiver executed the custom logic as expected. In this case, we are checking that the request was sent to the webhook with the correct data.

You can review this example to understand how you can test the event receiver and ensure that the custom logic is executed when the event is triggered in the `openedx-events-2-zapier`_ plugin.

This way you can ensure that the event receiver is working as expected and that the custom logic is executed when the event is triggered. If the event definition or payload changes in any way, you can catch the error in the test suite instead of in production.

.. _Tutor: https://docs.tutor.edly.io/
.. _Django Signals Documentation: https://docs.djangoproject.com/en/4.2/topics/signals/
.. _openedx-events-2-zapier: https://github.com/eduNEXT/openedx-events-2-zapier
.. _Open edX Django plugins: https://docs.openedx.org/en/latest/developers/concepts/platform_overview.html#new-plugin
.. _OEP-49: https://docs.openedx.org/projects/openedx-proposals/en/latest/best-practices/oep-0049-django-app-patterns.html#signals
.. _list of events: https://docs.openedx.org/projects/openedx-events/en/latest/reference/events.html
Loading
Loading