Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Publish generic signals to event bus #6

Merged
merged 19 commits into from
Jul 18, 2022

Conversation

timmc-edx
Copy link
Contributor

@timmc-edx timmc-edx commented Jul 13, 2022

Implement send_to_event_bus, accepting signal, data, topic, and key information.

Also:

Small improvements that should be upstreamed to the cookiecutter:

  • Clean up after docs build and before test-all
  • Ignore pii_report output dir

Merge checklist:

  • Documentation updated (not only docstrings)
  • Commits are squashed

Rebecca Graber and others added 7 commits July 13, 2022 14:49
`make test-all` passes now. (`requests` was missing from test.txt, but
`pytest` was passing for me because something else brings it into dev.txt.)

Improve test-all target. This should be upstreamed to the cookiecutter.

Also ignore pii_report output dir.
…aversal

...but also reduce coverage target to something reasonable.
Copy link
Contributor

@rgraber rgraber left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some docstring issues, reminder to call poll(), and request to remove extraneous package updates

edx_event_bus_kafka/publishing/event_producer.py Outdated Show resolved Hide resolved
serializer_schema: An Avro schema (nested dictionaries)
field_path: List of strings matching the 'name' of subfields

TODO: Move to openedx_events.event_bus.avro.serializer?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a blocker, but we should clarify with @robrap how we want to handle TODOs like this

edx_event_bus_kafka/publishing/event_producer.py Outdated Show resolved Hide resolved
edx_event_bus_kafka/publishing/event_producer.py Outdated Show resolved Hide resolved
edx_event_bus_kafka/publishing/event_producer.py Outdated Show resolved Hide resolved
requirements/ci.txt Show resolved Hide resolved
@rgraber
Copy link
Contributor

rgraber commented Jul 14, 2022

Forgot! We should also add settings files (devstack.py and production.py at least) so we can actually set the correct host/url. The edx-django-utils/plugins documentation has more information on how to create settings for plugins (even if this is more library than plugin)

@timmc-edx
Copy link
Contributor Author

Why do we need settings files? Won't that come from the Django settings of the IDA?

I was just thinking of hardcoding in some devstack-appropriate defaults e.g. getattr(settings, 'SCHEMA_REGISTRY_URL', 'http://edx.devstack.schema-registry:8081') -- or just defaulting them to None and failing silently if the event bus isn't configured. (Not sure what the right layer of code is for bailing out like that.)

):
ep.send_to_event_bus(signal, 'user_stuff', 'user.id', event_data)

#. Make or refresh a copy of this repo where it can be seen from inside devstack: ``rsync -sax -delete ./ ../src/event-bus-kafka/``
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[not blocking] Should we update this to point to...whatever documentation we have around how to use edx-platform plugins in general?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couldn't find anything, but I didn't look super hard. But I think it would be worth investigating instead if we can call a test cluster more directly, not requiring the test to be run inside of devstack.

Copy link
Contributor

@rgraber rgraber left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some nits, but nothing blocking. Thank you!!!

@timmc-edx timmc-edx merged commit f0f7386 into main Jul 18, 2022
@timmc-edx timmc-edx deleted the rsgraber/publish-arbitrary-events branch July 18, 2022 21:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants