Update pants-plugins/uses_services to support checking for rabbitmq

[cognifloyd]

Background

This is another part of introducing pants, as discussed in various TSC meetings.

Related PRs can be found in:

• pantsbuild milestone (which includes PRs since Initialize pantsbuild #5713)
• pantsbuild label (which also includes preparatory PRs that came before adding pantsbuild)

Overview of this PR

This PR builds on #5864 to further improve the DX (developer experience) by failing as early as possible if the development (or CI) environment does not have the required services.

This PR adds checks for rabbitmq, like #5864 added mongo checks. I also have draft branch to add redis checks.

Please see #5864 for a more detailed description of pants-plugins/uses_services.

is_rabbitmq_running.py script and the rule that runs it

is_rabbitmq_running.py is the part that checks to see if rabbitmq is running.

The pants plugin cannot import anything from our other st2* code, so is_rabbitmq_running.py is a minimal self-contained script that mirrors how st2 connects to rabbitmq. It should be as minimal as possible so that keeping it up-to-date with the core st2 code is not onerous.

The is_rabbitmq_running.py rule gets opened and run in a pex with the same rule logic as is_mongo_running.py and inspect_platform.py (see #5864).

Rabbitmq embeds all auth stuff in the connection url. So, there are not as many settings to worry about as there were with mongo. For now, this only supports the url hard-coded in the tests. Here is where the rule defines the default url:

st2/pants-plugins/uses_services/rabbitmq_rules.py

Lines 47 to 65 in adb9c4b

	@dataclass(frozen=True)
	class UsesRabbitMQRequest:
	"""One or more targets need a running rabbitmq service using these settings.
	The mq_* attributes represent the messaging settings from st2.conf.
	In st2 code, they come from:
	oslo_config.cfg.CONF.messaging.{url,cluster_urls}
	"""
	# These config opts for integration tests are in:
	# conf/st2.tests*.conf st2tests/st2tests/fixtures/conf/st2.tests*.conf
	# (changed by setting ST2_CONFIG_PATH env var inside the tests)
	# TODO: for unit tests: modify code to pull mq connect settings from env vars
	# TODO: for int tests: modify st2.tests*.conf on the fly to set the per-pantsd-slot vhost
	# and either add env vars for mq connect settings or modify conf files as well
	# with our version of oslo.config (newer are slower) we can't directly override opts w/ environment variables.
	mq_urls: list[str] = ["amqp://guest:guest@127.0.0.1:5672//"]

Here is the definition of the rule that runs is_rabbitmq_running.py:

st2/pants-plugins/uses_services/rabbitmq_rules.py

Lines 99 to 101 in adb9c4b

	async def rabbitmq_is_running(
	request: UsesRabbitMQRequest, platform: Platform
	) -> RabbitMQIsRunning:

So, when another rule Gets RabbitMQIsRunning with a UsesRabbitMQRequest, pants will also run the rule that generates Platform (described in #5864), and then it will run this is_rabbitmq_running rule.

The is_rabbbitmq_running rule either returns RabbitMQIsRunning() if it is running, or raises ServiceMissingError if it is not. By raising an error, this actually breaks a convention for pants rules. Exceptions stop everything and get shown to the user right away, and for most goals, pants wants to see some dataclass returned that has something like a succeeded boolean instead. But, we want to error as early as possible, so this breaks that convention on purpose.

wiring up the test goal so it runs is_rabbitmq_running when pytest runs on a target with the uses field.

The last piece that ties this all together is a rule that makes sure the is_rabbitmq_running rule runs before pytest runs (if pants is running it on a target with the uses field). Here is the definition of the rabbitmq_is_running_for_pytest rule:

st2/pants-plugins/uses_services/rabbitmq_rules.py

Lines 86 to 88 in adb9c4b

	async def rabbitmq_is_running_for_pytest(
	request: PytestUsesRabbitMQRequest,
	) -> PytestPluginSetup:

This rule is very simple. It just triggers running the is_rabbitmq_running rule:

st2/pants-plugins/uses_services/rabbitmq_rules.py

Lines 89 to 92 in adb9c4b

	# this will raise an error if rabbitmq is not running
	_ = await Get(RabbitMQIsRunning, UsesRabbitMQRequest())
	return PytestPluginSetup()

This rule needs the PytestUsesRabbitMQRequest which selects targets that have the uses field.

st2/pants-plugins/uses_services/rabbitmq_rules.py

Lines 73 to 79 in adb9c4b

	class PytestUsesRabbitMQRequest(PytestPluginSetupRequest):
	@classmethod
	def is_applicable(cls, target: Target) -> bool:
	if not target.has_field(UsesServicesField):
	return False
	uses = target.get(UsesServicesField).value
	return uses is not None and "rabbitmq" in uses

This request will be made by the pants-internal pytest rules thanks to this UnionRule, which is a way to declare that our class "implements" the PytestPluginSetupRequest:

st2/pants-plugins/uses_services/rabbitmq_rules.py

Line 176 in adb9c4b

UnionRule(PytestPluginSetupRequest, PytestUsesRabbitMQRequest),

If we need to add uses support to other targets, we will need to find similar UnionRules where we can inject our rule logic. For now, I've only wired this up so our uses rules will run for pytest.

[rush-skills]

LGTM.