Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/koning/vhost no leading slash #217

Merged
merged 22 commits into from
May 18, 2020
Merged
Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
a120c3a
Merge pull request #1 from LLNL/develop
koning Dec 4, 2019
d76d67f
Merge remote-tracking branch 'upstream/develop' into develop
koning Mar 6, 2020
c18866b
Merge branch 'develop' of github.com:llnl/merlin into develop
koning Mar 9, 2020
3a7d035
Merge branch 'develop' of github.com:llnl/merlin into develop
koning Mar 9, 2020
43c8e49
Merge branch 'develop' of github.com:llnl/merlin into develop
koning Mar 11, 2020
3bb9184
Merge branch 'develop' of github.com:llnl/merlin into develop
koning Mar 12, 2020
9776a3e
Merge branch 'develop' of github.com:llnl/merlin into develop
koning Mar 13, 2020
461aad3
Merge branch 'develop' of github.com:llnl/merlin into develop
koning Mar 18, 2020
0bad54c
Merge branch 'develop' of github.com:llnl/merlin into develop
koning Mar 23, 2020
3b8b4a4
Merge branch 'develop' of github.com:llnl/merlin into develop
koning Mar 25, 2020
7ee915b
Merge branch 'develop' of github.com:llnl/merlin into develop
koning Mar 25, 2020
3de4524
Merge branch 'develop' of github.com:llnl/merlin into develop
koning Mar 30, 2020
b3b3d81
Merge branch 'develop' of github.com:llnl/merlin into develop
koning Mar 31, 2020
58bedd7
Merge branch 'develop' of github.com:llnl/merlin into develop
koning Apr 2, 2020
40865c1
Merge branch 'develop' of github.com:koning/merlin into develop
koning Apr 27, 2020
273b02d
Merge branch 'develop' of github.com:llnl/merlin into develop
koning Apr 27, 2020
a3e21ca
Merge branch 'develop' of github.com:llnl/merlin into develop
koning May 11, 2020
a28a8a2
Merge remote-tracking branch 'upstream/develop' into develop
koning May 18, 2020
ddfcd92
The broker name can now be amqps (with ssl) or amqp (without ssl).
koning May 18, 2020
554e6e6
Undo fix typos.
koning May 18, 2020
a02b13c
Fix spacing.
koning May 18, 2020
49caab8
Add amqps conn for self doc.
koning May 18, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,14 @@ All notable changes to Merlin will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

### Added
- The broker name can now be amqps (with ssl) or amqp (without ssl).

### Changed
- The default rabbitmq vhost is now <user> instead of /<user>.

## [1.5.3]

### Fixed
Expand Down
12 changes: 8 additions & 4 deletions docs/source/merlin_config.rst
Original file line number Diff line number Diff line change
Expand Up @@ -30,16 +30,20 @@ app.yaml file, then the user's ``~/.merlin`` directory is checked.
using a socket.


Broker: ``rabbitmq``
--------------------
Broker: ``rabbitmq``, ``amqps``, ``amqp``
-----------------------------------------
Merlin constructs the following connection string from the relevant options in the
``broker`` section of the app.yaml file. If the ``port`` argument is not defined,
the default rabbitmq TLS port, 5671, will be used. See the :ref:`broker_rabbitmq_ssl`
section for more info about security with this broker.
section for more info about security with this broker. When the ``broker``
is ``amqp``, the default port will be 5672.


| The prototype url for this configuration is:
| ``amqps://{username}:{password}@{server}:{port}//{vhost}``
| ``{conn}://{username}:{password}@{server}:{port}/{vhost}``

Here ``conn`` is ``amqps`` (with ssl) when ``name`` is ``rabbitmq`` or ``amqps`` and
``amqp`` (without ssl) when name is ``amqp``.

::

Expand Down
27 changes: 18 additions & 9 deletions merlin/config/broker.py
Original file line number Diff line number Diff line change
Expand Up @@ -50,9 +50,9 @@

LOG = logging.getLogger(__name__)

BROKERS = ["rabbitmq", "redis", "rediss", "redis+socket"]
BROKERS = ["rabbitmq", "redis", "rediss", "redis+socket", "amqps", "amqp"]

RABBITMQ_CONNECTION = "amqps://{username}:{password}@{server}:{port}//{vhost}"
RABBITMQ_CONNECTION = "{conn}://{username}:{password}@{server}:{port}/{vhost}"
REDISSOCK_CONNECTION = "redis+socket://{path}?virtual_host={db_num}"
USER = getpass.getuser()

Expand All @@ -64,14 +64,16 @@ def read_file(filepath):
return quote(line, safe="")


def get_rabbit_connection(config_path, include_password):
def get_rabbit_connection(config_path, include_password, conn="amqps"):
"""
Given the path to the directory where the broker configurations are stored
setup and return the RabbitMQ connection string.

:param config_path : The path for ssl certificates and passwords
:param include_password : Format the connection for ouput by setting this True
"""
LOG.debug(f"Broker: connection = {conn}")

vhost = CONFIG.broker.vhost
LOG.debug(f"Broker: vhost = {vhost}")

Expand Down Expand Up @@ -99,11 +101,15 @@ def get_rabbit_connection(config_path, include_password):
port = CONFIG.broker.port
LOG.debug(f"Broker: RabbitMQ port = {port}")
except (AttributeError, KeyError):
port = 5671
if conn == "amqp":
port = 5672
else:
port = 5671
LOG.debug(f"Broker: RabbitMQ using default port = {port}")

# Test configurations.
rabbitmq_config = {
"conn": conn,
"vhost": vhost,
"username": username,
"password": "******",
Expand Down Expand Up @@ -213,16 +219,19 @@ def get_connection_string(include_password=True):
if broker not in BROKERS:
raise ValueError(f"Error: {broker} is not a supported broker.")

if broker == "rabbitmq":
if broker == "rabbitmq" or broker == "amqps":
return get_rabbit_connection(config_path, include_password)
koning marked this conversation as resolved.
Show resolved Hide resolved

if broker == "redis+socket":
elif broker == "amqp":
return get_rabbit_connection(config_path, include_password, conn="amqp")

elif broker == "redis+socket":
return get_redissock_connection(config_path, include_password)

if broker == "redis":
elif broker == "redis":
return get_redis_connection(config_path, include_password)

if broker == "rediss":
elif broker == "rediss":
return get_redis_connection(config_path, include_password, ssl=True)

return None
Expand Down Expand Up @@ -257,7 +266,7 @@ def get_ssl_config():
if not broker_ssl:
broker_ssl = True

if broker == "rabbitmq" or broker == "rediss":
if broker == "rabbitmq" or broker == "rediss" or broker == "amqps":
return broker_ssl

return False