Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adopt dedicated secrets management library #2728 #2756

Closed
wants to merge 3 commits into from
Closed

Adopt dedicated secrets management library #2728 #2756

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
abc6c31
Adopt dedicated secrets management library #2728
phillxnet Nov 30, 2023
7ae63b9
Adopt dedicated secrets management library - continued #2728
phillxnet Dec 1, 2023
d85b8be
Adopt dedicated secrets management library - build.sh mod #2728
phillxnet Dec 2, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 11 additions & 2 deletions build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,13 @@ if [ ! -d "jslibs" ]; then
echo
fi

# Ensure GNUPG is setup for 'pass' (Idempotent)
/usr/bin/gpg --quick-generate-key --batch --passphrase '' rockstor@localhost || true
# Init 'pass' in ~ using above GPG key, and generate Django SECRET_KEY
export Environment="PASSWORD_STORE_DIR=/root/.password-store"
/usr/bin/pass init rockstor@localhost
/usr/bin/pass generate --no-symbols --force python-keyring/rockstor/SECRET_KEY 100

# Collect all static files in the STATIC_ROOT subdirectory. See settings.py.
# /opt/rockstor/static
# Additional collectstatic options --clear --dry-run
Expand All @@ -67,8 +74,10 @@ echo
echo "ROCKSTOR BUILD SCRIPT COMPLETED"
echo
echo "If installing from source, from scratch, for development; i.e. NOT via RPM:"
echo "Note GnuPG & password-store ExecStartPre steps in /opt/rockstor/conf/rockstor-pre.service"
echo "1. Run 'cd /opt/rockstor'."
echo "2. Run 'systemctl start postgresql'."
echo "3. Run 'export DJANGO_SETTINGS_MODULE=settings'."
echo "4. Run 'poetry run initrock' as root (equivalent to rockstor-pre.service)."
echo "5. Run 'systemctl enable --now rockstor-bootstrap'."
echo "4. Run 'export PASSWORD_STORE_DIR=/root/.password-store'."
echo "5. Run 'poetry run initrock' as root (equivalent to rockstor-pre.service ExecStart)."
echo "6. Run 'systemctl enable --now rockstor-bootstrap'."
1 change: 1 addition & 0 deletions conf/rockstor-bootstrap.service
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ Requires=rockstor.service

[Service]
Environment="DJANGO_SETTINGS_MODULE=settings"
Environment="PASSWORD_STORE_DIR=/root/.password-store"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does it have to be in the root directory? Or would it be better to also have it somewhere below the /opt/rockstor tree to be part of the Rockstor package "boundaries"?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Hooverdan96 Thanks for the interest here, I'll answer piece-meal:

does it have to be in the root directory?

No.

Or would it be better to also have it somewhere below the /opt/rockstor tree to be part of the Rockstor package "boundaries"?

No, and my reasoning here is that the most secure location on the "/" system is /root at least with the default fs arrangement, we also use this (our) users gpg (GnuPG) key (generated via the new rockstor-pre.service additions) to encrypt these passwords (or 'pass' does it for us anyway). Plus we currently run under this user. So just a security thing really. If in /opt/rockstor or any other user home (with default group setting for users) other users can at least get to the parent directly. We will likely revisit these locations and concerns when we approach: #2700
as at that time we ourselves will not be able to access /root. But there is likely always going to be a need for us to assume root at some point (but likely through specific capabilities) but again, this can be approached as we consider the dedicated user issue. But this is not on the Milestone for our coming stable release.

For now I'm trying to use defaults of the user we run under and for pass this is it for the 'root' user, which we are currently. But as we are in the constrained env of systemd it needed to be re-established.

Copy link
Member Author

@phillxnet phillxnet Dec 1, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another element here, given it's all default for the 'root' user is that one can interact easily via the pass command when simply logged in as root. I.e.:

lbuildvm:~ # pass
Password Store
└── python-keyring
    └── rockstor
        ├── CLIENT_SECRET
        └── SECRET_KEY

Is what the code here creates on first services start / first boot/install.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And on subsequent (second in this case) reboots we get our current stable/full set of:

lbuildvm:~ # pass
Password Store
└── python-keyring
    └── rockstor
        ├── CLIENT_SECRET
        ├── SECRET_KEY_FALLBACK
        └── SECRET_KEY

Making use of Django's 4.1 onwards fallback mechanism to smooth over CLIENT_SECRET signing key rotation. Which we now do in this branch on every 'cycle' of rockstor-pre.service by renaming SECRET_KEY to SECRET_KEY_FALLBACK each service start and creating a new SECRET_KEY: thus honouring the current and last in-play SECRET_KEY. Hopefully allowing for a smoother transition and less failed sessions. But also maintaining a robust key rotation.

WorkingDirectory=/opt/rockstor
ExecStart=/opt/rockstor/.venv/bin/bootstrap
Type=oneshot
Expand Down
10 changes: 10 additions & 0 deletions conf/rockstor-pre.service
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,17 @@ Requires=postgresql.service

[Service]
Environment="DJANGO_SETTINGS_MODULE=settings"
Environment="PASSWORD_STORE_DIR=/root/.password-store"
WorkingDirectory=/opt/rockstor
# Avoid `pass` stdout leaking generated passwords (N.B. 2>&1 >/dev/null failed).
StandardOutput=null
# Idempotent: failure tolerated for pgp as key likely already exists (rc 2).
ExecStartPre=-/usr/bin/gpg --quick-generate-key --batch --passphrase '' rockstor@localhost
# Idempotent.
ExecStartPre=/usr/bin/pass init rockstor@localhost
# Rotate Django SECRET_KEY: failure tolerated in rename in case of no prior SECRET_KEY.
ExecStartPre=-/usr/bin/pass rename --force python-keyring/rockstor/SECRET_KEY python-keyring/rockstor/SECRET_KEY_FALLBACK
ExecStartPre=/usr/bin/pass generate --no-symbols --force python-keyring/rockstor/SECRET_KEY 100
ExecStart=/usr/local/bin/poetry run initrock
Type=oneshot
RemainAfterExit=yes
Expand Down
1 change: 1 addition & 0 deletions conf/rockstor.service
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ Requires=rockstor-pre.service

[Service]
Environment="DJANGO_SETTINGS_MODULE=settings"
Environment="PASSWORD_STORE_DIR=/root/.password-store"
WorkingDirectory=/opt/rockstor
ExecStart=/usr/local/bin/poetry run supervisord -c /opt/rockstor/etc/supervisord.conf
ExecStop=/usr/local/bin/poetry run supervisorctl shutdown
Expand Down
300 changes: 272 additions & 28 deletions poetry.lock
View file
Open in desktop

Large diffs are not rendered by default.

8 changes: 7 additions & 1 deletion pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,10 +81,16 @@ psutil = "==5.9.4"
pyzmq = "*"
distro = "*"
URLObject = "==2.1.1"
keyring-pass = "*"
# https://pypi.org/project/supervisor/ 4.1.0 onwards embeds unmaintained meld3
supervisor = "==4.2.4"

[tool.poetry.dev-dependencies]
# `poetry install --with dev`
[tool.poetry.group.dev]
optional = true

[tool.poetry.group.dev.dependencies]
black = "*"

[tool.poetry.scripts]
# https://python-poetry.org/docs/pyproject#scripts
Expand Down
90 changes: 57 additions & 33 deletions src/rockstor/scripts/initrock.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,9 @@
import stat
import sys
from tempfile import mkstemp

import secrets
import keyring
from keyring.errors import KeyringError
from django.conf import settings

from system import services
Expand All @@ -41,7 +43,10 @@
CONF_DIR = f"{BASE_DIR}conf"
DJANGO = f"{BASE_BIN}/django-admin"
DJANGO_MIGRATE_CMD = [DJANGO, "migrate", "--noinput"]
DJANGO_MIGRATE_SMART_MANAGER_CMD = DJANGO_MIGRATE_CMD + ["--database=smart_manager", "smart_manager"]
DJANGO_MIGRATE_SMART_MANAGER_CMD = DJANGO_MIGRATE_CMD + [
"--database=smart_manager",
"smart_manager",
]
STAMP = f"{BASE_DIR}/.initrock"
FLASH_OPTIMIZE = f"{BASE_BIN}/flash-optimize"
DJANGO_PREP_DB = f"{BASE_BIN}/prep_db"
Expand Down Expand Up @@ -210,10 +215,10 @@ def update_nginx(log):
log.exception("Exception while updating nginx: {e}".format(e=e))


def update_tz(log):
def update_tz():
# update timezone variable in settings.py
zonestr = os.path.realpath("/etc/localtime").split("zoneinfo/")[1]
log.info("system timezone = {}".format(zonestr))
logger.info("system timezone = {}".format(zonestr))
sfile = "{}/src/rockstor/settings.py".format(BASE_DIR)
fo, npath = mkstemp()
updated = False
Expand All @@ -226,7 +231,7 @@ def update_tz(log):
else:
tfo.write("TIME_ZONE = '{}'\n".format(zonestr))
updated = True
log.info("Changed timezone from {} to {}".format(curzone, zonestr))
logger.info("Changed timezone from {} to {}".format(curzone, zonestr))
else:
tfo.write(line)
if updated:
Expand Down Expand Up @@ -457,6 +462,19 @@ def establish_poetry_paths():
logger.info("### DONE establishing poetry path to binaries in local files.")


def set_api_client_secret():
"""
Set/reset the API client secret which is used internally by OAUTH_INTERNAL_APP = "cliapp",
and the Replication service. Ultimately retrieved in setting.py and intended to be installed
instance stable. Resources OS package pass as python-keyring backend via interface project keyring-pass.
"""
try:
keyring.set_password("rockstor", "CLIENT_SECRET", secrets.token_urlsafe(100))
logger.info("API CLIENT_SECRET set/reset successfully.")
except keyring.errors.PasswordSetError:
raise keyring.errors.PasswordSetError("Failed to set/reset API CLIENT_SECRET.")


def main():
loglevel = logging.INFO
if len(sys.argv) > 1 and sys.argv[1] == "-x":
Expand All @@ -476,7 +494,7 @@ def main():
"/C=US/ST=Rockstor user's state/L=Rockstor user's "
"city/O=Rockstor user/OU=Rockstor dept/CN=rockstor.user"
)
logging.info("Creating openssl cert...")
logger.info("Creating openssl cert...")
run_command(
[
OPENSSL,
Expand All @@ -492,8 +510,8 @@ def main():
dn,
]
)
logging.debug("openssl cert created")
logging.info("Creating rockstor key...")
logger.debug("openssl cert created")
logger.info("Creating rockstor key...")
run_command(
[
OPENSSL,
Expand All @@ -504,8 +522,8 @@ def main():
"{}/rockstor.key".format(cert_loc),
]
)
logging.debug("rockstor key created")
logging.info("Singing cert with rockstor key...")
logger.debug("rockstor key created")
logger.info("Singing cert with rockstor key...")
run_command(
[
OPENSSL,
Expand All @@ -521,44 +539,46 @@ def main():
"3650",
]
)
logging.debug("cert signed.")
logging.info("restarting nginx...")
logger.debug("cert signed.")
logger.info("restarting nginx...")
run_command([SYSTEMCTL, "restart", "nginx"])

logging.info("Checking for flash and Running flash optimizations if appropriate.")
logger.info("Checking for flash and Running flash optimizations if appropriate.")
run_command([FLASH_OPTIMIZE, "-x"], throw=False)
try:
logging.info("Updating the timezone from the system")
update_tz(logging)
logger.info("Updating the timezone from the system")
update_tz()
except Exception as e:
logging.error("Exception while updating timezone: {}".format(e.__str__()))
logging.exception(e)
logger.error("Exception while updating timezone: {}".format(e.__str__()))
logger.exception(e)

try:
logging.info("Initialising SSHD config")
logger.info("Initialising SSHD config")
bootstrap_sshd_config(logging)
except Exception as e:
logging.error("Exception while updating sshd config: {}".format(e.__str__()))
logger.error("Exception while updating sshd config: {}".format(e.__str__()))

db_already_setup = os.path.isfile(STAMP)
if not db_already_setup or keyring.get_password("rockstor", "CLIENT_SECRET") is None:
set_api_client_secret()
for db_stage_name, db_stage_items in zip(
["Tune Postgres", "Setup Databases"], [DB_SYS_TUNE, DB_SETUP]
):
if db_stage_name == "Setup Databases" and db_already_setup:
continue
logging.info(f"--DB-- {db_stage_name} --DB--")
logger.info(f"--DB-- {db_stage_name} --DB--")
for action, command in db_stage_items.items():
logging.info(f"--DB-- Running - {action}")
logger.info(f"--DB-- Running - {action}")
if action.startswith("migrate"):
run_command(command)
else:
run_command(["su", "-", "postgres", "-c", command])
logging.info(f"--DB-- Done with {action}.")
logging.info(f"--DB-- {db_stage_name} Done --DB--.")
logger.info(f"--DB-- Done with {action}.")
logger.info(f"--DB-- {db_stage_name} Done --DB--.")
if db_stage_name == "Setup Databases":
run_command(["touch", STAMP]) # file flag indicating db setup

logging.info("Running app database migrations...")
logger.info("Running app database migrations...")
fake_migration_cmd = DJANGO_MIGRATE_CMD + ["--fake"]
fake_initial_migration_cmd = DJANGO_MIGRATE_CMD + ["--fake-initial"]

Expand Down Expand Up @@ -591,30 +611,34 @@ def main():
run_command(DJANGO_MIGRATE_CMD + ["storageadmin"], log=True)
run_command(DJANGO_MIGRATE_SMART_MANAGER_CMD, log=True)

o, e, rc = run_command([DJANGO, "showmigrations", "--list", "oauth2_provider"], log=True)
o, e, rc = run_command(
[DJANGO, "showmigrations", "--list", "oauth2_provider"], log=True
)
logger.info(f"Prior migrations for oauth2_provider are: {o}")

# Run all migrations for oauth2_provider
run_command(DJANGO_MIGRATE_CMD + ["oauth2_provider"], log=True)

o, e, rc = run_command([DJANGO, "showmigrations", "--list", "oauth2_provider"], log=True)
o, e, rc = run_command(
[DJANGO, "showmigrations", "--list", "oauth2_provider"], log=True
)
logger.info(f"Post migrations for oauth2_provider are: {o}")

logging.info("DB Migrations Done")
logger.info("DB Migrations Done")

logging.info("Running Django prep_db.")
logger.info("Running Django prep_db.")
run_command([DJANGO_PREP_DB])
logging.info("Done")
logger.info("Done")

logging.info("Stopping firewalld...")
logger.info("Stopping firewalld...")
run_command([SYSTEMCTL, "stop", "firewalld"])
run_command([SYSTEMCTL, "disable", "firewalld"])
logging.info("Firewalld stopped and disabled")
logger.info("Firewalld stopped and disabled")

logging.info("Enabling and Starting atd...")
logger.info("Enabling and Starting atd...")
run_command([SYSTEMCTL, "enable", "atd"])
run_command([SYSTEMCTL, "start", "atd"])
logging.info("Atd enabled and started")
logger.info("Atd enabled and started")

update_nginx(logging)

Expand Down
23 changes: 19 additions & 4 deletions src/rockstor/settings.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,9 @@
# Django settings for Rockstor project.
import os
import distro
import secrets
import keyring
from huey import SqliteHuey
from keyring.errors import KeyringError

# By default, DEBUG = False, honour this by True only if env var == "True"
DEBUG = os.environ.get("DJANGO_DEBUG", "") == "True"
Expand Down Expand Up @@ -112,11 +113,25 @@
"pipeline.finders.PipelineFinder",
)

# Make this unique, and don't share it with anybody.
SECRET_KEY = "odk7(t)1y$ls)euj3$2xs7e^i=a9b&xtf&z=-2bz$687&^q0+3"
# Resource keyring for Django's cryptographic signing key.
# https://docs.djangoproject.com/en/4.2/ref/settings/#secret-key
# Used for Sessions, Messages, PasswordResetView tokens.
# "... not used for passwords of users and key rotation will not affect them."
SECRET_KEY = keyring.get_password("rockstor", "SECRET_KEY")

try:
secret_key_fallback = keyring.get_password("rockstor", "SECRET_KEY_FALLBACK")
if secret_key_fallback is not None:
# New in Django 4.1: https://docs.djangoproject.com/en/4.2/ref/settings/#secret-key-fallbacks
SECRET_KEY_FALLBACKS = [secret_key_fallback]
else:
print("No SECRET_KEY_FALLBACK - rotated on reboot / rockstor services restart.")
except keyring.errors.KeyringError:
print("KeyringError")


# API client secret
CLIENT_SECRET = secrets.token_urlsafe()
CLIENT_SECRET = keyring.get_password("rockstor", "CLIENT_SECRET")

# New in Django 1.8 to cover all prior TEMPLATE_* settings.
# https://docs.djangoproject.com/en/1.11/ref/templates/upgrading/
Expand Down