Skip to content

Commit

Permalink
Merge pull request #72 from StackStorm/bugfix-and-overrides
Browse files Browse the repository at this point in the history 
Bugfix and overrides
  • Loading branch information
@blag
blag authored Jul 1, 2020
2 parents 53a1ec9 + 7dbe815 commit 0f81bd0
Show file tree
Hide file tree
Showing 8 changed files with 381 additions and 76 deletions.
13 changes: 7 additions & 6 deletions .travis.yml
View file
Original file line number Diff line number Diff line change
@@ -1,14 +1,15 @@
language: python
python: 2.7
python:
- "2.7"
- "3.6"
sudo: false
os:
- linux
env:
- TOX_ENV=lint
- TOX_ENV=py27

install:
- pip install tox
- pip install -r test-requirements.txt -r requirements.txt

script:
- tox -e $TOX_ENV
- flake8 --config ./lint-configs/python/.flake8 st2auth_enterprise_ldap_backend/
- pylint -E --rcfile=./lint-configs/python/.pylintrc st2auth_enterprise_ldap_backend/
- nosetests -sv --with-coverage --cover-package=st2auth_enterprise_ldap_backend tests/unit
138 changes: 114 additions & 24 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,28 +16,43 @@ sudo dnf install python2-devel python3-devel openldap-devel

## Configuration Options

| option | required | default | description |
|----------------------------|----------|---------|--------------------------------------------------------------------------------------------------------------------------------|
| bind_dn | yes | | DN of the service account to bind with the LDAP server |
| bind_password | yes | | Password of the service account |
| base_ou | yes | | Base OU to search for user and group entries |
| group_dns | yes | | Which groups user must be member of to be granted access |
| group_dns_check | no | and | What kind of check to perform when validating user group membership (``and`` / ``or``). When ``and`` behavior is used, user needs to be part of all the specified groups and when ``or`` behavior is used, user needs to be part of at least one or more of the specified groups. |
| host | yes | | Hostname of the LDAP server |
| port | yes | | Port of the LDAP server |
| use_ssl | no | false | Use LDAPS to connect |
| use_tls | no | false | Start TLS on LDAP to connect |
| cacert | no | None | Path to the CA cert used to validate certificate |
| id_attr | no | uid | Field name of the user ID attribute |
| scope | no | subtree | Search scope (base, onelevel, or subtree) |
| network_timeout | no | 10.0 | Timeout for network operations (in seconds) |
| chase_referrals | no | false | True if the referrals should be automatically chased within the underlying LDAP C lib |
| debug | no | false | Enable debug mode. When debug mode is enabled all the calls (including the results) to LDAP server are logged |
| client_options | no | | A dictionary with additional Python LDAP client options which can be passed to ``set_connection()`` method |
| cache_user_groups_response | no | true | When true, LDAP user groups response is cached for 120 seconds (by default) in memory. This decreases load on LDAP server and increases performance when remote LDAP group to RBAC role sync is enabled and / or when the same user authenticates concurrency in a short time frame. Keep in mind that even when this feature is enabled, single (authenticate) request to LDAP server will still be performed when user authenticates to st2auth - authentication information is not cached - only user groups are cached. |
| cache_user_groups_ttl | no | 120 | How long (in seconds) |

## Configuration Example
| option | required | default | description |
|----------------------------|----------|----------------|--------------------------------------------------------------------------------------------------------------------------------|
| bind_dn | yes | | DN of the service account to bind with the LDAP server |
| bind_password | yes | | Password of the service account |
| base_ou | yes | | Base OU to search for user and group entries |
| group_dns | yes | | Which groups user must be member of to be granted access (group names are considered case-insensitive) |
| group_dns_check | no | `and` | What kind of check to perform when validating user group membership (`and` / `or`). When `and` behavior is used, user needs to be part of all the specified groups and when `or` behavior is used, user needs to be part of at least one or more of the specified groups. |
| host | yes | | Hostname of the LDAP server |
| port | yes | | Port of the LDAP server |
| use_ssl | no | `false` | Use LDAPS to connect |
| use_tls | no | `false` | Start TLS on LDAP to connect |
| cacert | no | `None` | Path to the CA cert used to validate certificate |
| id_attr | no | `uid` | Field name of the user ID attribute; ignored if `account_pattern` is specified. |
| account_pattern | no | `{id_attr}={{username}}` | LDAP subtree pattern to match user. The user's `username` is escaped and interpolated into this string (see example). |
| group_pattern | no | `(|(&(objectClass=*)(|(member={user_dn})(uniqueMember={user_dn})(memberUid={username}))))` | LDAP subtree pattern for user groups. Both `user_dn` and `username` are escaped and then interpolated into this string (see example). |
| scope | no | `subtree` | Search scope (base, onelevel, or subtree) |
| network_timeout | no | `10.0` | Timeout for network operations (in seconds) |
| chase_referrals | no | `false` | True if the referrals should be automatically chased within the underlying LDAP C lib |
| debug | no | `false` | Enable debug mode. When debug mode is enabled all the calls (including the results) to LDAP server are logged |
| client_options | no | | A dictionary with additional Python LDAP client options which can be passed to `set_connection()` method |
| cache_user_groups_response | no | `true` | When true, LDAP user groups response is cached for 120 seconds (by default) in memory. This decreases load on LDAP server and increases performance when remote LDAP group to RBAC role sync is enabled and / or when the same user authenticates concurrency in a short time frame. Keep in mind that even when this feature is enabled, single (authenticate) request to LDAP server will still be performed when user authenticates to st2auth - authentication information is not cached - only user groups are cached. |
| cache_user_groups_ttl | no | `120` | How long (in seconds) |

## Implementation Overview

The LDAP backend attempts a few different LDAP operations to authenticate users against an LDAP server:

1. Attempts an LDAP bind with the StackStorm service credentials `bind_dn` and `bind_password`.
2. Searches the LDAP server for the username provided by the StackStorm user, and saves the user's `bind_dn` attribute as `user_dn`.
3. Fetches the user's LDAP groups and compares them against the groups in the `group_dns` configuration using the logic from `group_dns_check`, the `user_dn` from step 2, and the `username` supplied by the StackStorm user.
4. Attempts to re-bind to the LDAP server with the StackStorm user's supplied username and password.

If all of the steps succeed, then the user is authenticated. If any of those steps fail, then the authentication fails.

## Configuration Examples

### Simple Configuration

Please refer to the [standalone mode](http://docs.stackstorm.com/config/authentication.html#setup-standalone-mode) in the configuration section for authentication for basic setup concept. The following is an example of the auth section in the StackStorm configuration file for the LDAP backend.

Expand All @@ -55,15 +70,15 @@ logging = /path/to/st2auth.logging.conf
api_url = http://myhost.example.com:9101/
```

Note: Key in the ``client_options`` dictionary must be an integer representing a LDAP constant option value.
Note: Key in the `client_options` dictionary must be an integer representing a LDAP constant option value.

For example:

```ini
backend_kwargs = {..., "client_options": {"20482": 9}}
```

In this case, "20482" represents ``ldap.OPT_TIMEOUT`` option.
In this case, "20482" represents `ldap.OPT_TIMEOUT` option.

To retrieve a integer value of a particular client option constant, you can run the following code:

Expand All @@ -72,6 +87,81 @@ import ldap
print(ldap.OPT_TIMEOUT)
```

Additionally, this simple example uses the default values for the `id_attr`, `account_pattern`, and `group_pattern` configuration options. This means that the user's account will be queried with the default LDAP search pattern `uid={username}`, and the groups will be queried with the default LDAP search pattern `(|(&(objectClass=*)(|(member={user_dn})(uniqueMember={user_dn})(memberUid={username}))))`.

### Configuration Specifying `id_attr`

If your LDAP server uses a different name for the user ID attribute, you can simply specify the `id_attr` configuration option.

```ini
[auth]
mode = standalone
backend = ldap
backend_kwargs = {"bind_dn": "CN=st2admin,ou=users,dc=example,dc=com", "bind_password": "foobar123", "base_ou": "dc=example,dc=com", "id_attr": "username", "group_dns": ["CN=st2users,ou=groups,dc=example,dc=com", "CN=st2developers,ou=groups,dc=example,dc=com"], "host": "identity.example.com", "port": 636, "use_ssl": true, "cacert": "/path/to/cacert.pem"}
enable = True
debug = False
use_ssl = True
cert = /path/to/mycert.crt
key = /path/to/mycert.key
logging = /path/to/st2auth.logging.conf
api_url = http://myhost.example.com:9101/
```

#### Explanation

In this example, the user's account will be queried with the LDAP search pattern `username={username}`, and the user's groups will be queried with the default LDAP search pattern from above.

### Configuration Overriding `account_pattern` and `group_pattern`

```ini
[auth]
mode = standalone
backend = ldap
backend_kwargs = {"bind_dn": "CN=st2admin,ou=users,dc=example,dc=com", "bind_password": "foobar123", "base_ou": "dc=example,dc=com", "account_pattern": "(&(objectClass=person)(|(accountName={username})(mail={username})))", "group_pattern": "(&(objectClass=userGroup)(|(member={user_dn})(uniqueMember={user_dn})(memberUsername={username})))", "group_dns": ["CN=st2users,ou=groups,dc=example,dc=com", "CN=st2developers,ou=groups,dc=example,dc=com"], "host": "identity.example.com", "port": 636, "use_ssl": true, "cacert": "/path/to/cacert.pem"}
enable = True
debug = False
use_ssl = True
cert = /path/to/mycert.crt
key = /path/to/mycert.key
logging = /path/to/st2auth.logging.conf
api_url = http://myhost.example.com:9101/
```

Note: You do not have to override both `account_pattern` and `group_pattern` together, you may only need to override one of them.

#### Explanation

It's easier to read the `account_pattern` string `(&(objectClass=person)(|(accountName={username})(mail={username})))` if it is reformatted to show the nesting:

```
(&
(objectClass=person)
(|
(accountName={username})
(mail={username})
)
)
```

The Python string format patterns `{username}` will be interpolated with the LDAP username of users who are authenticating with StackStorm. This means that the `account_pattern` search string will query the LDAP server for a user who has the LDAP `objectClass` attribute equal to `person` **and** whose LDAP `accountName` **or** LDAP `mail` attributes are equal to the username they submit to StackStorm.

For `group_pattern`, the usage is similar, but it has an additional `user_dn` variable that will be interpolated into the string:

```
(&
(objectClass=userGroup)
(|
(member={user_dn})
(uniqueMember={user_dn})
(memberUsername={username})
)
)
```

This search string will query for LDAP objects that have `objectClass` attributes equal to `userGroup` **and** whose LDAP `member` **or** `uniqueMember` attributes are equal to the user's LDAP `user_dn` value **or** whose LDAP `memberUsername` attributes are equal to the username they submit to StackStorm.

The `user_dn` value is the user's `bind_dn` attribute returned by the LDAP server in step 2.

## Running tests

Unit tests:
Expand Down
115 changes: 86 additions & 29 deletions dist_utils.py
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# -*- coding: utf-8 -*-
# Copyright (C) 2020 Extreme Networks, Inc - All Rights Reserved
# NOTE: This file is auto-generated - DO NOT EDIT MANUALLY
# Instead modify scripts/dist_utils.py and run 'make .sdist-requirements' to
# update dist_utils.py files for all components

# Copyright 2019 Extreme Networks, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
Expand All @@ -21,31 +25,29 @@

from distutils.version import StrictVersion

GET_PIP = 'curl https://bootstrap.pypa.io/get-pip.py | python'

try:
import pip
from pip import __version__ as pip_version
except ImportError as e:
print('Failed to import pip: %s' % (str(e)))
print('')
print('Download pip:\n%s' % (GET_PIP))
sys.exit(1)

try:
# pip < 10.0
from pip.req import parse_requirements
except ImportError:
# pip >= 10.0
# NOTE: This script can't rely on any 3rd party dependency so we need to use this code here
#
# TODO: Why can't this script rely on 3rd party dependencies? Is it because it has to import
# from pip?
#
# TODO: Dear future developer, if you are back here fixing a bug with how we parse
# requirements files, please look into using the packaging package on PyPI:
# https://packaging.pypa.io/en/latest/requirements/
# and specifying that in the `setup_requires` argument to `setuptools.setup()`
# for subpackages.
# At the very least we can vendorize some of their code instead of reimplementing
# each piece of their code every time our parsing breaks.
PY3 = sys.version_info[0] == 3

if PY3:
text_type = str
else:
text_type = unicode # NOQA

try:
from pip._internal.req.req_file import parse_requirements
except ImportError as e:
print('Failed to import parse_requirements from pip: %s' % (str(e)))
print('Using pip: %s' % (str(pip_version)))
sys.exit(1)
GET_PIP = 'curl https://bootstrap.pypa.io/get-pip.py | python'

__all__ = [
'check_pip_is_installed',
'check_pip_version',
'fetch_requirements',
'apply_vagrant_workaround',
Expand All @@ -54,30 +56,85 @@
]


def check_pip_is_installed():
"""
Ensure that pip is installed.
"""
try:
import pip # NOQA
except ImportError as e:
print('Failed to import pip: %s' % (text_type(e)))
print('')
print('Download pip:\n%s' % (GET_PIP))
sys.exit(1)

return True


def check_pip_version(min_version='6.0.0'):
"""
Ensure that a minimum supported version of pip is installed.
"""
check_pip_is_installed()

import pip

if StrictVersion(pip.__version__) < StrictVersion(min_version):
print("Upgrade pip, your version '{0}' "
"is outdated. Minimum required version is '{1}':\n{2}".format(pip.__version__,
min_version,
GET_PIP))
sys.exit(1)

return True


def fetch_requirements(requirements_file_path):
"""
Return a list of requirements and links by parsing the provided requirements file.
"""
links = []
reqs = []
for req in parse_requirements(requirements_file_path, session=False):
# Note: req.url was used before 9.0.0 and req.link is used in all the recent versions
link = getattr(req, 'link', getattr(req, 'url', None))
if link:
links.append(str(link))
reqs.append(str(req.req))

def _get_link(line):
vcs_prefixes = ['git+', 'svn+', 'hg+', 'bzr+']

for vcs_prefix in vcs_prefixes:
if line.startswith(vcs_prefix) or line.startswith('-e %s' % (vcs_prefix)):
req_name = re.findall('.*#egg=(.+)([&|@]).*$', line)

if not req_name:
req_name = re.findall('.*#egg=(.+?)$', line)
else:
req_name = req_name[0]

if not req_name:
raise ValueError('Line "%s" is missing "#egg=<package name>"' % (line))

link = line.replace('-e ', '').strip()
return link, req_name[0]

return None, None

with open(requirements_file_path, 'r') as fp:
for line in fp.readlines():
line = line.strip()

if line.startswith('#') or not line:
continue

link, req_name = _get_link(line=line)

if link:
links.append(link)
else:
req_name = line

if ';' in req_name:
req_name = req_name.split(';')[0].strip()

reqs.append(req_name)

return (reqs, links)


Expand Down
Loading

0 comments on commit 0f81bd0

Please sign in to comment.