Skip to content

Commit

Permalink
Add explicit username and password option for Mongodb (elastic#2889)
Browse files Browse the repository at this point in the history
We previously only had the option of specifying a user/pass in the
URL string, which is problematic because it results in the password
being indexed in Elasticsearch (elastic#2888).

This adds the option to specify a username/password at the module
configuration. To make this happen, I had to copy some unexported
functions from the mgo driver.
  • Loading branch information
tsg authored and ruflin committed Oct 31, 2016
1 parent b80e26e commit 6c7dfe4
Show file tree
Hide file tree
Showing 9 changed files with 287 additions and 9 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ https://github.com/elastic/beats/compare/v5.0.0...master[Check the HEAD diff]
- Add experimental filebeat metricset in the beats module. {pull}2297[2297]
- Add experimental libbeat metricset in the beats module. {pull}2339[2339]
- Add experimental docker module. Provided by Ingensi and @douaejeouit based on dockbeat.
- Add username and password config options to the MongoDB module. {pull}2889[2889]

*Packetbeat*
- Define `client_geoip.location` as geo_point in the mappings to be used by the GeoIP processor in the Ingest Node pipeline.
Expand Down
28 changes: 26 additions & 2 deletions metricbeat/docs/modules/mongodb.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ servers.
When configuring the `hosts` option, you must use MongoDB URLs of the following format:

-----------------------------------
[mongodb://][user:pass@]host[:port]
[mongodb://host[:port][?options]
-----------------------------------

The URL can be as simple as:
Expand All @@ -33,6 +33,21 @@ Or more complex like:
hosts: ["mongodb://myuser:mypass@localhost:40001", "otherhost:40001"]
----------------------------------------------------------------------

WARNING: In case you use username and password in the hosts array, this
information will be sent with each event as part of the `metricset.host` field.
To prevent sending username and password the config options `username` and
`password` can be used.

[source,yaml]
----
- module: mongodb
metricsets: ["status"]
hosts: ["localhost:27017"]
username: root
password: test
----



[float]
=== Compatibility
Expand All @@ -55,9 +70,18 @@ metricbeat.modules:
#period: 10s
# The hosts must be passed as MongoDB URLs in the format:
# [mongodb://][user:pass@]host[:port]
# [mongodb://][user:pass@]host[:port].
# Warning: specifying the user/password in the hosts array is possible
# but it will result in the password being present in the output documents.
# We recommend using the username and password options instead.
#hosts: ["localhost:27017"]
# Username to use when connecting to MongoDB. Empty by default.
#username: user
# Password to use when connecting to MongoDB. Empty by default.
#password: pass
----

[float]
Expand Down
11 changes: 10 additions & 1 deletion metricbeat/etc/beat.full.yml
Original file line number Diff line number Diff line change
Expand Up @@ -105,9 +105,18 @@ metricbeat.modules:
#period: 10s

# The hosts must be passed as MongoDB URLs in the format:
# [mongodb://][user:pass@]host[:port]
# [mongodb://][user:pass@]host[:port].
# Warning: specifying the user/password in the hosts array is possible
# but it will result in the password being present in the output documents.
# We recommend using the username and password options instead.
#hosts: ["localhost:27017"]

# Username to use when connecting to MongoDB. Empty by default.
#username: user

# Password to use when connecting to MongoDB. Empty by default.
#password: pass


#-------------------------------- MySQL Module -------------------------------
#- module: mysql
Expand Down
11 changes: 10 additions & 1 deletion metricbeat/metricbeat.full.yml
Original file line number Diff line number Diff line change
Expand Up @@ -105,9 +105,18 @@ metricbeat.modules:
#period: 10s

# The hosts must be passed as MongoDB URLs in the format:
# [mongodb://][user:pass@]host[:port]
# [mongodb://][user:pass@]host[:port].
# Warning: specifying the user/password in the hosts array is possible
# but it will result in the password being present in the output documents.
# We recommend using the username and password options instead.
#hosts: ["localhost:27017"]

# Username to use when connecting to MongoDB. Empty by default.
#username: user

# Password to use when connecting to MongoDB. Empty by default.
#password: pass


#-------------------------------- MySQL Module -------------------------------
#- module: mysql
Expand Down
11 changes: 10 additions & 1 deletion metricbeat/module/mongodb/_meta/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,15 @@
#period: 10s

# The hosts must be passed as MongoDB URLs in the format:
# [mongodb://][user:pass@]host[:port]
# [mongodb://][user:pass@]host[:port].
# Warning: specifying the user/password in the hosts array is possible
# but it will result in the password being present in the output documents.
# We recommend using the username and password options instead.
#hosts: ["localhost:27017"]

# Username to use when connecting to MongoDB. Empty by default.
#username: user

# Password to use when connecting to MongoDB. Empty by default.
#password: pass

17 changes: 16 additions & 1 deletion metricbeat/module/mongodb/_meta/docs.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ servers.
When configuring the `hosts` option, you must use MongoDB URLs of the following format:

-----------------------------------
[mongodb://][user:pass@]host[:port]
[mongodb://host[:port][?options]
-----------------------------------

The URL can be as simple as:
Expand All @@ -28,6 +28,21 @@ Or more complex like:
hosts: ["mongodb://myuser:mypass@localhost:40001", "otherhost:40001"]
----------------------------------------------------------------------

WARNING: In case you use username and password in the hosts array, this
information will be sent with each event as part of the `metricset.host` field.
To prevent sending username and password the config options `username` and
`password` can be used.

[source,yaml]
----
- module: mongodb
metricsets: ["status"]
hosts: ["localhost:27017"]
username: root
password: test
----



[float]
=== Compatibility
Expand Down
123 changes: 123 additions & 0 deletions metricbeat/module/mongodb/parseurl.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
package mongodb

import (
"errors"
"fmt"
"net/url"
"strings"

mgo "gopkg.in/mgo.v2"
)

/*
* Functions copied from the mgo driver to help with parsing the URL.
*
* http://bazaar.launchpad.net/+branch/mgo/v2/view/head:/session.go#L382
*/

type urlInfo struct {
addrs []string
user string
pass string
db string
options map[string]string
}

func parseURL(s string) (*urlInfo, error) {
if strings.HasPrefix(s, "mongodb://") {
s = s[10:]
}
info := &urlInfo{options: make(map[string]string)}
if c := strings.Index(s, "?"); c != -1 {
for _, pair := range strings.FieldsFunc(s[c+1:], isOptSep) {
l := strings.SplitN(pair, "=", 2)
if len(l) != 2 || l[0] == "" || l[1] == "" {
return nil, errors.New("connection option must be key=value: " + pair)
}
info.options[l[0]] = l[1]
}
s = s[:c]
}
if c := strings.Index(s, "@"); c != -1 {
pair := strings.SplitN(s[:c], ":", 2)
if len(pair) > 2 || pair[0] == "" {
return nil, errors.New("credentials must be provided as user:pass@host")
}
var err error
info.user, err = url.QueryUnescape(pair[0])
if err != nil {
return nil, fmt.Errorf("cannot unescape username in URL: %q", pair[0])
}
if len(pair) > 1 {
info.pass, err = url.QueryUnescape(pair[1])
if err != nil {
return nil, fmt.Errorf("cannot unescape password in URL")
}
}
s = s[c+1:]
}
if c := strings.Index(s, "/"); c != -1 {
info.db = s[c+1:]
s = s[:c]
}
info.addrs = strings.Split(s, ",")
return info, nil
}

func isOptSep(c rune) bool {
return c == ';' || c == '&'
}

// ParseURL parses the given URL and returns a DialInfo structure ready
// to be passed to DialWithInfo
func ParseURL(host, username, pass string) (*mgo.DialInfo, error) {
uinfo, err := parseURL(host)
if err != nil {
return nil, err
}
direct := false
mechanism := ""
service := ""
source := ""
for k, v := range uinfo.options {
switch k {
case "authSource":
source = v
case "authMechanism":
mechanism = v
case "gssapiServiceName":
service = v
case "connect":
if v == "direct" {
direct = true
break
}
if v == "replicaSet" {
break
}
fallthrough
default:
return nil, errors.New("unsupported connection URL option: " + k + "=" + v)
}
}

info := &mgo.DialInfo{
Addrs: uinfo.addrs,
Direct: direct,
Database: uinfo.db,
Username: uinfo.user,
Password: uinfo.pass,
Mechanism: mechanism,
Service: service,
Source: source,
}

if len(username) > 0 {
info.Username = username
}
if len(pass) > 0 {
info.Password = pass
}

return info, nil
}
78 changes: 78 additions & 0 deletions metricbeat/module/mongodb/parseurl_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
package mongodb

import (
"testing"

"github.com/stretchr/testify/assert"
)

func TestParseURL(t *testing.T) {
tests := []struct {
Name string
URL string
Username string
Password string
ExpectedAddr string
ExpectedUsername string
ExpectedPassword string
}{
{
Name: "basic test",
URL: "localhost:40001",
Username: "user",
Password: "secret",

ExpectedAddr: "localhost:40001",
ExpectedUsername: "user",
ExpectedPassword: "secret",
},
{
Name: "with schema",
URL: "mongodb://localhost:40001",
Username: "user",
Password: "secret",

ExpectedAddr: "localhost:40001",
ExpectedUsername: "user",
ExpectedPassword: "secret",
},
{
Name: "user password in url",
URL: "mongodb://user:secret@localhost:40001",
Username: "",
Password: "",

ExpectedAddr: "localhost:40001",
ExpectedUsername: "user",
ExpectedPassword: "secret",
},
{
Name: "user password overwride",
URL: "mongodb://user:secret@localhost:40001",
Username: "anotheruser",
Password: "anotherpass",

ExpectedAddr: "localhost:40001",
ExpectedUsername: "anotheruser",
ExpectedPassword: "anotherpass",
},
{
Name: "with options",
URL: "mongodb://localhost:40001?connect=direct&authSource=me",
Username: "anotheruser",
Password: "anotherpass",

ExpectedAddr: "localhost:40001",
ExpectedUsername: "anotheruser",
ExpectedPassword: "anotherpass",
},
}

for _, test := range tests {
info, err := ParseURL(test.URL, test.Username, test.Password)
assert.NoError(t, err, test.Name)
assert.Equal(t, info.Addrs[0], test.ExpectedAddr, test.Name)
assert.Equal(t, info.Username, test.ExpectedUsername, test.Name)
assert.Equal(t, info.Password, test.ExpectedPassword, test.Name)
}
}
Loading

0 comments on commit 6c7dfe4

Please sign in to comment.