Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[receiver/snowflakereceiver] added more detailed readme #18203

Merged
merged 4 commits into from
Feb 22, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions .chloggen/snowflake_add_readme.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: 'enhancement'

# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
component: snowflakereceiver

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: updated README.md to include more detailed documentation.

# One or more tracking issues related to the change
issues: [14754]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext:
37 changes: 36 additions & 1 deletion receiver/snowflakereceiver/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,42 @@
| Supported pipeline types | metrics |
| Distributions | [contrib] |

This receiver collects metrics from a Snowflake account by connecting to an account and running queries at set intervals.
This receiver collects metrics from a Snowflake account by connecting to and querying a Snowflake deployment.

## Configuration

The following settings are required:

* `username` (no default): Specifies username used to authenticate with Snowflake.
* `password` (no default): Specifies the password associated with designated username. Used to authenticate with Snowflake.
* `account` (no default): Specifies the account from which metrics are to be gathered.
* `warehouse` (no default): Specifies the warehouse, or unit of computer, designated for the metric gathering queries. Must be an existing warehouse in your Snowflake account.

The following settings are optional:

* `metrics` (default: see `DefaultMetricSettings` [here](./internal/metadata/generated_metrics.go)): Controls the enabling/disabling of specific metrics. For in depth documentation on the allowable metrics see [here](./documentation.md).
* `schema` (default: 'ACCOUNT_USAGE'): Snowflake DB schema containing usage statistics and metadata to be monitored.
* `database` (default: 'SNOWFLAKE'): Snowflake DB containing schema with usage statistics and metadata to be monitored.
* `role` (default: 'ACCOUNTADMIN'): Role associated with the username designated above. By default admin privileges are required to access most/all of the usage data.
* `collection_interval` (default: 30m): Collection interval for metrics receiver. The value for this setting must be readable by golang's [time.ParseDuration](https://pkg.go.dev/time#ParseDuration).

Example:
```yaml
receivers:
snowflake:
username: snowflakeuser
password: securepassword
account: bigbusinessaccount
warehouse: metricWarehouse
collection_interval: 18m
metrics:
snowflake.database.bytes_scanned.avg:
enabled: true
snowflake.database.bytes_deketed.avg:
enabled: false
```

The full list of settings exposed for this receiver are documented [here](./config.go) with a detailed sample configuration [here](./testdata/config.yaml)

[development]: https://github.com/open-telemetry/opentelemetry-collector#development
[contrib]:https://github.com/open-telemetry/opentelemetry-collector-releases/tree/main/distributions/otelcol-contrib
dmitryax marked this conversation as resolved.
Show resolved Hide resolved
50 changes: 50 additions & 0 deletions receiver/snowflakereceiver/config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,20 @@
package snowflakereceiver

import (
"path/filepath"
"testing"
"time"

"github.com/google/go-cmp/cmp"
"github.com/google/go-cmp/cmp/cmpopts"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"go.opentelemetry.io/collector/component"
"go.opentelemetry.io/collector/confmap/confmaptest"
"go.opentelemetry.io/collector/receiver/scraperhelper"
"go.uber.org/multierr"

"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/snowflakereceiver/internal/metadata"
)

func TestValidateConfig(t *testing.T) {
Expand Down Expand Up @@ -98,3 +108,43 @@ func TestValidateConfig(t *testing.T) {
})
}
}

func TestLoadConfig(t *testing.T) {
t.Parallel()

cm, err := confmaptest.LoadConf(filepath.Join("testdata", "config.yaml"))
require.NoError(t, err)
// LoadConf includes the TypeStr which NewFactory does not set
id := component.NewIDWithName(typeStr, "")
cmNoStr, err := cm.Sub(id.String())
require.NoError(t, err)

testMetrics := metadata.DefaultMetricsSettings()
testMetrics.SnowflakeDatabaseBytesScannedAvg.Enabled = true
testMetrics.SnowflakeQueryBytesDeletedAvg.Enabled = false

expected := &Config{
Username: "snowflakeuser",
Password: "securepassword",
Account: "bigbusinessaccount",
Warehouse: "metricWarehouse",
ScraperControllerSettings: scraperhelper.ScraperControllerSettings{
CollectionInterval: 18 * time.Minute,
},
Role: "customMonitoringRole",
Database: "SNOWFLAKE",
Schema: "ACCOUNT_USAGE",
Metrics: testMetrics,
}

factory := NewFactory()
cfg := factory.CreateDefaultConfig()

require.NoError(t, component.UnmarshalConfig(cmNoStr, cfg))
assert.NoError(t, component.ValidateConfig(cfg))

diff := cmp.Diff(expected, cfg, cmpopts.IgnoreUnexported(metadata.MetricSettings{}))
if diff != "" {
t.Errorf("config mismatch (-expected / +actual)\n%s", diff)
}
}
1 change: 1 addition & 0 deletions receiver/snowflakereceiver/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ go 1.19

require (
github.com/DATA-DOG/go-sqlmock v1.5.0
github.com/google/go-cmp v0.5.9
github.com/open-telemetry/opentelemetry-collector-contrib/internal/coreinternal v0.71.0
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatatest v0.71.0
github.com/snowflakedb/gosnowflake v1.6.17
Expand Down
1 change: 1 addition & 0 deletions receiver/snowflakereceiver/go.sum

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

14 changes: 14 additions & 0 deletions receiver/snowflakereceiver/testdata/config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
snowflake:
# required settings
username: snowflakeuser
password: securepassword
account: bigbusinessaccount
warehouse: metricWarehouse
# optional settings (i.e. have defaults)
collection_interval: 18m
metrics:
snowflake.database.bytes_scanned.avg:
enabled: true
snowflake.query.bytes_deleted.avg:
enabled: false
role: customMonitoringRole
dmitryax marked this conversation as resolved.
Show resolved Hide resolved