Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pkg/ottl] Added ottl functions for hashing strings #22968

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions .chloggen/ottl_hash_functions.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Use this changelog template to create an entry for release notes.
# If your change doesn't affect end users, such as a test fix or a tooling change,
# you should instead start your pull request title with [chore] or use the "Skip Changelog" label.

# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: enhancement

# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
component: pkg/ottl

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: "Add hash converters/functions for OTTL"

# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
issues: [22725]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext:
64 changes: 64 additions & 0 deletions pkg/ottl/ottlfuncs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -263,10 +263,13 @@ Unlike functions, they do not modify any input telemetry and always return a val
Available Converters:
- [Concat](#concat)
- [ConvertCase](#convertcase)
- [FNV](#fnv)
- [Int](#int)
- [IsMatch](#ismatch)
- [Log](#log)
- [ParseJSON](#parsejson)
- [SHA1](#sha1)
- [SHA256](#sha256)
- [SpanID](#spanid)
- [Split](#split)
- [TraceID](#traceid)
Expand Down Expand Up @@ -316,6 +319,25 @@ Examples:

- `ConvertCase(metric.name, "snake")`

### FNV

`FNV(value)`

The `FNV` Converter converts the `value` to an FNV hash/digest.

The returned type is int64.

`value` is either a path expression to a string telemetry field or a literal string. If `value` is another type an error is returned.

If an error occurs during hashing it will be returned.

Examples:

- `FNV(attributes["device.name"])`


- `FNV("name")`
TylerHelmuth marked this conversation as resolved.
Show resolved Hide resolved

### Int

`Int(value)`
Expand Down Expand Up @@ -424,6 +446,48 @@ Examples:

- `ParseJSON(body)`

### SHA1

`SHA1(value)`

The `SHA1` Converter converts the `value` to a sha1 hash/digest.

The returned type is string.

`value` is either a path expression to a string telemetry field or a literal string. If `value` is another type an error is returned.

If an error occurs during hashing it will be returned.

Examples:

- `SHA1(attributes["device.name"])`


- `SHA1("name")`

**Note:** According to the National Institute of Standards and Technology (NIST), SHA1 is no longer a recommended hash function. It should be avoided except when required for compatibility. New uses should prefer FNV whenever possible.

### SHA256

`SHA256(value)`

The `SHA256` Converter converts the `value` to a sha256 hash/digest.

The returned type is string.

`value` is either a path expression to a string telemetry field or a literal string. If `value` is another type an error is returned.

If an error occurs during hashing it will be returned.

Examples:

- `SHA256(attributes["device.name"])`


- `SHA256("name")`

**Note:** According to the National Institute of Standards and Technology (NIST), SHA256 is no longer a recommended hash function. It should be avoided except when required for compatibility. New uses should prefer FNV whenever possible.

### SpanID

`SpanID(bytes)`
Expand Down
47 changes: 47 additions & 0 deletions pkg/ottl/ottlfuncs/func_fnv.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

package ottlfuncs // import "github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl/ottlfuncs"

import (
"context"
"fmt"
"hash/fnv"

"github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl"
)

type FnvArguments[K any] struct {
Target ottl.StringGetter[K] `ottlarg:"0"`
}

func NewFnvFactory[K any]() ottl.Factory[K] {
return ottl.NewFactory("FNV", &FnvArguments[K]{}, createFnvFunction[K])
}

func createFnvFunction[K any](_ ottl.FunctionContext, oArgs ottl.Arguments) (ottl.ExprFunc[K], error) {
args, ok := oArgs.(*FnvArguments[K])

if !ok {
return nil, fmt.Errorf("FNVFactory args must be of type *FnvArguments[K]")
}

return FNVHashString(args.Target)
}

func FNVHashString[K any](target ottl.StringGetter[K]) (ottl.ExprFunc[K], error) {

return func(ctx context.Context, tCtx K) (interface{}, error) {
val, err := target.Get(ctx, tCtx)
if err != nil {
return nil, err
}
hash := fnv.New64a()
_, err = hash.Write([]byte(val))
if err != nil {
return nil, err
}
hashValue := hash.Sum64()
return int64(hashValue), nil
}, nil
}
82 changes: 82 additions & 0 deletions pkg/ottl/ottlfuncs/func_fnv_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

package ottlfuncs

import (
"context"
"testing"

"github.com/stretchr/testify/assert"

"github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl"
)

func Test_FNV(t *testing.T) {
TylerHelmuth marked this conversation as resolved.
Show resolved Hide resolved
tests := []struct {
name string
value interface{}
expected interface{}
err bool
}{
{
name: "string",
value: "hello world",
expected: int64(8618312879776256743),
},
{
name: "empty string",
value: "",
expected: int64(-3750763034362895579),
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
exprFunc, err := FNVHashString[interface{}](&ottl.StandardStringGetter[interface{}]{
Getter: func(context.Context, interface{}) (interface{}, error) {
return tt.value, nil
},
})
assert.NoError(t, err)
result, err := exprFunc(nil, nil)
if tt.err {
assert.Error(t, err)
} else {
assert.NoError(t, err)
}
assert.Equal(t, tt.expected, result)
})
}
}

func Test_FNVError(t *testing.T) {
tests := []struct {
name string
value interface{}
err bool
expectedError string
}{
{
name: "non-string",
value: 10,
expectedError: "expected string but got int",
},
{
name: "nil",
value: nil,
expectedError: "expected string but got nil",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
exprFunc, err := FNVHashString[interface{}](&ottl.StandardStringGetter[interface{}]{
Getter: func(context.Context, interface{}) (interface{}, error) {
return tt.value, nil
},
})
assert.NoError(t, err)
_, err = exprFunc(nil, nil)
assert.ErrorContains(t, err, tt.expectedError)
})
}
}
48 changes: 48 additions & 0 deletions pkg/ottl/ottlfuncs/func_sha1.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

package ottlfuncs // import "github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl/ottlfuncs"

import (
"context"
"crypto/sha1" // #nosec
"encoding/hex"
"fmt"

"github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl"
)

type SHA1Arguments[K any] struct {
Target ottl.StringGetter[K] `ottlarg:"0"`
}

func NewSHA1Factory[K any]() ottl.Factory[K] {
return ottl.NewFactory("SHA1", &SHA1Arguments[K]{}, createSHA1Function[K])
}

func createSHA1Function[K any](_ ottl.FunctionContext, oArgs ottl.Arguments) (ottl.ExprFunc[K], error) {
args, ok := oArgs.(*SHA1Arguments[K])

if !ok {
return nil, fmt.Errorf("SHA1Factory args must be of type *SHA1Arguments[K]")
}

return SHA1HashString(args.Target)
}

func SHA1HashString[K any](target ottl.StringGetter[K]) (ottl.ExprFunc[K], error) {

return func(ctx context.Context, tCtx K) (interface{}, error) {
val, err := target.Get(ctx, tCtx)
if err != nil {
return nil, err
}
hash := sha1.New() // #nosec
_, err = hash.Write([]byte(val))
if err != nil {
return nil, err
}
hashValue := hex.EncodeToString(hash.Sum(nil))
return hashValue, nil
TylerHelmuth marked this conversation as resolved.
Show resolved Hide resolved
}, nil
}
82 changes: 82 additions & 0 deletions pkg/ottl/ottlfuncs/func_sha1_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

package ottlfuncs

import (
"context"
"testing"

"github.com/stretchr/testify/assert"

"github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl"
)

func Test_SHA1(t *testing.T) {
rnishtala-sumo marked this conversation as resolved.
Show resolved Hide resolved
tests := []struct {
name string
value interface{}
expected interface{}
err bool
}{
{
name: "string",
value: "hello world",
expected: "2aae6c35c94fcfb415dbe95f408b9ce91ee846ed",
},
{
name: "empty string",
value: "",
expected: "da39a3ee5e6b4b0d3255bfef95601890afd80709",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
exprFunc, err := SHA1HashString[interface{}](&ottl.StandardStringGetter[interface{}]{
Getter: func(context.Context, interface{}) (interface{}, error) {
return tt.value, nil
},
})
assert.NoError(t, err)
result, err := exprFunc(nil, nil)
if tt.err {
assert.Error(t, err)
} else {
assert.NoError(t, err)
}
assert.Equal(t, tt.expected, result)
})
}
}

func Test_SHA1Error(t *testing.T) {
tests := []struct {
name string
value interface{}
err bool
expectedError string
}{
{
name: "non-string",
value: 10,
expectedError: "expected string but got int",
},
{
name: "nil",
value: nil,
expectedError: "expected string but got nil",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
exprFunc, err := SHA1HashString[interface{}](&ottl.StandardStringGetter[interface{}]{
Getter: func(context.Context, interface{}) (interface{}, error) {
return tt.value, nil
},
})
assert.NoError(t, err)
_, err = exprFunc(nil, nil)
assert.ErrorContains(t, err, tt.expectedError)
})
}
}
Loading