Skip to content

Commit

Permalink
[pkg/ottl] Revise OTTL function nomenclature (#22032)
Browse files Browse the repository at this point in the history
Revise OTTL function nomenclature

Co-authored-by: Evan Bradley <evan-bradley@users.noreply.github.com>
  • Loading branch information
evan-bradley and evan-bradley authored May 17, 2023
1 parent fe15100 commit c8da52b
Show file tree
Hide file tree
Showing 9 changed files with 183 additions and 172 deletions.
124 changes: 67 additions & 57 deletions pkg/ottl/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,39 +8,78 @@ The OTTL is signal agnostic; it is not aware of the type of telemetry on which i

## Grammar

The OTTL grammar includes Invocations, Values and Boolean Expressions.
The OTTL grammar includes function invocations, Values and Boolean Expressions. These parts all fit into a Statement, which is the basis of execution in the OTTL.

### Invocations
### Editors

Invocations represent a function call that transform the underlying telemetry payload. Invocations are made up of 2 parts:
Editors are functions that transform the underlying telemetry payload. They may return a value, but typically do not. There must be a single Editor Invocation in each OTTL statement.

An Editor is made up of 2 parts:

- a string identifier. The string identifier must start with a lowercase letter.
- zero or more Values (comma separated) surrounded by parentheses (`()`).

**The OTTL does not define any function implementations.**
Users must supply a map between string identifiers and the actual function implementation.
The OTTL will use this map and reflection to generate Invocations, that can then be invoked by the user.
See [ottlfuncs](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/pkg/ottl/ottlfuncs) for pre-made, usable functions.
**The OTTL has no built-in Editors.**
Users must supply a map between string identifiers and Editor implementations.
The OTTL will use this map to determine which implementation to call when executing a Statement.
See [ottlfuncs](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/pkg/ottl/ottlfuncs#editors) for pre-made, usable Editors.

### Converters

Example Invocations
- `route()`
- `set(field, 1)`
Converters are functions that convert data to a new format before being passed as a function argument or used in a Boolean Expression.
Converters are made up of 3 parts:

#### Invocation parameters
- a string identifier. The string identifier must start with an uppercase letter.
- zero or more Values (comma separated) surrounded by parentheses (`()`).
- a combination of zero or more a string key (`["key"]`) or int key (`[0]`)

The OTTL will use reflection to determine parameter types when parsing an invocation within a statement.
**The OTTL has no built-in Converters.**
Users must include Converters in the same map that Editors are supplied.
The OTTL will use this map and reflection to generate Converters that can then be invoked by the user.
See [ottlfuncs](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/pkg/ottl/ottlfuncs#converters) for pre-made, usable Converters.

When keys are supplied the value returned by the Converter will be indexed by the keys in order.
If keys are supplied to a Converter and the return value cannot be indexed, or if the return value doesn't support the
type of key supplied, OTTL will error. Supported values are:

| Type | Index Type |
|------------------|------------|
| `pcommon.Map` | `String` |
| `map[string]any` | `String` |
| `pcommon.Slice` | `Int` |
| `[]any` | `Int` |

Example Converters
- `Int()`
- `IsMatch(field, ".*")`
- `Split(field, ",")[1]`

### Function parameters

The following types are supported for single-value parameters in OTTL functions:

When developing functions that represent invocations, the following types are supported for single parameter values:
- `Setter`
- `GetSetter`
- `Getter`
- `PMapGetter`
- `FloatGetter`
- `FloatLikeGetter`
- `StringGetter`
- `StringLikeGetter`
- `Enum`
- `string`
- `float64`
- `int64`
- `bool`

For slice parameters, the following types are supported:

- `Getter`
- `PMapGetter`
- `FloatGetter`
- `FloatLikeGetter`
- `StringGetter`
- `StringLikeGetter`
- `string`
- `float64`
- `int64`
Expand All @@ -49,15 +88,16 @@ For slice parameters, the following types are supported:

### Values

Values are passed as input to an Invocation or are used in a Boolean Expression. Values can take the form of:
Values are passed as function parameters or are used in a Boolean Expression. Values can take the form of:

- [Paths](#paths)
- [Lists](#lists)
- [Literals](#literals)
- [Enums](#enums)
- [Converters](#converters)
- [Math Expressions](#math-expressions)

#### Paths
### Paths

A Path Value is a reference to a telemetry field. Paths are made up of lowercase identifiers, dots (`.`), and square brackets combined with a string key (`["key"]`) or int key (`[0]`). **The interpretation of a Path is NOT implemented by the OTTL.** Instead, the user must provide a `PathExpressionParser` that the OTTL can use to interpret paths. As a result, how the Path parts are used is up to the user. However, it is recommended that the parts be used like so:

Expand All @@ -76,7 +116,7 @@ Example Paths
- `attributes["nested"]["values"]`
- `cache["slice"][1]`

##### Contexts
#### Contexts

The package that handles the interpretation of a path is normally called a Context.
Contexts will have an implementation of `PathExpressionParser` that decides how an OTTL Path is interpreted.
Expand All @@ -85,7 +125,7 @@ The context's implementation will need to make decisions like what a dot (`.`) r
[There are OpenTelemetry-specific contexts provided for each signal here.](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/pkg/ottl/contexts)
When using OTTL it is recommended to use these contexts unless you have a specific need. Check out each context to view the paths it supports.

#### Lists
### Lists

A List Value comprises a sequence of Values.
Currently, list can only be created by the grammar to be used in functions or conditions;
Expand All @@ -97,7 +137,7 @@ Example List Values:
- `["1", "2", "3"]`
- `["a", attributes["key"], Concat(["a", "b"], "-")]`

#### Literals
### Literals

Literals are literal interpretations of the Value into a Go value. Accepted literals are:

Expand All @@ -116,51 +156,20 @@ Example Literals
- `nil`,
- `0x0001`

#### Enums
### Enums

Enums are uppercase identifiers that get interpreted during parsing and converted to an `int64`. **The interpretation of an Enum is NOT implemented by the OTTL.** Instead, the user must provide a `EnumParser` that the OTTL can use to interpret the Enum. The `EnumParser` returns an `int64` instead of a function, which means that the Enum's numeric value is retrieved during parsing instead of during execution.

Within the grammar Enums are always used as `int64`. As a result, the Enum's symbol can be used as if it is an Int value.

When defining a function that will be used as an Invocation by the OTTL, if the function needs to take an Enum then the function must use the `Enum` type for that argument, not an `int64`.
When defining an OTTL function, if the function needs to take an Enum then the function must use the `Enum` type for that argument, not an `int64`.

#### Converters

Converters are special functions that convert data to a new format before being passed to an Invocation or Boolean Expression.
Converters are made up of 3 parts:

- a string identifier. The string identifier must start with an uppercase letter.
- zero or more Values (comma separated) surrounded by parentheses (`()`).
- a combination of zero or more a string key (`["key"]`) or int key (`[0]`)

**The OTTL does not define any Converter implementations.**
Users must include Converters in the same map that invocations are supplied.
The OTTL will use this map and reflection to generate Converters that can then be invoked by the user.
See [ottlfuncs](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/pkg/ottl/ottlfuncs#converters) for pre-made, usable Converters.

When keys are supplied the value returned by the Converter will be indexed by the keys in order.
If keys are supplied to a Converter and the return value cannot be indexed, or if the return value doesn't support the
type of key supplied, OTTL will error. Supported values are:

| Type | Index Type |
|------------------|------------|
| `pcommon.Map` | `String` |
| `map[string]any` | `String` |
| `pcommon.Slice` | `Int` |
| `[]any` | `Int` |

Example Converters
- `Int()`
- `IsMatch(field, ".*")`
- `Split(field, ",")[1]`


#### Math Expressions
### Math Expressions

Math Expressions represent arithmetic calculations. They support `+`, `-`, `*`, and `/`, along with `()` for grouping.

Math Expressions currently only support `int64` and `float64`.
Math Expressions support `Paths` and `Invocations` that return supported types.
Math Expressions support `Paths` and `Editors` that return supported types.
Note that `*` and `/` take precedence over `+` and `-`.
Operations that share the same level of precedence will be executed in the order that they appear in the Math Expression.
Math Expressions can be grouped with parentheses to override evaluation precedence.
Expand All @@ -169,7 +178,7 @@ It is up to the function using the Math Expression to determine what to do with
Division by zero is gracefully handled with an error, but other arithmetic operations that would result in a panic will still result in a panic.
Division of integers results in an integer and follows Go's rules for division of integers.

Since Math Expressions support `Path` and `Invocation`, they are evaluated during data processing.
Since Math Expressions support `Path`s and `Converter`s as input, they are evaluated during data processing.
__As a result, in order for a function to be able to accept an Math Expressions as a parameter it must use a `Getter`.__

Example Math Expressions
Expand All @@ -180,7 +189,7 @@ Example Math Expressions

### Boolean Expressions

Boolean Expressions allow a decision to be made about whether an Invocation should be called. Boolean Expressions are optional. When used, the parsed statement will include a `Condition`, which can be used to evaluate the result of the statement's Boolean Expression. Boolean Expressions always evaluate to a boolean value (true or false).
Boolean Expressions allow a decision to be made about whether an Editor should be called. Boolean Expressions are optional. When used, the parsed statement will include a `Condition`, which can be used to evaluate the result of the statement's Boolean Expression. Boolean Expressions always evaluate to a boolean value (true or false).

Boolean Expressions consist of the literal string `where` followed by one or more Booleans (see below).
Booleans can be joined with the literal strings `and` and `or`.
Expand Down Expand Up @@ -211,7 +220,7 @@ Booleans can be negated with the `not` keyword such as
- `not name == "foo"`
- `not (IsMatch(name, "http_.*") and kind > 0)`

### Comparison Rules
## Comparison Rules

The table below describes what happens when two Values are compared. Value types are provided by the user of OTTL. All of the value types supported by OTTL are listed in this table.

Expand Down Expand Up @@ -246,10 +255,11 @@ Access to signal telemetry is provided to OTTL functions through a `TransformCon
### Getters and Setters

Getters allow for reading the following types of data. See the respective section of each Value type for how they are interpreted.

- [Paths](#paths).
- [Enums](#enums).
- [Literals](#literals).
- [Invocations](#invocations).
- [Converters](#converters).

It is possible to update the Value in a telemetry field using a Setter. For read and write access, the `GetSetter` interface extends both interfaces.

Expand Down
2 changes: 1 addition & 1 deletion pkg/ottl/expression.go
Original file line number Diff line number Diff line change
Expand Up @@ -352,7 +352,7 @@ func (p *Parser[K]) newGetter(val value) (Getter[K], error) {
}

func (p *Parser[K]) newGetterFromConverter(c converter) (Getter[K], error) {
call, err := p.newFunctionCall(invocation(c))
call, err := p.newFunctionCall(editor(c))
if err != nil {
return nil, err
}
Expand Down
2 changes: 1 addition & 1 deletion pkg/ottl/factory.go
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ type FunctionContext struct {
}

// Factory defines an OTTL function factory that will generate an OTTL
// function to be called based on an invocation within a statement.
// function to be called within a statement.
type Factory[K any] interface {
// Name is the canonical name to be used by the user when invocating
// the function generated by this Factory.
Expand Down
26 changes: 13 additions & 13 deletions pkg/ottl/functions.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,10 @@ type EnumParser func(*EnumSymbol) (*Enum, error)

type Enum int64

func (p *Parser[K]) newFunctionCall(inv invocation) (Expr[K], error) {
f, ok := p.functions[inv.Function]
func (p *Parser[K]) newFunctionCall(ed editor) (Expr[K], error) {
f, ok := p.functions[ed.Function]
if !ok {
return Expr[K]{}, fmt.Errorf("undefined function %v", inv.Function)
return Expr[K]{}, fmt.Errorf("undefined function %v", ed.Function)
}
args := f.CreateDefaultArguments()

Expand All @@ -43,12 +43,12 @@ func (p *Parser[K]) newFunctionCall(inv invocation) (Expr[K], error) {
// settability requirements. Non-pointer values are not
// modifiable through reflection.
if reflect.TypeOf(args).Kind() != reflect.Pointer {
return Expr[K]{}, fmt.Errorf("factory for %s must return a pointer to an Arguments value in its CreateDefaultArguments method", inv.Function)
return Expr[K]{}, fmt.Errorf("factory for %s must return a pointer to an Arguments value in its CreateDefaultArguments method", ed.Function)
}

err := p.buildArgs(inv, reflect.ValueOf(args).Elem())
err := p.buildArgs(ed, reflect.ValueOf(args).Elem())
if err != nil {
return Expr[K]{}, fmt.Errorf("error while parsing arguments for call to '%v': %w", inv.Function, err)
return Expr[K]{}, fmt.Errorf("error while parsing arguments for call to '%v': %w", ed.Function, err)
}
}

Expand All @@ -60,9 +60,9 @@ func (p *Parser[K]) newFunctionCall(inv invocation) (Expr[K], error) {
return Expr[K]{exprFunc: fn}, err
}

func (p *Parser[K]) buildArgs(inv invocation, argsVal reflect.Value) error {
if len(inv.Arguments) != argsVal.NumField() {
return fmt.Errorf("incorrect number of arguments. Expected: %d Received: %d", argsVal.NumField(), len(inv.Arguments))
func (p *Parser[K]) buildArgs(ed editor, argsVal reflect.Value) error {
if len(ed.Arguments) != argsVal.NumField() {
return fmt.Errorf("incorrect number of arguments. Expected: %d Received: %d", argsVal.NumField(), len(ed.Arguments))
}

argsType := argsVal.Type()
Expand All @@ -83,11 +83,11 @@ func (p *Parser[K]) buildArgs(inv invocation, argsVal reflect.Value) error {
return fmt.Errorf("ottlarg struct tag on field '%s' is not a valid integer: %w", argsType.Field(i).Name, err)
}

if argNum < 0 || argNum >= len(inv.Arguments) {
return fmt.Errorf("ottlarg struct tag on field '%s' has value %d, but must be between 0 and %d", argsType.Field(i).Name, argNum, len(inv.Arguments))
if argNum < 0 || argNum >= len(ed.Arguments) {
return fmt.Errorf("ottlarg struct tag on field '%s' has value %d, but must be between 0 and %d", argsType.Field(i).Name, argNum, len(ed.Arguments))
}

argVal := inv.Arguments[argNum]
argVal := ed.Arguments[argNum]

var val any
if fieldType.Kind() == reflect.Slice {
Expand Down Expand Up @@ -172,7 +172,7 @@ func (p *Parser[K]) buildSliceArg(argVal value, argType reflect.Type) (any, erro
}
}

// Handle interfaces that can be passed as arguments to OTTL function invocations.
// Handle interfaces that can be passed as arguments to OTTL functions.
func (p *Parser[K]) buildArg(argVal value, argType reflect.Type) (any, error) {
name := argType.Name()
switch {
Expand Down
Loading

0 comments on commit c8da52b

Please sign in to comment.