Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sdk/log: Add design doc #4954

Merged
merged 36 commits into from
Mar 13, 2024
Merged
Changes from 10 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
2a7b54c
Create Logs SDK design doc
pellared Feb 21, 2024
7268786
Update prototype PR hyperlink
pellared Feb 21, 2024
45e6efe
Fix typo
pellared Feb 21, 2024
6eff65a
Update DESIGN.md
pellared Mar 6, 2024
bc90093
Update DESIGN.md
pellared Mar 6, 2024
0cc45fa
Merge branch 'main' into sdk-log-design
pellared Mar 6, 2024
af0e848
Fix typos
pellared Mar 6, 2024
8b4a204
Document design choice
pellared Mar 6, 2024
f1ec9aa
Update DESIGN.md
pellared Mar 6, 2024
2e54ea5
Update limits
pellared Mar 6, 2024
6a85c02
Apply suggestions from code review
pellared Mar 6, 2024
3cb2da4
Update DESIGN.md
pellared Mar 7, 2024
0a474d2
Update DESIGN.md
pellared Mar 7, 2024
d657c99
Update DESIGN.md
pellared Mar 7, 2024
b24439c
Update DESIGN.md
pellared Mar 7, 2024
93826b7
Update DESIGN.md
pellared Mar 7, 2024
7c8e4c5
Update sdk/log/DESIGN.md
pellared Mar 7, 2024
8ba8991
Update DESIGN.md
pellared Mar 8, 2024
c4b798b
Add Record.AddAttributes
pellared Mar 8, 2024
3a81068
Update DESIGN.md
pellared Mar 8, 2024
7d68029
Update DESIGN.md
pellared Mar 8, 2024
f480907
Add Processor
pellared Mar 11, 2024
b1c39b5
Fix typo
pellared Mar 11, 2024
42eedc1
Rejected proposal: Embedd log.Record
pellared Mar 11, 2024
7dc3a89
Merge branch 'main' into sdk-log-design
pellared Mar 11, 2024
1bac331
Update DESIGN.md
pellared Mar 11, 2024
ab9e3ea
Update DESIGN.md
pellared Mar 11, 2024
86e6392
Update sdk/log/DESIGN.md
pellared Mar 11, 2024
ada07c9
Merge branch 'main' into sdk-log-design
pellared Mar 11, 2024
0d69655
Apply suggestions from code review
pellared Mar 12, 2024
2c8a57c
Update DESIGN.md
pellared Mar 12, 2024
716e780
Update DESIGN.md
pellared Mar 12, 2024
796adfe
Merge branch 'main' into sdk-log-design
pellared Mar 12, 2024
5c45fc0
Merge branch 'main' into sdk-log-design
pellared Mar 12, 2024
d96dbbb
Add SetTraceID SetSpanID SetTraceFlags to Record
pellared Mar 12, 2024
07ef850
Merge branch 'main' into sdk-log-design
pellared Mar 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
170 changes: 170 additions & 0 deletions sdk/log/DESIGN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,170 @@
# Logs SDK

## Abstract

`go.opentelemetry.io/otel/sdk/log` provides Logs SDK compliant with the
[specification](https://opentelemetry.io/docs/specs/otel/logs/sdk/).

The main and recommended use case is to configure the SDK to use an OTLP
exporter with a batch processor.[^1] Therefore, the design aims to be
high-performant in this scenario.

The prototype was created in
[#4955](https://github.com/open-telemetry/opentelemetry-go/pull/4955).

## Modules structure

The SDK is published as a single `go.opentelemetry.io/otel/sdk/log` Go module.

The exporters are going to be published as following Go modules:

- `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc`
- `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp`
- `go.opentelemetry.io/otel/exporters/stdout/stdoutlog`

## LoggerProvider

The [LoggerProvider](https://opentelemetry.io/docs/specs/otel/logs/sdk/#loggerprovider)
is defined as follows:

```go
type LoggerProvider struct {
embedded.LoggerProvider
}

func NewLoggerProvider(...Option) *LoggerProvider

func (*LoggerProvider) Logger(name string, options ...log.LoggerOption) log.Logger
pellared marked this conversation as resolved.
Show resolved Hide resolved

type Option interface { /* ... */ }

func WithResource(*resource.Resource) Option
```

## LogRecord limits

The [LogRecord limits](https://opentelemetry.io/docs/specs/otel/logs/sdk/#logrecord-limits)
are defined as follows:

```go
func WithAttributeCountLimit(limit int) Option

func WithAttributeValueLengthLimit(limit int) Option
pellared marked this conversation as resolved.
Show resolved Hide resolved
```

### LogRecordProcessor and LogRecordExporter

Both [LogRecordProcessor](https://opentelemetry.io/docs/specs/otel/logs/sdk/#logrecordprocessor)
and [LogRecordExporter](https://opentelemetry.io/docs/specs/otel/logs/sdk/#logrecordexporter)
are defined via an `Exporter` interface:

```go
func WithExporter(Exporter) Option {
return nil
}

type Exporter interface {
Export(ctx context.Context, records []*Record) error
pellared marked this conversation as resolved.
Show resolved Hide resolved
Shutdown(ctx context.Context) error
ForceFlush(ctx context.Context) error
}
```

The `Record` struct represents the [ReadWriteLogRecord](https://opentelemetry.io/docs/specs/otel/logs/sdk/#readwritelogrecord).

```go
type Record struct { /* ... */ }
pellared marked this conversation as resolved.
Show resolved Hide resolved
MrAlias marked this conversation as resolved.
Show resolved Hide resolved

func (r *Record) Timestamp()

func (r *Record) SetTimestamp(t time.Time)

func (r *Record) ObservedTimestamp() time.Time

func (r *Record) SetObservedTimestamp(t time.Time)

func (r *Record) Severity() log.Severity

func (r *Record) SetSeverity(level log.Severity)

func (r *Record) SeverityText() string

func (r *Record) SetSeverityText(text string)

func (r *Record) Body() log.Value

func (r *Record) SetBody(v log.Value)

func (r *Record) WalkAttributes(f func(log.KeyValue) bool)

// SetAttributes sets and overrides the attributes of the log record.
func (r *Record) SetAttributes(attrs ...log.KeyValue)

func (r *Record) TraceID() trace.TraceID

func (r *Record) SpanID() trace.SpanID

func (r *Record) TraceFlags() trace.TraceFlags

func (r *Record) Resource() resource.Resource

func (r *Record) InstrumentationScope() instrumentation.Scope

func (r *Record) AttributeValueLengthLimit() int

func (r *Record) AttributeCountLimit() int
```

The user can implement a custom [LogRecordProcessor](https://opentelemetry.io/docs/specs/otel/logs/sdk/#logrecordprocessor)
by implementing a `Exporter` decorator.

This is similar to the design of HTTP server middleware
which is a wrapper of `http.Handler`.[^2]

[Simple processor](https://opentelemetry.io/docs/specs/otel/logs/sdk/#simple-processor)
is a achieved by simply passing a bare-exporter.
pellared marked this conversation as resolved.
Show resolved Hide resolved

[Batching processor](https://opentelemetry.io/docs/specs/otel/logs/sdk/#batching-processor)
pellared marked this conversation as resolved.
Show resolved Hide resolved
is a achieved by wrapping an expoter with `Batcher`:
pellared marked this conversation as resolved.
Show resolved Hide resolved

```go
type Batcher struct { /* ... */ }

func NewBatchingExporter(exporter Exporter, opts ...BatchingOption) *Batcher

func (b *Batcher) Export(ctx context.Context, records []*Record)

func (b *Batcher) Shutdown(ctx context.Context) error

func (b *Batcher) ForceFlush(ctx context.Context) error

type BatchingOption interface { /* ... */ }

func WithInterval(d time.Duration) BatchingOption

func WithTimeout(d time.Duration) BatchingOption

func WithQueueSize(max int) BatchingOption

func WithBatchSize(max int) BatchingOption
pellared marked this conversation as resolved.
Show resolved Hide resolved
```

## Benchmarking

The benchmarks are supposed to test end-to-end scenarios.

However, in order avoid I/O that could affect the stability of the results,
the benchmarks are using an stdout exporter using `io.Discard`.

The benchmark results can be found in [the prototype](https://github.com/open-telemetry/opentelemetry-go/pull/4955).

## Rejected alternatives

## Open issues

The Logs SDK NOT be released as stable before all issues below are closed:

- [Fix what can be modified via ReadWriteLogRecord](https://github.com/open-telemetry/opentelemetry-specification/pull/3907)

[^1]: [OpenTelemetry Logging](https://opentelemetry.io/docs/specs/otel/logs)
[^2]: [Middleware - Go Web Examples](https://gowebexamples.com/basic-middleware/)