-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Copy contrib's storage interface to core (#3425)
As discussed during the SIG, we want to move storage extension to core, starting with the interface (this PR) so persistent buffer implementation (#2285) could use it **Link to tracking Issue:** #3424 **Testing:** Just the interface, no tests **Documentation:** README.md with API cc @djaglowski @tigrannajaryan
- Loading branch information
Showing
5 changed files
with
211 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
include ../../Makefile.Common |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,38 @@ | ||
# Storage | ||
|
||
**Status: under development; This is currently just the interface** | ||
|
||
A storage extension persists state beyond the collector process. Other components can request a storage client from the storage extension and use it to manage state. | ||
|
||
The `storage.Extension` interface extends `component.Extension` by adding the following method: | ||
``` | ||
GetClient(context.Context, component.Kind, config.ComponentID, string) (Client, error) | ||
``` | ||
|
||
The `storage.Client` interface contains the following methods: | ||
``` | ||
Get(context.Context, string) ([]byte, error) | ||
Set(context.Context, string, []byte) error | ||
Delete(context.Context, string) error | ||
Close(context.Context) error | ||
``` | ||
|
||
It is possible to execute several operations in a single transaction via `Batch`. The method takes a collection of | ||
`Operation` arguments (each of which contains `Key`, `Value` and `Type` properties): | ||
``` | ||
Batch(context.Context, ...Operation) error | ||
``` | ||
|
||
The elements itself can be created using: | ||
|
||
``` | ||
SetOperation(string, []byte) Operation | ||
GetOperation(string) Operation | ||
DeleteOperation(string) Operation | ||
``` | ||
|
||
Get operation results are stored in-place into the given Operation and can be retrieved using its `Value` property. | ||
|
||
Note: All methods should return error only if a problem occurred. (For example, if a file is no longer accessible, or if a remote service is unavailable.) | ||
|
||
Note: It is the responsibility of each component to `Close` a storage client that it has requested. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
// Copyright The OpenTelemetry Authors | ||
// | ||
// Licensed under the Apache License, Version 2.0 (the "License"); | ||
// you may not use this file except in compliance with the License. | ||
// You may obtain a copy of the License at | ||
// | ||
// http://www.apache.org/licenses/LICENSE-2.0 | ||
// | ||
// Unless required by applicable law or agreed to in writing, software | ||
// distributed under the License is distributed on an "AS IS" BASIS, | ||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
// See the License for the specific language governing permissions and | ||
// limitations under the License. | ||
|
||
// Package stanzareceiver implements a receiver that can be used by the | ||
// Opentelemetry collector to receive logs using the stanza log agent | ||
package storage |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,51 @@ | ||
// Copyright The OpenTelemetry Authors | ||
// | ||
// Licensed under the Apache License, Version 2.0 (the "License"); | ||
// you may not use this file except in compliance with the License. | ||
// You may obtain a copy of the License at | ||
// | ||
// http://www.apache.org/licenses/LICENSE-2.0 | ||
// | ||
// Unless required by applicable law or agreed to in writing, software | ||
// distributed under the License is distributed on an "AS IS" BASIS, | ||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
// See the License for the specific language governing permissions and | ||
// limitations under the License. | ||
|
||
package storage | ||
|
||
import "context" | ||
|
||
type nopClient struct{} | ||
|
||
var nopClientInstance Client = &nopClient{} | ||
|
||
// NewNopClient returns a nop client | ||
func NewNopClient() Client { | ||
return nopClientInstance | ||
} | ||
|
||
// Get does nothing, and returns nil, nil | ||
func (c nopClient) Get(context.Context, string) ([]byte, error) { | ||
return nil, nil // no result, but no problem | ||
} | ||
|
||
// Set does nothing and returns nil | ||
func (c nopClient) Set(context.Context, string, []byte) error { | ||
return nil // no problem | ||
} | ||
|
||
// Delete does nothing and returns nil | ||
func (c nopClient) Delete(context.Context, string) error { | ||
return nil // no problem | ||
} | ||
|
||
// Close does nothing and returns nil | ||
func (c nopClient) Close(context.Context) error { | ||
return nil | ||
} | ||
|
||
// Batch does nothing, and returns nil, nil | ||
func (c nopClient) Batch(context.Context, ...Operation) error { | ||
return nil // no result, but no problem | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,104 @@ | ||
// Copyright The OpenTelemetry Authors | ||
// | ||
// Licensed under the Apache License, Version 2.0 (the "License"); | ||
// you may not use this file except in compliance with the License. | ||
// You may obtain a copy of the License at | ||
// | ||
// http://www.apache.org/licenses/LICENSE-2.0 | ||
// | ||
// Unless required by applicable law or agreed to in writing, software | ||
// distributed under the License is distributed on an "AS IS" BASIS, | ||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
// See the License for the specific language governing permissions and | ||
// limitations under the License. | ||
|
||
package storage | ||
|
||
import ( | ||
"context" | ||
|
||
"go.opentelemetry.io/collector/component" | ||
"go.opentelemetry.io/collector/config" | ||
) | ||
|
||
// Extension is the interface that storage extensions must implement | ||
type Extension interface { | ||
component.Extension | ||
|
||
// GetClient will create a client for use by the specified component. | ||
// Each component can have multiple storages (e.g. one for each signal), | ||
// which can be identified using storageName parameter. | ||
// The component can use the client to manage state | ||
GetClient(ctx context.Context, kind component.Kind, id config.ComponentID, storageName string) (Client, error) | ||
} | ||
|
||
// Client is the interface that storage clients must implement | ||
// All methods should return error only if a problem occurred. | ||
// This mirrors the behavior of a golang map: | ||
// - Set doesn't error if a key already exists - it just overwrites the value. | ||
// - Get doesn't error if a key is not found - it just returns nil. | ||
// - Delete doesn't error if the key doesn't exist - it just no-ops. | ||
// Similarly: | ||
// - Batch doesn't error if any of the above happens for either retrieved or updated keys | ||
// This also provides a way to differentiate data operations | ||
// [overwrite | not-found | no-op] from "real" problems | ||
type Client interface { | ||
|
||
// Get will retrieve data from storage that corresponds to the | ||
// specified key. It should return (nil, nil) if not found | ||
Get(ctx context.Context, key string) ([]byte, error) | ||
|
||
// Set will store data. The data can be retrieved by the same | ||
// component after a process restart, using the same key | ||
Set(ctx context.Context, key string, value []byte) error | ||
|
||
// Delete will delete data associated with the specified key | ||
Delete(ctx context.Context, key string) error | ||
|
||
// Batch handles specified operations in batch. Get operation results are put in-place | ||
Batch(ctx context.Context, ops ...Operation) error | ||
|
||
// Close will release any resources held by the client | ||
Close(ctx context.Context) error | ||
} | ||
|
||
type opType int | ||
|
||
const ( | ||
Get opType = iota | ||
Set | ||
Delete | ||
) | ||
|
||
type operation struct { | ||
// Key specifies key which is going to be get/set/deleted | ||
Key string | ||
// Value specifies value that is going to be set or holds result of get operation | ||
Value []byte | ||
// Type describes the operation type | ||
Type opType | ||
} | ||
|
||
type Operation *operation | ||
|
||
func SetOperation(key string, value []byte) Operation { | ||
return &operation{ | ||
Key: key, | ||
Value: value, | ||
Type: Set, | ||
} | ||
} | ||
|
||
func GetOperation(key string) Operation { | ||
return &operation{ | ||
Key: key, | ||
Type: Get, | ||
} | ||
} | ||
|
||
func DeleteOperation(key string) Operation { | ||
return &operation{ | ||
Key: key, | ||
Type: Delete, | ||
} | ||
} |