-
Notifications
You must be signed in to change notification settings - Fork 2.1k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
service/dynamodb/expression: Add expression building utility for Dyna…
…moDB (#1527) Adds a new package, expression, to the SDK providing builder utilities to create DynamoDB expressions safely taking advantage of type safety. Adds several new builders to construct specific DynamoDB Expression types. These builders can be combined together to retrieve the expression name, and value aliases, along with the formatted expression string for the given expression type. Adds a runnable example under example/service/dynamodb/expression
- Loading branch information
Showing
19 changed files
with
8,890 additions
and
83 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,102 @@ | ||
# Example | ||
|
||
`scan` is an example how to use Amazon DynamoDB's `expression` package to fill | ||
the member fields of Amazon DynamoDB's Operation input types. | ||
|
||
## Representing DynamoDB Expressions | ||
|
||
In the example, the variable `filt` represents a `FilterExpression`. Note that | ||
DynamoDB item attributes are represented using the function `Name()` and | ||
DynamoDB item values are similarly represented using the function `Value()`. In | ||
this context, the string `"Artist"` represents the name of the item attribute | ||
that we want to evaluate and the string `"No One You Know"` represents the value | ||
we want to evaluate the item attribute against. The relationship between the two | ||
[operands](http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.OperatorsAndFunctions.html#Expressions.OperatorsAndFunctions.Syntax) | ||
are specified using the method `Equal()`. | ||
|
||
Similarly, the variable `proj` represents a `ProjectionExpression`. The list of | ||
item attribute names comprising the `ProjectionExpression` are specified as | ||
arguments to the function `NamesList()`. The `expression` package utilizes the | ||
type safety of Go and if an item value were to be used as an argument to the | ||
function `NamesList()`, a compile time error is returned. The pattern of | ||
representing DynamoDB Expressions by indicating relationships between `operands` | ||
with functions is consistent throughout the whole `expression` package. | ||
|
||
```go | ||
filt := expression.Name("Artist").Equal(expression.Value("No One You Know")) | ||
// let :a be an ExpressionAttributeValue representing the string "No One You Know" | ||
// equivalent FilterExpression: "Artist = :a" | ||
|
||
proj := expression.NamesList(expression.Name("SongTitle"), expression.Name("AlbumTitle")) | ||
// equivalent ProjectionExpression: "SongTitle, AlbumTitle" | ||
``` | ||
|
||
## Creating an `Expression` | ||
|
||
In the example, the variable `expr` is an instance of an `Expression` type. An | ||
`Expression` is built using a builder pattern. First, a new `Builder` is | ||
initialized by the `NewBuilder()` function. Then, types representing DynamoDB | ||
Expressions are added to the `Builder` by methods `WithFilter()` and | ||
`WithProjection()`. The `Build()` method returns an instance of an `Expression` | ||
and an error. The error will be either an `InvalidParameterError` or an | ||
`UnsetParameterError`. | ||
|
||
```go | ||
filt := expression.Name("Artist").Equal(expression.Value("No One You Know")) | ||
proj := expression.NamesList(expression.Name("SongTitle"), expression.Name("AlbumTitle")) | ||
|
||
expr, err := expression.NewBuilder().WithFilter(filt).WithProjection(proj).Build() | ||
if err != nil { | ||
fmt.Println(err) | ||
} | ||
``` | ||
|
||
## Filling in the fields of a DynamoDB `Scan` API | ||
|
||
In the example, the getter methods of the `Expression` type is used to get the | ||
formatted DynamoDB Expression strings. The `ExpressionAttributeNames` and | ||
`ExpressionAttributeValues` member field of the DynamoDB API must always be | ||
assigned when using an `Expression` since all item attribute names and values | ||
are aliased. That means that if the `ExpressionAttributeNames` and | ||
`ExpressionAttributeValues` member is not assigned with the corresponding | ||
`Names()` and `Values()` methods, the DynamoDB operation will run into a logic | ||
error. | ||
|
||
```go | ||
filt := expression.Name("Artist").Equal(expression.Value("No One You Know")) | ||
proj := expression.NamesList(expression.Name("SongTitle"), expression.Name("AlbumTitle")) | ||
expr, err := expression.NewBuilder().WithFilter(filt).WithProjection(proj).Build() | ||
if err != nil { | ||
fmt.Println(err) | ||
} | ||
|
||
input := &dynamodb.ScanInput{ | ||
ExpressionAttributeNames: expr.Names(), | ||
ExpressionAttributeValues: expr.Values(), | ||
FilterExpression: expr.Filter(), | ||
ProjectionExpression: expr.Projection(), | ||
TableName: aws.String("Music"), | ||
} | ||
``` | ||
|
||
## Usage | ||
|
||
`go run -tags example scan.go -table "<table_name>" -region "<optional_region>"` | ||
|
||
## Output | ||
|
||
``` | ||
{ | ||
Count: #SomeNumber, | ||
Items: [{ | ||
AlbumTitle: { | ||
#SomeAlbumTitle | ||
}, | ||
SongTitle: { | ||
#SomeSongTitle | ||
} | ||
}], | ||
... | ||
ScannedCount: #SomeNumber, | ||
} | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,88 @@ | ||
// +build example | ||
|
||
package main | ||
|
||
import ( | ||
"flag" | ||
"fmt" | ||
"os" | ||
|
||
"github.com/aws/aws-sdk-go/aws" | ||
"github.com/aws/aws-sdk-go/aws/session" | ||
"github.com/aws/aws-sdk-go/service/dynamodb" | ||
"github.com/aws/aws-sdk-go/service/dynamodb/expression" | ||
) | ||
|
||
func exitWithError(err error) { | ||
fmt.Fprintln(os.Stderr, err) | ||
os.Exit(1) | ||
} | ||
|
||
func main() { | ||
cfg := Config{} | ||
if err := cfg.Load(); err != nil { | ||
exitWithError(fmt.Errorf("failed to load config, %v", err)) | ||
} | ||
|
||
// Create the config specifying the Region for the DynamoDB table. | ||
// If Config.Region is not set the region must come from the shared | ||
// config or AWS_REGION environment variable. | ||
awscfg := &aws.Config{} | ||
if len(cfg.Region) > 0 { | ||
awscfg.WithRegion(cfg.Region) | ||
} | ||
|
||
// Create the session that the DynamoDB service will use. | ||
sess := session.Must(session.NewSession(awscfg)) | ||
|
||
// Create the DynamoDB service client to make the query request with. | ||
svc := dynamodb.New(sess) | ||
|
||
// Create the Expression to fill the input struct with. | ||
filt := expression.Name("Artist").Equal(expression.Value("No One You Know")) | ||
proj := expression.NamesList(expression.Name("SongTitle"), expression.Name("AlbumTitle")) | ||
expr, err := expression.NewBuilder().WithFilter(filt).WithProjection(proj).Build() | ||
if err != nil { | ||
exitWithError(fmt.Errorf("failed to create the Expression, %v", err)) | ||
} | ||
|
||
// Build the query input parameters | ||
params := &dynamodb.ScanInput{ | ||
ExpressionAttributeNames: expr.Names(), | ||
ExpressionAttributeValues: expr.Values(), | ||
FilterExpression: expr.Filter(), | ||
ProjectionExpression: expr.Projection(), | ||
TableName: aws.String(cfg.Table), | ||
} | ||
if cfg.Limit > 0 { | ||
params.Limit = aws.Int64(cfg.Limit) | ||
} | ||
|
||
// Make the DynamoDB Query API call | ||
result, err := svc.Scan(params) | ||
if err != nil { | ||
exitWithError(fmt.Errorf("failed to make Query API call, %v", err)) | ||
} | ||
|
||
fmt.Println(result) | ||
} | ||
|
||
type Config struct { | ||
Table string // required | ||
Region string // optional | ||
Limit int64 // optional | ||
} | ||
|
||
func (c *Config) Load() error { | ||
flag.Int64Var(&c.Limit, "limit", 0, "Limit is the max items to be returned, 0 is no limit") | ||
flag.StringVar(&c.Table, "table", "", "Table to Query on") | ||
flag.StringVar(&c.Region, "region", "", "AWS Region the table is in") | ||
flag.Parse() | ||
|
||
if len(c.Table) == 0 { | ||
flag.PrintDefaults() | ||
return fmt.Errorf("table name is required.") | ||
} | ||
|
||
return nil | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,84 +1,27 @@ | ||
// AttributeValue Marshaling and Unmarshaling Helpers | ||
// | ||
// Utility helpers to marshal and unmarshal AttributeValue to and | ||
// from Go types can be found in the dynamodbattribute sub package. This package | ||
// provides has specialized functions for the common ways of working with | ||
// AttributeValues. Such as map[string]*AttributeValue, []*AttributeValue, and | ||
// directly with *AttributeValue. This is helpful for marshaling Go types for API | ||
// operations such as PutItem, and unmarshaling Query and Scan APIs' responses. | ||
// | ||
// See the dynamodbattribute package documentation for more information. | ||
// https://docs.aws.amazon.com/sdk-for-go/api/service/dynamodb/dynamodbattribute/ | ||
// | ||
// AttributeValue Marshaling | ||
// | ||
// To marshal a Go type to an AttributeValue you can use the Marshal | ||
// functions in the dynamodbattribute package. There are specialized versions | ||
// of these functions for collections of AttributeValue, such as maps and lists. | ||
// | ||
// The following example uses MarshalMap to convert the Record Go type to a | ||
// dynamodb.AttributeValue type and use the value to make a PutItem API request. | ||
// | ||
// type Record struct { | ||
// ID string | ||
// URLs []string | ||
// } | ||
// | ||
// //... | ||
// | ||
// r := Record{ | ||
// ID: "ABC123", | ||
// URLs: []string{ | ||
// "https://example.com/first/link", | ||
// "https://example.com/second/url", | ||
// }, | ||
// } | ||
// av, err := dynamodbattribute.MarshalMap(r) | ||
// if err != nil { | ||
// panic(fmt.Sprintf("failed to DynamoDB marshal Record, %v", err)) | ||
// } | ||
// | ||
// _, err = svc.PutItem(&dynamodb.PutItemInput{ | ||
// TableName: aws.String(myTableName), | ||
// Item: av, | ||
// }) | ||
// if err != nil { | ||
// panic(fmt.Sprintf("failed to put Record to DynamoDB, %v", err)) | ||
// } | ||
// | ||
// AttributeValue Unmarshaling | ||
// | ||
// To unmarshal a dynamodb.AttributeValue to a Go type you can use the Unmarshal | ||
// functions in the dynamodbattribute package. There are specialized versions | ||
// of these functions for collections of AttributeValue, such as maps and lists. | ||
// | ||
// The following example will unmarshal the DynamoDB's Scan API operation. The | ||
// Items returned by the operation will be unmarshaled into the slice of Records | ||
// Go type. | ||
// | ||
// type Record struct { | ||
// ID string | ||
// URLs []string | ||
// } | ||
// | ||
// //... | ||
// | ||
// var records []Record | ||
// | ||
// // Use the ScanPages method to perform the scan with pagination. Use | ||
// // just Scan method to make the API call without pagination. | ||
// err := svc.ScanPages(&dynamodb.ScanInput{ | ||
// TableName: aws.String(myTableName), | ||
// }, func(page *dynamodb.ScanOutput, last bool) bool { | ||
// recs := []Record{} | ||
// | ||
// err := dynamodbattribute.UnmarshalListOfMaps(page.Items, &recs) | ||
// if err != nil { | ||
// panic(fmt.Sprintf("failed to unmarshal Dynamodb Scan Items, %v", err)) | ||
// } | ||
// | ||
// records = append(records, recs...) | ||
// | ||
// return true // keep paging | ||
// }) | ||
/* | ||
AttributeValue Marshaling and Unmarshaling Helpers | ||
Utility helpers to marshal and unmarshal AttributeValue to and | ||
from Go types can be found in the dynamodbattribute sub package. This package | ||
provides has specialized functions for the common ways of working with | ||
AttributeValues. Such as map[string]*AttributeValue, []*AttributeValue, and | ||
directly with *AttributeValue. This is helpful for marshaling Go types for API | ||
operations such as PutItem, and unmarshaling Query and Scan APIs' responses. | ||
See the dynamodbattribute package documentation for more information. | ||
https://docs.aws.amazon.com/sdk-for-go/api/service/dynamodb/dynamodbattribute/ | ||
Expression Builders | ||
The expression package provides utility types and functions to build DynamoDB | ||
expression for type safe construction of API ExpressionAttributeNames, and | ||
ExpressionAttribute Values. | ||
The package represents the various DynamoDB Expressions as structs named | ||
accordingly. For example, ConditionBuilder represents a DynamoDB Condition | ||
Expression, an UpdateBuilder represents a DynamoDB Update Expression, and so on. | ||
See the expression package documentation for more information. | ||
https://docs.aws.amazon.com/sdk-for-go/api/service/dynamodb/expression/ | ||
*/ | ||
package dynamodb |
Oops, something went wrong.