Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve docs for shape and a la carte shaping functions #5172

Merged
merged 3 commits into from
Jul 8, 2024
Merged

Improve docs for shape and a la carte shaping functions #5172

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
0212e13
Improve docs for shape and a la carte shaping functions
philrz Jul 7, 2024
f12bfde
Add missing .md that caused broken link
philrz Jul 7, 2024
7b9a253
PR feedback: Clarify where fill puts added fields
philrz Jul 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion docs/language/functions/cast.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ For complex types, the cast function visits each leaf value in `val` and
casts that value to the corresponding type in `t`.
When a complex value has multiple levels of nesting,
casting is applied recursively down the tree. For example, cast is recursively
applied to each element in array of records and recursively applied to each record.
applied to each element in an array of records and recursively applied to each record.

If `val` is a record (or if any of its nested value is a record):
* absent fields are ignored and omitted from the result,
Expand All @@ -41,6 +41,12 @@ to match the output type's order but rather just modifies the leaf values.
If a cast fails, an error is returned when casting to primitive types
and the input value is returned when casting to complex types.

:::tip
Many users seeking to `cast` record values prefer to use the
[`shape` function](./shape.md) which applies the `cast`, [`fill`](./fill.md),
and [`order`](./order.md) functions simultaneously.
:::

### Examples

_Cast primitives to type `ip`_
Expand Down
10 changes: 10 additions & 0 deletions docs/language/functions/fill.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,18 @@ present in the output type `t` but not in the input.
Filled fields are added with a `null` value. Filling is useful when
you want to be sure that all fields in a schema are present in a record.

The order of newly added fields relative to fields already present is
undefined. If maintaining relative order is important, consider applying the
[`order`](./order.md) or [`shape`](./shape.md) function.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fields are added in the order they appear in the input record.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For reference, fill essentially happens here:

zed/runtime/sam/expr/shaper.go

Lines 274 to 275 in f497079

} else if fill {
fields = append(fields, outField)

(That's inside a loop that, for fill, iterates over the fields of the input record.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After seeing @nwt's comments here we had a quick offline chat and agreed on some text to clarify where fill adds the fields rather than the "undefined" I had before. I've pushed a commit with that change.


If `val` is not a record, it is returned unmodified.

:::tip
Many users seeking the functionality of `fill` prefer to use the
[`shape` function](./shape.md) which applies the `fill`, [`cast`](./cast.md),
and [`order`](./order.md) functions simultaneously on a record.
:::

### Examples

_Fill a record_
Expand Down
5 changes: 5 additions & 0 deletions docs/language/functions/order.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,11 @@ the empty record type, i.e.,
```
order(val, <{}>)
```
:::tip
Many users seeking the functionality of `order` prefer to use the
[`shape` function](./shape.md) which applies the `order`, [`cast`](./cast.md),
and [`fill`](./fill.md) functions simultaneously on a record.
:::

:::tip Note
[Record expressions](../expressions.md#record-expressions) can also be used to
Expand Down
10 changes: 5 additions & 5 deletions docs/language/functions/shape.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,12 @@ shape(val: any, t: type) -> any
### Description

The _shape_ function applies the
[cast](cast.md),
[fill](fill.md), and
[order](order.md) functions to its input to provide an
overall data shaping operation.
[`cast`](cast.md),
[`fill`](fill.md), and
[`order`](order.md) functions to its input to provide an
overall [data shaping](../shaping.md) operation.

Note that _shape_ does not perform a _crop_ function so
Note that `shape` does not perform a [`crop` function](./crop.md) so
extra fields in the input are propagated to the output.

### Examples
Expand Down