Panic when attempting to output complex values as text/table

[philrz]

Repro is with Zed commit ce9626e.

This issue was reported in a community Slack thread. I confirmed that it started failing at commit ce9626e, which is associated with the arrival of the Arena changes in #5118.

$ zq -version
Version: v1.15.0-17-gce9626eb

$ echo '{"a":[1]}' | zq -f text 'yield this' -
panic: type conflict

goroutine 1 [running]:
github.com/brimdata/zed.(*Arena).new(0xc00060ee10, {0x28e2de0?, 0xc0004a8888}, 0x2000000000000000, 0x1, 0x0)
	/Users/phil/work/zed/arena.go:129 +0x398
github.com/brimdata/zed.(*Arena).New(0xc00060ee10?, {0x28e2de0?, 0xc0004a8888?}, {0xc000014f41?, 0x2?, 0x7?})
	/Users/phil/work/zed/arena.go:99 +0x12e
github.com/brimdata/zed.(*Value).DerefByColumn(0xc00041f9e8, 0xc0003c0330?, 0x0)
	/Users/phil/work/zed/value.go:445 +0xac
github.com/brimdata/zed/zio/textio.(*Writer).writeRecord(0xc0007abfc0, {0x100000c00060ee10, 0x2000001e00000000})
	/Users/phil/work/zed/zio/textio/writer.go:50 +0x26c
github.com/brimdata/zed/zio/textio.(*Writer).Write(0xc0007abfc0, {0xc00041fa60?, 0xc00041fa98?})
	/Users/phil/work/zed/zio/textio/writer.go:36 +0x196
github.com/brimdata/zed/zbuf.WriteBatch({0x4ab90018, 0xc0007abfc0}, {0x28f1400?, 0xc0004be700?})
	/Users/phil/work/zed/zbuf/batch.go:97 +0x70
github.com/brimdata/zed/zbuf.CopyPuller({0x4ab90018, 0xc0007abfc0}, {0x28dcba0, 0xc0003c0240})
	/Users/phil/work/zed/zbuf/batch.go:163 +0x76
github.com/brimdata/zed/cli/zq.(*Command).Run(0xc0000de1e0, {0xc00013c030, 0x2, 0x2})
	/Users/phil/work/zed/cli/zq/command.go:162 +0x76a
github.com/brimdata/zed/pkg/charm.path.run({0xc0001222d0?, 0x1, 0x1}, {0xc00013c030?, 0x2, 0x0?})
	/Users/phil/work/zed/pkg/charm/path.go:11 +0x79
github.com/brimdata/zed/pkg/charm.(*Spec).ExecRoot(0x100874b?, {0xc00013c010, 0x4, 0x4})
	/Users/phil/work/zed/pkg/charm/charm.go:63 +0x3f
main.main()
	/Users/phil/work/zed/cmd/zq/main.go:11 +0x5b

$ echo '{"a":[1]}' | zq -f table 'yield this' -
a
panic: type conflict

goroutine 1 [running]:
github.com/brimdata/zed.(*Arena).new(0xc00012a870, {0x28e2de0?, 0xc0005128a0}, 0x2000000000000000, 0x1, 0x0)
	/Users/phil/work/zed/arena.go:129 +0x398
github.com/brimdata/zed.(*Arena).New(0xc00012a870?, {0x28e2de0?, 0xc0005128a0?}, {0xc000014f41?, 0x2?, 0x7?})
	/Users/phil/work/zed/arena.go:99 +0x12e
github.com/brimdata/zed.(*Value).DerefByColumn(0xc00049fa70, 0xc000438330?, 0x0)
	/Users/phil/work/zed/value.go:445 +0xac
github.com/brimdata/zed/zio/tableio.(*Writer).Write(0xc0005289c0, {0x100000c00012a870, 0x2000001e00000000})
	/Users/phil/work/zed/zio/tableio/writer.go:64 +0x6f6
github.com/brimdata/zed/zbuf.WriteBatch({0x4ab4b758, 0xc0005289c0}, {0x28f1400?, 0xc00052a700?})
	/Users/phil/work/zed/zbuf/batch.go:97 +0x70
github.com/brimdata/zed/zbuf.CopyPuller({0x4ab4b758, 0xc0005289c0}, {0x28dcba0, 0xc000438240})
	/Users/phil/work/zed/zbuf/batch.go:163 +0x76
github.com/brimdata/zed/cli/zq.(*Command).Run(0xc0000f81e0, {0xc0001b4030, 0x2, 0x2})
	/Users/phil/work/zed/cli/zq/command.go:162 +0x76a
github.com/brimdata/zed/pkg/charm.path.run({0xc00019a2d8?, 0x1, 0x1}, {0xc0001b4030?, 0x2, 0x0?})
	/Users/phil/work/zed/pkg/charm/path.go:11 +0x79
github.com/brimdata/zed/pkg/charm.(*Spec).ExecRoot(0x100874b?, {0xc0001b4010, 0x4, 0x4})
	/Users/phil/work/zed/pkg/charm/charm.go:63 +0x3f
main.main()
	/Users/phil/work/zed/cmd/zq/main.go:11 +0x5b

No panic at the prior commit.

$ zq -version
Version: v1.15.0-16-g183e432f

$ echo '{"a":[1]}' | zq -f text 'yield this' -
1

$ echo '{"a":[1]}' | zq -f table 'yield this' -
a
1

[nwt]

This is happening because tableio.Writer and textio.Writer use a single arena with types from two zed.Contexts. (They create the second context for expr.NewFlattener. csvio.Writer and zeekio.Writer use NewFlattener in the same way but don't seem to be affected by this bug, presumably because they never create values in the arena using types created by the flattener's zed.Context.)

Some ways to fix this:

1. Make the query's zed.Context available to these writers. This feels wrong because we don't allow a zio.Writer to alter the query context anywhere else.
2. Add second arena. This works for the {a:[1]} example but feels fragile because it doesn't address an underlying problem with using two zed.Contexts here, which is that we can create record types in the flattener context that contain complex types from the query context.
3. Modify expr.Flattener.Flatten to translate all types (including those passed to FlattenFields) into the flattener context.
4. Replace expr.Flattener in these writers with an explicit walk of nested records that writes each field as they're encountered (rather than creating a new, flattened record value).

[nwt]

Fixed in #5278.

[philrz]

Verified in Zed commit 4a1ebf5.

Repeating the repro steps, we now once again see the expected outputs in text and table formats instead of a panic.

$ zq -version
Version: v1.17.0-64-g4a1ebf5b

$ echo '{"a":[1]}' | zq -f text 'yield this' -
1

$ echo '{"a":[1]}' | zq -f table 'yield this' -
a
1

Thanks @nwt!