refactor selector package layout

[warpfork]

The current traversal/selector package works, but I think at this point, after seeing how users interact with it (and what confuses them), we could make some changes that would make it auto-explain itself to users much better.

Problem Identification

The main problem with the current arrangement is that it makes it too easy for a new reader to think the selector.Selector type is something they should interact with... which is not the case.

The selector.Selector type is a "complied" thing that's used by other logic to get stuff done; it's often not the data type a user should be doing anything but pass around. Users shouldn't be implementing it; users shouldn't be calling its methods directly themselves; it's not meant to be part of a protocol API itself (its precursors, before "compilation", are!); etc. And yet people see it first, so they keep assuming it will be those things.

Proposed Solution

It would be better to have:

• traversal/selector/engine -- contains the engine.Selector interface, all its implementors, and the Compile functions
• traversal/selector -- contains the user-facing walk functions, and user-facing Compile functions (it mostly delegates)
• traversal/selector/parse -- contains things like the json-to-dmt jumpers
• traversal/selector/builder -- sure, this can come along too

The packages like selector/parse and selector/builder should just be returning an ipld.Node of the DMT of the Selector.

The whole traversal/selector.* namespace can be filled with things a typical user should see at first glance. Like: functions that glue the parse or builder features directly the engine.Compile and then pass all that straight through to starting a Walk. Having complete end-to-end features like that visible in the root package about selectors should increase ease-of-use and discoverability drastically.

The traversal/selector package also becomes free to import other packages like dagjson without fear, and free to use those to make nice one-step helper functions... because if some user is really dependency-conscious and doesn't want that, they are free to do the additional work to make their code use the low level subpackages directly. This frees us up to be much more helpful to the most typical users.

The huge majority of concrete types in the current selector package would all disappear into selector/engine and I think that would be for the best. Users need to be aware of those approximately "never".

Timeline and Change Strategy

Not immediate. At this time, I'm just making notes on the wish.

This would probably result in some API changes that are technically "breaking", so we'll want to decide when it's time for that. That said: the breakages I would expect to make will be extremely clear at compile time and almost certainly addressable by "sed"-scale updates in downstream consumers.

It's possible we could make these changes nonbreaking by using golang's alias feature heavily, but I'm not actually sure I'd want to do that -- it might obstruct fixing the communicational buggaboo about the selector.Selector type being an engine thing that should be mostly unseen, which would reduce what we win with this change. Still worth considering the possibility, though; might be less problematic than I initially fear, or there might be good middle grounds for removing most breakage for most users, etc.

[warpfork]

#249 is solid evidence that there is real user confusion right now, which we could fix by this work.

[warpfork]

An additional improvement we could make as part of this would be ensuring that the compiled selector type is a relatively closed one: a struct with unexported fields could have benefits, such as making it easier to have functions that consume it do a quick sanity check that they aren't holding onto a zero value, which would in turn let us make sure that compiled selector structure was, indeed, compiled at some point.

Currently, the selector type is an interface, and a fairly open one at that (e.g. it could technically be implemented even by types outside of that package -- which is fairly nonsensical; if we could've used a sum type for these, we would have). The result is that we sometimes, when debugging an issue, lose some (admittedly not much, but some) time needing to doublecheck that the selector value has in fact been compiled and made it through the validations that that implies.

[rvagg]

Lots more notes in #192 on this and related topics; closing that one out for now but want to capture context.