Releases: hofstadter-io/cuetils
v0.4.0
Changelog
This release introduces the flow command and package.
- Flow allows you to write task DAGs, which are properly run based on inter-task dependencies that are automatically inferred
- library of tasks for you to use, wide and growing variety
- ability to import flows & sets of tasks for reuse
see examples/flow and flow/tasks to see what is available. Docs will be added in the near future on https://docs.hofstadter.io
Commits
6263ed1 Merge pull request #35 from hofstadter-io/feat-pipeline
6941d77 add in-mem k/v store; ideate some new tasks
3ba33f9 add links to each output in youtube.cue example
15022f2 add lost changeset for passing pipeline flags
0d9af4f add new task os.Mkdir, enable append in os.WriteFile
90a11da add new tasks os.FileLock and os.FileUnlock
ce562e9 add some notes to ideas.md
6baa8b2 add tags to pipeline for data injection; there's a bug
096cedc add youtube api example
0b3a06e bugfixes
b76184e checkpoint before removing calls to t.Value()
b335fe9 cleanup ideas.md some more
350c53f cmd/cuetils: add verbose flag
d4c03e6 cmd/pipeline: add list & docs flags
65674a4 cmd/pipeline: add logging flag to api.Serve
f15d71d cmd/pipeline: cleanup pipeline task, and build flow.Controller when needed to ensure latest value updates are available
11ddb78 cmd/pipeline: cleanup, refactor, task:api/call
577078c cmd/pipeline: enable listing and doc printing for available pipelines
87f4215 cmd/pipeline: enable verbose printing during listing
8671d62 cmd/pipeline: fix bug in os.Getenv
9c65f7e cmd/pipeline: implement tags for injecting data from the command line
321b8f3 examples/pipeline: add new streamer example with twitch oauth and imported helper task
4f0dc48 finish s/pipeline/flow/g
5a77f05 get basic twitch bot working
9ca2391 implement api.Serve task with routes that support pipelines as their definition
4b1a37a implement db.Call.stmts to share a connection while processing a list of calls
5389d21 implement more of pipeline/os tasks; add several tests
6abd2c7 implement several pipeline/os tasks; general cleanup
4efb003 improve commands and how they are processed in the irc handler
e247dc0 in flow/tasks/*: add locks around CUE evaluator until concurrency safe
f7123ce initial pass at an ETL pipeline built on cue/flow and cuetils/ops
081e585 move around cue test & example files
2c62df5 pipeline largely working; many new task types and tests
29241a4 print tags when listing available pipelines; split up files in pipeline/
f9436ad propagate flag set if nil
910d391 prototype api.Serve task
2a2894d refactor pipeline pt1; add test cases
4cca9bc refactor pipeline task code; mostly fix issue?; update tests/cue; get twitch bot example working again
6d3596b rename example/pipeline to flow
520d1ff rename pipeline cmd to flow and regenerate
1eb0122 rename pipeline dir to flow
2d008ed repro in cuetils, but not cue/cmd
961c2c3 s/pipeline/flow/g in examples dir
8952918 s/pipeline/flow/g in flow dir
8e001fd start breaking down twitch cue files
de98ab9 test cleanup; remove empty file
689d54e update CUE and hof versions
Docker images
docker pull hofstadter/cuetils:v0.4.0-scratchdocker pull hofstadter/cuetils:v0.4-scratchdocker pull hofstadter/cuetils:v0-scratchdocker pull hofstadter/cuetils:latest-scratchdocker pull hofstadter/cuetils:v0.4.0docker pull hofstadter/cuetils:v0.4docker pull hofstadter/cuetils:v0docker pull hofstadter/cuetils:latestdocker pull hofstadter/cuetils:v0.4.0-debiandocker pull hofstadter/cuetils:v0.4-debiandocker pull hofstadter/cuetils:v0-debiandocker pull hofstadter/cuetils:latest-debian
v0.3.3
Changelog
5bb7f11 improve more empty results for diff
Docker images
docker pull hofstadter/cuetils:v0.3.3-scratchdocker pull hofstadter/cuetils:v0.3-scratchdocker pull hofstadter/cuetils:v0-scratchdocker pull hofstadter/cuetils:latest-scratchdocker pull hofstadter/cuetils:v0.3.3docker pull hofstadter/cuetils:v0.3docker pull hofstadter/cuetils:v0docker pull hofstadter/cuetils:latestdocker pull hofstadter/cuetils:v0.3.3-debiandocker pull hofstadter/cuetils:v0.3-debiandocker pull hofstadter/cuetils:v0-debiandocker pull hofstadter/cuetils:latest-debian
v0.3.2
Changelog
a25ff92 fix bug and cleanup diff
Docker images
docker pull hofstadter/cuetils:v0.3.2-scratchdocker pull hofstadter/cuetils:v0.3-scratchdocker pull hofstadter/cuetils:v0-scratchdocker pull hofstadter/cuetils:latest-scratchdocker pull hofstadter/cuetils:v0.3.2docker pull hofstadter/cuetils:v0.3docker pull hofstadter/cuetils:v0docker pull hofstadter/cuetils:latestdocker pull hofstadter/cuetils:v0.3.2-debiandocker pull hofstadter/cuetils:v0.3-debiandocker pull hofstadter/cuetils:v0-debiandocker pull hofstadter/cuetils:latest-debian
v0.3.1
Changelog
2a90591 cleanup some common funcs
8dfb2ff create common glob processors and reduce lots of code
07a7a35 fix go test
342dab0 fix transform bug, fix pick test
4ac0153 implement diff in Go
f3a698c implement insert in Go, some other cleanup
1d6903b implement mask in Go
cbd7b2d implement patch in Go
03e488f implement replace in Go
541090a implement upsert in Go
bd4c8b2 pass opts (flags) as pointer to more places and through recursion; start breaking out op funcs for differnt node types
e0a1bcc prep for rewriting some ops to Go
2c51178 proxy st ops and update cli to use
cce16a4 remove old replace implementation
937a998 reshape the CLI UX a bit
Docker images
docker pull hofstadter/cuetils:v0.3.1-scratchdocker pull hofstadter/cuetils:v0.3-scratchdocker pull hofstadter/cuetils:v0-scratchdocker pull hofstadter/cuetils:latest-scratchdocker pull hofstadter/cuetils:v0.3.1docker pull hofstadter/cuetils:v0.3docker pull hofstadter/cuetils:v0docker pull hofstadter/cuetils:latestdocker pull hofstadter/cuetils:v0.3.1-debiandocker pull hofstadter/cuetils:v0.3-debiandocker pull hofstadter/cuetils:v0-debiandocker pull hofstadter/cuetils:latest-debian
v0.3.0
Changelog
- always use cue.Load to process op code (globs in next version)
- fix issue with loading data files
- implement
extend(add only if not present) - fix issue in Walk not passing options on recursion
- enable more tests in CI, start Go tests
- start implementing ops in Go (pick so far)
- improve CLI args and short help messages
Commits
4f65451 clean up CLI args and short help; add two new command stubs
abef592 cleanup pick code
d21b59f disable cue tests until a bash enabled cue action is available
ffd8750 enable more tests
2d9be41 fix issue with json/yaml entrypoints
45d4a9f implement Go version of Pick
259d90f implement extend command
a121c4c pass options in Walk recursion
0830a61 remove unused file
ac38182 rename input funcs for reuse on all args
1f47c87 rename input processing funcs, refactor in consumers
Docker images
docker pull hofstadter/cuetils:v0.3.0-scratchdocker pull hofstadter/cuetils:v0.3-scratchdocker pull hofstadter/cuetils:v0-scratchdocker pull hofstadter/cuetils:latest-scratchdocker pull hofstadter/cuetils:v0.3.0docker pull hofstadter/cuetils:v0.3docker pull hofstadter/cuetils:v0docker pull hofstadter/cuetils:latestdocker pull hofstadter/cuetils:v0.3.0-debiandocker pull hofstadter/cuetils:v0.3-debiandocker pull hofstadter/cuetils:v0-debiandocker pull hofstadter/cuetils:latest-debian
v0.2.2
Changelog
f24692b add clean flag for picking out basic lits
6f2c1c5 add special case for pick with _; can use for converting formats, eventually selecting subobject or field
ce32612 update pick to support embedded scalars
Docker images
docker pull hofstadter/cuetils:v0.2.2-scratchdocker pull hofstadter/cuetils:v0.2-scratchdocker pull hofstadter/cuetils:v0-scratchdocker pull hofstadter/cuetils:latest-scratchdocker pull hofstadter/cuetils:v0.2.2docker pull hofstadter/cuetils:v0.2docker pull hofstadter/cuetils:v0docker pull hofstadter/cuetils:latestdocker pull hofstadter/cuetils:v0.2.2-debiandocker pull hofstadter/cuetils:v0.2-debiandocker pull hofstadter/cuetils:v0-debiandocker pull hofstadter/cuetils:latest-debian
v0.2.1
Changelog
a53e9cd fix pick cue
f436376 move cue tests
3cb1390 rename load* flags to just load and use for all args
cf76c89 return non-nil error when trying to overwrite without flag
bd461c6 support expressions as operator for a number of commands
bd09f4c update validate command to use new operator loading method
Docker images
docker pull hofstadter/cuetils:v0.2.1-scratchdocker pull hofstadter/cuetils:v0.2-scratchdocker pull hofstadter/cuetils:v0-scratchdocker pull hofstadter/cuetils:latest-scratchdocker pull hofstadter/cuetils:v0.2.1docker pull hofstadter/cuetils:v0.2docker pull hofstadter/cuetils:v0docker pull hofstadter/cuetils:latestdocker pull hofstadter/cuetils:v0.2.1-debiandocker pull hofstadter/cuetils:v0.2-debiandocker pull hofstadter/cuetils:v0-debiandocker pull hofstadter/cuetils:latest-debian
v0.2.0
Changelog
049fdc7 add a number of flags for upcoming changes
b0c3cd2 add validate test and example
e82efa1 add walk function, implement count with walk and Go
f6ea079 implement depth in Go with walk; fix bug in count
cc7c02d operator ,@ ; outname ; overwrite ; update pick to use these
b38d044 pass flags to helpers rather than importing there
dc8ec7b remove default from accum & outname flags
913b891 remove old code
df7953c update other commands to use new operator and output features
a704de8 update readme with cli examples
Docker images
docker pull hofstadter/cuetils:v0.2.0-scratchdocker pull hofstadter/cuetils:v0.2-scratchdocker pull hofstadter/cuetils:v0-scratchdocker pull hofstadter/cuetils:latest-scratchdocker pull hofstadter/cuetils:v0.2.0docker pull hofstadter/cuetils:v0.2docker pull hofstadter/cuetils:v0docker pull hofstadter/cuetils:latestdocker pull hofstadter/cuetils:v0.2.0-debiandocker pull hofstadter/cuetils:v0.2-debiandocker pull hofstadter/cuetils:v0-debiandocker pull hofstadter/cuetils:latest-debian
v0.1.2
Changelog
ab25472 add example details / show to each helper
8d54b03 add workflow to run tests
de9d328 adjust details
dc37150 consolidate pkg into structural, start transform / validate helpers
e188a84 enable cue.Load on some helpers, add tests for implemented commands
3932cbb fix build and test in workflow
e9c371c fix import in default.cue workflow
2dd86e2 implement transform command
66b5c49 implement validate command
3b71ebe prep for cue/load single inputs
4d8408a readme fixes
0709c8f remove test output files...
19e2cec wip readme
Docker images
docker pull hofstadter/cuetils:v0.1.2-scratchdocker pull hofstadter/cuetils:v0.1-scratchdocker pull hofstadter/cuetils:v0-scratchdocker pull hofstadter/cuetils:latest-scratchdocker pull hofstadter/cuetils:v0.1.2docker pull hofstadter/cuetils:v0.1docker pull hofstadter/cuetils:v0docker pull hofstadter/cuetils:latestdocker pull hofstadter/cuetils:v0.1.2-debiandocker pull hofstadter/cuetils:v0.1-debiandocker pull hofstadter/cuetils:v0-debiandocker pull hofstadter/cuetils:latest-debian
v0.1.1
Changelog
402c2ad add count helper; change basic->multi; fix depth for lists
be0adda add replace and upsert
Docker images
docker pull hofstadter/cuetils:v0.1.1-scratchdocker pull hofstadter/cuetils:v0.1-scratchdocker pull hofstadter/cuetils:v0-scratchdocker pull hofstadter/cuetils:latest-scratchdocker pull hofstadter/cuetils:v0.1.1docker pull hofstadter/cuetils:v0.1docker pull hofstadter/cuetils:v0docker pull hofstadter/cuetils:latestdocker pull hofstadter/cuetils:v0.1.1-debiandocker pull hofstadter/cuetils:v0.1-debiandocker pull hofstadter/cuetils:v0-debiandocker pull hofstadter/cuetils:latest-debian