Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[DAG] Start a new Pipelining design doc #813

Closed
enricorotundo opened this issue Oct 6, 2022 · 5 comments
Closed

[DAG] Start a new Pipelining design doc #813

enricorotundo opened this issue Oct 6, 2022 · 5 comments
Assignees
@enricorotundo

Comments

@enricorotundo
Copy link
Contributor

enricorotundo commented Oct 6, 2022 •
edited by aronchick
Loading

ETA: 2023Q1

Part of #190

WIP doc: https://hackmd.io/@usN-geg4Q_iFcXZ-UCZpoQ/rkW5FE3Mj
@enricorotundo enricorotundo self-assigned this Oct 6, 2022
@aronchick
Copy link
Collaborator

Thoughts:

  • Please look at https://openlineage.io/ - valuable?
  • If we are executed by an external DAG, how do we inherit their structure

@enricorotundo enricorotundo changed the title Start a new DAG design doc Start a new Pipelining design doc Oct 13, 2022
@enricorotundo
Copy link
Contributor Author

Design doc 1st iteration's here: https://hackmd.io/@usN-geg4Q_iFcXZ-UCZpoQ/rkW5FE3Mj

@enricorotundo enricorotundo mentioned this issue Oct 18, 2022
Closed
@aronchick
Copy link
Collaborator

@enricorotundo can you move this to our design doc repo? https://www.notion.so/pl-strflt/Design-Docs-b25744a00a01437bb4b963cb20d5fde6

@enricorotundo
Copy link
Contributor Author

enricorotundo commented Nov 2, 2022 •
edited
Loading

@aronchick Oh it should be already there (see Bacalhau Pipelines link) within an i-frame.

@aronchick
Copy link
Collaborator

Oops thanks!

@enricorotundo enricorotundo changed the title Start a new Pipelining design doc [DAG] Start a new Pipelining design doc Nov 16, 2022
@aronchick aronchick moved this to Triage in Bacalhau Main Nov 20, 2022
@aronchick aronchick mentioned this issue Nov 20, 2022
Closed
2 tasks
@aronchick aronchick moved this from Triage to In Progress in Bacalhau Main Nov 21, 2022
@aronchick aronchick moved this from In Progress to To Celebrate in Bacalhau Main Nov 21, 2022
@lukemarsden lukemarsden moved this from To Celebrate to Done in Bacalhau Main Nov 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@enricorotundo enricorotundo

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@aronchick @enricorotundo