Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improved pipeline validation #1045

Open
wants to merge 12 commits into
base: main
Choose a base branch
from
Open

Improved pipeline validation #1045

wants to merge 12 commits into from

Conversation

snopoke
Copy link
Collaborator

@snopoke snopoke commented Jan 16, 2025

Description

Improve pipeline validation to try and reduce the likelyhood of creating invalid pipelines like this one:

image

There is an issue with the UI updating (auto-save doesn't update the node / edge data) which I still need to resolve but I'll do that in a separate PR since this one is big enough.

User Impact

Better pipeline validation and error display.

Demo

image
image
image

@snopoke snopoke changed the title Sk/pipeline validations Improved pipeline validation Jan 16, 2025
@codecov-commenter
Copy link

❌ 2 Tests Failed:

Tests completed Failed Passed Skipped
820 2 818 0
View the top 2 failed tests by shortest run time
apps/pipelines/tests/test_runnable_builder.py::test_cyclical_graph
Stack Traces | 1.17s run time
.../pipelines/tests/test_runnable_builder.py:735: in test_cyclical_graph
    create_runnable(pipeline, nodes, edges)
.../pipelines/tests/utils.py:32: in create_runnable
    return PipelineGraph.build_runnable_from_pipeline(pipeline)
apps/pipelines/graph.py:89: in build_runnable_from_pipeline
    return cls.build_from_pipeline(pipeline).build_runnable()
apps/pipelines/graph.py:107: in build_runnable
    self._validate_no_parallel_nodes()
apps/pipelines/graph.py:138: in _validate_no_parallel_nodes
    raise PipelineBuildError(
E   apps.pipelines.exceptions.PipelineBuildError: Multiple edges connected to the same output

During handling of the above exception, another exception occurred:
.../pipelines/tests/test_runnable_builder.py:734: in test_cyclical_graph
    with pytest.raises(PipelineBuildError, match="A cycle was detected"):
E   AssertionError: Regex pattern did not match.
E    Regex: 'A cycle was detected'
E    Input: 'Multiple edges connected to the same output'
apps/pipelines/tests/test_runnable_builder.py::test_branching_pipeline
Stack Traces | 2.22s run time
.../pipelines/tests/test_runnable_builder.py:221: in test_branching_pipeline
    output = create_runnable(pipeline, nodes, edges).invoke(
.../pipelines/tests/utils.py:32: in create_runnable
    return PipelineGraph.build_runnable_from_pipeline(pipeline)
apps/pipelines/graph.py:89: in build_runnable_from_pipeline
    return cls.build_from_pipeline(pipeline).build_runnable()
apps/pipelines/graph.py:107: in build_runnable
    self._validate_no_parallel_nodes()
apps/pipelines/graph.py:138: in _validate_no_parallel_nodes
    raise PipelineBuildError(
E   apps.pipelines.exceptions.PipelineBuildError: Multiple edges connected to the same output

To view more test analytics, go to the Test Analytics Dashboard
📢 Thoughts on this report? Let us know!

Copy link
Collaborator

@SmittieC SmittieC left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good to me

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants