Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: remove pipeline experiments #214

Merged
merged 3 commits into from
Mar 3, 2025

Conversation

mathislucka
Copy link
Member

@mathislucka mathislucka commented Feb 28, 2025

Related Issues

Proposed Changes:

How did you test it?

Notes for the reviewer

Checklist

Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@mathislucka
Copy link
Member Author

@davidsbatista @Amnah199 with this PR, I also released the tool to compare traces. I thought about keeping it but since you need to import the old pipeline implementation and the new pipeline at the same time, you can't compare traces anyways with a new release.

If people want to compare old Haystack pipeline against new pipeline, they have to downgrade to haystack-ai==2.9 and haystack-experimental==0.7.

@bilgeyucel @dfokina should we turn a version of the AsyncPipeline notebook into a cookbook?

Copy link
Contributor

@bilgeyucel bilgeyucel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mathislucka the AsyncPipeline notebook is already a cookbook here except for the last "Running components concurrently" section. Is that part the only addition to the example?

@@ -44,8 +44,6 @@ that includes it. Once it reaches the end of its lifespan, the experiment will b
| [`InMemoryChatMessageStore`][7] | Memory Store | December 2024 | None | <a href="https://colab.research.google.com/github/deepset-ai/haystack-cookbook/blob/main/notebooks/conversational_rag_using_memory.ipynb" target="_parent"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> | [Discuss](https://github.com/deepset-ai/haystack-experimental/discussions/75) |
| [`Auto-Merging Retriever`][8] & [`HierarchicalDocumentSplitter`][9] | Document Splitting & Retrieval Technique | December 2024 | None | <a href="https://colab.research.google.com/github/deepset-ai/haystack-cookbook/blob/main/notebooks/auto_merging_retriever.ipynb" target="_parent"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> | [Discuss](https://github.com/deepset-ai/haystack-experimental/discussions/78) |
| [`LLMMetadataExtractor`][13] | Metadata extraction with LLM | December 2024 | None | <a href="https://colab.research.google.com/github/deepset-ai/haystack-experimental/blob/main/examples/metadata_extraction_with_llm_metadata_extractor.ipynb" target="_parent"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> | [Discuss](https://github.com/deepset-ai/haystack-experimental/discussions/130) |
| [`AsyncPipeline`][17] | Async Pipeline & Components | February 2024 | None | <a href="https://colab.research.google.com/github/deepset-ai/haystack-experimental/blob/main/examples/async_pipeline.ipynb" target="_parent"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> | [Discuss](https://github.com/deepset-ai/haystack-experimental/discussions/152) |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mathislucka Can you add ready-made super components and Agent to the list?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will do that in separate PRs. I just want to get this one merged so that I don't get merge conflicts (see #211)

@mathislucka
Copy link
Member Author

@mathislucka the AsyncPipeline notebook is already a cookbook here except for the last "Running components concurrently" section. Is that part the only addition to the example?

Oh, didn't see that. It's the old implementation from Madeesh that lists "running components concurrently" as a non-goal. I'd say we should update it. I can work on that.

@mathislucka mathislucka marked this pull request as ready for review March 3, 2025 09:42
@mathislucka mathislucka requested review from a team as code owners March 3, 2025 09:42
@mathislucka mathislucka requested review from dfokina and vblagoje and removed request for a team March 3, 2025 09:42
@coveralls
Copy link

Pull Request Test Coverage Report for Build 13627828667

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+7.7%) to 86.746%

Totals Coverage Status
Change from base Build 13620714261: 7.7%
Covered Lines: 1898
Relevant Lines: 2188

💛 - Coveralls

@coveralls
Copy link

Pull Request Test Coverage Report for Build 13627831409

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+7.7%) to 86.746%

Totals Coverage Status
Change from base Build 13620714261: 7.7%
Covered Lines: 1898
Relevant Lines: 2188

💛 - Coveralls

Copy link
Contributor

@davidsbatista davidsbatista left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mathislucka mathislucka merged commit 6f1b99a into main Mar 3, 2025
8 checks passed
@mathislucka mathislucka deleted the chore/remove-pipeline-asyncpipeline branch March 3, 2025 10:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Move Pipeline and AsyncPipeline experiments to Haystack Core
4 participants