New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: orm passage migration #2180

Merged

mattzh72 merged 36 commits into main from mindy/orm-passage-migration

Dec 11, 2024

Collaborator

mlong93 commented Dec 6, 2024

Please describe the purpose of this pull request.
Migrating passages to ORM, phasing out archival memory

How to test
How can we test your PR during review? What commands should we run? What outcomes should we expect?

Have you tested this PR?
Have you tested the latest commit on the PR? If so please provide outputs from your tests.

Related issues or PRs
Please link any related GitHub issues or PRs.

Is your PR over 500 lines of code?
If so, please break up your PR into multiple smaller PRs so that we can review them quickly, or provide justification for its length.

Additional context
Add any other context or screenshots about the PR here.

Mindy Long added 3 commits

December 6, 2024 13:48


          Added Passage ORM and PassageManager

561b9c2

* Test cases for PassageManager
* Additional mixins for File, User
* Better `list` method in SQLAlchemyBase
* Pydantic Passage model uses ORMBase


          gitignore changes

9923e3d


          Merge branch 'main' into mindy/orm-passage-migration

7a02d73

* Adding Messages migration

mlong93 had a problem deploying to Deployment

December 6, 2024 21:58

— with

GitHub Actions Failure


          fixed reading model env vars from .env

b9c0054

mlong93 had a problem deploying to Deployment

December 6, 2024 22:21

— with

GitHub Actions Failure

github-actions bot commented Dec 6, 2024

📦 Dependencies Alert:

Both poetry.lock and pyproject.toml have been modified

Please review these changes carefully to ensure they are intended (cc @sarahwooders @cpacker).

Mindy Long added 2 commits

December 6, 2024 14:31


          replaced archival memory with passage manager in Agent

e1b3795


          Changed passages to have a relation with organization, not user. Modi…

e74f636

…fied test_managers

mlong93 had a problem deploying to Deployment

December 6, 2024 23:47

— with

GitHub Actions Failure

github-actions bot commented Dec 6, 2024

📦 Dependencies Alert:

Both poetry.lock and pyproject.toml have been modified

Please review these changes carefully to ensure they are intended (cc @sarahwooders @cpacker).


          Replaced archival memory with PassageManager in Agent and Server

726c812

mlong93 had a problem deploying to Deployment

December 7, 2024 00:36

— with

GitHub Actions Failure


          Updated storage and connectors for new Passage schema

7003a0d

* Passage has a relationship with organization, not user
* Fixed filtering bug for None columns in Chroma connector filtering
* Removed outdated test cases for archival memory

github-actions bot commented Dec 7, 2024

📦 Dependencies Alert:

Both poetry.lock and pyproject.toml have been modified

Please review these changes carefully to ensure they are intended (cc @sarahwooders @cpacker).

mlong93 had a problem deploying to Deployment

December 7, 2024 00:39

— with

GitHub Actions Failure

github-actions bot commented Dec 7, 2024

📦 Dependencies Alert:

Both poetry.lock and pyproject.toml have been modified

Please review these changes carefully to ensure they are intended (cc @sarahwooders @cpacker).

mlong93 had a problem deploying to Deployment

December 7, 2024 02:18

— with

GitHub Actions Failure

github-actions bot commented Dec 7, 2024

📦 Dependencies Alert:

Both poetry.lock and pyproject.toml have been modified

Please review these changes carefully to ensure they are intended (cc @sarahwooders @cpacker).

Mindy Long added 2 commits

December 6, 2024 19:14


          Added SQLite and PGSQL support for passages

bfc234a

* Fixed passage-file foreign key constraint
* Phased out schemas/Passage


          added alembic migration script

51b36df

mlong93 force-pushed the mindy/orm-passage-migration branch from 5796a06 to 51b36df Compare

December 7, 2024 22:26

mlong93 had a problem deploying to Deployment

December 7, 2024 22:26

— with

GitHub Actions Failure

github-actions bot commented Dec 7, 2024

📦 Dependencies Alert:

Both poetry.lock and pyproject.toml have been modified

Please review these changes carefully to ensure they are intended (cc @sarahwooders @cpacker).


          Added cascading delete to file-passages

458ddc5

* Additional alembic migration
* Updated test managers, passing

mlong93 had a problem deploying to Deployment

December 7, 2024 23:01

— with

GitHub Actions Failure

github-actions bot commented Dec 7, 2024

📦 Dependencies Alert:

Both poetry.lock and pyproject.toml have been modified

Please review these changes carefully to ensure they are intended (cc @sarahwooders @cpacker).


          Passing server tests

6df44db

* renamed PassageModel to PassageModelLegacy
* modified load_data to take agent_id
* deleted passages before deleting agent_id
* debugged server::test_load_data, removed extraneous AgentMixin

mlong93 had a problem deploying to Deployment

December 7, 2024 23:55

— with

GitHub Actions Failure

github-actions bot commented Dec 7, 2024

📦 Dependencies Alert:

Both poetry.lock and pyproject.toml have been modified

Please review these changes carefully to ensure they are intended (cc @sarahwooders @cpacker).


          fixed server tests

189f123

mlong93 temporarily deployed to Deployment

December 10, 2024 17:33

— with

GitHub Actions Inactive

github-actions bot commented Dec 10, 2024

📦 Dependencies Alert:

pyproject.toml has been modified

Please review these changes carefully to ensure they are intended (cc @sarahwooders @cpacker).

Mindy Long added 2 commits

December 10, 2024 11:24


          made agent optional for Passage

09a0546


          updated agent.attach_source() for passage_managers and all references

7dc7e2f

mlong93 had a problem deploying to Deployment

December 10, 2024 19:38

— with

GitHub Actions Failure

github-actions bot commented Dec 10, 2024

📦 Dependencies Alert:

pyproject.toml has been modified

Please review these changes carefully to ensure they are intended (cc @sarahwooders @cpacker).


          fix: attaching source to agent in metadata

6d3ed2a

mlong93 temporarily deployed to Deployment

December 10, 2024 22:44

— with

GitHub Actions Inactive

github-actions bot commented Dec 10, 2024

📦 Dependencies Alert:

pyproject.toml has been modified

Please review these changes carefully to ensure they are intended (cc @sarahwooders @cpacker).


          final alembic script, added notes for Passages

69e0233

mlong93 had a problem deploying to Deployment

December 10, 2024 23:16

— with

GitHub Actions Failure

github-actions bot commented Dec 10, 2024

📦 Dependencies Alert:

pyproject.toml has been modified

Please review these changes carefully to ensure they are intended (cc @sarahwooders @cpacker).


          minor pr fixes and 1 monkeypatch

d3fba84

mlong93 temporarily deployed to Deployment

December 11, 2024 00:12

— with

GitHub Actions Inactive

github-actions bot commented Dec 11, 2024

📦 Dependencies Alert:

pyproject.toml has been modified

Please review these changes carefully to ensure they are intended (cc @sarahwooders @cpacker).


          removed prints

b0e1fb5

mlong93 temporarily deployed to Deployment

December 11, 2024 00:15

— with

GitHub Actions Inactive

github-actions bot commented Dec 11, 2024

📦 Dependencies Alert:

pyproject.toml has been modified

Please review these changes carefully to ensure they are intended (cc @sarahwooders @cpacker).


          1 less print

df40cab

mlong93 temporarily deployed to Deployment

December 11, 2024 00:17

— with

GitHub Actions Inactive

github-actions bot commented Dec 11, 2024

📦 Dependencies Alert:

pyproject.toml has been modified

Please review these changes carefully to ensure they are intended (cc @sarahwooders @cpacker).


          removing chroma

0c32f60

mlong93 temporarily deployed to Deployment

December 11, 2024 01:32

— with

GitHub Actions Inactive

github-actions bot commented Dec 11, 2024

📦 Dependencies Alert:

pyproject.toml has been modified

Please review these changes carefully to ensure they are intended (cc @sarahwooders @cpacker).

mattzh72 reviewed

View reviewed changes

letta/services/passage_manager.py Show resolved Hide resolved

mattzh72 reviewed

View reviewed changes

tests/test_server.py Show resolved Hide resolved

mattzh72 self-requested a review

December 11, 2024 02:09

mattzh72 approved these changes

View reviewed changes

Collaborator

mattzh72 left a comment

great work!

mattzh72 merged commit 31d2774 into main

34 of 35 checks passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet