add migrations for ML generated fields #1153
Conversation
| -- Add a generated summary to the reference to make it easier for the LLM to choose what to read | ||
| ALTER TABLE vulnerability.reference_content ADD COLUMN summary text NULL; | ||
|
|
||
| ALTER TABLE package.package ADD COLUMN readme_text text NULL; |
There was a problem hiding this comment.
A package already has package.description which often times the readme. Probably don't need to add another column to do the search.
There was a problem hiding this comment.
Why are we jamming the readme into that column? That isn't what I would expect, id expect it to have a short description. Also, often? When does it not? Depending on ecosystem?
Lets sort this out in standup because im interested!
| ALTER TABLE vulnerability.reference_content ADD COLUMN summary text NULL; | ||
|
|
||
| ALTER TABLE package.package ADD COLUMN readme_text text NULL; | ||
| ALTER TABLE package.package ADD COLUMN use_case_summary text NULL; |
There was a problem hiding this comment.
What will this column contain that is different than description?
There was a problem hiding this comment.
We discussed this a few times so I think you're familiar. By telling the LLM to summarize the use case specifically, and avoid all other descriptions, we get a much better vector proximity. It's not a general description, its just "what is this for"
| ); | ||
|
|
||
| -- Add a generated summary to the reference to make it easier for the LLM to choose what to read | ||
| ALTER TABLE vulnerability.reference_content ADD COLUMN summary text NULL; |
There was a problem hiding this comment.
what is the difference between this column and https://github.com/lunasec-io/lunasec/blob/master/lunatrace/bsl/hasura/migrations/lunatrace/1678406466712_add_parsed_content_to_reference_content/up.sql#LL1C81-L1C81?
There was a problem hiding this comment.
We also discussed this the other day, this is a short one to two sentence description of the content, so that we can display it in a list for the LLM to choose what to read.
Regenerate files for ml-migrations
No description provided.