-
Notifications
You must be signed in to change notification settings - Fork 52
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add custom metadata to uploaded files #93
Add custom metadata to uploaded files #93
Conversation
@@ -0,0 +1,256 @@ | |||
/* eslint-disable @typescript-eslint/no-unsafe-assignment */ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In order to use data: expect.any(String),
instead add the entiry data as string. same for ids, it will change on every run.
@@ -0,0 +1,256 @@ | |||
/* eslint-disable @typescript-eslint/no-unsafe-assignment */ | |||
/* eslint-disable @typescript-eslint/no-explicit-any */ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this was added to test a bad configuration, I know we must trust on TS, but on the file we have many any
s already, but was used for the last 2 tests, it kinda was just to add more branch coverage
id: nanoid(), | ||
metadata: { | ||
...(metadataMapper ? metadataMapper(document.metadata, index) : {}), | ||
...this.config.options?.metadata, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this was the biggest change here,
I'm destructuring the current metadata, to accept the provided metadata by the user, as we can do on raw texts
closing this PR as it was merged through #95. Thanks for the contribution! |
Hi, not sure if is intended, but users are not able to save custom metadata to the vector when uploading a file from ragchat.
Usage example:
Given a setup RagChat instance,
When adding a new context from sources (e.g., PDF),
And including custom metadata,
Then users should be able to visualize their custom metadata in the Upstash vector console.
Please refer to the attached screenshot for a visual example of the issue:
![Screenshot 2024-11-13 at 21 55 06](https://private-user-images.githubusercontent.com/4389565/385955720-0ced66b4-4334-45c3-8a26-c6cccbeeec6a.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3Mzk0NzQ5MTksIm5iZiI6MTczOTQ3NDYxOSwicGF0aCI6Ii80Mzg5NTY1LzM4NTk1NTcyMC0wY2VkNjZiNC00MzM0LTQ1YzMtOGEyNi1jNmNjY2JlZWVjNmEucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI1MDIxMyUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNTAyMTNUMTkyMzM5WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9NTk3ZWQ4YTFiZjZlNjQ4YWFlMDUyYTdkMDMxNTdlNmEwY2VmOGNmZDQ1ZTQ5MWNhNzc5NzkyNmM5MjdkNGRhMSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QifQ.qrLm--exBETpLi8mX-ZFd3X9EnZkSn6iaTA5M47hxxo)
Proposed solution on this PR extends
config.options?.metadata
formapDocumentsIntoInsertPayload
and turn it into a private method in order to be able to read the config.