Amazon Knowledge Base retriever support with new metadata feature?

[langchainuser404]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

retriever = AmazonKnowledgeBasesRetriever(
    knowledge_base_id=bedrock_knowledge_base_index_id,
    retrieval_config={"vectorSearchConfiguration": {"numberOfResults": 3}},
)

...
...

docs = knowledge_base_retriever.get_relevant_documents(question)
for doc in docs:
    print(doc.metadata)

Output:
{'location': {'type': 'S3', 's3Location': {'uri': 's3://info/dir/data.txt'}}, 'score': 0.47478439}

Description

Knowledge Bases for Amazon Bedrock now support metadata (press release).

Can Langchain Amazon Knowledge Bases retriever be integrated to this? I have tested so far, but I am not seeing any such metadata that I created custom.

My understanding is metadata is just added here: https://github.com/langchain-ai/langchainjs/pull/4035/files#diff-6c9ea40663ca59cf62a9488a979044d9140b820dfb09b42749db8880546f87c2R101-R104

Is my understanding correct? If not how I can I currently modify the retriever to pull my custom metadata that I am storing in bedrock knowledge base?

System Info

langchain==0.1.13

[dosubot[bot]]

Hey there, @langchainuser404! Great to see you diving into some exciting features again. How's everything going with your LangChain adventures?

Based on the context provided, it seems like you're trying to retrieve custom metadata from an Amazon Bedrock knowledge base using the AmazonKnowledgeBasesRetriever in LangChain JS. The current version of LangChain JS (0.1.30) does support the retrieval of custom metadata from Amazon Bedrock's Knowledge Bases. However, you mentioned that you're using LangChain version 0.1.13. It's possible that the version you're using doesn't have this feature yet.

To retrieve custom metadata, you need to extend the AmazonKnowledgeBaseRetriever class and override the getRelevantDocuments method. This method should be modified to include the retrieval of custom metadata. Here's an example of how you might implement this:

import { AmazonKnowledgeBaseRetriever } from "@langchain/community/retrievers/amazon_knowledge_base";

class CustomMetadataAmazonKnowledgeBaseRetriever extends AmazonKnowledgeBaseRetriever {
  constructor(options) {
    super(options);
  }

  async getRelevantDocuments(query) {
    const response = await super.getRelevantDocuments(query);
    const documentsWithCustomMetadata = response.map(doc => ({
      ...doc,
      customMetadata: "Your custom metadata here" // Replace this with actual logic to fetch custom metadata
    }));

    return documentsWithCustomMetadata;
  }
}

// Usage
const retriever = new CustomMetadataAmazonKnowledgeBaseRetriever({
  topK: 10,
  knowledgeBaseId: "YOUR_KNOWLEDGE_BASE_ID",
  region: "us-east-2",
  clientOptions: {
    credentials: {
      accessKeyId: "YOUR_ACCESS_KEY_ID",
      secretAccessKey: "YOUR_SECRET_ACCESS_KEY",
    },
  },
});

const docs = await retriever.getRelevantDocuments("How are clouds formed?");
console.log(docs);

This is a simplified example and you'll need to adjust it based on the actual API for fetching documents with custom metadata. The logic inside the getRelevantDocuments method should match how your custom metadata can be fetched or processed, as this will depend on the specifics of your Amazon Bedrock knowledge base and the nature of the custom metadata.

You can refer to the examples/src/retrievers/amazon_knowledge_bases.ts for more details on how to use the AmazonKnowledgeBaseRetriever.

As for the pull request you found, it seems to be related to the feature you're looking for. However, without more details about the changes made in the pull request, it's hard to say for sure if it's directly related to your issue. It would be helpful if you could provide more information about the changes made in the pull request.

I hope this helps! If you have any more questions or need further clarification, feel free to ask.

Sources

• examples/src/retrievers/amazon_knowledge_bases.ts
• libs/langchain-community/src/retrievers/amazon_knowledge_base.ts
• examples/src/document_transformers/metadata_tagger_custom_prompt.ts

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[jacoblee93]

Fixed in #5535!