From 0eeef24ece1f77d6ecdfcd74366e3bd395f24ce3 Mon Sep 17 00:00:00 2001 From: Alexis VIALARET Date: Wed, 21 Feb 2024 23:35:40 +0100 Subject: [PATCH] fix: doc --- docs/cookbook/configs/databases_configs.md | 43 +++++++++++++ .../configs/embedding_models_configs.md | 46 ++++++++++++++ docs/cookbook/configs/llms_configs.md | 62 +++++++++++++++++++ .../cookbook/configs/vector_stores_configs.md | 60 ++++++++++++++++++ docs/cookbook/cookbook.md | 6 ++ docs/cookbook/extend_ragconfig.md | 42 +++++++++++++ docs/cookbook/loading_documents.md | 29 +++++++++ mkdocs.yml | 12 +++- 8 files changed, 298 insertions(+), 2 deletions(-) create mode 100644 docs/cookbook/configs/databases_configs.md create mode 100644 docs/cookbook/configs/embedding_models_configs.md create mode 100644 docs/cookbook/configs/llms_configs.md create mode 100644 docs/cookbook/configs/vector_stores_configs.md create mode 100644 docs/cookbook/cookbook.md create mode 100644 docs/cookbook/extend_ragconfig.md create mode 100644 docs/cookbook/loading_documents.md diff --git a/docs/cookbook/configs/databases_configs.md b/docs/cookbook/configs/databases_configs.md new file mode 100644 index 0000000..4d3df4b --- /dev/null +++ b/docs/cookbook/configs/databases_configs.md @@ -0,0 +1,43 @@ +The database config is the "easiest" as it only requires a database URL. + +So far, `sqlite`, `mysql`, and `postgresql` are supported. + +CloudSQL on GCP, RDS on AWS, or Azure Database will allow you to deploy `mysql`, and `postgresql` database instances. + +!!! warning + If using `mysql` or `postgresql` you will need to also create a database, typically named `rag`, to be able to use it. + + You will also need to create a user, and get its password. Make sure there are no spacial characters in the password. + + +As the database URL contains a username and password, we don't want to have it directly in the `config.yaml`. + +Instead, we have: +```yaml +# backend/config.yaml +DatabaseConfig: &DatabaseConfig + database_url: {{ DATABASE_URL }} +``` + +And `DATABASE_URL` is coming from an environment variable. + +The connection strings are formated as follows: + +- **SQLite:** `sqlite:///database/rag.sqlite3` +```shell +export DATABASE_URL=sqlite:///database/rag.sqlite3 +``` + +- **mySQL:** `mysql://:@:/rag` +```shell +# The typical port is 3306 for mySQL +export DATABASE_URL=mysql://username:abcdef12345@123.45.67.89:3306/rag +``` + +- **postgreSQL:** `postgresql://:@:/rag` +```shell +# The typical port is 5432 for postgreSQL +export DATABASE_URL=postgresql://username:abcdef12345@123.45.67.89:5432/rag +``` + +When first testing the RAG locally, `sqlite` is the best since it requires no setup as the database is just a file on your machine. However, if you're working as part of a team, or looking to industrialize, you will need to deploy a `mysql`, or `postgresql` instance. diff --git a/docs/cookbook/configs/embedding_models_configs.md b/docs/cookbook/configs/embedding_models_configs.md new file mode 100644 index 0000000..4932464 --- /dev/null +++ b/docs/cookbook/configs/embedding_models_configs.md @@ -0,0 +1,46 @@ +## Locally hosted embedding model from Hugging Face + +This will download the selected model from the HF hub and make embeddings on the machine the backend is running on. +```shell +pip install sentence_transformers +``` + +```yaml +# backend/config.yaml +EmbeddingModelConfig: &EmbeddingModelConfig + source: HuggingFaceEmbeddings + source_config: + model_name : 'BAAI/bge-base-en-v1.5' +``` + + +## Artefact Azure-hosted embedding model + +```yaml +# backend/config.yaml +EmbeddingModelConfig: &EmbeddingModelConfig + source: OpenAIEmbeddings + source_config: + openai_api_type: azure + openai_api_key: {{ EMBEDDING_API_KEY }} + openai_api_base: https://poc-openai-artefact.openai.azure.com/ + deployment: embeddings + chunk_size: 500 +``` + +## AWS Bedrock + +!!! info "You will first need to login to AWS" + + ```shell + pip install boto3 + ``` + [Follow this guide to authenticate your machine](https://docs.aws.amazon.com/cli/latest/userguide/cli-authentication-user.html) + +```yaml +# backend/config.yaml +EmbeddingModelConfig: &EmbeddingModelConfig + source: BedrockEmbeddings + source_config: + model_id: 'amazon.titan-embed-text-v1' +``` diff --git a/docs/cookbook/configs/llms_configs.md b/docs/cookbook/configs/llms_configs.md new file mode 100644 index 0000000..9e949af --- /dev/null +++ b/docs/cookbook/configs/llms_configs.md @@ -0,0 +1,62 @@ +## Artefact Azure-hosted GPT4-turbo + +```yaml +# backend/config.yaml +LLMConfig: &LLMConfig + source: AzureChatOpenAI + source_config: + openai_api_type: azure + openai_api_key: {{ OPENAI_API_KEY }} + openai_api_base: https://genai-ds.openai.azure.com/ + openai_api_version: 2023-07-01-preview + deployment_name: gpt4 + temperature: 0.1 +``` + +## Local llama2 +!!! info "You will first need to install and run Ollama" + + [Download the Ollama application here](https://ollama.ai/download) + + Ollama will automatically utilize the GPU on Apple devices. + + ```shell + ollama run llama2 + ``` + +```yaml +# backend/config.yaml +LLMConfig: &LLMConfig + source: ChatOllama + source_config: + model: llama2 +``` + +## Vertex AI gemini-pro + +!!! warning + + Right now Gemini models' safety settings are **very** sensitive, and is is not possible to disable them. That makes this model pretty much useless for the time being as it blocks most requests and/or responses. + + Github issue to follow: https://github.com/langchain-ai/langchain/pull/15344#issuecomment-1888597151 + +!!! info "You will first need to login to GCP" + + ```shell + export PROJECT_ID= + gcloud config set project $PROJECT_ID + gcloud auth login + gcloud auth application-default login + ``` + +!!! info "" + Activate the Vertex APIs in your project + +```yaml +# backend/config.yaml +LLMConfig: &LLMConfig + source: ChatVertexAI + source_config: + model_name: gemini-pro + temperature: 0.1 +``` diff --git a/docs/cookbook/configs/vector_stores_configs.md b/docs/cookbook/configs/vector_stores_configs.md new file mode 100644 index 0000000..ca823ab --- /dev/null +++ b/docs/cookbook/configs/vector_stores_configs.md @@ -0,0 +1,60 @@ +## PostgreSQL + +As we need a backend SQL database to store conversation history and other info, using Postgres as a vector store is very attractive for us. Implementeing all this functionalities using the same technology reduces deployment overhead and complexity. + +[See the recipes for database configs here](databases_configs.md) + +```shell +pip install psycopg2-binary pgvector +``` + +```yaml +# backend/config.yaml +VectorStoreConfig: &VectorStoreConfig + source: PGVector + source_config: + connection_string: {{ DATABASE_URL }} + + retriever_search_type: similarity_score_threshold + retriever_config: + k: 20 + score_threshold: 0.5 + + insertion_mode: null +``` + +`top_k`: maximum number of documents to fetch. + +`score_threshold`: score below which a document is deemed irrelevant and not fetched. + +`insertion_mode`: `null` | `full` | `incremental`. [How document indexing and insertion in the vector store is handled.](https://python.langchain.com/docs/modules/data_connection/indexing#deletion-modes) + + +## Local Chroma + +```yaml +# backend/config.yaml +VectorStoreConfig: &VectorStoreConfig + source: Chroma + source_config: + persist_directory: vector_database/ + collection_metadata: + hnsw:space: cosine + + retriever_search_type: similarity_score_threshold + retriever_config: + k: 20 + score_threshold: 0.5 + + insertion_mode: null +``` + +`persist_directory`: where, locally the Chroma database will be persisted. + +`hnsw:space: cosine`: [distance function used. Default is `l2`.](https://docs.trychroma.com/usage-guide#changing-the-distance-function) Cosine is bounded [0; 1], making it easier to set a score threshold for retrival. + +`top_k`: maximum number of documents to fetch. + +`score_threshold`: score below which a document is deemed irrelevant and not fetched. + +`insertion_mode`: `null` | `full` | `incremental`. [How document indexing and insertion in the vector store is handled.](https://python.langchain.com/docs/modules/data_connection/indexing#deletion-modes) diff --git a/docs/cookbook/cookbook.md b/docs/cookbook/cookbook.md new file mode 100644 index 0000000..a388850 --- /dev/null +++ b/docs/cookbook/cookbook.md @@ -0,0 +1,6 @@ +Here you will find a repository of configurations that have proven to work. + +- [LLM Configuration samples](configs/llms_configs.md) +- [Embedding model Configuration samples](configs/embedding_models_configs.md) +- [Vector Store Configuration samples](configs/vector_stores_configs.md) +- [Database Configuration samples](configs/databases_configs.md) diff --git a/docs/cookbook/extend_ragconfig.md b/docs/cookbook/extend_ragconfig.md new file mode 100644 index 0000000..f211409 --- /dev/null +++ b/docs/cookbook/extend_ragconfig.md @@ -0,0 +1,42 @@ +As you tune this starter kit to your needs, you may need to add specific configuration that your RAG will use. + +For example, let's say you want to add the `foo` configuration parameter to your vector store configuration. + +First, add it to `config.py` in the part relavant to the vector store: + +```python +# ... + +@dataclass +class VectorStoreConfig: + # ... rest of the VectorStoreConfig ... + + foo: str = "bar" # We add foo param, of type str, with the default value "bar" + +# ... +``` + +This parameter will now be available in your `RAG` object configuration. + +```python +from pathlib import Path +from backend.rag_components.rag import RAG + +config_directory = Path("backend/config.yaml") +rag = RAG(config_directory) + +print(rag.config.vector_store.foo) +# > bar +``` + +if you want to override its default value. You can do that in your `config.yaml`: +```yaml +VectorStoreConfig: &VectorStoreConfig + # ... rest of the VectorStoreConfig ... + foo: baz +``` + +```python +print(rag.config.vector_store.foo) +# > baz +``` diff --git a/docs/cookbook/loading_documents.md b/docs/cookbook/loading_documents.md new file mode 100644 index 0000000..783260b --- /dev/null +++ b/docs/cookbook/loading_documents.md @@ -0,0 +1,29 @@ +## Loading documents + +The easiest but least flexible way to load documents to your RAG is to use the `RAG.load_file` method. It will semi-intellignetly try to pick the best Langchain loader and parameters for your file. + +```python +from pathlib import Path + +from backend.rag_components.rag import RAG + + +data_directory = Path("data") + +config_directory = Path("backend/config.yaml") +rag = RAG(config_directory) + +for file in data_directory.iterdir(): + if file.is_file(): + rag.load_file(file) +``` + +If you want more flexibility, you can use the `rag.load_documents` method which expects a list of `langchain.docstore.document` objects. + +**TODO: example** + +## Document indexing + +The document loader maintains an index of the loaded documents. You can change it in the configuration of your RAG at `vector_store.insertion_mode` to `None`, `incremental`, or `full`. + +[Details of what that means here.](https://python.langchain.com/docs/modules/data_connection/indexing) diff --git a/mkdocs.yml b/mkdocs.yml index 87038ba..ab96d02 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -18,8 +18,6 @@ nav: - Home: index.md - The frontend: frontend.md - The database: database.md - - Deployment: - - Admin Mode: deployment/admin_mode.md - The backend: - The backend: backend/backend.md - RAG and RAGConfig classes: backend/rag_ragconfig.md @@ -28,3 +26,13 @@ nav: - Memory and sessions: backend/plugins/conversational_rag_plugin - Authentication: backend/plugins/authentication.md - Secure user-based sessions: backend/plugins/user_based_sessions.md + - Deployment: + - Admin Mode: deployment/admin_mode.md + - Cookbook: + - config.yaml recipes: + - LLMs: cookbook/configs/llms_configs.md + - Vector Stores: cookbook/configs/vector_stores_configs.md + - Embedding Models: cookbook/configs/embedding_models_configs.md + - Databases: cookbook/configs/databases_configs.md + - How to load documents in the RAG: cookbook/loading_documents.md + - How to extend the RAGConfig: cookbook/extend_ragconfig.md