From ddf493f8e9c116de15e1311a8175208811bb8efa Mon Sep 17 00:00:00 2001 From: Adam Date: Thu, 30 Jan 2025 00:22:25 +0100 Subject: [PATCH] Updated documentation --- .../Choosing OpenAI Model for Translation.md | 73 ----- docs/docs/Tutorial/Quick Start Guide.md | 177 ----------- docs/docs/Tutorial/TranslatorMistralCloud.md | 115 -------- docs/docs/Tutorial/TranslatorOpenSource.md | 142 --------- docs/docs/index.md | 277 +++++++----------- docs/mkdocs.yml | 6 +- 6 files changed, 101 insertions(+), 689 deletions(-) diff --git a/docs/docs/Tutorial/Choosing OpenAI Model for Translation.md b/docs/docs/Tutorial/Choosing OpenAI Model for Translation.md index 1a90349..e69de29 100644 --- a/docs/docs/Tutorial/Choosing OpenAI Model for Translation.md +++ b/docs/docs/Tutorial/Choosing OpenAI Model for Translation.md @@ -1,73 +0,0 @@ -## Choosing OpenAI Model for Translation - -When using the `TranslatorOpenAI` class for translation tasks, you have two options to specify the model: - -1. Provide the model name as a string. -2. Use the `ModelForTranslator` enum to select from predefined models. - -### Option 1: Using Model Name as a String - -You can directly pass the model name as a string when creating an instance of `TranslatorOpenAI`. - -```python -from llmtranslate import TranslatorOpenAI - -# Example with a model name as a string -translator = TranslatorOpenAI( - api_key="YOUR_OPENAI_API_KEY", - model="gpt-4o-mini" -) - -translated_text = translator.translate( - text="Cześć jak się masz? Meu nome é Adam", - to_language="en" -) -print(translated_text) -``` - -### Option 2: Using the `ModelForTranslator` Enum - -Alternatively, you can use the `ModelForTranslator` enum to select a model. This enum contains predefined models that have been tested and come with specific versions. - -```python -from llmtranslate import TranslatorOpenAI -from llmtranslate import ModelForTranslator - -# Example using the ModelForTranslator enum -translator = TranslatorOpenAI( - api_key="YOUR_OPENAI_API_KEY", - model=ModelForTranslator.GPT_4o_mini -) - -translated_text = translator.translate( - text="Cześć jak się masz? Meu nome é Adam", - to_language="en" -) -print(translated_text) -``` - -### Available Models in `ModelForTranslator` Enum - -```python -from enum import Enum - -class ModelForTranslator(Enum): - BEST_BIG_MODEL = "gpt-4o" - BEST_SMALL_MODEL = "gpt-4o-mini" - GPT_4o = "gpt-4o" - GPT_4o_mini = "gpt-4o-mini" -``` - -- **BEST_BIG_MODEL**: `"gpt-4o"` - Represents the best large model available. -- **BEST_SMALL_MODEL**: `"gpt-4o-mini"` - Represents the best smaller model, optimized for speed. -- **GPT_4o**: `"gpt-4o"` - Specific version of GPT-4o. -- **GPT_4o_mini**: `"gpt-4o-mini"` - A smaller version of GPT-4o. - - - -### Summary - -- **Flexibility**: Choose between using a string or the enum based on your preference. -- **Predefined Models**: The `ModelForTranslator` enum provides a convenient way to select tested models with known performance characteristics. - - diff --git a/docs/docs/Tutorial/Quick Start Guide.md b/docs/docs/Tutorial/Quick Start Guide.md index 531d775..e69de29 100644 --- a/docs/docs/Tutorial/Quick Start Guide.md +++ b/docs/docs/Tutorial/Quick Start Guide.md @@ -1,177 +0,0 @@ -# Quick Start Guide - -## Installation - -You can install the llmtranslate library from PyPI: - -```bash -pip install llmtranslate -``` - -## Usage - -### Setting the OpenAI API Key - -Before using llmtranslate with OpenAI, you need to set your OpenAI API key. You can do this by creating an instance of the TranslatorOpenAI class. - -```python -from llmtranslate import TranslatorOpenAI - -# Set your OpenAI API key -translator = TranslatorOpenAI(api_key="YOUR_OPENAI_API_KEY", model="gpt-4o-mini") - -``` - -### Language Detection - -To detect the language of a given text: - -```python -from llmtranslate import TranslatorOpenAI - -# Set your OpenAI API key -translator = TranslatorOpenAI(api_key="YOUR_OPENAI_API_KEY", model="gpt-4o-mini") - -# Detect language -detected_language = translator.get_text_language("Hello world") -if detected_language is not None: - print(detected_language.ISO_639_1_code) # Output: 'en' - print(detected_language.ISO_639_2_code) # Output: 'eng' - print(detected_language.ISO_639_3_code) # Output: 'eng' - print(detected_language.language_name) # Output: 'English' - -``` - -!!! warning - If the translator does not detect any language, it will return None.
- Before using results of translator detection you should check if it returned correct result or None - -### Translation - -To translate text containing multiple languages into another language, you need to provide the ISO 639 language code for the target language. For a list of all ISO 639 language codes, you can refer to this [ISO 639-1 code list website](https://localizely.com/iso-639-1-list/). - -```python -from llmtranslate import TranslatorOpenAI - -# Set your OpenAI API key -translator = TranslatorOpenAI(api_key="YOUR_OPENAI_API_KEY", model="gpt-4o-mini") - -# Translate text -translated_text = translator.translate( - text="Cześć jak się masz? Meu nome é Adam", - to_language="en" # Use ISO 639-1 code for the target language -) -print(translated_text) # Output: "Hello how are you? My name is Adam" -``` - - -### Full Example - -Here is a complete example demonstrating how to use the library: - -```python -from llmtranslate import TranslatorOpenAI - -# Initialize the translator with your OpenAI API key -translator = TranslatorOpenAI(api_key="YOUR_OPENAI_API_KEY", model="gpt-4o-mini") - -# Detect language -detected_language = translator.get_text_language("jak ty się nazywasz") -if detected_language is not None: - print(detected_language.ISO_639_1_code) # Output: 'pl' - print(detected_language.ISO_639_2_code) # Output: 'pol' - print(detected_language.ISO_639_3_code) # Output: 'pol' - print(detected_language.language_name) # Output 'Polish' - -# Translate text -translated_text = translator.translate( - text="Cześć jak się masz? Meu nome é Adam", - to_language="en" -) -print(translated_text) # Output: "Hello how are you? My name is Adam" - -``` - -### Available OpenAI Models for Translation -The llmtranslate library provides access to various OpenAI models for translation. Below are the supported models and their use cases: - -```python -from llmtranslate import TranslatorOpenAI - -# Recommended for precise translation, high-precision model -translator = TranslatorOpenAI(api_key="YOUR_OPENAI_API_KEY", model="gpt-4o") - -# A budget-friendly option, balancing cost and quality -translator = TranslatorOpenAI(api_key="YOUR_OPENAI_API_KEY", model="gpt-4o-mini") - -``` - - -## Using Asynchronous Methods - -The `llmtranslate` library provides asynchronous methods to allow you to perform language detection and translation tasks efficiently in an async environment. If your application uses `asyncio` or another asynchronous framework, you can take full advantage of these async methods to avoid blocking your program while waiting for language detection or translation tasks to complete. - -### Example of Using Asynchronous Methods - -The following example demonstrates how to use the `async_get_text_language` and `async_translate_text` methods: - -```python -import asyncio -from llmtranslate import TranslatorOpenAI - -# Initialize the translator with your OpenAI API key -translator = TranslatorOpenAI(api_key="YOUR_OPENAI_API_KEY", model="gpt-4o-mini") - - -# Async function to detect language and translate text -async def detect_and_translate(): - # Detect language asynchronously - detected_language = await translator.async_get_text_language("Hola, ¿cómo estás?") - if detected_language is not None: - print(detected_language.ISO_639_1_code) # Output: 'es' - print(detected_language.language_name) # Output: 'Spanish' - - # Translate text asynchronously - translated_text = await translator.async_translate( - text="Cześć jak się masz? Meu nome é Adam", - to_language="en" # Use ISO 639-1 code for the target language - ) - print(translated_text) # Output: "Hello how are you? My name is Adam" - - -# Run the async function -asyncio.run(detect_and_translate()) -``` - -### Key Asynchronous Methods - -1. **`async_get_text_language(text: str)`**: - This method detects the language of the provided text asynchronously. - - **Parameters**: - - `text`: The input text whose language needs to be detected. - - **Returns**: A `TextLanguage` object containing the detected language's ISO 639-1, ISO 639-2, ISO 639-3 codes, and the language name. - - **Example**: - ```python - detected_language = await translator.async_get_text_language("Hallo, wie geht's?") - ``` - -2. **`async_translate_text(text: str, to_language: str)`**: - This method translates the input text asynchronously to the specified target language. - - **Parameters**: - - `text`: The input text to be translated. - - `to_language`: The target language in ISO 639-1 code. - - **Returns**: A string containing the translated text. - - **Example**: - ```python - translated_text = await translator.async_translate("Bonjour tout le monde", "en") - ``` - -### Why Use Asynchronous Methods? - -Using asynchronous methods allows your application to handle multiple tasks concurrently, improving efficiency, especially when dealing with large amounts of text or performing multiple translations simultaneously. This non-blocking behavior is ideal for web services, APIs, and any scenario requiring high responsiveness. - -### Running Asynchronous Functions - -Remember that asynchronous methods must be called within an `async` function. To execute them, you can use `asyncio.run()` as shown in the examples above. diff --git a/docs/docs/Tutorial/TranslatorMistralCloud.md b/docs/docs/Tutorial/TranslatorMistralCloud.md index b7776ca..e69de29 100644 --- a/docs/docs/Tutorial/TranslatorMistralCloud.md +++ b/docs/docs/Tutorial/TranslatorMistralCloud.md @@ -1,115 +0,0 @@ - -# TranslatorMistralCloud Class Documentation - -The `TranslatorMistralCloud` class is fully compatible with both `TranslatorOpenSourceLLM` and `TranslatorOpenAI` classes. It contains all their methods and uses models hosted on the Mistral cloud for translation, language detection, and other tasks, providing the same functionality with cloud-based execution. - - -## How to Use - -### Initialization - -To use the `TranslatorMistralCloud` class, initialize it with your Mistral API key. Optionally, you can specify a model, with `MISTRAL_LARGE` as the default. - -```python -from llmtranslate import TranslatorMistralCloud, ModelForTranslator - -translator = TranslatorMistralCloud( - api_key="your_api_key_here", - model="open-mistral-nemo" -) -``` - -### Choose Mistral model -The `TranslatorMistralCloud` class allows you to choose a translation model when initializing. By default, it uses `MISTRAL_LARGE`, but you can specify other models using the `ModelForTranslator` enum or as a string. - -**Example using Enum:** -```python -from llmtranslate import TranslatorMistralCloud, ModelForTranslator - -translator = TranslatorMistralCloud( - api_key="your_api_key_here", - model=ModelForTranslator.MISTRAL_NEMO -) -``` - -**Example using String** -```python -from llmtranslate import TranslatorMistralCloud, ModelForTranslator - -translator = TranslatorMistralCloud( - api_key="your_api_key_here", - model="open-mistral-nemo" -) - -``` - - - - -### Translate - - -Translates the provided text into the specified language using the ISO 639-1 code. - -- **Parameters:** - - `text`: The text to translate. - - `to_language`: Target language (default is English: `"eng"`). - - -```python -from llmtranslate import TranslatorMistralCloud -translator = TranslatorMistralCloud( - api_key="your_api_key_here", - model="open-mistral-nemo" -) -translated_text = translator.translate("Bonjour", "en") -print(translated_text) # Output: Hello -``` - -### Detect language - -Detects the language of the given text and returns its ISO 639-1 code. - - -```python -from llmtranslate import TranslatorMistralCloud -translator = TranslatorMistralCloud( - api_key="your_api_key_here", - model="open-mistral-nemo" -) -detected_language = translator.get_text_language("jak ty się nazywasz") -if detected_language is not None: - print(detected_language.ISO_639_1_code) # Output: 'pl' - print(detected_language.ISO_639_2_code) # Output: 'pol' - print(detected_language.ISO_639_3_code) # Output: 'pol' - print(detected_language.language_name) # Output 'Polish' -``` - -!!! warning - If the translator does not detect any language, it will return None.
- Before using results of translator detection you should check if it returned correct result or None - - - -## Example Usage - -```python -from llmtranslate import TranslatorMistralCloud -translator = TranslatorMistralCloud( - api_key="your_api_key_here", - model="open-mistral-nemo" -) - -# Translate text to English -translated_text = translator.translate("こんにちは", "en") -print(translated_text) # Output: Hello - -# Detect language -detected_language = translator.get_text_language("jak ty się nazywasz") -if detected_language is not None: - print(detected_language.ISO_639_1_code) # Output: 'pl' - print(detected_language.ISO_639_2_code) # Output: 'pol' - print(detected_language.ISO_639_3_code) # Output: 'pol' - print(detected_language.language_name) # Output 'Polish' - -``` diff --git a/docs/docs/Tutorial/TranslatorOpenSource.md b/docs/docs/Tutorial/TranslatorOpenSource.md index e6fb62b..e69de29 100644 --- a/docs/docs/Tutorial/TranslatorOpenSource.md +++ b/docs/docs/Tutorial/TranslatorOpenSource.md @@ -1,142 +0,0 @@ -# TranslatorOpenSource Class Documentation - -The `TranslatorOpenSource` class is designed for language detection and translation using open-source large language models (LLMs) deployed on a custom endpoint. This class enables interaction with these models to detect the language of text and translate between languages. - -## Initialization - -```python -from llmtranslate import TranslatorOpenSource - -# Create the translator object -translator = TranslatorOpenSource( - api_key="YOUR_API_KEY", - llm_endpoint="YOUR_LLM_ENDPOINT", - model="mistralai/Mistral-Nemo-Instruct-2407" -) -``` - - -The constructor initializes the `TranslatorOpenSource` class by setting up the API key, endpoint, and large language model (LLM). It also configures the maximum text length limits depending on the model. - -### Parameters: - -- **`api_key`**: - A string representing the API key that authenticates access to the LLM service. This key ensures that the service can securely connect to your model deployment. - -- **`llm_endpoint`**: - A string representing the endpoint URL where the LLM model is deployed. This is usually a REST API endpoint provided by your deployment platform (e.g., Hugging Face or other open-source model deployment platforms). - -- **`model`**: - A string representing the name of the model used for translation. In this case, the `mistralai/Mistral-Nemo-Instruct-2407` model is used. This model determines the capabilities and performance of the translation system. - - - For models in `MINI_MODELS`: - - `max_length = 30`: Limits the maximum length of text chunks that can be translated in a single request. - - `max_length_mini_text_chunk = 20`: Limits the maximum length of mini text chunks for smaller translations. - - For larger models (those not in `MINI_MODELS`): - - `max_length = 100`: Allows larger text chunks for translation. - - `max_length_mini_text_chunk = 50`: Allows larger mini text chunks for more efficient translation handling. - - - -## Example Usage - -### `get_text_language()` - -Detects the language of the provided text synchronously. - -```python -from llmtranslate import TranslatorOpenSource - -# Create the translator object -translator = TranslatorOpenSource( - api_key="YOUR_API_KEY", - llm_endpoint="YOUR_LLM_ENDPOINT", - model="mistralai/Mistral-Nemo-Instruct-2407" -) - -# Detect the language of a given text -detected_language = translator.get_text_language("Bonjour tout le monde") - -if detected_language is not None: - print(detected_language.ISO_639_1_code) # Output: 'fr' - print(detected_language.ISO_639_2_code) # Output: 'fra' - print(detected_language.ISO_639_3_code) # Output: 'fra' - print(detected_language.language_name) # Output: 'French' -``` - -### `async_get_text_language()` - -Detects the language of the provided text asynchronously. - -```python -import asyncio -from llmtranslate import TranslatorOpenSource - -# Create the translator object -translator = TranslatorOpenSource( - api_key="YOUR_API_KEY", - llm_endpoint="YOUR_LLM_ENDPOINT", - model="mistralai/Mistral-Nemo-Instruct-2407" -) - -# Async function to detect language -async def detect_language_async(): - detected_language = await translator.async_get_text_language("Hola, ¿cómo estás?") - if detected_language is not None: - print(detected_language.ISO_639_1_code) # Output: 'es' - print(detected_language.language_name) # Output: 'Spanish' - -# Run the async function -asyncio.run(detect_language_async()) -``` - -### `translate()` - -Translates text synchronously from one language to another. - -```python -from llmtranslate import TranslatorOpenSource - -# Create the translator object -translator = TranslatorOpenSource( - api_key="YOUR_API_KEY", - llm_endpoint="YOUR_LLM_ENDPOINT", - model="mistralai/Mistral-Nemo-Instruct-2407" -) - -# Translate text from one language to another -translated_text = translator.translate( - text="Hallo, wie geht's?", - to_language="en" # Target language in ISO 639-1 format -) -print(translated_text) # Output: "Hello, how are you?" -``` - -### `async_translate()` - -Translates text asynchronously from one language to another. - -```python -import asyncio -from llmtranslate import TranslatorOpenSource - -# Create the translator object -translator = TranslatorOpenSource( - api_key="YOUR_API_KEY", - llm_endpoint="YOUR_LLM_ENDPOINT", - model="mistralai/Mistral-Nemo-Instruct-2407" -) - - -# Async function to translate text -async def translate_text_async(): - translated_text = await translator.async_translate( - text="Cześć, jak się masz?", - to_language="en" # Target language in ISO 639-1 format - ) - print(translated_text) # Output: "Hello, how are you?" - - -# Run the async function -asyncio.run(translate_text_async()) -``` diff --git a/docs/docs/index.md b/docs/docs/index.md index 9364ee4..6b7faf3 100644 --- a/docs/docs/index.md +++ b/docs/docs/index.md @@ -7,223 +7,146 @@ [![codecov](https://codecov.io/github/adam-pawelek/llmtranslate/graph/badge.svg?token=WCQOJC032S)](https://codecov.io/github/adam-pawelek/llmtranslate) [![Downloads](https://static.pepy.tech/badge/llmtranslate)](https://pepy.tech/project/llmtranslate) -## Overview -llmtranslate is a Python library designed to identify the language of a given text and translate text between multiple languages using OpenAI's GPT-4o. The library is especially useful for translating text containing multiple languages into a single target language. - -## Features - -- **Language Detection:** Identify the language of a given text in ISO 639-1 format. -- **Translation:** Translate text containing multiple languages into another language in ISO 639-1 format. - -## Documentation -Comprehensive documentation, including detailed usage information, is available at https://llm-translate.com - -## Requirements - -To use this library, you must have an OpenAI API key. This key allows the library to utilize OpenAI's GPT-4o for translation and language detection. +The `llmtranslate` library is a Python tool that leverages LangChain's [ChatModels](https://python.langchain.com/docs/integrations/chat/) to translate text between languages and recognize the language of a given text. This library provides both synchronous and asynchronous translators to accommodate various use cases. +📖 **Documentation:** [llm-translate.com](https://llm-translate.com/) ## Installation -You can install the llmtranslate library from PyPI: - -```bash -pip install llmtranslate -``` - -## Usage - -### Setting the OpenAI API Key - -Before using llmtranslate with OpenAI, you need to set your OpenAI API key. You can do this by creating an instance of the TranslatorOpenAI class. - -```python -from llmtranslate import TranslatorOpenAI - -# Set your OpenAI API key -translator = TranslatorOpenAI(api_key="YOUR_OPENAI_API_KEY", model="gpt-4o-mini") - -``` - -### Language Detection - -To detect the language of a given text: - -```python -from llmtranslate import TranslatorOpenAI - -# Set your OpenAI API key -translator = TranslatorOpenAI(api_key="YOUR_OPENAI_API_KEY", model="gpt-4o-mini") - -# Detect language -detected_language = translator.get_text_language("Hello world") -if detected_language is not None: - print(detected_language.ISO_639_1_code) # Output: 'en' - print(detected_language.ISO_639_2_code) # Output: 'eng' - print(detected_language.ISO_639_3_code) # Output: 'eng' - print(detected_language.language_name) # Output: 'English' - -``` - -!!! warning - If the translator does not detect any language, it will return None.
- Before using results of translator detection you should check if it returned correct result or None +To ensure proper dependency management, install the required libraries in the following order: +1. Install LangChain chat libraries for your preferred LLM. For example: + ```bash + pip install langchain-openai + ``` -### Translation - -To translate text containing multiple languages into another language, you need to provide the ISO 639 language code for the target language. For a list of all ISO 639 language codes, you can refer to this [ISO 639-1 code list website](https://localizely.com/iso-639-1-list/). - -```python -from llmtranslate import TranslatorOpenAI - -# Set your OpenAI API key -translator = TranslatorOpenAI(api_key="YOUR_OPENAI_API_KEY", model="gpt-4o-mini") - -# Translate text -translated_text = translator.translate( - text="Cześć jak się masz? Meu nome é Adam", - to_language="en" # Use ISO 639-1 code for the target language -) -print(translated_text) # Output: "Hello how are you? My name is Adam" -``` - +2. Install the `llmtranslate` library: + ```bash + pip install llmtranslate + ``` -### Full Example +--- -Here is a complete example demonstrating how to use the library: +## Example Using OpenAI's GPT +### Using `Translator` (Synchronous) ```python -from llmtranslate import TranslatorOpenAI +from llmtranslate import Translator +from langchain_openai import ChatOpenAI -# Initialize the translator with your OpenAI API key -translator = TranslatorOpenAI(api_key="YOUR_OPENAI_API_KEY", model="gpt-4o-mini") +# Initialize the LLM and Translator +llm = ChatOpenAI(model_name="gpt-4o", openai_api_key="your_openai_api_key") +translator = Translator(llm=llm) -# Detect language -detected_language = translator.get_text_language("jak ty się nazywasz") -if detected_language is not None: - print(detected_language.ISO_639_1_code) # Output: 'pl' - print(detected_language.ISO_639_2_code) # Output: 'pol' - print(detected_language.ISO_639_3_code) # Output: 'pol' - print(detected_language.language_name) # Output 'Polish' +# Detect the language of the text +text_language = translator.get_text_language("Hi how are you?") +if text_language: + print(text_language.ISO_639_1_code) # Output: en + print(text_language.ISO_639_2_code) # Output: eng + print(text_language.ISO_639_3_code) # Output: eng + print(text_language.language_name) # Output: English # Translate text -translated_text = translator.translate( - text="Cześć jak się masz? Meu nome é Adam", - to_language="en" -) -print(translated_text) # Output: "Hello how are you? My name is Adam" - +translated_text = translator.translate("Bonjour tout le monde", "English") +print(translated_text) # Output: Hello everyone ``` -### Available OpenAI Models for Translation -The llmtranslate library provides access to various OpenAI models for translation. Below are the supported models and their use cases: +--- - -```python -from llmtranslate import TranslatorOpenAI - -# Recommended for precise translation, high-precision model -translator = TranslatorOpenAI(api_key="YOUR_OPENAI_API_KEY", model="gpt-4o") - -# A budget-friendly option, balancing cost and quality -translator = TranslatorOpenAI(api_key="YOUR_OPENAI_API_KEY", model="gpt-4o-mini") - -``` - -## Using Asynchronous Methods - -The `llmtranslate` library provides asynchronous methods to allow you to perform language detection and translation tasks efficiently in an async environment. If your application uses `asyncio` or another asynchronous framework, you can take full advantage of these async methods to avoid blocking your program while waiting for language detection or translation tasks to complete. - -### Example of Using Asynchronous Methods - -The following example demonstrates how to use the `async_get_text_language` and `async_translate_text` methods: +### Using `AsyncTranslator` (Asynchronous) ```python import asyncio -from llmtranslate import TranslatorOpenAI +from llmtranslate import AsyncTranslator +from langchain_openai import ChatOpenAI -# Initialize the translator with your OpenAI API key -translator = TranslatorOpenAI(api_key="YOUR_OPENAI_API_KEY", model="gpt-4o-mini") +# Initialize the LLM and AsyncTranslator +llm = ChatOpenAI(model_name="gpt-4o", openai_api_key="your_openai_api_key") -# Async function to detect language and translate text -async def detect_and_translate(): - # Detect language asynchronously - detected_language = await translator.async_get_text_language("Hola, ¿cómo estás?") - if detected_language is not None: - print(detected_language.ISO_639_1_code) # Output: 'es' - print(detected_language.language_name) # Output: 'Spanish' - - # Translate text asynchronously - translated_text = await translator.async_translate( - text="Cześć jak się masz? Meu nome é Adam", - to_language="en" # Use ISO 639-1 code for the target language +async def translate_text(): + translator = AsyncTranslator( + llm=llm, + max_translation_chunk_length=100, + max_translation_chunk_length_multilang=50, + max_concurrent_llm_calls=10 ) - print(translated_text) # Output: "Hello how are you? My name is Adam" - - -# Run the async function -asyncio.run(detect_and_translate()) + tasks = [ + translator.get_text_language("Hi how are you?"), + translator.translate("Hi how are you?", "Spanish") + ] + results = await asyncio.gather(*tasks) + # Output the detected language information + text_language = results[0] + if results: + print(text_language.ISO_639_1_code) # Output: en + print(text_language.ISO_639_2_code) # Output: eng + print(text_language.ISO_639_3_code) # Output: eng + print(text_language.language_name) # Output: English + + # Output the translated text + print(results[1]) # Output: Hola, ¿cómo estás? + + +# Run the asynchronous translation +asyncio.run(translate_text()) ``` -### Key Asynchronous Methods - -1. **`async_get_text_language(text: str)`**: - This method detects the language of the provided text asynchronously. - - **Parameters**: - - `text`: The input text whose language needs to be detected. - - **Returns**: A `TextLanguage` object containing the detected language's ISO 639-1, ISO 639-2, ISO 639-3 codes, and the language name. - - **Example**: - ```python - detected_language = await translator.async_get_text_language("Hallo, wie geht's?") - ``` - -2. **`async_translate_text(text: str, to_language: str)`**: - This method translates the input text asynchronously to the specified target language. - - **Parameters**: - - `text`: The input text to be translated. - - `to_language`: The target language in ISO 639-1 code. - - **Returns**: A string containing the translated text. - - **Example**: - ```python - translated_text = await translator.async_translate("Bonjour tout le monde", "en") - ``` - -### Why Use Asynchronous Methods? - -Using asynchronous methods allows your application to handle multiple tasks concurrently, improving efficiency, especially when dealing with large amounts of text or performing multiple translations simultaneously. This non-blocking behavior is ideal for web services, APIs, and any scenario requiring high responsiveness. - -### Running Asynchronous Functions - -Remember that asynchronous methods must be called within an `async` function. To execute them, you can use `asyncio.run()` as shown in the examples above. - +--- +## Key Parameters +### `max_translation_chunk_length` +- **Description**: Defines the maximum length (in characters) of a text chunk to be translated in a single call when the text is in one language. +- **Recommendation**: If translations are not accurate, try reducing this value as weaker LLMs struggle with large chunks of text. +### `max_translation_chunk_length_multilang` +- **Description**: Defines the maximum length (in characters) of a text chunk when the text contains multiple languages. +- **Recommendation**: Reduce this value for better accuracy with multi-language inputs. +### `max_concurrent_llm_calls` +- **Description**: Limits the number of concurrent calls made to the LLM. +- **Default**: 10 +- **Usage**: Adjust this parameter to align with your LLM provider's concurrency limits. +--- -### Setting the Azure OpenAI API Key -If you are using Azure's OpenAI services, you need to set your Azure OpenAI API key along with additional required parameters. Use the TranslatorAzureOpenAI class for this. +## Other Supported LLMs Examples +You can find more examples of LangChain Chat models in the documentation at: +- [llm-translate.com](https://llm-translate.com/) +- [LangChain Documentation](https://python.langchain.com/docs/integrations/chat/) +### Anthropic's Claude +To create an instance of an Anthropic-based Chat LLM: +```bash + pip install langchain-anthropic + pip install llmtranslate +``` ```python -from llmtranslate import TranslatorAzureOpenAI +from langchain_anthropic import Anthropic +from llmtranslate import Translator +llm = Anthropic(model="claude-2", anthropic_api_key="your_anthropic_api_key") +translator = Translator(llm=llm) +``` -# Set your Azure OpenAI API key and related parameters -translator = TranslatorAzureOpenAI( - azure_endpoint="YOUR_AZURE_ENDPOINT", - api_key="YOUR_AZURE_API_KEY", - api_version="YOUR_API_VERSION", - azure_deployment="YOUR_AZURE_DEPLOYMENT" -) +### Mistral via API +To create an instance of a Chat LLM using the Mistral API: +```bash +pip install -qU langchain_mistralai +pip install llmtranslate +``` +```python +from langchain_mistral import MistralChat +from llmtranslate import Translator +llm = MistralChat(api_key="your_mistral_api_key") +translator = Translator(llm=llm) ``` +--- + ## Supported Languages diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml index 4132d28..896af21 100644 --- a/docs/mkdocs.yml +++ b/docs/mkdocs.yml @@ -13,11 +13,7 @@ nav: - Supported languages: - gpt-4o: Supported languages/gpt-4o.md - gpt-4o-mini: Supported languages/gpt-4o-mini.md - - Tutorial: - - Quick Start Guide - OpenAI: Tutorial/Quick Start Guide.md - - Choosing OpenAI Model for Translation: Tutorial/Choosing OpenAI Model for Translation.md - - TranslatorMistralCloud: Tutorial/TranslatorMistralCloud.md - - TranslatorOpenSource: Tutorial/TranslatorOpenSource.md + markdown_extensions: