Skip to content

This project aims to evaluate and compare different Large Language Models (LLMs) for the task of extracting email signature information and structuring it into a JSON format. The evaluated models are OpenAI GPT-3.5 turbo and Anthropic Claude 3. The project includes prompt engineering, testing, and iteration to achieve the best results.

Notifications You must be signed in to change notification settings

edogola4/Email-Signature-Extraction-with-LLMs

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

7 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Email-Signature-Extraction-with-LLMs

Email Signature Extractor & Evaluator

This project is designed to evaluate the performance of different Large Language Models (LLMs) in extracting email signature information and structuring it into a JSON format.

Project Structure

  • config.py: Contains the OpenAI API key.
  • functions.py: Defines the functions used for prompt evaluation and result analysis.
  • main.py: The main script that runs the evaluation.

Setup

  1. Clone the repository:

    git clone https://github.com/your-repo/email_signature_extractor_evaluator.git
    cd email_signature_extractor_evaluator
  2. Set up a Python virtual environment:

    python3 -m venv .venv
    source .venv/bin/activate
  3. Install the required packages:

    pip install openai transformers
  4. Set your OpenAI API key in config.py:

    # config.py
    OPENAI_API_KEY = "your_openai_api_key_here"

Usage

Run the main script to evaluate the prompt against the test cases:

python main.py


Contact

For any questions or issues, please contact [brandon14ogola@gmail.com].

About

This project aims to evaluate and compare different Large Language Models (LLMs) for the task of extracting email signature information and structuring it into a JSON format. The evaluated models are OpenAI GPT-3.5 turbo and Anthropic Claude 3. The project includes prompt engineering, testing, and iteration to achieve the best results.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published