UpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding use-cases), perform root cause analysis on failure cases and give insights on how to resolve them.

[ACL 2024] User-friendly evaluation framework: Eval Suite & Benchmarks: UHGEval, HaluEval, HalluQA, etc.

Code for the EMNLP 2024 paper "Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps"

An Easy-to-use Hallucination Detection Framework for LLMs.

Official repo for the paper PHUDGE: Phi-3 as Scalable Judge. Evaluate your LLMs with or without custom rubric, reference answer, absolute, relative and much more. It contains a list of all the available tool, methods, repo, code etc to detect hallucination, LLM evaluation, grading and much more.

[ACL 2024] ANAH & [NeurIPS 2024] ANAH-v2

VideoHallucer, The first comprehensive benchmark for hallucination detection in large video-language models (LVLMs)

up-to-date curated list of state-of-the-art Large vision language models hallucinations research work, papers & resources

[ACL 2024] An Easy-to-use Hallucination Detection Framework for LLMs.

This repo hosts the Python SDK and related examples for AIMon, which is a proprietary, state-of-the-art system for detecting LLM quality issues such as Hallucinations. It can be used during offline evals, continuous monitoring or inline detection. We offer various model quality metrics that are fast, reliable and cost-effective.

Detecting Hallucinations in Large Language Model Generations using Graph Structures

Fact-checking with Iterative Retrieval and Verification

Fully automated LLM evaluator

Chrome extension for the ATLAS project.

Antibodies for LLMs hallucinations (grouping LLM as a judge, NLI, reward models)

Binary hallucination detection classifier using logistic regression

🔢Hallucination detector for Large Language Models.

Competition: SemEval-2024 Task-6 - SHROOM, a Shared-task on Hallucinations and Related Observable Overgeneration Mistakes

Different approaches to evaluate RAG !!!