[NeurIPS 2025] BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models
-
Updated
Sep 20, 2025 - Python
#
llms-benchmarking
Here are 44 public repositories matching this topic...
Benchmark that evaluates LLMs using 759 NYT Connections puzzles extended with extra trick words
-
Updated
Sep 20, 2025 - Python
How good are LLMs at chemistry?
-
Updated
Sep 11, 2025 - Python
Python SDK for experimenting, testing, evaluating & monitoring LLM-powered applications - Parea AI (YC S23)
metrics good-first-issue llm prompt-engineering generative-ai llmops llm-eval llm-tools llm-evaluation llm-evaluation-toolkit llms-benchmarking llm-evaluation-framework
-
Updated
Feb 13, 2025 - Python
Language Model for Mainframe Modernization
-
Updated
Aug 23, 2024 - Python
Develop reliable AI apps
-
Updated
Sep 2, 2025 - Python
-
Updated
Mar 14, 2025 - Python
Training and Benchmarking LLMs for Code Preference.
-
Updated
Nov 15, 2024 - Python
The data and implementation for the experiments in the paper "Flows: Building Blocks of Reasoning and Collaborating AI".
ai competitive-programming agents competitive-programming-contests competitive-coding llms llms-reasoning llms-benchmarking aiflows
-
Updated
Feb 12, 2024 - Python
Restore safety in fine-tuned language models through task arithmetic
-
Updated
Mar 28, 2024 - Python
A minimalist benchmarking tool designed to test the routine-generation capabilities of LLMs.
-
Updated
Nov 28, 2024 - Python
🐝 SwarmBench: Benchmarking LLMs' Swarm Intelligence
-
Updated
May 21, 2025 - Python
Plancraft is a minecraft environment and agent suite to test planning capabilities in LLMs
minecraft planning interactive-environments llm multimodal-large-language-models llms-benchmarking agentic-ai
-
Updated
Jul 9, 2025 - Python
Join 15k builders to the Real-World ML Newsletter ⬇️⬇️⬇️
-
Updated
Apr 19, 2024 - Python
FaceXBench: Evaluating Multimodal LLMs on Face Understanding
attributes face face-recognition gender-classification facial-expression-recognition race-detection age-estimation crowd-counting face-antispoofing multimodal face-segmentation face-perception deepfake-detection headpose-estimation llms llms-benchmarking multimodal-llms
-
Updated
Feb 4, 2025 - Python
Belief-Bias evaluation of local LLMs
-
Updated
Jul 3, 2025 - Python
The MERIT Dataset is a fully synthetic, labeled dataset created for training and benchmarking LLMs on Visually Rich Document Understanding tasks. It is also designed to help detect biases and improve interpretability in LLMs, where we are actively working. This repository is actively maintained, and new features are continuously being added.
donut biases synthetic-dataset-generation layoutlm synthetic-dataset layoutxlm token-classification layoutlmv3 layoutlmv2 llms-benchmarking idefics2
-
Updated
Jul 16, 2025 - Python
A platform that enables users to perform private benchmarking of machine learning models. The platform facilitates the evaluation of models based on different trust levels between the model owners and the dataset owners.
platform benchmarking inference secure private mpc contamination trusted-execution-environment confidential-computing large-language-models llms-benchmarking private-benchmarking ezpc
-
Updated
Sep 16, 2024 - Python
Official repository for "RoMath: A Mathematical Reasoning Benchmark in 🇷🇴 Romanian 🇷🇴"
-
Updated
Feb 14, 2025 - Python
Improve this page
Add a description, image, and links to the llms-benchmarking topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the llms-benchmarking topic, visit your repo's landing page and select "manage topics."