Evaluation library for your LLM applications
Ragas supercharges your LLM application evaluations with tools to objectively measure performance, synthesize test case scenarios, and gain insights by leveraging production data.
Evaluating and testing LLM applications is a challenging, time-consuming, and often boring process. Ragas aims provide a suite of tools that could supercharge your evaluation workflows and make it more efficient and fun using state-of-the-art research. We are also building an open ecosystem, that fosters sharing of ideas to make the evaluation process better and collaborates with other tools in the market to make it a seamless experience.
- Metrics: Different LLM based and non LLM based metrics to objectively evaluate your LLM applications such as RAG, Agents, etc.
- Test Data Generation: Synthesize high-quality datasets covering wide variety of scenarios for comprehensive testing of your LLM applications.
- Integrations: Seamless integration with all major LLM applications frameworks like langchain and observability tools.
From release:
pip install ragasAlternatively, from source:
pip install git+https://github.com/explodinggradients/ragasIf you want to get more involved with Ragas, check out our discord server. It's a fun community where we geek out about LLM, Retrieval, Production issues, and more.
We track very basic usage metrics to guide us to figure out what our users want, what is working, and what's not. As a young startup, we have to be brutally honest about this which is why we are tracking these metrics. But as an Open Startup, we open-source all the data we collect. You can read more about this here. Ragas does not track any information that can be used to identify you or your company. You can take a look at exactly what we track in the code
To disable usage-tracking you set the RAGAS_DO_NOT_TRACK flag to true.
