# The Eval Index

> The living leaderboard of LLM and agent evaluation & benchmark tooling, ranked
> daily by momentum (stars, push-recency, rising-newness) from live GitHub signals.

Updated: 2026-07-21T10:52:00.152769+00:00
Tools indexed: 274

## Top eval tools by momentum

- [langfuse/langfuse](https://github.com/langfuse/langfuse) — momentum 87, ⭐31562 — Observability — 🪢 Open source AI engineering platform: LLM evals, observability, metrics, prompt management, playgro
- [mlflow/mlflow](https://github.com/mlflow/mlflow) — momentum 86, ⭐27134 — Observability — The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all
- [promptfoo/promptfoo](https://github.com/promptfoo/promptfoo) — momentum 85, ⭐23456 — Red Teaming & Safety — Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare p
- [comet-ml/opik](https://github.com/comet-ml/opik) — momentum 85, ⭐20743 — Observability — Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehe
- [openobserve/openobserve](https://github.com/openobserve/openobserve) — momentum 85, ⭐20321 — Observability — Open source observability platform for logs, metrics, traces, frontend monitoring, pipelines and LLM
- [Tencent/WeKnora](https://github.com/Tencent/WeKnora) — momentum 84, ⭐18668 — RAG Eval — Open-source LLM knowledge platform: turn raw documents into a queryable RAG, an autonomous reasoning
- [confident-ai/deepeval](https://github.com/confident-ai/deepeval) — momentum 84, ⭐16996 — Eval Frameworks — The LLM Evaluation Framework
- [dataelement/bisheng](https://github.com/dataelement/bisheng) — momentum 82, ⭐11725 — Observability — BISHENG is an open LLM devops platform for next generation Enterprise AI applications. Powerful and 
- [Sahir619/fable-method](https://github.com/Sahir619/fable-method) — momentum 82, ⭐1765 — Agent Eval — The Fable Workflow: how Claude Fable 5 worked, distilled into skills any model can run, with the eva
- [Arize-ai/phoenix](https://github.com/Arize-ai/phoenix) — momentum 81, ⭐10650 — Observability — AI Observability & Evaluation
- [oumi-ai/oumi](https://github.com/oumi-ai/oumi) — momentum 80, ⭐9363 — Eval Frameworks — Easily fine-tune, evaluate and deploy Gemma 4, Qwen3.5, Qwen3.6, gpt-oss, DeepSeek-R1, or any open s
- [NVIDIA/garak](https://github.com/NVIDIA/garak) — momentum 79, ⭐8516 — Eval Frameworks — the LLM vulnerability scanner
- [open-compass/opencompass](https://github.com/open-compass/opencompass) — momentum 79, ⭐7219 — Benchmarks — OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, Inter
- [katanemo/plano](https://github.com/katanemo/plano) — momentum 79, ⭐6882 — Observability — Plano is an AI-native proxy server and data plane for agentic apps. Smart LLM routing, observability
- [traceloop/openllmetry](https://github.com/traceloop/openllmetry) — momentum 78, ⭐7316 — Observability — Open-source observability for your GenAI or LLM application, based on OpenTelemetry
- [jeinlee1991/chinese-llm-benchmark](https://github.com/jeinlee1991/chinese-llm-benchmark) — momentum 78, ⭐6301 — Agent Eval — 非线智能 NoneLinear - ReLE评测：中文AI大模型能力评测（持续更新）：目前已囊括374个大模型，覆盖chatgpt、gpt-5.4、谷歌gemini-3.1-pro、Claude-4.
- [Giskard-AI/giskard-oss](https://github.com/Giskard-AI/giskard-oss) — momentum 78, ⭐5662 — Red Teaming & Safety — 🐢 Open-Source Evaluation & Testing library for LLM Agents
- [coze-dev/coze-loop](https://github.com/coze-dev/coze-loop) — momentum 78, ⭐5628 — Observability — Next-generation AI Agent Optimization Platform: Cozeloop addresses challenges in AI agent developmen
- [GoogleCloudPlatform/agent-starter-pack](https://github.com/GoogleCloudPlatform/agent-starter-pack) — momentum 77, ⭐6520 — Observability — Ship AI Agents to Google Cloud in minutes, not months. Production-ready templates with built-in CI/C
- [Kiln-AI/Kiln](https://github.com/Kiln-AI/Kiln) — momentum 77, ⭐4968 — RAG Eval — Build, Evaluate, and Optimize AI Systems. Includes evals, RAG, agents, fine-tuning, synthetic data g
- [Marker-Inc-Korea/AutoRAG](https://github.com/Marker-Inc-Korea/AutoRAG) — momentum 77, ⭐4934 — RAG Eval — AutoRAG: Now your agent can find anything in your computer. It gets smarter if you are using it freq
- [Andyyyy64/whichllm](https://github.com/Andyyyy64/whichllm) — momentum 76, ⭐5916 — Benchmarks — Find the local LLM that actually runs and performs best on your hardware. Ranked by real, recency-aw
- [pydantic/logfire](https://github.com/pydantic/logfire) — momentum 76, ⭐4381 — Observability — AI observability platform for production LLM and agent systems.
- [Agenta-AI/agenta](https://github.com/Agenta-AI/agenta) — momentum 76, ⭐4319 — Observability — The open-source workspace for building and running AI agents. Build agents through chat, share them 
- [open-compass/VLMEvalKit](https://github.com/open-compass/VLMEvalKit) — momentum 76, ⭐4294 — Benchmarks — Open-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchma
- [Tencent/AI-Infra-Guard](https://github.com/Tencent/AI-Infra-Guard) — momentum 76, ⭐4163 — Red Teaming & Safety — A full-stack AI Red Teaming platform securing AI ecosystems via OpenClaw Security Scan, Agent Scan, 
- [tensorzero/tensorzero](https://github.com/tensorzero/tensorzero) — momentum 75, ⭐11693 — Observability — TensorZero is an open-source LLMOps platform that unifies an LLM gateway, observability, evaluation,
- [Helicone/helicone](https://github.com/Helicone/helicone) — momentum 75, ⭐5975 — Observability — 🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC 
- [EvolvingLMMs-Lab/lmms-eval](https://github.com/EvolvingLMMs-Lab/lmms-eval) — momentum 75, ⭐4324 — Benchmarks — One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks
- [truera/trulens](https://github.com/truera/trulens) — momentum 75, ⭐3450 — Observability — Evaluation and Tracking for LLM Experiments and AI Agents
- [langwatch/langwatch](https://github.com/langwatch/langwatch) — momentum 75, ⭐3411 — Observability — The platform for LLM evaluations and AI agent testing
- [harbor-framework/harbor](https://github.com/harbor-framework/harbor) — momentum 75, ⭐3342 — Agent Eval — Framework for evaluating and improving agents
- [modelscope/evalscope](https://github.com/modelscope/evalscope) — momentum 75, ⭐3115 — RAG Eval — A streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and p
- [lmnr-ai/lmnr](https://github.com/lmnr-ai/lmnr) — momentum 75, ⭐3103 — Observability — Laminar - open-source observability platform purpose-built for AI agents. YC S24.
- [FreedomIntelligence/Awesome-AI4Med](https://github.com/FreedomIntelligence/Awesome-AI4Med) — momentum 74, ⭐2841 — Benchmarks — A curated list of medical LLMs, multimodal systems, datasets, benchmarks, and more. 🏥
- [openlit/openlit](https://github.com/openlit/openlit) — momentum 74, ⭐2628 — Observability — Open source platform for AI Engineering: OpenTelemetry-native LLM Observability, GPU Monitoring, Gua
- [ifixai-ai/iFixAi](https://github.com/ifixai-ai/iFixAi) — momentum 74, ⭐1586 — Red Teaming & Safety — Catch your AI's mistakes and blind spots before your customers or regulators do. iFixAi runs 45 insp
- [future-agi/future-agi](https://github.com/future-agi/future-agi) — momentum 74, ⭐1467 — Observability — Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applicati
- [benchflow-ai/awesome-evals](https://github.com/benchflow-ai/awesome-evals) — momentum 74, ⭐743 — Agent Eval — A curated, non-BS library of the best resources for building and evaluating AI agents — papers, blog
- [AgentOps-AI/agentops](https://github.com/AgentOps-AI/agentops) — momentum 73, ⭐5719 — Observability — Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most