#3737th of 57 in Gateways and observability

trulens

Tracing and LLM-judge evaluation for agents and RAG apps on OpenTelemetry

Stars
3.7k
License
MIT
Last commit
Nov 2026
Last release
Oct 2026
Language
Python
Stars, last 29 weeks+398
25 Apr 2026: 3,262 stars2 May 2026: 3,279 stars9 May 2026: 3,295 stars16 May 2026: 3,317 stars23 May 2026: 3,330 stars30 May 2026: 3,344 stars6 Jun 2026: 3,352 stars13 Jun 2026: 3,371 stars20 Jun 2026: 3,385 stars27 Jun 2026: 3,396 stars4 Jul 2026: 3,411 stars11 Jul 2026: 3,429 stars18 Jul 2026: 3,439 stars25 Jul 2026: 3,453 stars1 Aug 2026: 3,472 stars8 Aug 2026: 3,489 stars15 Aug 2026: 3,506 stars22 Aug 2026: 3,514 stars29 Aug 2026: 3,524 stars5 Sept 2026: 3,534 stars12 Sept 2026: 3,545 stars19 Sept 2026: 3,562 stars26 Sept 2026: 3,574 stars3 Oct 2026: 3,590 stars10 Oct 2026: 3,597 stars17 Oct 2026: 3,603 stars24 Oct 2026: 3,613 stars31 Oct 2026: 3,625 stars7 Nov 2026: 3,638 stars14 Nov 2026: 3,660 stars

3,262 on 25 Apr 20263,660 on 14 Nov 2026

From GitHub's daily star counts.

Checked on GitHub.

Overview

TruLens instruments Python LLM apps with an `@instrument` decorator that records each step as an OpenTelemetry span, including latency, inputs, outputs, tokens and cost. It scores those spans with LLM-judge metrics, such as the RAG triad and seven agent evaluators, either inline as the app runs or in batch over a dataset. A leaderboard compares scores, latency and cost across app versions.

Who it is for: Engineers evaluating and debugging LLM agents and RAG apps in Python

Strengths

  • Traces are OTel spans, exportable to any OTLP backend such as Jaeger or Datadog
  • Seven agent evaluators, including plan adherence, tool selection and tool calling
  • Runs inline or as batch evals over a dataset via the Run API
  • Judge providers include OpenAI, Bedrock, Gemini, LiteLLM, HuggingFace and Snowflake Cortex

Weaknesses

  • Evaluation relies on LLM judges, so most setups need an external model provider
  • Python only; instrumentation is by decorator or framework package
  • Each provider and app integration is a separate pip package to install
  • Headline accuracy figures come from the maintainers' own papers and cited benchmarks

What it needs

  • no GPU
  • Needs LLM provider for judges (OpenAI, Bedrock, Gemini, LiteLLM, HuggingFace, Snowflake Cortex)
  • Models: OpenAI, Azure OpenAI, LiteLLM, Google Gemini, AWS Bedrock

What changed

  1. “Streamlit-in-Snowflake dashboard support has been removed. (#2784)”

From the release notes. No flag means the note does not say so, not that the upgrade is safe.

Also in Gateways and observability

See all 57
Also in Gateways and observability
RankProjectScore
1OmniRouteLLM gateway routing one OpenAI-compatible endpoint across 372 providers88.5k starsMIT96 out of 100
2headroomCompresses tool outputs, logs and history before they reach the LLM79.9k starsApache-2.092 out of 100
3new-apiGateway that unifies LLM providers behind OpenAI, Anthropic and Gemini APIs52.1k starsAGPL-3.0known advisory in latest version91 out of 100
#4MLflowTracing, evaluation and prompt registry for LLM apps, plus ML tracking Live demo ↗ (opens in a new tab)28.9k starsApache-2.091 out of 100
#5LiteLLMOpenAI-format gateway and Python SDK for calling 100+ LLM providers63.5k starscustom license88 out of 100