#120120th of 242 in ML frameworks and libraries

mteb

Python toolkit for benchmarking embedding models on retrieval and other tasks

Stars
3.5k
License
Apache-2.0
Last commit
Nov 2026
Last release
Nov 2026
Language
Python
Stars, last 29 weeks+303
25 Apr 2026: 3,215 stars2 May 2026: 3,224 stars9 May 2026: 3,240 stars16 May 2026: 3,249 stars23 May 2026: 3,260 stars30 May 2026: 3,269 stars6 Jun 2026: 3,281 stars13 Jun 2026: 3,290 stars20 Jun 2026: 3,298 stars27 Jun 2026: 3,313 stars4 Jul 2026: 3,329 stars11 Jul 2026: 3,340 stars18 Jul 2026: 3,352 stars25 Jul 2026: 3,361 stars1 Aug 2026: 3,374 stars8 Aug 2026: 3,379 stars15 Aug 2026: 3,388 stars22 Aug 2026: 3,395 stars29 Aug 2026: 3,405 stars5 Sept 2026: 3,413 stars12 Sept 2026: 3,419 stars19 Sept 2026: 3,426 stars26 Sept 2026: 3,435 stars3 Oct 2026: 3,444 stars10 Oct 2026: 3,453 stars17 Oct 2026: 3,464 stars24 Oct 2026: 3,475 stars31 Oct 2026: 3,489 stars7 Nov 2026: 3,501 stars14 Nov 2026: 3,518 stars

3,215 on 25 Apr 20263,518 on 14 Nov 2026

From GitHub's daily star counts.

Checked on GitHub.

Overview

MTEB is a Python library and CLI that evaluates text and multimodal embedding models across a catalog of tasks and named benchmarks, such as MTEB(eng, v2). Models load through mteb.get_model, which falls back to SentenceTransformer for models not implemented in MTEB. It also backs the public MTEB leaderboard on Hugging Face.

Who it is for: Engineers and researchers comparing or submitting embedding models

Strengths

  • Installs with pip or uv and runs from Python or the `mteb run` CLI
  • Named benchmarks carry citations and can export task tables as LaTeX
  • Feeds the public Hugging Face MTEB leaderboard, with documented steps to add models
  • Falls back to SentenceTransformer for models without a custom MTEB implementation

Weaknesses

  • Evaluation only; it does not train, fine-tune or serve embedding models
  • Task and model catalogs live in external docs, not the README

What it needs

  • Docker
  • Needs sentence-transformers
  • Models: SentenceTransformer models, custom model definitions

What changed

From the release notes. No flag means the note does not say so, not that the upgrade is safe.

Also in ML frameworks and libraries

See all 242
Also in ML frameworks and libraries
RankProjectScore
1transformersModel-definition library for pretrained text, vision, audio and multimodal models169.3k starsApache-2.094 out of 100
2ultralyticsPython library and CLI for training and running YOLO vision models63.5k starsAGPL-3.094 out of 100
3pytorch-image-modelsPyTorch library of image models, pretrained weights and training scripts37.3k starsApache-2.094 out of 100
#4pytorchPython tensor library with GPU support and tape-based autograd105.8k starscustom license93 out of 100
#5supervisionPython toolkit for loading, annotating and counting computer vision detections Live demo ↗ (opens in a new tab)51.7k starsMIT92 out of 100