ArchestackBest of Open-Source AI: List on GitHub ↗3 stars (opens in a new tab)

#1414th of 22 in Model serving

laya-mlx

MLX port of Laya typed-decision models for Apple Silicon

Documentation ↗ (opens in a new tab)Website ↗ (opens in a new tab)Repository on GitHub ↗ (opens in a new tab)
Stars
6.9k
License
Apache-2.0
Last commit
Oct 2026
Last release
Oct 2026
Language
Python
Stars, last 4 weeks+6,918
15 Sept 2026: 0 stars22 Sept 2026: 5,588 stars29 Sept 2026: 6,617 stars6 Oct 2026: 6,784 stars13 Oct 2026: 6,918 stars

0 on 15 Sept 20266,918 on 13 Oct 2026

From GitHub's daily star counts.

Checked on GitHub.

Overview

Runs the Laya encoder checkpoints (322M to 421M parameters) natively on Apple Silicon with MLX. It answers typed questions (choice, score, true/false probability) in one forward pass with zero output tokens. Reported M3 Max median latency is 13.4 ms for the English model and 7.4 ms for the multilingual one. It is an independent port covering inference and conversion, not training.

Who it is for: Apple Silicon developers needing fast local classification and scoring

Strengths

  • Single forward pass, no token decoding or JSON generation; P50 13.42 ms English on M3 Max
  • Matched upstream answers on 63/63 validation questions in FP32 and FP16 per checkpoint
  • No PyTorch, Transformers runtime or cloud API; runs locally after first weight download
  • Includes language router, confidence gating, shortlist and tournament helpers for large choice sets

Weaknesses

  • Apple Silicon only (macOS 14+, Python 3.11+); no CUDA or Linux path
  • Limited to choice, score and noul questions; not a general text-generation server
  • No training or fine-tuning; upstream batch, HTTP/MCP server and hooks APIs are not included
  • Context capped at 512 or 1,024 tokens; many-label choices need shortlisting or tournament

What it needs

  • GPU required
  • Needs MLX, Hugging Face Hub (checkpoint download)
  • Models: convaiinnovations/laya (ModernBERT-large, 421M), convaiinnovations/laya-multilingual (mmBERT-base, 322M), convaiinnovations/laya-typed-decisions (ModernBERT-large, 421M)

Also in Model serving

See all 22
Also in Model serving
RankProjectScore
1LocalAIOne OpenAI-compatible server for text, speech, image and video models49.5k starsMIT81 out of 100
2llama.cppC/C++ inference engine serving GGUF models over an OpenAI-compatible API131.1k starsMIT80 out of 100
3OllamaRuns open-weight models locally behind a CLI and REST API183.1k starsMIT79 out of 100
#4vLLMInference engine for Hugging Face LLMs with PagedAttention and continuous batching93.7k starsApache-2.076 out of 100
#5colibriC inference engine that runs huge MoE models by streaming experts from disk41.8k starsApache-2.071 out of 100