#3939th of 41 in Training and fine-tuning
text-to-text-transfer-transformer
Reference code for the T5 text-to-text transformer, with dataset and training tooling
- Stars
- 6.6k
- License
- Apache-2.0
- Last commit
- Oct 2026
- Last release
- Apr 2020
- Language
- Python
6,472 on 4 Apr 20266,567 on 31 Oct 2026
From GitHub's daily star counts.
Checked on GitHub.
Overview
The t5 library is the code behind the T5 paper. It defines text-to-text Tasks and Mixtures on tf.data, evaluates them with built-in metrics, and trains or fine-tunes released checkpoints through Mesh TensorFlow (TPU or GPU) or an experimental Hugging Face PyTorch shim. The README says it is no longer actively developed and recommends T5X (JAX/Flax) for new work.
Who it is for: Researchers reproducing T5 results or fine-tuning T5 checkpoints on TPUs
Strengths
- Task and Mixture classes bundle data, preprocessors, tokenizer and metrics for multi-task training
- Operative configs for the paper's experiments are published in gs://t5-data/experiments
- Fine-tunes released T5 checkpoints from the command line via t5_mesh_transformer and gin configs
- Exports a SavedModel for serving; Colab tutorial runs on a free TPU
Weaknesses
- TensorFlow/MeshTF version is no longer actively developed; README points to T5X
- Hugging Face PyTorch shim is experimental, single-GPU, and subject to change
- Workflow assumes GCP TPUs and GCS buckets; GPU use is a secondary path
- Building C4 needs about 7 TB download and 335 CPU-days of preprocessing
What it needs
- GPU optional
- Needs TensorFlow, Mesh TensorFlow, TensorFlow Datasets, SentencePiece, gin-config, Google Cloud TPU and GCS (for TPU use)
- Models: T5 pre-trained checkpoints, Mesh TensorFlow Transformer, Hugging Face Transformers (PyTorch, experimental)
Quick answers
Is text-to-text-transfer-transformer open source?
Yes. Its code is on GitHub under the Apache-2.0 license.
Does text-to-text-transfer-transformer need a GPU?
No. It runs on a CPU and can use a GPU when you have one.
Also in Training and fine-tuning
See all 41| Rank | Project | Score |
|---|---|---|
| 1 | unslothDesktop and web app to run and fine-tune local models | 88 out of 100 |
| 2 | peftPython library for LoRA and other parameter-efficient fine-tuning of Transformers models | 86 out of 100 |
| 3 | deepspeedPyTorch library for distributed training of very large models with ZeRO | 85 out of 100 |
| #4 | verlReinforcement learning training library for LLM post-training with PPO and GRPO | 84 out of 100 |
| #5 | trlPost-training library with SFT, DPO, GRPO and KTO trainers | 83 out of 100 |