Local AI by VRAM Tier - 8GB | 12GB | 16GB | 24GB | 32GB
Deep dives: Chat | Coding | Translation | Vision | Speech | Agents
Updated July 2026: transcription has a new accuracy leader board (NVIDIA Parakeet and Canary, IBM Granite Speech), and the open-weight text-to-speech race has moved on from last spring’s picks to Step-Audio-EditX and Fish Audio S2 Pro. The VRAM combos below are rebuilt around the current models.
ElevenLabs charges $22/month for voice cloning. Otter.ai charges $17/month for transcription. Both send your audio to their servers. Running speech models locally gives you unlimited voice cloning, unlimited transcription, and complete privacy - all for the one-time cost of the GPU you already own.
This guide covers both halves of local speech: speech-to-text (STT / transcription) and text-to-speech (TTS / voice generation). The two tasks have very different hardware requirements, and you can often run both simultaneously.
Part 1: Speech-to-Text (Transcription)
The Models
For three years, Whisper was the default answer for local transcription. It still supports 99+ languages and remains the broadest option, but as of mid-2026 it no longer holds the accuracy crown. A cluster of newer open-weight models now sits at the top of the Open ASR Leaderboard for English and major European languages:
| Model | Params | VRAM / Compute | Reported WER (avg) | License | Source |
|---|---|---|---|---|---|
| IBM Granite Speech 4.1 2B | 2B | Runs on a single consumer GPU | ~5.33% | Apache 2.0 | HF card |
| NVIDIA Canary-Qwen-2.5B | 2.5B | ~A6000/RTX 5090 class | ~5.63% | CC-BY-4.0 | HF card |
| NVIDIA Parakeet TDT 0.6B v3 | 600M | ~2 GB to load | ~6.34% | CC-BY-4.0 | HF card |
| Whisper large-v3-turbo | 809M | ~6 GB | ~7.75% | MIT | HF card |
| Whisper large-v3 | 1.5B | ~10 GB | ~7.88% | MIT | HF card |
Word Error Rate (WER) is measured on standard English benchmarks; lower is better. One caveat worth stating plainly: the numbers above are each model’s own reported average on the Open ASR Leaderboard, not a single live ranking we re-ran ourselves, and the exact 1-2-3 order shifts as the leaderboard is updated. Treat them as “all three of the new models beat Whisper on English WER,” not as a precise podium.
The Recommendation: Parakeet TDT 0.6B v3 for English, Whisper for everything else
Two picks, depending on your languages.
If you mostly transcribe English (or the 25 languages it covers), start with NVIDIA Parakeet TDT 0.6B v3. At 600M parameters it loads in roughly 2GB, runs extremely fast, handles up to ~3 hours of audio in one pass with local attention, and still beats full-size Whisper on WER. That combination of tiny footprint and top-tier accuracy is what makes it the new default - it leaves almost your entire GPU free for a TTS model or an LLM. If you want the absolute lowest WER and only need English, Canary-Qwen-2.5B edges it out, at the cost of more VRAM and a 40-second-per-utterance limit.
If you need broad multilingual coverage, stay on Whisper. Its 99+ language support is still unmatched by the newer models, and the tooling around it is excellent: whisper.cpp (v1.8.6, June 2026, now with native voice-activity detection) and Faster-Whisper (CTranslate2, roughly 4x faster on GPU than the reference code at the same accuracy). Whisper large-v3-turbo, at ~6GB, remains the sensible all-rounder for mixed-language audio.
STT by VRAM Tier
| Tier | Best English pick | Best multilingual pick | VRAM used | Room for TTS? |
|---|---|---|---|---|
| 8GB | Parakeet TDT 0.6B v3 (~2GB) | Whisper large-v3-turbo (~6GB) | 2-6 GB | Yes - Parakeet leaves ~6GB free |
| 12GB | Parakeet / Canary-Qwen | Whisper large-v3-turbo | 2-6 GB | Yes - comfortable |
| 16GB | Canary-Qwen-2.5B | Whisper large-v3 (full) | 5-10 GB | Yes - most TTS models |
| 24GB | Canary-Qwen-2.5B | Whisper large-v3 (full) | 5-10 GB | Yes - everything fits |
| 32GB | Canary-Qwen-2.5B | Whisper large-v3 (full) | 5-10 GB | Yes - full pipeline + LLM |
The headline change from last year: because Parakeet transcribes at top accuracy in ~2GB, the “STT eats most of my 8GB card” problem is largely gone. You can now run great transcription and a voice-cloning TTS model together on an entry GPU.
We have a detailed setup guide: How to Self-Host Whisper: Replace Otter.ai (the workflow is the same if you swap in Parakeet via NVIDIA NeMo).
Part 2: Text-to-Speech (Voice Generation)
TTS is where the real variety is, and it is also where the most has changed since spring. The current Artificial Analysis open-weights TTS leaderboard is topped by two models that did not exist when this guide first ran.
The Landscape
| Model | Params | VRAM | Voice Clone? | Languages | License | Notes |
|---|---|---|---|---|---|---|
| Step-Audio-EditX | 3B | ~12 GB (AWQ 4-bit ~6-8 GB) | Yes (zero-shot) | EN/ZH/JA/KO + dialects | Apache 2.0 | Highest-ranked open TTS; can iteratively edit emotion/style |
| Fish Audio S2 Pro | ~5B | ~17 GB (bf16, measured) | Yes | 80+ | Non-commercial | #2 Elo; great multilingual, but read the license |
| Kokoro-82M | 82M | <1 GB | No | 8 | Apache 2.0 | Speed king; CPU-capable; 54 preset voices |
| Chatterbox (Multilingual V3) | 500M | ~4-6 GB | Yes (few-sec sample) | 23+ | MIT | Emotion-exaggeration control; built-in watermark |
| Piper | Varies | <1 GB | No (pre-trained) | 20+ | MIT | Runs real-time on a Raspberry Pi, no GPU |
A note on VRAM honesty: official cards for several of these do not publish an exact GB figure. The Step-Audio-EditX (~12GB) and Fish S2 Pro (~17GB) numbers come from the projects and from single-tester measurements respectively, and Kokoro/Chatterbox “under 1GB / 4-6GB” figures are community-reported. Expect real usage to move with your quantization and batch size.
Model Breakdown
Step-Audio-EditX - The new quality leader. StepFun’s 3B model currently tops the open-weights Elo board. Beyond zero-shot cloning, its trick is editing: you can take generated (or reference) audio and iteratively push the emotion, style, or delivery, rather than re-rolling from scratch. Apache 2.0 licensed. An AWQ 4-bit build brings it down to roughly 6-8GB if you are tight on VRAM.
Fish Audio S2 Pro - The multilingual heavyweight. S2 Pro covers 80+ languages and sits at #2 on the leaderboard. The catch, and it is a real one for a “run it yourself” audience: the weights ship under a non-commercial license. Fine for personal projects; you need a separate commercial license from Fish Audio to use it in a product.
Kokoro-82M - The speed king. At just 82M parameters, Kokoro runs comfortably on CPU and sips VRAM on a GPU. It has no voice cloning (54 fixed voices across 8 languages), but for reading text aloud, notifications, or a local voice assistant’s output, nothing else touches its efficiency. Apache 2.0.
Chatterbox - The expressive cloner. Resemble AI’s Chatterbox does zero-shot cloning from a few seconds of audio, and its Multilingual V3 update pushed cross-language consistency and added emotion-exaggeration control across 23+ languages. MIT licensed, with built-in Perth watermarking on generated audio.
Piper - The edge champion. Piper runs on a Raspberry Pi at real-time speed with no GPU. Quality is a step below the neural cloners, but the zero-VRAM footprint means it can run alongside anything - ideal for home automation and smart-home responses.
Two older models are still in active use and have not been superseded for their niches: XTTS-v2 (Coqui, strong multilingual default) and F5-TTS (flow-matching zero-shot cloning). We flag them because plenty of existing pipelines still run them well, though both are now below the current leaderboard leaders.
TTS by VRAM Tier {#tts-tiers}
8GB VRAM {#8gb}
GPUs: RTX 4060, RTX 3060 8GB, RTX 3070
| Combo | STT | TTS | Total VRAM | Use Case |
|---|---|---|---|---|
| Speed focus | Parakeet TDT (2GB) | Kokoro-82M (<1GB) | ~3 GB | Fast transcription + readback, tons of headroom |
| Voice clone | Parakeet TDT (2GB) | Chatterbox (4-6GB) | ~6-8 GB | Transcribe and clone together, finally |
| Edge/CPU | (CPU Whisper) | Piper (<1GB) | ~1 GB | 20+ languages, no GPU needed |
Best combo: Parakeet TDT 0.6B v3 + Chatterbox. The big shift this year is that a top-accuracy transcriber now costs ~2GB, so an 8GB card can run transcription and voice cloning simultaneously - something that required a 12GB+ card in the previous generation.
12GB VRAM {#12gb}
GPUs: RTX 3060 12GB, RTX 4070
| Combo | STT | TTS | Total VRAM |
|---|---|---|---|
| Best quality | Parakeet TDT (2GB) | Step-Audio-EditX AWQ (6-8GB) | ~8-10 GB |
| Multilingual | Whisper turbo (6GB) | Chatterbox (4-6GB) | ~10-12 GB |
| Efficient | Parakeet TDT (2GB) | Kokoro (<1GB) | ~3 GB (+room for a small LLM) |
Best combo: Parakeet + Step-Audio-EditX (AWQ 4-bit). Leaderboard-topping transcription and text-to-speech together, with room to spare.
16GB VRAM {#16gb}
GPUs: RTX 4060 Ti 16GB, RTX 5060, Arc A770
| Combo | STT | TTS | Total VRAM |
|---|---|---|---|
| Full quality | Canary-Qwen (5-6GB) | Step-Audio-EditX (12GB) | ~16-18 GB (tight - use AWQ build) |
| Voice assistant | Parakeet (2GB) | Chatterbox (6GB) | ~8 GB (+8GB for an LLM) |
| Multilingual | Whisper large-v3 (10GB) | Kokoro (<1GB) | ~11 GB |
Best combo: Parakeet + Chatterbox + a small LLM. 16GB is enough for a complete voice-in, AI-process, voice-out assistant with everything resident at once.
24GB VRAM {#24gb}
GPUs: RTX 3090, RTX 4090
| Combo | STT | TTS | LLM | Total VRAM |
|---|---|---|---|---|
| Voice assistant | Parakeet (2GB) | Step-Audio-EditX (12GB) | Qwen3 8B-class (~6GB) | ~20 GB |
| Power pipeline | Canary-Qwen (6GB) | Fish S2 Pro (17GB) | - | ~23 GB (non-commercial) |
| Full stack | Parakeet (2GB) | Chatterbox (6GB) | 14B-class LLM (~11GB) | ~19 GB |
Best combo: Parakeet + Step-Audio-EditX + an 8B-class LLM. A complete local voice assistant - speak, get a smart answer read back in a cloned, emotion-controllable voice - with headroom left over.
32GB VRAM {#32gb}
GPUs: RTX 5090
| Combo | STT | TTS | LLM | Total VRAM |
|---|---|---|---|---|
| Premium assistant | Canary-Qwen (6GB) | Step-Audio-EditX (12GB) | 14B-class LLM (~11GB) | ~29 GB |
| Multilingual studio | Whisper large-v3 (10GB) | Fish S2 Pro (17GB) | - | ~27 GB (non-commercial) |
Best combo: Canary-Qwen + Step-Audio-EditX + a 14B-class LLM. Maximum quality across transcription, expressive cloned speech, and a genuinely capable conversational model, all on one card.
Cross-Tier Summary
Best STT by Tier
| Tier | English pick | Multilingual pick | Reported WER |
|---|---|---|---|
| 8GB+ | Parakeet TDT 0.6B v3 | Whisper large-v3-turbo | ~6.34% / ~7.75% |
| 16GB+ | Canary-Qwen-2.5B | Whisper large-v3 | ~5.63% / ~7.88% |
Best TTS by Tier
| Tier | Model | Voice Clone | License | Notes |
|---|---|---|---|---|
| Any (CPU) | Piper / Kokoro-82M | No | MIT / Apache 2.0 | Real-time, tiny footprint |
| 8-12GB | Chatterbox / Step-Audio-EditX AWQ | Yes | MIT / Apache 2.0 | Cloning on entry hardware |
| 16GB+ | Step-Audio-EditX | Yes | Apache 2.0 | Top-ranked, editable delivery |
| 16GB+ | Fish Audio S2 Pro | Yes | Non-commercial | 80+ languages, license permitting |
Quick Start
Transcription with Faster-Whisper (multilingual)
pip install faster-whisper
# Python
from faster_whisper import WhisperModel
model = WhisperModel("large-v3-turbo", device="cuda")
segments, info = model.transcribe("meeting.mp3")
for segment in segments:
print(f"[{segment.start:.1f}s] {segment.text}")Transcription with Parakeet (English, via NVIDIA NeMo)
pip install nemo_toolkit['asr']
# Python
import nemo.collections.asr as nemo_asr
asr = nemo_asr.models.ASRModel.from_pretrained("nvidia/parakeet-tdt-0.6b-v3")
print(asr.transcribe(["meeting.wav"])[0].text)TTS with Kokoro (zero GPU needed)
pip install kokoro-onnx
# Works on CPU - no GPU required
from kokoro_onnx import Kokoro
kokoro = Kokoro("kokoro-v1.0.onnx", "voices-v1.0.bin")
audio, sr = kokoro.create("Your text here", voice="af_heart")For complete setup guides with Docker, speaker identification, and production configurations: