Local AI by Editorial Team

8GB VRAM: Every AI Task You Can Run Locally in 2026

Run local AI on 8GB GPUs - chat, coding, vision, speech, and agents. Current model picks and honest limits for RTX 4060, RTX 3070, and similar cards.

Updated July 13, 2026

NVIDIA graphics card closeup showing cooling fans
Photo via Unsplash

Local AI by VRAM Tier - 8GB | 12GB | 16GB | 24GB | 32GB

Deep dives: Chat | Coding | Translation | Vision | Speech | Agents

Updated July 2026: the big 8GB change is speech - NVIDIA’s Parakeet transcribes at top accuracy in about 2GB, so an entry card can finally run transcription, text-to-speech, and a small model together. Agent and vision picks are updated too.

You have 8GB of VRAM. A year ago that meant toy models and disappointment. In 2026 it means a genuinely useful local AI setup across chat, coding, translation, vision, speech, and basic agent tasks - all running privately on your own hardware.

Your Hardware

8GB VRAM cards include the RTX 4060, RTX 3060 8GB, RTX 3070, RTX 2080, and GTX 1080. These retail for $250-350 (RTX 4060) or less used.

The math: at Q4_K_M quantization (4-bit), a model uses roughly 0.6-0.8 GB per billion parameters. That means 8GB fits models up to about 9B parameters with 1-2GB left for context. Larger models need more aggressive quantization or simply won’t fit.

Quick Reference

Use CaseBest PickVRAM UsedNote
ChatQwen3.5-9B~7 GB54-58 tok/s, offloads to long context
Coding (autocomplete)Qwen2.5-Coder 7B~5 GB88.4% HumanEval, FIM
Coding (chat)Qwen3.5-9B~7 GBStrong small-model coder
TranslationPurpose-built translation model~3 GBBeats a general model of the same size
VisionQwen3-VL-8B~6 GBDocs, charts, screenshots
Speech (STT)Parakeet TDT 0.6B v3~2 GB~6.34% WER, English + 24 langs
Speech (TTS)Kokoro-82M<1 GBCPU-capable, 8 languages
AgentsLFM2.5-8B-A1Bunder ~6 GBBuilt for tool calling

Chat & General Assistant

Qwen3.5-9B remains the best all-round 8GB chat model - about 6.96GB at Q4 with 32K context, 54-58 tok/s, and the ability to offload to much longer context. It handles everyday questions, brainstorming, summarization, and light analysis well. For a lighter footprint, Gemma 4 E4B at ~5GB leaves more room for other models.

Full comparison: local chat model guide.

Coding

Autocomplete: Qwen2.5-Coder 7B at 88.4% HumanEval with FIM support and 128K context. At ~5GB it leaves room for context. Plug it into Continue or Tabby for Copilot-style inline suggestions.

Chat-based coding: Qwen3.5-9B again - the strongest small code-chat option at this tier.

Full comparison and IDE setup: local coding model guide.

Translation

A purpose-built translation model at 3-4GB (such as the TranslateGemma line) beats a general model of the same size on translation quality, and leaves plenty of headroom on an 8GB card. For very rare language pairs, a 200-language model like NLLB-200 1.3B is literal but accurate.

Full comparison: local translation model guide.

Vision

Qwen3-VL-8B at ~6GB reads documents, understands charts, and describes photos. Point it at a screenshot to extract an error message, or photograph a whiteboard for structured notes. For an ultralight option that leaves room for another model, MiniCPM-V 4.6 runs in roughly 4GB (or ~2GB as a CPU GGUF).

Full comparison: local vision model guide.

Speech

This is where 8GB changed most. Transcription: NVIDIA Parakeet TDT 0.6B v3 loads in about 2GB, transcribes English (and 24 more languages) at roughly 6.34% WER - beating full-size Whisper - and leaves most of your card free. For broad multilingual audio, Whisper large-v3-turbo (~6GB) is still the pick.

Text-to-speech: Kokoro-82M runs at a fraction of a gigabyte, on CPU if needed, with natural built-in voices across 8 languages (no cloning).

Both together, plus more: because Parakeet only needs ~2GB, you can now run transcription + Kokoro TTS + a small chat model at the same time on a single 8GB card - a full voice pipeline that used to require a 12GB+ GPU.

Full comparison and pipeline setups: local speech model guide.

Agents (Tool Use)

LFM2.5-8B-A1B is a mixture-of-experts model built for on-device agents: it writes function calls by default, stays under ~6GB, and posts strong tool-use scores (Tau2-Bench Telecom 88.07). It handles 2-3 step tool chains reliably - enough for smart-home control, single API calls, and basic automations. Qwen3.5-9B is a dependable alternative if you want one model for both chat and tools.

Full comparison and setup: local agent model guide.

Getting Started

# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh

# Pull the essentials for 8GB
ollama pull qwen3.5:9b           # Chat + coding chat
ollama pull qwen2.5-coder:7b     # Autocomplete
ollama pull qwen3-vl:8b          # Vision

# Start chatting
ollama run qwen3.5:9b

For a web interface, pair Ollama with Open WebUI. For coding, add Continue or Tabby. Model tags shift as libraries update - check the Ollama library for the current ones.

Honest Limits

8GB is the entry tier. What it still can’t do well:

  • Complex multi-step reasoning - models under the 14B class lose coherence on long logic chains
  • Two large models at once - fitting two 7-9B models simultaneously is not realistic; you switch between them (small speech/vision models are the exception, as above)
  • Large context - after a 7GB model loads, you have ~1GB for context, so long documents get truncated
  • Reliable long agents - 2-3 step chains work, 5+ step workflows fail often

If you keep hitting these walls, a 12GB card (RTX 3060 12GB, ~$200 used) or 16GB card (RTX 4060 Ti 16GB, ~$400) is a meaningful upgrade. See our 12GB guide or 16GB guide.