Local AI by VRAM Tier - 8GB | 12GB | 16GB | 24GB | 32GB
Deep dives: Chat | Coding | Translation | Vision | Speech | Agents
Updated July 2026: the autocomplete crown still belongs to Qwen2.5-Coder, but the agentic (SWE-bench) picks moved to the Qwen 3.6 generation, with Devstral Small 2 and Qwen3-Coder-Next as new open-weight options. Picks below are rebuilt around them.
GitHub Copilot costs $19/month, sends your code to Microsoft’s servers, and still gets things wrong. Running a coding model locally costs nothing per query, keeps your proprietary code on your machine, and - depending on your GPU - gets surprisingly close in quality.
The question is which model to run. There are two distinct jobs: autocomplete (fill-in-the-middle / FIM, the inline suggestions as you type) and agentic coding (asking questions, debugging, and having the model actually resolve issues across files). Some models do both. Most do one well.
The Benchmarks That Matter
- HumanEval - generate a Python function from a docstring. The classic test; most strong models now score 88%+
- SWE-bench Verified - fix a real GitHub issue autonomously. The closest thing to actual software engineering, and the benchmark that separates toys from tools
- FIM support - does the model support fill-in-the-middle for autocomplete? Not all chat models do
- Context window - how much of your codebase you can paste in at once
- License - whether you can use it in commercial work
8GB VRAM {#8gb}
GPUs: RTX 4060, RTX 3060 8GB, RTX 3070
Candidates
| Model | Params | HumanEval | FIM | VRAM (Q4) | License |
|---|---|---|---|---|---|
| Qwen2.5-Coder 7B | 7B | 88.4% | Yes | ~5 GB | Apache 2.0 |
| DeepSeek Coder V2 Lite | 16B / 2.4B active (MoE) | 81.1% | Yes | ~5 GB | DeepSeek license |
For autocomplete: Qwen2.5-Coder 7B
The Qwen2.5-Coder 7B is still the best small autocomplete model. At 88.4% HumanEval it beats far larger older models, supports FIM natively, handles 128K context, and covers 92+ languages. At ~5GB it leaves 3GB free on an 8GB card. Plug it into Continue or Tabby for inline suggestions that rival Copilot on common patterns.
Budget MoE: DeepSeek Coder V2 Lite - 16B total but only 2.4B active, so it runs light (~5GB), supports FIM, and scores 81.1% HumanEval. Less community tooling than Qwen, but a solid free alternative.
The honest take: 8GB coding models handle autocompletion well but struggle with complex refactoring, multi-file changes, and large-codebase understanding. Use them for the routine 80% and reach for a bigger model or the cloud for the hard 20%.
For all use cases at this level, see our 8GB VRAM complete guide.
12GB VRAM {#12gb}
GPUs: RTX 3060 12GB, RTX 4070
Winner: Qwen2.5-Coder 14B
At around 89% HumanEval with FIM support and 128K context, the Qwen2.5-Coder 14B is the pick for this tier. It fits in ~9GB, leaving ~3GB for context. It handles multi-language projects, framework-specific patterns, and generates tests alongside implementations - and it remains the strongest dedicated autocomplete model that fits a 12GB card.
For agentic use (running the model through a tool that edits files), 12GB is tight; the models that resolve real issues well start at the 16GB tier below.
For all use cases at this level, see our 12GB VRAM complete guide.
16GB VRAM {#16gb}
GPUs: RTX 4060 Ti 16GB, RTX 5060, Intel Arc A770, AMD RX 7800 XT
Candidates
| Model | Params | SWE-bench Verified | FIM | VRAM (Q4) | License |
|---|---|---|---|---|---|
| Qwen3.6-35B-A3B | 35B / 3B active (MoE) | 73.4% | No | ~16.6-22 GB | Apache 2.0 |
| Qwen2.5-Coder 14B | 14B | (autocomplete) | Yes | ~9 GB | Apache 2.0 |
Winner (agentic): Qwen3.6-35B-A3B
The current coding-tier testing calls Qwen3.6-35B-A3B the new default here: a mixture-of-experts model scoring 73.4% on SWE-bench Verified, meaning it resolves real GitHub issues, not just completions. VRAM is the catch - roughly 16.6GB at aggressive quant up to ~22GB at Q4, and an MoE keeps all experts resident, so on a 16GB card you are running it at Q3 with modest context. If it is too tight, drop to the 24GB tier’s dense options or stay on Qwen2.5-Coder 14B for autocomplete.
Autocomplete on the same card: keep Qwen2.5-Coder 14B (~9GB) installed for fast FIM inline suggestions and switch to the bigger model for agentic sessions.
For all use cases at this level, see our 16GB VRAM complete guide.
24GB VRAM {#24gb}
GPUs: RTX 3090, RTX 4090
This tier is where local coding genuinely competes with cloud assistants.
Candidates
| Model | Params | HumanEval | SWE-bench Verified | FIM | VRAM (Q4) | License |
|---|---|---|---|---|---|---|
| Qwen3.6-27B | 27B dense | - | 77.2% | No | ~17 GB | Apache 2.0 |
| Qwen2.5-Coder 32B | 32B | 92.7% | - | Yes | ~20 GB | Apache 2.0 |
| Devstral Small 2 | 24B dense | - | 68.0% | No | ~15 GB | Apache 2.0 |
For agentic coding: Qwen3.6-27B
The Qwen3.6-27B dense model scores 77.2% on SWE-bench Verified - higher than the 397B model it undercuts - and fits in about 17GB at Q4, leaving context headroom on a 24GB card. Run it through Aider or OpenHands and it resolves real issues autonomously. This is the strongest agentic coder that fits a single 24GB GPU.
For autocomplete: Qwen2.5-Coder 32B
92.7% HumanEval with FIM, 128K context, and 92+ languages at ~20GB. Still the best local autocomplete model at any tier - the reason a lot of RTX 4090 owners cancelled Copilot.
Apache-licensed alternative: Devstral Small 2 from Mistral - 24B dense, Apache 2.0, 256K context, 68.0% SWE-bench Verified, and only ~15GB at Q4 (per its hardware guide). The long context and permissive license make it a strong agentic pick when you want room for big repos.
For all use cases at this level, see our 24GB VRAM complete guide.
32GB VRAM {#32gb}
GPUs: RTX 5090
Winner: Qwen2.5-Coder 32B (Q6) for autocomplete, Qwen3.6-27B for agentic
At 32GB you can run Qwen2.5-Coder 32B at Q6_K (~28GB) for the cleanest autocomplete - fewer hallucinated API calls than Q4 - or run Qwen3.6-27B with maximum context for agentic work. You can also comfortably run Devstral Small 2 at Q8 (~26GB) if you prefer its 256K context and Apache license.
The stretch: Qwen3-Coder-Next is an 80B / 3B-active MoE, Apache 2.0, 262K context, scoring 70.6% SWE-bench Verified. It is the most capable open coding model here, but be realistic about VRAM: its hardware guide shows an IQ2_XXS build squeezing onto a 24GB card at degraded quality, and full-quality use really needs 48GB+. On a single 32GB card, expect to run it at a low quant or with CPU offload and reduced speed - impressive, but not a smooth interactive experience yet.
For all use cases at this level, see our 32GB VRAM complete guide.
Cross-Tier Summary
| Tier | Autocomplete Pick | Agentic Pick | Key figure |
|---|---|---|---|
| 8GB | Qwen2.5-Coder 7B | - | 88.4% HumanEval |
| 12GB | Qwen2.5-Coder 14B | - | ~89% HumanEval |
| 16GB | Qwen2.5-Coder 14B | Qwen3.6-35B-A3B | 73.4% SWE-bench |
| 24GB | Qwen2.5-Coder 32B | Qwen3.6-27B | 92.7% HE / 77.2% SWE-bench |
| 32GB | Qwen2.5-Coder 32B (Q6) | Qwen3.6-27B / Devstral Small 2 | 92.7% HE / 77.2% SWE-bench |
IDE Integration
Autocomplete with Continue (VS Code / JetBrains)
# Pull your autocomplete model
ollama pull qwen2.5-coder:7b # 8GB tier
ollama pull qwen2.5-coder:14b # 12-16GB tier
ollama pull qwen2.5-coder:32b # 24GB+ tierInstall Continue in VS Code or JetBrains, point it at your local Ollama server (http://localhost:11434), and enable tab autocomplete. The experience is close to Copilot. Full walkthrough: Self-Host Code Completion with Continue + Ollama.
Autocomplete with Tabby
Tabby is purpose-built for self-hosted completion - model management, codebase indexing for context-aware suggestions, and plugins for VS Code, JetBrains, and Vim. Setup: Self-Host Tabby AI Code Completion.
Agentic coding with Aider
pip install aider-chat
aider --model ollama/qwen3.6:27bAider sends your files to the local model, gets proposed changes, and applies them with git integration. At 77.2% SWE-bench Verified, Qwen3.6-27B resolves real issues, not just completions. Check the Ollama library for the exact current model tags.
What Local Coding Models Can’t Do (Yet)
- Full repository understanding - they see what you paste in, not your entire codebase. Indexing (Tabby) helps but is not the same as a cloud IDE agent
- Complex multi-file refactoring - models below the 27B tier struggle to stay consistent across many files
- Reliable test generation - they generate tests, but the tests often do not compile or test the wrong thing. Always review
- Unfamiliar frameworks - obscure libraries get hallucinated APIs
The 24GB+ tier with Qwen3.6-27B and Qwen2.5-Coder 32B genuinely threatens paid coding assistants for most workflows. Below that, local models are best as a supplement to your own expertise rather than a replacement for it.