Local AI by VRAM Tier - 8GB | 12GB | 16GB | 24GB | 32GB
Deep dives: Chat | Coding | Translation | Vision | Speech | Agents
Updated July 2026: purpose-built tool-callers arrived (Liquid AI’s LFM2.5 at 8GB, ServiceNow’s Apriel-1.6-15B at 16GB), and GLM-4.7-Flash became the high-tier agent pick. Scores below are aggregator-sourced and drawn from different benchmarks, so treat them as directional rather than a single ranked ladder.
AI agents that browse the web, query databases, control smart-home devices, and chain tools together to solve problems - all running on your own GPU. That’s the promise. The reality depends on whether your model can reliably call functions without hallucinating parameters or losing track of a multi-step plan.
Function calling is the hardest task for small models. A chat model can get away with an imprecise answer. A model calling a function with the wrong argument crashes the workflow. There’s no partial credit.
The Benchmarks That Matter
- BFCL - the Berkeley Function Calling Leaderboard. Tests single- and multi-turn calls, serial and parallel execution. The standard tool-use benchmark
- TAU-bench / TAU2 / τ²-bench - real-world tool-augmented tasks (retail, airline, telecom domains). Tests whether a model completes end-to-end workflows, not just single calls
- IFEval - instruction-following precision, a good proxy for “does it stay in the required output format?”
A grounding note before the tables: the official BFCL page is JavaScript-only and did not render for scraping, so the function-calling numbers here come from aggregators (llm-stats, Awesome Agents) and model cards. Different rows use different benchmarks (BFCL vs τ²-bench vs TAU2) that are not directly comparable - a higher number in one column does not automatically beat a lower number in another. Use them to shortlist, then test on your own tools.
What Makes a Good Agent Model?
Beyond benchmarks, practical reliability depends on:
- Structured output - does it format JSON function calls correctly, every time?
- Stop-token discipline - does it stop after the call, or keep generating text that breaks parsing?
- Multi-step planning - can it chain 3+ tool calls without losing the goal?
- Error recovery - when a tool returns an error, does it retry with corrected parameters or spiral?
- Determinism - at temperature 0, is the output parseable every run?
Models explicitly trained for tool use (LFM2.5, Apriel, the Qwen agent line, GLM-4.7-Flash) handle points 1-3 far more consistently than general chat models retrofitted with a tool template.
8GB VRAM {#8gb}
GPUs: RTX 4060, RTX 3060 8GB, RTX 3070
Candidates
| Model | Key figure | VRAM | License | Note |
|---|---|---|---|---|
| LFM2.5-8B-A1B | Tau2 Telecom 88.07, IFEval 91.84 | under ~6 GB | (model card) | MoE built for tool calls |
| Qwen3.5-9B | strong native tool format | ~7 GB | Apache 2.0 | Reliable all-rounder |
Winner: LFM2.5-8B-A1B
Liquid AI’s LFM2.5-8B-A1B is a mixture-of-experts model (8.3B total, ~1.5B active) built specifically for on-device agents. It writes Pythonic function calls by default, scores 88.07 on Tau2-Bench Telecom and 91.84 on IFEval, and stays under ~6GB. That combination of a tool-first design and a tiny footprint makes it the new 8GB agent pick. One caveat: a BFCL figure circulating for it could not be confirmed on the live leaderboard, so we quote the Tau2/IFEval numbers its release actually documents.
Reliable alternative: Qwen3.5-9B. The Qwen tool-call format is well supported across frameworks and parses cleanly, making the 9B a dependable general agent at ~7GB when you want one model for chat and tools.
The honest take: 8GB agent models handle 2-3 step tool chains reliably. Beyond that they start losing context. Great for smart-home control, single API calls, and basic retrieval; not for long multi-tool research workflows.
For all use cases at this level, see our 8GB VRAM complete guide.
12GB VRAM {#12gb}
GPUs: RTX 3060 12GB, RTX 4070
Winner: Qwen3.5-9B (headroom), or step toward Apriel-1.6-15B
At 12GB, run Qwen3.5-9B with room for large tool responses (database rows, web-page contents), or move up to the reasoning-first Apriel-1.6-15B-Thinker if your agent has to decide which tool to use, not just how to call it. Apriel’s predecessor ran roughly 10-13GB in independent testing, so on 12GB expect a tight fit with limited context; it is more comfortable at 16GB.
For all use cases at this level, see our 12GB VRAM complete guide.
16GB VRAM {#16gb}
GPUs: RTX 4060 Ti 16GB, RTX 5060, Intel Arc A770
Winner: Apriel-1.6-15B-Thinker
Apriel-1.6-15B-Thinker is the standout mid-tier agent model: a 15B dense model under an MIT license, 131K context, scoring 63.50 on BFCL v3 and 69 on Tau-Bench Telecom (with solid retail and airline numbers too). The reasoning training shows up as better planning on multi-step tasks, and MIT licensing plus a single-GPU footprint make it easy to deploy anywhere. 16GB gives it the context room its predecessor lacked at 12GB.
Keep a fast fallback: Qwen3.5-9B at ~7GB for simple, high-frequency automations where Apriel’s deliberation is overkill.
For all use cases at this level, see our 16GB VRAM complete guide.
24GB VRAM {#24gb}
GPUs: RTX 3090, RTX 4090
This is where local agents get serious - long tool-response chains and multi-step workflows.
Candidates
| Model | Key figure | VRAM | License | Note |
|---|---|---|---|---|
| GLM-4.7-Flash | τ²-bench 79.5, SWE-bench Verified 59.2 | ~24 GB | MIT | Strong end-to-end agent |
| Qwen3.5-27B | BFCL v4 68.5% | ~16 GB | Apache 2.0 | Reliable function calling |
| Qwen3.5-35B-A3B | BFCL v4 67.3% | ~18-22 GB | Apache 2.0 | Faster MoE |
Winner: GLM-4.7-Flash
GLM-4.7-Flash is a MIT-licensed 30B-A3B mixture-of-experts model that scores 79.5 on τ²-bench and 59.2 on SWE-bench Verified - the strongest end-to-end agent result that fits a 24GB card. Its MoE design keeps inference fast, which matters for agents: every tool call is a round trip, so a 5-step workflow means 5+ inference passes. Unsloth’s guide covers the quant options.
Reliable function-caller: Qwen3.5-27B at 68.5% BFCL v4 (with the 35B-A3B MoE close behind at 67.3%) is the pick when the priority is clean, parseable single- and multi-turn calls rather than long autonomous planning. Both fit 24GB with context headroom.
For all use cases at this level, see our 24GB VRAM complete guide.
32GB VRAM {#32gb}
GPUs: RTX 5090
Winner: GLM-4.7-Flash at full precision
At 32GB, run GLM-4.7-Flash at full precision with generous context and KV-cache headroom - the same 79.5 τ²-bench capability, now able to hold long tool histories without eviction. For heavier planning, Qwen3.6-27B is a strong dense alternative (one source reports a TAU2 of 94.2 under its own test methodology, which is not directly comparable to GLM’s τ²-bench figure, so treat the two as different measurements rather than a head-to-head).
For all use cases at this level, see our 32GB VRAM complete guide.
Cross-Tier Summary
| Tier | Best Pick | Benchmark shown | Practical Chain Length |
|---|---|---|---|
| 8GB | LFM2.5-8B-A1B | Tau2 Telecom 88.07 | 2-3 steps |
| 12GB | Qwen3.5-9B / Apriel-1.6-15B | - | 4-5 steps |
| 16GB | Apriel-1.6-15B-Thinker | BFCL v3 63.50 | 5-6 steps |
| 24GB | GLM-4.7-Flash | τ²-bench 79.5 | 8-10 steps |
| 32GB | GLM-4.7-Flash (full precision) | τ²-bench 79.5 | 10+ steps |
Setting Up a Local Agent
With Ollama + MCP (Model Context Protocol)
MCP connects your local model to external data sources and tools. Qwen-Agent has native MCP support and handles tool routing automatically:
pip install qwen-agent
ollama pull qwen3.5:9b
# Configure your MCP servers in the agent config; Qwen-Agent routes the callsWith LM Studio + Function Calling
LM Studio exposes OpenAI-compatible function calling, so it works with any framework expecting the OpenAI API format. Load your model, enable the API server, and point your framework at http://localhost:1234/v1. Check the Ollama library for the exact current model tags.
What Local Agents Can and Can’t Do
Works well:
- Smart-home control (lights, thermostat, media)
- Simple API integrations (weather, notifications, database queries)
- File operations (organize, rename, extract from documents)
- RAG pipelines (search, retrieve, answer)
- 2-3 step workflows on 8GB, 8-10 steps on 24GB+
Struggles with:
- Complex web browsing (multi-step forms, CAPTCHAs)
- Long planning horizons (10+ steps below the 24GB tier)
- Ambiguous tool selection when several tools could work
- Error recovery from unexpected states (timeouts, format changes)
- Reliable parallel tool execution (3+ tools at once)
Not realistic yet:
- Fully autonomous agents running for hours unsupervised
- Research tasks requiring judgment about source reliability
- Multi-agent coordination
The 24GB+ tier with GLM-4.7-Flash is where local agents become genuinely useful for real workflows. Below that, they work as automation assistants for well-defined, simple tasks.
Security Note
Running an agent locally does not make it safe. A local model with shell access can still delete files or expose data. Always:
- Sandbox agents in containers (Docker, Firejail)
- Limit tool permissions to what’s actually needed
- Log all tool calls for review
- Never give agents write access to critical systems without confirmation gates
For a complete walkthrough, see our AI Agent Containment guide.