Six months ago I pulled the plug on my GitHub Copilot subscription and switched everything to a local stack built on Continue.dev and Ollama. My code never leaves my machine, the latency is surprisingly good on a mid-range GPU, and the monthly cost is exactly zero. What follows is exactly what I settled on — the parts that stuck, with the dead ends cut.
Quick Start — Get Running in 5 Minutes
You need two pieces: Ollama (the local model server) and the Continue VS Code extension (the IDE interface). They communicate over a local HTTP port on 127.0.0.1:11434.
1. Install Ollama
Download the installer from the official site for your OS, or on Linux run:
curl -fsSL https://ollama.com/install.sh | sh
Verify it started correctly:
ollama --version
# ollama version is 0.3.x
2. Pull a Coding Model
For a machine with 8 GB VRAM (or 16 GB RAM for CPU-only), qwen2.5-coder:7b is my go-to — fast, accurate on completions, and fits comfortably:
# 7B — good balance of speed and quality
ollama pull qwen2.5-coder:7b
# 1.5B — if RAM is limited or you want instant completions
ollama pull qwen2.5-coder:1.5b
# 14B — if you have a 24 GB GPU and want GPT-4o-level chat quality
ollama pull qwen2.5-coder:14b
Confirm the model is available:
ollama list
3. Install the Continue Extension
Open VS Code, go to the Extensions panel, search for Continue (publisher: Continue.dev), and click Install. After installation a Continue sidebar icon appears on the left.
4. Point Continue at Ollama
Click the Continue sidebar icon → gear icon → Open config.json. Replace the default contents with this minimal working config:
{
"models": [
{
"title": "Qwen2.5 Coder 7B",
"provider": "ollama",
"model": "qwen2.5-coder:7b"
}
],
"tabAutocompleteModel": {
"title": "Qwen2.5 Coder 1.5B",
"provider": "ollama",
"model": "qwen2.5-coder:1.5b"
},
"allowAnonymousTelemetry": false
}
Save the file. Open any source file and start typing — press Tab to accept an autocomplete suggestion, or Ctrl+L (Cmd+L on Mac) to open the chat panel. You’re in.
Deep Dive — Understanding the Configuration
The quick start config works. But Continue has three areas most people never touch — and that’s where the real gains are.
Separating Chat and Autocomplete Models
Continue treats chat and autocomplete as separate workloads. The right model size is different for each:
- Chat /
modelsarray: You are sending long prompts and expecting detailed explanations. A 7B–14B model is worth the extra latency here. - Autocomplete /
tabAutocompleteModel: This fires on every keystroke. It needs to respond in under 300 ms or it feels laggy. A 1.5B–3B model is the right tool.
Context Providers — the Feature Most People Miss
Ever ask an AI assistant about your code and get an answer completely disconnected from your actual codebase? That’s what happens without context providers. They let Continue pull in live information — open files, your git diff, terminal output, VS Code diagnostics — automatically. Without them, the model only sees what you manually paste into chat. Add these to your config:
"contextProviders": [
{ "name": "code" },
{ "name": "docs" },
{ "name": "diff" },
{ "name": "terminal" },
{ "name": "problems" },
{
"name": "folder",
"params": { "nFinal": 10 }
}
]
Now in the chat box you can type @ to reference a file, the current git diff, terminal output, or VS Code diagnostics. The model gets real context instead of guessing.
Codebase Embeddings (Optional but Worth It)
To give the model visibility beyond just your open files, add an embeddings provider. Continue indexes your codebase locally and does a semantic search before each chat query:
"embeddingsProvider": {
"provider": "ollama",
"model": "nomic-embed-text"
}
ollama pull nomic-embed-text
Trigger a re-index with Ctrl+Shift+P → Continue: Reindex Codebase. Once indexed, the @codebase context provider becomes available in chat. On a 20K+ line project, it surfaces functions you forgot existed and connects patterns across files the model would otherwise miss entirely.
Advanced Usage
Using Multiple Models for Different Tasks
Register several models in the models array and switch between them in the chat panel. My production config has three:
"models": [
{
"title": "Qwen2.5 Coder 14B (Deep questions)",
"provider": "ollama",
"model": "qwen2.5-coder:14b"
},
{
"title": "Qwen2.5 Coder 7B (Daily driver)",
"provider": "ollama",
"model": "qwen2.5-coder:7b"
},
{
"title": "Mistral 7B (Non-code writing)",
"provider": "ollama",
"model": "mistral:7b"
}
]
The model switcher sits at the top of the chat panel. I reach for the 14B model when debugging a gnarly issue and drop back to 7B for routine tasks to keep response times fast.
Custom Slash Commands
Slash commands let you define reusable prompts. Add a slashCommands section:
"slashCommands": [
{
"name": "review",
"description": "Review selected code for bugs and style issues",
"prompt": "Review the following code. Point out bugs, potential edge cases, and style issues. Be direct and specific:\n\n{{{ input }}}"
},
{
"name": "docstring",
"description": "Write a docstring for selected function",
"prompt": "Write a concise docstring for this function. Include parameter types and return value:\n\n{{{ input }}}"
}
]
Select a block of code, open chat, type /review and hit Enter. The model receives the selection with your custom prompt attached.
Project-Level Rules with .continuerules
Drop a .continuerules file in your project root to give the model standing instructions for that codebase:
This project uses Python 3.11 with strict type hints.
All functions must have type annotations.
Use snake_case for variables and functions.
Prefer pathlib over os.path.
Do not suggest any deprecated Python 2 patterns.
Continue automatically injects this file into every chat prompt when you are working in that project. No more repeating the same context every session.
Practical Tips from Six Months of Daily Use
Hardware reality check: A 7B model on a machine with an NVIDIA RTX 3060 (12 GB VRAM) completes a typical chat response in 3–8 seconds. CPU-only on a 16-core machine with 32 GB RAM takes 20–40 seconds for the same response — usable for chat, too slow for autocomplete. CPU-only? Stick to the 1.5B model for everything.
Keep autocomplete lightweight: The biggest mistake I see is running a 7B model for autocomplete. The latency breaks the flow. I’ve run this split — 1.5B for tab completion, 7B for chat — across Python, TypeScript, and Go projects for six months without changing it once. The 1.5B model misses a few complex completions. But at under 200 ms, it feels like a native IDE feature rather than waiting on an API call.
Use @diff for code review: Before committing, open Continue chat and type @diff Review this for issues before I commit. The model sees your staged changes and flags problems that linting misses — logic errors, missing null checks, inconsistent naming.
Restart Ollama if responses degrade: After a long session, Ollama sometimes accumulates memory and responses slow down noticeably. A quick systemctl restart ollama (Linux) or restarting the app (Windows/Mac) brings it back to normal.
Model selection by task type:
- Python / JS / TS:
qwen2.5-coderseries — trained on a large code corpus, best autocomplete accuracy I have tested. - Explanation and architecture discussion:
llama3.1:8bormistral:7b— better at structured explanations than the coder-tuned models. - Dockerfile / shell scripting: Either works; the coder models have a slight edge.
Privacy benefit is real: On enterprise projects with NDAs, running completely offline removes the risk of proprietary code hitting third-party servers. That alone justified the switch on two client projects before I even cared about the cost saving.
The full setup described here — embeddings, context providers, multiple models, slash commands — takes about 15 minutes from scratch. After that, Ollama runs as a background service and Continue just works. The only recurring task is an occasional restart when memory builds up after a long session.

