Continue.dev + Ollama in VS Code: Run a Free AI Code Assistant Locally (GitHub Copilot Replacement)

AI tutorial - IT technology blog
AI tutorial - IT technology blog

Six months ago I pulled the plug on my GitHub Copilot subscription and switched everything to a local stack built on Continue.dev and Ollama. My code never leaves my machine, the latency is surprisingly good on a mid-range GPU, and the monthly cost is exactly zero. What follows is exactly what I settled on — the parts that stuck, with the dead ends cut.

Quick Start — Get Running in 5 Minutes

You need two pieces: Ollama (the local model server) and the Continue VS Code extension (the IDE interface). They communicate over a local HTTP port on 127.0.0.1:11434.

1. Install Ollama

Download the installer from the official site for your OS, or on Linux run:

curl -fsSL https://ollama.com/install.sh | sh

Verify it started correctly:

ollama --version
# ollama version is 0.3.x

2. Pull a Coding Model

For a machine with 8 GB VRAM (or 16 GB RAM for CPU-only), qwen2.5-coder:7b is my go-to — fast, accurate on completions, and fits comfortably:

# 7B — good balance of speed and quality
ollama pull qwen2.5-coder:7b

# 1.5B — if RAM is limited or you want instant completions
ollama pull qwen2.5-coder:1.5b

# 14B — if you have a 24 GB GPU and want GPT-4o-level chat quality
ollama pull qwen2.5-coder:14b

Confirm the model is available:

ollama list

3. Install the Continue Extension

Open VS Code, go to the Extensions panel, search for Continue (publisher: Continue.dev), and click Install. After installation a Continue sidebar icon appears on the left.

4. Point Continue at Ollama

Click the Continue sidebar icon → gear icon → Open config.json. Replace the default contents with this minimal working config:

{
  "models": [
    {
      "title": "Qwen2.5 Coder 7B",
      "provider": "ollama",
      "model": "qwen2.5-coder:7b"
    }
  ],
  "tabAutocompleteModel": {
    "title": "Qwen2.5 Coder 1.5B",
    "provider": "ollama",
    "model": "qwen2.5-coder:1.5b"
  },
  "allowAnonymousTelemetry": false
}

Save the file. Open any source file and start typing — press Tab to accept an autocomplete suggestion, or Ctrl+L (Cmd+L on Mac) to open the chat panel. You’re in.

Deep Dive — Understanding the Configuration

The quick start config works. But Continue has three areas most people never touch — and that’s where the real gains are.

Separating Chat and Autocomplete Models

Continue treats chat and autocomplete as separate workloads. The right model size is different for each:

  • Chat / models array: You are sending long prompts and expecting detailed explanations. A 7B–14B model is worth the extra latency here.
  • Autocomplete / tabAutocompleteModel: This fires on every keystroke. It needs to respond in under 300 ms or it feels laggy. A 1.5B–3B model is the right tool.

Context Providers — the Feature Most People Miss

Ever ask an AI assistant about your code and get an answer completely disconnected from your actual codebase? That’s what happens without context providers. They let Continue pull in live information — open files, your git diff, terminal output, VS Code diagnostics — automatically. Without them, the model only sees what you manually paste into chat. Add these to your config:

"contextProviders": [
  { "name": "code" },
  { "name": "docs" },
  { "name": "diff" },
  { "name": "terminal" },
  { "name": "problems" },
  {
    "name": "folder",
    "params": { "nFinal": 10 }
  }
]

Now in the chat box you can type @ to reference a file, the current git diff, terminal output, or VS Code diagnostics. The model gets real context instead of guessing.

Codebase Embeddings (Optional but Worth It)

To give the model visibility beyond just your open files, add an embeddings provider. Continue indexes your codebase locally and does a semantic search before each chat query:

"embeddingsProvider": {
  "provider": "ollama",
  "model": "nomic-embed-text"
}
ollama pull nomic-embed-text

Trigger a re-index with Ctrl+Shift+P → Continue: Reindex Codebase. Once indexed, the @codebase context provider becomes available in chat. On a 20K+ line project, it surfaces functions you forgot existed and connects patterns across files the model would otherwise miss entirely.

Advanced Usage

Using Multiple Models for Different Tasks

Register several models in the models array and switch between them in the chat panel. My production config has three:

"models": [
  {
    "title": "Qwen2.5 Coder 14B (Deep questions)",
    "provider": "ollama",
    "model": "qwen2.5-coder:14b"
  },
  {
    "title": "Qwen2.5 Coder 7B (Daily driver)",
    "provider": "ollama",
    "model": "qwen2.5-coder:7b"
  },
  {
    "title": "Mistral 7B (Non-code writing)",
    "provider": "ollama",
    "model": "mistral:7b"
  }
]

The model switcher sits at the top of the chat panel. I reach for the 14B model when debugging a gnarly issue and drop back to 7B for routine tasks to keep response times fast.

Custom Slash Commands

Slash commands let you define reusable prompts. Add a slashCommands section:

"slashCommands": [
  {
    "name": "review",
    "description": "Review selected code for bugs and style issues",
    "prompt": "Review the following code. Point out bugs, potential edge cases, and style issues. Be direct and specific:\n\n{{{ input }}}"
  },
  {
    "name": "docstring",
    "description": "Write a docstring for selected function",
    "prompt": "Write a concise docstring for this function. Include parameter types and return value:\n\n{{{ input }}}"
  }
]

Select a block of code, open chat, type /review and hit Enter. The model receives the selection with your custom prompt attached.

Project-Level Rules with .continuerules

Drop a .continuerules file in your project root to give the model standing instructions for that codebase:

This project uses Python 3.11 with strict type hints.
All functions must have type annotations.
Use snake_case for variables and functions.
Prefer pathlib over os.path.
Do not suggest any deprecated Python 2 patterns.

Continue automatically injects this file into every chat prompt when you are working in that project. No more repeating the same context every session.

Practical Tips from Six Months of Daily Use

Hardware reality check: A 7B model on a machine with an NVIDIA RTX 3060 (12 GB VRAM) completes a typical chat response in 3–8 seconds. CPU-only on a 16-core machine with 32 GB RAM takes 20–40 seconds for the same response — usable for chat, too slow for autocomplete. CPU-only? Stick to the 1.5B model for everything.

Keep autocomplete lightweight: The biggest mistake I see is running a 7B model for autocomplete. The latency breaks the flow. I’ve run this split — 1.5B for tab completion, 7B for chat — across Python, TypeScript, and Go projects for six months without changing it once. The 1.5B model misses a few complex completions. But at under 200 ms, it feels like a native IDE feature rather than waiting on an API call.

Use @diff for code review: Before committing, open Continue chat and type @diff Review this for issues before I commit. The model sees your staged changes and flags problems that linting misses — logic errors, missing null checks, inconsistent naming.

Restart Ollama if responses degrade: After a long session, Ollama sometimes accumulates memory and responses slow down noticeably. A quick systemctl restart ollama (Linux) or restarting the app (Windows/Mac) brings it back to normal.

Model selection by task type:

  • Python / JS / TS: qwen2.5-coder series — trained on a large code corpus, best autocomplete accuracy I have tested.
  • Explanation and architecture discussion: llama3.1:8b or mistral:7b — better at structured explanations than the coder-tuned models.
  • Dockerfile / shell scripting: Either works; the coder models have a slight edge.

Privacy benefit is real: On enterprise projects with NDAs, running completely offline removes the risk of proprietary code hitting third-party servers. That alone justified the switch on two client projects before I even cared about the cost saving.

The full setup described here — embeddings, context providers, multiple models, slash commands — takes about 15 minutes from scratch. After that, Ollama runs as a background service and Continue just works. The only recurring task is an occasional restart when memory builds up after a long session.

Share: