It was 2 AM when the alerts started firing. Production database cluster throwing connection pool errors — the kind that cascade into full outages if you don’t catch them fast. I remembered writing a post-mortem about an almost identical issue eight months ago in my Obsidian vault. Over 600 notes. Searching “connection pool” returned 23 results. None of them were the right one.
I wasted 15 minutes clicking through notes while the incident was live. Found it eventually — titled “PostgreSQL connection exhaustion notes (2023 Q2)”. The title had zero keywords I searched for. That night, I decided to fix this properly.
The Root Cause: Why Keyword Search Fails You
Traditional search is dumb in a very specific way. It matches strings, not meaning. Your brain encoded “connection pool exhaustion” as knowledge, but your future self might search for “too many connections” or “database timeout” or “pg_stat_activity overloaded”. Keyword search doesn’t know these phrases are the same problem.
The technical term is semantic gap. The word you typed and the words in your notes carry the same meaning but share zero characters. Every Obsidian user who writes frequently hits this wall. Dataview queries, tags, backlinks — they all still depend on you remembering exactly how you named things months ago.
What you actually need is vector similarity search: convert every note into a mathematical representation of its meaning (an embedding), then search by conceptual closeness rather than string matching.
Three Approaches — and Why One Wins
Option 1: Cloud AI Plugins
Obsidian has plugins that hook into OpenAI or Claude to search your vault semantically. Setup takes five minutes. The fatal problem: your notes leave your machine. If you’re documenting production infrastructure, internal architecture decisions, or anything even slightly sensitive, sending that to a cloud API is a non-starter. Also, it costs money per query and breaks whenever you hit rate limits — which will happen at 2 AM when you need it most.
Option 2: Local LLM Only (Ollama Without a Vector Store)
You can run Ollama locally and feed it your notes manually by copy-pasting. This obviously doesn’t scale past a handful of files. With 600+ notes you’d spend more time copy-pasting than the search problem itself is worth.
Option 3: Local Embeddings + Vector Database (The Right Way)
This is what I ended up building. The stack:
- Obsidian — your existing note vault (plain Markdown files on disk)
- Ollama — runs embedding models locally, zero cloud dependency
- ChromaDB — lightweight vector database, runs in-process or as a persistent server
- Python script — indexes the vault, answers questions using retrieved context
Everything runs on your machine. Your notes never leave. Query time is under a second on modern hardware, and the whole thing survives airplane mode.
Building the Stack
Step 1: Install Ollama and Pull an Embedding Model
Install Ollama, then pull a dedicated embedding model. Don’t use your chat model for embeddings — they’re architecturally different and optimized for different tasks. Using the wrong one degrades retrieval accuracy significantly.
# Install Ollama (Linux)
curl -fsSL https://ollama.com/install.sh | sh
# Pull a dedicated embedding model
# nomic-embed-text: fast, accurate, 137M params
ollama pull nomic-embed-text
# Verify it's available
ollama listAlso pull a lightweight chat model for answering questions:
ollama pull llama3.2:3b # Fast, fits in 4GB RAM
# or
ollama pull mistral:7b # Better reasoning, needs 8GB RAMStep 2: Install Python Dependencies
pip install chromadb ollamaChromaDB can run entirely in-process with persistent storage — no separate server process needed for a personal knowledge base. This also means no port conflicts, no startup order issues.
Step 3: Index Your Obsidian Vault
This script walks your vault, reads every Markdown file, generates embeddings via Ollama, and upserts them into ChromaDB. It skips unchanged files on subsequent runs using content hashing, so re-indexing after adding new notes takes seconds instead of minutes.
import chromadb
import ollama
import hashlib
from pathlib import Path
VAULT_PATH = "/path/to/your/obsidian/vault"
CHROMA_PATH = "./chroma_db"
EMBED_MODEL = "nomic-embed-text"
COLLECTION_NAME = "obsidian_notes"
def get_content_hash(content: str) -> str:
return hashlib.md5(content.encode()).hexdigest()
def index_vault():
client = chromadb.PersistentClient(path=CHROMA_PATH)
collection = client.get_or_create_collection(
name=COLLECTION_NAME,
metadata={"hnsw:space": "cosine"}
)
# Get already-indexed IDs to skip unchanged files
existing_ids = set(collection.get()["ids"])
vault = Path(VAULT_PATH)
md_files = list(vault.rglob("*.md"))
print(f"Found {len(md_files)} notes in vault")
indexed = 0
for md_file in md_files:
content = md_file.read_text(encoding="utf-8", errors="ignore").strip()
if not content:
continue
relative_path = str(md_file.relative_to(vault))
versioned_id = f"{relative_path}::{get_content_hash(content)}"
if versioned_id in existing_ids:
continue # File unchanged, skip
# Embed (truncate to model's token limit)
response = ollama.embed(model=EMBED_MODEL, input=content[:8000])
embedding = response["embeddings"][0]
collection.upsert(
ids=[versioned_id],
embeddings=[embedding],
documents=[content[:2000]], # Store excerpt for context window
metadatas=[{
"file": relative_path,
"title": md_file.stem,
"path": str(md_file)
}]
)
indexed += 1
print(f" Indexed: {md_file.stem}")
print(f"Done. Added {indexed} notes. Total in DB: {collection.count()}")
if __name__ == "__main__":
index_vault()Run it once. On 600 notes it takes about 3-4 minutes on CPU. After that, re-running only processes changed or new files.
Step 4: Query Your Knowledge Base
import chromadb
import ollama
import sys
CHROMA_PATH = "./chroma_db"
EMBED_MODEL = "nomic-embed-text"
CHAT_MODEL = "llama3.2:3b"
COLLECTION_NAME = "obsidian_notes"
def ask(question: str, top_k: int = 5):
client = chromadb.PersistentClient(path=CHROMA_PATH)
collection = client.get_or_create_collection(name=COLLECTION_NAME)
# Embed the question with the SAME model used for indexing
q_embedding = ollama.embed(model=EMBED_MODEL, input=question)["embeddings"][0]
results = collection.query(
query_embeddings=[q_embedding],
n_results=top_k,
include=["documents", "metadatas", "distances"]
)
docs = results["documents"][0]
metas = results["metadatas"][0]
distances = results["distances"][0]
print("\n=== Relevant Notes Found ===")
for meta, dist in zip(metas, distances):
similarity = 1 - dist
print(f" [{similarity:.2f}] {meta['title']} ({meta['file']})")
# Build context for the LLM
context = "\n\n---\n\n".join([
f"[Source: {m['title']}]\n{doc}"
for doc, m in zip(docs, metas)
])
prompt = f"""You are a personal assistant with access to the user's notes.
Answer the question based ONLY on the provided notes.
If the notes don't contain enough information, say so directly.
NOTES:
{context}
QUESTION: {question}
ANSWER:"""
response = ollama.chat(
model=CHAT_MODEL,
messages=[{"role": "user", "content": prompt}]
)
print("\n=== Answer ===")
print(response["message"]["content"])
if __name__ == "__main__":
question = " ".join(sys.argv[1:]) if len(sys.argv) > 1 else "What topics are in my notes?"
ask(question)Now search works the way your brain does:
python ask.py "how did I fix that postgres connection issue"
python ask.py "nginx reverse proxy with SSL configuration"
python ask.py "what memory limits did we set for the API container"Keeping It Current: Auto-Indexing
Run the indexer on a cron job so new notes get picked up automatically:
# Runs every hour, appends to log
0 * * * * /usr/bin/python3 /path/to/index_vault.py >> /tmp/kb_index.log 2>&1If you have long notes — post-mortems, architecture docs, anything over 500 lines — split them into chunks before indexing. Chunking means retrieval returns the relevant section instead of the whole file, which makes the LLM’s context window much more useful:
def chunk_text(text: str, chunk_size: int = 400) -> list[str]:
words = text.split()
return [
" ".join(words[i:i + chunk_size])
for i in range(0, len(words), chunk_size)
]Real-World Performance Numbers
After running this for three months on my actual vault (680 notes):
- Initial indexing: ~4 minutes for 680 notes, CPU only, nomic-embed-text
- Incremental re-index: 3-10 seconds for changed or new files
- Query + LLM answer: 8-15 seconds with llama3.2:3b on CPU; 2-4 seconds with a GPU
- Storage overhead: ChromaDB uses roughly 45MB for 680 notes
In my real-world experience, building this kind of local knowledge retrieval system is one of the essential skills to master if you’re serious about actually reusing what you write. The gap between “having notes” and “having a searchable knowledge base” is enormous in practice. I’ve recovered forgotten solutions, cross-referenced old architecture decisions, and answered “didn’t we solve this already?” in seconds instead of minutes.
The 2 AM scenario I described at the top isn’t hypothetical anymore. Last month — same kind of alert, different database. Ran the query script. Got the relevant post-mortem excerpts in 9 seconds. Resolved in 12 minutes instead of 45.
Things That Will Catch You Off Guard
- nomic-embed-text truncates at 8192 tokens. Notes longer than ~6000 words get silently clipped. Chunk them first or you’ll get incomplete embeddings.
- Always use the same embedding model for indexing and querying. Mixing models produces nonsense results — the vector spaces are completely different mathematical structures.
- ChromaDB in-process blocks concurrent access. If you want to query from multiple terminals simultaneously, run
chroma run --path ./chroma_dbas a standalone server and connect withchromadb.HttpClient()instead. - First query after a fresh Ollama start is slow. The model loads into memory on first call (~3-5 seconds extra). Subsequent queries are fast. Not a bug.
The stack is deliberately minimal — no LangChain, no orchestration layer, no cloud dependency. Just Python, two libraries, and local models each doing one job. When something breaks at 2 AM, simple systems with fewer moving parts are dramatically easier to diagnose than layered abstractions. The extra 15 minutes of setup simplicity pays for itself every time you actually need to use this under pressure.
