The 2:15 AM Documentation Deadlock
It is 2:15 AM, and I am staring at a 504 Gateway Timeout error on a legacy deployment. I know I fixed this exact issue six months ago.
I even remember writing down the specific Nginx buffer settings that solved it. But searching my 1,200 Markdown files for “Nginx timeout” returns dozens of irrelevant results, and grep is failing because I probably used the word “latency” instead of “timeout” in my original note. My personal knowledge base has become a digital attic—full of value, but impossible to navigate.
This frustration is why semantic search is becoming essential. Tools like Notion AI or Obsidian Copilot find information based on intent rather than just matching characters. However, feeding your private intellectual property into a third-party cloud is a massive privacy risk. I wanted the intelligence of a Large Language Model (LLM) without the data leakage. That search for privacy led me to Khoj.
Why Your Current Search Feels Broken
Standard search is literal. If you search for “database performance,” a basic tool will ignore a note titled “Scaling Postgres Queries” simply because the characters don’t match. This rigid approach forces you to remember exactly how you phrased a thought months ago.
Semantic search works differently by using vector embeddings. Think of it as a mathematical map of human language.
It converts your text into a series of numbers (a vector) that represents its meaning. When you ask, “How do I make my site faster?”, the system looks for notes that are mathematically “close” to that concept, surfacing documents about caching, CDNs, or image compression. Khoj manages this intensive processing locally, allowing you to query PDFs, Markdown vaults, and even GitHub repos using natural language.
Khoj: The Privacy-First AI Assistant
Khoj is an open-source AI assistant designed to live on your own hardware. It does more than just search; it can summarize long research papers, chat with your notes to find contradictions, and even generate new content based on your writing style. Using Docker to deploy Khoj ensures your setup is portable and isolated from the rest of your system.
Hardware Requirements
- Docker & Docker Compose: Ensure you are running the latest version.
- RAM: 8GB is the minimum, but 16GB is recommended if you plan to run local LLMs.
- Storage: At least 10GB of free space for the vector database and model weights.
- Model Choice: An OpenAI API key for speed, or an Ollama instance for 100% local offline use.
Deploying Khoj with Docker
I prefer using Docker Compose because it makes the setup repeatable and easy to update. We need to configure the Khoj container to see your local files while keeping the configuration persistent. Here is a reliable docker-compose.yml to get you started.
version: '3.8'
services:
khoj:
image: khoj/khoj:latest
container_name: khoj
volumes:
- ./khoj_config:/home/khoj/.khoj
- /home/user/my_notes:/home/khoj/data:ro
ports:
- "42110:42110"
environment:
- KHOJ_DOMAIN=localhost
restart: unless-stopped
Pay close attention to the volume mapping: /home/user/my_notes:/home/khoj/data:ro. Replace the first path with the actual location of your Obsidian vault or PDF folder. I highly recommend using the :ro (read-only) flag. This ensures the AI can read and index your thoughts without any risk of accidentally modifying or deleting your original files.
Launching the Instance
Fire up your terminal and run:
docker-compose up -dOnce the container initializes, open your browser to http://localhost:42110. During the first run, Khoj will begin indexing your files. If you have a large library of 2,000+ files, expect this to take about 15 to 30 minutes as it builds the mathematical map of your data.
Connecting Your Knowledge Base
Inside the web interface, head to the “Content Sources” settings. This is where you fine-tune what Khoj actually looks at. You can link local directories, specific GitHub repositories, or even connect your Notion workspace if you are still in the process of migrating away from the cloud.
To keep the experience entirely local, point Khoj toward an Ollama instance. This setup ensures your data never leaves your network. In the settings, change the model provider to “Ollama” and set the URL to http://host.docker.internal:11434. This allows the Docker container to communicate with the Ollama service running on your host machine.
Optimizing Search Relevance
My initial search results were cluttered with noise from .git folders and node_modules. To fix this, create a .khojignore file in your data directory. Just like a .gitignore, this file tells the AI which folders to skip. Excluding log files and dependency folders makes the vector database leaner and significantly improves search speed.
Performance and Resource Management
Running local AI can be taxing on your CPU. If your fans are spinning too loud or your system feels sluggish, you can throttle Docker’s resource consumption in your compose file:
deploy:
resources:
limits:
cpus: '4'
memory: 6G
From a security standpoint, this architecture is a major win. Even if you choose to use an OpenAI model for the chat interface, the “Retrieval” part of RAG (Retrieval-Augmented Generation) happens on your machine. Only the tiny, relevant snippets of text are sent to the LLM as context, rather than your entire library. If you use Ollama, your privacy is absolute.
Final Thoughts
By 3:00 AM, my local Khoj instance was fully indexed. I asked, “How did I fix the Nginx proxy timeout?” and it immediately surfaced the exact proxy_read_timeout configuration I had buried in a project folder from last year. No more scrolling. No more guessing.
Transitioning to a semantic search system is a fundamental shift in how you interact with your own data. It turns a static pile of notes into an interactive second brain that actually understands you. If you value your privacy and your time, stop hitting CTRL+F and start building your own local AI search engine.
