Comparing Approaches for Vector Search
Full-text search engines like Elasticsearch rely on exact term overlap via BM25 scoring. That model fails when users speak in synonyms. A query for “budget laptop for software engineering” won’t surface a product titled “cheap developer notebook” because the tokens don’t match.
Dense vector embeddings solve this vocabulary mismatch. Models map text to points in a high-dimensional space where conceptual similarity correlates with geometric distance. But once you turn sentences into arrays of 384 or 1,536 floats, choosing where and how to search them determines whether your pipeline scales or stalls under traffic:
- PostgreSQL with pgvector: A pragmatic choice if you already run Postgres in your stack. You query relational tables and compute cosine similarity within standard SQL. The catch? Building HNSW indexes over 5 million rows requires gigabytes of dedicated maintenance work memory, and latency spikes when complex joins run alongside vector distance calculations.
- Two-Stage Post-Filtering: This setup queries an approximate nearest neighbor (ANN) index first, then throws away any results that fail metadata checks. It breaks down fast under high selectivity. If you ask for the top 10 items where
category = "electronics", but 95% of the nearest neighbors happen to be"clothing", the query returns almost nothing unless you over-fetch hundreds of candidate rows. - Qdrant (Single-Stage Filtering): Written in Rust, Qdrant builds payload metadata directly into its HNSW graph traversal. Instead of pruning candidates after the vector search finishes, it checks boolean filters at every step through the graph nodes. Unmatched vectors never enter the distance calculation queue.
Architecture Trade-Offs at a Glance
Every vector strategy makes a deliberate compromise between latency, memory consumption, and infrastructure complexity.
1. PostgreSQL + pgvector
- Strengths: Zero new infrastructure. You retain standard ACID guarantees, row-level locks, and familiar relational joins against existing business tables.
- Weaknesses: Memory usage climbs steep during index construction. On large multi-tenant collections, combined filter-and-vector queries frequently fall back to slow sequential scans if the planner misjudges filter selectivity.
2. Naive Post-Filtering
- Strengths: Trivial to assemble on top of standalone ANN libraries like FAISS or Annoy.
- Weaknesses: Terrible worst-case latency. Narrow filters force you to scan an unpredictable number of neighbors—often 500 to 1,000 points—just to return 10 valid matches.
3. Qdrant Single-Stage Engine
- Strengths: Sub-10ms search latency on collections with tens of millions of records. It supports memory-mapped files (mmap) on disk and offers native 8-bit scalar quantization, reducing RAM demands by up to 75% with under 1% loss in recall.
- Weaknesses: It adds an independent stateful service to monitor, back up, and scale alongside your primary datastore.
Recommended Stack: Qdrant + FastEmbed
Running Qdrant alongside FastEmbed in Python creates a lean, production-grade retrieval pipeline. For a document corpus under 500,000 items, a 384-dimensional model like BAAI/bge-small-en-v1.5 or all-MiniLM-L6-v2 delivers solid semantic accuracy while staying cheap to run. In our benchmarks on an AWS c6i.xlarge instance (4 vCPUs, 8 GB RAM), this setup maintains p95 query latencies below 8 milliseconds under 120 concurrent requests per second.
The system breaks down into three distinct components:
- Qdrant Daemon: Runs inside Docker or Kubernetes, hosting the vector graph, inverted payload indexes, and memory-mapped segment files.
- Inference Layer: Python handles lightweight tokenization and ONNX runtime embedding generation via FastEmbed, keeping CPU overhead minimal.
- Tuned HNSW Index: Tuned graph parameters balance index build speeds against fast real-time query exploration.
Implementation Walkthrough
Let’s walk through configuring the daemon, initializing the collection schema, indexing key fields, and querying with complex filters.
Step 1: Launch Qdrant in Docker
Start the Qdrant container with persistent volume storage. Expose port 6333 for HTTP operations and 6334 for high-throughput gRPC communication:
docker run -d --name qdrant-server \
-p 6333:6333 -p 6334:6334 \
-v $(pwd)/qdrant_storage:/qdrant/storage:z \
qdrant/qdrant:v1.12.1Step 2: Install Python Libraries
Grab the official client along with FastEmbed, which ships with ONNX Runtime to avoid heavy PyTorch dependencies:
pip install qdrant-client fastembedStep 3: Define Collection Schema and HNSW Settings
Dense FP32 vectors eat memory quickly. Storing 1 million 384-dimensional vectors in raw 32-bit floats demands around 1.5 GB of RAM just for the raw coordinates, plus graph overhead. By enabling INT8 scalar quantization, Qdrant compresses every 4-byte float into a single 1-byte integer, cutting that footprint down to ~400 MB. Setting on_disk=True for vectors ensures only hot vectors and quantized representations stay pinned in RAM.
from qdrant_client import QdrantClient
from qdrant_client.http import models
client = QdrantClient(host="localhost", port=6333)
COLLECTION_NAME = "tech_kb_articles"
VECTOR_SIZE = 384 # Matches BAAI/bge-small-en-v1.5 output dimensions
client.recreate_collection(
collection_name=COLLECTION_NAME,
vectors_config=models.VectorParams(
size=VECTOR_SIZE,
distance=models.Distance.COSINE,
on_disk=True # Keeps raw FP32 vectors on disk, paging them via mmap
),
hnsw_config=models.HnswConfigDiff(
m=16, # 16 links per node; keeps graph traversal fast and lightweight
ef_construct=100, # Evaluates 100 candidate neighbors per point during insertion
on_disk=False # Keeps the HNSW navigation graph in RAM for quick hops
),
quantization_config=models.ScalarQuantization(
scalar=models.ScalarQuantizationConfig(
type=models.ScalarType.INT8,
quantile=0.99, # Drops extreme outliers to preserve dynamic range
always_ram=True # Keeps the compact 1-byte representation in memory
)
)
)
print(f"Collection '{COLLECTION_NAME}' configured.")Step 4: Index Payload Metadata
Filtering without payload indexes forces Qdrant to scan raw metadata JSON blobs on disk during search. Creating explicit field indexes builds inverted lists and numeric trees, ensuring instant filter checks during graph traversal:
# Keyword index for exact string matching on categories
client.create_payload_index(
collection_name=COLLECTION_NAME,
field_name="category",
field_schema=models.PayloadSchemaType.KEYWORD,
)
# Numeric index to enable fast b-tree range evaluations
client.create_payload_index(
collection_name=COLLECTION_NAME,
field_name="view_count",
field_schema=models.PayloadSchemaType.INTEGER,
)
print("Payload indexes registered.")Step 5: Generate Embeddings and Batch Insert
Generate document embeddings using FastEmbed and upload them into Qdrant alongside their metadata payloads:
from fastembed import TextEmbedding
from uuid import uuid4
embedding_model = TextEmbedding(model_name="BAAI/bge-small-en-v1.5")
documents = [
{
"text": "Troubleshooting connection timeouts and DNS errors in Docker bridge networks.",
"category": "devops",
"status": "published",
"view_count": 1250
},
{
"text": "Configuring rate limiting, token buckets, and proxy buffering in Nginx.",
"category": "devops",
"status": "published",
"view_count": 3400
},
{
"text": "Writing clean asynchronous database queries with Python AsyncIO and SQLAlchemy 2.0.",
"category": "python",
"status": "published",
"view_count": 5100
},
{
"text": "Internal draft detailing legacy PostgreSQL replication failover procedures.",
"category": "database",
"status": "draft",
"view_count": 40
}
]
texts = [doc["text"] for doc in documents]
embeddings = list(embedding_model.embed(texts))
points = [
models.PointStruct(
id=str(uuid4()),
vector=emb.tolist(),
payload=doc
)
for doc, emb in zip(documents, embeddings)
]
client.upsert(
collection_name=COLLECTION_NAME,
points=points
)
print(f"Upserted {len(points)} records into '{COLLECTION_NAME}'.")Step 6: Query with Single-Stage Payload Filters
Execute a filtered search against the collection. Qdrant evaluates the filter criteria dynamically while walking the HNSW graph:
query_text = "how to fix packet drops and network timeouts inside containers"
query_vector = list(embedding_model.embed([query_text]))[0].tolist()
search_filter = models.Filter(
must=[
models.FieldCondition(
key="category",
match=models.MatchValue(value="devops")
),
models.FieldCondition(
key="status",
match=models.MatchValue(value="published")
)
],
should=[
models.FieldCondition(
key="view_count",
range=models.Range(gte=1000)
)
]
)
results = client.search(
collection_name=COLLECTION_NAME,
query_vector=query_vector,
query_filter=search_filter,
limit=3,
search_params=models.SearchParams(
hnsw_ef=64, # Search depth; higher values yield better recall at slight latency cost
quantization=models.QuantizationSearchParams(
rescore=True # Re-ranks top candidates against raw FP32 vectors on disk
)
)
)
for hit in results:
print(f"Score: {hit.score:.4f} | Category: {hit.payload['category']} | Text: {hit.payload['text']}")The rescore=True setting delivers the best of both worlds. Qdrant traverses the graph using lightweight 8-bit quantized vectors in RAM to locate candidate items in fractions of a millisecond. Once it narrows the search to the top candidates, it reads the original FP32 vectors from disk to re-score the final list. You preserve full floating-point accuracy without paying the RAM cost to keep uncompressed vectors in memory.
