Deploying Qdrant on Docker: A 6-Month Production Review for RAG and Semantic Search

Table of Contents

Context & Why: Moving Vector Workloads to Rust

Six months ago, our team hit a wall with our semantic search infrastructure. We were serving customer-facing RAG workflows across 20 million chunks embedded with OpenAI’s 1536-dimensional models. Our legacy setup—PostgreSQL with pgvector running on an AWS r6i.4xlarge instance—struggled under load. During bulk re-indexing runs, p99 search latency spiked from 45ms to well over 140ms. Worse, Postgres consumed upwards of 160GB RAM, frequently flirting with out-of-memory crashes.

Relational databases and document stores remain great for structured transactional records. But high-dimensional vector search demands specialized memory layouts and hardware acceleration. We evaluated Milvus, Pinecone, and Qdrant before standardizing on Qdrant.

Its Rust codebase eliminated garbage-collection pauses. SIMD acceleration (AVX-512) maximized compute throughput on x86 chips. After migration, our p99 query latency dropped to 11.8ms under 450 QPS. RAM usage fell by 68% once we enabled scalar quantization.

Installation: Production-Ready Docker & Docker Compose Setup

Single-container ad-hoc instances fail fast in production. You need predictable disk persistence, explicit resource limits, and automated health checks configured through Docker Compose.

Project Directory Layout

mkdir -p /opt/qdrant/{data,config,snapshots}
cd /opt/qdrant

Docker Compose Configuration

Create a robust docker-compose.yml file targeting the stable v1.8.4 release:

version: '3.8'

services:
  qdrant:
    image: qdrant/qdrant:v1.8.4
    container_name: qdrant_production
    restart: unless-stopped
    ports:
      - "6333:6333" # REST API & Web UI
      - "6334:6334" # gRPC API
    environment:
      - QDRANT__SERVICE__HTTP_PORT=6333
      - QDRANT__SERVICE__GRPC_PORT=6334
      - QDRANT__SERVICE__ENABLE_CORS=true
      - QDRANT__SERVICE__API_KEY=${QDRANT_API_KEY}
    volumes:
      - ./data:/qdrant/storage
      - ./config/config.yaml:/qdrant/config/production.yaml:ro
      - ./snapshots:/qdrant/snapshots
    ulimits:
      nofile:
        soft: 65535
        hard: 65535
    deploy:
      resources:
        limits:
          cpus: '8.0'
          memory: 32G
        reservations:
          cpus: '4.0'
          memory: 16G
    healthcheck:
      test: ["CMD-SHELL", "curl -sf http://localhost:6333/healthz || exit 1"]
      interval: 15s
      timeout: 5s
      retries: 3
      start_period: 10s

Generate a 256-bit API key and write it to .env before launching the container daemon:

echo "QDRANT_API_KEY=$(openssl rand -hex 32)" > .env
docker compose up -d

Configuration: Optimizing Storage, HNSW, and Quantization

Stock defaults prioritize quick local prototyping over indexing throughput. For collections holding tens of millions of vectors, fine-tune the indexing thresholds and offload JSON payloads to NVMe disk.

Production `config/config.yaml`

storage:
  storage_path: /qdrant/storage
  snapshots_path: /qdrant/snapshots
  on_disk_payload: true

service:
  max_request_size_mb: 32
  max_workers: 8

hnsw_index:
  m: 16
  ef_construct: 128
  full_scan_threshold: 10000
  max_indexing_threads: 4
  on_disk: false

Configuring Scalar Quantization for Memory Efficiency

Raw 32-bit floating-point vectors get expensive at scale. Twenty million 1536-dimensional vectors require roughly 122.8 GB of RAM strictly for vector data (20,000,000 * 1536 * 4 bytes). Enabling 1-byte scalar quantization (int8) compresses that footprint down to ~30.7 GB. In our recall evaluations against a test set of 10,000 gold queries, accuracy dropped by only 0.8% while throughput doubled.

Here is our production collection provisioning script using the official Python client:

import os
from qdrant_client import QdrantClient
from qdrant_client.http import models

client = QdrantClient(
    url="http://localhost:6333",
    api_key=os.getenv("QDRANT_API_KEY"),
    prefer_grpc=True
)

# Initialize collection with scalar quantization
client.create_collection(
    collection_name="knowledge_base",
    vectors_config=models.VectorParams(
        size=1536,
        distance=models.Distance.COSINE,
        on_disk=False  # Keep graph in RAM for fastest traversal
    ),
    optimizers_config=models.OptimizersConfigDiff(
        indexing_threshold=20000,
        memmap_threshold=50000
    ),
    quantization_config=models.ScalarQuantization(
        scalar=models.ScalarQuantizationConfig(
            type=models.ScalarType.INT8,
            quantile=0.99,
            always_ram=True
        )
    )
)

# Index metadata fields for rapid boolean/equality filtering
client.create_payload_index(
    collection_name="knowledge_base",
    field_name="tenant_id",
    field_schema=models.PayloadSchemaType.KEYWORD
)
client.create_payload_index(
    collection_name="knowledge_base",
    field_name="created_at",
    field_schema=models.PayloadSchemaType.INTEGER
)

Verification & Monitoring: Health Checks and Production Metrics

After standing up the cluster, validate ingestion and query mechanics before routing real user traffic.

1. Testing Batch Upsert and Filtered Search

Verify tenant-scoped semantic queries directly in Python:

# Insert sample document point
client.upsert(
    collection_name="knowledge_base",
    points=[
        models.PointStruct(
            id="c56a4180-65aa-42ec-a945-5fd21dec0538",
            vector=[0.012] * 1536,
            payload={
                "tenant_id": "tenant_acme_corp",
                "doc_title": "Qdrant Production Architecture",
                "created_at": 1717200000
            }
        )
    ]
)

# Query with pre-filtering
search_hits = client.query_points(
    collection_name="knowledge_base",
    query=[0.012] * 1536,
    query_filter=models.Filter(
        must=[
            models.FieldCondition(
                key="tenant_id",
                match=models.MatchValue(value="tenant_acme_corp")
            )
        ]
    ),
    limit=5,
    with_payload=True
)

for hit in search_hits.points:
    print(f"Score: {hit.score:.4f} | Title: {hit.payload['doc_title']}")

2. Prometheus Metrics Integration

Qdrant exposes open telemetry metrics over HTTP on /metrics. Wire it directly into your central scraper:

scrape_configs:
  - job_name: 'qdrant'
    metrics_path: '/metrics'
    bearer_token: 'YOUR_QDRANT_API_KEY'
    static_configs:
      - targets: ['qdrant_production:6333']

3. Key Metrics to Watch

app_telemetry_rest_requests_duration_seconds_bucket: Tracks search latency distribution. Alert if p99 exceeds 35ms over a 5-minute rolling window.
collections_vector_count: Total vector records per collection. Use this to track growth trends.
process_resident_memory_bytes: Memory allocated directly by the process. Set warnings at 80% container memory ceiling to prevent Linux OOM kills.
cluster_pending_operations: Unprocessed optimizer queue depth. Persistent spikes indicate that index compaction is falling behind batch write volume.

Operational Lessons from 6 Months of Uptime

Three concrete patterns kept our infrastructure stable:

Payload indexing is mandatory: Always build dedicated payload indexes for tenancy or timestamp fields. Filtering unindexed fields forces full segment scans that kill query concurrency.
Switch ingestion pipelines to gRPC: Sending vectors via gRPC over port 6334 cut our ingest CPU utilization by 28% and boosted write throughput from 1,800 to 4,200 points/sec per worker.
Automate snapshot rotation: Take point-in-time snapshots nightly via the /collections/{name}/snapshots API, compress them, and sync them to an S3 cold bucket. Snapshot creation is lock-free and never blocks read traffic.