Six months ago, our team ran into a wall that hits almost every retrieval-augmented generation (RAG) system in production: parsing messy, layout-heavy PDFs. We needed to ingest over 45,000 quarterly financial filings and engineering specs.
Each document was crammed with three-column layouts, nested 12-column balance sheets, and inline mathematical notation. Traditional text extractors flattened everything into an unreadable mess, scrambling tabular rows and turning differential equations into gibberish.
We swapped out our fragile regex pipelines for IBM Docling. Since that rollout, our extraction error rates dropped by roughly 82%, processing tens of thousands of complex pages reliably. Here is an honest operational breakdown of how Docling compares to legacy tooling and how to deploy it inside your ingestion stack.
Document Extraction Approaches Compared
Parsing PDF documents for vector retrieval usually boils down to three primary architectural paradigms:
1. Rule-Based and Spatial Extractors (PyPDF, pdfplumber, PyMuPDF)
These libraries extract raw character glyphs based on the bounding box coordinates embedded in the PDF byte stream. They run blazingly fast—often taking under 40 milliseconds per page—and consume practically no RAM.
- Where they stumble: Multi-column layouts trip them up easily. Sentences often splice together across columns. Borderless tables get scrambled, and mathematical typography dissolves into unsearchable symbol fragments.
2. Heavy Vision-Language Pipelines (Nougat, OCR + Multimodal LLMs)
This approach rasterizes pages into high-resolution images, passing them directly to vision models or frontier multimodal LLMs like GPT-4o or Claude 3.5 Sonnet to transcribe into Markdown.
- Where they stumble: Cloud API costs quickly spiral out of control. Transcribing a 100-page report can cost $1.50 to $3.00, while latency crawls to 3 to 8 seconds per page. Even worse, LLMs still suffer from occasional hallucinated numbers in dense balance sheets.
3. Layout-Aware Vision Models (IBM Docling)
Docling hits the sweet spot between speed and accuracy. It pairs specialized layout detection models—trained on datasets like DocLayNet—with targeted table-structure predictors (TableFormer). Docling recovers natural reading order, reconstructs merged cells, and converts formulas directly to LaTeX without calling external APIs.
Pros and Cons: 6 Months in Production
Processing millions of pages taught us where Docling excels and where it requires careful babysitting.
Pros
- Reliable Reading Order: Docling tracks the flow of multi-column papers cleanly, preventing unrelated paragraphs from bleeding into each other.
- Resilient Table Structure: It accurately reconstructs multi-row headers and merged cells into clean Markdown or HTML tables, eliminating corrupted numbers in retrieved context.
- Native LaTeX Formulas: Math expressions parse straight into LaTeX strings, keeping equations fully intact for math-aware embedding models.
- Air-Gapped and Private: Everything executes inside your own VPC—no sensitive financial or clinical records leave your security perimeter.
- Downstream Chunking: Built-in chunkers integrate directly with LangChain and LlamaIndex while honoring document section boundaries.
Cons
- Compute Demands: Unlike PyPDF’s near-zero footprint, Docling runs PyTorch deep learning models under the hood. On a 4-core CPU, expect roughly 1.5 to 3.5 seconds per page. High-throughput queues definitely need GPU acceleration.
- Memory Footprint: Worker processes typically consume 2.5GB to 4GB of RAM each. You must tune container concurrency carefully to prevent Kubernetes out-of-memory (OOM) kills.
- Scanned Document Fallbacks: If a PDF lacks an embedded text layer and forces fallback to EasyOCR or Tesseract, processing time can jump to over 6 seconds per page.
Recommended Architecture and Environment Setup
Never run Docling synchronously inside your web application server. Instead, decouple ingestion using a background worker queue like Celery, Temporal, or AWS SQS. Provision each worker node with at least 4 vCPUs and 8GB of RAM, or attach a cost-effective GPU such as an NVIDIA T4 (16GB VRAM), which slashes page conversion times down to 350 milliseconds.
Install the core package alongside PyTorch with CUDA support:
# Install Docling and chunking utilities
pip install docling docling-core
# Install PyTorch with CUDA 12.1 acceleration
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121Pin these package versions in your requirements.txt to prevent unexpected model weights from downloading during production rollouts.
Implementation Guide: Building the Extraction Pipeline
The following script shows our end-to-end extraction pipeline: configuring the parser, pulling structured tables and formulas, and generating chunk embeddings.
Step 1: Document Conversion and Feature Configuration
Docling provides high-level conversion through DocumentConverter, allowing precise control over OCR engines and table parsing depth:
from pathlib import Path
from docling.document_converter import DocumentConverter, PdfFormatOption
from docling.datamodel.pipeline_options import PdfPipelineOptions, TableStructureOptions
def configure_converter() -> DocumentConverter:
# Configure parsing and OCR switches
pipeline_options = PdfPipelineOptions()
pipeline_options.do_ocr = True
pipeline_options.do_table_structure = True
pipeline_options.table_structure_options = TableStructureOptions(
do_cell_matching=True
)
# Initialize converter with custom options
converter = DocumentConverter(
format_options={
"pdf": PdfFormatOption(pipeline_options=pipeline_options)
}
)
return converter
converter = configure_converter()
result = converter.convert("sample_financial_report.pdf")
doc = result.documentStep 2: Inspecting Tables, Formulas, and Markdown Output
After conversion completes, access the document as unified Markdown, or inspect structural elements individually:
# Export full document to clean Markdown
markdown_content = doc.export_to_markdown()
print("--- Extracted Document Markdown Sample ---")
print(markdown_content[:500])
# Inspect individual detected tables
print(f"\nDetected tables: {len(doc.tables)}")
for idx, table in enumerate(doc.tables):
print(f"\n--- Table {idx + 1} ---")
# Export directly to Pandas DataFrame
df = table.export_to_dataframe()
print(df.head())
# Extract mathematical expressions
if hasattr(doc, "formulas"):
for idx, formula in enumerate(doc.formulas):
print(f"Formula {idx + 1}: {formula.text}")Step 3: Layout-Aware Chunking for RAG Embeddings
Fixed-character splitters routinely slice tables in half, ruining semantic context during vector search. Docling’s HybridChunker respects structural boundaries, keeping related data together:
from docling.chunking import HybridChunker
# Initialize chunker targeting 512 tokens with layout awareness
chunker = HybridChunker(max_tokens=512, merge_peers=True)
chunks = list(chunker.chunk(doc))
print(f"Total semantic chunks generated: {len(chunks)}\n")
for idx, chunk in enumerate(chunks[:3]):
print(f"[Chunk {idx + 1}] Length: {len(chunk.text)} characters")
print(chunk.text)
# Preserve section metadata for vector payload filtering
print(f"Metadata: {chunk.meta.headings}\n")Passing these structured chunks into your embedding model gives the retrieval engine clean, coherent context. Tables stay intact, sentences follow natural reading order, and mathematical expressions retain their exact formulas.
