Posted inAI

Structured Data Extraction with BAML and Python: Faster LLM Output and Visual Prompt Testing

AI tutorial - IT technology blog
AI tutorial - IT technology blog

Real-World Problem: When LLMs Break Production Data Pipelines

Nothing breaks a production microservice faster than an unparseable LLM response. During staging, everything looks flawless. You configure an endpoint expecting clean JSON, run thirty test payloads, and watch the schemas validate without a hitch.

Then traffic hits production. Edge cases show up immediately. A model wraps its response in unexpected markdown fences like ```json ... ```. Another inserts conversational filler like “Sure! Here is your requested JSON:” right before the opening brace. Worse, unescaped double quotes inside nested user strings trigger fatal json.decoder.JSONDecodeError exceptions, dropping required fields entirely.

Fixing these bugs usually degrades into tedious trial and error. Engineers copy failed payloads from production Datadog logs into a web playground. They tack on desperate prompt instructions like “Return ONLY valid raw JSON, no markdown”. Next, they patch Python f-strings and push a hotfix. The feedback cycle is slow. It burns developer time, costs money, and provides zero compile-time guarantees.

Root Cause Analysis: Why Standard LLM Pipelines Fail Schema Validation

To fix brittle extraction, we have to recognize where conventional Python tooling falls short:

1. String-Based Prompts Break Type Safety

Most frameworks treat prompts as raw f-strings or Jinja templates. You might maintain a pristine Pydantic model in schemas.py, but your prompt template has no static connection to it. Rename a field in Python, and your prompt remains untouched until runtime validation blows up in production.

2. High Latency in Native JSON Schema Modes

LLM providers offer native JSON modes and structured outputs via grammar-constrained decoding. While effective at enforcing schemas, constrained token masking adds noticeable compute overhead. In high-concurrency workloads, time-to-first-token (TTFT) can jump by 200ms to 400ms. For deep nested arrays, the model can freeze, hallucinate default values, or truncate mid-payload when it hits tight token limits.

3. The Wall Between IDEs and Web Playgrounds

Prompt iteration usually happens in web consoles—OpenAI Playground, Anthropic Console, or Google AI Studio. Your application logic and test fixtures live in VS Code. This split forces constant context switching. Testing edge cases means manually shuffling mock JSON between browser tabs and Python test runners.

Solutions Compared: Pydantic vs. Instructor vs. BAML

Developers have tackled structured generation with three primary patterns:

  • Raw Prompts + json.loads(): Simple to implement without third-party dependencies. However, it offers zero validation guarantees. Any stray comma or preamble crashes the parser.
  • Pydantic Output Parsers (e.g., LangChain): Injects schema definitions into prompt text and retries when parsing fails. The catch? Each retry triggers a fresh API call. A 700ms request quickly balloons into a 2.1-second latency spike that triples your token bill.
  • Instructor: Patches the official OpenAI or Anthropic SDKs to bind Pydantic models directly to function calling. It works reliably and preserves Python typing. However, prompt templates remain raw strings in your code, and running interactive prompt experiments still demands custom test scripts.
  • BAML (Basically A Made-up Language): A domain-specific configuration language backed by a native Rust compiler and an integrated VS Code previewer. BAML compiles prompts and schemas directly into typed Python, TypeScript, or Ruby clients. You get compile-time validation, sub-millisecond parsing, and an in-editor test playground.

Step-by-Step Implementation: Building a Resume Extractor with BAML and Python

BAML moves prompt templates and target schemas out of Python strings and into structured configuration files. The compiler handles validation and code generation ahead of time.

Step 1: Install the BAML CLI and Extension

Install the BAML runtime in your Python virtual environment:

pip install baml-py

Next, install the BAML extension from the VS Code Marketplace. Initialize the workspace in your project root:

baml-cli init

This scaffolds a baml_src/ directory containing client definitions, sample prompts, and configuration files.

Step 2: Declare Models and Prompts in BAML

Create a schema and extraction function in baml_src/extract_resume.baml:

client<llm> FastModel {
  provider openai
  options {
    model "gpt-4o-mini"
    api_key env.OPENAI_API_KEY
  }
}

class WorkExperience {
  company string
  title string
  years_active int
  responsibilities string[]
}

class CandidateProfile {
  name string
  email string
  skills string[]
  experience WorkExperience[]
}

function ExtractCandidate(raw_text: string) -> CandidateProfile {
  client FastModel
  prompt #"
    Extract candidate details from the following resume text:
    ---
    {{ raw_text }}
    ---

    {{ ctx.output_format }}
  "#
}

Pay attention to the {{ ctx.output_format }} macro. BAML translates your CandidateProfile schema into concise, model-specific instructions automatically. You never have to manually explain schema formats or JSON rules in prompt text.

Step 3: Test Prompts Visually in VS Code

Open baml_src/extract_resume.baml in VS Code. Look right above the function ExtractCandidate declaration. Click the Open Playground code lens.

An interactive test panel opens alongside your code. Paste raw resume text into the test input and click Run. The playground fires a request directly to the model. You can inspect token counts, examine raw provider responses, and verify parsed data against your schema—all without writing temporary test harnesses or bouncing into a browser.

Step 4: Generate Type-Safe Python Bindings

Generate the Python client code using the CLI:

baml-cli generate

This creates an auto-generated baml_client package in your repository. The generated module contains full type signatures, IDE autocomplete definitions, and runtime deserialization powered by a compiled Rust engine.

Step 5: Call the BAML Function from Python

Import the generated client into your application workflow:

import os
from baml_client import b
from baml_client.types import CandidateProfile

os.environ["OPENAI_API_KEY"] = "your-api-key-here"

resume_data = """
Alex Chen - [email protected]
Senior Backend Engineer with 6 years experience across Python, Rust, and Go.
Led microservice migration at FinTech Corp (2020-2023), boosting throughput by 42%.
Implemented distributed OpenTelemetry tracing and maintained multi-region EKS clusters.
"""

def process_resume(text: str) -> CandidateProfile:
    # Fully typed call with sync (b.ExtractCandidate) or async (b.ExtractCandidate.async_run)
    profile: CandidateProfile = b.ExtractCandidate(raw_text=text)
    return profile

result = process_resume(resume_data)

print(f"Extracted: {result.name} ({result.email})")
print(f"Skills: {', '.join(result.skills)}")
for job in result.experience:
    print(f"- {job.title} at {job.company} ({job.years_active} years)")

Key Takeaways for Production Architectures

  • Treat Prompts as Compiled Assets: Manage prompts and schemas with a dedicated compiler rather than scattered string concatenations across your codebase.
  • Shift Prompt Testing Left: Catch schema drifts and hallucinated keys in the IDE playground during development, not in production APM alerts.
  • Eliminate Costly Retry Loops: BAML’s native Rust parser repairs partial, quoted, or slightly malformed outputs in under 2ms. You get clean typed data on the first attempt without burning tokens on fallback retries.
Share:
Tags: