LLM Testing with DeepEval: Measure Hallucination and Automate CI/CD in Python

AI tutorial - IT technology blog
AI tutorial - IT technology blog

Real-World Problem: Silent Failures in LLM Features

Shipping an LLM feature usually feels triumphant—until real users start hitting edge cases. Traditional unit tests check predictable inputs against exact outputs. Send 2 + 2, expect 4. If a function throws an unhandled exception or dumps a 500 status, your test runner flags it within seconds.

Generative models strip away that safety net. An API call can return an HTTP 200 with clean, parseable JSON, yet deliver nonsense. The output might invent non-existent features or ignore what the user asked.

We hit this exact wall last quarter. Our team trimmed a system prompt to reduce token latency by 15%. All 240 of our standard Pytest unit tests passed. Two days later, an enterprise customer flagged that our documentation bot was hallucinating API endpoints that never existed in version 3.x. The code worked flawlessly; the language output failed silently.

Root Cause Analysis: Why Traditional Testing Misses LLM Regressions

Standard testing tools expect deterministic code. LLMs violate those assumptions in four fundamental ways:

  • Non-deterministic output: Even set to temperature=0.0, models can produce subtle wording variations across runs. Hardcoded string assertions like assert response == expected fail almost immediately.
  • Semantic nuances: Checking whether an answer is helpful or factually grounded requires semantic context. Simple substring searches or regex rules cannot evaluate factual consistency.
  • Prompt and retrieval drift: Changing a prompt prefix, adjusting vector chunk sizes from 500 to 800 tokens, or swapping base models can inject hallucinations across dozens of queries. None of these changes trigger standard runtime tracebacks.
  • The review bottleneck: Asking engineers to spot-check 50 completions in a spreadsheet adds hours to review cycles. It is slow, expensive, and inevitably misses rare edge cases.

Solutions Compared: Finding the Right Evaluation Strategy

Teams typically attempt three strategies to guard LLM quality:

1. Manual Review & Spreadsheets

Engineers export batches of prompt completions to Google Sheets, scoring each on a 1-to-5 scale. Human reviewers bring reliable intuition, but this setup cannot scale past a few dozen iterations. It is costly, introduces personal reviewer bias, and cannot block a broken pull request in a build pipeline.

2. Ad-Hoc LLM-as-a-Judge Scripts

Developers write quick helper scripts asking a model like gpt-4o to judge, “Is this answer relevant? Reply YES or NO.” While quick to sketch out, bespoke judge scripts rarely maintain consistent scoring thresholds across team members. Managing raw prompts for multiple evaluation criteria quickly becomes its own maintenance burden.

3. Dedicated Evaluation Frameworks (DeepEval)

DeepEval plugs production-grade evaluation metrics directly into Pytest. It gives you standardized metrics—like Hallucination, Answer Relevancy, Faithfulness, and Contextual Precision—complete with calibrated scoring thresholds. If a prompt tweak causes relevancy to drop below your designated threshold, the build fails just like an ordinary broken test.

Implementation: DeepEval Quality Gates in Practice

Let’s set up DeepEval, write targeted tests for relevancy and hallucination, and enforce these checks automatically on every pull request.

Step 1: Environment Setup

Create an isolated virtual environment and install DeepEval along with Pytest:

python3 -m venv venv
source venv/bin/activate
pip install deepeval==1.2.0 pytest==8.2.0

DeepEval uses an evaluation model (GPT-4o by default) to score output quality. Export your API credentials in your active shell:

export OPENAI_API_KEY="sk-proj-your-api-key"

Step 2: Testing Answer Relevancy

The AnswerRelevancyMetric scores whether a response answers the user’s explicit question without rambling or adding irrelevant claims.

Create tests/test_relevancy.py:

import pytest
from deepeval import assert_test
from deepeval.test_case import LLMTestCase
from deepeval.metrics import AnswerRelevancyMetric

def test_answer_relevancy():
    user_input = "How do I reset my account password?"
    model_output = (
        "To reset your account password, click 'Forgot Password' on the login screen, "
        "enter your registered email address, and follow the link delivered to your inbox."
    )
    
    # Score ranges from 0.0 to 1.0. 0.7 requires strong direct alignment.
    metric = AnswerRelevancyMetric(threshold=0.7)
    
    test_case = LLMTestCase(
        input=user_input,
        actual_output=model_output
    )
    
    assert_test(test_case, [metric])

Run the test through the DeepEval CLI runner:

deepeval test run tests/test_relevancy.py

DeepEval extracts statements from the completion, checks them against the query, and produces a score between 0 and 1. If the score misses the 0.7 bar, the test fails and prints the specific lines flagged as off-topic.

Step 3: Measuring and Catching Hallucinations

Hallucination testing compares your model’s answer against the source documents provided by your retrieval layer. HallucinationMetric checks whether the completion makes claims unsupported by those reference chunks.

Create tests/test_hallucination.py:

import pytest
from deepeval import assert_test
from deepeval.test_case import LLMTestCase
from deepeval.metrics import HallucinationMetric

def test_hallucination_guard():
    retrieved_docs = [
        "Our refund policy allows customers to request a full refund within 30 days of purchase.",
        "Refunds are credited back to the original payment method within 5 to 7 business days."
    ]
    
    user_input = "Can I get a cash refund in your physical store?"
    # Simulated faulty completion introducing an unsubstantiated policy
    faulty_output = (
        "You can get a full refund within 30 days. "
        "We also provide cash refunds at any retail store location."
    )
    
    # For HallucinationMetric, threshold is the MAXIMUM acceptable hallucination score.
    # 0.0 demands zero ungrounded statements.
    hallucination_metric = HallucinationMetric(threshold=0.1)
    
    test_case = LLMTestCase(
        input=user_input,
        actual_output=faulty_output,
        context=retrieved_docs
    )
    
    assert_test(test_case, [hallucination_metric])

Execute the test suite:

deepeval test run tests/test_hallucination.py

Because the completion fabricated an in-store cash refund policy not present in the context, DeepEval assigns a high contradiction score (typically ~0.50). This easily breaches the 0.1 threshold and causes the assertion to fail with a clear diagnostic log.

Step 4: Integrating DeepEval into GitHub Actions CI/CD

Manual testing during development is helpful, but continuous automation keeps quality steady. Add .github/workflows/llm_eval.yml to run these checks on every PR targeting main:

name: LLM Evaluation CI

on:
  pull_request:
    branches:
      - main

jobs:
  evaluate-llm:
    runs-on: ubuntu-latest
    steps:
      - name: Check out repository
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'
          cache: 'pip'

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt
          pip install deepeval pytest

      - name: Run DeepEval Quality Gates
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
        run: |
          deepeval test run tests/ --json-report

Once this workflow runs, prompt edits are verified automatically alongside regular application code. If a teammate alters prompt instructions or swaps chunking parameters in a way that introduces hallucinations, GitHub Actions fails the check. Broken outputs get caught in code review—not in front of your customers.

Share: