Posted inProgramming

What Is a Webhook? How It Works, Pros & Cons, and Real-World Use Cases

Programming tutorial - IT technology blog
Programming tutorial - IT technology blog

Polling vs. Webhooks: Two Ways to Get Notified

When I first built integrations between services, my default approach was polling — hit an API endpoint every 30 seconds, check if something changed, act on it. It works. But after running that in production for a few months, the cracks start showing: wasted API calls, rate limit headaches, and latency that’s always at least half your polling interval.

Webhooks flip the model. Instead of your app asking “did anything happen?” on a schedule, the external service tells you the moment something happens. Think of it as the difference between checking your mailbox every hour versus having the postman ring your doorbell.

A webhook is an HTTP callback — a POST request sent by one system to a URL you specify, triggered by a specific event. Payment processed? Stripe POSTs to your endpoint. New commit pushed? GitHub POSTs to your CI server. Code deployed? Your monitoring tool POSTs to your Slack webhook URL.

Here’s a side-by-side to make this concrete:

# Polling approach (runs every 30s)
while True:
    response = requests.get('https://api.example.com/orders?status=new')
    for order in response.json():
        process_order(order)
    time.sleep(30)

# Webhook approach (your endpoint, called by the external service)
@app.route('/webhook/new-order', methods=['POST'])
def handle_new_order():
    order = request.json
    process_order(order)
    return '', 200

The polling version runs whether or not anything happened. The webhook version runs exactly when an order arrives. At low traffic, the difference is trivial. At scale, it’s the difference between 50,000 wasted API calls per day and zero.

Pros and Cons After 6 Months in Production

What Works Well

  • Real-time delivery — Events arrive within milliseconds of occurring. My payment confirmation emails now go out in under a second after a Stripe charge succeeds.
  • Reduced server load — No idle polling loops. Your server only does work when there’s actual work to do.
  • Simpler code — The webhook handler is just a small HTTP endpoint. No state machine to track “last checked at” timestamps.
  • Third-party ecosystem — GitHub, Stripe, Shopify, Twilio, Slack — they all speak webhooks natively. If a service is worth integrating with, it almost certainly has webhook support.

Where It Gets Painful

  • Your endpoint must be publicly reachable — During local development, this is annoying. You need a tunnel tool like ngrok or localtunnel to expose your localhost.
  • You become the receiver, not the caller — You can’t easily retry if your server is down when the event fires. Most services retry a few times, but if you’re down long enough, you lose events.
  • Duplicate delivery is real — Network timeouts cause retries. I’ve processed the same payment event twice in production. Idempotency checks are not optional.
  • Security is your responsibility — Anyone who knows your endpoint URL can POST to it. You must verify the request actually came from the expected source.

The honest take: webhooks are strictly better than polling for event-driven use cases, but they shift operational responsibility to you. Polling is dumb and reliable; webhooks are smart and require care.

Recommended Setup for Production

1. Verify Webhook Signatures

Every major webhook provider ships a signature header. For GitHub it’s X-Hub-Signature-256, for Stripe it’s Stripe-Signature (which also embeds a timestamp to block replay attacks). Always verify before touching the payload.

import hmac
import hashlib

def verify_github_signature(payload_body: bytes, signature_header: str, secret: str) -> bool:
    """Verify GitHub webhook signature."""
    if not signature_header:
        return False
    
    hash_object = hmac.new(
        secret.encode('utf-8'),
        msg=payload_body,
        digestmod=hashlib.sha256
    )
    expected_signature = 'sha256=' + hash_object.hexdigest()
    
    # Use hmac.compare_digest to prevent timing attacks
    return hmac.compare_digest(expected_signature, signature_header)

Six months of production traffic, zero spoofed requests. I’ve seen maybe a dozen malformed POSTs hit the endpoint — automated scanners, mostly. The HMAC check bounces every one.

2. Return 200 Fast, Process Async

Your webhook endpoint should respond within 2–5 seconds or the sender assumes failure and retries. Don’t do heavy processing inline — push the job to a queue and return immediately.

import redis
import json

r = redis.Redis()

@app.route('/webhook/stripe', methods=['POST'])
def stripe_webhook():
    payload = request.get_data()
    sig_header = request.headers.get('Stripe-Signature')
    
    # Verify signature first
    if not verify_stripe_signature(payload, sig_header):
        return 'Invalid signature', 400
    
    # Push to queue, return immediately
    event = json.loads(payload)
    r.lpush('webhook_queue', json.dumps(event))
    
    return '', 200  # Respond fast

# Separate worker processes this queue
def worker():
    while True:
        _, job = r.brpop('webhook_queue')
        process_stripe_event(json.loads(job))

3. Handle Duplicates with Idempotency Keys

Store processed event IDs. Before handling any event, check whether you’ve already seen it.

def process_stripe_event(event: dict) -> None:
    event_id = event['id']  # e.g., 'evt_1ABC123...'
    
    # Check if already processed
    if r.sismember('processed_events', event_id):
        print(f'Skipping duplicate event: {event_id}')
        return
    
    # Process the event
    if event['type'] == 'payment_intent.succeeded':
        handle_payment(event['data']['object'])
    
    # Mark as processed (expire after 7 days)
    r.sadd('processed_events', event_id)
    r.expire('processed_events', 604800)

4. Use ngrok for Local Development

# Install ngrok
brew install ngrok  # macOS
# or download from ngrok.com for Linux/Windows

# Start your local server on port 5000, then expose it
ngrok http 5000

# ngrok gives you a public URL like:
# https://a1b2c3d4.ngrok.io
# Use this as your webhook URL during testing

Implementation Guide: GitHub Webhook for Auto-Deploy

Here’s a real pattern I use: push to main branch → webhook fires → server pulls and restarts.

Step 1: Set Up the Webhook Receiver

from flask import Flask, request, abort
import hmac, hashlib, subprocess, os

app = Flask(__name__)
SECRET = os.environ['GITHUB_WEBHOOK_SECRET']

@app.route('/deploy', methods=['POST'])
def deploy():
    # 1. Verify signature
    sig = request.headers.get('X-Hub-Signature-256', '')
    body = request.get_data()
    expected = 'sha256=' + hmac.new(SECRET.encode(), body, hashlib.sha256).hexdigest()
    if not hmac.compare_digest(sig, expected):
        abort(403)
    
    # 2. Only act on push to main
    data = request.json
    if data.get('ref') != 'refs/heads/main':
        return 'Ignored', 200
    
    # 3. Trigger deploy (non-blocking)
    subprocess.Popen(['/opt/scripts/deploy.sh'])
    return 'Deploying', 200

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=8080)

Step 2: Register the Webhook on GitHub

# Via GitHub CLI
gh api repos/{owner}/{repo}/hooks \
  --method POST \
  --field 'name=web' \
  --field 'active=true' \
  --field 'events[]=push' \
  --field 'config[url]=https://yourdomain.com/deploy' \
  --field 'config[content_type]=json' \
  --field 'config[secret]=your_secret_here'

Step 3: Run Behind a Reverse Proxy

# Nginx config snippet
location /deploy {
    proxy_pass http://127.0.0.1:8080;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header Host $host;
    
    # Rate limit to prevent abuse
    limit_req zone=webhook burst=10 nodelay;
}

Where Webhooks Pay Off Most

Auto-deploy is one pattern. Here’s where I’ve consistently seen webhooks outperform polling:

  • Payment processing — Stripe and PayPal fire events for successful charges, refunds, and disputes. Essential for async payment flows where you can’t block the user waiting for confirmation.
  • CI/CD pipelines — GitHub and GitLab webhooks trigger builds on every push or PR update. Faster feedback loops than any polling interval can achieve.
  • Chat integrations — Slack and Discord incoming webhooks let you post notifications from any service with a single HTTP POST. No OAuth, no SDK — just a URL.
  • E-commerce — Shopify webhooks notify you when orders are placed, inventory drops below a threshold, or customers sign up. Some high-volume stores process thousands of these per hour.
  • Monitoring alerts — PagerDuty, Datadog, and Grafana all support outbound webhooks to custom endpoints when alerts fire. Useful for routing alerts to internal tooling.

My personal rule: if I’m polling an external API more than once per minute, I look for a webhook alternative first. The overhead of standing up a webhook endpoint is real — but it’s worth it almost every time.

Share:
Tags: