“serp api python” is one of the highest-intent developer searches in web data, and most tutorials answer it with a hello-world and stop — before pagination, before rate discipline, before the geo and device parameters that make results actually correct. This is the tutorial we wanted when we were the one searching: a complete, runnable pattern with the traps marked in the places they bite.

The Setup (Two Minutes, No Ceremony)

Thordata’s SERP API is an HTTP endpoint with a key — no SDK required, though the thordata-sdk package wraps it for Python if you prefer. The direct version, so you can see the shape:

import requests

def serp(query, api_key, location="United States", device="mobile",
         engine="google", fmt="json"):
    r = requests.post(
        "https://api.thordata.com/serp",
        json={"q": query, "engine": engine, "location": location,
              "device": device, "output": fmt},
        headers={"Authorization": f"Bearer {api_key}"},
        timeout=30,
    )
    r.raise_for_status()
    return r.json()

One design note before the traps: the location and device defaults are doing real work. Search results vary by geography and device class — a national-desktop scrape tells you about a world that most of your users don’t live in. Pricing context: responses bill per successful request at roughly $0.70–$1.20 per 1,000 by volume, and trial credits exist for the evaluation phase — so the tutorial below is genuinely free to run for a new account.

Trap One: The First Page Is a Lie (Pagination)

SERPs return a page of results, and the page you get is not “the results” — it’s one window. The trap bites in two directions: teams that treat page one as complete, and teams that paginate by naively re-requesting with an incremented start offset and discover inconsistent result sets (results shift between fetches; offsets aren’t stable snapshots).

The working pattern:

def serp_all(query, api_key, location, max_pages=3):
    results = []
    seen_urls = set()                      # dedup across pages: offsets drift
    for page in range(max_pages):
        payload = serp(query, api_key, location=location)
        organic = payload.get("organic", [])
        if not organic:
            break                          # empty page: stop, don't guess
        for item in organic:
            if item["url"] not in seen_urls:
                seen_urls.add(item["url"])
                results.append(item)
    return results

The seen_urls set is the quiet correctness fix: cross-page dedup handles the drift that offset-based pagination pretends doesn’t exist.

Trap Two: Parsing the JSON You Were Handed (Don’t)

The second trap is psychological: teams receive clean, structured JSON — organic array, positions, titles, URLs, features — and then write a “parser just to normalize a few things.” Don’t. The structured response *is* the normalization; every downstream parser is a schema-drift liability the next time the provider updates fields. Read the documented schema, consume it directly, and pin your dependency on the documented field names (raise loudly on absence rather than defaulting to None silently — a missing position should be an event, not a mystery).

def extract_rank(payload, target_domain):
    for item in payload.get("organic", []):
        if target_domain in item.get("url", ""):
            return item["position"]
    return None   # absent from page one — a fact, not an error

Trap Three: Rate Discipline (Be a Client, Not a Flood)

The third trap is the retry storm: a timeout triggers a retry, the retry triggers the same timeout, and a burst of concurrency “to speed things up” turns a transient failure into a rate-shaped one. The working discipline for per-request-billed APIs:

Sequential by default, concurrent by measurement — parallelize only where you’ve measured the win
Timeouts short and honest (30s), retries few and backed-off (two, exponential)
Failures are data — log the query, the market, the error class; your block-shaped and quota-shaped failures have different signatures and different fixes

Trap Four: The Agent-Shaped Future of This Exact Tutorial

The last trap is temporal: the tutorial above teaches you to *drive* the API from Python — but if the consumer of your search results is an LLM agent, the modern integration isn’t Python code at all. The Model Context Protocol endpoint exposes the same SERP machinery as agent-discoverable tools: the Thordata MCP server gives Claude Desktop, Cursor, Cline, Windsurf, or any MCP/JSON-RPC runtime live search, engine listings, request history, and usage statistics — with the agent reading the engine schemas and choosing its own parameters. New accounts get 500 free calls and the MCP tier draws on the same response balance (no separate subscription). The same tutorial, two consumption shapes: your code calls serp(), or your agent calls the tool.

The Complete Working Pattern

Assembled — a monitoring-grade loop with every trap handled:

def track_keyword(query, market, api_key, store, max_pages=2):
    payload_pages = [serp(query, api_key, location=market)
                     for _ in range(max_pages)]
    organic = dedup_pages(payload_pages)          # trap one
    record = {
        "query": query, "market": market,
        "target_rank": extract_rank(payload_pages[0], "yourdomain.com"),
        "observed_at": utc_now_iso(),             # freshness is a field, not hope
        "result_count": len(organic),
    }
    store.append(record)                          # schema consumed directly — trap two
    return record

Wrap it in a scheduler, store observations as events, and you have the minimum honest rank tracker — geo-pinned, timestamped, deduped, parser-free. Add the MCP endpoint when an agent joins the pipeline, and the same backend serves both consumption shapes.

FAQ

Node.js, PHP, Go, PHP — same patterns?
Yes — the traps (pagination drift, parser temptation, rate discipline) are language-independent, and the HTTP endpoint shape ports directly. The MCP path is even more language-agnostic: it speaks JSON-RPC, so any runtime in the supported client list connects without porting anything.

How do I keep costs predictable at monitoring scale?
Per-response billing makes it arithmetic: keywords × markets × devices × frequency × rate. A 5,000-keyword daily national+mobile tracker at volume rates prices in the low hundreds per month — know the multiplication before the deployment, not after the invoice.

What about historical SERP data for backtesting?
Collect today what you’ll want last year’s version of — scheduled monitoring is how history gets recorded. Licensed SERP datasets cover the pre-existing archive; your own scheduled collection covers everything after. Both land in the same warehouse if you plan the schema once.

Where’s the repo version of this tutorial?
The patterns above are the portable core; the MCP server documentation carries the current endpoint specifics and tool schemas. Bring the traps list to whichever integration you build — the hello-world is the easy 10%, and the traps are the 90% that shows up in production.