EN
English
简体中文
Log inGet started for free

Blog

Proxies

a-real-estate-data-rfp-annotated-what-a-good-answer-actually-looks-like

A Real Estate Data RFP, Annotated: What a Good Answer Actually Looks Like

Property-data collection RFPs ask the wrong questions in 90% of drafts (“How many IPs do you have?” “Can you scrape Zillow?”), and vendors answer them without anyone noticing the questions that determine project survival — coverage semantics, geo fidelity, change-detection economics, and structured-field depth — went unasked. This is a real estate data RFP read line by line, with a good answer, a red flag, and the architecture behind it.

If you’re a proptech data lead, a brokerage analytics team, or a market-research firm assembling listing data, you’ve drafted this document before. Let’s annotate it.

Requirement 1: “Collect listing data from major portals (Zillow, Realtor.com, Redfin, Homes.com) and 40+ regional MLS sites”

What the RFP means: coverage of the platforms that matter plus the long tail that differentiates you.

A good answer distinguishes the two tiers explicitly. For the majors: pre-built scrapers with structured field extraction (price, beds, baths, square footage, days-on-market, status, image sets), not raw HTML hand-offs. Thordata’s catalog covers 120+ pre-built targets including Zillow, with results delivered as JSON, CSV, or XLSX at roughly $0.50–$1.00 per 1,000 results — and the field schema is stable across portal redesigns, because parser maintenance is the vendor’s bill, not yours. For the MLS long tail: raw residential proxy capability with the same account and dashboard, city-targeted where the MLS is city-scoped, at published volume pricing ($2.00/GB down to $0.65/GB).

Red flag: an answer that promises “any site” without distinguishing which are maintained scrapers versus which are your engineering problem.

Requirement 2: “Data must reflect what a prospective buyer in that metro actually sees”

What the RFP means: geo fidelity, and it’s quietly the hardest requirement in the document.

Property markets are hyper-local, and portal behavior is location-sensitive: what a search returns from a Phoenix IP differs from a Phoenix-residential IP on mobile. Centralized collection produces “average Arizona” data for a Phoenix product. The correct answer names targeting granularity — country, state, city, and ASN-level, without surcharge — and session behavior: sticky sessions for search-and-paginate flows (Thordata supports up to 90 minutes) so a full result set comes from one coherent session, not forty unrelated ones.

A good answer also volunteers how it proves this: exit-IP verification on sample runs, geo-match rates, and a willingness to let you audit a metro with your own local testers. Red flag: “we have 195 countries” as an answer to a question about Phoenix.

Requirement 3: “Refresh cadence: daily for active markets, weekly for the long tail; detect price changes within 24 hours”

What the RFP means: this is a change-detection budget question dressed as a feature.

The instinct is to re-fetch everything at the highest cadence, which is how collection budgets die. The economics that work: per-result pricing makes daily refreshs a linear decision (a full 100,000-listing sweep in one metro is about $50 at the volume rate, not a “let me check the traffic budget” conversation), and structured status fields make delta-detection trivial — you diff records, not HTML.

A good answer separates the pipeline: high-frequency deltas on hot inventory (status flips, price cuts, new listings), low-frequency completeness checks on the long tail, and scheduled collection jobs for both rather than cron scripts you maintain. Red flag: pricing that makes full refreshs feel expensive — if daily sweeps require an apology email to finance, the architecture fights your requirements.

Requirement 4: “Images, not just metadata: photo sets for every active listing, updated when the set changes”

What the RFP means: image collection at scale, the budget line that surprises teams.

Listing images are storage, bandwidth, and deduplication questions. A good answer: the same record-based architecture (fetch structured records, diff image URLs, pull only changed assets), object-storage delivery, and record-level pricing on the metadata path so the expensive path — bytes — only runs on actual change. For teams that also need visual understanding models (staging detection, photo quality scoring), the same vendor’s dataset catalog serves pre-built image-text records from about $0.25 per 1,000 records — the training-side counterpart to the collection pipeline, in one account.

Red flag: a flat “per GB of media” line with no change-detection strategy.

Requirement 5: “Off-market and coming-soon signals where legal: pre-listing indicators, days since last sale, withdrawal velocity”

What the RFP means: this is a search-visibility question wearing a data-lawyer costume — because half these signals live in how properties and agents surface in search, not just on portals.

The serious proptech roadmap now includes: monitoring what appears for “homes in [suburb]” queries (portals, indexers, and — increasingly — answer-layer placements), tracking agent-brand SERP presence, and timing market-entry intelligence with search demand. The SERP monitoring solution is the fit: scheduled structured collection of organic results, local pack presence, and ads at roughly $0.70 per 1,000 responses, geo-pinned per market, delivered as records that join into the same warehouse as listing data. A continuous SERP data crawling job replaces the manual “check Google from incognito” routine that every market-research team quietly maintains.

Red flag: a data vendor whose answer to visibility questions is “that’s a different tool.”

Requirement 6: “Compliance posture: robots.txt respect, rate limiting, documented sourcing, deletion handling on request”

What the RFP means: you need to survive your own legal review and your next enterprise deal.

A good answer here is concrete: documented opt-in sourcing for residential pools (Thordata describes its model this way), rate-limit discipline built into managed collection rather than left to your engineer’s conscience, and a deletion process with stated turnaround. If your requirement is a signed enterprise agreement with named SLAs and DPAs, that’s an enterprise-motion vendor’s strength — and a legitimate one; the published self-serve model above serves technically self-sufficient teams, and neither posture is a proxy for quality.

Red flag: either direction — refusing to discuss provenance, or refusing to discuss what a contract can’t give you.

Requirement 7: “Integration: Python, scheduled jobs, n8n, output to our warehouse”

What the RFP means: total week-to-first-data matters more than any feature line.

Annotated example of the call the winning architecture makes:

from thordata import Thordata

client = Thordata(api_key="YOUR_API_KEY")

def metro_sweep(city: str, page: int = 1):
    result = client.scrape(
        scraper="zillow_listings",       # maintained parser, stable schema
        query=f"{city},AZ",
        geo="US-AZ",                      # collected as a local would see it
        output_format="json",
    )
    for listing in result.records:
        warehouse.upsert({
            "mls_like_id": listing["zpid"],
            "price": listing["price"],
            "status": listing["status"],  # for sale | pending | off
            "days_on_market": listing.get("days_on_market"),
            "photos_hash": hash(tuple(listing["photo_urls"])),
            "collected_at": result.created_at,
        })

The comment that matters: every field here is the vendor’s maintenance problem, not yours. Portal redesigns are their outage, not yours.

Scoring the Answers

A useful evaluation rubric weights these five questions most heavily: (1) does the vendor maintain parsers with a stable schema, (2) does geo fidelity survive an audit in your worst metro, (3) does per-result pricing make your target refresh cadence cheap enough to not argue about, (4) does the same account cover the long tail of sites with raw proxy capability, and (5) is provenance documented in writing. Two platforms typically score (1) and (3) with a single product; the ones that score all five — collection, pre-built scrapers, unlocker for the hostile 5%, SERP-side monitoring, and record-priced datasets — are the ones that shrink your vendor count without shrinking your risk. Run the free trial credits against one metro, one week, one warehouse table — and let the join between listing data and search-visibility data make the argument the RFP couldn’t.