Why Your Product Feed Never Matches Your Supplier’s: A Field Diagnostic

Catalog mismatch — the same product existing as three different records in your feed, your supplier’s feed, and the marketplace — is a multimodal data problem disguised as a string-matching bug. Titles lie, SKUs drift, and image sets diverge. This diagnostic walks through the five root causes in the order a field engineer actually encounters them, ending with the record structure that fixes matching at the source.

The ticket usually reads: “supplier feed says 41,203 SKUs, our catalog shows 39,871, marketplace listings are duplicated.” Someone proposes fuzzy string matching. It helps for a week. Then a supplier renames 4,000 products for SEO and the matching layer collapses again. Here is the diagnostic, in the order the failures actually stack.

Diagnosis 1: You Are Matching on the Field That Lies Most

Titles are marketing copy, not identity. “Ultra-Comfort Ergonomic Office Chair with Lumbar Support (Black)” and “Pro Ergo Chair — Black” are the same product; “V2” and “V2 (2025 revision)” might not be. Any matching pipeline keyed primarily on title text inherits the supplier’s marketing calendar.

The fix isn’t a better string algorithm — it’s matching on richer evidence: the image set, the attribute block, the category path, and the description, weighted together. Which is a way of saying: matching is a multimodal problem, and text-only pipelines are doing it with one hand tied.

Diagnosis 2: Your Embeddings Were Trained on the Wrong Distribution

Teams that do move to embedding-based matching usually grab a general-purpose model. Then they discover e-commerce is a dialect: “PS5 DualSense Edge” needs to land near “PlayStation 5 Edge controller,” and “1TB” matters more than “fast.” General embeddings get you 80% of the way; the last 20% — the part that determines whether 4,000 products match or 600 do — comes from domain-adapted similarity, tuned on real catalog pairs.

That tuning set is a data problem: paired records across marketplaces and suppliers, with images and attributes aligned. This is precisely what structured multimodal datasets sell — records from 100+ domains at roughly $0.25 per 1,000 records, with image references and text fields aligned per record. A few hundred thousand records of cross-marketplace product pairs is the difference between a matching layer you retune quarterly and one you rebuild annually.

Diagnosis 3: One Side of the Match Is Stale

Matching runs fine — then the supplier publishes a Tuesday update, and 800 records change images, attributes, or existence. If your pipeline matches against a snapshot, every supplier edit becomes a temporary mismatch, and your dedup metrics quietly degrade until someone reruns the full job.

The fix is operational: scheduled collection. The same infrastructure that supplies the datasets also runs recurring collection jobs — Thordata’s platform pairs its dataset catalog with scraper APIs (120+ pre-built targets, ~$0.50–$1.00 per 1,000 results) and raw residential collection (published slider, $2.00/GB down to $0.65/GB at volume), so the supplier side of the match refreshes on a schedule instead of on a crisis.

Diagnosis 4: The Marketplace Layer Doubles Everything

You fix supplier matching, and the marketplace feed arrives with its own listings for the same products — different titles, retouched images, different SKUs. Now the match is three-way, and record identity across three vocabularies needs a join key that isn’t any side’s SKU.

The working pattern: build the match on multimodal evidence, then assign your own canonical ID, and keep the provenance of every side attached:

# Illustrative: multimodal record → canonical entity
def resolve(record):
    vec = embed(
        title=record["title"],
        attributes=record["attributes"],   # brand, size, color, capacity
        image=record["image_url"],         # image embedding joins the text
    )
    match = index.search(vec, top_k=3)
    if match.score > THRESHOLD:
        return match.canonical_id           # existing product
    return mint_new_canonical_id(record)    # genuinely new SKU

# Every side keeps its own ID, mapped to the canonical one
catalog.upsert({
    "canonical_id": resolve(supplier_record),
    "source": "supplier_a",
    "source_sku": supplier_record["sku"],
    "image_hash": hash_image(supplier_record["image_url"]),
})

The image_hash line matters more than it looks: image identity survives title rewrites, which makes it the most stable join key in a marketing-driven catalog.

Diagnosis 5: Nobody Can Audit the Match

The final failure is organizational. A buyer asks why two listings merged (or didn’t), and the answer is “the model said so.” Without provenance — which fields matched, from which source records, at what score — every matching decision is unauditable, and the catalog team loses trust in the whole layer.

This is where record-level structure pays its second dividend: if the dataset and collection layers deliver records with source, timestamp, and field-level lineage, the matching layer can expose its evidence per decision. Auditable matching survives procurement reviews and buyer skepticism; unauditable matching gets replaced by interns with spreadsheets.

The Diagnostic as a Checklist

FAQ

How much does the data side of this actually cost?
Less than the engineering around it. A 300,000-record multimodal pilot from a structured dataset catalog is on the order of $75 at per-record pricing; the scheduled collection that keeps it fresh scales with your supplier count. The expensive version of this problem is the one you’re already paying for — duplicated listings, mismatched inventory, and manual reconciliation.

We’re a marketplace ourselves, not a retailer. Different problem?
Same problem, amplified: you have N suppliers, not one, so Diagnosis 4 multiplies and the canonical-ID layer becomes your actual product. Marketplaces that solve this well treat the matching layer as core IP, fed by dataset-scale evidence.

Does search data help catalog matching?
Indirectly but really: product queries are a third vocabulary for the same entities, and watching how products surface in search — through SERP monitoring at roughly $0.70 per 1,000 structured responses — both validates matches (do these two “different” SKUs surface for the same queries?) and catches demand-side drift. Teams running continuous SERP data crawling alongside catalog pipelines report that query overlap is their cheapest matching signal after image identity.

Where do we start in one week?
Extract 500 hard pairs from your current mismatches, hand-label them, and test your matching layer against them. If it fails more than a third — and with title-keyed matching, it usually does — pull a filtered multimodal dataset shard and re-run. The delta is your business case, and it takes a week, not a quarter. For the search-side validation, structured SERP data pilots with the same free trial credits as the scraper APIs.

The one-sentence version of the whole diagnostic: your feed doesn’t match because identity was never a text problem — it’s an evidence problem, and the teams that win it buy their evidence in structured records instead of extracting it from strings.