Enterprise AI agents retrieve from a corpus that is mostly text, while the knowledge they’re asked about — products, packaging, screenshots, video — is mostly not text. Multimodal retrieval quality is decided almost entirely by the alignment data you index, not the model you query. Five lessons from teams that fixed their agents’ blindness, with the dataset strategy that worked.

A support agent that can quote your return policy from memory but cannot answer “is this the right charger for my laptop?” from a product photo is not a niche failure. It’s the default outcome of building retrieval over text alone.

Lesson 1: Retrieval Quality Is Dataset Quality

The uncomfortable arithmetic: in most enterprises, the documents agents are asked about skew heavily visual — product catalogs, UI screenshots, packaging, how-to video. A text-only retrieval layer can only reach the caption, the title, and the alt text, which is a thin shadow of what the user’s question is actually about.

Teams that fix this do not start with a better model. They start with a better index: image-text pairs where the text genuinely describes the image, at corpus scale. That is a data problem, and it has a market solution — pre-built multimodal datasets deliver aligned records from 100+ domains at roughly $0.25 per 1,000 records, a price point at which indexing a million aligned pairs costs a few hundred dollars.

The test for whether your agent has this problem is one line: ask it about something in an image you know it should have access to. If it answers from text metadata or hallucinates, your index — not your model — is the bottleneck.

Lesson 2: Alignment Beats Volume

The second lesson is about what “good” multimodal data means. A million images with generic titles (“IMG_2041”, “product photo”) are worth less than a hundred thousand records where the text field genuinely describes the visual content — product names, attributes, contexts.

This is why record structure matters more than record count when evaluating datasets. A well-formed record carries the image reference plus the aligned text plus the metadata that lets you slice safely:

# Indexing a multimodal dataset shard into a retrieval store
for record in dataset.stream(filter={"domain": "ecommerce_listings"}):
    if not record.get("title"):          # skip unaligned records
        continue
    doc = {
        "id": record["record_id"],
        "image": record["image_url"],
        "text": f"{record['title']}. {record.get('attributes', '')}",
        "market": record.get("market"),
        "language": record.get("language"),
    }
    index.upsert(embed_multimodal(doc))   # joint image-text embedding

The if not record.get(“title”) guard is the whole lesson in one line: unaligned records poison multimodal indexes. Filtering them out before embedding costs nothing and improves answer quality measurably.

Lesson 3: Diversity Is a Coverage Problem, Not a Virtue

Teams discover Lesson 3 the hard way, usually as a user complaint: the agent answers perfectly about popular products from one market and confidently wrong about everything else. The index over-represents whatever the collection path over-represented.

The countermeasure is sourcing discipline. Dataset providers that collect through geo-targeted infrastructure — Thordata’s collection network spans 190+ countries with city-level IP targeting — produce corpora where the “long tail” of markets and content types is actually present. When you evaluate a multimodal dataset, don’t ask “how many records?”; ask “what fraction of my deployment distribution does this cover?” and test it against a holdout drawn from your real users.

Lesson 4: Agents Need Freshness, Not Just Recall

A retrieval index is a snapshot, and enterprises rot their snapshots constantly: products change, packaging updates, screenshots go stale with every release. The teams that keep agents trustworthy treat index refresh as a scheduled operation, not a re-index project.

Two mechanisms make this affordable. First, scheduled collection: providers operating both datasets and collection infrastructure (as Thordata does) can run recurring jobs that feed new records to your storage on a cadence. Second, freshness-aware retrieval: timestamp each document and prefer recent records for questions about current state — a pattern borrowed from web search, applied internally.

Lesson 5: Some Agent Questions Are About the Outside World

The final lesson surprised at least one team on this list. Their support agent kept getting questions it couldn’t answer from any internal corpus: “why is your product cheaper on that other site,” “did you stop selling the blue one,” “why doesn’t your page come up when I search anymore.” These are questions about the live web — and specifically about search results.

The fix was to treat SERP data as another retrievable corpus. With the SERP monitoring solution, structured Google and Bing results — organic positions, shopping placements, ads — flow into the agent’s index at roughly $0.70 per 1,000 responses at volume, refreshed on a schedule. Questions about visibility, pricing in search, and brand presence in results stopped being out-of-scope; the agent answers them from the same retrieval layer that serves everything else.

This pattern generalizes: monitoring-oriented data feeds — rank tracking, competitor SERP presence — make excellent agent corpora because they arrive structured, timestamped, and continuously. The continuous SERP data crawling service delivers exactly that shape of feed.

FAQ

Do we need a multimodal model to benefit from multimodal datasets? Not necessarily. Even text-only agents improve when the indexing pipeline uses aligned captions — the dataset’s text fields carry information the raw corpus lacked. The multimodal model unlocks the full value, but the data investment pays off at every stage.

How much aligned data is enough? The honest answer: enough to cover your deployment distribution. Measure with a visual holdout set — a hundred questions about images your users actually ask about — and index until accuracy plateaus. Most teams plateau far below “everything”; they plateau at “everything my users actually ask.”

Can we mix our proprietary data with purchased datasets? Yes, and it’s the standard architecture: purchased breadth underneath, proprietary depth on top, both flowing into the same retrieval store. Keep provenance metadata attached so you always know which answers rest on which data class.

What does this cost relative to the LLM stack? A rounding error. At $0.25 per 1,000 records, dataset acquisition for a serious agent index runs a few hundred dollars; the SERP feed at $0.70 per 1,000 responses scales with monitoring scope, not model usage. Compare that to the cost of one confidently wrong answer reaching a customer — support escalations, lost trust — and the economics stop being a debate.

Where do we start in one week? Day 1–2: build the visual holdout from real user questions. Day 3: pull a filtered dataset shard and index it. Day 4–5: run the holdout against the enriched index and measure. If the improvement is visible, add the freshness layer — scheduled collection plus the SERP monitoring feed for the outside-world questions. One week is enough to know whether your agent’s blindness is fixable at data cost — and it almost always is.