Search
Search is a pipeline of distinct operations. Finding a useful URL, retrieving its contents, and generating an answer require different contracts.
On this page
Three different outputs
| Output | What it represents |
|---|---|
| Search snippet | A short excerpt associated with a ranked result. Useful for discovery; potentially partial or stale. |
| Page content | Text extracted from a retrieved document. Retrieval may fail or return only part of a page. |
| Generated answer | A model-produced synthesis. It needs its own provenance and evaluation. |
For example, Tavily Search exposes answer and raw-content options, while Tavily Extract retrieves content from URLs. Treat these as distinct outputs even when a provider can return several in one response.
Keep your types honest
type SearchHit = {
url: string;
title: string;
snippet: string;
};
type RetrievedPage = {
url: string;
content: string;
fetchedAt: string;
status: 'complete' | 'partial';
};
type GeneratedAnswer = {
text: string;
sourceUrls: string[];
provider: string;
};
// Never fill RetrievedPage.content with SearchHit.snippet.Build a deliberate pipeline
- Discover: search with a bounded result count and explicit domain or time constraints where supported.
- Select: choose which URLs warrant retrieval. Preserve titles and source URLs.
- Retrieve: extract selected pages, keeping per-page failures and retrieval timestamps.
- Synthesize: generate an answer only when requested, with references to the retrieved evidence.
- Evaluate: check citation support, freshness, missing evidence, and output quality separately.
Retrieved pages are untrusted input. If a model consumes them, treat instructions found inside a page as content to analyze, not permission to change the application's behavior or disclose private data.
Control cost and privacy
Avoid retrieving every hit by default. Limit result counts, extraction size, concurrency, and total time. Confirm current billing in the provider's own documentation; do not assume that search, extraction, and answer generation share a price or quota.
- Keep secrets and unnecessary personal information out of search queries.
- Use cache keys that include relevant query options and authorization scope.
- Preserve the provider and retrieval time when cached content is returned.
- Apply the same domain and data policies to fallback providers.
SDK SDK is an independent discovery hub. Provider examples link to official documentation. App-owned examples are illustrative patterns you can adapt.