How to evaluate Exa against other search providers and native model search, on public and Exa-built benchmarks.
ATLAS grades search agents on the completeness and accuracy of wide, deep research tasks. We give the same agent, prompt and budget a different search API on each run and score the answer tables. Exa reaches the highest row F1 of any search API.
No public benchmark drills into the day-to-day tasks a search agent performs: finding code and documentation, researching companies and people, and reading financial filings. So we built our own. Each chart compares Exa's search API with the other search APIs on the same queries.
Everything above measures Exa's search API. This section measures Exa Agent, our research agent, against other research agents and frontier models.
These results are for Exa Agent, not the search API. Exa Agent plans and runs many searches itself, so we compare it with other research agents on benchmarks that test an agent's ability to find and enrich multiple entities.
In addition to existing industry benchmarks, we developed ATLAS, a new benchmark for wide and deep search that tests the accuracy and completeness of search agents against challenging real-world research tasks.
For deep research tasks, Exa Agent is pareto dominant across total number of entities found and cost per verified result.
We put each provider through 8,985 tasks across 13 test suites. People + Company Search, Finance and our other coding sets are built from production traffic and stay private.
The model (GPT-6 Luna), prompt, loop, and budget are fixed. Only the search API changes.
OpenAI (GPT-6 Luna) and Anthropic (Claude Haiku 5.5) run with their own hosted search. The model and search backend are scored together, so the native searchers sit alongside the search APIs rather than separately.
See below for details of settings used.
Every provider's results on a benchmark are graded by the same LLM judge with the same rubric, recreating the upstream grading method for public benchmarks as closely as possible.
Public benchmarks load from pinned upstream versions. Each run records every request, answer, grade, and cost.
Each provider runs on its own defaults except the request settings below. These values never change between providers.
{
"model": "openai/gpt-6-luna",
"reasoning_effort": "medium",
"max_tokens": 32000,
"max_rounds": 25
}POST https://api.exa.ai/search
{
"type": "auto",
"numResults": 10,
"contents": {
"highlights": true
}
}POST https://api.perplexity.ai/search
{
"search_type": "web",
"max_results": 10
}POST https://api.parallel.ai/v1/search
{
"mode": "advanced",
"advanced_settings": {
"max_results": 10
}
}GET https://api.search.brave.com/res/v1/llm/context ?count=10 &maximum_number_of_tokens_per_url=4096
POST https://api.openai.com/v1/responses
{
"model": "gpt-6-luna",
"reasoning": { "effort": "medium" },
"tools": [
{
"type": "web_search",
"search_context_size": "high",
"external_web_access": true,
"return_token_budget": "unlimited"
}
]
}Runs on anthropic/claude-haiku-5-5: Anthropic's hosted web search only works with Claude models, so this arm cannot use the shared GPT-6 Luna agent. Search runs through Programmatic Tool Calling (PTC), and there is no cap on the number of searches dispatched.
POST https://api.anthropic.com/v1/messages
{
"model": "claude-haiku-5-5",
"output_config": { "effort": "high" },
"tools": [
{
"type": "web_search_20260318",
"allowed_callers": ["code_execution_20260120"],
"max_uses": null,
"response_inclusion": "excluded"
}
]
}