“Exa consistently shows higher quality than other AI search providers we tested. What Exa is able to do in 1 search took our old provider multiple searches, taking 5 times as long.”
About CodeRabbit
CodeRabbit is an AI code review platform that helps engineering teams catch issues and move PRs faster through review cycles. Its agent works across repo context, code diffs, custom rules, and review comment history. When a review comment depends on outside context, it checks libraries, APIs, docs, packages, public repos, changelogs, and commits against web search evidence.
Prior to using Exa, that verification often required several search calls. Switching to Exa Search cut CodeRabbit's web-search volume by about 70-75% while preserving, and sometimes improving, review quality.
In a four-day sample, web search ran for 7-10% of PRs on a given day.
Web-cited AI code review verification
CodeRabbit uses Exa Search to provide critical context from web search, technical docs, packages, and release notes. When repository context alone is not enough, a targeted verification question can return structured evidence that the review agent uses to confirm, refine, or suppress a comment.
The Challenge
Verifying review comments against fast-moving external context
The hardest review cases depend on information outside the repo.
- Library behavior: A review comment may depend on whether a specific library version supports a method or changed behavior recently.
- API and package changes: Third-party APIs, packages, and SDKs may deprecate parameters, introduce breaking changes, or alter expected usage.
- Public implementation examples: Public repositories can confirm whether a proposed implementation pattern is common, current, or contradicted by real usage.
- Docs, changelogs, commits, and issues: Review feedback can be stronger when it is grounded in current documentation, release notes, package metadata, commits, or public issue context.
With the prior agentic web search provider, Deep Search-style checks could require multiple calls and take about 30-45 seconds.
The Solution
Real-time context for review comments
CodeRabbit integrated Exa Search in its agentic code review context engine to bring in real-time context from web search, docs, packages, and public release notes. It runs only when outside context can change the code review recommendation.
Verify dependencies
Check whether a specific library version supports a method, changed behavior, or introduced a breaking change.Check docs and changelogs
Review current documentation, migration guides, release notes, and deprecation notices.Ground review decisions
Use cited context to decide when to make, refine, or avoid a review comment.Why Exa
One agentic /search call instead of many search calls
The key shift was moving from repeated web-search calls to Deep Search. Exa performs multi-step search and synthesis behind a single API call.
- Prior provider: About 30-45 seconds for some Deep Search-style questions.
- Exa Search: P50 7.1 seconds and P95 11.1 seconds.
Results
Faster verification with fewer search calls
- 70-75% fewer web-search calls: Search reduced call volume for the same verification workflow.
- Lower latency for Deep Search checks: P50 7.1 seconds and P95 11.1 seconds.
- Current web citations: CodeRabbit can verify comments against current libraries, APIs, docs, repos, commits, and packages.
- Selective production use: Web search runs only when a review needs outside context, about 7-10% of PRs in a four-day sample.