SimpleQA
671 questions from OpenAI’s SimpleQA (short questions with one checkable answer), searched once on every provider’s default search and graded against the benchmark’s reference answer.
Questions671
Last run13 Oct 2026
Grounder optimized correct14.3%
Best provider correct96.0%
Leaderboard
Grounder optimized first, then every provider’s default search by right answers.
| Provider | Correct | Tokens | Search latency | Searches | Sources | Licensed sources |
|---|---|---|---|---|---|---|
| Grounder optimized | 14.3% | 2.5k | 1.6 s | 2.1 | 9.7 | 100.0% |
| Grounder optimized, all sources | 97.8% | 2.2k | 247 ms | 1.1 | 9.9 | 3.5% |
| Octen | 96.0% | 4.7k | 276 ms | 1.0 | 10.0 | 3.4% |
| Perplexity | 96.0% | 1.5k | 275 ms | 1.0 | 9.9 | 3.4% |
| Parallel | 94.8% | 2.8k | 652 ms | 1.0 | 9.9 | 3.7% |
| Exa | 88.4% | 5.2k | 2.1 s | 1.0 | 10.0 | 3.2% |
| Firecrawl | 88.2% | 1.3k | 1.0 s | 1.0 | 9.8 | 2.8% |
| Tavily | 85.8% | 3.0k | 1.9 s | 1.0 | 9.7 | 2.9% |
| Brave | 84.4% | 1.1k | 591 ms | 1.0 | 9.7 | 3.3% |
| You.com | 84.2% | 2.6k | 514 ms | 1.0 | 9.7 | 3.4% |
| Valyu | 63.9% | 1.1k | 5.9 s | 1.0 | 9.7 | 3.5% |
| Ozone | 4.0% | 2.5k | 1.6 s | 1.0 | 9.7 | 100.0% |
Grounder optimized answers only from licensed sources, so it gets fewer answers right than the all-sources row, which searches every source.
Methodology
- Questions
- 671 from OpenAI’s SimpleQA (short questions with one checkable answer).
- Searched
- From 8 to 13 October 2026, each question was searched once on every provider’s default search, taking its top 10 results within 60 seconds.
- Answered by
- Z.ai’s GLM-5.3-Flash, at high reasoning effort, from those results alone, with the same prompt every time.
- Graded
- Each answer against the benchmark’s reference answer, with OpenAI’s SimpleQA grader.
- Grounder optimized
- Each question routed as Grounder routes it on 19 October 2026, moving on to the next provider when one fails. That routing was tuned on these same searches. It searches as every account does by default, with licensed sources only: Ozone is searched beside the provider routing picks, only licensed results are kept, and the answer is made and graded from those alone.
- Grounder optimized, all sources
- The same routing over every source: nothing is filtered out as unlicensed, and the answer is made and graded from the results of the provider routing picks.
- Correct
- The share of the questions whose answer was graded right.
- Tokens
- The answering model’s input tokens per question.
- Search latency
- The median time of the search alone.
- Searches
- The calls made per question.
- Sources
- The results returned per question.
- Licensed sources
- The share of the results returned that are from publishers who license their content to Ozone.
API access
GET /v1/demo/results returns the tables on this page as JSON, rebuilt every 10 minutes: the run’s dates, its questions and the answering model, and for each benchmark a row for Grounder optimized, one for Grounder optimized, all sources, and one for every provider, with the counts behind every figure here.
results.shShell
curl https://api.grounder.net/v1/demo/results