INDEPENDENT MODEL COMPARISON
Jev & frontier models
Accuracy, speed, cost, and the value of knowing when to defer.
Benchmark results
| Model | Score | Answered | Provider time | Cost (USD) |
|---|
Per-item result matrix
Correctness and recorded metrics for every model and item. Hover or focus a cell for time, cost, and confidence. Question text and answer letters are withheld.
BRING YOUR OWN KEY
Run an original benchmark
Ten original SAT-style questions. Each run sends 10 requests to the chosen model and uses your Gateway credits.
Your key goes directly from this browser to Vercel AI Gateway. It is never sent to this site’s server or saved in history.
Keys are cleared after a run or when you leave the page. Results stay in this browser’s local storage; clear them on shared devices. Gateway/provider policies apply to submitted questions. Model availability may vary. Failed requests are not automatically retried.
View the 10 original questions
About the data
The published archive contains numerical observations from three private benchmark runs: AGIEval SAT (100 questions) and two GPQA Diamond runs (198 questions each). Question text, options, answer letters, explanations, raw model responses, and request identifiers are omitted. Restricted datasets cannot be rerun here.
Scores count unsuccessful answers as incorrect. Time is summed provider duration, not wall-clock latency. Cost uses Jev’s market price and other models’ returned cost. Unknown measurements are shown as —. Historical runs used the local app’s minimum supported reasoning settings; new browser runs use Gateway default reasoning settings and may not be directly comparable.
These are exploratory comparisons, not official leaderboard results or scaled SAT scores. Confidence is a model output, not a guarantee. AGIEval provenance · GPQA provenance