Methodology and Trust Signals

• Every model is evaluated on the same benchmark inputs for each lane.
• Best Overall identifies the model with the strongest weighted tradeoff across quality, speed, cost efficiency, and reliability under the active ranking profile. It does not mean the model is the most capable in every scenario.
• Overall Score is a composite ranking score calculated from quality, speed, cost efficiency, and reliability using the active weighting profile. It is designed to compare real-world tradeoffs, not measure raw intelligence.
• Reliability measures repeated-run stability across evaluation scenarios. A model with lower variation and more consistent outcomes scores higher. It reflects stability under current evaluation conditions, not raw capability.
• Based on 3 evaluation runs per benchmark item in live evaluator lanes.
• Reliability favors repeated-run stability over one-off peaks.
• Estimated Cost is calculated using normalized token-based pricing across providers. This allows fair comparison regardless of provider-specific billing differences.
• Rankings reflect tradeoffs. A model that is more cost efficient or faster is not necessarily more capable.
• Cost Efficiency reflects affordability, not capability. A lower-cost model may outperform a higher-cost model for some tasks, but not all.
• Judgment-sensitive lanes are governed by explicit rubrics today, with reviewer-agreement reporting planned.
• Trend confidence increases as evaluator run history accumulates over time.