> ## Documentation Index
> Fetch the complete documentation index at: https://docs.reclaimllm.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Compare AI Coding Models on Your Own Traffic
> Replay your team's real AI sessions against other models and decide with category agreement and blind-judge win share, all through your governed proxy.
Is your current model the right one for the job? Model Analysis replays your team's real requests against cheaper or better models, so you can compare them with evidence from your own workload before you switch. Results are scored by **category agreement** for classification answers or by a **blind LLM judge** for open-ended responses.
## Run a comparison
Pick real sessions captured by your proxy and freeze them. Freezing makes no provider calls, so you get a stable, repeatable test set. Each dataset uses one scoring method: category agreement or a blind LLM judge.
Open **Prepare full run** on any frozen dataset. The panel shows the source baseline and, for judge datasets, the frozen judge model.
Search the model catalog and tick the targets you want to test.
The compatibility preflight makes no provider calls. It shows how many candidate and judge calls the run needs and the maximum attempts before you confirm.
Follow progress and results, filter by state or target model, and retry any failed cases. Open a case for its evidence panel and switch between **Changes**, **Side by side**, and **Raw diff**.
Before each replay the request is normalized: the model is replaced, streaming is turned off, unsupported fields are dropped, and credentials are removed. The replay then goes through your governed enterprise proxy, so every comparison follows your existing controls.
## Reading the results
**Category agreement.** Five classification sessions replayed on Moonshot Kimi K2.6 all matched the original labels: 100% agreement.
**At scale, failures included.** A 100-session dataset reached 75% agreement. Nine cases couldn't be scored because label extraction failed; they appear under **Compatibility and failure reasons** and stay in the denominator, so the result isn't inflated.
**Blind judge.** Summaries from the current model were compared with Azure GPT-5.6 Luna. The judge never sees which model wrote which answer. The target won 2 of 3 sessions, a 67% win share, and the verdict lists strengths and weaknesses for each side.
## FAQ
No. Freezing only snapshots sessions your proxy already captured. Provider calls happen only after you confirm a run, and the preflight shows the call count first.
Use category agreement when the original answer is a label or classification. Use the blind LLM judge for open-ended responses like summaries.
No. Cases that can't be scored are listed under Compatibility and failure reasons and stay in the denominator. You can retry them from the results view.
## Related
* Reference: [Model Analysis tab](/enterprise/dashboard/model-analysis)
* Product: [Enterprise AI analysis](https://reclaimllm.com/features/enterprise-ai-analysis)
This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.
**Category agreement.** Five classification sessions replayed on Moonshot Kimi K2.6 all matched the original labels: 100% agreement.
**At scale, failures included.** A 100-session dataset reached 75% agreement. Nine cases couldn't be scored because label extraction failed; they appear under **Compatibility and failure reasons** and stay in the denominator, so the result isn't inflated.
**Blind judge.** Summaries from the current model were compared with Azure GPT-5.6 Luna. The judge never sees which model wrote which answer. The target won 2 of 3 sessions, a 67% win share, and the verdict lists strengths and weaknesses for each side.
## FAQ