One influence engine, every objective.

Pick any objective, apply the influence engine step by step, and watch the leaderboard react.

The framework at a glance

Input
Pairwise votes
fit one base Bradley–Terry model
→
Engine
Influence scores
rank candidate actions, no refitting
→
Act
Greedy Drop·Add·Flip
apply the highest-leverage change
From A Unified Perturbation Framework for Analyzing Leaderboard Stability and Manipulation, an oral at the CTB Workshop @ ICML 2026. Read the paper: arXiv:2605.15761.
Chapter 3 · Specification of matches

Which matches are influential? Structure, not exposure.

Signed Spearman correlations between one-step match influence scores and four match-level covariates on Chatbot Arena 55k. Raw match count (exposure) is a weak, often negative proxy — while bridge variance and closeness spike under Flip. Pick an objective and compare the five actions.

From thesis appendix §Specification of matches — signed Spearman ρ ∈ [−1,1], 1sN influence, grouped by objective and action. Sign is objective-relative. (Bridge var. and closeness share a value in the source data.)
Chapter 1 · Influence sandbox

Choose an objective. Apply the influence engine. Watch it react.

Set the objective and action, apply one action at a time, and see the influential match, the objective curve (actual refit values), and the leaderboard change — with the goal marked reached or not. The Influence ranking panel shows the full pool of candidate matches, sorted by influence; the greedy engine simply walks down it — watch each pick light up as you apply actions. In/out Top-k restricts the attacker to injecting new votes, so it shows every outcome available on the pair it lands on instead.

Objective
k
Direction
Criterion
Player Action

Live leaderboard · rating ±95% CI

0/ 3
Chapter 2 · Player removal (deprecation)

Remove one model. The whole board reshuffles.

Each row’s bar and number are that model’s influence score — a first-order estimate (∆τ) of how much its removal perturbs the ranking. Click a model to deprecate it and watch the board re-fit and reorder.

Arena leaderboard bar = |influence score| · click to remove

What predicts removal influence? player feature ↔ |player influence| · pick a leaderboard

Influence score = first-order ∆(Kendall τ) from compute_player_influence (joint-Newton). Removal itself re-fits BT live. Anchor mixtral-8x7b can’t be removed.
Chapter 4 · Dataset robustness

Every leaderboard has a robustness fingerprint.

Three normalized axes — how hard to break Top-1, how stable the CIs, how stable the ranking. All seven leaderboards at once.

The Top-1 spoke collapses to the center almost everywhere — breaking #1 is cheap on every crowdsourced arena. Only MT-Bench resists; ATP and WebDev cave on ranking and CI too.