Community last in Round 01, top-5 in Round 02 multi-turn. The great inversion.
Composite Score
78.6
/100 · canonical
Arena ELO (R1)
1470
±44 · n=215
Multi-Turn ELO (R2)
1525
±43 · n=212
Reliability Rank
#6
failure-mode rubric
▌ Section 02 · The Lede
What this model is for.
GPT-4.1 finishes last on community ELO (#11) and #3 on reliability — the great Round 01 inversion. Tied #1 on narrative momentum with MiniMax (F8 score 4.30), tied in the top tier on lore (4.30, with Gemma and Mistral, behind DeepSeek's 4.50). The community didn't enjoy writing with it, but the failure-mode rubric says it doesn't break. The pick when reliability matters more than vibes — and a useful test case for the engagement-vs-reliability split this entire benchmark exists to surface.
▌ Section 03 · At a Glance
Cross-test position
GPT-4.1 sits at #16 on Cost · Latency, the caveat to watch.
Composite
5
Arena ELO
11
Multi-Turn
5
Rubric
10
Adversarial
6
Cost · Latency
16
▌ Section 04 · Strength & Weakness
Where it shines. Where it stumbles.
▲ Strength
Tied #1 on narrative momentum (4.30/5, with MiniMax). Failure-rank #3 in the pool. Top tier on lore consistency (tied with Gemma + Mistral at 4.30, behind DeepSeek's 4.50).
▼ Weakness
Bottom community ELO (#11/11). High cost ($2/1k tokens). Voters consistently rank it last on engagement despite the strong reliability profile.
▌ Section 05 · Failure Modes
Per-axis breakdown.
Six adversarial probes per session, twenty sessions per model, judged by Sonnet 4 against a fixed rubric. Further right = the model handled the failure mode better. Each axis is drawn as a band on the rubric’s 1 to 5 scale, not a number: judges disagree by about 0.3 at the model level, so overlapping bands are a tie. The right column is the rank within the rp-bench pool.
F1 · Agency
Doesn't write your character's actions
15
#11
F2 · POV / Tense
Holds 2nd-person, present-tense narration
15
#7
F3 · Lore
Doesn't break worldbuilding
15
#3
F8 · Momentum
Pushes scene forward when user goes passive
15
#1
F12 · Instruction Drift
Keeps to the system prompt
15
#8
F13 · Context Attention
Holds character cards 50+ turns deep
15
#11
“Last on the community board — third on the failure-rank board.”
Round 01 verdict · The great inversion
▌ Section 06 · Subjective Dimensions
Engagement · Voice · Collaboration.
All three dimensions scored 1 to 5 by the Sonnet 4 LLM judge across twenty 12-turn multi-turn sessions. The same battery feeds the failure-mode rubric above; these are the subjective half of that judgment, drawn as bands with no number for the same reason.
Engagement
15
Tone Consistency
15
Collaboration
15
▌ Section 07 · Behavioral Metrics
How it writes.
Quantitative signals from the same 20 multi-turn sessions, compared against the population mean across all 11 models.
Avg words / turn
212↓
pop avg 265 · -20%
Unique-word ratio
0.688↑
pop avg 0.657 · +5%
Repetition score
0.031↓
pop avg 0.048 · -35%
▌ Section 08 · Flaw Hunter
Adversarial probe score.
Score of 100 minus deductions across 22 fail-mode flag types on adversarial 12-turn sessions. Further right = fewer flaws caught. Drawn as a band, not a number: about ten points either way is rater noise, so bands that overlap are tied.
▌ Craft band
-20100
From the Round 01/02 flaw-hunter pool. The current card carries the newer single-rater band.
Fatal/sess 0.75
Major/sess 6.83
▌ Top flaws caught
purple_proserecycled_descriptionagency_violation
▌ Section 09 · Sample Responses
Highest- and lowest-rated turns.
▌ Pending Round 02
Best- and worst-rated sample responses ship with the raw-vote endpoint in Round 02. When that lands, this section will surface the model’s highest- and lowest-scoring blind-arena turns side by side, scored on the same rubric the leaderboard uses.
▌ Round 01 verdict
GPT-4.1 is the cleanest argument the round produces for why we run two leaderboards. The failure-mode rubric ranks it #3 — tied #1 on narrative momentum with MiniMax, top-tier on lore behind DeepSeek — and the community ranks it dead last. Both are real signals, measuring different things. Pick it when reliability matters more than vibes. Pick Sonnet 4.5 if cost isn't a constraint and you want both.