Deep context window, brittle on adversarial probes.
Composite Score
35.7
/100 · canonical
Arena ELO (R1)
n/a
joined post-R01
Multi-Turn ELO (R2)
1513
±46 · n=161
Reliability Rank
#13
failure-mode rubric
▌ Section 02 · The Lede
What this model is for.
Google's Gemini 3.1 Pro is the premium-tier Google entrant in Round 02 — $12.20 per million tokens, the third-highest price in the pool behind both Opus models. Multi-turn voters were unkind: it lands at #14 on Round 02 ELO at 1487, well behind the Flash Lite sibling that costs 1/68th as much. Adversarial coverage is solid where it exists (F1 agency 4.43, F12 instruction drift 4.33), but the fatal-flaw rate is the killer — 1.00 per session, second-worst in the field, and the flaw-hunter score sits at 29.2 with the kind of variance that implies one bad probe can break a session. The 1M-token context window is the main argument for picking this over its cheaper sibling.
▌ Section 03 · At a Glance
Cross-test position
Gemini 3.1 Pro sits at #18 on Cost · Latency, the caveat to watch.
Composite
14
Arena ELO
n/a
Multi-Turn
7
Rubric
12
Adversarial
13
Cost · Latency
18
▌ Section 04 · Strength & Weakness
Where it shines. Where it stumbles.
▲ Strength
Strong tone consistency (4.62/5). 1M-token context window, tied with the Flash Lite sibling and GPT-4.1.
▼ Weakness
Fatal-flaw rate of 1.00 per session — second-highest in the field. Multi-turn ELO of 1487 makes it the worst-value premium-tier model in the pool.
▌ Section 05 · Failure Modes
Per-axis breakdown.
Six adversarial probes per session, twenty sessions per model, judged by Sonnet 4 against a fixed rubric. Further right = the model handled the failure mode better. Each axis is drawn as a band on the rubric’s 1 to 5 scale, not a number: judges disagree by about 0.3 at the model level, so overlapping bands are a tie. The right column is the rank within the rp-bench pool.
▌ Coverage: 4/6F3 · Lore · F8 · Momentum not yet run on this model. Upstream rolls these out incrementally as new models join the pool.
F1 · Agency
Doesn't write your character's actions
15
#8
F2 · POV / Tense
Holds 2nd-person, present-tense narration
15
#15
F3 · Lore
not yet run on this model
n/a
F8 · Momentum
not yet run on this model
n/a
F12 · Instruction Drift
Keeps to the system prompt
15
#6
F13 · Context Attention
Holds character cards 50+ turns deep
15
#13
“Premium-tier price, mid-pack votes. The 1M context is the only argument.”
Round 02 verdict · Tough price-to-rank
▌ Section 06 · Subjective Dimensions
Engagement · Voice · Collaboration.
All three dimensions scored 1 to 5 by the Sonnet 4 LLM judge across twenty 12-turn multi-turn sessions. The same battery feeds the failure-mode rubric above; these are the subjective half of that judgment, drawn as bands with no number for the same reason.
Engagement
15
Tone Consistency
15
Collaboration
15
▌ Section 07 · Behavioral Metrics
How it writes.
Quantitative signals from the same 20 multi-turn sessions, compared against the population mean across all 11 models.
Avg words / turn
263↓
pop avg 265 · -1%
Unique-word ratio
0.668↑
pop avg 0.657 · +2%
Repetition score
0.040↓
pop avg 0.048 · -17%
▌ Section 08 · Flaw Hunter
Adversarial probe score.
Score of 100 minus deductions across 22 fail-mode flag types on adversarial 12-turn sessions. Further right = fewer flaws caught. Drawn as a band, not a number: about ten points either way is rater noise, so bands that overlap are tied.
▌ Craft band
-20100
From the Round 01/02 flaw-hunter pool. The current card carries the newer single-rater band.
Fatal/sess 1.00
Major/sess 6.75
▌ Top flaws caught
recycled_descriptionpurple_proseagency_violation
▌ Section 09 · Sample Responses
Highest- and lowest-rated turns.
▌ Pending Round 02
Best- and worst-rated sample responses ship with the raw-vote endpoint in Round 02. When that lands, this section will surface the model’s highest- and lowest-scoring blind-arena turns side by side, scored on the same rubric the leaderboard uses.
▌ Round 01 verdict
Hard to recommend in Round 02. Gemini 3.1 Pro is more expensive than Sonnet 4.5 (which destroys it on every axis) and slower than Opus 4.7 (which beats it on multi-turn ELO by 140 points). Pick this when the 1M-token context window is the load-bearing feature — anywhere else, the Flash Lite sibling is 67× cheaper for adjacent multi-turn quality.