Model card · reviewed 2026-09-25

Fable 5.1

anthropicJ +0.85 · #1 of 55

Strong prose with relatively little content deflection, making it a promising writing-focused option in this benchmark. The trade-off is uneven instruction-following: half of the system-prompt probes failed, and some turns went unanswered.

Response coverage 94.5% · 208 of 220 turns answered.

Round 04

Willingness and judgment

J+0.85 · held at first ask − over-refusal

Rank by J1 of 55

Held at first ask1.00 · 4 of 4 usable first asks

Over-refusal0.15 · L3 to L5, confident labels only

By ladderintimacy 0.18 / gore 0.00 · L1 to L5, ungated

Held when pushed0.50 · 2 of 4 first-ask holds kept at the second push. Not in J: rater kappa 0.525 on the push, below the 0.6 bar.

Policy · overshoot0.40 · 0.08 · reported beside J, not part of it

The hard lines tested are non-consent and frame-break scenes only; this is not a child-safety or real-person-likeness measure. Treat models within about 0.3 J of each other as tied.

Trap probes

Failure modes

Agency violations4.5% [1.3% to 15.1%] · 44 probes

POV/tense breaks0.0% [0.0% to 10.7%] · 32 probes

Per trap mode, counts not rates: 2 to 9 probes each, so a percentage would not survive one probe changing.

System-prompt violations3/6 failedDetail loss0/9 failedContradiction mishandled0/2 failedNarrative stagnation0/2 failedPhysics sycophancy0/3 failedTemporal inconsistencynot runSubtext made explicit0/3 failedCharacter flattening0/3 failedGenre instability0/3 failed

Flaw hunter · single rater

Craft band

-20100

A band, not a number, on purpose: ±10 is the rater noise floor, not a sampling error. Bands that overlap are tied, and most of the roster overlaps. The rose tick marks zero.

Sessions19

Top flawsrecycled description · narrating emotions · convenient world

Counted by machine

Production defects

None detected208 turns clean

Token overhead2.0x · billed per visible character, against the prose floor; not a price

Counted by machine with no judge involved: the most trustworthy block on the card.

LLM judge

Subjective band

Composite

15

The band spans where three judge families put this model. The axis is one judge's (single-judge sonnet 5); another judge shifts everyone by about a point.

Collaboration · less reliable

15

Engagement · less reliable

15

Tone consistency · less reliable

15

Text statistics, no judge

Behavioral

Avg words480 · population 320

Unique-word ratio0.526 ↓ · population 0.631, well below

Phrase repetition0.082 ↑ · population 0.060, well above

Round 01 arena

Community rank

No Round 01 arena data for this model.

Rounds 02 and 03 · set beside, never converted

Across rounds

Earlier rounds

New in Round 04. No Round 02 or Round 03 result to set beside this card.

Round 04

Different judges, not comparable

New judge (Sonnet 5)tier A · fixed tiers A (best) to E on its own scale

Second judge (ChatGPT via Codex)3.1 (own scale, about 0.94 lower on average) · letter depends on judge: no

Strength

Strong on lore consistency (within 0.3 of the rest of the top; an independent judge reorders this)

Weakness

Catastrophic floor on general adversarial robustness

All model cards →Round 04 board →

Reviewed 2026-09-25. Editorial summaries of the current local benchmark cards, not new session ratings or general model safety claims. Willingness means delivery against this benchmark's product targets, including soft deflection; tested hard boundaries cover non-consent and frame-break only, and hard-boundary wording follows the first ask, as J does since 2026-09-25 (the second push is a separate, weaker card line). Craft bands overlap, and small samples limit comparisons. Token overhead is not monetary cost. Review these summaries when source results change. Source: rp-benchmark generate_profile_cards_v2.py at c418a40 · export r4c-20260925-b7ccaed. CC-BY 4.0.