{ "benchmark": "The Aggregate - Lichess Pairwise Strength (25 Pairs)", "models": [ { "model": "anthropic/claude-opus-5.5", "correct": 22, "n": 25, "accuracy": 88.0, "baseline_accuracy": 74.0, "invalid_confidence": 0, "consistent_reversals": 5, "reversals": [ { "pair": "pair_09_reverse", "original": "pair_09", "prediction": { "stronger": "A", "probability_a_stronger": 0.55, "explanation": "Both targets are weak. In Game A, Black played a coherent Dutch setup. However, 14...Bd6? and 15...Rd8 let White play 16.Bxd6 and win a piece. After that, Black drifted and lost a long game ending in 46.Re8#. In Game B, Black blundered more crudely. 10...Nxd4 11...Bxd4 walked into 12.Bb5+ c6 13.Qxd4, dropping a piece in the opening. Later 26...Nxc4 ignored the back-rank threats, and 28.Qg8+ ended the game. Game B's errors look more elementary and came earlier, while Game A's Black at least resisted for longer. Slight edge to A." }, "consistent": true }, { "pair": "pair_15_reverse", "original": "pair_15", "prediction": { "stronger": "A", "probability_a_stronger": 0.82, "explanation": "Game A White plays a coherent Yugoslav-style kingside attack: 7.Bh6, 9.g4, 11.h4 and 13.hxg6. The sacrifice 26.Rxh7! and 27.Qxh7 is followed by accurate conversion, ending with 35.g6 and the mate 38.Nh6#. Game B White plays poorly. 2.Bc4 and 5.Bd3 waste tempi, and 16.Be4?? allows 16...Nxg2+, losing castling rights and material. White then ignores threats: 19.Nfg5, 22.Bc3 and 23.Nd4 let Black take 23...Bxh1, and White is quickly mated after 29.Kg2 Qxf3#. These are elementary tactical oversights, so Game A's target is clearly stronger." }, "consistent": true }, { "pair": "pair_13_reverse", "original": "pair_13", "prediction": { "stronger": "A", "probability_a_stronger": 0.68, "explanation": "Game A White played a coherent Exchange Caro-Kann setup (4.Bd3, 5.c3, 7.O-O, 9.Bg5, 10.f4) and only collapsed late, into a smothered mate after 24.Bb4?? Ne2+ 25.Kh1 Qg1+. Game B White opened with the bizarre 2.Qf3 and 3.g4, then got a dangerous position after 6...Nxf2 7.Kxf2. He won only because Black blundered repeatedly: 11...Nb5+ and 13...Qf2+ dropped material, and 22...Qxg1+ gave away the queen. Both games show low-level play, but A's opening and middlegame were far more principled, while B's opening was poor and his win came from the opponent's errors. A is likely stronger." }, "consistent": true }, { "pair": "pair_12_reverse", "original": "pair_12", "prediction": { "stronger": "B", "probability_a_stronger": 0.4, "explanation": "Game A Black wasted many tempi early (3...Bd7, 9...Bf5, 10...Bd7, 12...Bg4, 15...Be6). He later found the fork 18...Ncxb3 and finished with 31...Qf2#, but against a very loose White (8.h4, 19.f5, 25.Rf1?). Game B Black set up coherently with 4...b5, 5...c4, 10...Nd5 and 11...f5, which pushed back the White bishop. He then punished errors efficiently with 15...Nf4, 16...Nxg2+, 22...Bxe4, 23...Bxh1, 25...Nh3+ and the clean mate 29...Qxf3#. Both opponents were weak, but Game B's target showed sounder structure and sharper conversion. That edge favors B." }, "consistent": true }, { "pair": "pair_18_reverse", "original": "pair_18", "prediction": { "stronger": "A", "probability_a_stronger": 0.8, "explanation": "Game A White played coherent developing chess, regained the piece with 12.Bb5+ and 13.Qxd4, and created pressure with 17.b4, 20.bxc5 and 22.Bf4+. 26.Re3 was loose, but play was mostly purposeful. Game B Black played very weakly. The flank pawn moves 4...b5, 7...f6, 8...h5 and 9...g5 neglected development. Black allowed 17.Nxf6+ and the fork 20.Nb6, losing the exchange. Then 22...Qd8?? put the queen on a square attacked by the g5 bishop (23.Bxd8). These are basic blunders, so Game A's target was likely the stronger player." }, "consistent": true } ], "rows": [ { "pair": "pair_01", "game_a": "game_16", "color_a": "black", "game_b": "game_19", "color_b": "white", "rating_a": 1569, "rating_b": 2668, "gap": 1099, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.25, "correct": 1, "brier": 0.0625, "old_estimate_a": 1100, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black played a dubious gambit (2...e6) and then the unsound 6...Bxf2+, ending down material for pawns. 11...Ng4 12.h3 Bxf3 13.hxg4 Bxe2 only traded pieces without recovering the deficit. Black then drifted into a worse endgame and resigned after 27.Re6, showing little compensation or resourcefulness. Game B White handled a QGD Exchange competently: 8.Bxf6 led to a queen trade, 22.Nxh5 won a pawn, and 25.h5 with 28.h6 pushed the passed pawn. White then used the rook actively with 33.Re7+ and 34.Ra7, and punished 34...Nxf5 with 35.Ra8+ and 36.Rxh8. Game B's play is more coherent and technically sound.", "confidence_valid": true }, { "pair": "pair_02", "game_a": "game_01", "color_a": "black", "game_b": "game_24", "color_b": "white", "rating_a": 1159, "rating_b": 762, "gap": 397, "gap_band": "200-499", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.55, "correct": 1, "brier": 0.20249999999999996, "old_estimate_a": 1200, "old_estimate_b": 1250, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Both games are low-level. In Game A, Black allowed a fork chain with 15.Nd5 and 16.Nf6+, where 7...Qd7 left the queen exposed, giving up the queen for minor pieces. Afterwards Black played reasonably: 19...Nd4, 20...gxf3 and 30...Bxg3 picked up material, and Black converted after White's 40.Qb8+?? blunder. In Game B, White faced a very weak opponent (5...a5, 7...f6, 8...h5) but played dubiously, with 15.g3 and 16.gxh4. White also gave up a knight with 17.Nxf6+ for little, then won because Black blundered the queen with 22...Qd8?? into 23.Bxd8. Game A's Black showed somewhat more tactical resourcefulness against a stronger opponent, so I lean slightly to A.", "confidence_valid": true }, { "pair": "pair_03", "game_a": "game_01", "color_a": "white", "game_b": "game_06", "color_b": "white", "rating_a": 1363, "rating_b": 1112, "gap": 251, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.58, "correct": 1, "brier": 0.17640000000000003, "old_estimate_a": 1250, "old_estimate_b": 1250, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "Both targets look weak. Game A White found 15.Nd5 and 16.Nf6+, forking king and queen to win the queen and reach a winning position. Its later collapse was dramatic: 40.Qb8+?? gave away the queen, and 42.Rxe8 left a lost ending. Game B White played an aimless opening (6.Bb1, 8.h4, 9.b3) and allowed 18...Ncxb3, losing the exchange. 25.Rf1?? dropped another exchange, and 27.Ke1?? hung the f6 knight to Qxf6. White was then quickly mated. Game B White shows more repeated basic tactical oversights, while Game A White showed some tactical vision before blundering. I lean slightly to A.", "confidence_valid": true }, { "pair": "pair_04", "game_a": "game_22", "color_a": "black", "game_b": "game_08", "color_b": "black", "rating_a": 557, "rating_b": 1791, "gap": 1234, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.15, "correct": 1, "brier": 0.0225, "old_estimate_a": 1000, "old_estimate_b": 1450, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black played poorly: 6...Nxf2?! was a dubious sacrifice, and 11...Nb5+? simply hung the knight to 13.Qxb5. Black then dropped more material and finished with 22...Qxg1+??, giving away the queen. The opponent's 2.Qf3 and 3.g4 suggest low-level play. Game B Black played a sound opening (...e5, ...d5, ...Bb4, ...O-O) and won with tactics: 15...Ndxc5 recovered material, and 22...Bxf3+ followed by 24...Nf3+ won the exchange and more. The opponent did blunder with 15.Bxc5 and 22.Rd4, but Black's play was coherent and accurate. B looks clearly stronger.", "confidence_valid": true }, { "pair": "pair_05", "game_a": "game_09", "color_a": "white", "game_b": "game_07", "color_b": "black", "rating_a": 2193, "rating_b": 1593, "gap": 600, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.6, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 1700, "old_estimate_b": 1350, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A White handled a Benoni-type opening soundly (6.g3, 9.Nge2, 10.O-O) against a clearly competent opponent, but was outplayed tactically: 21...Bxc3 22.bxc3 Qxa3 cost two pawns, and the later 32.Rxe7 Qf4+ allowed a winning endgame for Black. Still, White kept active play (28.e5, 30.f5) against strong opposition. Game B Black played a solid Slav and found 21...Bc2, forking queen and rook, but this was against a very weak opponent: 21.Bf3?? and 22.Bh5?? simply dropped the queen. That gives little evidence about Black's real strength. The tougher opposition in Game A slightly favors White there.", "confidence_valid": true }, { "pair": "pair_06", "game_a": "game_17", "color_a": "black", "game_b": "game_25", "color_b": "black", "rating_a": 1639, "rating_b": 838, "gap": 801, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.72, "correct": 1, "brier": 0.07840000000000001, "old_estimate_a": 1550, "old_estimate_b": 950, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black plays a coherent Italian with sensible developing moves (…Bc5, …O-O, …d5, …Bg4). The resignation at move 24 likely followed a pawn loss or tactical issue after 24.Rd5, but the play up to that point is solid and positional. Game B Black plays a chaotic opening: 2…Bd7, 3…c6, 4…b5, 5…Na6. Black does win the a1 rook, but then blunders. After 25.Bxc3, Black is down material, and Black later allows mate with 39.Rxd8#. Game B's opponent was also weak (3.Qf3, 4.g4). On balance, A's target shows more competent play.", "confidence_valid": true }, { "pair": "pair_07", "game_a": "game_14", "color_a": "black", "game_b": "game_08", "color_b": "white", "rating_a": 1188, "rating_b": 1310, "gap": 122, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.6, "correct": 0, "brier": 0.36, "old_estimate_a": 1350, "old_estimate_b": 1300, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A Black played an odd opening (1...d6 2...e6 3...h6) but punished White's 10.Bxd5?? with 10...Bxh4, winning a piece. Afterward Black consolidated reasonably, and 24...Re8 and 26...Rad8 led to 29...Rd4 and 30...Rxd3, picking up the bishop. Game B White played a passive setup and then blundered: 15.Bxc5?? simply dropped a piece. That was followed by 21.Kg2?, allowing 22...Bxf3+, and then 23.Kg1, allowing the fork 24...Nf3+. That is repeated tactical blindness. Both opponents were weak, so the evidence is limited. Still, A's target showed better tactical awareness, while B's target made several unforced blunders.", "confidence_valid": true }, { "pair": "pair_08", "game_a": "game_02", "color_a": "black", "game_b": "game_20", "color_b": "black", "rating_a": 2410, "rating_b": 1903, "gap": 507, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.68, "correct": 1, "brier": 0.10239999999999996, "old_estimate_a": 1350, "old_estimate_b": 1450, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game A Black punished errors cleanly. 9...Nxe4 took a free pawn after 8.Nd5, and 18...Bxh4 exploited 18.h4. After the queen trade, 21...Re2 led to 22...Rxb2, 24...Rxh3 and the mating 25...Rh1#, showing efficient, accurate conversion. Admittedly White there was weak. Game B Black played poorly against a standard kingside attack. 6...Re8 and 7...Bh8 wasted tempi and dropped the dark-squared bishop's defensive role. 13...fxg6 opened lines, and 17...Nxh6 handed White the h-file. Black's queenside play (b4, a4, b3) was far too slow, and the king collapsed after 26.Rxh7. Black B's opponent attacked competently, but B's opening and defensive understanding looks weaker than A's tactical alertness.", "confidence_valid": true }, { "pair": "pair_09", "game_a": "game_12", "color_a": "black", "game_b": "game_03", "color_b": "black", "rating_a": 1811, "rating_b": 1901, "gap": 90, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.45, "correct": 1, "brier": 0.2025, "old_estimate_a": 1350, "old_estimate_b": 1300, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Both targets are weak, so this is close to a guess. Game A Black's 10...Nxd4 combination fails to 12.Bb5+, and 13.Qxd4 wins a piece. Black then missed 13...cxb5, which would have limited the damage, and later ignored his back rank: 26...Nxc4 allowed 27.Bxh8 and 28.Qg8+. Game B Black blundered with 14...Bd6 into the 15.Ng6 fork, losing material, and later dropped the exchange to 34.Ne7+. Still, he defended longer and more coherently against a cleaner opponent, with ideas like 22...e5 and 24...Bxd5. That gives a slight edge to Game B's target.", "confidence_valid": true }, { "pair": "pair_10", "game_a": "game_23", "color_a": "black", "game_b": "game_09", "color_b": "black", "rating_a": 564, "rating_b": 2177, "gap": 1613, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.2, "correct": 1, "brier": 0.04000000000000001, "old_estimate_a": 1200, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black blundered badly with 13...Bxh3?, an unsound sacrifice. After 14.gxh3 f5 15.Nxf5 they were simply down material, then wasted tempi with the queen (19...Qg3, 21...Qe6, 22...Qg6) and resigned early. Game B Black played a coherent Benoni/Benko-style plan with ...Na6-c7, ...b5, ...Rb8 and ...Ng4-e5-c4. They won material with 21...Bxc3 and 22...Qxa3, kept active rooks with 25...Rb2 and 26...Qd2, and converted cleanly with 35...Bxd5, ...Rdd2 and 41...f3 mating ideas. B shows much stronger understanding and technique.", "confidence_valid": true }, { "pair": "pair_11", "game_a": "game_21", "color_a": "black", "game_b": "game_10", "color_b": "black", "rating_a": 743, "rating_b": 2501, "gap": 1758, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.25, "correct": 1, "brier": 0.0625, "old_estimate_a": 1300, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black played a dubious opening that dropped pawns: 3...Qe7 and 4...f6 allowed 5.Qxd5 and 7.Qxb7. Black then let a knight go with 26...Nd5 and 28...Ng4, allowing 29.exd5. The rook-versus-knight ending was also handled slowly, and White blundered constantly, so the result says little. Game B Black played a coherent Queen's Gambit-style setup with ...Nbd7, ...c5 and ...b6/Bb7. Black kept the structure sound and found active ideas: 24...Qc2 hit the rook and 25...Ba6 won the exchange (26...Bxf1). This shows tactical awareness against a competent opponent. B's play looks considerably more controlled.", "confidence_valid": true }, { "pair": "pair_12", "game_a": "game_05", "color_a": "black", "game_b": "game_06", "color_b": "black", "rating_a": 1867, "rating_b": 1382, "gap": 485, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.58, "correct": 1, "brier": 0.17640000000000003, "old_estimate_a": 1400, "old_estimate_b": 1450, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game A Black played purposeful moves: the knight raid ...Nf4, ...Nxg2+ winning material, ...fxe4, then a coordinated attack with ...Nbd3, ...Bxh1, ...Nh3+ and a clean finish with ...Qxf3#. The opponent was weak (Bc4-d3-e2 wandering, 16.Be4 allowing Nxg2+). Game B Black played more passively and loosely, with odd bishop shuffles (...Bd7-f5-d7-g4-e6) and 16...Bxg5 giving up the bishop. Black still won material via ...Ncxb3 and ...Nxa1 and later mated with ...Qf2#, but White blundered heavily (Rf1?). Both targets exploited weak opponents, but Game A's attack looked more coherent, so A is slightly favoured.", "confidence_valid": true }, { "pair": "pair_13", "game_a": "game_22", "color_a": "white", "game_b": "game_04", "color_b": "white", "rating_a": 717, "rating_b": 2508, "gap": 1791, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.35, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1200, "old_estimate_b": 1450, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White opens with the unorthodox 2.Qf3 and 3.g4. It wins mainly because Black collapses: 6...Nxf2?? is unsound, and 22...Qxg1+?? gives away the queen. White's own play (13.Qxb5, 17.Qxc5, 21.Qxb7) just collects loose material, which says little about skill against a weak opponent. Game B White plays a coherent Exchange Caro-Kann setup (4.Bd3, 5.c3, Ne2, O-O) and a sensible queenside pawn storm (14.b4, 16.a4, 20.a5) against a competent opponent. It does err: 23.Qb2 lets Black win c5, and 24.Bb4? allows 24...Ne2+ 25.Kh1 Qg1+ 26.Rxg1 Nf2#, a smothered mate. Even so, B's overall understanding and the stronger opposition point to B being the stronger target.", "confidence_valid": true }, { "pair": "pair_14", "game_a": "game_18", "color_a": "black", "game_b": "game_14", "color_b": "white", "rating_a": 2641, "rating_b": 1249, "gap": 1392, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.78, "correct": 1, "brier": 0.04839999999999999, "old_estimate_a": 1600, "old_estimate_b": 1250, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black played a coherent game. After the slightly odd 10...Kf8, the plan of ...h5-h4 opened the h-file, and 15...hxg3 and 16...Bh3 traded off White's key bishop. Black finished tactically with 30...Re2+, 31...Rh2 and 33...Qxg3+. Game B White blundered badly. 8.Nh4 followed by 9.Qf3 let 9...Nd5 and 10...Bxh4 win a knight, so 10.Bxd5 simply lost a piece. Later 30.Rg6?? left the d3 bishop hanging to 30...Rxd3. These are basic tactical oversights, while Black in Game A showed better understanding and calculation.", "confidence_valid": true }, { "pair": "pair_15", "game_a": "game_05", "color_a": "white", "game_b": "game_20", "color_b": "white", "rating_a": 1120, "rating_b": 1932, "gap": 812, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.22, "correct": 1, "brier": 0.0484, "old_estimate_a": 1150, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White played a wandering opening (2.Bc4, 5.Bd3, 6.Be2, 11.Be4, 12.Bd3) and then blundered badly: 16.Be4?? allowed Nxg2+, and 22.Bc3/23.Nd4 dropped the h1 rook. This is a collapse to simple tactics. Game B White ran a coherent Pirc/150 attack: Be3, Qd2, f3, O-O-O, g4, h4, h5 and hxg6, then Rxh6. The 26.Rxh7 sacrifice and the precise finish ending in 38.Nh6# show real attacking skill. Black in B did misplay with 7...Bh8 and 23...Bg7, but B's play is still clearly more competent.", "confidence_valid": true }, { "pair": "pair_16", "game_a": "game_11", "color_a": "black", "game_b": "game_02", "color_b": "white", "rating_a": 1273, "rating_b": 1575, "gap": 302, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.68, "correct": 0, "brier": 0.4624000000000001, "old_estimate_a": 1400, "old_estimate_b": 1200, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A Black chose an odd opening (2...Bc5, 3...Qf6) but then played coherently: 10...Nxd5 won a pawn, 15...Nxe2+ took the bishop, and the endgame was handled actively with 31...dxc4 and 36...Rxc4. Black only collapsed at the end with 40...e3??, allowing the back-rank mate Rd8+. Game B White made more basic errors. 8.Nd5 followed by 9.c3 let Black play 9...Nxe4 for a free pawn. 18.h4?? dropped another pawn, and 20.Qh3 traded queens into a wrecked structure. White then let a rook into e2 and b2, and 25.Bg5?? allowed Rh1#. Black B's poorer play overall points to A being the stronger target.", "confidence_valid": true }, { "pair": "pair_17", "game_a": "game_13", "color_a": "white", "game_b": "game_16", "color_b": "white", "rating_a": 1126, "rating_b": 1752, "gap": 626, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.3, "correct": 1, "brier": 0.09, "old_estimate_a": 1250, "old_estimate_b": 1350, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White played oddly throughout: 5.Be3 then 9.Bd2, 10.Qa1 and 11.Nb1 undeveloping, and 16.Rc5?? dropping the exchange to ...b6. Despite later gaining material after Black's errors (28.Ne8 forking/winning the exchange), White ignored back-rank danger and allowed 35...Rc1#. Game B White handled a sloppy opponent sensibly: accepted 6...Bxf2+ (a bad sacrifice), 13.hxg4 kept the extra piece, and the follow-up was solid with central control (10.d4, 17.Rae1). White also traded queens with 25.Qe2 to convert. Black's 27...Nc7?? hangs f6 or leads to resignation-level loss. B's play shows cleaner technique, so B is likely stronger.", "confidence_valid": true }, { "pair": "pair_18", "game_a": "game_24", "color_a": "black", "game_b": "game_12", "color_b": "white", "rating_a": 765, "rating_b": 1877, "gap": 1112, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.25, "correct": 1, "brier": 0.0625, "old_estimate_a": 1050, "old_estimate_b": 1550, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black plays a chaotic opening with many flank pawn moves (4...b5, 5...a5, 8...h5, 9...g5), which leaves the kingside weak. Black then allows 17.Nxf6+ and 20.Nb6 forking the rook. The decisive error is 22...Qd8??, which simply loses the queen to 23.Bxd8. Game B White is imprecise early: 10.Be3 lets Black win the d4 pawn. Afterward, though, White plays coherently. 12.Bb5+ forces 12...c6, then 13.Qxd4 regains the piece, followed by queenside pressure with 17.b4 and 20.bxc5. White finishes with active tactics (22.Bf4+, 25.Be5, 27.Bxh8) and 28.Qg8+. Game B White looks stronger.", "confidence_valid": true }, { "pair": "pair_19", "game_a": "game_17", "color_a": "white", "game_b": "game_10", "color_b": "white", "rating_a": 1514, "rating_b": 2689, "gap": 1175, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.45, "correct": 1, "brier": 0.2025, "old_estimate_a": 1550, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A (White) played a sound but simple Italian. However, 11.g4 loosened the kingside, and the win came abruptly after 24.Rd5 c6 (likely a flag or resignation), not from clear technique. Game B (White) handled a double-fianchetto English/Catalan structure competently: 12.dxc5, 13.Nd4 and 16.Nxe6 created targets, and 20.Nb5 with 24.Nxa7 won material. White then erred with 25.Rf1? allowing ...Ba6 to win the exchange, and the queen wanderings (Qf4-h4-f4) were imprecise. Game B's opening and middlegame understanding look somewhat more sophisticated, though both players show errors. The evidence is close, so this is a slight lean toward B.", "confidence_valid": true }, { "pair": "pair_20", "game_a": "game_11", "color_a": "white", "game_b": "game_23", "color_b": "white", "rating_a": 1228, "rating_b": 557, "gap": 671, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.57, "correct": 1, "brier": 0.18490000000000004, "old_estimate_a": 1450, "old_estimate_b": 1350, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A White handled a messy game soundly. After Black's odd 2...Bc5 and 3...Qf6 he won the bishop pair with 8.Nxc5, though 15.Be2 (allowing Nxe2+) and 19.Qh4 (allowing Bxf3 to wreck his pawns) were loose. In the endgame he played actively with 30.f6, 34.Bb6, 38.Rb5 and 40.Rd5, then finished with the back-rank mate 41.Rd8+. Game B White played aimlessly early: 2.Bf4, 5.Bb5, 6.Bg5 moves the bishop around. Weakening moves like 17.f3 let Black's queen roam. He mainly won material because Black gave pieces away (13...Bxh3, 15...Bh4). Neither player is convincing, but A showed somewhat more technique.", "confidence_valid": true }, { "pair": "pair_21", "game_a": "game_21", "color_a": "white", "game_b": "game_15", "color_b": "white", "rating_a": 739, "rating_b": 2157, "gap": 1418, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.2, "correct": 1, "brier": 0.04000000000000001, "old_estimate_a": 1250, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White plays an unsound opening (2.Qd3, 4.f4) and later blunders badly: 11.Rc1/12.Rxc7 lets Black's bishop roam, 19.a3? allows ...Nc2+ and ...Nxf1+, and 24.a4 drops more material. White did fight back, as Black blundered too (26...Nd5, 30...Nxh1), but later 39.Ne5+?? gives a knight away and 66.Nxh3?? loses to the fork ...Nf2+. Game B White plays a coherent Slav, wins the b-pawn and a piece via 24.Rd1/25.Qxc8+ tactics, and handles the minor-piece endgame reasonably. White then lost the king-and-pawn ending's winning chances but still held a correct draw (57.Kh1, 58.Kh2). Game B's play is clearly more competent.", "confidence_valid": true }, { "pair": "pair_22", "game_a": "game_04", "color_a": "black", "game_b": "game_13", "color_b": "black", "rating_a": 2718, "rating_b": 1551, "gap": 1167, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.68, "correct": 1, "brier": 0.10239999999999996, "old_estimate_a": 1550, "old_estimate_b": 1300, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black played a coherent Caro-Kann exchange plan (…Bg4xe2, …O-O-O, …Ne4, …f5, …g5) and finished with the sharp queen sacrifice 25…Qg1+ 26.Rxg1 Nf2#, a smothered mate. Game B Black won mainly because White blundered repeatedly (8.Rb3, 16.Rc5 losing the exchange, 35.Nxb5?? allowing …Rc1#). Black's own play there was loose: early queen wandering with 6…Qxb2, and the odd 26…Rf6/28…Re6 that dropped the exchange to Nc7. That suggests a weaker player benefiting from a weak opponent. A's opponent was more competent, and A's tactical finish was cleaner.", "confidence_valid": true }, { "pair": "pair_23", "game_a": "game_25", "color_a": "white", "game_b": "game_18", "color_b": "white", "rating_a": 738, "rating_b": 2426, "gap": 1688, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.3, "correct": 1, "brier": 0.09, "old_estimate_a": 1100, "old_estimate_b": 1450, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A is very low-level on both sides. White's 3.Qf3, 4.g4 and 6.h4 neglect development, and 13.Nh3 allowed 13...Nc2+ forking king and rook. White won mainly because Black blundered repeatedly, e.g. 16...Bxh3 and the later collapse of Black's king. Game B's White played a coherent double-fianchetto setup, including 8.d4, 9.Bb2 and 13.Bd4, and handled a sharp middlegame reasonably. White then collapsed with 32.Qxd5?? into ...Qh3+ and ...Qxg3+, a serious error. Even so, Game B's overall structure and the opponent's quality suggest a stronger level of play than Game A's chaotic game.", "confidence_valid": true }, { "pair": "pair_24", "game_a": "game_03", "color_a": "white", "game_b": "game_15", "color_b": "black", "rating_a": 1782, "rating_b": 2154, "gap": 372, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.66, "correct": 0, "brier": 0.43560000000000004, "old_estimate_a": 1550, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "In Game A, White played a coherent London setup and punished Black's loosening play with clean tactics: 15.Ng6, 16.Bxd6, 23.Nxd5 winning material, and 34.Ne7+ with 35.Nxg8. White then converted smoothly, finishing with 43.Bxg6+ and 46.Re8#. In Game B, Black's play was error-prone. 15...Qd5 allowed White's knights to invade, and 24...Nxb7?? opened the c-file for 25.Qxc8+, dropping a rook to a simple tactic. Black only drew because White also misplayed. The king-and-pawn ending was also handled imprecisely, since the rook pawn let White's king reach the drawing corner. Overall, A's target looks more tactically reliable.", "confidence_valid": true }, { "pair": "pair_25", "game_a": "game_19", "color_a": "black", "game_b": "game_07", "color_b": "white", "rating_a": 2474, "rating_b": 1862, "gap": 612, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.75, "correct": 1, "brier": 0.0625, "old_estimate_a": 1550, "old_estimate_b": 1200, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black played a coherent QGD Exchange with ...Bf5-g6 and ...h5 into an endgame. The errors came late: 22...c5 allowed Nxh5 to win a pawn, and 33...Kd8 and 34...Nxf5 let the h-pawn promote. Overall this was reasonable play against a competent opponent. Game B White made gross blunders. 21.Bf3?? allowed 21...Bc2, trapping the queen, and 22.Bh5 made it worse, losing the queen for little. Earlier play such as 18.Rb3 and 19.b5 was also aimless. Hanging the queen to a simple bishop tactic suggests a weaker player than A's endgame inaccuracies do.", "confidence_valid": true } ] }, { "model": "anthropic/claude-sonnet-5.5", "correct": 22, "n": 25, "accuracy": 88.0, "baseline_accuracy": 76.0, "invalid_confidence": 0, "consistent_reversals": 3, "reversals": [ { "pair": "pair_09_reverse", "original": "pair_09", "prediction": { "stronger": "B", "probability_a_stronger": 0.4, "explanation": "Game A Black played a Stonewall-like setup but drifted. 8...d6 and 9...Qc8 were passive, and 12...bxc5 and 13...d5 left weaknesses. 14...Bd6 let White trade it off with Bxd6. 22...e5 and 23...Nd7 allowed Nxd5 and the loss of a pawn. After that Black held on for a long game, resisting for 46 moves with some tactical awareness, but it was clearly worse throughout. Game B Black made more serious errors: 7...Bxf3 and 8...exd4 were fine, but 9...Qe7+ and 10...Nxd4 lost material to Bb5+ and Qxd4. 19...f5 and 23...Rd4 were blunders, and 25...Rd2 and 26...Nxc4 lost to Qg8+. Both players are weak and Game B Black looks slightly worse, with more tactical oversights in a short game. Game A Black held out longer and played the opening more coherently, so A is slightly stronger and B is the pick only by a slim margin." }, "consistent": false }, { "pair": "pair_15_reverse", "original": "pair_15", "prediction": { "stronger": "A", "probability_a_stronger": 0.72, "explanation": "Game A White played a coherent Pirc/Modern attack: Be3-Bh6, O-O-O, then g4 and h4-h5 pawn storm. The h-file break (13.hxg6, 26.Rxh7) was precise. White converted with 28.fxg6+ and 31.Rxd8, and mated with 38.Nh6#. Black's play was weak, but White handled the attack and endgame cleanly. Game B White opened with a passive, odd line (2.Bc4, 6.Be2, 7.d3 allowing ...cxd3) and lost time with the bishop moves. 10.e5 and 11.Be4 let Black play ...f5, then 15.c4 and 16.Be4 allowed 16...Nxg2+ with a winning attack. White lost to a tactic and was mated quickly, suggesting weaker play. Game A looks stronger, though Black's opposition was also weak there." }, "consistent": true }, { "pair": "pair_13_reverse", "original": "pair_13", "prediction": { "stronger": "B", "probability_a_stronger": 0.4, "explanation": "Both White players look weak. In Game A, White drifted into trouble: 8.Qe1 and 9.Bg5 were awkward, 10.f4 weakened the king, and 20.a5 and 21.c4 were slow moves while Black's attack came. White also missed Black's threats and allowed 24...Ne2+ and 26...Nf2#. In Game B, White opened with 2.Qf3 and 3.g4, which are very dubious, and 6.h3 allowed a knight sacrifice. White did find resources, such as 8.gxh6, 13.Qxb5 and 17.Qxc5, and won after Black blundered 22...Qxg1+. White's tactical alertness was a bit better here, but Black's play was poor. This is a close call; I give B a slight edge because B's White punished mistakes and A's White was mated in a game with no major unforced opening blunders from Black." }, "consistent": false }, { "pair": "pair_12_reverse", "original": "pair_12", "prediction": { "stronger": "A", "probability_a_stronger": 0.6, "explanation": "Game A Black met a weird, loose White setup (4.Bd3, 6.Bb1, 8.h4) with sound play. Black found 13...Nd4 and 18...Ncxb3, picked up the rook on a1 with 19...Nxa1, then mated accurately with 27...Qxf6, 28...Kh6 and 31...Qf2#. Black did miss some defensive nuances, but converted cleanly. Game B Black played a sharp, tactical game (15...Nf4, 16...Nxg2+, 23...Bxh1, 29...Qxf3#), but White blundered badly with 15.c4 and 16.Be4, so Black's win says less about Black's own strength. Black's opening play in B was decent (the ...b5 and ...c4 push). Both opponents look weak, and Game A's opponent looks slightly weaker and more erratic, but A Black's play was somewhat more precise and controlled. Slight edge to A." }, "consistent": true }, { "pair": "pair_18_reverse", "original": "pair_18", "prediction": { "stronger": "A", "probability_a_stronger": 0.8, "explanation": "Game B's Black played very weakly: 2...d6, 3...c6 and the pawn pushes 4...b5, 5...a5, 7...f6, 8...h5, 9...g5 ignored development and weakened the king. Black then dropped the queen with 22...Qd8, allowing 23.Bxd8, after already losing the exchange on 20...Qc6. In Game A, Black (the opponent) blundered, but White played sensibly: 12.Bb5+ and 13.Qxd4 won a piece, and 17.b4 and 19.Qb3 kept the initiative. White then found 22.Bf4+, 25.Be5 and the finishing 27.Bxh8 and 28.Qg8+. White's play was solid and tactically alert. Black in Game A was also weak, but that is evidence about the opponent, not the target. Target A looks stronger." }, "consistent": true } ], "rows": [ { "pair": "pair_01", "game_a": "game_16", "color_a": "black", "game_b": "game_19", "color_b": "white", "rating_a": 1569, "rating_b": 2668, "gap": 1099, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.4, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 1500, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A, Black: Black played a speculative 6...Bxf2+ sacrifice that gave up a piece for a pawn, then played reasonably (12...Bxf3, 17...Rxe1). The game ended abruptly after 27.Re6 Nc7, and Black was worse by then. The opening sacrifice suggests an intermediate-level player. Game B, White: White played a standard Queen's Gambit Exchange with Bg5, traded queens, and played a coherent plan (h4-h5, Rac1, Ne2-g3). Its 22.Nxh5 won a pawn, and it converted the endgame cleanly with 33.Re7+ and 36.Rxh8. Black in Game B also played reasonably but slipped. White in B looks somewhat more solid and technically sound than Black in A, though both are probably club-level players, so the evidence is modest.", "confidence_valid": true }, { "pair": "pair_02", "game_a": "game_01", "color_a": "black", "game_b": "game_24", "color_b": "white", "rating_a": 1159, "rating_b": 762, "gap": 397, "gap_band": "200-499", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.62, "correct": 1, "brier": 0.1444, "old_estimate_a": 1600, "old_estimate_b": 1550, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Both targets are fairly weak, but Black in Game A played more purposefully. In Game A, Black's 15...Nxg5 and 18...Bb7 were reasonable, and 19...Nd4 and 30...Bxg3 showed tactical awareness. Black then converted the endgame with 38...h3 and 40...Rxb8. Black did make loose moves such as 2...d5 and 12...h5, and White was also weak (17.Nxd7 was a blunder-ish trade). In Game B, White won only because Black blundered with 20...Qc6 and 22...Qd8. White's own play was sloppy: 16.gxh4 and 17.Nxf6+ were fine, but 11.Nh5 and 20.Nb6 were speculative. Black in Game B played extremely poorly, which suggests a low-rated pool, and White's win says little about their strength. Game A's opponent was somewhat more competent and Black still outplayed them, which points to A being stronger.", "confidence_valid": true }, { "pair": "pair_03", "game_a": "game_01", "color_a": "white", "game_b": "game_06", "color_b": "white", "rating_a": 1363, "rating_b": 1112, "gap": 251, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.6, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 1550, "old_estimate_b": 1550, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "Both White players lost. In Game A, White lasted 42 moves in a messy game. Opening with 2.d3 and 3.Nc3 was passive, but 10.O-O-O and 11.h4 showed some attacking intent. 15.Nd5 and 17.Nxd7 were reasonable, and White kept pace until Black's passed e-pawn and 30...Bxg3 decided it. Black also played loosely, so White's resistance was fairly normal for the level. In Game B, White's play was much weaker: 8.h4 and 9.b3 loosened the position, 13.Ng5 and 14.Qd3 drifted, and 18.Qe3 allowed 18...Ncxb3 and 19...Nxa1. After 24.Nf6+ Kg7 25.Rf1, White hung material and was mated by 31...Qf2#, having ignored threats with 30.Qxb7. Game B's White therefore looks weaker.", "confidence_valid": true }, { "pair": "pair_04", "game_a": "game_22", "color_a": "black", "game_b": "game_08", "color_b": "black", "rating_a": 557, "rating_b": 1791, "gap": 1234, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.3, "correct": 1, "brier": 0.09, "old_estimate_a": 1450, "old_estimate_b": 1550, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black met 2.Qf3 with decent development, but 5...Ng4 and 6...Nxf2 was an unsound sacrifice. Black followed with 9...Bc5 and 13...Qf2+, yet lost material. Black then blundered with 20...Qf4 and 22...Qxg1+, which gave up the queen, so the game was messy and error-prone. Game B Black played a solid opening with 3...Bb4, 5...O-O and 7...Bg4. The play was coherent: 9...e4 and 10...Nxe4 won a pawn, 14...c5 was a good central break, and 22...Bxf3+ followed by 24...Nf3+ finished cleanly. White in Game B was passive. Black in B appears stronger and more consistent.", "confidence_valid": true }, { "pair": "pair_05", "game_a": "game_09", "color_a": "white", "game_b": "game_07", "color_b": "black", "rating_a": 2193, "rating_b": 1593, "gap": 600, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.6, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 1750, "old_estimate_b": 1550, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A (White) played a reasonable King's Indian/Benoni setup with 9.Nge2, 10.O-O and 12.Bd2. It was then outplayed: 13.Qc1 and 14.h3 allowed ...Ng4-e5-c4 and ...Qxa3 (22...Qxa3 won a pawn), and 19.f4 allowed 19...Nd4. White still fought on with activity (25.Rxa7, 32.Rxe7), so this looks like a solid mid-level player with some tactical lapses. Game B (White) made a serious blunder, 21.Bf3 followed by 22.Bh5, which lost the queen for a bishop and the exchange. The earlier play was passive, and 13.Ba3 and 17.b4 were not harmful but gave Black easy equality. Black in Game B played accurately, but that player is not the target. Target B (Black) played a standard Slav/QGD structure well, winning 12...Bxe4 and 21...Bc2 tactically, though against a weak opponent. Evidence for B is limited because the opponent was weak, while Game A's Black was stronger. Slight lean to A.", "confidence_valid": true }, { "pair": "pair_06", "game_a": "game_17", "color_a": "black", "game_b": "game_25", "color_b": "black", "rating_a": 1639, "rating_b": 838, "gap": 801, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.66, "correct": 1, "brier": 0.11559999999999998, "old_estimate_a": 1600, "old_estimate_b": 1500, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black played a standard Italian setup with sensible development (4...O-O, 6...d5, 8...Bg4, 9...Bb6). Black traded pieces and kept a fairly solid position. The 12...h6 and 13...Nf4 moves were reasonable, though 18...g5 weakened the king and Black eventually lost slowly. Game B Black played a bizarre opening (2...Bd7, 3...c6, 4...b5, 5...Na6) and left the king exposed. Black did win material with tactics (13...Nc2+, 14...Nxa1, 16...Bxh3), but later blundered with 20...Rxd4+ and 21...Rxc4, then dropped the queen with 23...Qc8 and 24...Rxc3+. Overall Game B Black looks weaker and more erratic, so A is more likely stronger.", "confidence_valid": true }, { "pair": "pair_07", "game_a": "game_14", "color_a": "black", "game_b": "game_08", "color_b": "white", "rating_a": 1188, "rating_b": 1310, "gap": 122, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.64, "correct": 0, "brier": 0.4096, "old_estimate_a": 1550, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A, Black: answered the early e5 with ...c6 and ...Be7 and developed sensibly. Played ...Nd5 and ...Bxh4 to win the knight pair, then ...Qf5 and ...Qg6 to improve the queen. ...f5 and ...Rd5 were active, and ...Rd4 followed by ...Rxd3 won a piece after White's Rg6 blunder. This was a solid, purposeful game against a mediocre opponent. Game B, White: played the opening passively and allowed ...e4 with tempo. 15.Bxc5 and 16.Ne5 were questionable, 18.Qxb6 axb6 left a worse structure, and 21.Kg2 walked into ...Ng5. 22.Rd4 and 24.h4 lost to ...Nf3+ smothered-style tactics. White's play had more inaccuracies and a tactical oversight, while Black in Game A was steadier. The evidence is modest, and both opponents seem to be around the same level.", "confidence_valid": true }, { "pair": "pair_08", "game_a": "game_02", "color_a": "black", "game_b": "game_20", "color_b": "black", "rating_a": 2410, "rating_b": 1903, "gap": 507, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.66, "correct": 1, "brier": 0.11559999999999998, "old_estimate_a": 1550, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game A, Black: plays the Open Spanish normally. After White's weak 8.Nd5, Black takes 9...Nxe4 and 10...exd4, then 14...Nc4 and 18...Bxh4 win material and tempo. Black then finishes cleanly with 21...Re2, 22...Rxb2 and 25...Rh1#. White blundered, but Black punished each error efficiently. Game B, Black: plays a King's Indian/Modern setup but loses a pawn structure battle. 7...Bh8 is odd, and 17...Nxh6 followed by 18.Rxh6 lets White's attack run. 24...b3 and 25...Kf7 are dubious, and 26...Nxh7 loses material. Black loses to a mating attack with lots of errors, including 28...Kf8. Black in A looks a bit stronger, though A's opponent was very weak, which tempers the evidence.", "confidence_valid": true }, { "pair": "pair_09", "game_a": "game_12", "color_a": "black", "game_b": "game_03", "color_b": "black", "rating_a": 1811, "rating_b": 1901, "gap": 90, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.35, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1500, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black (Black) played a Modern setup and traded well early, but 12...c6 and 14...O-O-O left the king exposed. 16...Kb8 was passive, and 19...f5 and 20...dxc5 let White's pieces dominate. 23...Rd4 and 24...Qc5 allowed 25.Be5 with a decisive attack. Black blundered into 28.Qg8+ by playing 26...Nxc4, which lost quickly. Game B Black (Black) played a Stonewall-like Dutch with 1...f5 and 8...d6. It was passive early, but Black defended tenaciously for 46 moves, with 22...e5 and 24...Bxd5, and survived into a long endgame. Black blundered on 33...Nc8 and 34.Ne7+, yet resisted longer than Game A's Black and kept a more coherent structure. Game A's collapse in 28 moves suggests weaker play, though both players are probably similar in strength.", "confidence_valid": true }, { "pair": "pair_10", "game_a": "game_23", "color_a": "black", "game_b": "game_09", "color_b": "black", "rating_a": 564, "rating_b": 2177, "gap": 1613, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.3, "correct": 1, "brier": 0.09, "old_estimate_a": 1600, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black played the opening reasonably (4...Bf5, 8...O-O) but erred with 13...Bxh3 14.gxh3 f5 15.Nxf5, which dropped material; 15...Bh4 and 16...Qxh4 then lost a piece. Later 18...Qxe5 and 20...Qxh3+ grabbed pawns, but White had a strong attack, and Black lost quickly in 24 moves. Game B Black handled a King's Indian-style Benoni well: 11...Nxb5, 13...Ng4 and 15...Nc4 were active, 21...Bxc3 and 22...Qxa3 won material, and the tactical finish with 39...Rh2+ and 41...f3 was accurate. Black still allowed some complications (25.Rxa7), but overall played at a higher and more consistent level than A Black.", "confidence_valid": true }, { "pair": "pair_11", "game_a": "game_21", "color_a": "black", "game_b": "game_10", "color_b": "black", "rating_a": 743, "rating_b": 2501, "gap": 1758, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.35, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1500, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black met 2.Qd3 with 2...e5 and 4...f6, which left Black down a pawn after 7.Qxb7. Black then found 7...Qb4+ and 8...Bxb4+, followed by 10...Bxc2, which won back material and gave Black activity. Black won a long endgame. White's play was weak, and Black's conversion was uneven, with a long rook-and-knight ending that took until move 69. Game B Black played a solid, principled Queen's Gambit-style setup (Nbd7, Be7, ...c5, ...b6, ...Bb7). Black played 12...Nxc5, 15...Ne6, and 17...Bd6 sensibly. Black then tactically punished White's 24.Nxa7 with 24...Qc2 and 26...Bxf1, winning quickly. Black in B looks more coherent and cleaner, though White in B also blundered. Game A's opponent also played poorly, and Black's play there was shakier. I lean B.", "confidence_valid": true }, { "pair": "pair_12", "game_a": "game_05", "color_a": "black", "game_b": "game_06", "color_b": "black", "rating_a": 1867, "rating_b": 1382, "gap": 485, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.4, "correct": 0, "brier": 0.36, "old_estimate_a": 1600, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game A (Black): White played poorly, with 10.e5 and 11.Be4 allowing 11...f5, then 15.c4 and 16.Be4. Black found 16...Nxg2+ and later 23...Bxh1, 25...Nh3+ and 29...Qxf3#, a strong attack. But White's opening and defense were weak, so the opponent looks low-rated, and Black's 2...e6 and 4...b5 are standard Sicilian-gambit-like play at club level. Game B (Black): Black played a solid King's Indian/Philidor setup and took the chance after 13.Ng5 with 13...Nd4, 16...Bxg5 and 18...Ncxb3, winning material. White blundered with 14.Qd3 and 19.f5, and Black then converted accurately with 25...Bxf1, 27...Qxf6 and 31...Qf2#. Black's 30...Rc1+ was a precise finish. Both opponents were weak and both targets played tactically well, so it is close. Black in B showed slightly more positional understanding and calculation in a more complex game, so I lean B.", "confidence_valid": true }, { "pair": "pair_13", "game_a": "game_22", "color_a": "white", "game_b": "game_04", "color_b": "white", "rating_a": 717, "rating_b": 2508, "gap": 1791, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.4, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 1450, "old_estimate_b": 1500, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A: White opened 2.Qf3 and 3.g4, which are dubious. Black's 6...Nxf2 sacrifice and 8...Qf6+ gave Black a strong attack, yet White survived. Black then blundered with 13...Qf2+ and 16...cxd5, and White won with 17.Qxc5 and 21.Qxb7. This points to a weak game from both sides, and White's opening moves were poor. Game B: White played a reasonable Caro-Kann Exchange with solid development (Bd3, c3, Ne2, Qe1), but 10.f4 and 18.Nc5 gave Black chances. The decisive error was 21.c4 and 22.b5, which ignored Black's threats, and White lost to a knight-and-queen-sacrifice mate. White in B played more principled chess for most of the game, while White in A relied on opponent errors. B is slightly more likely stronger, but with low confidence.", "confidence_valid": true }, { "pair": "pair_14", "game_a": "game_18", "color_a": "black", "game_b": "game_14", "color_b": "white", "rating_a": 2641, "rating_b": 1249, "gap": 1392, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.66, "correct": 1, "brier": 0.11559999999999998, "old_estimate_a": 1600, "old_estimate_b": 1500, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A, Black: answered the Sicilian setup well with ...d5 and ...Nf6, and took the h-file with ...h5-h4 and ...hxg3. Black won with 26...Bxc3 and 28...Nxd5, and finished with ...Qh3+ and ...Qxg3+. ...Kf8 and ...h5 were a bit loose, but White's play was also weak (Bxc5+ and Qxd5 did not hold). Game B, White: opened with 2.Bc4 and 4.e5, and the 8.Nh4 and 9.Qf3 plan lost a piece after 10...Bxh4. Black's play was fairly accurate there, but White's blunders were basic. White also dropped a bishop with 30.Rg6 Rxd3. Both games show club-level play, but White in B made the larger errors, so target A looks somewhat stronger.", "confidence_valid": true }, { "pair": "pair_15", "game_a": "game_05", "color_a": "white", "game_b": "game_20", "color_b": "white", "rating_a": 1120, "rating_b": 1932, "gap": 812, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.3, "correct": 1, "brier": 0.09, "old_estimate_a": 1450, "old_estimate_b": 1700, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White played a Bc4/c3 setup against the Sicilian, then went wrong with 10.e5 and 11.Be4, which allowed 11...f5. After 15.c4 Nf4 and 16.Be4?, Black's 16...Nxg2+ won material with a strong attack, and White never recovered. Overall that was a loss of control in the opening and middlegame, with White missing the tactics. Game B White played a coherent Pirc attack with Be3, Qd2, Bh6 and a pawn storm (g4, h4-h5). White found the 26.Rxh7 combination, then 29.Qxh8+ and the 33.Nf5 and 38.Nh6# finish, which converted accurately. Black's defence was weak, but White's play was more purposeful and tactically sound, so B looks stronger.", "confidence_valid": true }, { "pair": "pair_16", "game_a": "game_11", "color_a": "black", "game_b": "game_02", "color_b": "white", "rating_a": 1273, "rating_b": 1575, "gap": 302, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.4, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 1600, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A, Black: played an early Qf6/Qb6 opening and lost the queen trade with 8...Qxc5 fine, but then 23...e4 and 24...d5 let White gain f-pawn space. Black allowed 25.f5 and 30.f6, then 36...Rxc4 and 38...Rc1 lost to a back-rank mate after 41...Re8. Black showed decent tactics like 15...Nxe2+ but was passive and missed the mating net. Game B, White: knew the Ruy Lopez to move 7, but 8.Nd5 and 9.c3 lost a pawn to 9...Nxe4. Then 16.Bh6, 18.h4 and 20.Qh3 were poor, allowing 18...Bxh4 and 21...Re2, and White lost badly with a hung b2 pawn. White in Game B played worse, with more blunders, while Black in Game A was only moderately weak. Slight edge to B's target being weaker, so A is slightly favored, but A's Black also lost. The choice is close.", "confidence_valid": true }, { "pair": "pair_17", "game_a": "game_13", "color_a": "white", "game_b": "game_16", "color_b": "white", "rating_a": 1126, "rating_b": 1752, "gap": 626, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.3, "correct": 1, "brier": 0.09, "old_estimate_a": 1450, "old_estimate_b": 1550, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White played an odd, passive opening: 3.dxc5 Qa5+ 4.Bd2 Qxc5 5.Be3 Qb5 6.Nc3 Qxb2 lost a pawn and allowed the queen raid. Later 7.Rb1 Qa3 8.Rb3 and 16.Rc5 were dubious rook moves. 20.Ne4 and 23.Nd6 were also shaky, and 28.Ne8 and 29.Nc7 lost material. White then missed the final back-rank mate (35.Nxb5 Rc1#). Game B White took a pawn after 2.exd5, and Black's 6...Bxf2+ was an unsound sacrifice. White handled it cleanly with 7.Kxf2, 10.d4, 11.Bg5 and 13.hxg4, then traded into a won endgame. Black's play was also weak, but White was accurate and reasonable. Game B White looks somewhat stronger, though both are modest in level.", "confidence_valid": true }, { "pair": "pair_18", "game_a": "game_24", "color_a": "black", "game_b": "game_12", "color_b": "white", "rating_a": 765, "rating_b": 1877, "gap": 1112, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.3, "correct": 1, "brier": 0.09, "old_estimate_a": 1450, "old_estimate_b": 1550, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black played weakly: 2...d6 and 3...c6 with the pawn pushes 4...b5, 5...a5, 6...b4, 8...h5 and 9...g5 ignored development and weakened the king. 11...c5 and 13...Nd4 left material hanging, and 16...Bg4 allowed 17.Nxf6+. 20...Qc6 then 21...Qxa8 lost to 22.Bd5 and 23.Bxd8, a queen trap-style blunder. White in A was also loose but won tactically. Game B White played a sound opening (Bd3, Nbd2, Be3) and punished 7...Bxf3 and 10...Nxd4 with solid moves like 12.Bb5+, 13.Qxd4 and 17.b4. It then found 22.Bf4+, 23.Bxc4 and 25.Be5, picking up material and finishing with 28.Qg8+. B's opponent was also inaccurate, but White in B played cleanly, while A's Black blundered repeatedly. Black in A looks weaker than White in B.", "confidence_valid": true }, { "pair": "pair_19", "game_a": "game_17", "color_a": "white", "game_b": "game_10", "color_b": "white", "rating_a": 1514, "rating_b": 2689, "gap": 1175, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.35, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1650, "old_estimate_b": 1700, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White played an Italian-style opening but allowed 11...Bg6 and 13...Nf4 trades. The 24 moves are not very accurate: 11.g4 loosened the king, and 15.Kg2 left it exposed. White then won against a weak Black who dropped material (24...c6 ended the game, probably after a blunder). Black's play (18...g5, 22...Ne5) was loose, suggesting a modest opponent. Game B White played a coherent English with 3.g3, Bb2, Rc1, and 13.Nd4, so the structure and development look sound. But 20.Nb5 and 24.Nxa7 grabbed a pawn and left White's pieces offside and exposed to 24...Qc2 and 26...Bxf1, and White lost. Black in Game B played well and punished White's mistakes, which suggests a stronger opponent. Overall B's play was more theoretically sound but A's opponent was weaker. I lean slightly to B.", "confidence_valid": true }, { "pair": "pair_20", "game_a": "game_11", "color_a": "white", "game_b": "game_23", "color_b": "white", "rating_a": 1228, "rating_b": 557, "gap": 671, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.62, "correct": 1, "brier": 0.1444, "old_estimate_a": 1600, "old_estimate_b": 1700, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game A White (A) played a long, coherent game. They traded the dark-squared bishop with 8.Nxc5 and handled the queen trade on move 25 well. They pushed f6 (30.f6) and the passed pawns, then found the finishing 40.Rd5 and 42.Rxe8#. Black blundered the endgame, though White did make a few inaccuracies, such as 17.h3 and the weakened structure after 19...Bxf3 20.gxf3. Game B White (B) played the slow, passive-looking 3.h3 and 5.Bb5, and 10.Bxc6 gave up the bishop pair. They then allowed 13...Bxh3 14.gxh3 f5, which exposed their king. White won only because Black blundered with 14...f5 15.Nxf5 and 15...Bh4, and the opponent's play was weak throughout (16...Qxh4 and the later queen moves). B's win was largely a gift, and A's performance was more consistent and showed better technique, so A is slightly more likely the stronger player.", "confidence_valid": true }, { "pair": "pair_21", "game_a": "game_21", "color_a": "white", "game_b": "game_15", "color_b": "white", "rating_a": 739, "rating_b": 2157, "gap": 1418, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.25, "correct": 1, "brier": 0.0625, "old_estimate_a": 1450, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White played 2.Qd3, an early queen sortie, then 4.f4 and 5.Qxd5. Grabbing 7.Qxb7 allowed 7...Qb4+ and an immediate queen trade, with Black's rook and bishop active. It later lost the h1 rook to 30...Nxh1 and lost the endgame. These are weak, beginner-like choices. Game B White followed standard Queen's Gambit Accepted theory and developed sensibly. It kept the structure sound with 9.Nbd2, 10.O-O and 13.Bxa6. It traded into an even endgame with 25.Qxc8+ and 26.Rxd5. It held a draw accurately against a decent opponent with Black's active king play. B looks clearly stronger and more consistent, though the sample is small.", "confidence_valid": true }, { "pair": "pair_22", "game_a": "game_04", "color_a": "black", "game_b": "game_13", "color_b": "black", "rating_a": 2718, "rating_b": 1551, "gap": 1167, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.6, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 1600, "old_estimate_b": 1550, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black played the Caro-Kann sensibly (4...Nc6, 6...Bg4, 10...Bxe2) and then punished White's weak kingside play. 18...Bxc5 and 19...g5 were active, 21...gxf4 opened lines, and 22...Nd4 followed by 23...Qxc5 and 24...Ne2+ led to a clean smothered-style mate with 26...Nf2#. The opponent was weak, so the win is only modest evidence. Game B Black grabbed the b2 pawn with 6...Qxb2 and won, but the opponent was also weak: White's 16.Rc5 lost an exchange, and 28.Ne8 and 30.Nxe6 were poor. Black's play was solid, yet 21...Qe5 and 26...Rf6 were fairly ordinary, and the endgame mate was handed over. Both opponents look similar in strength. A's attack was slightly more convincing, so I lean to A.", "confidence_valid": true }, { "pair": "pair_23", "game_a": "game_25", "color_a": "white", "game_b": "game_18", "color_b": "white", "rating_a": 738, "rating_b": 2426, "gap": 1688, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.35, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1550, "old_estimate_b": 1550, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "Game A: White played the opening poorly (2.e3 and 4.g4 gambits, 5.h3 and 6.h4 pawn-pushing, 3.Qf3 early queen). Black's 13...Nc2+ and 16...Bxh3 were strong, and White was probably losing around move 20. White won only because Black blundered, e.g. 23...Qc8 and 24...Rxc3+. The result suggests a low-rated, chaotic game. Game B: White played a solid Sicilian setup with 3.b3, 4.g3 and 7.O-O, and the moves were reasonable, though 10.Re1+ and 14.c4 led to a worse position. The loss came from tactics: 31.Kf1? and 32.Qxd5? allowed 32...Qh3+ and 33...Qxg3+. White's opening play was more principled in B, so B is likely stronger, but the evidence is weak and the margin small.", "confidence_valid": true }, { "pair": "pair_24", "game_a": "game_03", "color_a": "white", "game_b": "game_15", "color_b": "black", "rating_a": 1782, "rating_b": 2154, "gap": 372, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.35, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1700, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White (Stronger? unclear): played a Stonewall-like Bf4 setup with an early h4 and Ng5. The play was aggressive but mostly sound, and Black's defence was loose (e.g. 9...Qc8, 20...Qd7, 23...Nd7 allowing Nxd5). White won with Nxd5 and Qxc5, then cleanly finished with Rd8 and Re8#. Overall this is a club-level performance with tactical alertness. Game B Black: handled a Slav and Queen's Gambit Accepted line with ...b5/...b4 accurately, and found ...Qd5 and ...Rxa4. Black navigated the tactical sequence (15.Ncxa5 through 26.Rxd5 exd5) and held an ending a pawn down against a stronger-looking White; the 41...Kxc5 pawn race was precise, and the draw was secure. Black's play was more accurate and consistent against a competent opponent, so B is slightly more likely to be stronger.", "confidence_valid": true }, { "pair": "pair_25", "game_a": "game_19", "color_a": "black", "game_b": "game_07", "color_b": "white", "rating_a": 2474, "rating_b": 1862, "gap": 612, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.4, "correct": 0, "brier": 0.36, "old_estimate_a": 1600, "old_estimate_b": 1500, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A, Black: allowed a queen trade and doubled f-pawns, then played passively. 13...b5 and 14...Ke7 were okay, but 22...c5 and 23...c4+ let White build a kingside attack. Black blundered with 36...b4, overlooking the h-pawn advance; 34...Nxf5 gave up the knight and the rook trade was poor, so the h-pawn was decisive. Black's play looks around club level. Game B, White: played a standard Slav/Queen's Gambit structure sensibly through move 16, but 21.Bf3 allowed 21...Bc2, and 22.Bh5 Bxd1 23.Bxg6 Bxb3 lost the exchange and more, a major tactical oversight (the queen was en prise to a deflection). Both are weak; White in B blundered a queen-trade sequence, whereas Black in A held a roughly equal game for a long time. Slight lean to B being stronger because B's opening play was more structured, though the evidence is thin.", "confidence_valid": true } ] }, { "model": "google/gemini-3.8-flash", "correct": 22, "n": 25, "accuracy": 88.0, "baseline_accuracy": 86.0, "invalid_confidence": 0, "consistent_reversals": 4, "reversals": [ { "pair": "pair_09_reverse", "original": "pair_09", "prediction": { "stronger": "B", "probability_a_stronger": 0.22, "explanation": "Game A's Black demonstrated very low tactical awareness, blundering a full bishop outright on move 16 with 16...Kf7?? instead of recapturing on d6, quickly followed by passive play and further tactical oversights (23...Nd7 allowing 24. Qb3). In contrast, Game B's Black committed a common tactical miscalculation on move 10 (10...Nxd4? 11. Nxd4 Bxd4), missing White's subtle intermezzo 12. Bb5+ before recapturing. Despite being down a piece, Black in Game B demonstrated dynamic resourcefulness and aggressive counterplay (21...c4, 23...Rd4, and 25...Rd2 creating serious mating threats) before ultimately falling to back-rank tactics. Target B showed significantly higher tactical and positional understanding." }, "consistent": false }, { "pair": "pair_15_reverse", "original": "pair_15", "prediction": { "stronger": "A", "probability_a_stronger": 0.88, "explanation": "Game A White executed a textbook English/150 Attack against the Modern Defense with high tactical precision. After building a pawn storm with g4-h4-h5, White correctly sacrificed on h7 (26. Rxh7! Nxh7 27. Qxh7 Rh8 28. fxg6+), followed by a decisive queen sacrifice on h8 to force mate, showing strong attacking knowledge. In contrast, Game B White violated basic opening principles by moving the bishop five times in the first 16 moves (Bc4-d3-Be2-d3-e4), then blundered material immediately with 16. Be4? Nxg2+ 17. Kf1 fxe4. White's king was trapped and subsequently mated on f3 after a series of heavy tactical blunders." }, "consistent": true }, { "pair": "pair_13_reverse", "original": "pair_13", "prediction": { "stronger": "A", "probability_a_stronger": 0.85, "explanation": "Game A White demonstrates solid positional knowledge in a standard Exchange Caro-Kann, castling early and executing a thematic queenside minority-style expansion (14. b4, 15. Nb3, 16. a4, 18. Nc5). Although White completely overlooks Black's spectacular smothered mate sequence starting with 24... Ne2+, the quality of play up to that point reflects a coherent, intermediate plan. In contrast, Game B White displays novice tendencies right from the start, playing erratic early queen and pawn moves (2. Qf3, 3. g4) that provoke rapid black piece development and a king walk to e3 by move 9. Despite winning after Black collapsed, Game B White exhibits much lower baseline competence." }, "consistent": true }, { "pair": "pair_12_reverse", "original": "pair_12", "prediction": { "stronger": "A", "probability_a_stronger": 0.58, "explanation": "Both players punished their opponents' dubious setups efficiently, but Player A demonstrated slightly higher positional polish. In Game A, Black foresaw White's battery threat with 7...g6 and 8...h5, then executed a high-level knight maneuver (13...Nd4, 14...Nd7!, 17...Nc5!) that immediately dismantled White's position via 18...Ncxb3! and 19...Nxa1. In Game B, Black played a very strong tactical game but missed an immediate tactic (13...Nxc3 winning a central pawn cleanly) before capitalizing on White's blunders starting with 16. Be4??. Player A showed slightly cleaner calculation and prophylaxis throughout." }, "consistent": true }, { "pair": "pair_18_reverse", "original": "pair_18", "prediction": { "stronger": "A", "probability_a_stronger": 0.88, "explanation": "Player A (White) demonstrates sound tactical alertness and conversion. After Black grabbed a central pawn, White immediately responded with 12. Bb5+! and 13. Qxd4, winning a piece. White then handled the initiative cleanly, finding forceful tactical moves like 22. Bf4+!, 24. Rfe1, and 25. Be5 to finish cleanly. In Game B, Black played an erratic, overextended opening (pushing pawns with 4...b5, 5...a5, 7...f6, 8...h5, 9...g5 while leaving pieces undeveloped). Although Black gained some tactics, Black completely collapsed with the catastrophic one-move queen blunder 22...Qd8?? leaving the queen entirely hanging to 23. Bxd8." }, "consistent": true } ], "rows": [ { "pair": "pair_01", "game_a": "game_16", "color_a": "black", "game_b": "game_19", "color_b": "white", "rating_a": 1569, "rating_b": 2668, "gap": 1099, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.03, "correct": 1, "brier": 0.0009, "old_estimate_a": 1450, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black played an unsound opening line and immediately sacrificed a full minor piece for zero compensation with 6...Bxf2+ 7.Kxf2, after which White smoothly consolidated with Rf1 and Kg1, leaving Black a piece down without counterplay. In contrast, Game B White demonstrated solid positional understanding and endgame technique. White strategically compromised Black's pawn structure (8.Bxf6 and 9.Qxf6), systematically exploited Black's weaknesses on the kingside (16.Nhf4, 22.Nxh5), created an unstoppable passed h-pawn, and executed a decisive tactical wrap-up (34.Ra7 and 35.Ra8+). Game B White shows vastly superior chess strength.", "confidence_valid": true }, { "pair": "pair_02", "game_a": "game_01", "color_a": "black", "game_b": "game_24", "color_b": "white", "rating_a": 1159, "rating_b": 762, "gap": 397, "gap_band": "200-499", "low_band": "one<1000", "gold": "A", "choice": "B", "probability_a": 0.22, "correct": 0, "brier": 0.6084, "old_estimate_a": 1250, "old_estimate_b": 1250, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "Player B showed much sounder opening fundamentals (developing pieces naturally with Nf3, Nc3, Bc4, and castling) compared to Black in Game A, who pushed kingside pawns recklessly (4...f5, 8...h6, 9...g5). Furthermore, Black in Game A committed an elementary tactical oversight with 15...Nxg5??, directly walking into the 16. Nf6+ fork that dropped their queen for just a minor piece, only winning because White completely hung their queen on move 40. In contrast, White in Game B maintained solid piece coordination, navigated Black's flank aggression, and capitalized on tactics with 20. Nb6 and 22. Bd5 to decisively win material.", "confidence_valid": true }, { "pair": "pair_03", "game_a": "game_01", "color_a": "white", "game_b": "game_06", "color_b": "white", "rating_a": 1363, "rating_b": 1112, "gap": 251, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.78, "correct": 1, "brier": 0.04839999999999999, "old_estimate_a": 1200, "old_estimate_b": 1250, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Target A demonstrated good tactical alertness with 16. Nf6+, winning Black's queen, and built a completely winning position before suffering an apparent mouse-slip or massive lapse with 40. Qb8+. Up to that point, White in Game A navigated tactical complications and pushed pawns purposefully. In contrast, Target B played a very passive and awkward opening (4. Bd3, 6. Bb1) and was outplayed step-by-step. Black quickly dominated the board with active knights and bishops (18... Ncxb3), while Game B White shed material easily (20. Bxa1, 25. Rf1) and succumbed swiftly to 31... Qf2#.", "confidence_valid": true }, { "pair": "pair_04", "game_a": "game_22", "color_a": "black", "game_b": "game_08", "color_b": "black", "rating_a": 557, "rating_b": 1791, "gap": 1234, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.08, "correct": 1, "brier": 0.0064, "old_estimate_a": 800, "old_estimate_b": 1500, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B's Black played sound, disciplined chess, exploiting the pin on f3 with 9...e4 and winning material cleanly with the 14...c5 fork. Black then executed a completely winning endgame sequence with 20...Bg4 and 21...Ng5 to force White's resignation. In contrast, Game A's Black played an unsound, chaotic attack (6...Nxf2), missed capturing White's queen on move 17 (17...Bf5 instead of 17...Qxc5), and blundered away their queen multiple times (20...Qf4 and 22...Qxg1+), showing very low tactical awareness.", "confidence_valid": true }, { "pair": "pair_05", "game_a": "game_09", "color_a": "white", "game_b": "game_07", "color_b": "black", "rating_a": 2193, "rating_b": 1593, "gap": 600, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.35, "correct": 0, "brier": 0.42250000000000004, "old_estimate_a": 1900, "old_estimate_b": 1650, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game B Black showed excellent positional handling of the Slav/London structure, trading pieces harmoniously and setting up a decisive tactical strike with 20...Qg6! (threatening mate on g2) followed by 21...Bc2! to cleanly net a rook. In Game A, White handled the Benoni setup sluggishly (13. Qc1, 14. h3, 15. Kh2), allowing Black free rein on the queenside. White then blundered a pawn with 21. Qxe2? and drifted into a completely passive, dominated position along the b-file and second rank. Target B exhibited cleaner tactical precision and overall coordination than Target A.", "confidence_valid": true }, { "pair": "pair_06", "game_a": "game_17", "color_a": "black", "game_b": "game_25", "color_b": "black", "rating_a": 1639, "rating_b": 838, "gap": 801, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.88, "correct": 1, "brier": 0.0144, "old_estimate_a": 1650, "old_estimate_b": 1300, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Player A demonstrates sound opening principles and structured positional understanding in a Giuoco Pianissimo, coordinating pieces logically (5...Nc6, 6...d5, 8...Bg4) and managing trades cleanly until 24...c6. In contrast, Player B plays an extremely low-level game characterized by bizarre early pawn thrusts and awkward setups (2...Bd7, 4...b5, 7...Rd8). Despite winning a rook with 13...Nc2+, Black in Game B displayed severe tactical blindness, needlessly sacrificing material with 24...Rxc3+ and passively walking into back-rank and pin disasters (30...Be7?, 32...Nxf5?) culminating in 39. Rxd8#.", "confidence_valid": true }, { "pair": "pair_07", "game_a": "game_14", "color_a": "black", "game_b": "game_08", "color_b": "white", "rating_a": 1188, "rating_b": 1310, "gap": 122, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.28, "correct": 1, "brier": 0.07840000000000001, "old_estimate_a": 1450, "old_estimate_b": 1250, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A features severe tactical oversights by both players characteristic of lower ratings. In Game A, Black (Target A) missed winning a free knight with 15...Qxe4 (playing 15...Rd8 instead), and on move 17 played 17...Qf6, completely hanging the undefended rook on e8 (which White inexplicably failed to capture). In Game B, White (Target B) played a somewhat passive opening and lost a piece to an alert fork (14...c5), but Black in Game B demonstrated strong tactical pressure (20...Bg4 and 21...Ng5). Target B's play was flawed, but avoiding basic one-move blunders suggests a higher rating than Target A.", "confidence_valid": true }, { "pair": "pair_08", "game_a": "game_02", "color_a": "black", "game_b": "game_20", "color_b": "black", "rating_a": 2410, "rating_b": 1903, "gap": 507, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.68, "correct": 1, "brier": 0.10239999999999996, "old_estimate_a": 1650, "old_estimate_b": 1650, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "Game A's Black demonstrated clean, disciplined positional and tactical play throughout. In the Spanish opening, Black capitalized cleanly on White's pawn sacrifice with 9...Nxe4, consolidated comfortably with 13...Na5 and 15...g6, and methodically punished White's overextension on the kingside with 18...Bxh4 and 21...Re2, executing a swift finish. Game B's Black mishandled the defense against White's standard kingside storm in the Modern/Pirc, spending too many tempi shuffling knights (14...Ne5, 15...Nf7, 16...Nd7, 18...Nf8) while queenside counterplay lagged behind. This passive defense allowed White a decisive breakthrough via 23. Qh2 and 26. Rxh7.", "confidence_valid": true }, { "pair": "pair_09", "game_a": "game_12", "color_a": "black", "game_b": "game_03", "color_b": "black", "rating_a": 1811, "rating_b": 1901, "gap": 90, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.35, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1500, "old_estimate_b": 1600, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black exhibits severe tactical blind spots typical of lower ratings. After falling for the elementary deflection tactic 10...Nxd4? 11.Nxd4 Bxd4 12.Bb5+!, Black orchestrated a dynamic counterattack but suffered complete tactical tunnel vision at the climax: Black missed a mate-in-two (27...Qxf2+ 28.Kh1 Qxg2#) and instead played 27...Nxe3??, walking straight into 28.Qg8# on the back rank. Game B Black played a coherent Dutch setup; although Black overlooked the tactical hook 15...Rd8? permitting 16.Bxd6, Black alertly spotted the fork threat and avoided immediate disaster with 16...Kf7. Overall, Game B Black demonstrates better strategic coherence and board awareness than Game A Black.", "confidence_valid": true }, { "pair": "pair_10", "game_a": "game_23", "color_a": "black", "game_b": "game_09", "color_b": "black", "rating_a": 564, "rating_b": 2177, "gap": 1613, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.12, "correct": 1, "brier": 0.0144, "old_estimate_a": 1200, "old_estimate_b": 2050, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B Black demonstrated strong strategic mastery in a Benoni setup, methodically executing the typical queenside expansion (...b5, ...Nc7, ...Rb8), winning a pawn cleanly with 21...Bxc3 and 22...Qxa3, and converting decisively with active rook play (...Rb2, ...Rdd2) to weave a mating net. In contrast, Game A Black blundered a pawn immediately with 11...e5?, followed up with an unsound piece sacrifice 13...Bxh3, and never achieved real compensation before losing quickly. Player B's understanding of opening plans, piece coordination, and technical execution reflects a considerably higher playing strength.", "confidence_valid": true }, { "pair": "pair_11", "game_a": "game_21", "color_a": "black", "game_b": "game_10", "color_b": "black", "rating_a": 743, "rating_b": 2501, "gap": 1758, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.04, "correct": 1, "brier": 0.0016, "old_estimate_a": 1300, "old_estimate_b": 2150, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B demonstrates high-level, mature positional chess and sharp tactics. Black in Game B navigated a classical Catalan/English structure cleanly, coordinated pieces purposefully, and executed a decisive tactical finish starting with 24...Qc2!, culminating in 27...Ne8!, leaving White completely lost. In stark contrast, Game A features chaotic, low-rated beginner/intermediate play: Black played the unsound 2...e5, 3...Qe7, and 4...f6, blundering early on (allowing 7. Qxb7), followed by both sides missing simple tactical themes (e.g., White dropping the h1-rook to ...Nxh1). Black in Game B plays with vastly superior chess understanding.", "confidence_valid": true }, { "pair": "pair_12", "game_a": "game_05", "color_a": "black", "game_b": "game_06", "color_b": "black", "rating_a": 1867, "rating_b": 1382, "gap": 485, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.42, "correct": 0, "brier": 0.3364000000000001, "old_estimate_a": 1550, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Both target players won cleanly with Black, but Game B faced a more coherent kingside setup and demonstrated superior coordination. Game B Black neutralized White's h-pawn thrust, found energetic knight outposts (13...Nd4, 17...Nc5), and executed a dominant queenside infiltration (18...Ncxb3, 19...Nxa1). The conversion was completely clinical through 29...Rac8 and mate. In Game A, White wasted several tempi shuffling the bishop (Bd3-Be2, Be4-Bd3-Be4) and blundered outright with 16. Be4?? allowing 16...Nxg2+, leaving Black with an easier tactical task to finish the game.", "confidence_valid": true }, { "pair": "pair_13", "game_a": "game_22", "color_a": "white", "game_b": "game_04", "color_b": "white", "rating_a": 717, "rating_b": 2508, "gap": 1791, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.08, "correct": 1, "brier": 0.0064, "old_estimate_a": 850, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A features very low-level, chaotic play: White plays dubious early queen moves like 2. Qf3 and 3. g4, allowing their king to wander to e3 on move 9, while both sides repeatedly hang material. In contrast, Game B shows solid positional opening knowledge in the Caro-Kann Exchange Variation (3. exd5, 4. Bd3, 5. c3) and coherent opposite-side castling plans. Although White in Game B blundered into a classic smothered mate tactic starting with 24. Bb4? allowing 24...Ne2+, their strategic understanding and move quality across the middlegame reflect a vastly higher established rating than White in Game A.", "confidence_valid": true }, { "pair": "pair_14", "game_a": "game_18", "color_a": "black", "game_b": "game_14", "color_b": "white", "rating_a": 2641, "rating_b": 1249, "gap": 1392, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.83, "correct": 1, "brier": 0.028900000000000012, "old_estimate_a": 1850, "old_estimate_b": 1350, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Player A (Black) conducted an energetic, purposeful kingside attack featuring early dynamic play with ...h5-h4. When White miscalculated with 30. Rd1, Black punished it crisply with 30...Re2+! followed by 31...Rh2! and 32...Qh3+, forcing resignation due to unstoppable mate. In contrast, Player B (White) displayed weak positional understanding with moves like 8. Nh4, outright hung a knight on e4 at move 15 (which Black missed), and eventually resigned after blatantly hanging the d3-bishop in one move with 30. Rg6??. Target A demonstrates far superior board vision and tactical sharpness.", "confidence_valid": true }, { "pair": "pair_15", "game_a": "game_05", "color_a": "white", "game_b": "game_20", "color_b": "white", "rating_a": 1120, "rating_b": 1932, "gap": 812, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.03, "correct": 1, "brier": 0.0009, "old_estimate_a": 1200, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White shows poor opening understanding, moving the light-squared bishop five times in the first twelve moves (Bc4-d3-e2-d3-e4-d3) before blundering tactically with 16. Be4?? (allowing 16...Nxg2+ and dropping decisive material). In contrast, Game B White executes a textbook Yugoslav-style kingside storm against the Pirc/Modern. Game B White methodically opens the h-file with 11. h4, 12. h5, and 22. Rdh1, followed by a crisp tactical breakthrough with 26. Rxh7! and 28. fxg6+, efficiently winning Black's queen and converting the ending directly into a forced checkmate.", "confidence_valid": true }, { "pair": "pair_16", "game_a": "game_11", "color_a": "black", "game_b": "game_02", "color_b": "white", "rating_a": 1273, "rating_b": 1575, "gap": 302, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.22, "correct": 1, "brier": 0.0484, "old_estimate_a": 1250, "old_estimate_b": 1450, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Player B displays far superior foundational opening and strategic understanding. Playing a standard Ruy Lopez, Game B White followed sensible development schemes (8.Nd5, 10.d4, 15.Qd3) before collapsing in tactical defense and suffering 25...Rh1#. In contrast, Game A Black plays an unprincipled early queen sortie with Scholar's Mate aspirations (3...Qf6), misses White's hanging piece after 34.Bb6? by retreating with 34...Rdd6, and finally commits a catastrophic back-rank blunder with 40...e3?? walking directly into mate in two (41.Rd8+ Re8 42.Rxe8#). These errors are hallmarks of a much lower rating pool.", "confidence_valid": true }, { "pair": "pair_17", "game_a": "game_13", "color_a": "white", "game_b": "game_16", "color_b": "white", "rating_a": 1126, "rating_b": 1752, "gap": 626, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.15, "correct": 1, "brier": 0.0225, "old_estimate_a": 1200, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Player B exhibits solid fundamentals, comfortably defending against Black's unsound sacrifice 6...Bxf2+ by safely repositioning the king with 8.Rf1 and 9.Kg1. From there, B smoothly coordinates, develops pieces, controls central files, and converts the material advantage cleanly without any tactical oversights. In contrast, Player A plays haphazardly, needlessly giving up pawn structure and falling into a cheap exchange tactic with 17.c3? b6, and later blunders a simple back-rank checkmate in one move with 35.Nxb5?? Rc1#.", "confidence_valid": true }, { "pair": "pair_18", "game_a": "game_24", "color_a": "black", "game_b": "game_12", "color_b": "white", "rating_a": 765, "rating_b": 1877, "gap": 1112, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.08, "correct": 1, "brier": 0.0064, "old_estimate_a": 1150, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Player B demonstrates far superior tactical vision and positional understanding compared to Player A. In Game B, White cleverly won material with 12. Bb5+ followed by 13. Qxd4, then converted with active piece play (22. Bf4+, 25. Be5, 28. Qg8#). In contrast, Player A pushed pawns recklessly on the flanks leaving their king completely vulnerable (7...f6, 8...h5, 9...g5), and finally blundered their queen outright in one move with 22...Qd8?? allowing 23. Bxd8.", "confidence_valid": true }, { "pair": "pair_19", "game_a": "game_17", "color_a": "white", "game_b": "game_10", "color_b": "white", "rating_a": 1514, "rating_b": 2689, "gap": 1175, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.23, "correct": 1, "brier": 0.0529, "old_estimate_a": 1700, "old_estimate_b": 2050, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B exhibits high-level strategic comprehension from both sides in a sophisticated English Opening. White coordinates pieces harmoniously with 11. Rc1, 13. Nd4, and 17. Rfd1, maintaining strong pressure until a tactical oversight with 24. Nxa7? allowed 24...Qc2!. In contrast, Game A features more rudimentary play: White unnecessarily weakens the kingside with 11. g4 and 12. Neg5, misplacing pieces and leaving holes that Black could have exploited. Even though White won Game A (likely by an early resignation or time-out on move 24), Player B's opening sophistication and positional maneuvering reflect a considerably higher established rating.", "confidence_valid": true }, { "pair": "pair_20", "game_a": "game_11", "color_a": "white", "game_b": "game_23", "color_b": "white", "rating_a": 1228, "rating_b": 557, "gap": 671, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.88, "correct": 1, "brier": 0.0144, "old_estimate_a": 1300, "old_estimate_b": 1300, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "Player A demonstrates vastly superior tactical awareness and board vision compared to Player B. In Game A, White develops logically, navigates the opening soundly, and finishes cleanly with an alert mating net (40. Rd5! e3 41. Rd8+). In contrast, Game B features severe tactical blunders by both players. White in Game B hung a piece with 14. Nxf5? (walked into bishop coverage from e7) and later inexplicably hung their queen in broad daylight with 22. Qe5?? (which Black failed to capture with 22...Qg6). The density of basic, one-move blunders in Game B is characteristic of a much lower rating tier.", "confidence_valid": true }, { "pair": "pair_21", "game_a": "game_21", "color_a": "white", "game_b": "game_15", "color_b": "white", "rating_a": 739, "rating_b": 2157, "gap": 1418, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.02, "correct": 1, "brier": 0.0004, "old_estimate_a": 1250, "old_estimate_b": 2200, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White demonstrates beginner-level play, starting with the awkward 2. Qd3 and repeatedly missing simple tactical combinations, allowing 20...Ne3+ winning a bishop on f1 and later blundering an entire rook via 28...Ng4 and 29...Nf2. In contrast, Game B White shows advanced positional and tactical understanding in a theoretical Slav Defense, executing a sharp queen-sac sequence (23. dxc5 Nxc5 24. Rd1 Nxb7 25. Qxc8+!) to reach a favorable endgame, which was then converted into a theoretically precise draw. Target B is far stronger.", "confidence_valid": true }, { "pair": "pair_22", "game_a": "game_04", "color_a": "black", "game_b": "game_13", "color_b": "black", "rating_a": 2718, "rating_b": 1551, "gap": 1167, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.88, "correct": 1, "brier": 0.0144, "old_estimate_a": 1950, "old_estimate_b": 1250, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Player A demonstrates sound opening knowledge in the Caro-Kann Exchange Variation and excellent strategic understanding (16...Ne4, 17...f5, 19...g5), culminating in a flawlessly executed smothered mate combination starting with 24...Ne2+. In contrast, Player B exhibits low-tier opening habits with early, risky queen sorties (3...Qa5+, 5...Qb5, 6...Qxb2) and poor tactical awareness, allowing the devastating 28. Ne8 and 29. Nc7 fork to drop an entire rook. Player B only won because White committed an elementary back-rank blunder on move 35. Player A is far more polished and established.", "confidence_valid": true }, { "pair": "pair_23", "game_a": "game_25", "color_a": "white", "game_b": "game_18", "color_b": "white", "rating_a": 738, "rating_b": 2426, "gap": 1688, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1350, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Player B exhibits solid positional understanding and coherent opening play (2. Nf3, 3. b3, 4. g3, 7. O-O), followed by purposeful piece activity and central play, only collapsing tactically late with 30. Rd1? allowing the decisive ...Re2+. In contrast, Player A plays like a beginner: violating opening principles with 3. Qf3 and an absurd flurry of flank pawn pushes (4. g4, 5. h3, 6. h4, 7. g5, 8. h5), while completely neglecting development and overlooking the elementary fork 13...Nc2+ dropping a full rook. Despite Black later blundering in Game A, Player A's overall grasp of chess fundamentals is vastly inferior to Player B's.", "confidence_valid": true }, { "pair": "pair_24", "game_a": "game_03", "color_a": "white", "game_b": "game_15", "color_b": "black", "rating_a": 1782, "rating_b": 2154, "gap": 372, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.22, "correct": 1, "brier": 0.0484, "old_estimate_a": 1800, "old_estimate_b": 2200, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Player B exhibits substantially higher chess understanding and tactical refinement. In an advanced Slav/a4-b4 line, Black handled Queen's-side piece coordination cleanly and executed the tactical liquidation 24...Nxb7! 25.Qxc8+ Nxc8 26.Rxd5 exd5, transitioning into a slightly worse but holdable endgame that was drawn with textbook king activity and pawn-play technique. In contrast, White in Game A plays an aggressive early h4/Ng5 London attack against a severely underperforming opponent who collapsed with 15...Rd8? and 16...Kf7?. While White efficiently converted the material advantage, the level of opposition and strategic complexity in Game B reflects a significantly higher established rating.", "confidence_valid": true }, { "pair": "pair_25", "game_a": "game_19", "color_a": "black", "game_b": "game_07", "color_b": "white", "rating_a": 2474, "rating_b": 1862, "gap": 612, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.72, "correct": 1, "brier": 0.07840000000000001, "old_estimate_a": 1750, "old_estimate_b": 1550, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Player A (Black) demonstrated a solid grasp of structural and positional nuances in an offbeat Queen's Gambit Declined endgame, handling doubled f-pawns and coordinating pieces reasonably well until collapsing under heavy pressure late in the game (33...Kd8? and 34...Nxf5?). In contrast, Player B (White) adopted a timid, passive setup with little initiative and committed a severe one-move tactical blunder with 21. Bf3?, completely missing the direct fork 21...Bc2, which immediately dropped a full exchange and prompted resignation on move 24.", "confidence_valid": true } ] }, { "model": "google/gemini-3.1-pro-preview", "correct": 21, "n": 25, "accuracy": 84.0, "baseline_accuracy": 92, "invalid_confidence": 0, "consistent_reversals": 4, "reversals": [ { "pair": "pair_09_reverse", "original": "pair_09", "prediction": { "stronger": "B", "probability_a_stronger": 0.1, "explanation": "Black in Game A misses a completely free piece with the inexplicable 16...Kf7?? (instead of Rxd6 or Qxd6) and struggles with passive play throughout (9...Qc8). Black in Game B demonstrates much higher tactical awareness, winning a pawn early with the pin 10...Nxd4 and launching a complex sequence with the fork 21...c4 and double attack 23...Rd4. Although Black B ultimately blunders a back-rank mate (27...Nxe3?? allowing 28.Qg8+), their overall strategic and tactical play before that mistake suggests a much higher rating than Black A's beginner-level blunders." }, "consistent": false }, { "pair": "pair_15_reverse", "original": "pair_15", "prediction": { "stronger": "A", "probability_a_stronger": 0.95, "explanation": "White A shows excellent understanding of attacking principles in the Pirc Defense, culminating in a brilliant, calculated Queen sacrifice (26. Rxh7! and 29. Qxh8+!) to weave a mating net. In contrast, White B plays the opening poorly (wasting tempi with Bc4-d3-e2-d3) and blunders a piece with 16. Be4?? Nxg2+. White A's cohesive, deeply tactical play demonstrates a much higher playing strength than White B's aimless and blunder-prone performance." }, "consistent": true }, { "pair": "pair_13_reverse", "original": "pair_13", "prediction": { "stronger": "A", "probability_a_stronger": 0.95, "explanation": "White A shows solid opening knowledge in the Caro-Kann Exchange, developing logically and launching a queenside attack (14. b4, 20. a5) before falling victim to a famous and complex smothered mate combination by Black. White B, in contrast, plays a highly dubious 2. Qf3 and 3. g4, allows their king to be chased to e3 by move 9, and survives only because Black blunders pieces away. White A's positional understanding and structural play are vastly superior to White B's chaotic, beginner-level moves, making White A clearly the stronger player." }, "consistent": true }, { "pair": "pair_12_reverse", "original": "pair_12", "prediction": { "stronger": "B", "probability_a_stronger": 0.3, "explanation": "Black B demonstrates impressive tactical vision and calculation in a sharp Sicilian, punishing White with complex sequences like 15... Nf4, 16... Nxg2+, and 17... fxe4, culminating in a beautiful mating net (27... Be4+). Black A also plays a good game, systematically dismantling White after exploiting the weak 18. Qe3 with Ncxb3, but White A's opening play was extremely bizarre and passive (6. Bb1, 8. h4). Black B's ability to navigate a more complex middlegame and execute advanced combinations suggests a higher rating." }, "consistent": true }, { "pair": "pair_18_reverse", "original": "pair_18", "prediction": { "stronger": "A", "probability_a_stronger": 0.95, "explanation": "White in Game A showed decent tactical awareness, playing 12. Bb5+ to recover a piece and eventually spotting the winning back-rank mate with 27. Bxh8 Nxe3 28. Qg8+. In contrast, Black in Game B played a very dubious, pawn-pushing opening (7... f6, 8... h5, 9... g5), completely neglecting development, and ultimately blundered their queen on d8 to a simple bishop attack. Target A's grasp of fundamentals and tactics is far superior to Target B's." }, "consistent": true } ], "rows": [ { "pair": "pair_01", "game_a": "game_16", "color_a": "black", "game_b": "game_19", "color_b": "white", "rating_a": 1569, "rating_b": 2668, "gap": 1099, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1100, "old_estimate_b": 2000, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Black in Game A plays a highly dubious opening with 2... e6 and unprovokedly sacrifices a bishop for no compensation with 6... Bxf2+, showing a lack of basic tactical and positional understanding. In contrast, White in Game B demonstrates high-level positional play in a QGD Exchange, strategically damaging Black's pawn structure, maneuvering knights effectively (Nh3-f4-g3), and executing a precise endgame conversion with the tactical sequence 34. Ra7 and 35. Ra8+. White B is clearly the stronger, more advanced player.", "confidence_valid": true }, { "pair": "pair_02", "game_a": "game_01", "color_a": "black", "game_b": "game_24", "color_b": "white", "rating_a": 1159, "rating_b": 762, "gap": 397, "gap_band": "200-499", "low_band": "one<1000", "gold": "A", "choice": "B", "probability_a": 0.15, "correct": 0, "brier": 0.7224999999999999, "old_estimate_a": 1200, "old_estimate_b": 900, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Black in Game A demonstrates very low tactical awareness, blundering their queen to a basic one-move knight fork with 15... Nxg5?? 16. Nf6+. Black only wins because White commits an equally egregious queen blunder (40. Qb8+??). White in Game B, despite some awkward maneuvering like 11. Nh5, shows much better tactical vision by finding resources like 17. Nxf6+ to escape a trap and 20. Nb6 to win material. White B's ability to spot and execute these tactics suggests a higher rating than Black A's beginner-level blunders.", "confidence_valid": true }, { "pair": "pair_03", "game_a": "game_01", "color_a": "white", "game_b": "game_06", "color_b": "white", "rating_a": 1363, "rating_b": 1112, "gap": 251, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.7, "correct": 1, "brier": 0.09000000000000002, "old_estimate_a": 1250, "old_estimate_b": 1200, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Target A played a somewhat standard opening and spotted a nice tactical sequence (15. Nd5 followed by the 16. Nf6+ royal fork) to win the queen, even though they later self-destructed with a horrific endgame blunder (40. Qb8+??). Target B played an extremely bizarre and unprincipled opening setup (4. Bd3, 6. Bb1, 7. Qc2) and was easily outplayed tactically by Black's knights, dropping an exchange to a simple fork (18... Ncxb3) and getting checkmated. A's grasp of opening fundamentals and early tactical awareness suggest a higher rating than B's beginner-like maneuvering.", "confidence_valid": true }, { "pair": "pair_04", "game_a": "game_22", "color_a": "black", "game_b": "game_08", "color_b": "black", "rating_a": 557, "rating_b": 1791, "gap": 1234, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.02, "correct": 1, "brier": 0.0004, "old_estimate_a": 900, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Black in Game B demonstrates solid, principled play, controlling the center, developing logically, and executing strong tactics such as the 14... c5 fork and the game-winning 24... Nf3+. In stark contrast, Black in Game A plays recklessly and blunders multiple pieces outright, such as hanging a knight with 11... Nb5+ (13. Qxb5) and a bishop with 16... cxd5 (17. Qxc5), before eventually blundering their queen as well. Target B is clearly a much more experienced and competent player.", "confidence_valid": true }, { "pair": "pair_05", "game_a": "game_09", "color_a": "white", "game_b": "game_07", "color_b": "black", "rating_a": 2193, "rating_b": 1593, "gap": 600, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.15, "correct": 0, "brier": 0.7224999999999999, "old_estimate_a": 1950, "old_estimate_b": 1550, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Target A (White in Game A) plays a passive and slightly awkward opening (8. a3, 12. Bd2, 13. Qc1) and is outplayed positionally before blundering material with 21. Qxe2 Bxc3. They are eventually ground down in a lost endgame. Target B (Black in Game B) plays a solid and very well-developed Slav Defense, easily equalizing. They demonstrate excellent tactical awareness by exploiting White's inaccuracies, culminating in the sharp sequence 20... Qg6 and 21... Bc2!, which cleanly wins a rook through an intermediate exchange of queens. Target B's clean, mistake-free play strongly suggests a higher rating.", "confidence_valid": true }, { "pair": "pair_06", "game_a": "game_17", "color_a": "black", "game_b": "game_25", "color_b": "black", "rating_a": 1639, "rating_b": 838, "gap": 801, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.85, "correct": 1, "brier": 0.022500000000000006, "old_estimate_a": 1750, "old_estimate_b": 1150, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Target A (Black in Game A) played a standard Italian Game, demonstrating solid opening principles, logical piece development (...Bg4, ...Nf4), and a good understanding of pawn structure. The game ended abruptly, likely due to a timeout or disconnect, but Black's play was positionally sound. In contrast, Target B (Black in Game B) played a chaotic, beginner-level opening (...Bd7, ...b5, ...Na6, ...Rd8) against an opponent who recklessly pushed kingside pawns early on. Although Black B found some basic tactical tricks like 20...Rxd4+, their overall strategic understanding was poor, culminating in severe blunders and getting checkmated. Target A's solid positional play strongly suggests a higher rating.", "confidence_valid": true }, { "pair": "pair_07", "game_a": "game_14", "color_a": "black", "game_b": "game_08", "color_b": "white", "rating_a": 1188, "rating_b": 1310, "gap": 122, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.65, "correct": 0, "brier": 0.42250000000000004, "old_estimate_a": 1150, "old_estimate_b": 1350, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Target A (Black) played a somewhat passive but solid game, avoiding major tactical errors before eventually capitalizing on White's blatant blunder (30...Rxd3). Target B (White) demonstrated a very passive and disjointed opening setup (1.Nc3, 2.d3, 3.e3, 5.g3), fell for a simple pawn fork (14...c5), and later allowed a devastating knight fork (24.h4 Nf3+). Target A's ability to maintain a solid position and avoid basic tactical oversights suggests a higher rating than Target B's error-prone and excessively passive play.", "confidence_valid": true }, { "pair": "pair_08", "game_a": "game_02", "color_a": "black", "game_b": "game_20", "color_b": "black", "rating_a": 2410, "rating_b": 1903, "gap": 507, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.15, "correct": 0, "brier": 0.7224999999999999, "old_estimate_a": 1650, "old_estimate_b": 2000, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Black B demonstrates advanced opening knowledge (such as the thematic 7...Bh8 to preserve the bishop) and executes a coherent queenside counterattack in a highly complex position, ultimately falling to a brilliant combination by a strong opponent. In contrast, Black A plays a solid game but simply capitalizes on basic blunders from a much weaker White opponent, such as dropping a pawn with 9. c3 Nxe4 and walking into a simple mate-in-one with 25. Bg5. Black B shows a significantly higher level of strategic understanding.", "confidence_valid": true }, { "pair": "pair_09", "game_a": "game_12", "color_a": "black", "game_b": "game_03", "color_b": "black", "rating_a": 1811, "rating_b": 1901, "gap": 90, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.15, "correct": 1, "brier": 0.0225, "old_estimate_a": 1800, "old_estimate_b": 1500, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Black in Game A blunders a full piece early to a basic tactic (12... c6 13. Qxd4) and later walks into a simple back-rank mate (27... Nxe3 28. Qg8+). Black in Game B displays a better positional understanding in the Dutch Defense, completing development and castling before breaking down to a complex knight maneuver by White. Although Black B also drops a piece with 16... Kf7 (missing 16... Rxd6), the overall opening play and resistance suggest a more experienced player than Black A's basic tactical blindness.", "confidence_valid": true }, { "pair": "pair_10", "game_a": "game_23", "color_a": "black", "game_b": "game_09", "color_b": "black", "rating_a": 564, "rating_b": 2177, "gap": 1613, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1050, "old_estimate_b": 2250, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Target B demonstrates a sophisticated understanding of the Benoni structure, executing typical queenside expansion (10... b5) and excellent knight maneuvers (14... Ne5, 15... Nc4). Target B seamlessly converts a tactical advantage into a won endgame with precise technique (32... Qf4+). In contrast, Target A plays a chaotic and unsound game, sacrificing a piece dubiously with 13... Bxh3, and blundering another piece outright with 15... Bh4??. Target B's positional depth and tactical precision heavily outweigh Target A's poor tactical vision.", "confidence_valid": true }, { "pair": "pair_11", "game_a": "game_21", "color_a": "black", "game_b": "game_10", "color_b": "black", "rating_a": 743, "rating_b": 2501, "gap": 1758, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1100, "old_estimate_b": 2150, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B Black demonstrates a solid understanding of positional chess, smoothly navigating a complex Reti/Tarrasch structure and executing a sharp tactical sequence (24... Qc2!). In contrast, Game A Black plays dubiously in the opening (2... e5?!), blunders a pawn with 6... Bf5 allowing 7. Qxb7, and relies on basic knight forks against an equally weak opponent in a chaotic, low-level game. Target B is significantly stronger.", "confidence_valid": true }, { "pair": "pair_12", "game_a": "game_05", "color_a": "black", "game_b": "game_06", "color_b": "black", "rating_a": 1867, "rating_b": 1382, "gap": 485, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.6, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 1550, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Both targets played excellent attacking games against opponents who misplayed the opening. In Game A, Black was extremely sharp, finding tactical shots like 15... Nf4, 16... Nxg2+, and the crushing 24... Qb7! to create mating nets. In Game B, Black also punished White's bizarre setup (6. Bb1) efficiently with knights maneuvering to d4 and c5. However, Game A Black's tactical execution was slightly crisper and more complex, suggesting a higher level of rating.", "confidence_valid": true }, { "pair": "pair_13", "game_a": "game_22", "color_a": "white", "game_b": "game_04", "color_b": "white", "rating_a": 717, "rating_b": 2508, "gap": 1791, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 950, "old_estimate_b": 1950, "baseline_choice": "B", "baseline_credit": 1, "explanation": "White A plays a nonsensical opening (2. Qf3, 3. g4) and exposes their king (9. Ke3), surviving only because Black inexplicably hangs pieces (13. Qxb5, 17. Qxc5). This indicates a beginner level. In contrast, White B plays a theoretical Caro-Kann Exchange, develops logically, and launches a credible queenside attack (14. b4, 20. a5). Although White B falls for a classic smothered mate trap (24...Ne2+ 25.Kh1 Qg1+ 26.Rxg1 Nf2#), their overall positional understanding and structural play are vastly superior to White A.", "confidence_valid": true }, { "pair": "pair_14", "game_a": "game_18", "color_a": "black", "game_b": "game_14", "color_b": "white", "rating_a": 2641, "rating_b": 1249, "gap": 1392, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.95, "correct": 1, "brier": 0.0025000000000000044, "old_estimate_a": 2000, "old_estimate_b": 1100, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Black in Game A demonstrated strong tactical awareness and attacking play, launching a successful kingside attack with moves like 11...h5 and finishing with the crushing sequence starting with 31...Rh2!. In contrast, White in Game B showed a lack of basic board awareness, hanging a piece in the opening with 10. Bxd5 Bxh4 (leaving the h4 knight unprotected) and blundering a bishop at the end with 30. Rg6 Rxd3. Target A is clearly the stronger player.", "confidence_valid": true }, { "pair": "pair_15", "game_a": "game_05", "color_a": "white", "game_b": "game_20", "color_b": "white", "rating_a": 1120, "rating_b": 1932, "gap": 812, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1250, "old_estimate_b": 2150, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Target B (White in Game B) demonstrated a much higher level of play than Target A. White in Game B executed a thematic and well-calculated kingside attack against the Pirc/Modern defense, complete with long castling, a pawn storm (h4-h5), and brilliant sacrifices like 26. Rxh7 and the queen sacrifice 29. Qxh8+, leading to a beautiful checkmate. In contrast, White in Game A struggled with basic opening principles (2. Bc4, 5. Bd3) and blundered a piece to a simple tactic (16. Be4?? Nxg2+), getting completely crushed. The difference in tactical and positional understanding is vast.", "confidence_valid": true }, { "pair": "pair_16", "game_a": "game_11", "color_a": "black", "game_b": "game_02", "color_b": "white", "rating_a": 1273, "rating_b": 1575, "gap": 302, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.4, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 1150, "old_estimate_b": 1300, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Black in Game A plays a very dubious opening (1...e5, 3...Qf6) and ultimately blunders a simple back-rank mate (40...e3??). White in Game B plays a standard Ruy Lopez but hangs pawns (9. c3, 18. h4) and completely misses a mating net (24. Bc2??). Both players show significant tactical weaknesses, but White in Game B exhibits slightly better opening principles than Black's early queen sorties in Game A, suggesting B might be slightly stronger.", "confidence_valid": true }, { "pair": "pair_17", "game_a": "game_13", "color_a": "white", "game_b": "game_16", "color_b": "white", "rating_a": 1126, "rating_b": 1752, "gap": 626, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.1, "correct": 1, "brier": 0.010000000000000002, "old_estimate_a": 1050, "old_estimate_b": 1400, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White demonstrates very low-level play, with bizarre maneuvers like 10. Qa1 and 11. Nb1, dropping an exchange with 16. Rc5 Ndb4 17. c3 b6, and eventually blundering a simple back-rank mate with 35. Nxb5 Rc1#. Game B White, facing an early speculative sacrifice (6...Bxf2+), calmly defends by castling by hand (7. Kxf2, 8. Rf1, 9. Kg1) and efficiently trades down to consolidate the extra piece into a winning endgame. White B's play is fundamentally sound and coordinated, whereas White A plays erratically and blunders severely.", "confidence_valid": true }, { "pair": "pair_18", "game_a": "game_24", "color_a": "black", "game_b": "game_12", "color_b": "white", "rating_a": 765, "rating_b": 1877, "gap": 1112, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 850, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Target A (Black in Game A) demonstrates extremely poor fundamentals, making numerous weakening pawn moves in the opening (7...f6, 8...h5, 9...g5) while neglecting development, and ultimately blunders their queen with 22...Qd8??. Target B (White in Game B) plays logically and demonstrates strong tactical awareness, such as recovering a piece with 12. Bb5+ and 13. Qxd4, creating attacking chances with 16. Qa4, exploiting a fork with 25. Be5, and finding a decisive back-rank combination with 28. Qg8+. Target B's play is vastly superior.", "confidence_valid": true }, { "pair": "pair_19", "game_a": "game_17", "color_a": "white", "game_b": "game_10", "color_b": "white", "rating_a": 1514, "rating_b": 2689, "gap": 1175, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.2, "correct": 1, "brier": 0.04000000000000001, "old_estimate_a": 1800, "old_estimate_b": 2000, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B White demonstrates a sophisticated understanding of positional play in a Reti/Catalan system, developing harmoniously and maneuvering well (13. Nd4, 15. Qf4) before a tactical oversight at the end (24. Nxa7 allowing Qc2). In contrast, Game A White plays a less refined opening, pointlessly wasting tempi (12. Neg5 h6 13. Ne4) and unnecessarily weakening the kingside with 11. g4 to break a pin. Therefore, Game B White likely has a higher rating despite losing.", "confidence_valid": true }, { "pair": "pair_20", "game_a": "game_11", "color_a": "white", "game_b": "game_23", "color_b": "white", "rating_a": 1228, "rating_b": 557, "gap": 671, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.85, "correct": 1, "brier": 0.022500000000000006, "old_estimate_a": 1200, "old_estimate_b": 1100, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Target A demonstrates a better understanding of development and tactics, finding nice moves like 7. Ne4 to exploit the opponent's early Queen sortie, though they do hang a pawn later with 10. c3. Target B plays the opening decently but then makes several massive blunders that both players fail to notice. For instance, in Game B, White plays 15. Nxf5 hanging the knight to Black's rook on f8, and later 22. Qe5?? hanging the Queen outright to Black's Queen on e6, which Black completely ignores with 22... Qg6. This mutual blindness to hanging pieces suggests Target B is a beginner, making Target A the stronger player.", "confidence_valid": true }, { "pair": "pair_21", "game_a": "game_21", "color_a": "white", "game_b": "game_15", "color_b": "white", "rating_a": 739, "rating_b": 2157, "gap": 1418, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.02, "correct": 1, "brier": 0.0004, "old_estimate_a": 1000, "old_estimate_b": 2250, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Target A (White) played a highly irregular, beginner-level opening with 2. Qd3 and later completely collapsed tactically, blundering a knight fork (19. a3 Nc2+) and a full rook (29. exd5 Nf2). In contrast, Target B (White) navigated a theoretical Slav Defense (6. Nb1, 7. Bxc4) with deep positional understanding, solid piece coordination, and competent endgame technique to secure a draw in a pawn endgame. Target B's play demonstrates a vastly higher level of tactical and strategic maturity.", "confidence_valid": true }, { "pair": "pair_22", "game_a": "game_04", "color_a": "black", "game_b": "game_13", "color_b": "black", "rating_a": 2718, "rating_b": 1551, "gap": 1167, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.95, "correct": 1, "brier": 0.0025000000000000044, "old_estimate_a": 2150, "old_estimate_b": 1150, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Target A played a principled Caro-Kann, castled queenside, and executed a beautiful smothered mate sequence starting with 22... Nd4! Target A demonstrated strong positional understanding and advanced tactical vision. In contrast, Target B played a dubious opening (3... Qa5+), went pawn-grabbing early (6... Qxb2), and blundered material (28... Re6? leading to 31. Rxc6). Target B only won because White blundered a basic back-rank mate at the end (35. Nxb5?? Rc1#). Target A's cohesive, high-level play heavily outclasses Target B's erratic and opportunistic style.", "confidence_valid": true }, { "pair": "pair_23", "game_a": "game_25", "color_a": "white", "game_b": "game_18", "color_b": "white", "rating_a": 738, "rating_b": 2426, "gap": 1688, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1100, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White shows a lack of opening fundamentals with bizarre early moves like 3. Qf3, 4. g4, and pushing flank pawns to h5 and g5, later falling for a simple knight fork with 13... Nc2+. In contrast, Game B White demonstrates a solid understanding of opening principles, piece development, and structural play against the Sicilian Defense (e.g., fianchetto, challenging the center with d4). Although Game B White blunders at the end with 32. Qxd5 allowing a mating net, their overall positional understanding and tactical awareness throughout the game indicate a significantly higher rating than Game A White.", "confidence_valid": true }, { "pair": "pair_24", "game_a": "game_03", "color_a": "white", "game_b": "game_15", "color_b": "black", "rating_a": 1782, "rating_b": 2154, "gap": 372, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.35, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1800, "old_estimate_b": 2250, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Target B (Black) demonstrates deep positional and tactical understanding in the Slav Defense, navigating a highly complex sequence from moves 22-26 that results in an equal endgame. The thematic 22...c5 break and the ensuing mass exchanges (24...Nxb7 25. Qxc8+ Nxc8 26. Rxd5 exd5) show precise calculation. Target A (White) plays a decent attacking game in the Dutch, punishing Black's blunders (such as 15...Rd8? allowing 16. Bxd6), but early moves like 8.h4 and 11.Nh3 are somewhat committal and less refined than B's play. B's ability to maintain equality in a sharp theoretical variation suggests a higher rating.", "confidence_valid": true }, { "pair": "pair_25", "game_a": "game_19", "color_a": "black", "game_b": "game_07", "color_b": "white", "rating_a": 2474, "rating_b": 1862, "gap": 612, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.75, "correct": 1, "brier": 0.0625, "old_estimate_a": 1850, "old_estimate_b": 1400, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Target A (Black) played a solid opening and navigated a complex endgame reasonably well until falling for a multi-step tactical trap (33... Kd8 34. Ra7 Nxf5 35. Ra8+) which lost the rook. Target B (White) played a rather passive opening (3. e3, 5. Nbd2) and then completely missed a basic one-move double attack with 21. Bf3? Bc2, forking the Queen and Rook. Target A demonstrated deeper strategic understanding and positional play compared to Target B's passive play and elementary tactical oversight.", "confidence_valid": true } ] }, { "model": "google/gemini-3.5-flash", "correct": 21, "n": 25, "accuracy": 84.0, "baseline_accuracy": 80.0, "invalid_confidence": 0, "consistent_reversals": 3, "reversals": [ { "pair": "pair_09_reverse", "original": "pair_09", "prediction": { "stronger": "B", "probability_a_stronger": 0.15, "explanation": "Game B's Black, despite an early tactical oversight (allowing 12. Bb5+ and 13. Qxd4), demonstrated a much higher level of tactical awareness and resourcefulness. Black's counterplay with 21... c4!, 23... Rd4!, and 25... Rd2! created severe threats, showing strong dynamic calculation before ultimately succumbing to a back-rank mate. In contrast, Game A's Black showed poor tactical vision and passive play. Black allowed 15. Ng6, and after 15... Rd8, missed that 16. Bxd6! left Black helpless due to the Ne7+ fork threat (since 16... Rxd6 17. Ne7+ wins the queen). Black's subsequent defense was weak, leading to a slow, one-sided defeat." }, "consistent": false }, { "pair": "pair_15_reverse", "original": "pair_15", "prediction": { "stronger": "A", "probability_a_stronger": 0.95, "explanation": "Game A White demonstrates strong understanding of the Modern/Pirc, launching a well-coordinated kingside attack (9. g4, 11. h4, 13. hxg6) and executing a precise, thematic mating sequence beginning with the rook sacrifice 26. Rxh7! and subsequent queen sacrifice 29. Qxh8+. In contrast, Game B White shows poor positional and tactical awareness, misplacing pieces early (2. Bc4, 5. Bd3), blundering material to simple threats (15... Nf4), and collapsing quickly into a mating net. Game A White played at a much higher level." }, "consistent": true }, { "pair": "pair_13_reverse", "original": "pair_13", "prediction": { "stronger": "A", "probability_a_stronger": 0.95, "explanation": "In Game A, White plays a standard Exchange Caro-Kann with coherent positional plans, including a queenside pawn storm (b4, a4) and a strong knight on c5. Although White overlooks a tactical smothered mate sequence starting with 24...Ne2+, the play prior shows solid intermediate understanding. In contrast, Game B White shows a severe lack of opening principles, playing 2. Qf3 and 3. g4, leading to the king being exposed on e3 by move 9. Game B is highly chaotic and typical of beginner-level play, making White A the much stronger and higher-rated player." }, "consistent": true }, { "pair": "pair_12_reverse", "original": "pair_12", "prediction": { "stronger": "A", "probability_a_stronger": 0.6, "explanation": "Game A Black demonstrated superior positional understanding and precise calculation. The sequence starting with 12...Bg4, 13...Nd4, 14...Nd7, and 17...Nc5 shows a sophisticated plan to target White's weak queenside, executed flawlessly with the tactical shot 18...Ncxb3! winning material. Game B Black also played strongly but showed slightly less precision, missing the immediate win of a pawn with 13...Nxc3 (playing 13...Nc6 instead), and opting for the unnecessarily complicated 16...Nxg2+ instead of the simpler 16...fxe4. Game A Black's play was exceptionally clean and structured." }, "consistent": false }, { "pair": "pair_18_reverse", "original": "pair_18", "prediction": { "stronger": "A", "probability_a_stronger": 0.98, "explanation": "In Game A, White (target) plays a clean game, finding the tactical refutation 12. Bb5+ and 13. Qxd4 to win a piece, and handles a complex endgame well with 22. Bf4+ and 26. Re3, leading to mate. In contrast, Black (target) in Game B plays extremely weak, non-developing pawn moves in the opening (such as 7... f6 and 9... g5). Black then displays very poor tactical vision, repeatedly missing a simple winning queen fork with ...Nf3+ from move 18 onwards, and ultimately blunders their own queen with 22... Qd8??." }, "consistent": true } ], "rows": [ { "pair": "pair_01", "game_a": "game_16", "color_a": "black", "game_b": "game_19", "color_b": "white", "rating_a": 1569, "rating_b": 2668, "gap": 1099, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.03, "correct": 1, "brier": 0.0009, "old_estimate_a": 1300, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "In Game A, Black plays a dubious opening gambit (2... e6) and an unsound piece sacrifice (6... Bxf2+?) that leaves them down a piece for no compensation. Black's play lacks tactical and positional foundation. In contrast, Game B (White) demonstrates high-level positional and endgame play. White expertly exploits Black's pawn weaknesses, maneuvers the knight efficiently (Nh3-f4-g3-f5), and executes a precise rook endgame. White secures the win with a neat tactical sequence starting with 34. Ra7 and 35. Ra8+, winning Black's rook. Target B is vastly more accomplished.", "confidence_valid": true }, { "pair": "pair_02", "game_a": "game_01", "color_a": "black", "game_b": "game_24", "color_b": "white", "rating_a": 1159, "rating_b": 762, "gap": 397, "gap_band": "200-499", "low_band": "one<1000", "gold": "A", "choice": "B", "probability_a": 0.25, "correct": 0, "brier": 0.5625, "old_estimate_a": 1150, "old_estimate_b": 1250, "baseline_choice": "B", "baseline_credit": 0, "explanation": "In Game A, Black (Target A) committed a severe tactical oversight on move 15, playing 15... Nxg5, which completely ignored the threat on f6 and allowed the simple knight fork 16. Nf6+ winning the queen. In Game B, White (Target B) displayed far better tactical awareness and solid positional understanding. White successfully navigated Black's kingside threats, found a resource with 17. Nxf6+ to save the queen, executed a strong fork with 20. Nb6, and promptly punished Black's queen blunder with 23. Bxd8. Target B's play suggests a much higher rating.", "confidence_valid": true }, { "pair": "pair_03", "game_a": "game_01", "color_a": "white", "game_b": "game_06", "color_b": "white", "rating_a": 1363, "rating_b": 1112, "gap": 251, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.75, "correct": 1, "brier": 0.0625, "old_estimate_a": 1250, "old_estimate_b": 1150, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A White showed reasonable tactical awareness, capitalizing on Black's mistake with the fork 16. Nf6+ to win the queen. Despite throwing away a completely winning position with the colossal blunder 40. Qb8+?? (likely a mouse slip or time-trouble hallucination), their play prior was competent. In contrast, Game B White showed poor fundamental understanding, wasting tempi with passive moves like 4. Bd3 and 6. Bb1. Game B White failed to develop effectively, lost material easily, and was quickly mated on f2 with no counterplay, suggesting a lower playing strength.", "confidence_valid": true }, { "pair": "pair_04", "game_a": "game_22", "color_a": "black", "game_b": "game_08", "color_b": "black", "rating_a": 557, "rating_b": 1791, "gap": 1234, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1000, "old_estimate_b": 1450, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B Black demonstrates solid positional play, sound development, and sharp tactical awareness. Black exploits a pin with 7... Bg4, spots the tactical piece win 14... c5, and cleanly converts the advantage, finishing with the fork 24... Nf3+. In contrast, Game A Black plays an chaotic game, starting with an unsound sacrifice 6... Nxf2, followed by numerous positional errors, and concluding with a catastrophic blunder of the queen with 22... Qxg1+.", "confidence_valid": true }, { "pair": "pair_05", "game_a": "game_09", "color_a": "white", "game_b": "game_07", "color_b": "black", "rating_a": 2193, "rating_b": 1593, "gap": 600, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.2, "correct": 0, "brier": 0.6400000000000001, "old_estimate_a": 1550, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game B Black showed excellent positional play and tactical alertness in a Slav Defense. Black equalized smoothly, then launched a decisive tactical sequence with 20...Qg6 (threatening mate) and 21...Bc2, utilizing a fork on White's queen and rook to win a full rook. Conversely, Game A White played passively in the opening and middlegame (such as 15. Kh2 and 16. Rb1), allowed Black to dominate the queenside, blundered a pawn on a3, and defended poorly, ultimately getting mated on the back rank.", "confidence_valid": true }, { "pair": "pair_06", "game_a": "game_17", "color_a": "black", "game_b": "game_25", "color_b": "black", "rating_a": 1639, "rating_b": 838, "gap": 801, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.85, "correct": 1, "brier": 0.022500000000000006, "old_estimate_a": 1650, "old_estimate_b": 1050, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Black in Game A plays standard, principled opening moves, castling early and maintaining a solid positional structure before a final blunder with 24...c6, which hangs the e5-knight. In contrast, Black in Game B plays extremely erratically from the start, choosing bizarre moves like 2...Bd7 and 7...Rd8. Despite capturing a rook after 13...Nc2+, Black displays poor board vision, blundering back material with 24...Rxc3+ and 32...Nxf5, leading to a quick checkmate. Black in Game A shows much higher baseline chess competence.", "confidence_valid": true }, { "pair": "pair_07", "game_a": "game_14", "color_a": "black", "game_b": "game_08", "color_b": "white", "rating_a": 1188, "rating_b": 1310, "gap": 122, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.75, "correct": 0, "brier": 0.5625, "old_estimate_a": 1200, "old_estimate_b": 1200, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "In Game A, Black (Target A) displayed reasonable tactical awareness, capitalizing on White's mistake to win a knight with 10... Bxh4 and safely converting the material advantage to a win. In contrast, in Game B, White (Target B) played a very passive opening and exhibited poor tactical vision. Target B overlooked a simple pawn fork, losing a piece to 14... c5, and later allowed a devastating pin with 20. Nf3 Bg4 and 21... Ng5, leading to a quick collapse. Target A's play was significantly more solid and tactically alert.", "confidence_valid": true }, { "pair": "pair_08", "game_a": "game_02", "color_a": "black", "game_b": "game_20", "color_b": "black", "rating_a": 2410, "rating_b": 1903, "gap": 507, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.65, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1500, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 0, "explanation": "In Game A, Black plays a highly logical Ruy Lopez, quickly punishing White's inaccuracy. Black coordinates pieces excellently with 13...Na5 and 14...Nc4, wins a pawn cleanly with 18...Bxh4, and executes a swift mating attack with 24...Rxh3 and 25...Rh1#. In Game B, Black's passive Modern Defense setup (such as 6...Re8) allows White to build a quick kingside storm. Black fails to generate effective counterplay on the queenside, defensive maneuvers like 18...Nf8 are too slow, and Black ultimately collapses to a straightforward sacrifice on h7. Target A exhibits superior tactical alertness and opening preparation.", "confidence_valid": true }, { "pair": "pair_09", "game_a": "game_12", "color_a": "black", "game_b": "game_03", "color_b": "black", "rating_a": 1811, "rating_b": 1901, "gap": 90, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.3, "correct": 1, "brier": 0.09, "old_estimate_a": 1250, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black exhibits weak tactical vision. They blunder a piece early to a simple two-ply combination (10... Nxd4? 11. Nxd4 Bxd4 12. Bb5+ and 13. Qxd4) and later succumb to a basic back-rank mate (26... Nxc4? 27. Bxh8 Nxe3 28. Qg8+). In contrast, Game B Black demonstrates superior opening knowledge and positional understanding in a complex Dutch Defense setup. Although Game B Black also makes a tactical oversight on move 15 (15... Rd8? allowing 16. Bxd6 with a Ne7+ fork threat), their overall strategic play and standard developmental plans suggest a significantly higher established rating than Game A Black.", "confidence_valid": true }, { "pair": "pair_10", "game_a": "game_23", "color_a": "black", "game_b": "game_09", "color_b": "black", "rating_a": 564, "rating_b": 2177, "gap": 1613, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1150, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "In Game A, Black shows poor tactical and positional awareness, dropping a pawn with 11...e5? and then sacrificing a bishop unsoundly with 13...Bxh3?. In contrast, Game B's Black plays a highly coherent Benoni setup, executing a thematic queenside expansion (10...b5). Black shows great positional understanding, wins a pawn cleanly with 21...Bxc3 and 22...Qxa3, and converts the endgame flawlessly to create a mating net with 41...f3. Game B's Black is clearly the more established and stronger player.", "confidence_valid": true }, { "pair": "pair_11", "game_a": "game_21", "color_a": "black", "game_b": "game_10", "color_b": "black", "rating_a": 743, "rating_b": 2501, "gap": 1758, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1200, "old_estimate_b": 2000, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A features a chaotic, lower-rated game. Black plays a highly questionable opening with 2...e5 and 3...Qe7, which blocks development, and the game proceeds with numerous tactical and positional inconsistencies from both sides. In contrast, Game B shows a high level of theoretical understanding and positional control from Black. Black navigates a standard English Defence setup flawlessly, developing logically, maintaining a solid pawn structure, and capitalizing on White's inaccuracies with sharp tactical precision starting with 24...Qc2, 25...Ba6, and 27...Ne8, forcing an immediate resignation. Player B is clearly the more established and stronger player.", "confidence_valid": true }, { "pair": "pair_12", "game_a": "game_05", "color_a": "black", "game_b": "game_06", "color_b": "black", "rating_a": 1867, "rating_b": 1382, "gap": 485, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.7, "correct": 1, "brier": 0.09000000000000002, "old_estimate_a": 1450, "old_estimate_b": 1450, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "In Game A, Black demonstrates excellent opening understanding with an active Sicilian setup, followed by precise, energetic tactical calculation. Moves like 15...Nf4, the tactical sequence 16...Nxg2+ and 17...fxe4, followed by 20...Nb4! show high-level tactical vision and a direct, punishing attack. In Game B, Black plays a somewhat passive and disjointed opening, including the repetitive maneuver 3...Bd7, 9...Bf5, and 10...Bd7. While Game B's Black successfully exploits White's passive play later on, the overall cohesion, opening preparation, and tactical sharpness of Game A's Black suggest a significantly higher established rating.", "confidence_valid": true }, { "pair": "pair_13", "game_a": "game_22", "color_a": "white", "game_b": "game_04", "color_b": "white", "rating_a": 717, "rating_b": 2508, "gap": 1791, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.03, "correct": 1, "brier": 0.0009, "old_estimate_a": 1050, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White plays highly irregular, low-level chess, with an early queen excursion (2. Qf3), a premature pawn storm (3. g4), and walking their king to e3 by move 9. Both players commit severe errors. Conversely, Game B White demonstrates solid opening and positional understanding in the Caro-Kann Exchange, developing logically (4. Bd3, 6. Ne2) and organizing a queenside attack (14. b4, 20. a5). Although Game B White ultimately misses a tactical smothered mate sequence starting with 24...Ne2+, their level of play prior to this is vastly superior, indicating a much higher established rating.", "confidence_valid": true }, { "pair": "pair_14", "game_a": "game_18", "color_a": "black", "game_b": "game_14", "color_b": "white", "rating_a": 2641, "rating_b": 1249, "gap": 1392, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.95, "correct": 1, "brier": 0.0025000000000000044, "old_estimate_a": 1850, "old_estimate_b": 1150, "baseline_choice": "A", "baseline_credit": 1, "explanation": "In Game A, Black demonstrated high-level play, employing a strong kingside attack (11...h5, 14...h4, 16...Bh3) and maintaining excellent piece activity despite losing castling rights. Black's tactical execution was sharp, culminating in a decisive mating net (29...Re8, 30...Re2+, and 32...Qh3+). In contrast, Game B White showed poor tactical awareness, misplacing a knight with 8. Nh4, which allowed 10...Bxh4, and later hanging a bishop with 30. Rg6?? Rxd3. Black in Game A is clearly the much stronger player.", "confidence_valid": true }, { "pair": "pair_15", "game_a": "game_05", "color_a": "white", "game_b": "game_20", "color_b": "white", "rating_a": 1120, "rating_b": 1932, "gap": 812, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.01, "correct": 1, "brier": 0.0001, "old_estimate_a": 1150, "old_estimate_b": 1950, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White shows a low level of play, making severe tactical blunders such as 16. Be4? (allowing 16... Nxg2+ and losing central control), 23. Nd4? (hanging the h1-rook), and 27. Qxd3?? (hanging the queen to a simple discovery). In contrast, Game B White exhibits advanced positional and attacking understanding in a Pirc/Modern structure, orchestrating a high-level kingside assault (7. Bh6, 11. h4, 22. Qh2). Game B White executes a beautiful, calculation-heavy combination starting with 25. Rxh7! and the queen sacrifice 28. Qxh8+! to transition into a winning endgame, finishing with an elegant mating net.", "confidence_valid": true }, { "pair": "pair_16", "game_a": "game_11", "color_a": "black", "game_b": "game_02", "color_b": "white", "rating_a": 1273, "rating_b": 1575, "gap": 302, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.4, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 1150, "old_estimate_b": 1350, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black plays the Englund Gambit (1...e5) and follows with naive, beginner-like early queen development (3...Qf6), which strongly suggests a lower established rating. Although Game A Black later coordinates their pieces reasonably well, they blunder a basic mate in two with 40...e3??. Game B White plays a highly standard and theoretical Ruy Lopez opening up to move 8, suggesting a more established understanding of chess principles, even though they suffer a tactical collapse, blundering the e4-pawn on move 9, the h4-pawn on move 18, and mate in one with 25. Bg5??. Game B White's opening competence makes them likely the higher-rated player overall.", "confidence_valid": true }, { "pair": "pair_17", "game_a": "game_13", "color_a": "white", "game_b": "game_16", "color_b": "white", "rating_a": 1126, "rating_b": 1752, "gap": 626, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.15, "correct": 1, "brier": 0.0225, "old_estimate_a": 1000, "old_estimate_b": 1500, "baseline_choice": "B", "baseline_credit": 1, "explanation": "White in Game B demonstrated significantly better tactical and positional awareness than White in Game A. In Game A, White committed several large tactical errors, including 17. c3? (which allowed a knight fork) and the final blunder 35. Nxb5??, which completely overlooked a simple back-rank checkmate. In contrast, White in Game B responded calmly to an early sacrifice on f2, consolidated their material advantage, and converted the game smoothly without giving Black any counterplay.", "confidence_valid": true }, { "pair": "pair_18", "game_a": "game_24", "color_a": "black", "game_b": "game_12", "color_b": "white", "rating_a": 765, "rating_b": 1877, "gap": 1112, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.1, "correct": 1, "brier": 0.010000000000000002, "old_estimate_a": 1150, "old_estimate_b": 1450, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Target B (Game B, White) demonstrated solid tactical awareness. They quickly capitalized on Black's mistake with the combination 12. Bb5+ and 13. Qxd4 to win a piece, and converted the game cleanly, finding the mating sequence starting with 28. Qg8+. Conversely, Target A (Game A, Black) exhibited poor board vision. They missed a simple, winning royal fork with 18...Nf3+ (which would have won White's queen) and later blundered their own queen in a single move with 22...Qd8??. Target B is clearly the stronger, more established player.", "confidence_valid": true }, { "pair": "pair_19", "game_a": "game_17", "color_a": "white", "game_b": "game_10", "color_b": "white", "rating_a": 1514, "rating_b": 2689, "gap": 1175, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.58, "correct": 0, "brier": 0.3364, "old_estimate_a": 1700, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White played a solid, blunder-free game. While the opening was relatively simple, White maintained active pieces, coordinated well, and capitalized on Black's late-game errors to secure a win. In contrast, Game B White displayed good positional understanding in an English Opening but suffered a severe tactical collapse. 24. Nxa7?? was a decisive blunder that overlooked 24...Qc2, which forks the b2-bishop and f1-rook and immediately loses the game. Player A's tactical consistency and lack of critical errors suggest a more reliable established playing strength.", "confidence_valid": true }, { "pair": "pair_20", "game_a": "game_11", "color_a": "white", "game_b": "game_23", "color_b": "white", "rating_a": 1228, "rating_b": 557, "gap": 671, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.65, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1200, "old_estimate_b": 1250, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game A White showed better overall coordination and successfully executed a back-rank mate with 41. Rd8+ and 42. Rxe8#. Although White made errors like hanging a bishop with 14. Bc4 and 34. Bb6, the general structure of the game was reasonable. In contrast, Game B White committed an egregious one-move blunder by playing 22. Qe5, which hung the queen directly to Black's queen on e6 (which Black incredibly missed with 22... Qg6). This level of basic tactical oversight suggests Game B is of a lower rating band than Game A.", "confidence_valid": true }, { "pair": "pair_21", "game_a": "game_21", "color_a": "white", "game_b": "game_15", "color_b": "white", "rating_a": 739, "rating_b": 2157, "gap": 1418, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.02, "correct": 1, "brier": 0.0004, "old_estimate_a": 1100, "old_estimate_b": 2000, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White shows low-level play, starting with the bizarre 2. Qd3, and subsequently blundering a full piece with 21. Kd2? allowing 21... Nxf1+. In contrast, Game B White exhibits sophisticated positional understanding in a main-line Slav Defense, managing complex queenless middlegame maneuvering (e.g., 14. Nc4, 18. N3a5) and demonstrating precise endgame technique to secure a draw. Game B White is vastly superior in tactical awareness and strategic planning.", "confidence_valid": true }, { "pair": "pair_22", "game_a": "game_04", "color_a": "black", "game_b": "game_13", "color_b": "black", "rating_a": 2718, "rating_b": 1551, "gap": 1167, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.95, "correct": 1, "brier": 0.0025000000000000044, "old_estimate_a": 1900, "old_estimate_b": 1050, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black demonstrates excellent positional understanding and high tactical sharpness, playing a solid Caro-Kann and executing a flawless smothered mate sequence (24...Ne2+ 25.Kh1 Qg1+ 26.Rxg1 Nf2#). In contrast, Game B Black plays an erratic opening with early queen moves (3...Qa5+, 5...Qb5) and exhibits weak tactical vision, missing a simple winning queen fork with 17...Nc2+ after 17.c3, instead playing 17...b6. Game B only ends because White blunders into a simple back-rank mate with 35.Nxb5?? Rc1#.", "confidence_valid": true }, { "pair": "pair_23", "game_a": "game_25", "color_a": "white", "game_b": "game_18", "color_b": "white", "rating_a": 738, "rating_b": 2426, "gap": 1688, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1100, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "In Game A, White's opening play is very weak, including premature queen moves (3. Qf3) and random pawn pushes (4. g4, 6. h4), quickly blundering a rook to a basic fork with 13. Nh3 Nc2+. In contrast, Game B's White plays a coherent and logical game, employing a recognized sideline (3. b3) against the Sicilian, developing pieces harmoniously (7. O-O, 10. Re1+), and executing sound plans. While Game B's White eventually blunders the king's safety with 30. Rd1?, their overall strategic and tactical competence is vastly superior to the beginner-level play shown by White in Game A.", "confidence_valid": true }, { "pair": "pair_24", "game_a": "game_03", "color_a": "white", "game_b": "game_15", "color_b": "black", "rating_a": 1782, "rating_b": 2154, "gap": 372, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.15, "correct": 1, "brier": 0.0225, "old_estimate_a": 1850, "old_estimate_b": 2000, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B (Black) demonstrates a much higher level of chess understanding and technical skill. Black navigates a sharp, theoretical Slav Defense (3...dxc4, 5...b4), coordinates pieces accurately in a queenless middlegame (15...Qd5, 16...Rxa4), and calculates a pawn-down minor piece endgame into a precise drawn king-and-pawn endgame (39...Bxc5, 48...Kd4). In contrast, Game A (White) plays a superficial opening with early flank attacks (8. h4, 11. Nh3) and wins primarily due to Black's major tactical blunder (16...Kf7?? allowing 17. Ne5+). Game B's positional accuracy and endgame precision suggest a significantly higher established rating.", "confidence_valid": true }, { "pair": "pair_25", "game_a": "game_19", "color_a": "black", "game_b": "game_07", "color_b": "white", "rating_a": 2474, "rating_b": 1862, "gap": 612, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.75, "correct": 1, "brier": 0.0625, "old_estimate_a": 1650, "old_estimate_b": 1600, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Player A (Black) demonstrates a much higher baseline of chess understanding than Player B (White). In Game A, Black navigates a highly theoretical Queen's Gambit Declined Exchange line, maintaining a reasonable position through the middlegame (e.g., 10... h5, 12... Nd7) before succumbing to White's strong endgame play. In contrast, Player B plays very passively in the opening (3. e3, 5. Nbd2) and exhibits poor tactical vision. After Black's 20... Qg6, which threatens mate on g2, White plays 21. Bf3? completely overlooking the basic fork 21... Bc2, which wins the exchange and the game on the spot. Player A is clearly the stronger, more established player.", "confidence_valid": true } ] }, { "model": "google/gemini-3.7-flash", "correct": 21, "n": 25, "accuracy": 84.0, "baseline_accuracy": 80, "invalid_confidence": 0, "consistent_reversals": 4, "reversals": [ { "pair": "pair_09_reverse", "original": "pair_09", "prediction": { "stronger": "B", "probability_a_stronger": 0.25, "explanation": "Game B Black demonstrates significantly higher dynamic awareness and tactical initiative despite losing. After falling for the tactic 10...Nxd4 11.Nxd4 Bxd4 12.Bb5+ c6 13.Qxd4, Black fought back energetically with 21...c4, 22...Rd4, and 25...Rd2, generating dangerous mating threats on f2 before missing the back-rank vulnerability on move 27. In contrast, Game A Black played passively, walking into simple knight forks and pins (15...Rd8, 22...e5 allowing 23.Nxd5 dropping the bishop on b7) and succumbed without creating any counterplay." }, "consistent": false }, { "pair": "pair_15_reverse", "original": "pair_15", "prediction": { "stronger": "A", "probability_a_stronger": 0.88, "explanation": "Game A White executed a textbook, deeply calculated kingside assault against the Modern Defence. White demonstrated strong strategic planning (h4-h5, g4-g5, doubling rooks) and accurate tactical calculation, cleanly executing the decisive 26. Rxh7! followed by queen sacrifices and a forced checkmate with 38. Nh6#. In contrast, Game B White displayed poor opening fundamentals, moving the bishop five times by move 12 (Bc4-d3-e2-d3-e4), instantly lost a pawn to 16...Nxg2+, and completely collapsed tactically without generating any counterplay." }, "consistent": true }, { "pair": "pair_13_reverse", "original": "pair_13", "prediction": { "stronger": "A", "probability_a_stronger": 0.88, "explanation": "Game A features a coherent, principled Caro-Kann Exchange middlegame with thematic opposite-side castling pawn storms (14. b4, 16. a4, 20. a5) before White blunders into a classic smothered mate sequence starting with 24...Ne2+. In contrast, Game B shows typical beginner play from move 2: White plays 2. Qf3 and 3. g4, unnecessarily exposing their king to 9. Ke3 under heavy tactical threats. Black subsequently misses taking White's queen on 17. Qxc5 before hanging their own queen with 22...Qxg1+. Game A's target demonstrates far superior opening and positional understanding." }, "consistent": true }, { "pair": "pair_12_reverse", "original": "pair_12", "prediction": { "stronger": "B", "probability_a_stronger": 0.38, "explanation": "Game B's Black played a crisp and principled Sicilian setup, efficiently seizing queenside space with ...b5 and ...c4, coordinating pieces smoothly (...Bb7, ...Qc7), and aggressively punishing White's passive structure with 15...Nf4 and 20...Nb4. In contrast, Game A's Black played slightly awkwardly in the opening, moving their bishop repeatedly (3...Bd7, 9...Bf5, 10...Bd7) against an unorthodox White attack. While Game A Black converted accurately after White's tactical blunders on moves 16-19, Game B Black demonstrated sharper overall opening handling and dynamic piece coordination." }, "consistent": true }, { "pair": "pair_18_reverse", "original": "pair_18", "prediction": { "stronger": "A", "probability_a_stronger": 0.88, "explanation": "Game A White played a clean, coordinated game. After falling a pawn behind temporarily, White calculated 12. Bb5+ c6 13. Qxd4 to regain the material with an initiative, subsequently opening lines against Black's king with 20. bxc5 and 21. a4, and concluding accurately with 25. Be5 and 28. Qg8#. In contrast, Game B Black played an unsound opening with excessive, weakening pawn pushes (4...b5, 6...b4, 7...f6, 8...h5, 9...g5). Despite obtaining counterplay later, Black suffered a catastrophic tactical blindness, playing 22...Qd8?? and hanging their queen in one move to 23. Bxd8." }, "consistent": true } ], "rows": [ { "pair": "pair_01", "game_a": "game_16", "color_a": "black", "game_b": "game_19", "color_b": "white", "rating_a": 1569, "rating_b": 2668, "gap": 1099, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1300, "old_estimate_b": 2000, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Target B (White in Game B) demonstrated high-level strategic and endgame skill. After doubling Black's pawns with 8. Bxf6 and trading queens, White accurately navigated the endgame with Kd2, Nh3-f4, exploited Black's kingside weaknesses to win pawns (22. Nxh5), and converted effortlessly with 32. Re1 and 35. Ra8+. In contrast, Target A (Black in Game A) played an unsound gambit and blundered a full piece with 6...Bxf2+? without any compensation, subsequently losing without mounting any real resistance.", "confidence_valid": true }, { "pair": "pair_02", "game_a": "game_01", "color_a": "black", "game_b": "game_24", "color_b": "white", "rating_a": 1159, "rating_b": 762, "gap": 397, "gap_band": "200-499", "low_band": "one<1000", "gold": "A", "choice": "B", "probability_a": 0.18, "correct": 0, "brier": 0.6724000000000001, "old_estimate_a": 1250, "old_estimate_b": 1300, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Player A (Black) exhibited serious tactical oversights, notably blundering their queen on move 15 with 15...Nxg5?? allowing the elementary fork 16. Nf6+, and only winning after White threw away a winning position with 40. Qb8+??. In contrast, Player B (White) played a much more coherent opening, handled Black's kingside thrust reasonably, and seized practical chances with tactical awareness (20. Nb6 and 22. Bd5 winning material before capturing Black's blundered queen on 23. Bxd8).", "confidence_valid": true }, { "pair": "pair_03", "game_a": "game_01", "color_a": "white", "game_b": "game_06", "color_b": "white", "rating_a": 1363, "rating_b": 1112, "gap": 251, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.62, "correct": 1, "brier": 0.1444, "old_estimate_a": 1300, "old_estimate_b": 1350, "baseline_choice": "B", "baseline_credit": 0, "explanation": "White in Game A demonstrated solid tactical awareness by spotting the intermediate fork 16.Nf6+, winning Black's queen. White then navigated the complications well and pushed a dangerous passed pawn to e7 before suffering a catastrophic, single-move blunder with 40.Qb8+??. In contrast, White in Game B played an awkward opening setup (4.Bd3 and 6.Bb1), lost control of the dark squares and key outposts (13...Nd4, 18...Ncxb3), and showed little resistance as Black quickly broke through to deliver checkmate on move 31. Despite the endgame blunder, Player A displayed superior tactical vision and positional competence.", "confidence_valid": true }, { "pair": "pair_04", "game_a": "game_22", "color_a": "black", "game_b": "game_08", "color_b": "black", "rating_a": 557, "rating_b": 1791, "gap": 1234, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 850, "old_estimate_b": 1550, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Target A plays a chaotic, low-accuracy game characterized by unsound piece sacrifices (6...Nxf2), huge tactical oversights (failing to capture White's Queen on move 17 with 17...Bf5 instead of 17...Qxc5), and ultimately blundering the Queen entirely (22...Qxg1+). In contrast, Target B demonstrates solid opening principles, spots a clean tactical fork to win material (14...c5!), and smoothly converts the advantage with accurate tactical strikes (21...Ng5 pinning the knight and 24...Nf3+ winning the rook). Target B shows vastly superior tactical vision and positional understanding.", "confidence_valid": true }, { "pair": "pair_05", "game_a": "game_09", "color_a": "white", "game_b": "game_07", "color_b": "black", "rating_a": 2193, "rating_b": 1593, "gap": 600, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.28, "correct": 0, "brier": 0.5184, "old_estimate_a": 1850, "old_estimate_b": 1700, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Target B (Black) demonstrated clean, mature handling of the Slav Defense, effortlessly completing development, coordinating pieces toward White's kingside, and instantly exploiting tactical weaknesses with 20...Qg6! threatening g2 and setting up the decisive 21...Bc2 fork winning a rook. Target A (White) played rather passively against the Benoni, lost queenside control, blundered a pawn via 21.Qxe2? Bxc3 22.bxc3 Qxa3, and allowed Black's heavy pieces to completely dominate the 2nd rank and finish the game.", "confidence_valid": true }, { "pair": "pair_06", "game_a": "game_17", "color_a": "black", "game_b": "game_25", "color_b": "black", "rating_a": 1639, "rating_b": 838, "gap": 801, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.88, "correct": 1, "brier": 0.0144, "old_estimate_a": 1700, "old_estimate_b": 1200, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black demonstrates solid fundamental chess understanding, navigating an open game with sound opening principles (6...d5, 8...Bg4), active piece placement (13...Nf4), and classical central play (21...Rd8, 22...Ne5). Although Black resigned or flagged after 24...c6, their position remained completely playable and logical throughout. In contrast, Game B Black plays an erratic, passive opening (2...Bd7, 7...Rd8) and, despite winning a trapped rook on a1, displays terrible king safety and coordination. Game B Black completely mishandles White's uncoordinated attack, missing major tactical threats (22...Qc5 allowing 23.Qa8+) and collapsing rapidly into checkmate (39.Rxd8#).", "confidence_valid": true }, { "pair": "pair_07", "game_a": "game_14", "color_a": "black", "game_b": "game_08", "color_b": "white", "rating_a": 1188, "rating_b": 1310, "gap": 122, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.72, "correct": 0, "brier": 0.5184, "old_estimate_a": 1500, "old_estimate_b": 1350, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A Black demonstrated solid positional fundamentals, defending their center, contesting the d-file, and cleanly punishing White's final blunder with 30...Rxd3 (though missing the free knight earlier on move 15). In contrast, Game B White played an extremely passive and disjointed opening setup (Nc3, d3, e3, g3), walked into a simple pawn fork with 14...c5 losing a piece on c5 without compensation, and subsequently collapsed to the pin on f3 with 20...Bg4 and 21...Ng5. Target A displayed much better board vision and coordination.", "confidence_valid": true }, { "pair": "pair_08", "game_a": "game_02", "color_a": "black", "game_b": "game_20", "color_b": "black", "rating_a": 2410, "rating_b": 1903, "gap": 507, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.78, "correct": 1, "brier": 0.04839999999999999, "old_estimate_a": 1650, "old_estimate_b": 1700, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Target A (Game A Black) demonstrated solid opening understanding and sharp tactical awareness. After White's inaccurate gambit play, Black smoothly collected material (9...Nxe4), activated pieces dynamically (13...Na5, 14...Nc4, 21...Re2), and delivered a flawless finish with 24...Rxh3 and 25...Rh1#. Conversely, Target B (Game B Black) defended passively against a standard Yugoslav-style kingside pawn storm, misplaying the kingside defense with 23...Bg7? and 25...Kf7?, allowing White a crushing breakthrough (26. Rxh7) and a clean tactical finish.", "confidence_valid": true }, { "pair": "pair_09", "game_a": "game_12", "color_a": "black", "game_b": "game_03", "color_b": "black", "rating_a": 1811, "rating_b": 1901, "gap": 90, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.22, "correct": 1, "brier": 0.0484, "old_estimate_a": 1550, "old_estimate_b": 1500, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A Black displayed low tactical awareness early on, blundering a full piece to a simple deflection motif on move 10 (10...Nxd4? 11.Nxd4 Bxd4 12.Bb5+ c6 13.Qxd4) and later allowing a direct back-rank mate with 27...Nxe3? 28.Qg8#. In contrast, Game B Black demonstrated a solid understanding of Dutch Defense structures (1...f5 through 7...O-O) and coordinated play. While Game B Black was outplayed by White's active knights starting around 15...Rd8, they showed better positional understanding, calculation, and board vision than Game A Black.", "confidence_valid": true }, { "pair": "pair_10", "game_a": "game_23", "color_a": "black", "game_b": "game_09", "color_b": "black", "rating_a": 564, "rating_b": 2177, "gap": 1613, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.12, "correct": 1, "brier": 0.0144, "old_estimate_a": 1250, "old_estimate_b": 1950, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Player B displays a sophisticated understanding of opening strategy and positional play in the Benoni/KID setup, executing a classic queenside expansion (...Na6-c7, ...b5) and active piece maneuvers (14...Ne5, 15...Nc4). Player B converted smoothly with 21...Bxc3, heavy-piece infiltration (...Rb2, ...Qd2), and precise technique in the endgame. In contrast, Player A demonstrated poor positional and tactical awareness, dropping a central pawn with 11...e5? and following up with an unsound piece sacrifice (13...Bxh3?) and weak defense that led to an early resignation after 24.Qh2.", "confidence_valid": true }, { "pair": "pair_11", "game_a": "game_21", "color_a": "black", "game_b": "game_10", "color_b": "black", "rating_a": 743, "rating_b": 2501, "gap": 1758, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.08, "correct": 1, "brier": 0.0064, "old_estimate_a": 1200, "old_estimate_b": 2150, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B Black demonstrates significantly higher chess understanding, playing a classical, principled setup against White's English Opening (1...e6, 4...Nbd7, 7...c5). Black coordinated their pieces harmoniously in the middlegame and executed a decisive tactical sequence with 24...Qc2!, exploiting the fork on d1 and b2, followed cleanly by 25...Ba6 and 27...Ne8. In contrast, Game A Black plays chaotic and suboptimal chess from the opening (2...e5, 3...Qe7, 4...f6), dropping pawns and allowing Queen tactics (7.Qxb7), surviving only through White's blunders in an erratic low-level endgame.", "confidence_valid": true }, { "pair": "pair_12", "game_a": "game_05", "color_a": "black", "game_b": "game_06", "color_b": "black", "rating_a": 1867, "rating_b": 1382, "gap": 485, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.64, "correct": 1, "brier": 0.1296, "old_estimate_a": 1600, "old_estimate_b": 1550, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black displayed excellent opening understanding and tactical sharpness against White's unorthodox Sicilian play. Moves like 4...b5, 5...c4, 11...f5, and the precise tactical combination starting with 16...Nxg2+ followed by 17...fxe4 and 20...Nb4/21...Nbd3 demonstrated high-level positional control and tactical accuracy. In Game B, both players looked less seasoned; Black's passive 3...Bd7 and retreating tempo loss 9...Bf5 and 10...Bd7 signaled less sophisticated opening knowledge, even though Black converted well later with 13...Nd4 and 17...Nc5. Game A Black demonstrated higher overall proficiency.", "confidence_valid": true }, { "pair": "pair_13", "game_a": "game_22", "color_a": "white", "game_b": "game_04", "color_b": "white", "rating_a": 717, "rating_b": 2508, "gap": 1791, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.08, "correct": 1, "brier": 0.0064, "old_estimate_a": 900, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A features chaotic, low-level opening play with 2. Qf3 and 3. g4 by White, followed by reckless piece sacrifices and mutual tactical blunders throughout a wild king hunt. In contrast, Game B displays coherent positional understanding in a Caro-Kann Exchange Variation, with White executing standard strategic concepts like queenside expansion (14. b4, 15. Nb3, 16. a4). Although White in Game B blundered into a classic smothered mate sequence starting with 24. Bb4?? Ne2+, their overall opening preparation, pawn structure handling, and strategic coordination indicate a much higher established rating than Game A's White.", "confidence_valid": true }, { "pair": "pair_14", "game_a": "game_18", "color_a": "black", "game_b": "game_14", "color_b": "white", "rating_a": 2641, "rating_b": 1249, "gap": 1392, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.82, "correct": 1, "brier": 0.03240000000000002, "old_estimate_a": 2050, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black displayed strong tactical and positional awareness, conducting a purposeful kingside assault starting with 11...h5, coordinating pieces effectively, and executing a decisive mating sequence with 30...Re2+ and 31...Rh2. In contrast, Game B White played passively and aimlessly (e.g., 8. Nh4, allowing 15...Qxe4 which Black missed), before abruptly blundering a full minor piece with 30. Rg6?? Rxd3. Target A demonstrated much higher tactical vigilance and overall strategic cohesion.", "confidence_valid": true }, { "pair": "pair_15", "game_a": "game_05", "color_a": "white", "game_b": "game_20", "color_b": "white", "rating_a": 1120, "rating_b": 1932, "gap": 812, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.04, "correct": 1, "brier": 0.0016, "old_estimate_a": 1400, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Player B demonstrates a high level of strategic and tactical competence against the Modern Defense. White in Game B executes a cohesive kingside attack (g4, h4-h5, Qh2, f4-f5), culminating in a precise queen sacrifice and checkmate (26. Rxh7! through 38. Nh6#). In contrast, Player A shows poor opening handling and low tactical awareness, repeatedly shuffling the light-squared bishop (Bc4-d3-e2-d3-e4), blundering a piece with 16. Be4? fxe4, and collapsing rapidly to a decisive mating attack on f3.", "confidence_valid": true }, { "pair": "pair_16", "game_a": "game_11", "color_a": "black", "game_b": "game_02", "color_b": "white", "rating_a": 1273, "rating_b": 1575, "gap": 302, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.28, "correct": 1, "brier": 0.07840000000000001, "old_estimate_a": 1400, "old_estimate_b": 1500, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Player B demonstrates standard Ruy Lopez opening principles and reasonable piece coordination before collapsing to tactical oversights in the endgame (allowing 24...Rxh3 and mate). In contrast, Player A plays the suspect 1...e5 against 1.d4 followed by an early queen sortie (3...Qf6), fails to capture a hanging bishop (34...Rdd6 instead of 34...Rxb6), and blunders into an elementary back-rank checkmate on move 40 with 40...e3?? allowing 41.Rd8+.", "confidence_valid": true }, { "pair": "pair_17", "game_a": "game_13", "color_a": "white", "game_b": "game_16", "color_b": "white", "rating_a": 1126, "rating_b": 1752, "gap": 626, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.12, "correct": 1, "brier": 0.0144, "old_estimate_a": 1250, "old_estimate_b": 1600, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Player B shows far superior tactical and strategic discipline. Against Black's speculative piece sacrifice (6...Bxf2+), Player B cleanly executed artificial castling (8.Rf1, 9.Kg1), consolidated the extra piece smoothly, and converted the advantage without any concessions. In contrast, Player A in Game A played awkwardly in the opening (10.Qa1), permitted tactical vulnerabilities like 17.c3 (allowing 17...Nc2+), and ultimately blundered a basic back-rank mate with 35.Nxb5?? Rc1#.", "confidence_valid": true }, { "pair": "pair_18", "game_a": "game_24", "color_a": "black", "game_b": "game_12", "color_b": "white", "rating_a": 765, "rating_b": 1877, "gap": 1112, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1250, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Target B (White in Game B) demonstrated solid tactical vision and strategic competence. After Black grabbed a central pawn with 10...Nxd4 and 11...Bxd4, White immediately punished the tactic with the intermediate check 12.Bb5+! winning a piece (13.Qxd4). White then converted the advantage cleanly, concluding with 28.Qg8+. In contrast, Target A (Black in Game A) played an opening with excessive weakening pawn pushes (7...f6, 8...h5, 9...g5, 11...c5) leaving the king vulnerable, missed multiple tactics, and ultimately hung the queen outright with 22...Qd8?? allowing 23.Bxd8.", "confidence_valid": true }, { "pair": "pair_19", "game_a": "game_17", "color_a": "white", "game_b": "game_10", "color_b": "white", "rating_a": 1514, "rating_b": 2689, "gap": 1175, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.32, "correct": 1, "brier": 0.1024, "old_estimate_a": 1750, "old_estimate_b": 2050, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B features much deeper opening understanding and strategic sophistication in a standard English/Reti structure (1. c4 through 18. Qh4) before White tactical slip with 24. Nxa7? allowing 24...Qc2. In contrast, Game A displays significantly lower-tier play, with White playing crude pawn-pushing and awkwardly placed knights (11. g4, 12. Neg5 h6 13. Ne4), facing an opponent who resigned prematurely after 24. Rd5. The overall structural and positional competence in Game B strongly points to a higher established rating.", "confidence_valid": true }, { "pair": "pair_20", "game_a": "game_11", "color_a": "white", "game_b": "game_23", "color_b": "white", "rating_a": 1228, "rating_b": 557, "gap": 671, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "B", "probability_a": 0.22, "correct": 0, "brier": 0.6084, "old_estimate_a": 1450, "old_estimate_b": 1350, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game B White demonstrates much better tactical awareness and defensive control. In Game B, White played a coherent opening, cleanly won material after Black's positional errors (11. dxe5 and 14. gxh3), and calmly diffused Black's kingside counterplay to force resignation (20. Qe2 and 24. Qh2). In contrast, Game A White made egregious one-move tactical blunders, outright hanging a bishop on move 14 with 14. Bc4 (which Black failed to capture with 14...Qxc4) and again on move 34 with 34. Bb6 (allowing 34...Rxb6). Game A White only won because Black overlooked a basic back-rank mate (41. Rd8+).", "confidence_valid": true }, { "pair": "pair_21", "game_a": "game_21", "color_a": "white", "game_b": "game_15", "color_b": "white", "rating_a": 739, "rating_b": 2157, "gap": 1418, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.03, "correct": 1, "brier": 0.0009, "old_estimate_a": 1150, "old_estimate_b": 2150, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White opens with the dubious 2. Qd3 and demonstrates poor tactical awareness throughout the game, allowing Black to win a rook via ...Nf2 (moves 28-30) after previously missing multiple knight forks and hanging pawns. In contrast, Game B White exhibits sound positional understanding in the Slav Defense (4. e3, 5. a4), executes clean tactical simplifications (23. dxc5 through 27. Nxb7), and plays a technically solid knight and pawn endgame leading into a straightforward drawn king-and-pawn finish. Player B demonstrates much higher tactical and endgame competence.", "confidence_valid": true }, { "pair": "pair_22", "game_a": "game_04", "color_a": "black", "game_b": "game_13", "color_b": "black", "rating_a": 2718, "rating_b": 1551, "gap": 1167, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.85, "correct": 1, "brier": 0.022500000000000006, "old_estimate_a": 1950, "old_estimate_b": 1300, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black displayed high-level Caro-Kann opening understanding, purposeful strategic play on both flanks, and flawless tactical calculation, concluding with a textbook smothered mate starting with 24...Ne2+ and 25...Qg1+. In contrast, Game B featured sloppy opening play with risky queen excursions (5...Qb5, 6...Qxb2). Game B Black later missed a tactical sequence, falling into a knight fork and dropping an exchange on move 29 (28.Ne8 Re6 29.Nc7), only winning due to White blundering back-rank mate with 35.Nxb5??", "confidence_valid": true }, { "pair": "pair_23", "game_a": "game_25", "color_a": "white", "game_b": "game_18", "color_b": "white", "rating_a": 738, "rating_b": 2426, "gap": 1688, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.15, "correct": 1, "brier": 0.0225, "old_estimate_a": 1200, "old_estimate_b": 1950, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White shows weak opening fundamentals with random flank pawn pushes (3. Qf3, 4. g4, 6. h4, 8. h5) and outright blundered a full rook on move 13 (13. Nh3 Nc2+), winning only because Black repeatedly blundered back. In contrast, Game B White plays a coherent opening with standard piece development and central control (3. b3, 5. Bg2, 7. O-O, 8. d4). Even though Game B White eventually lost due to tactical oversights in time trouble (30. Rd1 Re2+ 31. Kf1 Rh2), the overall strategic and tactical comprehension is significantly higher than Game A White's.", "confidence_valid": true }, { "pair": "pair_24", "game_a": "game_03", "color_a": "white", "game_b": "game_15", "color_b": "black", "rating_a": 1782, "rating_b": 2154, "gap": 372, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.22, "correct": 1, "brier": 0.0484, "old_estimate_a": 1700, "old_estimate_b": 2150, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B demonstrates much higher-level chess understanding and precision from both players. Target B navigates a complex Slav structure smoothly, executing accurate piece coordination (14...Nb6, 15...Qd5) and sharp liquidating tactics (22...c5! through 26...exd5). Target B then displays flawless endgame calculation to secure a pawn-ending draw (42...d4!, 47...d3+). In contrast, Game A features cruder opening play with an aggressive flank thrust (8. h4, 9. Ng5), where White capitalized easily on Black's severe tactical blunders (such as 16...Kf7 and passive piece placement) without facing substantial resistance.", "confidence_valid": true }, { "pair": "pair_25", "game_a": "game_19", "color_a": "black", "game_b": "game_07", "color_b": "white", "rating_a": 2474, "rating_b": 1862, "gap": 612, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.72, "correct": 1, "brier": 0.07840000000000001, "old_estimate_a": 1850, "old_estimate_b": 1650, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Player A (Black) played a complex endgame against a strong opponent, demonstrating reasonable strategic ideas (queenside pawn expansion, king activity) before succumbing to tactical tricks on move 34 (34...Nxf5? allowing 35. Ra8+). Player B (White) played passively with unambitious developing moves and committed a basic tactical oversight with 21. Bf3??, walking straight into the 21...Bc2 fork winning the rook on b3 and leading to an immediate resignation.", "confidence_valid": true } ] }, { "model": "qwen/qwen3.8-max-prime", "correct": 21, "n": 25, "accuracy": 84.0, "baseline_accuracy": 58.0, "invalid_confidence": 0, "consistent_reversals": 3, "reversals": [ { "pair": "pair_09_reverse", "original": "pair_09", "prediction": { "stronger": "B", "probability_a_stronger": 0.3, "explanation": "Game A Black played poorly: 16...Kf7 blundered a piece, 19...gxf6 ruined structure, and the position collapsed after 23...exf4 tactics, showing weak defense. Game B Black opened solidly, played active counterplay (10...Nxd4, 24...Rd2, 26...Nxc4), and only lost after a tactical oversight near the end. Black B demonstrated stronger middlegame understanding and more accurate piece play overall." }, "consistent": false }, { "pair": "pair_15_reverse", "original": "pair_15", "prediction": { "stronger": "A", "probability_a_stronger": 0.82, "explanation": "Game A White executes a coherent kingside attack: 13.hxg6 opens the h-file, 24.f5 gains space, and the rook sacrifice 26.Rxh7! leads to a forced mating net ending 38.Nh6#. Game B White plays passively and inaccurately: 11.Bd3 retreats the bishop awkwardly, allows 15...Nf4 and the devastating 16...Nxg2+ tactic; White never generates counterplay and is mated by move 29. Game A White shows superior tactical vision and coordinated attacking technique, while Game B White's position collapses from strategic and tactical errors." }, "consistent": true }, { "pair": "pair_13_reverse", "original": "pair_13", "prediction": { "stronger": "B", "probability_a_stronger": 0.28, "explanation": "Game A White plays a passive, inaccurate line, allowing Black a crushing attack: 13.O-O-O? hands Black open-file play, 17.Be1? is a blunder losing structure, and 20.a5? ignores the knight invasion, leading to the forced 24...Ne2+ 25...Qg1+ 26...Nf2# mate. Game B White plays a quirky opening but converts a chaotic middlegame with reasonable resourcefulness: 15.Qc4, 18.Qxd5, and 21.Qxb7 gain material; despite a shaky 23.Nxg1, the win is secured against an opponent who blunders repeatedly. B's White, despite imperfections, shows more practical competence than A's White, who collapses quickly." }, "consistent": true }, { "pair": "pair_12_reverse", "original": "pair_12", "prediction": { "stronger": "A", "probability_a_stronger": 0.78, "explanation": "Game A Black met the odd 1.c4 setup with standard plans (…Nf6, …Be7, …0-0) and exploited White's blunders, converting the knight with clean calculation like 18…Ncxb3, 19…Nxa1, and finishing neatly with 30…Rc1+ 31…Qf2#. Game B Black also won, but allowed sloppy tactics: 16…Nxg2+ relied on White not taking, and later moves like 23…Bxh1 and 26…Bxf3 won only amid chaotic blunders by White. A's target showed more controlled, accurate handling of a weaker opponent, while B's target benefited from sharper, more accidental tactics, suggesting A slightly stronger." }, "consistent": false }, { "pair": "pair_18_reverse", "original": "pair_18", "prediction": { "stronger": "A", "probability_a_stronger": 0.88, "explanation": "Game A: White (target) played a clean London setup, calmly recapturing 13.Qxd4, and finished with the decisive 27.Bxh8, winning. Black blundered 9...exd4? and 25...Rd2?. Game B: Black (target) opened poorly with 4...b5, 5...a5, 6...b4, overextended with 8...h5/9...g5, and fell to 16.gxh4!, losing material before 23.Bxd8. White A converted clearly; Black B was outplayed." }, "consistent": true } ], "rows": [ { "pair": "pair_01", "game_a": "game_16", "color_a": "black", "game_b": "game_19", "color_b": "white", "rating_a": 1569, "rating_b": 2668, "gap": 1099, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.6, "correct": 0, "brier": 0.36, "old_estimate_a": 1200, "old_estimate_b": 1500, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black sacrifices the bishop unsoundly (6...Bxf2+), but recovers with sound positional moves: 12...Bxf3, 17...Rxe1, 22...cxd4, and the simplifying 25...Qxe2, reaching a roughly defensible rook ending before a late collapse. Game B White plays more passively and strategically shaky: 7.Qf3 invites trades, 9.Qxf6 gxf6 improves Black's structure, 10.h4 and 22.Nxh5 chase loose pawns while Black equalizes; the win owes more to opponent errors after 30...dxc4. A's middlegame resourcefulness under a bad opening suggests a slightly stronger established player.", "confidence_valid": true }, { "pair": "pair_02", "game_a": "game_01", "color_a": "black", "game_b": "game_24", "color_b": "white", "rating_a": 1159, "rating_b": 762, "gap": 397, "gap_band": "200-499", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.68, "correct": 1, "brier": 0.10239999999999996, "old_estimate_a": 1300, "old_estimate_b": 1300, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "Target A (Black, Game A) handled a chaotic middlegame competently, finding accurate trades (15...Nxg5, 17...Kxd7) and converting a winning endgame. Target B (White, Game B) won only through Black's severe blunder 22...Qxd8??, losing the queen; before that, Black had outplayed White with 13...Nd4 and 16...Bg4. Black's 8...Qd7 and 26...a5 in Game A show coherent plans, while White in Game B played passively (7.Ne2, 10.Nd2) and relied on an opponent's one-move oversight. A's opponent offered a tougher fight; A's play against it was stronger.", "confidence_valid": true }, { "pair": "pair_03", "game_a": "game_01", "color_a": "white", "game_b": "game_06", "color_b": "white", "rating_a": 1363, "rating_b": 1112, "gap": 251, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.62, "correct": 1, "brier": 0.1444, "old_estimate_a": 1150, "old_estimate_b": 1400, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Both whites lost, but Game B's collapse was far more one-sided. B's White played passively (6.Bb1, 7.Qc2), then after 17...Nc5 and 18...Ncxb3 lost material with no compensation; 24.Nf6+ Kg7 simply hung into 25...Bxf1, and Black mated with 31...Qf2#. Game A's White built a playable kingside attack (11.h4, 14.Bg5, 16.Nf6+), creating real tactical problems, and only lost after a long, complex endgame where 30.dxe4 allowed 30...Bxg3. A showed more competence in a richer middlegame.", "confidence_valid": true }, { "pair": "pair_04", "game_a": "game_22", "color_a": "black", "game_b": "game_08", "color_b": "black", "rating_a": 557, "rating_b": 1791, "gap": 1234, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.3, "correct": 1, "brier": 0.09, "old_estimate_a": 800, "old_estimate_b": 1550, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black (target) collapses tactically: after 5...Ng4 6.h3 Nxf2 7.Kxf2, Black plays 8...Qf6+ 9.Ke3, then later loses material with 11...Nb5+ 12.Ke2 Qf6 13.Qxb5, dropping a knight for nothing. Black's king safety and coordination are poor throughout, and White wins comfortably. Game B Black plays solidly: ...e4 9...Nxe4 gains a central foothold, ...c5 15...Ndxc5 regains the pawn, and the endgame technique is clean—21...Ng5, 22...Bxf3+ 23...Be4, 24...Nf3+ forcing mate threats. Black's play is coherent and decisive, indicating a higher established strength than Game A Black.", "confidence_valid": true }, { "pair": "pair_05", "game_a": "game_09", "color_a": "white", "game_b": "game_07", "color_b": "black", "rating_a": 2193, "rating_b": 1593, "gap": 600, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.72, "correct": 1, "brier": 0.07840000000000001, "old_estimate_a": 1450, "old_estimate_b": 1600, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game A White handled a Modern Benoni with reasonable theory (6.g3, 9.Nge2) but collapsed tactically: 15.Kh2? allowed ...Nc4, and 19.f4? lost the d4-knight sequence, leaving Black a clear pawn up. Despite losing, White found the resourceful 28.e5 and kept resistance into a complex rook ending. Game B Black was outplayed positionally; 21...Bc2?? hangs material to 22.Bh5! and Black resigned after 22...Bxd1 23.Bxg6 Bxb3 24.Bd3, losing the exchange and position outright. A's errors were subtle; B blundered a full exchange in a quiet position, indicating lower strength.", "confidence_valid": true }, { "pair": "pair_06", "game_a": "game_17", "color_a": "black", "game_b": "game_25", "color_b": "black", "rating_a": 1639, "rating_b": 838, "gap": 801, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.87, "correct": 1, "brier": 0.016900000000000002, "old_estimate_a": 1050, "old_estimate_b": 1200, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game A Black played solid, standard rapid lines: …Bc5, …d5 counter-strike, …Nxd5, …Rfe8, and only resigned after 24.Rd5. Game B Black made bizarre, non-developing moves: 2…Bd7, 4…b5, 6…Qc7, 7…Rd8, then won material but collapsed tactically after 20…Rxd4+, allowing the forced mate 39.Rxd8#. B's opening was incoherent and defense hopeless; A showed ordinary club competence.", "confidence_valid": true }, { "pair": "pair_07", "game_a": "game_14", "color_a": "black", "game_b": "game_08", "color_b": "white", "rating_a": 1188, "rating_b": 1310, "gap": 122, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.42, "correct": 1, "brier": 0.17639999999999997, "old_estimate_a": 1400, "old_estimate_b": 1350, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A Black wins mostly from White blundering away a rook: 29...Rd4 30.Rg6?? Rxd3, with Black's play (16...Na6, 18...Qg6, 22...Rd8) competent but unexceptional. Game B White, despite losing, played 16.Ne5 and 17.Bxe4, creating real complications, and only lost to a slick finish 22...Bxf3+ 23.Kg1 Be4 24...Nf3+. B's active middlegame ideas and tighter calculation against stronger opposition suggest a slightly higher rating.", "confidence_valid": true }, { "pair": "pair_08", "game_a": "game_02", "color_a": "black", "game_b": "game_20", "color_b": "black", "rating_a": 2410, "rating_b": 1903, "gap": 507, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.68, "correct": 1, "brier": 0.10239999999999996, "old_estimate_a": 1350, "old_estimate_b": 1550, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game A Black handled the Ruy Lopez solidly, exploiting White's weak Nd5 (Game A 9…Nxe4) and later capitalizing on White's flawed 18.h4 to win a pawn and deliver a clean mate (Game A 25…Rh1#). Game B Black, in the Pirc (Game B 1…g6, 7…Bh8), was repeatedly passive; 25…Kf7 and 27…Rh8 invited disaster, and Black failed to counter White's g-file attack, leading to mate on move 38. Game B Black showed weaker prophylaxis and middlegame judgment than Game A Black.", "confidence_valid": true }, { "pair": "pair_09", "game_a": "game_12", "color_a": "black", "game_b": "game_03", "color_b": "black", "rating_a": 1811, "rating_b": 1901, "gap": 90, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.35, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1250, "old_estimate_b": 1200, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A Black played dubious opening moves like 8...exd4 and 9...Qe7+, and blundered material at 25...Nxc4, but stayed competitively active until then. Game B Black committed severe strategic and tactical errors: 16...Kf7?? allowed 17.Ne5+ forking king and queen, 23...Nd7?? dropped the queen, 31...f4?? and 36...Rg7?? lost more, and ultimately got checkmated. B Black's repeated blunders and king exposure indicate a substantially weaker player than A Black.", "confidence_valid": true }, { "pair": "pair_10", "game_a": "game_23", "color_a": "black", "game_b": "game_09", "color_b": "black", "rating_a": 564, "rating_b": 2177, "gap": 1613, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.15, "correct": 1, "brier": 0.0225, "old_estimate_a": 1100, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "In Game A, Black plays a dubious 13...Bxh3 sacrifice that loses material after 14.gxh3 and then wastes moves with a wandering queen (17...Qg5+, 19...Qg3, 21...Qe6, 22...Qg6, 23...Qh6+) before resigning down a full piece. In Game B, Black executes a coherent Benko-style plan with 10...b5, 13...Ng4-e5-c4, and 19...Nd4, winning cleanly via 21...Bxc3 and a decisive attack ending with 39...Rdd2 and 41...f3. Game B's Black shows clearly superior strategic and tactical play.", "confidence_valid": true }, { "pair": "pair_11", "game_a": "game_21", "color_a": "black", "game_b": "game_10", "color_b": "black", "rating_a": 743, "rating_b": 2501, "gap": 1758, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.18, "correct": 1, "brier": 0.0324, "old_estimate_a": 1150, "old_estimate_b": 1400, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A's Black wins but against a very weak opponent (White's 2.Qd3, 4.f4, and 10.Rxc1 blundering Bxc2). Black plays solidly but nothing exceptional; the win stems from White's opening disaster. Game B's Black demonstrates stronger strategic play: 12...Nxc5 recapture, 15...Ne6 redeployment, 18...Rf8 prophylaxis, and the decisive 23...Qc2 and 26...Bxf1 tactics exploiting White's overextension (24.Nxa7). Game B's opponent played a coherent opening and middlegame, making Black's clean conversion more impressive evidence of higher established strength.", "confidence_valid": true }, { "pair": "pair_12", "game_a": "game_05", "color_a": "black", "game_b": "game_06", "color_b": "black", "rating_a": 1867, "rating_b": 1382, "gap": 485, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.68, "correct": 1, "brier": 0.10239999999999996, "old_estimate_a": 1650, "old_estimate_b": 1500, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black shows sharper tactical vision: 15...Nf4! sets up 16...Nxg2+!, and after 17.Kf1 the sequence 21...Nbd3, 22...Bxe4, 23...Bxh1 wins decisively, finishing with 29...Qxf3#. Game B Black plays solidly but relies heavily on opponent blunders: White's 13.Ng5?, 17.Qe3?, and especially 24.Nf6+? sacrifice material gratuitously. B's 18...Ncxb3 is good but less forcing than A's coordinated knight-bishop attack. A's opponent resisted longer and made fewer outright hangs, so A's win required more accurate, creative play.", "confidence_valid": true }, { "pair": "pair_13", "game_a": "game_22", "color_a": "white", "game_b": "game_04", "color_b": "white", "rating_a": 717, "rating_b": 2508, "gap": 1791, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "A", "probability_a": 0.66, "correct": 0, "brier": 0.43560000000000004, "old_estimate_a": 1100, "old_estimate_b": 1300, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White survived a chaotic opening (6...Nxf2 7.Kxf2) and converted material with accurate tactics: 13.Qxb5, 17.Qxc5, 22.Rg1 repulsing 21...Qg5. The win against speculative Black shows practical strength. Game B White chose a passive Exchange Caro-Kann (4.Bd3, 5.c3, 8.Qe1), drifted after 13...O-O-O, and lost to a standard smothered pattern (24...Ne2+ 25.Kh1 Qg1+ 26.Rxg1 Nf2#) without counterplay. A's active resourcefulness exceeds B's generic play, though A's 2.Qf3/3.g4 opening was also dubious.", "confidence_valid": true }, { "pair": "pair_14", "game_a": "game_18", "color_a": "black", "game_b": "game_14", "color_b": "white", "rating_a": 2641, "rating_b": 1249, "gap": 1392, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.72, "correct": 1, "brier": 0.07840000000000001, "old_estimate_a": 1350, "old_estimate_b": 1200, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black shows coherent Sicilian plans: ...h5-hxg3 opening the h-file, the strong 20...Ng8 redeployment, and the clean finish 30...Re2+, 32...Qh3+, 33...Qxg3+. Errors are modest (10...Kf8). Game B White plays dubious moves: 8.Nh4 wasting time, the unnecessary 10.Bxd5 giving up the bishop pair, awkward 17.Nd6, passive 21.c4, and the losing tactical oversight allowing 30...Rxd3. White's positional drift and final collapse indicate noticeably weaker play than Black's purposeful, accurate attacking execution.", "confidence_valid": true }, { "pair": "pair_15", "game_a": "game_05", "color_a": "white", "game_b": "game_20", "color_b": "white", "rating_a": 1120, "rating_b": 1932, "gap": 812, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.08, "correct": 1, "brier": 0.0064, "old_estimate_a": 1350, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "In Game A, White plays passively and blunders: 16.Kf1 allows 16...Nxg2+ losing the h1-rook and king safety, and White's pieces are quickly overwhelmed (23...Bxh1). In Game B, White executes a coherent, aggressive plan (7.Bh6, 11.h5, 16.g5), keeps pressure, and delivers a precise mating finish (31.Rxd8, 37.g7+, 38.Nh6#). Game A's target is outplayed from the opening and loses badly, while Game B's target dominates with strong attacking technique. The evidence strongly favors B as the higher-rated player.", "confidence_valid": true }, { "pair": "pair_16", "game_a": "game_11", "color_a": "black", "game_b": "game_02", "color_b": "white", "rating_a": 1273, "rating_b": 1575, "gap": 302, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.22, "correct": 1, "brier": 0.0484, "old_estimate_a": 1300, "old_estimate_b": 1200, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Target A (Black, Game A) mishandled a dubious opening (1...e5 2...Bc5 3...Qf6), allowed White a comfortable edge after 8.Nxc5, and later blundered into 42.Rxe8#. Target B (White, Game B) chose the passive 8.Nd5, lost a pawn after 9...Nxe4, then allowed 18...Bxh4 winning material, and finally permitted 25...Rh1# with the rook on h1. Both lost, but A's opening was objectively poor and the mate resulted from a clear oversight in a simplified position, whereas B's errors were more tactical in a complex middlegame. B's play suggests a somewhat higher baseline.", "confidence_valid": true }, { "pair": "pair_17", "game_a": "game_13", "color_a": "white", "game_b": "game_16", "color_b": "white", "rating_a": 1126, "rating_b": 1752, "gap": 626, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.34, "correct": 1, "brier": 0.11560000000000002, "old_estimate_a": 1300, "old_estimate_b": 1400, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White played several dubious moves: 10.Qa1 was awkward, 18.Qb2 dropped the c5 rook to 18...bxc5, and 28.Ne8/29.Nc7 were poorly timed, losing material and being checkmated. Game B White, though winning partly due to Black's blunder 27...Nc7 (allowing the winning tactic), generally kept control after absorbing the Bxf2+ sac (7.Kxf2), coordinated well with 11.Bg5 and 23.Nxd4, and maintained a solid edge into the endgame. White B showed better overall piece play; White A made more elementary errors and lost convincingly.", "confidence_valid": true }, { "pair": "pair_18", "game_a": "game_24", "color_a": "black", "game_b": "game_12", "color_b": "white", "rating_a": 765, "rating_b": 1877, "gap": 1112, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.12, "correct": 1, "brier": 0.0144, "old_estimate_a": 1100, "old_estimate_b": 1300, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A target (Black) played bizarrely with 3...c6, 4...b5, 7...f6, and 8...h5, weakening the kingside and allowing White a winning attack; after 21.Nxa8 Black resigned at 23.Bxd8, showing weak strategic understanding. Game B target (White) played solidly with d4/Nf3/e3, handled Black's tactics calmly after 10...Nxd4, and converted with accurate moves like 25.Be5, 27.Bxh8, and the mating finish 28.Qg8+, demonstrating clearly superior technique and composure.", "confidence_valid": true }, { "pair": "pair_19", "game_a": "game_17", "color_a": "white", "game_b": "game_10", "color_b": "white", "rating_a": 1514, "rating_b": 2689, "gap": 1175, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.32, "correct": 1, "brier": 0.1024, "old_estimate_a": 1250, "old_estimate_b": 1200, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A's White played solid, accurate moves—central control, timely exchanges (16...Rxe4 20...Rxe4), and a clean rook lift ending with 24.Rd5, earning a win with minimal errors. Game B's White collapsed badly: 24.Nxa7?? hangs material and structure, allowing 24...Qc2, then 26.Bxf1?? loses to 26...Ne8, resigning immediately. Despite losing, B's opening (Catalan setup, 12...Nxc5, 15...Ne6) showed higher-level strategic understanding; the blunder was time-pressure induced. A won but with less demanding play and a simpler structure.", "confidence_valid": true }, { "pair": "pair_20", "game_a": "game_11", "color_a": "white", "game_b": "game_23", "color_b": "white", "rating_a": 1228, "rating_b": 557, "gap": 671, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.72, "correct": 1, "brier": 0.07840000000000001, "old_estimate_a": 1500, "old_estimate_b": 1050, "baseline_choice": "A", "baseline_credit": 1, "explanation": "In Game A, White shows solid middlegame understanding: developing naturally, exploiting the bishop pair, and converting with precise tactics like 30.f6 and 41.Rd8+ Re8 42.Rxe8#. In Game B, White plays passively (Bb5 then Bxc6 voluntarily, 6.Bg5 awkwardly), allows 13...Bxh3! winning a pawn with attacking chances, and only wins because Black blunders repeatedly (15...Bh4?, 20...Qxh3+? losing the queen after 21.Qh2). Game A's White demonstrates clearer strategic control and cleaner conversion than Game B's White, who relied on opponent errors.", "confidence_valid": true }, { "pair": "pair_21", "game_a": "game_21", "color_a": "white", "game_b": "game_15", "color_b": "white", "rating_a": 739, "rating_b": 2157, "gap": 1418, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.22, "correct": 1, "brier": 0.0484, "old_estimate_a": 800, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White blunders the opening with 2.Qd3, loses material through 5.Qxd5/7.Qxb7, and never recovers. Game B White plays a solid Slav (5.a4, 7.Bxc4, 11.Nb3), reaches an equal middlegame, and navigates the complex knight maneuvers (15.Nxa5, 17.Rxa4, 25.Qxc8+) into a drawn king-and-pawn ending with accurate opposition technique. B's opening knowledge and endgame precision far exceed A's.", "confidence_valid": true }, { "pair": "pair_22", "game_a": "game_04", "color_a": "black", "game_b": "game_13", "color_b": "black", "rating_a": 2718, "rating_b": 1551, "gap": 1167, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.72, "correct": 1, "brier": 0.07840000000000001, "old_estimate_a": 1550, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black plays a coherent Caro-Kann, coordinates queenside castling with ...O-O-O and ...Kb8, then executes a clean tactical finish: 21...gxf4, 22...Nd4, 23...Qxc5, and the forcing 24...Ne2+ 25...Qg1+ 26...Nf2#. Game B Black's opening is awkward: the queen raid 3...Qa5+, 5...Qb5, 6...Qxb2 wastes tempos, and later ...b6/...bxc5 leaves a clumsy structure; Black wins mainly because White self-destructs with 28.Ne8? and 30.Nxe6?. A's active piece play and precise smothered-mate finish show stronger established technique.", "confidence_valid": true }, { "pair": "pair_23", "game_a": "game_25", "color_a": "white", "game_b": "game_18", "color_b": "white", "rating_a": 738, "rating_b": 2426, "gap": 1688, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "A", "probability_a": 0.82, "correct": 0, "brier": 0.6723999999999999, "old_estimate_a": 1350, "old_estimate_b": 1300, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A's White faced absurd openings (2…Bd7, 3…c6, 4…b5) yet played coherently, winning material cleanly (18.Bxa1, 20.Bxc4) and converting with 30.Rd7+, 38.Qd5+, 39.Rxd8#. Game B's White made real errors: 11.g4? weakening the kingside, 16.hxg3 allowing 16…Bh3, and losing after 30…Re2+ 31.Kf1 Rh2 32.Qxd5 Qh3+ 33.Ke1 Qxg3#. B's White collapsed against a much more competent opponent, suggesting a substantially lower established rating than A's White.", "confidence_valid": true }, { "pair": "pair_24", "game_a": "game_03", "color_a": "white", "game_b": "game_15", "color_b": "black", "rating_a": 1782, "rating_b": 2154, "gap": 372, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.72, "correct": 0, "brier": 0.5184, "old_estimate_a": 1500, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White shows strong tactical awareness: 16.Bxd6 wins material, 19.Bxf6 shatters Black's kingside, and 23.Nxd5 exploits the pin, converting to a clean win. Game B Black plays the Slav solidly but drifts passively; after 15...Qd5 and 16...Rxa4, Black trades into a simplified endgame (24...Nxb7 25.Qxc8+) without generating winning chances, ultimately drawing a king-and-pawn race. White's active piece coordination and decisive finishing in Game A suggest a higher established rating than Black's competent but reactive play in Game B.", "confidence_valid": true }, { "pair": "pair_25", "game_a": "game_19", "color_a": "black", "game_b": "game_07", "color_b": "white", "rating_a": 2474, "rating_b": 1862, "gap": 612, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.68, "correct": 1, "brier": 0.10239999999999996, "old_estimate_a": 1300, "old_estimate_b": 1400, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game A Black defended the Exchange Caro-Kann solidly; the loss resulted from a concrete tactical slip (30...dxc4 allowing 31.d5 and the rook raid 33.Re7+/37.h7), not systemic errors. Game B White played passively (b3, a4), lost time after 12.Nxe4, then blundered a whole queen with 22.Bh5??, ignoring that 22...Bxd1 captures the queen. Game B White's queen oversight indicates weaker tactical alertness than Game A Black's eventual endgame lapse, so A was likely the higher-rated player.", "confidence_valid": true } ] }, { "model": "anthropic/claude-fable-5.1", "correct": 20, "n": 25, "accuracy": 80.0, "baseline_accuracy": 82.0, "invalid_confidence": 0, "consistent_reversals": 3, "reversals": [ { "pair": "pair_09_reverse", "original": "pair_09", "prediction": { "stronger": "B", "probability_a_stronger": 0.4, "explanation": "Game A Black (Dutch) played passively and committed serious tactical oversights: 15...Rd8?? allowed 16.Bxd6 winning a piece, then 18...Re8 and 19...gxf6 shed more material, and the game dragged on with further losses (Rook on g8 lost to 34.Ne7+). Game B Black played a coherent plan: 9...Qe7+ and 10...Nxd4 won a pawn via a concrete tactic against a loose opponent, castled queenside, pressed with 23...Rd4 and 24...Qc5, and only lost to a back-rank blunder 25...Rd2?? allowing 27.Bxh8 and Qg8+. Black in B showed more tactical awareness and purposeful play, despite one decisive error; Black in A showed repeated piece-hanging blunders typical of a weaker player." }, "consistent": false }, { "pair": "pair_15_reverse", "original": "pair_15", "prediction": { "stronger": "A", "probability_a_stronger": 0.82, "explanation": "Game A White played a coherent Yugoslav-style attack vs the Pirc: Bh6, O-O-O, g4-h4-h5, hxg6, then a clean exchange sacrifice 26.Rxh7 leading to a forced win and mate with 38.Nh6#. Black's defense (7...Bh8, 14...Ne5, 16...Nd7 shuffles) was weak, but White's execution was accurate and purposeful. Game B White played a passive, poorly understood anti-Sicilian (2.Bc4, 5.Bd3, 6.Be2, 7.d3 losing tempi), then blundered with 15.c4?? allowing 15...Nf4 and 16...Nxg2+, dropped the h1 rook, and was mated by move 29. Game B White's opponent also showed only modest play, yet White collapsed completely, indicating a notably lower level than Game A's White." }, "consistent": true }, { "pair": "pair_13_reverse", "original": "pair_13", "prediction": { "stronger": "A", "probability_a_stronger": 0.7, "explanation": "Game A White played a coherent Caro-Kann Exchange setup (4.Bd3, 5.c3, 7.O-O, 9.Bg5, 14.b4/16.a4 pawn storm vs the queenside king) before collapsing tactically (23.Qb2?? allowing 24...Ne2+ and the smothered-style 25...Qg1+ 26.Rxg1 Nf2#). Game B White played 2.Qf3, 3.g4, 5.Qd3, 6.h3 and walked the king to e3/d1 — chaotic beginner-level play that only won because Black blundered pieces (11...Nb5+ hanging, 17...Bf5, 22...Qxg1+ giving the queen). Game B's opponent was clearly very weak, suggesting a low rating pool, so Game A's White is likely higher rated despite losing." }, "consistent": true }, { "pair": "pair_12_reverse", "original": "pair_12", "prediction": { "stronger": "B", "probability_a_stronger": 0.35, "explanation": "Game A Black (target) played oddly in the opening (3...Bd7, 9...Bf5 then 10...Bd7, 12...Bg4, 15...Be6 wandering bishop) and faced a very weak opponent (6.Bb1, hanging a1 rook and material); the win came mostly from White's blunders like 18.Qe3 and 24.Nf6+. Game B Black (target) played a coherent Sicilian setup (…b5, …Bb7, …Nd5-f4), exploited White's 11.Be4/16.Be4 errors with 16...Nxg2+, then conducted a clean tactical sequence (21...Nbd3, 23...Bxh1, 25...Nh3+, 29...Qxf3#). Game B's opponent was also somewhat more sensible, suggesting a higher level overall." }, "consistent": false }, { "pair": "pair_18_reverse", "original": "pair_18", "prediction": { "stronger": "A", "probability_a_stronger": 0.82, "explanation": "Game A White played coherent, if imperfect, chess: a sound opening, correctly recovered the pawn with 12.Bb5+ and 13.Qxd4 after Black's dubious 10...Nxd4, then exploited Black's loosening 19...f5 with 22.Bf4+, 25.Be5 and the finishing 28.Qg8+ after Black's 26...Nxc4 oversight. Game B Black played a chaotic flank-pawn opening (4...b5, 5...a5, 7...f6, 8...h5, 9...g5), left pieces undeveloped, allowed 17.Nxf6+ and 20.Nb6 forking the rook, and finally hung the queen with 22...Qd8?? to 23.Bxd8 — basic tactical blindness. White in A shows clearly higher established strength." }, "consistent": true } ], "rows": [ { "pair": "pair_01", "game_a": "game_16", "color_a": "black", "game_b": "game_19", "color_b": "white", "rating_a": 1569, "rating_b": 2668, "gap": 1099, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.2, "correct": 1, "brier": 0.04000000000000001, "old_estimate_a": 1100, "old_estimate_b": 1550, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black plays a speculative Scandinavian gambit with 3...Bxe6 and the unsound 6...Bxf2+ sacrifice, then 11...Ng4?? losing a piece outright (12.h3 Bxf3 13.hxg4), and resigns a pawn down in a lost ending after 27.Re6. That is a beginner-level piece blunder. Game B White conducts a solid Exchange QGD, trades sensibly (8.Bxf6, 9.Qxf6), centralizes the king for the queenless middlegame (12.Kd2, 19.Kxd3), wins a pawn with 22.Nxh5, and converts with the h-pawn and 33.Re7+ 34.Ra7 35.Ra8+. Technique is clean and purposeful; Black's defence in B was also reasonable. White in B shows clearly higher established strength.", "confidence_valid": true }, { "pair": "pair_02", "game_a": "game_01", "color_a": "black", "game_b": "game_24", "color_b": "white", "rating_a": 1159, "rating_b": 762, "gap": 397, "gap_band": "200-499", "low_band": "one<1000", "gold": "A", "choice": "B", "probability_a": 0.3, "correct": 0, "brier": 0.48999999999999994, "old_estimate_a": 1200, "old_estimate_b": 1250, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game A Black played a dubious setup (2...d5, 4...f5, 7...Qd7, 12...h5) and allowed 15.Nd5 Nxg5 16.Nf6+ losing the queen for two minor pieces; White then blundered badly (26.Qc6?, 27.Re1?? allowing e4 and Bxg3-Bxe1, and finally 40.Qb8+??), so Black's win reflects a weak opponent more than skill. Game B White handled a bizarre Black setup (3...c6, 4...b5, 7...f6, 8...h5, 9...g5) soundly, developed, castled, and punished tactically with 17.Nxf6+, 20.Nb6 winning the exchange and 22.Bd5 trapping/winning the queen. White in B showed better fundamentals and tactical alertness; Black in A showed poor opening understanding and missed basic tactics.", "confidence_valid": true }, { "pair": "pair_03", "game_a": "game_01", "color_a": "white", "game_b": "game_06", "color_b": "white", "rating_a": 1363, "rating_b": 1112, "gap": 251, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.58, "correct": 1, "brier": 0.17640000000000003, "old_estimate_a": 1300, "old_estimate_b": 900, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A White played a coherent if passive setup, found the tactical shot 15.Nd5/16.Nf6+ winning the queen for two pieces, then blundered a rook with 27.Re1?? allowing 27...e4 and later 40.Qb8+?? hallucinating a promotion trick; still, the earlier play showed genuine tactical awareness. Game B White drifted badly: 6.Bb1 and 7.Qc2 wasted tempi, 9.b3 and 13.Ng5 allowed ...Nd4 and ...Ncxb3 forking queen and rook, dropping the a1 rook and then c4, Rf1 for free, and finally walked into 31...Qf2# after grabbing pawns. Black in A also played weakly, making both opponents comparable. A's White seems slightly stronger.", "confidence_valid": true }, { "pair": "pair_04", "game_a": "game_22", "color_a": "black", "game_b": "game_08", "color_b": "black", "rating_a": 557, "rating_b": 1791, "gap": 1234, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.2, "correct": 1, "brier": 0.04000000000000001, "old_estimate_a": 800, "old_estimate_b": 1400, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black played a speculative Alekhine against a weak Qf3/g4 setup but then sacrificed unsoundly (6...Nxf2, 11...Nb5+) and hung the bishop and knight, then blundered the queen with 22...Qxg1+ into a hopeless position; the whole game shows poor calculation and piece-hanging. Game B Black played coherent developing moves (3...Bb4, 8...Bxc3, 9...e4), kept pieces coordinated, exploited White's errors (15...Ndxc5, 17...Nxe4) and finished with a clean tactical sequence 21...Ng5, 22...Bxf3+, 24...Nf3+ winning material. Game B's opponent was also weak, but Black B's play is markedly more sound than Black A's.", "confidence_valid": true }, { "pair": "pair_05", "game_a": "game_09", "color_a": "white", "game_b": "game_07", "color_b": "black", "rating_a": 2193, "rating_b": 1593, "gap": 600, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.4, "correct": 0, "brier": 0.36, "old_estimate_a": 1500, "old_estimate_b": 1400, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A White played a reasonable Benoni setup but drifted: 12.Bd2/13.Qc1 was passive, 15.Kh2 and 16.Rb1 allowed Nxd2 and the collapse of the queenside pawns (22...Qxa3, 23...Qxc3), then 28.e5/29.g4 opened the king and 32.Rxe7 lost to 32...Qf4+ trading into a hopeless rook ending. Game B Black played solid Slav development, correctly recaptured 8...exd5, equalized cleanly, and spotted the tactic 21...Bc2 winning the exchange; White's 22.Bh5?? blundered the queen, which Black handled accurately. Black in B showed sounder positional play against a fairly competent opponent; White in A was outplayed throughout.", "confidence_valid": true }, { "pair": "pair_06", "game_a": "game_17", "color_a": "black", "game_b": "game_25", "color_b": "black", "rating_a": 1639, "rating_b": 838, "gap": 801, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.8, "correct": 1, "brier": 0.03999999999999998, "old_estimate_a": 1450, "old_estimate_b": 950, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black played a coherent Italian: ...d5 break, ...Bg4, ...Nf4 exchange, ...Rxe4 sac for play; the game ended early (likely resignation/flag) without gross blunders shown. Game B Black played bizarre, incoherent moves from the start (2...Bd7, 4...b5, 5...Na6, 7...Rd8), then after winning material with ...Nc2+ and ...Nxa1 repeatedly misplayed: 19...Qg5 wasted tempi, 24...Rxc3+ gave back a rook, 25...f6/28...e5 weakened the king, and allowed a mating net. Game B White was also very weak (2.e3 Qf3 g4 h3), indicating a low-rated pairing. Black in Game A shows clearly more structured understanding.", "confidence_valid": true }, { "pair": "pair_07", "game_a": "game_14", "color_a": "black", "game_b": "game_08", "color_b": "white", "rating_a": 1188, "rating_b": 1310, "gap": 122, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.55, "correct": 0, "brier": 0.30250000000000005, "old_estimate_a": 1300, "old_estimate_b": 1250, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A Black played a passive setup (1...d6 2...e6 3...h6, 5...Bxd6 6...Be7 retreat) but found reasonable simplifications (10...Bxh4, 11...Bg5) and won material after White's 29.fxe5?? 30.Rg6?? blunder (30...Rxd3). Black also made odd shuffles (15...Rd8, 21...Rd5, 22...Rd8). Game B White built a solid Van Geet/fianchetto setup, but 14.Rfe1? allowed 14...c5 winning the Bb4 and a piece via 15.Bxc5 Ndxc5, then 20.Nf3 Bg4 21.Kg2 Ng5 22.Rd4?? Bxf3+ dropped more material. White B's errors were unforced piece-losing blunders against modest opposition; Black A's play, while passive, contained fewer outright material losses. Slight edge to A.", "confidence_valid": true }, { "pair": "pair_08", "game_a": "game_02", "color_a": "black", "game_b": "game_20", "color_b": "black", "rating_a": 2410, "rating_b": 1903, "gap": 507, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.62, "correct": 1, "brier": 0.1444, "old_estimate_a": 1450, "old_estimate_b": 1350, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black played a sound Ruy Lopez, exploited 9...Nxe4 after White's inaccurate 8.Nd5/9.c3, then found the tactical sequence 18...Bxh4! 19...Qxh4 and the rook invasion 21...Re2 to 25...Rh1#, a crisp conversion against a weak White. Game B Black played a passive Pirc setup (7...Bh8, 15...Nf7, 16...Nd7) and allowed a known mating pattern; 25...Kf7?? and 26...Nxh7 walked into Rxh7/Qxh7 and total collapse, and the queenside counterplay (24...b3, 26...a4) was far too slow. Game B's White showed decent attacking technique, but Black's defensive understanding was weak. Game A Black looks the more competent player.", "confidence_valid": true }, { "pair": "pair_09", "game_a": "game_12", "color_a": "black", "game_b": "game_03", "color_b": "black", "rating_a": 1811, "rating_b": 1901, "gap": 90, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.42, "correct": 1, "brier": 0.17639999999999997, "old_estimate_a": 1300, "old_estimate_b": 1200, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A Black played a coherent Modern setup and won a pawn with 10...Nxd4, but then collapsed: 14...O-O-O into a half-open b/a file position, 19...f5 and 21...c4 lost material to 22.Bf4+ and 23.Bxc4, and 24...Qc5?? allowed 25.Be5 forking the rooks. Game B Black (Dutch) made positional errors (13...d5 allowing 14.Ne5, 15...Rd8? dropping the bishop to 16.Bxd6, 18...Re8 losing the exchange after Bxf6/Ng6) and lost badly, but the losses were gradual and the opponent played competently; Game A's White was also weak (5.a3, 17.b4), yet Black still fell for simple tactics. Both look low-rated; slight edge to B for longer resistance and more normal piece development.", "confidence_valid": true }, { "pair": "pair_10", "game_a": "game_23", "color_a": "black", "game_b": "game_09", "color_b": "black", "rating_a": 564, "rating_b": 2177, "gap": 1613, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.12, "correct": 1, "brier": 0.0144, "old_estimate_a": 1300, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black played a speculative, unsound 13...Bxh3 sacrifice after 12...Be7 (allowing the pawn loss), then dropped more material with 15...Bh4 and resigned by move 24 with a hopeless position — typical of a lower-rated player chasing cheap tactics. Game B Black handled a Benoni-style structure with purpose (7...Na6-c7, 10...b5, 13...Ng4-e5-c4 to win the bishop pair), then converted with precise tactics: 19...Nd4, 22...Qxa3, the 32...Qf4+ trade into a won ending, and a clean mating net with 38...Rdd2, 39...Rh2+, 41...f3. Game B's Black shows far superior strategic and tactical competence.", "confidence_valid": true }, { "pair": "pair_11", "game_a": "game_21", "color_a": "black", "game_b": "game_10", "color_b": "black", "rating_a": 743, "rating_b": 2501, "gap": 1758, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.3, "correct": 1, "brier": 0.09, "old_estimate_a": 1000, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black won a very weak opponent's gambit (2.Qd3, 5.Qxd5 losing material) but then played erratically: 23...Nxg2 and 24...Nf4 followed by 25...Bxa4?? and 27...Nh6 lost pieces, and the knight on h1 was stranded for 30+ moves in a sloppy endgame. Game B Black handled a mainstream Catalan/Réti setup soundly (7...c5, 11...Rc8, 15...Ne6), then found 23...Bc5 and 24...Qc2 25...Ba6 winning the exchange with a crushing attack, forcing resignation after 27...Ne8. The opponent in B also played a coherent, higher-level game. B's play is markedly more consistent and tactically sharp.", "confidence_valid": true }, { "pair": "pair_12", "game_a": "game_05", "color_a": "black", "game_b": "game_06", "color_b": "black", "rating_a": 1867, "rating_b": 1382, "gap": 485, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.42, "correct": 0, "brier": 0.3364000000000001, "old_estimate_a": 1250, "old_estimate_b": 1050, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black played a coherent Sicilian with ...b5/...c4 harassing the bishop, then exploited White's loose play with Nf4/Nxg2+ tactics and a clean mating attack; but White in A was very weak (Bc4/Bd3/Be2 shuffles, Be4 hanging to Nxg2+). Game B Black faced a slightly more organized opponent (Nc3, Qc2, Bb2 setup) and converted via ...Bxg5, ...Ncxb3 winning the exchange, ...Bxc4, and the final ...Rc1+ ...Qf2# mating net, though ...Bf5/Bd7/Bg4/Be6 bishop wandering and ...Bxg5 were loose. Both are similar club level; B's calculation of multiple material-winning tactics against a less cooperative opponent gives a slight edge.", "confidence_valid": true }, { "pair": "pair_13", "game_a": "game_22", "color_a": "white", "game_b": "game_04", "color_b": "white", "rating_a": 717, "rating_b": 2508, "gap": 1791, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.3, "correct": 1, "brier": 0.09, "old_estimate_a": 1000, "old_estimate_b": 1450, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White played 2.Qf3 and 3.g4 against the Alekhine, then walked the king to e3 after 6...Nxf2; the win came only because Black (Game A) blundered pieces with 11...Nb5+?, 13...Qf2+ and 21...Qg5?? dropping the queen. Both A players look weak. Game B White played a coherent Caro-Kann Exchange setup (4.Bd3, 5.c3, 6.Ne2, 7.O-O), a sensible queenside pawn storm (14.b4, 16.a4, 20.a5, 22.b5), and lost to a sharp smothered-mate combination (24...Ne2+, 25...Qg1+) by a competent opponent. Game B White's structured play and stronger opposition suggest a higher rating than A's chaotic opening.", "confidence_valid": true }, { "pair": "pair_14", "game_a": "game_18", "color_a": "black", "game_b": "game_14", "color_b": "white", "rating_a": 2641, "rating_b": 1249, "gap": 1392, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.6, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 1650, "old_estimate_b": 1250, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black played a reasonable Sicilian setup, handled 10.Re1+ Kf8 by going for a kingside attack (11...h5, 14...h4, 16...Bh3) and converted tactically with 29...Re8, 30...Re2+ and 32...Qh3+ winning material, though 20...Ng8 and 22...Be5 were loose. Game B White's play was weaker: 2.Bc4 and 4.e5 sideline, 8.Nh4?/9.Qf3 lost a piece to 9...Nd5 10.Bxd5 Bxh4, then aimless knight shuffling (17.Nd6 18.Ne4) and finally 30.Rg6?? dropping the bishop to 30...Rxd3. Opponents in both games were modest, but A's target showed better tactical awareness.", "confidence_valid": true }, { "pair": "pair_15", "game_a": "game_05", "color_a": "white", "game_b": "game_20", "color_b": "white", "rating_a": 1120, "rating_b": 1932, "gap": 812, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.12, "correct": 1, "brier": 0.0144, "old_estimate_a": 1000, "old_estimate_b": 1600, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White played an incoherent opening (2.Bc4, 5.Bd3, 6.Be2, 7.d3 losing tempi), then 15.c4?? and 16.Be4?? allowed 16...Nxg2+ and a collapsing position, finishing with 27.Qxd3 walking into mate by 29...Qxf3#. Game B White executed a thematic Pirc/150 Attack plan: 7.Bh6, 9.g4, 11.h4, 12.h5, 13.hxg6, then 17.f4 Nxh6 18.Rxh6, doubled rooks 22.Rdh1, and converted cleanly with 26.Rxh7 Nxh7 27.Qxh7 and a precise mating net ending in 38.Nh6#. Game B White showed clear opening knowledge and tactical calculation; Game A White showed neither.", "confidence_valid": true }, { "pair": "pair_16", "game_a": "game_11", "color_a": "black", "game_b": "game_02", "color_b": "white", "rating_a": 1273, "rating_b": 1575, "gap": 302, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.4, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 1150, "old_estimate_b": 1250, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black opened with 1...e5 2...Bc5 3...Qf6 (dubious, queen out early), then shuffled the queen (Qb6, Qc5, Qa5+, Qc7) and later allowed an elementary back-rank mate with 41...Re8 42.Rxe8#, having also lost material to tactics like 36...Rxc4 37.Rab1. Game B White played a coherent Ruy Lopez with Nd5/c3/d4 ideas, but blundered a pawn to 9...Nxe4 and then hung material with 18.h4?? Bxh4, collapsing afterward. Both are weak, but Game B White's opening knowledge and structure look more developed than Game A Black's queen-wandering and back-rank oversight; slight edge to B.", "confidence_valid": true }, { "pair": "pair_17", "game_a": "game_13", "color_a": "white", "game_b": "game_16", "color_b": "white", "rating_a": 1126, "rating_b": 1752, "gap": 626, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.25, "correct": 1, "brier": 0.0625, "old_estimate_a": 1150, "old_estimate_b": 1450, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White played very oddly: 3.dxc5 then 4.Bd2 5.Be3 wasting tempi, allowed 6...Qxb2, then 8.Rb3, 10.Qa1, 10.Nb1 and 13.Ra3/14.Rc3/16.Rc5?? dropping an exchange, and finally allowed 35...Rc1# back-rank mate. Black in A was also weak (early queen hunting). Game B White handled the Bxf2+ sacrifice calmly (7.Kxf2, 8.Rf1, 9.Kg1), won material cleanly with 12.h3/13.hxg4, consolidated with Qf2/Rae1 and converted via 25.Qe2 trade and 27.Re6, with the opponent resigning. Game B White showed solid, error-free play; Game A White showed basic tactical blindness.", "confidence_valid": true }, { "pair": "pair_18", "game_a": "game_24", "color_a": "black", "game_b": "game_12", "color_b": "white", "rating_a": 765, "rating_b": 1877, "gap": 1112, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.15, "correct": 1, "brier": 0.0225, "old_estimate_a": 900, "old_estimate_b": 1450, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black played a chaotic, non-developing opening (3...c6, 4...b5, 5...a5, 7...f6, 8...h5, 9...g5) and then hung a knight with 16...Bg4 (17.Nxf6+), lost a rook to 20.Nb6 and finally 22...Qd8?? dropping the queen — very weak, beginner-level play. Game B White showed decent fundamentals: 7.h3 provoking Bxf3, recognizing 10.Be3 was fine after Nxd4 since 12.Bb5+ regained material, then exploited Black's loose king with 20.bxc5, 22.Bf4+, 25.Be5 and finished with 28.Qg8+ winning. White in B made some inaccuracies but clearly played coherent chess, far above A's target.", "confidence_valid": true }, { "pair": "pair_19", "game_a": "game_17", "color_a": "white", "game_b": "game_10", "color_b": "white", "rating_a": 1514, "rating_b": 2689, "gap": 1175, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.38, "correct": 1, "brier": 0.1444, "old_estimate_a": 1500, "old_estimate_b": 1500, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "Game A White played 2.Bc4 then 6.a3 and the aimless 10.h3/11.g4/12.Neg5/13.Ne4 knight shuffle, and 24.Rd5 (Black can take with ...cxd5) looks like a blunder; the win came from Black resigning abruptly, not strong play. Game B White handled a Catalan/Reti structure with coherent plans (8.cxd5, 12.dxc5, 13.Nd4, 15.Qf4 pressuring d5), only collapsing after 23.Qc7?/24.Nxa7 allowed 24...Qc2 and 25...Ba6 winning the exchange. Game B's positional understanding and opponent quality suggest a higher rating despite the loss.", "confidence_valid": true }, { "pair": "pair_20", "game_a": "game_11", "color_a": "white", "game_b": "game_23", "color_b": "white", "rating_a": 1228, "rating_b": 557, "gap": 671, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "B", "probability_a": 0.4, "correct": 0, "brier": 0.36, "old_estimate_a": 1250, "old_estimate_b": 1450, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game A White faced a weak opponent (1...e5 2.d5 Bc5, Qf6/Qb6 wandering) and played loosely: 7.Ne4 lost the d5 pawn, 19.Qh4 allowed Bxf3 wrecking the kingside, 34.Bb6?/35.Bc7? dropped the c4 pawn, and the win came only via Black's blunder 41...Re8?? allowing Rxe8#. Game B White played a coherent London setup, calmly refuted Black's 13...Bxh3 sacrifice with 15.Nxf5 and 16.Nxh4 winning two pieces, then consolidated (19.Re1, 20.Qe2, 23.Rg1) forcing resignation. Game B's opponent also played more reasonably (Nf6, Bf5, castling), suggesting a higher rating pool. B shows sounder technique.", "confidence_valid": true }, { "pair": "pair_21", "game_a": "game_21", "color_a": "white", "game_b": "game_15", "color_b": "white", "rating_a": 739, "rating_b": 2157, "gap": 1418, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.1, "correct": 1, "brier": 0.010000000000000002, "old_estimate_a": 850, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White played 2.Qd3 and 5.Qxd5?? 6.fxe5?? 7.Qxb7?? grabbing pawns while undeveloped, then lost the whole queenside and later a rook to knight forks; endgame technique was aimless (Nf3/Ng5 shuffling) and the opponent also blundered badly (e.g. 8.Qxb4 trade allowed, missed wins), suggesting a low-rated pairing. Game B White handled a Slav structure competently, won material with 15.Ncxa5 and 16.Nb7, reached a favorable ending with 25.Qxc8+ 26.Rxd5 liquidation, and even when outplayed in the king-pawn ending found the drawing resource 49.e5 and held a defensive stalemate pattern. Game B White is clearly the stronger player.", "confidence_valid": true }, { "pair": "pair_22", "game_a": "game_04", "color_a": "black", "game_b": "game_13", "color_b": "black", "rating_a": 2718, "rating_b": 1551, "gap": 1167, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.6, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 1600, "old_estimate_b": 1300, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black played a coherent Caro-Kann exchange setup (6...Bg4, 10...Bxe2 trading the bishop, 16...Ne4, 17...f5) and finished with a clean 25...Qg1+ 26.Rxg1 Nf2# smothered mate; White's play (Qe1, Bg5-h4-e1) was weak but not absurd. Game B Black grabbed material with 6...Qxb2 and won, but allowed a repeatedly harassed queen and lost an exchange to 28.Ne8/29.Nc7 (28...Re6?), winning only because White blundered terribly (Rc5?, Nxb5?? allowing Rc1#). White in Game B was very weak (Qa1, Rb3-a3-c3-c5), suggesting a lower-rated pairing. Game A's opposition and Black's technique look somewhat higher-level.", "confidence_valid": true }, { "pair": "pair_23", "game_a": "game_25", "color_a": "white", "game_b": "game_18", "color_b": "white", "rating_a": 738, "rating_b": 2426, "gap": 1688, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.3, "correct": 1, "brier": 0.09, "old_estimate_a": 1100, "old_estimate_b": 1550, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White played a bizarre, unsound opening (2.e3, 3.Qf3, 4.g4, 5.h3, 6.h4, 7.g5, 8.h5), lost the exchange and rook on a1 to 13...Nc2+, and only won because Black blundered badly (23...Qc8?? losing the queen and later Rxc3+ for nothing). The opponent's play (2...Bd7, 3...c6, 4...b5, 5...Na6) indicates a very low-rated pairing. Game B White showed coherent development (b3/g3 fianchetto, 10.Re1+ Kf8, 11.Nxd4, 14.c4, 21.cxd5 c5) and was even better before 29.Qd4?? allowed 29...Re8 and 30...Re2+ with a losing attack. Despite losing, Game B White's overall move quality and opponent's Sicilian handling suggest a higher established rating than Game A's target.", "confidence_valid": true }, { "pair": "pair_24", "game_a": "game_03", "color_a": "white", "game_b": "game_15", "color_b": "black", "rating_a": 1782, "rating_b": 2154, "gap": 372, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.4, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 1450, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White plays a loose London setup with speculative 8.h4 and 9.Ng5/11.Nh3 shuffling, then 14.Ne5/15.Ng6 tactics work only because Black (Game A) blundered with 14...Bd6?? and 16...Kf7; White's conversion (24.Qb3, 37.Rd8) is competent but the opponent's level was low. Game B Black handles a theoretical Slav line (3...dxc4, 4...b5, 8...a5, 11...Ba6), keeps material balance through complex trades (15...Qd5, 16...Rxa4, 24...Nxb7 25.Qxc8+ Nxc8), and plays a long, accurate king-and-pawn endgame, though 56...h4?? spoiled a win with ...Kf2/...h3 ideas earlier. Overall Black in B shows more structured, sound chess against a stronger opponent.", "confidence_valid": true }, { "pair": "pair_25", "game_a": "game_19", "color_a": "black", "game_b": "game_07", "color_b": "white", "rating_a": 2474, "rating_b": 1862, "gap": 612, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.6, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 1400, "old_estimate_b": 1250, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game B White blundered a full exchange tactically with 21. Bf3?? allowing 21...Bc2 forking queen and rook, then resigned early after 23...Bxb3 — a clear tactical oversight by a lower-level player, though earlier play (8.cxd5, 10.b3, 17.b4 minority plan) was coherent. Game A Black lost a long endgame after a sound opening (6...Bf5 7...Bg6, trading into a doubled-pawn structure), with errors like 22...c5 and 29...Nd8 letting the h-pawn and d5 break decide; no single piece-losing blunder. Both appear modest, but Game A Black's play was more consistent and lacked a one-move tactical collapse.", "confidence_valid": true } ] }, { "model": "google/gemini-3-flash-preview", "correct": 20, "n": 25, "accuracy": 80.0, "baseline_accuracy": 82.0, "invalid_confidence": 0, "consistent_reversals": 5, "reversals": [ { "pair": "pair_09_reverse", "original": "pair_09", "prediction": { "stronger": "B", "probability_a_stronger": 0.15, "explanation": "Player B (Black) demonstrates significantly higher tactical competence. They successfully navigated a complex opening to win a pawn with 10...Nxd4 and found the strong active resource 23...Rd4. While they eventually lost to a back-rank tactic after 27...Nxe3, their play was coordinated and purposeful. In contrast, Player A (Black) showed poor board vision. They blundered a piece to the simple 16.Bxd6 (after 16...Kf7??) and failed to mount any resistance against White's knights. The game A sequence ending in 33...Nc8?? (allowing 34.Ne7+) suggests a much lower level of tactical awareness compared to Player B's middle-game play." }, "consistent": true }, { "pair": "pair_15_reverse", "original": "pair_15", "prediction": { "stronger": "A", "probability_a_stronger": 0.95, "explanation": "Target A (White) demonstrated a sophisticated understanding of the Yugoslav-style attack against the Modern Defense. Moves like 7. Bh6, 12. h5, and the thematic exchange sacrifice 26. Rxh7! followed by the queen sacrifice 29. Qxh8+ show high-level calculation and positional aggression. In contrast, Target B (White) displayed poor tactical awareness. 16. Be4? hung a central pawn with check (16... Nxg2+), and 17. Kf1? immediately blundered a piece to 17... fxe4. Target B's collapse was rapid and characterized by basic errors, whereas Target A conducted a clinical attack and endgame transition." }, "consistent": true }, { "pair": "pair_13_reverse", "original": "pair_13", "prediction": { "stronger": "A", "probability_a_stronger": 0.85, "explanation": "Player A demonstrates a solid understanding of opening principles and positional play in the Caro-Kann Exchange (1-8. Qe1). They execute a coherent plan with 14. b4 and 18. Nc5, only losing to a spectacular tactical oversight leading to smothered mate. In contrast, Player B plays a highly irregular and dubious opening (2. Qf3, 3. g4) and wanders their king to the center early (9. Ke3). While Player B wins, the quality of play is much lower, characterized by fundamental violations of opening theory and numerous tactical errors by both sides that Player A avoided until the final blunder." }, "consistent": true }, { "pair": "pair_12_reverse", "original": "pair_12", "prediction": { "stronger": "B", "probability_a_stronger": 0.35, "explanation": "Player B demonstrates sophisticated opening preparation against the Bowdler Attack, utilizing 4...b5 and 5...c4 to seize space and 11...f5! to challenge the center. Black's coordination is exemplary, specifically the knight maneuvers to f4 and d3 (15...Nf4, 20...Nb4, 21...Nbd3) and the precise tactical sequence starting with 25...Nh3+. While Player A efficiently exploited White’s eccentric and passive setup (4. Bd3, 6. Bb1) with strong centralization (13...Nd4, 17...Nc5), the level of resistance was significantly lower. Player B’s systematic dismantling of a more conventional defense and superior theoretical application suggest a higher established rating." }, "consistent": true }, { "pair": "pair_18_reverse", "original": "pair_18", "prediction": { "stronger": "A", "probability_a_stronger": 0.9, "explanation": "White (A) demonstrates solid tactical vision and opening principles. White’s sequence 11. Nxd4 Bxd4 12. Bb5+ is a clever tactical intermezzo to recover a piece and win an exchange. Throughout the game, White coordinates pieces well and finds a clean finish with 28. Qg8+. Conversely, Black (B) plays a highly dubious opening, prioritizing weakening pawn pushes (7... f6, 8... h5, 9... g5) over development. Black (B) exhibits poor board vision, culminating in the catastrophic one-move blunder of the queen with 22... Qd8. Player A’s ability to handle tactics and convert a winning position suggests a much higher established rating than Player B's erratic and blunder-prone play." }, "consistent": true } ], "rows": [ { "pair": "pair_01", "game_a": "game_16", "color_a": "black", "game_b": "game_19", "color_b": "white", "rating_a": 1569, "rating_b": 2668, "gap": 1099, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.1, "correct": 1, "brier": 0.010000000000000002, "old_estimate_a": 1400, "old_estimate_b": 2200, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Target B demonstrates a much higher level of positional understanding and endgame technique. In Game B, White navigates a Queen's Gambit Exchange structure expertly, utilizing the h-pawn and knight maneuvers (11. Nh3, 16. Nhf4) to exploit Black's doubled pawns. White converts the technical endgame efficiently with 32. Re1 and 37. h7. In contrast, Target A (Black) plays a dubious opening (2... e6) and an unsound piece sacrifice (6... Bxf2+). After the initial aggression fails, Black lacks positional compensation and eventually collapses, ending with the blunder 27... Nc7 which loses the knight on d5 to the rook. The disparity in strategic depth and tactical stability clearly favors Target B.", "confidence_valid": true }, { "pair": "pair_02", "game_a": "game_01", "color_a": "black", "game_b": "game_24", "color_b": "white", "rating_a": 1159, "rating_b": 762, "gap": 397, "gap_band": "200-499", "low_band": "one<1000", "gold": "A", "choice": "B", "probability_a": 0.35, "correct": 0, "brier": 0.42250000000000004, "old_estimate_a": 1400, "old_estimate_b": 1300, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Target B shows better fundamental understanding and tactical alertness. In Game B, White develops logically (4. Bc4, 13. O-O) and exploits Black's blunders, such as the fork with 20. Nb6 and the hanging queen 23. Bxd8. In contrast, Target A (Black) in Game A commits a severe tactical oversight by allowing the simple fork 16. Nf6+, losing the queen. Although Black eventually wins, it is primarily due to White's even more egregious blunder (40. Qb8+), which hangs the queen for nothing. Target B's play is significantly more stable and demonstrates a higher floor of tactical competence than Target A.", "confidence_valid": true }, { "pair": "pair_03", "game_a": "game_01", "color_a": "white", "game_b": "game_06", "color_b": "white", "rating_a": 1363, "rating_b": 1112, "gap": 251, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.7, "correct": 1, "brier": 0.09000000000000002, "old_estimate_a": 1350, "old_estimate_b": 1250, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Player A shows superior tactical awareness and opening knowledge compared to Player B. In Game A, White executes a strong tactical sequence beginning with 15. Nd5 and 16. Nf6+ to win the Queen for two pieces. While Player A ultimately collapses in the endgame, blundering the Queen back with 40. Qb8+, their play was coherent until the time pressure phase. Player B, by contrast, plays a nonsensical opening (4. Bd3, 6. Bb1) and shows poor tactical vision, allowing a simple knight fork on a1 (19. f5) and failing to defend against a basic mating sequence. Player A’s peak level within the game is significantly higher.", "confidence_valid": true }, { "pair": "pair_04", "game_a": "game_22", "color_a": "black", "game_b": "game_08", "color_b": "black", "rating_a": 557, "rating_b": 1791, "gap": 1234, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 850, "old_estimate_b": 1600, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black displays extremely low-level play, losing two minor pieces to simple captures (13. Qxb5 and 17. Qxc5) and hanging their queen with 22... Qxg1+. In contrast, Game B Black follows sound opening principles (2... d5, 5... O-O) and demonstrates tactical competence. Black exploits the pin on the f3 knight with 9... e4 and 21... Ng5, wins a piece with the pawn fork 14... c5, and finishes the game with the fork 24... Nf3+. Game B Black's play is consistent with a much higher rating, whereas Game A Black lacks basic board vision and coordination.", "confidence_valid": true }, { "pair": "pair_05", "game_a": "game_09", "color_a": "white", "game_b": "game_07", "color_b": "black", "rating_a": 2193, "rating_b": 1593, "gap": 600, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.3, "correct": 0, "brier": 0.48999999999999994, "old_estimate_a": 1800, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Target B showed superior tactical vision and efficiency. The maneuver 18... Qe6 and 20... Qg6 prepared a clever tactical shot, 21... Bc2, which won the exchange and forced resignation. B's play was solid and purposeful throughout the Slav structure. Target A, playing White, struggled with coordination in a Benoni. Allowing the loss of the bishop pair with 16. Rb1 and failing to handle the pressure on the kingside (culminating in the 39-41 sequence) suggests a lower level of tactical and positional awareness compared to Target B. A's 13. Qc1 and 19. f4 were also somewhat imprecise, leading to a disadvantageous position.", "confidence_valid": true }, { "pair": "pair_06", "game_a": "game_17", "color_a": "black", "game_b": "game_25", "color_b": "black", "rating_a": 1639, "rating_b": 838, "gap": 801, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.9, "correct": 1, "brier": 0.009999999999999995, "old_estimate_a": 1800, "old_estimate_b": 1050, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black demonstrates sound opening principles (3... Bc5, 6... d5) and maintains a balanced, coordinated position throughout the middlegame. In contrast, Game B Black plays an erratic opening (2... Bd7, 5... Na6) and shows poor tactical awareness. Despite winning a rook early, Game B Black collapses with unnecessary exchanges (24... Rxc3+) and fails to defend a simple mating threat, ultimately allowing 39. Rxd8#. Player A's play shows a significantly higher level of foundational chess knowledge and calculation compared to Player B's chaotic performance, which is characterized by uncoordinated development and severe blunders under pressure.", "confidence_valid": true }, { "pair": "pair_07", "game_a": "game_14", "color_a": "black", "game_b": "game_08", "color_b": "white", "rating_a": 1188, "rating_b": 1310, "gap": 122, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.65, "correct": 0, "brier": 0.42250000000000004, "old_estimate_a": 1300, "old_estimate_b": 1300, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "Target A (Black) demonstrated solid fundamentals, winning material early (10... Bxh4) and maintaining a clear advantage through the endgame. While Black missed a winning tactical opportunity on move 15, their play remained cohesive and focused. Conversely, Target B (White) displayed poor tactical awareness, blundering a full piece to a basic pawn fork (14... c5). Target B subsequently failed to manage the pin on the f3-knight, allowing the decisive fork 24. Nf3+. The disparity in tactical oversight suggests Target A is the stronger player; Target B's loss of material to a simple one-move threat indicates a lower level of established play compared to Target A's overall stability.", "confidence_valid": true }, { "pair": "pair_08", "game_a": "game_02", "color_a": "black", "game_b": "game_20", "color_b": "black", "rating_a": 2410, "rating_b": 1903, "gap": 507, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.65, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1800, "old_estimate_b": 1950, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Player A (Black) played a precise and disciplined game, efficiently punishing White's inaccuracies. Moves like 9...Nxe4 and 14...Nc4 demonstrate strong tactical awareness, while the infiltration with 21...Re2 and 23...Rb3 showed high technique in converting an advantage. In contrast, Player B (Black) struggled to handle White’s aggressive kingside storm in a Modern Defense. Allowing the h-file to open with 17...Nxh6 and failing to generate sufficient counter-pressure led to a rapid collapse. While Player B faced a strong attack, Player A's clean execution and tactical alertness suggest a more established skill level.", "confidence_valid": true }, { "pair": "pair_09", "game_a": "game_12", "color_a": "black", "game_b": "game_03", "color_b": "black", "rating_a": 1811, "rating_b": 1901, "gap": 90, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.58, "correct": 0, "brier": 0.3364, "old_estimate_a": 1900, "old_estimate_b": 1700, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Player A (Black) shows better tactical resourcefulness and coordination despite losing a piece to a common discovery (12. Bb5+). Their counterplay, particularly with 23... Rd4 and 25... Rd2, forced White to find precise defensive moves. In contrast, Player B (Black) mismanaged the pressure in the Dutch Defense, committing a severe positional and safety error with 16... Kf7, which led to a passive and structural collapse. While both players suffered from tactical oversights (the back-rank mate in Game A and the exchange loss in Game B), Player A's ability to generate complications while down material indicates a slightly higher level of established strength compared to Player B's more passive deterioration.", "confidence_valid": true }, { "pair": "pair_10", "game_a": "game_23", "color_a": "black", "game_b": "game_09", "color_b": "black", "rating_a": 564, "rating_b": 2177, "gap": 1613, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.1, "correct": 1, "brier": 0.010000000000000002, "old_estimate_a": 1100, "old_estimate_b": 2050, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Target B demonstrates a clear understanding of Benoni structures, utilizing effective maneuvering (9... Nc7, 14... Ne5, 15... Nc4) and tactical alertness to secure a positional advantage. The conversion of the endgame was precise, culminating in a clean mating net (41... f3). In contrast, Target A displayed significant tactical weaknesses in Game A. They hung a pawn with 11... e5 and failed to capitalize on White's blunder 15. Nxf5, instead playing 15... Bh4?? which allowed White to save the knight. Target A's speculative sacrifice 13... Bxh3 was unsound and lacked follow-through, indicating a much lower level of tactical and positional awareness compared to Target B.", "confidence_valid": true }, { "pair": "pair_11", "game_a": "game_21", "color_a": "black", "game_b": "game_10", "color_b": "black", "rating_a": 743, "rating_b": 2501, "gap": 1758, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.15, "correct": 1, "brier": 0.0225, "old_estimate_a": 1300, "old_estimate_b": 2300, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B Black demonstrates significantly higher skill, playing a sophisticated English defense with sound development (10... Bb7, 14... Rfe8). Black's tactical execution is precise, notably 24... Qc2! which wins material by targeting the loose rook and bishop. In contrast, Game A Black plays an erratic opening (3... Qe7, 4... f6) and succeeds largely because White blunders a piece (21. Kd2? allowing Nxf1+). The quality of calculation and opening knowledge in Game B suggests a much higher established rating than the scrappy, blunder-prone play seen in Game A.", "confidence_valid": true }, { "pair": "pair_12", "game_a": "game_05", "color_a": "black", "game_b": "game_06", "color_b": "black", "rating_a": 1867, "rating_b": 1382, "gap": 485, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.65, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1600, "old_estimate_b": 1700, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Black in Game A demonstrates superior opening knowledge and tactical precision. Black plays a thematic Sicilian setup and immediately punishes White's inaccuracies (13. b3 and 16. Be4) with the powerful 15... Nf4 and 16... Nxg2+. The subsequent coordination of the knights (20... Nb4, 21... Nbd3) and the clinical finish (25... Nh3+ and 27... Be4+) show a high level of proficiency. In Game B, Black plays more passively with 3... Bd7 and 10... Bd7. While Black B exploits White's eccentric play (6. Bb1, 8. h4) well with 13... Nd4, the overall level of theoretical accuracy and calculated aggression in Game A suggests a stronger established player.", "confidence_valid": true }, { "pair": "pair_13", "game_a": "game_22", "color_a": "white", "game_b": "game_04", "color_b": "white", "rating_a": 717, "rating_b": 2508, "gap": 1791, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.15, "correct": 1, "brier": 0.0225, "old_estimate_a": 900, "old_estimate_b": 1950, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White shows a lack of fundamental principles, developing the queen prematurely (2. Qf3), pushing flank pawns (3. g4), and walking the king to the center early (9. Ke3) in a chaotic sequence. In contrast, Game B White plays a standard Exchange Caro-Kann with logical development (4. Bd3, 6. Ne2, 12. Nd2) and positional awareness, establishing a strong outpost with 18. Nc5. Although Game B White loses to a classic smothered mate tactic (24. Bb4?? Ne2+), their overall opening knowledge and structural play suggest a significantly higher established rating than Game A White.", "confidence_valid": true }, { "pair": "pair_14", "game_a": "game_18", "color_a": "black", "game_b": "game_14", "color_b": "white", "rating_a": 2641, "rating_b": 1249, "gap": 1392, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.85, "correct": 1, "brier": 0.022500000000000006, "old_estimate_a": 1900, "old_estimate_b": 1250, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Player A (Black) demonstrates sophisticated tactical and attacking skills in a sharp Sicilian. Navigating a king-in-center position after 10...Kf8, they correctly prioritized a kingside initiative (11...h5, 14...h4). The concluding sequence (30...Re2+ to 33...Qxg3+) shows precise calculation. In contrast, Player B (White) played a more aimless middlegame, exemplified by the unnecessary 10.Bxd5. Crucially, Player B committed a primitive tactical error with 30.Rg6, simply leaving the d3-bishop en prise to 30...Rxd3. The difference in tactical alertness and strategic depth between the two players is significant.", "confidence_valid": true }, { "pair": "pair_15", "game_a": "game_05", "color_a": "white", "game_b": "game_20", "color_b": "white", "rating_a": 1120, "rating_b": 1932, "gap": 812, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1250, "old_estimate_b": 2050, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Player A demonstrates poor opening fundamentals, wasting tempi with 2. Bc4 and 5. Bd3, and then commits a massive tactical blunder with 16. Be4?, losing a piece to 16... Nxg2+ and 17... fxe4. Player A's defense is non-existent, leading to a quick mate. In contrast, Player B plays a sophisticated, thematic kingside attack against the Pirc Defense. White correctly executes the h-pawn storm (11. h4), coordinates pieces well (14. Nce2), and finds a decisive exchange sacrifice (26. Rxh7!). Player B's ability to simplify into a won endgame (29. Qxh8+) and finish with a clean mate (38. Nh6#) indicates a significantly higher established playing strength.", "confidence_valid": true }, { "pair": "pair_16", "game_a": "game_11", "color_a": "black", "game_b": "game_02", "color_b": "white", "rating_a": 1273, "rating_b": 1575, "gap": 302, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.4, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 1400, "old_estimate_b": 1600, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Target B (White) demonstrates a more mature understanding of opening principles, navigating a main-line Ruy Lopez (5. Nc3, 7. O-O) and coordinating pieces logically. Despite the catastrophic one-move blunder (25. Bg5?? allowing 25... Rh1#), the prior play was coherent. In contrast, Target A (Black) chose a dubious opening (1... e5 vs 1. d4) and an early queen development (3... Qf6) that typically characterizes lower-rated play. Target A also blundered a back-rank mate (41... Re8#) in a relatively simple endgame. Target B’s sounder development and opening knowledge suggest a higher established rating.", "confidence_valid": true }, { "pair": "pair_17", "game_a": "game_13", "color_a": "white", "game_b": "game_16", "color_b": "white", "rating_a": 1126, "rating_b": 1752, "gap": 626, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.15, "correct": 1, "brier": 0.0225, "old_estimate_a": 1050, "old_estimate_b": 1600, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Target B shows significantly better tactical awareness and fundamental coordination. In Game A, White loses a pawn early (6... Qxb2), allows unnecessary pressure, and ultimately blunders a simple back-rank mate with 35. Nxb5??. In contrast, Target B handles an unsound sacrifice (6. Be2 Bxf2+) with composure, maintains the material advantage through clean development (17. Rae1, 26. Rxe2), and forces a resignation through solid positional play. Target A's failure to see the back-rank threat is a strong indicator of a lower rating compared to Target B's steady conversion.", "confidence_valid": true }, { "pair": "pair_18", "game_a": "game_24", "color_a": "black", "game_b": "game_12", "color_b": "white", "rating_a": 765, "rating_b": 1877, "gap": 1112, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.15, "correct": 1, "brier": 0.0225, "old_estimate_a": 1150, "old_estimate_b": 2200, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Target B (White) demonstrates superior tactical awareness and positional stability. In Game B, White calculates a favorable exchange sequence starting with 12. Bb5+ and finishes the game with a clean back-rank mate (28. Qg8+). Target A (Black), conversely, plays an extremely loose and weakening opening with 7...f6 and 9...g5, creating numerous holes in their position. Furthermore, Target A blunders their queen in a single move with 22...Qd8, allowing 23. Bxd8. While Target A shows some tactical aggression (16...Bg4), their lack of basic safety and the catastrophic blunder at the end suggest a lower established rating compared to Target B's coordinated play.", "confidence_valid": true }, { "pair": "pair_19", "game_a": "game_17", "color_a": "white", "game_b": "game_10", "color_b": "white", "rating_a": 1514, "rating_b": 2689, "gap": 1175, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.35, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1850, "old_estimate_b": 2150, "baseline_choice": "B", "baseline_credit": 1, "explanation": "White B demonstrates sophisticated positional play in an English/Catalan structure (1. c4, 3. g3, 9. d4), showing high-level opening knowledge. The coordination and maneuvering (13. Nd4, 15. Qf4, 20. Nb5) suggest a strong established rating. White A plays a standard Italian with a somewhat aggressive but loosening kingside expansion (11. g4). While White A capitalized on Black's late blunder (24... c6) to win, the strategic depth in Game B is more characteristic of a higher-rated player. Even with White B's tactical oversight (24. Nxa7), the overall quality of move selection in the first 20 moves points towards B having a higher established Lichess rapid rating.", "confidence_valid": true }, { "pair": "pair_20", "game_a": "game_11", "color_a": "white", "game_b": "game_23", "color_b": "white", "rating_a": 1228, "rating_b": 557, "gap": 671, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "B", "probability_a": 0.35, "correct": 0, "brier": 0.42250000000000004, "old_estimate_a": 1400, "old_estimate_b": 1250, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Target B demonstrates a more solid and principled understanding of the opening (London System) and tactical consolidation. 13. Nd4 in Game B is a strong centralizing move that exploits Black's weaknesses (c6 and f5). Target B also showed good technique by offering a queen trade with 22. Qe5 while a piece up. In contrast, Target A's play was more erratic. Move 14. Bc4 in Game A was a wasted tempo that allowed Black to trade off an influential knight, and White’s kingside structure was unnecessarily shattered (20. gxf3). Target A was technically losing the endgame and only won due to Black’s major back-rank oversight (41. Rd8+), whereas Target B maintained control throughout.", "confidence_valid": true }, { "pair": "pair_21", "game_a": "game_21", "color_a": "white", "game_b": "game_15", "color_b": "white", "rating_a": 739, "rating_b": 2157, "gap": 1418, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1100, "old_estimate_b": 2150, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Target A demonstrates a lack of basic opening principles and tactical awareness, playing 2. Qd3 and later hanging a piece to a simple sequence (20. Kd1 Ne3+ 21. Kd2 Nxf1+). The endgame play in Game A is erratic, with multiple blunders from both sides. In contrast, Target B shows a solid grasp of opening theory in a complex Slav line (3. Nc3 dxc4 4. e3 b5 5. a4). Target B handles a delicate positional struggle, accurately navigates a queen trade at move 25, and demonstrates precise knowledge of the drawn king-and-pawn endgame. The difference in technical proficiency between the two players is substantial.", "confidence_valid": true }, { "pair": "pair_22", "game_a": "game_04", "color_a": "black", "game_b": "game_13", "color_b": "black", "rating_a": 2718, "rating_b": 1551, "gap": 1167, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.85, "correct": 1, "brier": 0.022500000000000006, "old_estimate_a": 2050, "old_estimate_b": 1250, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Target A (Black) exhibits a sophisticated understanding of the Caro-Kann, coordinating a complex kingside attack while managing queenside pressure (16...Ne4, 19...g5). The execution of the classic smothered mate (24...Ne2+ through 26...Nf2#) demonstrates high tactical awareness. Target B (Black) plays a suspicious opening, making multiple early queen moves (3...Qa5+, 5...Qb5, 6...Qxb2) to grab a pawn, a strategy often punished at higher levels. While Target B won via a back-rank mate, it was facilitated by a significant oversight by White (34. Nc3??). Target A’s strategic cohesion and tactical finish suggest a higher established rating.", "confidence_valid": true }, { "pair": "pair_23", "game_a": "game_25", "color_a": "white", "game_b": "game_18", "color_b": "white", "rating_a": 738, "rating_b": 2426, "gap": 1688, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1100, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White plays a primitive opening, making five pawn moves and an early queen sortie (3. Qf3) before developing any minor pieces. White misses a basic fork (13. Nh3 Nc2+), losing a full rook, and only wins because Black fails to convert. Conversely, Game B White exhibits sophisticated positional play in a 3. b3 Sicilian, prioritizing development (9. Bb2, 10. Re1+) and central control. Game B involves high-level themes like the h-pawn thrust and piece maneuvering. Although Game B White blunders at the end (32. Qxd5??), the technical proficiency and opening knowledge displayed are significantly higher than the disorganized play seen from Game A White.", "confidence_valid": true }, { "pair": "pair_24", "game_a": "game_03", "color_a": "white", "game_b": "game_15", "color_b": "black", "rating_a": 1782, "rating_b": 2154, "gap": 372, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.2, "correct": 1, "brier": 0.04000000000000001, "old_estimate_a": 1900, "old_estimate_b": 2150, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Target B shows superior technical skill, navigating a complex Slav Defense mainline (4... b5, 5... b4). Black's maneuvering (14... Nb6) and the thematic 22... c5 break indicate deep theoretical knowledge. The tactical sequence 24... Nxb7 and the flawless transition to a drawn pawn endgame (39... Bxc5, 48... Kd4) suggest a high rating. Conversely, Target A wins against a weak Dutch Defense (1... f5) where Black blunders early (9... Qc8, 15... Rd8). While Target A efficiently capitalizes with 15. Ng6 and 24. Qb3, the overall quality of opposition and the lack of complex positional requirements in Game A make Target B the likely stronger player.", "confidence_valid": true }, { "pair": "pair_25", "game_a": "game_19", "color_a": "black", "game_b": "game_07", "color_b": "white", "rating_a": 2474, "rating_b": 1862, "gap": 612, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.6, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 2100, "old_estimate_b": 1750, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black demonstrates a reasonable understanding of the QGD Exchange structure, employing active king centralization (19... Kd6) and seeking counterplay in a strategically difficult endgame. Although Black ultimately loses to a skewer (35. Ra8+), the game reached a high degree of technical complexity. In contrast, Game B White plays a very passive opening and commits a basic tactical oversight by missing the 21... Bc2 fork, which immediately loses the exchange in a stable, quiet position. The nature of White's blunder in Game B, occurring without significant pressure, suggests a lower established rating compared to the more sophisticated defensive effort shown by Black in Game A.", "confidence_valid": true } ] }, { "model": "google/gemini-3.6-flash", "correct": 20, "n": 25, "accuracy": 80.0, "baseline_accuracy": 86.0, "invalid_confidence": 0, "consistent_reversals": 4, "reversals": [ { "pair": "pair_09_reverse", "original": "pair_09", "prediction": { "stronger": "B", "probability_a_stronger": 0.25, "explanation": "Player B demonstrates a significantly higher level of tactical awareness and dynamic play compared to Player A. In Game B, despite early tactical errors (10...Nxd4 losing a piece to 12. Bb5+), Black found an impressive offensive sequence starting with 21...c4! 22. Bf4+ Ka8 23. Bxc4 Rd4! and 25...Rd2!, creating a winning attack before blundering to a back-rank mate on move 27. In contrast, Player A played passively in the Dutch, blundered with 16...Kf7 and 22...e5, and allowed White to dominate without generating any active counterplay or tactical ideas." }, "consistent": true }, { "pair": "pair_15_reverse", "original": "pair_15", "prediction": { "stronger": "A", "probability_a_stronger": 0.95, "explanation": "Game A White executed a textbook kingside attack in an English Attack setup against the Modern Defense, demonstrating high tactical sharpness with aggressive pushes like 9. g4 and 11. h4, followed by a decisive tactical sequence starting with 26. Rxh7! and 29. Qxh8+!. In contrast, Game B White played poorly in the opening, wasting multiple tempos moving the bishop repeatedly (2. Bc4, 5. Bd3, 6. Be2, 8. Bxd3), creating loose pawns with 13. b3, and blundering material to simple tactics starting with 16. Be4 Nxg2+. Game A White shows far superior strategic planning and tactical calculation." }, "consistent": true }, { "pair": "pair_13_reverse", "original": "pair_13", "prediction": { "stronger": "A", "probability_a_stronger": 0.88, "explanation": "Game A White demonstrates solid opening and positional understanding in an Exchange Caro-Kann, executing standard plans with 4. Bd3, 14. b4, and 18. Nc5. Although Game A White misses Black's sharp tactical combination ending in a smothered mate (24...Ne2+), their overall play reflects a decent positional baseline. In contrast, Game B White displays beginner-level opening play with 2. Qf3, 3. g4, and 4. g5, unnecessarily weakening their kingside and walking their king out to e3 by move 9. Game A White shows far superior fundamental chess knowledge and logic than Game B White." }, "consistent": true }, { "pair": "pair_12_reverse", "original": "pair_12", "prediction": { "stronger": "A", "probability_a_stronger": 0.62, "explanation": "Both players punished their opponents' opening inaccuracies and won cleanly, but Black in Game A showed smoother, more precise strategic planning. In Game A, Black systematically outplayed White by installing dominant central knights (13...Nd4, 17...Nc5), breaking through White's center, and converting the attack with flawless piece coordination (29...Rac8 and 31...Qf2#). In Game B, Black missed an immediate tactical win of a pawn with 13...Nxc3 and benefited from White's gross blunder 16. Be4 hanging a full bishop. Game A's play demonstrates slightly higher established control and positional understanding." }, "consistent": false }, { "pair": "pair_18_reverse", "original": "pair_18", "prediction": { "stronger": "A", "probability_a_stronger": 0.95, "explanation": "Player A (White) demonstrated strong tactical vision and positional play. They exploited Black's tactical error with 12. Bb5+ to win a piece, followed up with energetic play (22. Bf4+, 25. Be5), and smoothly converted into a checkmating attack (28. Qg8#). Conversely, Player B (Black) exhibited poor opening strategy by pushing numerous wing pawns (4...b5, 5...a5, 7...f6, 8...h5, 9...g5) while delaying development. Player B's tactical awareness was notably weak, missing threats throughout and ultimately hanging their queen in a single move with 22...Qd8?? to White's 22. Bd5." }, "consistent": true } ], "rows": [ { "pair": "pair_01", "game_a": "game_16", "color_a": "black", "game_b": "game_19", "color_b": "white", "rating_a": 1569, "rating_b": 2668, "gap": 1099, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.1, "correct": 1, "brier": 0.010000000000000002, "old_estimate_a": 1400, "old_estimate_b": 1900, "baseline_choice": "B", "baseline_credit": 1, "explanation": "In Game A, Black (Target A) played an unsound piece sacrifice with 6...Bxf2+? for minimal compensation and later blundered with 27...Nc7, hanging the knight. In contrast, Target B (White in Game B) demonstrated strong positional understanding in a standard QGD Carlsbad structure. White systematically exploited Black's weak f- and h-pawns through precise knight maneuvering (11.Nh3, 16.Nhf4, 22.Nxh5). White then cleanly converted the endgame with active piece play (33.Re7+, 34.Ra7) to create a decisive passed pawn (37.h7). Target B displayed much higher strategic depth and tactical soundness than Target A.", "confidence_valid": true }, { "pair": "pair_02", "game_a": "game_01", "color_a": "black", "game_b": "game_24", "color_b": "white", "rating_a": 1159, "rating_b": 762, "gap": 397, "gap_band": "200-499", "low_band": "one<1000", "gold": "A", "choice": "B", "probability_a": 0.2, "correct": 0, "brier": 0.6400000000000001, "old_estimate_a": 1350, "old_estimate_b": 1150, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black played overly aggressive pawn pushes early on (4...f5, 9...g5) and committed a major tactical oversight with 15...Nxg5, allowing White's 16. Nf6+ fork to win Black's queen for a knight. Black only won because White blundered back heavily later (including hanging the queen with 40. Qb8+). In contrast, Game B White played a solid opening (1. e4, 2. Nf3, 4. Bc4) and demonstrated far superior tactical alertness, successfully capturing Black's hanging rook with 21. Nxa8 and taking Black's queen after 22...Qd8 23. Bxd8 to seal a clean victory without committing gross blunders.", "confidence_valid": true }, { "pair": "pair_03", "game_a": "game_01", "color_a": "white", "game_b": "game_06", "color_b": "white", "rating_a": 1363, "rating_b": 1112, "gap": 251, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.78, "correct": 1, "brier": 0.04839999999999999, "old_estimate_a": 1400, "old_estimate_b": 1300, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A White demonstrated good tactical vision by finding 16. Nf6+ to win Black's queen, subsequently building a completely dominant position before suffering a catastrophic late blunder (40. Qb8+). Game B White showed poor opening planning (4. Bd3, 6. Bb1), lost central control, allowed Black's knights to infiltrate unpunished (13...Nd4, 17...Nc5), and quickly collapsed under pressure (25. Rf1, 26...Nxg5), leading to mate on move 31. Despite Game A White's single-move blunder, their overall tactical awareness and positional play were significantly superior to Game B White.", "confidence_valid": true }, { "pair": "pair_04", "game_a": "game_22", "color_a": "black", "game_b": "game_08", "color_b": "black", "rating_a": 557, "rating_b": 1791, "gap": 1234, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1100, "old_estimate_b": 1500, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black played wildly and inaccurately, sacrificing material unsoundly with 6...Nxf2, missing basic tactical defenses, and ultimately blundering a queen outright with 22...Qxg1+. In contrast, Game B Black demonstrated solid positional understanding, tactical awareness, and smooth execution. Game B Black effectively capitalized on White's passivity, finding the winning tactic 14...c5 to pick up a piece, followed by a crisp sequence with 20...Bg4, 21...Ng5, and 24...Nf3+ to force resignation. Game B Black clearly displays a much higher established rating.", "confidence_valid": true }, { "pair": "pair_05", "game_a": "game_09", "color_a": "white", "game_b": "game_07", "color_b": "black", "rating_a": 2193, "rating_b": 1593, "gap": 600, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.2, "correct": 0, "brier": 0.6400000000000001, "old_estimate_a": 1850, "old_estimate_b": 1550, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A White played passively out of the opening against a Benoni setup with an early 6. g3. White struggled for a plan, conceded the queenside initiative, lost a pawn with 22. bxc3, and allowed Black's pieces to dominate the position down to a hopeless rook endgame. In contrast, Game B Black played a very solid and classical Slav Defense structure, maneuvered smoothly in the middlegame, and showed sharp tactical alertness by punishing White's blunder (21. Bf3) with 21...Bc2!, winning the queen after 22. Bh5 Bxd1. Game B Black displayed much cleaner overall technique and tactical awareness.", "confidence_valid": true }, { "pair": "pair_06", "game_a": "game_17", "color_a": "black", "game_b": "game_25", "color_b": "black", "rating_a": 1639, "rating_b": 838, "gap": 801, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.85, "correct": 1, "brier": 0.022500000000000006, "old_estimate_a": 1650, "old_estimate_b": 1100, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black demonstrates solid opening knowledge and positional principles in an Italian Game (6...d5, active piece placement). Black maintained a balanced position until a single tactical oversight on move 24 (24...c6, allowing 25.Rxe5). In contrast, Game B Black plays unprovoked passive/erratic moves early on (2...Bd7, 7...Rd8). Despite winning a full rook via 13...Nc2+, Game B Black showed poor board vision, voluntarily sacrificing material back (24...Rxc3+) and allowing White's rook and queen to infiltrate for a decisive mate on d8. Game A Black shows far higher fundamental chess competence.", "confidence_valid": true }, { "pair": "pair_07", "game_a": "game_14", "color_a": "black", "game_b": "game_08", "color_b": "white", "rating_a": 1188, "rating_b": 1310, "gap": 122, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.8, "correct": 0, "brier": 0.6400000000000001, "old_estimate_a": 1300, "old_estimate_b": 1200, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Target A (Black) played a solid positional game despite a passive opening (6...Be7) and missing a free knight with 15...Qxe4. However, A maintained structural integrity, defended well, and accurately punished White's 30. Rg6 blunder with 30...Rxd3. In contrast, Target B (White) demonstrated severe tactical oversights early in Game B. B allowed a troubling pin with 9...e4, missed the basic 14...c5 fork that lost a bishop, and later allowed 20...Bg4 and 21...Ng5, walking into a fatal knight fork on f3. Target A showed far better fundamental tactical awareness and stability.", "confidence_valid": true }, { "pair": "pair_08", "game_a": "game_02", "color_a": "black", "game_b": "game_20", "color_b": "black", "rating_a": 2410, "rating_b": 1903, "gap": 507, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.88, "correct": 1, "brier": 0.0144, "old_estimate_a": 1650, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game A Black displayed excellent tactical precision and solid opening knowledge in the Ruy Lopez. Black accurately seized central control (14...Nc4), punished White's premature flank push 18.h4 with 18...Bxh4! 19.Nxh4 Qxh4, and executed a swift, decisive conversion ending in 25...Rh1#. In contrast, Game B Black defended poorly against White's kingside storm in a Pirc structure. Facing severe pressure on the h-file, Game B Black ignored White's threats with 24...b3? and collapsed to the blunder 25...Kf7??, allowing 26.Rxh7! which won White a queen and rook. Game A Black demonstrated significantly higher tactical awareness and positional understanding.", "confidence_valid": true }, { "pair": "pair_09", "game_a": "game_12", "color_a": "black", "game_b": "game_03", "color_b": "black", "rating_a": 1811, "rating_b": 1901, "gap": 90, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.65, "correct": 0, "brier": 0.42250000000000004, "old_estimate_a": 1400, "old_estimate_b": 1300, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Player A played a coherent opening (Modern Defense) and reached a reasonable position before falling victim to a two-move tactical trick with 10...Nxd4, missing White's 12. Bb5+ intermediate check which won the d4-bishop. In contrast, Player B displayed much worse tactical vision in Game B: after White played 16. Bxd6, Black played 16...Kf7??, failing to even recapture the bishop on d6 and giving away a full piece for free. Player B continued to make severe one-move blunders such as 22...e5, hanging the d5-pawn. Player A's mistake was a classic intermediate check oversight, while Player B demonstrated a lack of basic board awareness.", "confidence_valid": true }, { "pair": "pair_10", "game_a": "game_23", "color_a": "black", "game_b": "game_09", "color_b": "black", "rating_a": 564, "rating_b": 2177, "gap": 1613, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1250, "old_estimate_b": 1950, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Player B demonstrates a far superior positional and tactical understanding. In Game B, Black plays a well-executed Benoni structure with 10...b5, coordinates pieces purposefully (13...Ng4, 15...Nc4, 19...Nd4), wins a pawn with 21...Bxc3, and executes a dominant endgame conversion with double rooks (38...Rdd2 and 41...f3 mate threat). In contrast, Player A plays poorly in Game A, hanging a pawn with 11...e5 and blundering a full bishop with 13...Bxh3 14.gxh3 without adequate compensation, leading to a swift defeat. Player B is clearly the stronger player.", "confidence_valid": true }, { "pair": "pair_11", "game_a": "game_21", "color_a": "black", "game_b": "game_10", "color_b": "black", "rating_a": 743, "rating_b": 2501, "gap": 1758, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1300, "old_estimate_b": 1950, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Player B demonstrates high positional understanding and opening knowledge in a standard English/Catalan setup (1...e6, 2...d5, 7...c5, 10...Bb7). Player B executed a decisive tactical sequence starting with 24...Qc2!, exploiting White's back rank and bishop on b2, leading to a quick victory. In contrast, Player A played very low-level opening moves (2...e5, 3...Qe7, 4...f6), giving away pawns early for no compensation against White's eccentric 2. Qd3. Although Player A eventually won after White blundered a rook and knight, Player B's clean, high-level tactical and strategic play indicates a much higher established rating.", "confidence_valid": true }, { "pair": "pair_12", "game_a": "game_05", "color_a": "black", "game_b": "game_06", "color_b": "black", "rating_a": 1867, "rating_b": 1382, "gap": 485, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.58, "correct": 1, "brier": 0.17640000000000003, "old_estimate_a": 1800, "old_estimate_b": 1550, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Both players capitalized well on unorthodox opening play by their opponents, but Black in Game A displayed superior dynamic precision and tactical awareness. Game A Black launched a energetic Sicilian expansion with 4...b5 and 5...c4, then efficiently punished White's clumsy piece coordination (13. b3?) with a series of sharp tactical blows starting with 15...Nf4!, 16...Nxg2+!, 17...fxe4!, and 20...Nb4!. In Game B, Black played slightly passively early on (3...Bd7) before taking advantage of White's poor setup (4. Bd3, 6. Bb1) to win material. Game A Black showed higher tactical sharpness and dynamic initiative overall.", "confidence_valid": true }, { "pair": "pair_13", "game_a": "game_22", "color_a": "white", "game_b": "game_04", "color_b": "white", "rating_a": 717, "rating_b": 2508, "gap": 1791, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1100, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White displays a clear lack of basic opening and tactical principles, playing dubious moves like 2. Qf3, 3. g4, and moving the king to e3 on move 9. Game A White wins only because Black blunders the queen multiple times (17...Bf5 missing 17...Qxc5, and 22...Qxg1+). In contrast, Game B White demonstrates solid positional understanding in a standard Caro-Kann Exchange Variation, executing a coherent queenside pawn storm with 14. b4, 15. Nb3, and 16. a4. Although Game B White falls for a classic smothered mate tactic starting with 24...Ne2+, Game B White shows vastly superior established playing strength compared to Game A White.", "confidence_valid": true }, { "pair": "pair_14", "game_a": "game_18", "color_a": "black", "game_b": "game_14", "color_b": "white", "rating_a": 2641, "rating_b": 1249, "gap": 1392, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.85, "correct": 1, "brier": 0.022500000000000006, "old_estimate_a": 1900, "old_estimate_b": 1150, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black played a remarkably sharp and accurate tactical game despite losing castling rights early (10...Kf8). Black organized a powerful Kingside attack with 14...h4, 16...Bh3, and finished with a decisive mating sequence involving 30...Re2+, 31...Rh2, and 33...Qxg3+. In contrast, Game B White displayed poor opening understanding with 8. Nh4, allowing Black to win a piece for nothing with 9...Nd5 10. Bxd5 Bxh4. Game B White later blundered a full piece outright with 30. Rg6 Rxd3. Game A Black demonstrated far superior tactical alertness and calculation.", "confidence_valid": true }, { "pair": "pair_15", "game_a": "game_05", "color_a": "white", "game_b": "game_20", "color_b": "white", "rating_a": 1120, "rating_b": 1932, "gap": 812, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1400, "old_estimate_b": 1900, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White played very weakly, opening with the passive 2. Bc4, misplacing pieces with 5. Bd3, and collapsing rapidly after tactical blunders like 13. b3 (hanging the c3 pawn) and 16. Be4?? (allowing 16...Nxg2+ and losing a piece). White in Game A offered no resistance and was mated on move 29. In contrast, Game B White demonstrated clear strategic understanding of the Yugoslav-style attack against the Modern Defense with 8. O-O-O, 11. h4, and 15. g5. Game B White conducted a powerful kingside storm, brilliantly sacrificed with 25. Rxh7! to dismantle Black's defense, and executed a clean tactical win ending in checkmate with 38. Nh6#.", "confidence_valid": true }, { "pair": "pair_16", "game_a": "game_11", "color_a": "black", "game_b": "game_02", "color_b": "white", "rating_a": 1273, "rating_b": 1575, "gap": 302, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.25, "correct": 1, "brier": 0.0625, "old_estimate_a": 1350, "old_estimate_b": 1350, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "Target A (Game A Black) plays the dubious Englund Gambit (1...e5), misses a clean piece win on move 14 (14...Nf4 instead of 14...Qxc4 winning the bishop on c4), and ultimately blunders a simple back-rank checkmate with 41...Re8??. Target B (Game B White) demonstrates solid theoretical opening understanding in the Ruy Lopez through move 17. Although Target B collapses quickly after 18. h4? and gets mated on move 25, the overall quality of opening play and lack of gross early tactical blunders suggest Target B has a higher established rating than Target A.", "confidence_valid": true }, { "pair": "pair_17", "game_a": "game_13", "color_a": "white", "game_b": "game_16", "color_b": "white", "rating_a": 1126, "rating_b": 1752, "gap": 626, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.15, "correct": 1, "brier": 0.0225, "old_estimate_a": 1150, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B White played cleanly and displayed solid tactical awareness. After Black's unsound piece sacrifice with 6...Bxf2+ 7.Kxf2, Game B White consolidated comfortably, castled by hand, smoothly centralized forces, simplified the position, and maintained a clear winning advantage throughout without any major flaws. In contrast, Game A White showed weak tactical oversight, hanging the b2-pawn early with 6.Nc3, allowing tactical vulnerabilities like 17.c3? (permitting 17...Nc2+ fork), and ultimately blundering a back-rank checkmate in one with 35.Nxb5?? Rc1#. Game B White demonstrates significantly higher established playing strength.", "confidence_valid": true }, { "pair": "pair_18", "game_a": "game_24", "color_a": "black", "game_b": "game_12", "color_b": "white", "rating_a": 765, "rating_b": 1877, "gap": 1112, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.08, "correct": 1, "brier": 0.0064, "old_estimate_a": 1050, "old_estimate_b": 1550, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A's target (Black) played an excessively dubious opening, severely weakening the kingside with 7...f6, 8...h5, and 9...g5 while leaving the king uncastled. Black capped off a chaotic performance by hanging their queen to a simple bishop attack with 22...Qd8 23.Bxd8. Conversely, Game B's target (White) displayed strong positional awareness and sharp tactical execution. White utilized neat tactical finesses such as 12.Bb5+ c6 13.Qxd4 to regain material, launched a decisive queenside expansion with 17.b4 and 21.a4, and spotted the powerful tactical strike 25.Be5, converting smoothly into a back-rank mate with 28.Qg8+.", "confidence_valid": true }, { "pair": "pair_19", "game_a": "game_17", "color_a": "white", "game_b": "game_10", "color_b": "white", "rating_a": 1514, "rating_b": 2689, "gap": 1175, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.65, "correct": 0, "brier": 0.42250000000000004, "old_estimate_a": 1700, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Player A played a solid, clean positional game without major mistakes, smoothly consolidating after 11. g4 with 15. Kg2 and maintaining strong central/d-file control (21. dxe4, 23. Rd1, 24. Rd5), ultimately forcing Black to resign in a lost position. In contrast, Player B built a nice English opening setup but suffered a catastrophic tactical oversight with 24. Nxa7??, missing Black's double attack 24...Qc2! which attacked both the bishop on b2 and the rook on f1. Player B failed to defend the tactics, losing material rapidly (26...Bxf1) and collapsing immediately.", "confidence_valid": true }, { "pair": "pair_20", "game_a": "game_11", "color_a": "white", "game_b": "game_23", "color_b": "white", "rating_a": 1228, "rating_b": 557, "gap": 671, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.65, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1400, "old_estimate_b": 1300, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Player A demonstrates superior positional structure and tactical awareness compared to Player B. In Game A, White quickly seized an advantage against the Englund Gambit with 7. Ne4! and maintained good piece activity, eventually capitalizing cleanly on Black's back-rank oversight with 41. Rd8+. While White blundered a bishop with 14. Bc4 (which Black missed), Player A's general play was much more cohesive. In Game B, White played passive early moves (3. h3, 7. Bxf6) and blundered a full queen with 22. Qe5, which Black miraculously failed to take with 22... Qxe5. Game B exhibits a significantly lower level of tactical oversight.", "confidence_valid": true }, { "pair": "pair_21", "game_a": "game_21", "color_a": "white", "game_b": "game_15", "color_b": "white", "rating_a": 739, "rating_b": 2157, "gap": 1418, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1150, "old_estimate_b": 2150, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White demonstrates weak opening knowledge with 2. Qd3 and poor tactical vision, blundering material with 21. Kd2 (allowing 21...Nxf1+) and failing to see 29...Nf2 winning the rook on h1. In contrast, Game B White plays sound, standard opening theory in the Slav Defense (1. d4, 2. c4, 3. Nc3), shows good positional understanding, cleanly wins a pawn with 15. Ncxa5, and maintains strong tactical stability throughout the game into a classic drawn pawn endgame. Game B White exhibits vastly superior opening preparation, tactical awareness, and overall playing strength.", "confidence_valid": true }, { "pair": "pair_22", "game_a": "game_04", "color_a": "black", "game_b": "game_13", "color_b": "black", "rating_a": 2718, "rating_b": 1551, "gap": 1167, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.88, "correct": 1, "brier": 0.0144, "old_estimate_a": 1850, "old_estimate_b": 1300, "baseline_choice": "A", "baseline_credit": 1, "explanation": "In Game A, Black played a very high-quality Caro-Kann defense, castling queenside and accurately handling White's pawn storm before executing a brilliant smothered mate combination (24...Ne2+ 25. Kh1 Qg1+ 26. Rxg1 Nf2#). In contrast, Game B's Black displayed poor tactical awareness, blundering material with 28...Re6 29. Nc7! and losing a piece on c6 (31. Rxc6). Game B Black was heavily losing and only won because White committed a fatal back-rank blunder with 35. Nxb5??, allowing 35...Rc1#. Game A Black demonstrated vastly superior tactical calculation and positional judgment.", "confidence_valid": true }, { "pair": "pair_23", "game_a": "game_25", "color_a": "white", "game_b": "game_18", "color_b": "white", "rating_a": 738, "rating_b": 2426, "gap": 1688, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.15, "correct": 1, "brier": 0.0225, "old_estimate_a": 1150, "old_estimate_b": 1700, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White shows extremely weak opening fundamentals, pushing flank pawns recklessly with 4. g4, 6. h4, 7. g5, and 8. h5 while neglecting piece development, and blundering a full rook with 13. Nh3 Nc2+. Although White eventually won after Black blundered back, the initial quality of play indicates a much lower rating. Game B White plays a sound, disciplined opening (Snyder Sicilian), castles early, and manages a complex positional battle logically up to move 29. Even though Game B White loses to a late-game tactical miscalculation (30. Rd1 Re2+), their overall chess understanding and move quality are vastly superior to Game A White.", "confidence_valid": true }, { "pair": "pair_24", "game_a": "game_03", "color_a": "white", "game_b": "game_15", "color_b": "black", "rating_a": 1782, "rating_b": 2154, "gap": 372, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.35, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1550, "old_estimate_b": 2200, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B's target (Black) demonstrates high positional understanding and theoretical fluency in a sharp Slav Defense line. Black executes the tactical break 22...c5! smoothly, leading to a forcing sequence (24...Nxb7! 25.Qxc8+ Nxc8 26.Rxd5 exd5) that transforms into a balanced endgame, which Black defends flawlessly to a draw. In contrast, Game A's target (White) plays a somewhat premature early flank advance with 8. h4 and 11. Nh3, though White effectively capitalizes on Black's major blunder (16...Kf7??). Overall, Black in Game B shows superior overall chess culture, calculation depth, and endgame technique.", "confidence_valid": true }, { "pair": "pair_25", "game_a": "game_19", "color_a": "black", "game_b": "game_07", "color_b": "white", "rating_a": 2474, "rating_b": 1862, "gap": 612, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.65, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1750, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Player A (Black) demonstrates good opening familiarity in a Carlsbad Queen's Gambit Declined (6...Bf5) and plays logically against White's flank pawn storm, only collapsing in a difficult, already-losing endgame (34...Nxf5?). In contrast, Player B (White) plays passively in a symmetrical Slav structure and commits a basic tactical oversight in a balanced middlegame with 21. Bf3??, completely missing Black's 21...Bc2 which forks the queen and rook and immediately loses significant material. Player A shows significantly better baseline positional understanding and tactical awareness throughout the main phase of the game.", "confidence_valid": true } ] }, { "model": "openai/gpt-5.6-sol", "correct": 20, "n": 25, "accuracy": 80.0, "baseline_accuracy": 80.0, "invalid_confidence": 0, "consistent_reversals": 5, "reversals": [ { "pair": "pair_09_reverse", "original": "pair_09", "prediction": { "stronger": "B", "probability_a_stronger": 0.27, "explanation": "Both Black players made major tactical errors, but B showed more purposeful calculation afterward. In Game A, Black inexplicably played 16...Kf7 after 16.Bxd6 instead of recovering the bishop with 16...Rxd6, then allowed 18.Be7 and 19.Bxf6, remaining tactically vulnerable through 34.Ne7+ and 35.Nxg8. Game B’s Black also blundered with 10...Nxd4? 11.Nxd4 Bxd4 12.Bb5+ c6 13.Qxd4, losing a piece, but later created coherent counterplay with 23...Rd4, 25...Rd2, 26...Nxc4, and 27...Nxe3. That resourcefulness suggests B was more likely the stronger established rapid player." }, "consistent": true }, { "pair": "pair_15_reverse", "original": "pair_15", "prediction": { "stronger": "A", "probability_a_stronger": 0.97, "explanation": "Game A White showed a coherent opposite-side-castling attack and calculated a long forcing combination: 22.Rdh1, 24.f5, and especially 26.Rxh7! Nxh7 27.Qxh7, culminating in 29.Qxh8+ Bxh8 30.Rxh8+ and 31.Rxd8, followed by an accurate mating attack. Game B White repeatedly mishandled threats: 16.Be4? allowed 16...Nxg2+ 17.Kf1 fxe4, losing the bishop; 22.Bc3 Bxe4 and 23.Nd4 Bxh1 then lost further material. Finally, 27.Qxd3? Be4+ 28.Nf3 Bxd3+ dropped the queen and led to mate. A displayed substantially better tactical awareness and coordination." }, "consistent": true }, { "pair": "pair_13_reverse", "original": "pair_13", "prediction": { "stronger": "A", "probability_a_stronger": 0.86, "explanation": "Game A White showed substantially more established positional knowledge: coherent Caro-Kann development, 13.Bh4 followed by queenside expansion with 14.b4, 16.a4, 18.Nc5 and 20.a5. The fatal 24.Bb4? and 25.Kh1? overlooked Black’s sharp ...Ne2+–...Qg1+ mating pattern, but this was one tactical collapse. Game B White’s 2.Qf3, 3.g4, 6.h3 and exposed king walk to e3/d1 indicate much weaker fundamentals. White won mainly because Black sacrificed on f2 without enough force and then allowed 13.Qxb5, 17.Qxc5, and the queen-losing 22...Qxg1+ 23.Nxg1." }, "consistent": true }, { "pair": "pair_12_reverse", "original": "pair_12", "prediction": { "stronger": "B", "probability_a_stronger": 0.3, "explanation": "Game B’s Black showed the more coherent calculation and attacking coordination. The sequence 16...Nxg2+ 17.Kf1 fxe4 used a forcing check to win White’s bishop, and 22...Bxe4 23.Nd4 Bxh1 converted the advantage by taking the rook. The follow-up 24...Qb7, 25...Nh3+, and 27...Be4+ led cleanly to mate. Game A’s Black also exploited errors well—18...Ncxb3 and 19...Nxa1 won a rook, followed by 25...Bxf1—but the opening involved several slow bishop moves and the conversion relied more heavily on White’s obvious material blunders. Overall, B displayed stronger tactical foresight and coordination." }, "consistent": true }, { "pair": "pair_18_reverse", "original": "pair_18", "prediction": { "stronger": "A", "probability_a_stronger": 0.98, "explanation": "Game A White displayed substantially better tactical awareness: 12.Bb5+ and 13.Qxd4 punished Black’s premature pawn grab, while 22.Bf4+, 23.Bxc4, and the concluding 26.Re3! Nxc4 27.Bxh8 Nxe3 28.Qg8+ formed a coherent attacking sequence. Game B Black played an extremely weakening opening with ...f6, ...h5, and ...g5 while leaving the king centralized, then overlooked 20.Nb6 and lost the a8 rook after 21.Nxa8. Most decisively, 22...Qd8?? simply placed the queen where 23.Bxd8 captured it. The contrast strongly favors A despite single-game uncertainty." }, "consistent": true } ], "rows": [ { "pair": "pair_01", "game_a": "game_16", "color_a": "black", "game_b": "game_19", "color_b": "white", "rating_a": 1569, "rating_b": 2668, "gap": 1099, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.07, "correct": 1, "brier": 0.004900000000000001, "old_estimate_a": 1600, "old_estimate_b": 1950, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s White showed substantially stronger, coherent play: the queen trade with 8.Bxf6 and 9.Qxf6, king centralization, the maneuver Nh3-f4-g3-h5, and 22.Nxh5 followed by the passed h-pawn were purposeful. White also found the clean tactical finish 33.Re7+ and 35.Ra8+, winning the h8-rook. Game A’s Black chose the highly unsound 6...Bxf2+, leaving White materially ahead despite the resource 12...Bxf3 13.hxg4 Bxe2. Black then traded queens while a piece down and resigned after 27...Nc7. B is much more likely to have had the higher established rapid strength.", "confidence_valid": true }, { "pair": "pair_02", "game_a": "game_01", "color_a": "black", "game_b": "game_24", "color_b": "white", "rating_a": 1159, "rating_b": 762, "gap": 397, "gap_band": "200-499", "low_band": "one<1000", "gold": "A", "choice": "B", "probability_a": 0.27, "correct": 0, "brier": 0.5328999999999999, "old_estimate_a": 1450, "old_estimate_b": 1450, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "Game A Black showed fighting resourcefulness with 19...Nd4, 27...e4 and the promotion defense 40...Rxb8 41...Rbxe8, but the elementary queen loss after 15...Nxg5? 16.Nf6+ Ke7 17.Nxd7 is strong evidence against established strength; victory depended on White’s later collapse. Game B White handled an eccentric attack calmly, castled, and exploited tactics accurately: 17.Nxf6+ cleared lines, 20.Nb6 drove the queen while threatening the rook, 21.Nxa8 won material, and 23.Bxd8 punished Black’s final blunder. Although Game B’s opponent was very weak, White made fewer serious errors and displayed clearer tactical awareness.", "confidence_valid": true }, { "pair": "pair_03", "game_a": "game_01", "color_a": "white", "game_b": "game_06", "color_b": "white", "rating_a": 1363, "rating_b": 1112, "gap": 251, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.86, "correct": 1, "brier": 0.019600000000000003, "old_estimate_a": 1450, "old_estimate_b": 1400, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A White showed substantially better tactical awareness and coordination: 15.Nd5 followed by 16.Nf6+ and 17.Nxd7 won Black’s queen, while 26.Qc6 and the later checks kept pressure. The conversion was poor—especially 40.Qb8+?, which allowed ...Rxb8 and liquidation—but the preceding play displayed real calculation. Game B White spent many opening tempi on Bd3-b1, a3, h4, and b3, then overextended with 15.f3 and 16.f4. More seriously, White allowed 18...Ncxb3 and 19...Nxa1, played 25.Rf1? Bxf1, and failed to address the mating attack ending in 31...Qf2#. A therefore looks more established despite also losing.", "confidence_valid": true }, { "pair": "pair_04", "game_a": "game_22", "color_a": "black", "game_b": "game_08", "color_b": "black", "rating_a": 557, "rating_b": 1791, "gap": 1234, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.04, "correct": 1, "brier": 0.0016, "old_estimate_a": 1000, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s Black played a coherent, controlled game: 5...O-O completed king safety, 9...e4 and 14...c5 challenged the center, and 15...Ndxc5 correctly recovered the bishop. After the queen trade, 20...Bg4, 21...Ng5, 22...Bxf3+, and 24...Nf3+ formed a purposeful winning sequence. Game A’s Black instead chose the highly speculative 6...Nxf2, then lost the knight after 11...Nb5+ 13.Qxb5 and a bishop after 15...d5 17.Qxc5. The final 22...Qxg1+ simply allowed 23.Nxg1. B showed substantially stronger coordination, calculation, and risk control.", "confidence_valid": true }, { "pair": "pair_05", "game_a": "game_09", "color_a": "white", "game_b": "game_07", "color_b": "black", "rating_a": 2193, "rating_b": 1593, "gap": 600, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.38, "correct": 0, "brier": 0.3844, "old_estimate_a": 1900, "old_estimate_b": 1650, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game B’s Black played a coherent Slav setup and showed clear tactical awareness with 20...Qg6 and 21...Bc2, exploiting White’s misplaced rook and queen; after 22.Bh5, 22...Bxd1 23.Bxg6 Bxb3 converted the sequence cleanly. Game A’s White showed good opening familiarity and active ideas such as 25.Rxa7 and 28.e5, but 19.f4 allowed ...Nd4 followed by exchanges that exposed the queenside, while 22...Qxa3 and 23...Qxc3 won pawns. Later, White’s rooks became vulnerable to the coordinated invasion 38...Rdd2, 39...Rh2+, and 40...Rbg2+. B’s game was shorter and opponent-assisted, but B displayed fewer weaknesses.", "confidence_valid": true }, { "pair": "pair_06", "game_a": "game_17", "color_a": "black", "game_b": "game_25", "color_b": "black", "rating_a": 1639, "rating_b": 838, "gap": 801, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.84, "correct": 1, "brier": 0.02560000000000001, "old_estimate_a": 1850, "old_estimate_b": 1250, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black played a coherent, principled game: 6...d5 challenged the center, 13...Nf4 created activity, and 16...Bxe4 followed by 19...Rfe8 and 21...Rd8 showed purposeful coordination. The unexplained ending after 24...c6 does not reflect an obvious collapse. Game B Black found the strong fork 13...Nc2+ and tactical 20...Rxd4+, but the opening setup (...Bd7, ...b5, ...Na6, ...Rd8) was awkward, and 24...Rxc3+ gave up a rook without sufficient return. Most decisively, 32...Nxf5 allowed 33.Ra8 and 34.Rxc8, losing the queen and leading to mate. A appears more consistently skilled.", "confidence_valid": true }, { "pair": "pair_07", "game_a": "game_14", "color_a": "black", "game_b": "game_08", "color_b": "white", "rating_a": 1188, "rating_b": 1310, "gap": 122, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.78, "correct": 0, "brier": 0.6084, "old_estimate_a": 1550, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A’s Black played more consistently and purposefully: 9...Nd5 answered White’s pressure, 19...f5 challenged the centralized knight, and 21...Rd5 followed by ...Rd8/...Rad8 coordinated both rooks before 30...Rxd3 punished the loose bishop. Game B’s White showed reasonable development and simplification with 17.Bxe4 and 18.Qxb6, but the passive opening conceded initiative, and 22.Rd4? overlooked 22...Bxf3+, where the bishop could escape because ...Ng5 protected f3. Then 24.h4? allowed 24...Nf3+, also forking the rook on d4. These tactical oversights make A more likely to have had the higher established rapid strength.", "confidence_valid": true }, { "pair": "pair_08", "game_a": "game_02", "color_a": "black", "game_b": "game_20", "color_b": "black", "rating_a": 2410, "rating_b": 1903, "gap": 507, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.88, "correct": 1, "brier": 0.0144, "old_estimate_a": 1650, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game A Black played a coherent tactical game: 9...Nxe4 exploited the center, 18...Bxh4! converted White’s kingside aggression into material and initiative, and 21...Re2 followed by ...Rxb2/...Rxh3 produced the precise 25...Rh1# finish. Game B Black’s queenside counterplay was too slow while the king came under a direct attack; 24...b3 and especially 25...Kf7 failed to address White’s threats, and 26...Nxh7 allowed the forcing sequence 27.Qxh7 Rh8 28.fxg6+ through 31.Rxd8, leaving Black decisively down. A showed stronger tactical awareness and defensive judgment.", "confidence_valid": true }, { "pair": "pair_09", "game_a": "game_12", "color_a": "black", "game_b": "game_03", "color_b": "black", "rating_a": 1811, "rating_b": 1901, "gap": 90, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.78, "correct": 0, "brier": 0.6084, "old_estimate_a": 1600, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A Black showed more coherent development and active counterplay with 14...O-O-O, 21...c4, and the rook invasion 23...Rd4–25...Rd2. They did blunder a piece through 10...Nxd4?! 11.Nxd4 Bxd4 12.Bb5+ c6 13.Qxd4 and later missed 28.Qg8+. Game B Black’s tactical awareness looked substantially weaker: 15...Rd8 ignored the attack on f8, 16...Kf7 failed to address the lost bishop and rook, and 18...Re8 allowed 19.Bxf6. Further material losses followed with little compensation. Thus A is more likely to have had the higher established rapid strength.", "confidence_valid": true }, { "pair": "pair_10", "game_a": "game_23", "color_a": "black", "game_b": "game_09", "color_b": "black", "rating_a": 564, "rating_b": 2177, "gap": 1613, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.03, "correct": 1, "brier": 0.0009, "old_estimate_a": 1350, "old_estimate_b": 2050, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s Black showed substantially stronger strategic and tactical control: 19...Nd4!, 20...Nxe2 and 21...Bxc3 dismantled White’s queenside, while 25...Rb2 and 32...Qf4+ transitioned cleanly into a winning rook invasion. Game A’s Black instead launched the unsound 13...Bxh3? and compounded it with 15...Bh4, surrendering both bishops for insufficient material. The ensuing queen checks never produced a decisive attack, leaving Black down a minor piece and resigning after 24.Qh2.", "confidence_valid": true }, { "pair": "pair_11", "game_a": "game_21", "color_a": "black", "game_b": "game_10", "color_b": "black", "rating_a": 743, "rating_b": 2501, "gap": 1758, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.08, "correct": 1, "brier": 0.0064, "old_estimate_a": 1450, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s Black showed much stronger positional coherence: sound development through 14...Rfe8, purposeful pressure with 18...Rf8 and 19...Qf7, then accurate tactics after 22.Rxc8 Bxc8. The sequence 24...Qc2, 25...Ba6 and 26...Bxf1 decisively exploited White’s loose back rank. Game A’s Black found good ideas such as 7...Qb4+ and the knight incursion 19...Nc2+–23...Nxg2, but the opening was highly irregular and the large advantage required a long, meandering conversion, with the knight stranded on h1 and many inefficient rook maneuvers. B looks substantially more established.", "confidence_valid": true }, { "pair": "pair_12", "game_a": "game_05", "color_a": "black", "game_b": "game_06", "color_b": "black", "rating_a": 1867, "rating_b": 1382, "gap": 485, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.64, "correct": 1, "brier": 0.1296, "old_estimate_a": 1850, "old_estimate_b": 1600, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black showed more purposeful calculation: 16...Nxg2+! 17.Kf1 fxe4 won the bishop while preserving the knight, and 22...Bxe4 23...Bxh1 followed by ...Nh3+ and the forced mating sequence was precise. Game B Black also converted well, especially with 18...Ncxb3 19...Nxa1 and 25...Bxf1, but the opening involved several inefficient bishop moves (...Bd7-f5-d7), and much of the win followed White’s conspicuous tactical errors. A’s play looks more consistently coordinated and tactically sophisticated.", "confidence_valid": true }, { "pair": "pair_13", "game_a": "game_22", "color_a": "white", "game_b": "game_04", "color_b": "white", "rating_a": 717, "rating_b": 2508, "gap": 1791, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.08, "correct": 1, "brier": 0.0064, "old_estimate_a": 1200, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B White showed much more coherent opening and strategic play: normal development, castling, and a purposeful queenside expansion with 14.b4, 16.a4, 18.Nc5, and 22.b5. The decisive error was tactical—24.Bb4? allowed 24...Ne2+ and the forced mating pattern. Game A White played extremely unsoundly with 2.Qf3, 3.g4, 6.h3, and an exposed king walk to e3/d1, succeeding mainly because Black repeatedly hung material, notably 13...Qf2+ allowing 17.Qxc5 and 22...Qxg1+ losing the queen to 23.Nxg1. B is therefore much likelier to have had the stronger established rapid level.", "confidence_valid": true }, { "pair": "pair_14", "game_a": "game_18", "color_a": "black", "game_b": "game_14", "color_b": "white", "rating_a": 2641, "rating_b": 1249, "gap": 1392, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.9, "correct": 1, "brier": 0.009999999999999995, "old_estimate_a": 1800, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black showed coherent opening play and strong tactical conversion: 29...Re8, 30...Re2+ and 31...Rh2 created a decisive attack, while 32...Qh3+ punished White’s greedy 32.Qxd5. Game B White used several inefficient moves, notably 8.Nh4 and repeated queen/knight maneuvers, then apparently resigned after 30...Rxd3 despite the immediate 31.Rxe8+, which wins Black’s hanging rook with check. Missing that elementary resource at the game’s conclusion is much stronger evidence of lower established strength than Game A Black’s occasional awkwardness such as 20...Ng8.", "confidence_valid": true }, { "pair": "pair_15", "game_a": "game_05", "color_a": "white", "game_b": "game_20", "color_b": "white", "rating_a": 1120, "rating_b": 1932, "gap": 812, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.06, "correct": 1, "brier": 0.0036, "old_estimate_a": 1450, "old_estimate_b": 1900, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B White showed a coherent attacking plan: opposite-side castling followed by g4, h4-h5, Rdh1 and Qh2, then calculated 26.Rxh7! Nxh7 27.Qxh7, converting into a decisive material attack and finishing accurately with 38.Nh6#. Game A White spent many tempi retreating the same bishop and overlooked elementary tactics: 16.Be4? allowed 16...Nxg2+ 17.Kf1 fxe4, losing the bishop, while 22.Bc3? Bxe4 and 23.Nd4? Bxh1 led to collapse. B’s target displayed substantially stronger planning and tactical awareness.", "confidence_valid": true }, { "pair": "pair_16", "game_a": "game_11", "color_a": "black", "game_b": "game_02", "color_b": "white", "rating_a": 1273, "rating_b": 1575, "gap": 302, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.72, "correct": 0, "brier": 0.5184, "old_estimate_a": 1600, "old_estimate_b": 1400, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A Black showed more sustained competence: despite the dubious 1...e5 and early queen moves, they completed development, castled, found useful play with 23...e4, 24...d5, 28...Nd3, and won the c4-pawn with 36...Rxc4. The final 40...e3? overlooked the back-rank mating sequence 41.Rd8+ Re8 42.Rxe8#, but it was one late tactical failure. Game B White played a coherent opening but deteriorated sharply: 18.h4?! exposed the king, 20.Qh3 invited a damaging queen trade, and 25.Bg5?? allowed the immediate 25...Rh1#. Missing such a direct mate, after already permitting ...Re2 and ...Rxh3, is stronger evidence of lower established strength.", "confidence_valid": true }, { "pair": "pair_17", "game_a": "game_13", "color_a": "white", "game_b": "game_16", "color_b": "white", "rating_a": 1126, "rating_b": 1752, "gap": 626, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.18, "correct": 1, "brier": 0.0324, "old_estimate_a": 1500, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B White responded calmly to Black’s dubious 6...Bxf2+: 7.Kxf2, 8.Rf1 and 9.Kg1 restored king safety. The tactical 12.h3 correctly met ...Bxf3 with 13.hxg4, winning the g4-knight, and White then simplified sensibly with 25.Qe2 Qxe2 26.Rxe2. Game A White showed some tactical awareness with 28.Ne8–30.Nxe6 and 31.Rxc6, but the opening rook excursions (8.Rb3, 13.Ra3, 14.Rc3, 16.Rc5) culminated in losing that rook after 18.Qb2? bxc5. Most decisively, 35.Nxb5?? overlooked the immediate 35...Rc1#.", "confidence_valid": true }, { "pair": "pair_18", "game_a": "game_24", "color_a": "black", "game_b": "game_12", "color_b": "white", "rating_a": 765, "rating_b": 1877, "gap": 1112, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1050, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black showed repeated strategic and tactical weakness: 7...f6, 8...h5, and 9...g5 badly loosened the position, while 20...Qc6 allowed 21.Nxa8 to win the rook. Most decisively, 22...Qd8 simply placed the queen where White’s bishop could take it with 23.Bxd8. Game B White played much more coherently and tactically: 12.Bb5+ was a strong zwischenzug enabling 13.Qxd4, the queenside expansion with 17.b4 was purposeful, and 25.Be5 followed by 28.Qg8+ demonstrated awareness of the mating attack despite material complications.", "confidence_valid": true }, { "pair": "pair_19", "game_a": "game_17", "color_a": "white", "game_b": "game_10", "color_b": "white", "rating_a": 1514, "rating_b": 2689, "gap": 1175, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.35, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1750, "old_estimate_b": 1750, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "Game B White showed stronger established opening and positional habits: 6.b3/7.Bb2, 11.Rc1, 14.Qd2, and 17.Rfd1 formed a coherent setup, while 20.Nb5 created active pressure. The decisive error was concrete but isolated: after saving the rook from 24...Qc2 with 25.Rf1, White overlooked 25...Ba6 and allowed 26...Bxf1. Game A White remained tactically sound, but the plan with 6.a3, 11.g4, and the 12.Neg5–13.Ne4 retreat was comparatively loose and inefficient. Game A’s unexplained finish after 24...c6 also provides little evidence of conversion skill. Thus B looks likelier to have had the higher established rapid strength despite losing.", "confidence_valid": true }, { "pair": "pair_20", "game_a": "game_11", "color_a": "white", "game_b": "game_23", "color_b": "white", "rating_a": 1228, "rating_b": 557, "gap": 671, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.76, "correct": 1, "brier": 0.0576, "old_estimate_a": 1700, "old_estimate_b": 1600, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A White showed more coherent strategic and tactical play: 19.Qh4 followed by f4-f5, g4, f6 created a dangerous kingside bind, while 31.c4 and the rook activation 37.Rab1–38.Rb5–40.Rd5 culminated accurately in 41.Rd8+ and 42.Rxe8#. Game B White correctly accepted Black’s unsound 13...Bxh3 sacrifice and found 15.Nxf5 and 16.Nxh4, but the opening plan with early h3, Bb5, Bg5 and repeated exchanges was less convincing, and 17.f3 exposed the king unnecessarily. Game B’s result depended heavily on Black’s crude sacrifice and premature resignation, whereas A demonstrated stronger sustained planning.", "confidence_valid": true }, { "pair": "pair_21", "game_a": "game_21", "color_a": "white", "game_b": "game_15", "color_b": "white", "rating_a": 739, "rating_b": 2157, "gap": 1418, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.06, "correct": 1, "brier": 0.0036, "old_estimate_a": 1350, "old_estimate_b": 2100, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B White showed much stronger opening discipline and calculation: standard Slav development, then the forcing sequence 24.Rd1 Nxb7 25.Qxc8+ Nxc8 26.Rxd5 exd5 27.Nxb7, followed by purposeful simplification with 31.Bxb4 and 34.Bxc5. Game A White’s 2.Qd3 and repeated pawn-grabbing with 5.Qxd5 and 7.Qxb7 neglected development. More seriously, 19.a3? allowed 19...Nc2+ and the knight’s checking route to 21...Nxf1+, costing a rook and forcing prolonged king exposure. Although A resisted for many moves, B’s play was substantially more coherent and tactically controlled.", "confidence_valid": true }, { "pair": "pair_22", "game_a": "game_04", "color_a": "black", "game_b": "game_13", "color_b": "black", "rating_a": 2718, "rating_b": 1551, "gap": 1167, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.82, "correct": 1, "brier": 0.03240000000000002, "old_estimate_a": 1950, "old_estimate_b": 1550, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black showed a coherent plan: 13...O-O-O, 15...Rc8, 16...Ne4 and 17...f5 coordinated the pieces before the kingside pawn attack. The finish 24...Ne2+ 25.Kh1 Qg1+ 26.Rxg1 Nf2# was a precise forced mating combination. Game B Black found useful tactics with 17...b6 and ultimately 35...Rc1#, but the early queen expedition 5...Qb5 6...Qxb2 cost many tempi, and later 28...Re6 29.Nc7 Ra7 30.Nxe6 allowed White to recover material and gain the initiative. B’s win depended more heavily on White’s final mating blunder.", "confidence_valid": true }, { "pair": "pair_23", "game_a": "game_25", "color_a": "white", "game_b": "game_18", "color_b": "white", "rating_a": 738, "rating_b": 2426, "gap": 1688, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.1, "correct": 1, "brier": 0.010000000000000002, "old_estimate_a": 1300, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B White showed substantially more coherent development and positional play: 7.O-O, 8.d4, 9.Bb2, 14.c4, and 23.Rc1 followed recognizable plans, while 27.Bxc5+ and 28.Qxc3 handled tactics sensibly. The late 32.Qxd5? allowed ...Qh3+ and ...Qxg3+, but it was one tactical failure. Game A White’s 3.Qf3, 4.g4, repeated h-pawn moves, and 13.Nh3? allowed ...Nc2+ and ...Nxa1, indicating much weaker fundamentals. A’s eventual mating attack, including 30.Rd7+ and 34.Rxc8, was resourceful but depended heavily on Black’s repeated defensive errors.", "confidence_valid": true }, { "pair": "pair_24", "game_a": "game_03", "color_a": "white", "game_b": "game_15", "color_b": "black", "rating_a": 1782, "rating_b": 2154, "gap": 372, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.2, "correct": 1, "brier": 0.04000000000000001, "old_estimate_a": 1800, "old_estimate_b": 2050, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s Black showed stronger positional and tactical control. The sequence 24...Nxb7! 25.Qxc8+ Nxc8 26.Rxd5 exd5 deliberately simplified into a sound endgame, followed by active king play with 32...Ke8, 33...Kd7 and the well-timed 42...d4. Game A’s White spotted several clear opportunities—16.Bxd6, 23.Nxd5 and 34.Ne7+—and converted decisively, but these arose largely from Black’s loose play, especially 14...Bd6, 15...Rd8 and the exposed king. White A’s early 8.h4 and repeated knight maneuvers were effective but less convincing as evidence of established strength than Black B’s sustained, coherent play.", "confidence_valid": true }, { "pair": "pair_25", "game_a": "game_19", "color_a": "black", "game_b": "game_07", "color_b": "white", "rating_a": 2474, "rating_b": 1862, "gap": 612, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.62, "correct": 1, "brier": 0.1444, "old_estimate_a": 1800, "old_estimate_b": 1500, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black showed more sustained positional competence: 13...b5, 16...Rhg8, king centralization through ...Ke7-d6, and the timely 22...c5 created active counterplay in the queenless middlegame. Black ultimately mishandled White’s h-pawn and blundered with 34...Nxf5?, allowing 35.Ra8+ and 36.Rxh8. Game B White also played a coherent Slav structure, but 21.Bf3? permitted 21...Bc2, a straightforward double attack on Qd1 and Rb3; 22.Bh5 Bxd1 23.Bxg6 Bxb3 then left White decisively down material. A’s longer stretch of purposeful play slightly outweighs the comparable final tactical oversight.", "confidence_valid": true } ] }, { "model": "openai/gpt-6-sol", "correct": 19, "n": 25, "accuracy": 76.0, "baseline_accuracy": 80.0, "invalid_confidence": 0, "consistent_reversals": 4, "reversals": [ { "pair": "pair_09_reverse", "original": "pair_09", "prediction": { "stronger": "A", "probability_a_stronger": 0.77, "explanation": "Game B’s Black appears to miscalculate early: 10...Nxd4 11.Nxd4 Bxd4 overlooks 12.Bb5+ c6 13.Qxd4, leaving Black a piece down for a pawn. Black’s later activity does not overcome that deficit. Game A’s Black also makes serious mistakes, notably 33...Nc8 allowing 34.Ne7+ and 35.Nxg8, but plays a more competitive game for considerably longer. On this limited evidence, A is likelier to have been the stronger established rapid player." }, "consistent": false }, { "pair": "pair_15_reverse", "original": "pair_15", "prediction": { "stronger": "A", "probability_a_stronger": 0.98, "explanation": "Game A’s White coordinated a purposeful kingside attack with 22.Rdh1 and 23.Qh2, then calculated the sequence 26.Rxh7 through 31.Rxd8 to win Black’s queen. Game B’s White repeatedly retreated the bishop in the opening, then allowed 16...Nxg2+ and 22...Bxe4, followed by 23...Bxh1. Despite different opponents, A’s attacking coordination and tactical play are much stronger evidence of established playing strength." }, "consistent": true }, { "pair": "pair_13_reverse", "original": "pair_13", "prediction": { "stronger": "A", "probability_a_stronger": 0.96, "explanation": "Game A’s White showed a coherent plan against Black’s queenside castling with 14.b4, 16.a4 and 20.a5, though 24.Bb4 overlooked a decisive mating sequence. Game B’s White played 2.Qf3 and 3.g4, then exposed the king with 7.Kxf2 and 9.Ke3. B won largely after Black allowed 13.Qxb5 and 17.Qxc5, losing pieces. A’s play looks more indicative of established strength despite the loss." }, "consistent": true }, { "pair": "pair_12_reverse", "original": "pair_12", "prediction": { "stronger": "B", "probability_a_stronger": 0.28, "explanation": "Game B’s Black showed a more coherent plan: ...a6, ...b5 and ...c4 gained space, then 15...Nf4, 16...Nxg2+ and 17...fxe4 exploited White’s exposed bishop. The later 20...Nb4, 21...Nbd3 and 22...Bxe4 continued the attack. Game A’s Black also converted effectively with 18...Ncxb3 and 19...Nxa1, but spent several early moves repositioning the same bishop with 9...Bf5, 10...Bd7 and 12...Bg4. B looks likelier to have had the higher established strength." }, "consistent": true }, { "pair": "pair_18_reverse", "original": "pair_18", "prediction": { "stronger": "A", "probability_a_stronger": 0.98, "explanation": "Game A’s White handled tactics well: 12.Bb5+ followed by 13.Qxd4 recovered the bishop, and 27.Bxh8 took a rook despite Black’s counterplay. Game B’s Black made repeated weakening pawn moves (...f6, ...h5, ...g5, ...h4), lost a rook after 20.Nb6 and 21.Nxa8, then placed the queen on d8 where 23.Bxd8 won it. A’s play shows substantially better tactical awareness." }, "consistent": true } ], "rows": [ { "pair": "pair_01", "game_a": "game_16", "color_a": "black", "game_b": "game_19", "color_b": "white", "rating_a": 1569, "rating_b": 2668, "gap": 1099, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.04, "correct": 1, "brier": 0.0016, "old_estimate_a": 1450, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s White played a coherent endgame: 22.Nxh5 won a pawn, 25.h5 and 28.h6 created a dangerous passer, and 34.Ra7 followed by 35.Ra8+ exploited Black’s knight capture. Game A’s Black chose the unsound-looking 6...Bxf2+, then lost another minor piece in the sequence 11...Ng4 12.h3 Bxf3 13.hxg4 Bxe2 14.Qxe2, leaving White with a substantial material advantage. B’s play is stronger evidence of established skill, regardless of the results.", "confidence_valid": true }, { "pair": "pair_02", "game_a": "game_01", "color_a": "black", "game_b": "game_24", "color_b": "white", "rating_a": 1159, "rating_b": 762, "gap": 397, "gap_band": "200-499", "low_band": "one<1000", "gold": "A", "choice": "B", "probability_a": 0.16, "correct": 0, "brier": 0.7055999999999999, "old_estimate_a": 1400, "old_estimate_b": 1500, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game A’s Black played actively, but 15...Nxg5 allowed 16.Nf6+ and 17.Nxd7, losing the queen. Black’s eventual win depended heavily on White’s later errors, including 40.Qb8+??. Game B’s White handled the tactics more reliably: 20.Nb6 attacked Black’s queen while threatening the a8 rook, and 22.Bd5 kept pressure on the queen before 23.Bxd8 won it. B therefore looks likelier to have been the stronger established player.", "confidence_valid": true }, { "pair": "pair_03", "game_a": "game_01", "color_a": "white", "game_b": "game_06", "color_b": "white", "rating_a": 1363, "rating_b": 1112, "gap": 251, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.88, "correct": 1, "brier": 0.0144, "old_estimate_a": 1400, "old_estimate_b": 1300, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A’s White spotted a concrete tactic with 15.Nd5 and 16.Nf6+, winning Black’s queen with 17.Nxd7. Although 40.Qb8+ was a decisive blunder, White had pursued a dangerous passed pawn with 38.e6 and 39.e7. Game B’s White showed less consistent awareness of threats: 19.f5 allowed ...Nxa1 to take a rook, 25.Rf1 allowed ...Bxf1 to take the other, and 30.Qxb7 overlooked the mating attack. The contrast favors A despite both losses.", "confidence_valid": true }, { "pair": "pair_04", "game_a": "game_22", "color_a": "black", "game_b": "game_08", "color_b": "black", "rating_a": 557, "rating_b": 1791, "gap": 1234, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.01, "correct": 1, "brier": 0.0001, "old_estimate_a": 1150, "old_estimate_b": 1600, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A’s Black played ambitiously, but 6...Nxf2 sacrificed a knight without sufficient compensation. After 11...Nb5+ 12.Ke2 Qf6, White simply took that knight with 13.Qxb5; Black then also allowed 17.Qxc5. Game B’s Black played much more coherently: 14...c5 challenged White’s center, 15...Ndxc5 recaptured the bishop, and 20...Bg4 followed by 22...Bxf3+ and 24...Nf3+ showed coordinated tactics.", "confidence_valid": true }, { "pair": "pair_05", "game_a": "game_09", "color_a": "white", "game_b": "game_07", "color_b": "black", "rating_a": 2193, "rating_b": 1593, "gap": 600, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.25, "correct": 0, "brier": 0.5625, "old_estimate_a": 1800, "old_estimate_b": 1800, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "Game B’s Black played a sound opening and found the tactical 21...Bc2, attacking White’s queen and rook; after 22.Bh5, Black collected both with 22...Bxd1 and 23...Bxb3. Game A’s White fought back actively with 25.Rxa7 and 32.Rxe7, but allowed Black’s queen to take pawns on a3 and c3 and later struggled against active rooks. B’s play looks more controlled, though White’s abrupt collapse in Game B limits confidence.", "confidence_valid": true }, { "pair": "pair_06", "game_a": "game_17", "color_a": "black", "game_b": "game_25", "color_b": "black", "rating_a": 1639, "rating_b": 838, "gap": 801, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.97, "correct": 1, "brier": 0.0009000000000000016, "old_estimate_a": 1600, "old_estimate_b": 1250, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A’s Black played a coherent opening with 6...d5, then coordinated exchanges and central pressure with 20...Rxe4, 21...Rd8, and 24...c6. Game B’s Black spotted tactics against White’s exposed king, notably 13...Nc2+ and 20...Rxd4+, but developed awkwardly and ultimately left the queen on c8 to be taken after 33...Bd8 34.Rxc8. A’s play looks substantially more reliable, regardless of the recorded results.", "confidence_valid": true }, { "pair": "pair_07", "game_a": "game_14", "color_a": "black", "game_b": "game_08", "color_b": "white", "rating_a": 1188, "rating_b": 1310, "gap": 122, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.95, "correct": 0, "brier": 0.9025, "old_estimate_a": 1650, "old_estimate_b": 1300, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A’s Black handled the exchanges coherently with 20...Nxc5 and 28...Qxe5, then exploited White’s exposed bishop with 29...Rd4 and 30...Rxd3. Game B’s White made more consequential errors: 15.Bxc5 gave up a bishop for a pawn after 15...Ndxc5, and 17.Bxe4 Nxe4 lost the other bishop. Black then won White’s remaining knight with 22...Bxf3+. A looks substantially stronger despite an unpolished opening.", "confidence_valid": true }, { "pair": "pair_08", "game_a": "game_02", "color_a": "black", "game_b": "game_20", "color_b": "black", "rating_a": 2410, "rating_b": 1903, "gap": 507, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.98, "correct": 1, "brier": 0.0004000000000000007, "old_estimate_a": 1650, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game A’s Black played a coherent game: 18...Bxh4 met White’s advance, and 21...Re2 followed by 22...Rxb2 and 24...Rxh3 turned the open lines into a mating attack. Game B’s Black pursued queenside counterplay with 24...b3, but missed a decisive danger: after 27...Rh8 and 28...Kf8, White’s 29.Qxh8+ Bxh8 30.Rxh8+ led to 31.Rxd8, winning Black’s queen. A’s play looks substantially more controlled.", "confidence_valid": true }, { "pair": "pair_09", "game_a": "game_12", "color_a": "black", "game_b": "game_03", "color_b": "black", "rating_a": 1811, "rating_b": 1901, "gap": 90, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.82, "correct": 0, "brier": 0.6723999999999999, "old_estimate_a": 1600, "old_estimate_b": 1600, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "Both Black players made serious tactical mistakes, but A showed more purposeful counterplay after losing material to White’s 12.Bb5+ and 13.Qxd4, using ...c5, ...c4 and ...Rd4 to create threats. B’s 16...Kf7 left White’s bishop on d6 unchallenged; it then took the knight with 19.Bxf6. Later, B allowed 34.Ne7+ to win a rook and 38.Rxg8+ to win the queen. A’s play looks flawed, but B’s repeated missed threats suggest lower established strength.", "confidence_valid": true }, { "pair": "pair_10", "game_a": "game_23", "color_a": "black", "game_b": "game_09", "color_b": "black", "rating_a": 564, "rating_b": 2177, "gap": 1613, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.02, "correct": 1, "brier": 0.0004, "old_estimate_a": 1450, "old_estimate_b": 1950, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s Black played a coherent queenside plan with 10...b5 and 12...Rb8, then used 19...Nd4 and 21...Bxc3 to win material before converting with active rooks. Game A’s Black made a speculative 13...Bxh3 sacrifice and followed it with 15...Bh4, allowing 16.Nxh4 and leaving White with an extra knight. That large tactical oversight makes B the much likelier stronger player, despite the uncertainty of judging one game each.", "confidence_valid": true }, { "pair": "pair_11", "game_a": "game_21", "color_a": "black", "game_b": "game_10", "color_b": "black", "rating_a": 743, "rating_b": 2501, "gap": 1758, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.04, "correct": 1, "brier": 0.0016, "old_estimate_a": 1450, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s Black played a coherent opening against a normally developing opponent, then found active moves: 24...Qc2, 25...Ba6 and 26...Bxf1 won a rook, while 27...Ne8 attacked White’s queen. Game A’s Black exploited major errors by an opponent who brought the queen out on move 2, but chose a risky early ...f6 and later allowed 29.exd5 to take a knight. B’s play is stronger evidence of established rapid strength.", "confidence_valid": true }, { "pair": "pair_12", "game_a": "game_05", "color_a": "black", "game_b": "game_06", "color_b": "black", "rating_a": 1867, "rating_b": 1382, "gap": 485, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.68, "correct": 1, "brier": 0.10239999999999996, "old_estimate_a": 1700, "old_estimate_b": 1600, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A’s Black played a more coherent game: 11...f5 drove back White’s bishop, 16...Nxg2+ set up 17...fxe4, and 20...Nb4 followed by 21...Nbd3 supported the decisive 22...Bxe4. Game B’s Black also spotted strong tactics, notably 18...Ncxb3 and 25...Bxf1, but the early 9...Bf5 10...Bd7 lost time. Both opponents made serious mistakes, so the comparison is uncertain.", "confidence_valid": true }, { "pair": "pair_13", "game_a": "game_22", "color_a": "white", "game_b": "game_04", "color_b": "white", "rating_a": 717, "rating_b": 2508, "gap": 1791, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.2, "correct": 1, "brier": 0.04000000000000001, "old_estimate_a": 1350, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s White showed more coherent play: 7.O-O completed development, and 14.b4 and 20.a5 pursued a queenside plan. White then missed the mating threat with 24.Bb4, but that single oversight is less telling than Game A’s White playing 2.Qf3 and 3.g4, then exposing the king with 9.Ke3. Game A’s win also relied heavily on Black leaving pieces available to 13.Qxb5 and 17.Qxc5.", "confidence_valid": true }, { "pair": "pair_14", "game_a": "game_18", "color_a": "black", "game_b": "game_14", "color_b": "white", "rating_a": 2641, "rating_b": 1249, "gap": 1392, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.91, "correct": 1, "brier": 0.008099999999999994, "old_estimate_a": 1750, "old_estimate_b": 1550, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A’s Black coordinated a kingside attack with 14...h4, 16...Bh3, and the decisive 30...Re2+ 31...Rh2 32...Qh3+. Game B’s White developed some active ideas, but after 29...Rd4 attacked the bishop on d3, 30.Rg6 allowed 30...Rxd3, losing it. A’s sustained tactical play is stronger evidence of established strength, despite the risk of 10...Kf8.", "confidence_valid": true }, { "pair": "pair_15", "game_a": "game_05", "color_a": "white", "game_b": "game_20", "color_b": "white", "rating_a": 1120, "rating_b": 1932, "gap": 812, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.01, "correct": 1, "brier": 0.0001, "old_estimate_a": 1300, "old_estimate_b": 1900, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s White built a coherent kingside attack with 8.O-O-O, 11.h4 and 22.Rdh1, then calculated 26.Rxh7 through 31.Rxd8, winning Black’s queen. Game A’s White repeatedly retreated the bishop and lost it after 16.Be4 Nxg2+ 17.Kf1 fxe4; later, 27.Qxd3 allowed 27...Be4+ and the queen was lost. Despite the limited evidence of one game each, B’s play shows substantially stronger planning and tactical awareness.", "confidence_valid": true }, { "pair": "pair_16", "game_a": "game_11", "color_a": "black", "game_b": "game_02", "color_b": "white", "rating_a": 1273, "rating_b": 1575, "gap": 302, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.88, "correct": 0, "brier": 0.7744, "old_estimate_a": 1550, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A’s Black played an eccentric opening but then coordinated well: 10...Nxd5 recovered a pawn, 23...e4 challenged the center, and 25...Qxg3+ simplified. The late 40...e3?? allowed a back-rank mate. Game B’s White made more basic errors: 9.c3 allowed ...Nxe4, 18.h4 led to the loss of a knight after 18...Bxh4 19.Nxh4 Qxh4, and 24.Bc2 failed to address the rook’s attack on h3. Despite A’s final blunder, A appears more likely to have been the stronger established player.", "confidence_valid": true }, { "pair": "pair_17", "game_a": "game_13", "color_a": "white", "game_b": "game_16", "color_b": "white", "rating_a": 1126, "rating_b": 1752, "gap": 626, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.08, "correct": 1, "brier": 0.0064, "old_estimate_a": 1350, "old_estimate_b": 1600, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s White handled Black’s unsound 6...Bxf2+ calmly with 7.Kxf2, 8.Rf1 and 9.Kg1, then developed and simplified with 25.Qe2 while ahead. Game A’s White found useful tactics with 30.Nxe6 and 31.Rxc6, but spent many opening moves shuffling the rook, lost it to 18...bxc5, and missed the mating threat before 35...Rc1#. B’s play looks more consistently controlled.", "confidence_valid": true }, { "pair": "pair_18", "game_a": "game_24", "color_a": "black", "game_b": "game_12", "color_b": "white", "rating_a": 765, "rating_b": 1877, "gap": 1112, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.02, "correct": 1, "brier": 0.0004, "old_estimate_a": 1250, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s White played a coherent opening and found tactical resources: 12.Bb5+ followed by 13.Qxd4 recovered the bishop, while 22.Bf4+ and 27.Bxh8 led to a decisive attack. Game A’s Black pushed several kingside pawns without developing or securing the king, then allowed 20.Nb6 and 21.Nxa8 to lose a rook. Black also missed 22.Bd5, which attacked the queen and led to 23.Bxd8. Despite some imperfect play by B’s White, B shows substantially stronger tactical awareness.", "confidence_valid": true }, { "pair": "pair_19", "game_a": "game_17", "color_a": "white", "game_b": "game_10", "color_b": "white", "rating_a": 1514, "rating_b": 2689, "gap": 1175, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.18, "correct": 1, "brier": 0.0324, "old_estimate_a": 1550, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s White shows more coherent development and planning: 7.Bb2, 11.Rc1, 17.Rfd1 and 22.Rxc8. The later 25.Rf1? allowed 25...Ba6 and 26...Bxf1, but that oversight does not erase the stronger earlier play. Game A’s White spent several moves retreating or repositioning the same pieces—12.Neg5 13.Ne4 and 19.Bb3 20.Bc2—after weakening the kingside with 11.g4. Game A’s result alone is not persuasive evidence of greater established strength.", "confidence_valid": true }, { "pair": "pair_20", "game_a": "game_11", "color_a": "white", "game_b": "game_23", "color_b": "white", "rating_a": 1228, "rating_b": 557, "gap": 671, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.77, "correct": 1, "brier": 0.05289999999999999, "old_estimate_a": 1650, "old_estimate_b": 1600, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A White showed more sustained planning against a relatively resilient opponent: 25.f5 secured a queen trade, 30.f6 restricted Black, and 38.Rb5 followed by 40.Rd5 created the decisive rook threat. Game B White correctly accepted Black’s unsound bishop play with 14.gxh3 and 16.Nxh4, then handled the queen checks, but much of that advantage came from Black giving up material. A’s play across the longer, contested game is stronger evidence of established strength.", "confidence_valid": true }, { "pair": "pair_21", "game_a": "game_21", "color_a": "white", "game_b": "game_15", "color_b": "white", "rating_a": 739, "rating_b": 2157, "gap": 1418, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.04, "correct": 1, "brier": 0.0016, "old_estimate_a": 1250, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s White developed and castled normally, then handled the tactical sequence 24.Rd1 Nxb7 25.Qxc8+ Nxc8 26.Rxd5 to reach a manageable endgame. Game A’s White spent the opening moving the queen with 2.Qd3, 5.Qxd5 and 7.Qxb7 while leaving pieces undeveloped, then allowed 19...Nc2+ and 21...Nxf1+ to cost material. B’s play is much stronger evidence of established rapid strength, despite the different results.", "confidence_valid": true }, { "pair": "pair_22", "game_a": "game_04", "color_a": "black", "game_b": "game_13", "color_b": "black", "rating_a": 2718, "rating_b": 1551, "gap": 1167, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.88, "correct": 1, "brier": 0.0144, "old_estimate_a": 1800, "old_estimate_b": 1500, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A’s Black built a coherent attack with 16...Ne4, 17...f5 and 22...Nd4, then finished with the calculated 24...Ne2+ 25...Qg1+ 26...Nf2#. Game B’s Black won after White ignored the threat to the rook on c5, but the early queen excursion with 5...Qb5 and 6...Qxb2 was risky, and 28...Re6 allowed 29.Nc7 to fork the rooks. A’s play is more convincing evidence of established strength.", "confidence_valid": true }, { "pair": "pair_23", "game_a": "game_25", "color_a": "white", "game_b": "game_18", "color_b": "white", "rating_a": 738, "rating_b": 2426, "gap": 1688, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.08, "correct": 1, "brier": 0.0064, "old_estimate_a": 1200, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s White developed and contested the center with 7.O-O, 8.d4 and 11.Nxd4, then navigated exchanges with 27.Bxc5+ and 28.Qxc3. The late 32.Qxd5? was a serious mistake, but Game A’s White showed more persistent problems: 3.Qf3 and 4.g4 neglected development, and 13.Nh3 allowed 13...Nc2+ 14.Kd2 Nxa1. A’s win also depended on Black’s 33...Bd8, which allowed 34.Rxc8. B looks more likely to have had the higher established strength despite losing.", "confidence_valid": true }, { "pair": "pair_24", "game_a": "game_03", "color_a": "white", "game_b": "game_15", "color_b": "black", "rating_a": 1782, "rating_b": 2154, "gap": 372, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.34, "correct": 1, "brier": 0.11560000000000002, "old_estimate_a": 1800, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s Black shows more sustained positional control: 16...Rxa4 finds counterplay, 21...Ne4 and 22...c5 challenge White’s center, and 39...Bxc5 leads to an active king-and-pawn ending. Game A’s White exploits Black’s errors well, notably with 17.Ne5+ and 34.Ne7+, but the early 8.h4, 9.Ng5, 11.Nh3 sequence spends several tempi on the same knight. A’s tactical win is impressive, but B’s play against a more resistant opponent is slightly stronger evidence of established strength.", "confidence_valid": true }, { "pair": "pair_25", "game_a": "game_19", "color_a": "black", "game_b": "game_07", "color_b": "white", "rating_a": 2474, "rating_b": 1862, "gap": 612, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.27, "correct": 0, "brier": 0.5328999999999999, "old_estimate_a": 1700, "old_estimate_b": 1650, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game B’s White developed coherently and pursued a clear queenside plan with 17.b4 and 19.b5, though 21.Bf3 and 22.Bh5 missed that ...Bc2 would let Black take the queen and then the rook on b3. Game A’s Black also played a reasonable opening, but allowed 22.Nxh5, made little progress against White’s advancing h-pawn, and overlooked 35.Ra8+ followed by 36.Rxh8. B’s play before the tactical mistake looks more convincing.", "confidence_valid": true } ] }, { "model": "openai/gpt-6.1-sol", "correct": 19, "n": 25, "accuracy": 76.0, "baseline_accuracy": 80, "invalid_confidence": 14, "consistent_reversals": 4, "reversals": [ { "pair": "pair_09_reverse", "original": "pair_09", "prediction": { "stronger": "B", "probability_a_stronger": 0.34, "explanation": "Game B Black showed more purposeful counterplay: 17...Nd5, 21...c4, and the rook invasion 23...Rd4–25...Rd2 created concrete threats, culminating in 26...Nxc4 winning a bishop. However, 10...Nxd4 allowed the bishop-losing sequence through 13.Qxd4, and 27...Nxe3 overlooked White’s decisive checking attack. Game A Black also generated activity, but 33...Nc8 allowed 34.Ne7+ followed by Nxg8, while 44...Kh8 simply left the attacked rook on g6 to be captured. B’s stronger coordination and tactical counterplay narrowly outweigh those serious mistakes." }, "consistent": false }, { "pair": "pair_15_reverse", "original": "pair_15", "prediction": { "stronger": "A", "probability_a_stronger": 0.96, "explanation": "Game A White showed coherent attacking play with opposite-side castling, doubling rooks via 22.Rdh1 and calculating 26.Rxh7! followed by 28.fxg6+, 29.Qxh8+ and 31.Rxd8 to recover Black’s queen. The finish, 35.g6, 36.Ng5 and 38.Nh6#, demonstrates coordinated tactical execution. Game B White repeatedly lost opening tempi with the bishop and overlooked concrete threats: 16.Be4 allowed ...Nxg2+ and ...fxe4, while 23.Nd4 permitted ...Bxh1. Finally, 27.Qxd3 allowed ...Be4+, winning the queen. Although A benefited from defensive mistakes, its sustained planning and calculation look substantially stronger than B’s play." }, "consistent": true }, { "pair": "pair_13_reverse", "original": "pair_13", "prediction": { "stronger": "A", "probability_a_stronger": 0.94, "explanation": "Game A White showed coherent development and planning: 7.O-O, 12.Nd2, 15.Nb3 and 18.Nc5, followed by a queenside pawn attack against Black’s castled king. White nevertheless underestimated Black’s counterplay and missed the smothered-mate sequence after 24.Bb4 Ne2+. Game B White’s 2.Qf3, 3.g4 and 6.h3 neglected development and exposed the king, which wandered through f2, e3, e2 and d1. Although White collected material with 13.Qxb5 and 17.Qxc5, the win depended heavily on Black’s speculative sacrifices and especially 22...Qxg1+, surrendering the queen for a rook. A’s overall play suggests substantially stronger established ability despite the loss." }, "consistent": true }, { "pair": "pair_12_reverse", "original": "pair_12", "prediction": { "stronger": "B", "probability_a_stronger": 0.73, "explanation": "Game B Black showed more coherent development and coordinated tactical play: 11...f5 undermined White’s center, 15...Nf4 and 16...Nxg2+ exploited the bishop’s vulnerability, and ...Nb4–d3 intensified the attack. Particularly convincing was 27...Be4+, uncovering the f8-rook’s check while attacking White’s queen. Game A Black also found effective tactics, notably 17...Nc5–18...Ncxb3 and 25...Bxf1, but the opening featured substantial bishop shuffling (...Bd7–f5–d7–g4–e6) against White’s very inefficient development. Both opponents made serious errors, but B’s sustained coordination provides the stronger evidence." }, "consistent": true }, { "pair": "pair_18_reverse", "original": "pair_18", "prediction": { "stronger": "A", "probability_a_stronger": 0.97, "explanation": "Game A White showed substantially better coordination and tactical awareness: 12.Bb5+ followed by 13.Qxd4 recovered the sacrificed material, while 22.Bf4+, 24.Rfe1 and the final 28.Qg8+ demonstrated purposeful attacking play. Game B Black neglected development and king safety with 7...f6, 8...h5 and 9...g5. More decisively, 20...Qc6 allowed Nb6–Nxa8, losing a rook, and 22...Qd8 placed the queen directly on White’s bishop diagonal, allowing 23.Bxd8. Although A benefited from inaccurate opposition, B’s repeated basic material oversights strongly favor A." }, "consistent": true } ], "rows": [ { "pair": "pair_01", "game_a": "game_16", "color_a": "black", "game_b": "game_19", "color_b": "white", "rating_a": 1569, "rating_b": 2668, "gap": 1099, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.93, "correct": 1, "brier": null, "old_estimate_a": 1550, "old_estimate_b": 1900, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A’s Black played the speculative 6...Bxf2+, sacrificing a bishop without a convincing continuation. The sequence 12...Bxf3 13.hxg4 Bxe2 left Black a minor piece down, and subsequent exchanges offered little compensation. Game B’s White showed substantially more coherent planning: 12.Kd2 activated the king after the queen trade, 22.Nxh5 won a pawn, and h5–h6 created a dangerous passer. White then combined central pressure with seventh-rank activity through 31.d5, 32.Re1 and 33.Re7+, followed by 35.Ra8+ and 36.Rxh8. Black’s cooperation in Game B limits certainty, but White’s sustained coordination is much stronger evidence of established playing strength.", "confidence_valid": false }, { "pair": "pair_02", "game_a": "game_01", "color_a": "black", "game_b": "game_24", "color_b": "white", "rating_a": 1159, "rating_b": 762, "gap": 397, "gap_band": "200-499", "low_band": "one<1000", "gold": "A", "choice": "B", "probability_a": 0.79, "correct": 0, "brier": null, "old_estimate_a": 1300, "old_estimate_b": 1500, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game B White developed more coherently and spotted 20.Nb6, forking Black’s queen and rook, followed by 21.Nxa8. White also exploited 22...Qd8 with 23.Bxd8, though 15.g3 and the subsequent knight sacrifice show imperfect play. Game A Black weakened the position with early pawn pushes and, crucially, allowed 15...Nxg5 16.Nf6+ followed by 17.Nxd7, losing the queen to a straightforward fork. Black showed resourcefulness with 27...e4 and 30...Bxg3–31...Bxe1, but the eventual win depended heavily on White’s errors, especially 40.Qb8+. B’s play provides stronger evidence of established playing strength.", "confidence_valid": false }, { "pair": "pair_03", "game_a": "game_01", "color_a": "white", "game_b": "game_06", "color_b": "white", "rating_a": 1363, "rating_b": 1112, "gap": 251, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.79, "correct": 1, "brier": 0.044099999999999986, "old_estimate_a": 1300, "old_estimate_b": 1200, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A White showed more coherent development, castled early, and exploited Black’s queen placement with 15.Nd5 and 16.Nf6+, winning the queen on d7. White later made serious errors: 31.e5 allowed ...Bxe1, and 40.Qb8+ overlooked Black’s second rook in the promotion combination. Game B White spent opening tempi on Bd3–b1 and f3–f4 while leaving the king uncastled. After 18...Ncxb3, 19.f5 allowed ...Nxa1; later 25.Rf1 simply permitted ...Bxf1, followed by losing the knight to ...Qxf6. Despite facing a more coherent opponent, B’s repeated development and material-safety failures suggest lower established strength.", "confidence_valid": true }, { "pair": "pair_04", "game_a": "game_22", "color_a": "black", "game_b": "game_08", "color_b": "black", "rating_a": 557, "rating_b": 1791, "gap": 1234, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.97, "correct": 1, "brier": null, "old_estimate_a": 800, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B Black develops coherently with ...O-O, ...Re8 and ...Nd7, then exploits White’s mistakes: 14...c5 15.Bxc5 Ndxc5 wins a bishop, while 22...Bxf3+ wins another piece and 24...Nf3+ forks the king and both rooks. Game A Black attacks an exposed king but repeatedly sacrifices material without sufficient compensation: 6...Nxf2, 11...Nb5+ allowing Qxb5, and ultimately 22...Qxg1+ allowing Nxg1, exchanging the queen for a rook. Although both opponents offer opportunities, B shows much stronger development, tactical execution and material judgment.", "confidence_valid": false }, { "pair": "pair_05", "game_a": "game_09", "color_a": "white", "game_b": "game_07", "color_b": "black", "rating_a": 2193, "rating_b": 1593, "gap": 600, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.64, "correct": 0, "brier": null, "old_estimate_a": 1900, "old_estimate_b": 1800, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game B Black showed coordinated tactical play: 18...Qe6 and 20...Qg6 created kingside threats, while 21...Bc2 forked White’s queen and rook. After 22.Bh5, Black correctly chose 22...Bxd1 and 23...Bxb3, emerging a rook ahead despite losing the queen. Game A White handled a more demanding position and generated activity with 25.Rxa7 and 28.e5, but allowed the sequence 19...Nd4 through 23...Qxc3 to cost two pawns, then failed to contain Black’s second-rank rooks and mating net. B’s opponent supplied a major tactical opportunity, so the comparison remains uncertain, but B’s cleaner execution slightly outweighs A’s greater complexity.", "confidence_valid": false }, { "pair": "pair_06", "game_a": "game_17", "color_a": "black", "game_b": "game_25", "color_b": "black", "rating_a": 1639, "rating_b": 838, "gap": 801, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.96, "correct": 1, "brier": 0.001600000000000003, "old_estimate_a": 1750, "old_estimate_b": 1050, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black showed coherent development and central play with 6...d5, 8...Bg4, and 13...Nf4, then coordinated the rooks through 19...Rfe8 and 21...Rd8. The final position does not indicate an obvious decisive blunder. Game B Black found tactics such as 13...Nc2+ and 20...Rxd4+, but against highly unconventional opposition. Black’s early ...Bd7, ...Na6, and ...Rd8 neglected kingside development, while 32...Nxf5 and 33...Bd8 allowed 34.Rxc8, losing the queen. A’s play is substantially more consistent with established strength.", "confidence_valid": true }, { "pair": "pair_07", "game_a": "game_14", "color_a": "black", "game_b": "game_08", "color_b": "white", "rating_a": 1188, "rating_b": 1310, "gap": 122, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.87, "correct": 0, "brier": 0.7569, "old_estimate_a": 1550, "old_estimate_b": 1250, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A Black showed better tactical awareness: 10...Bxh4 answered White’s capture on d5 by recovering a knight, and 20...Nxc5 removed White’s advanced knight. Black then coordinated the rooks with 26...Rad8 and 29...Rd4, exploiting 30.Rg6 with 30...Rxd3. The opening was slow, notably ...h6 and the bishop retreat to e7, but subsequent play was coherent. Game B White developed reasonably, yet 15.Bxc5 surrendered a bishop for a pawn without adequate compensation. Later, 22.Rd4 left the knight on f3 available to ...Bxf3+, and 24.h4 allowed ...Nf3+, forking both rooks. These repeated material oversights favor A.", "confidence_valid": true }, { "pair": "pair_08", "game_a": "game_02", "color_a": "black", "game_b": "game_20", "color_b": "black", "rating_a": 2410, "rating_b": 1903, "gap": 507, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.76, "correct": 1, "brier": 0.0576, "old_estimate_a": 1650, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game A Black played more coherently: 9...Nxe4 exploited the center, 18...Bxh4 won another pawn through 19.Nxh4 Qxh4, and 21...Re2 converted the queen trade into active rook play. The finish 24...Rxh3 and 25...Rh1# correctly used the b7 bishop. Game B Black showed sensible queenside counterplay, but the repeated knight maneuvers consumed time while White built a clear kingside attack. Black's 25...Kf7 allowed the forcing sequence beginning 26.Rxh7, ultimately losing the queen on d8; 36...c4 also ignored the mating threat. A's opposition was weaker, which limits confidence, but A demonstrated better tactical control.", "confidence_valid": true }, { "pair": "pair_09", "game_a": "game_12", "color_a": "black", "game_b": "game_03", "color_b": "black", "rating_a": 1811, "rating_b": 1901, "gap": 90, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.66, "correct": 1, "brier": null, "old_estimate_a": 1600, "old_estimate_b": 1500, "baseline_choice": "A", "baseline_credit": 0, "explanation": "B looks somewhat stronger despite both losses. Game A Black’s 10...Nxd4 and 11...Bxd4 overlooked 12.Bb5+ followed by 13.Qxd4, losing a piece. Later, 27...Nxe3 allowed the decisive back-rank check 28.Qg8+. Game B Black showed more coherent central play with 11...c5 and 13...d5, then purposeful rook activity through 27...Rac8, 28...Rc4 and 29...Rg8. B’s 16...Kf7 and subsequent king exposure were serious weaknesses, but the overall development and counterplay suggest a somewhat higher established strength than A’s repeated tactical oversights.", "confidence_valid": false }, { "pair": "pair_10", "game_a": "game_23", "color_a": "black", "game_b": "game_09", "color_b": "black", "rating_a": 564, "rating_b": 2177, "gap": 1613, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.98, "correct": 1, "brier": null, "old_estimate_a": 1300, "old_estimate_b": 2100, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B Black shows coherent Benoni play: 10...b5, the knight maneuver ...Ng4–e5–c4, and 19...Nd4 followed by ...Nxe2 and ...Bxc3 simplify effectively and win queenside pawns. Black then coordinates the rooks with 25...Rb2 and finishes with 38...Rdd2, 39...Rh2+ and 40...Rbg2+, creating a mating net. Game A Black’s 13...Bxh3, 14...f5 and 15...Bh4 sacrifice substantial material without a convincing continuation; the subsequent queen checks produce little. B’s sustained positional planning and tactical execution strongly suggest the higher established rapid strength.", "confidence_valid": false }, { "pair": "pair_11", "game_a": "game_21", "color_a": "black", "game_b": "game_10", "color_b": "black", "rating_a": 743, "rating_b": 2501, "gap": 1758, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.96, "correct": 1, "brier": null, "old_estimate_a": 1250, "old_estimate_b": 2050, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B Black shows coherent development and purposeful coordination: 14...Rfe8, 17...Bd6 and 19...Qf7 improve the position, while 23...Bc5, 24...Qc2 and 25...Ba6 exploit White’s pawn grab to win the exchange with 26...Bxf1. Game A Black’s opening, especially 2...e5 and 3...Qe7, allows substantial pawn losses without convincing compensation. Although 19...Nc2+ and 20...Ne3+ exploit White’s mistakes effectively, 25...Bxa4 simply permits 26.bxa4, surrendering a bishop. A’s eventual victory relies much more heavily on the opponent’s repeated tactical errors; B’s play is substantially more consistent and sophisticated.", "confidence_valid": false }, { "pair": "pair_12", "game_a": "game_05", "color_a": "black", "game_b": "game_06", "color_b": "black", "rating_a": 1867, "rating_b": 1382, "gap": 485, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.67, "correct": 1, "brier": 0.10889999999999997, "old_estimate_a": 1650, "old_estimate_b": 1550, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black showed more coherent development and sustained tactical coordination: 15...Nf4, 16...Nxg2+ and 17...fxe4 exploited White’s bishop placement, while 21...Nbd3 and 22...Bxe4 led to winning the rook. The finish with 24...Qb7, 25...Nh3+ and 27...Be4+ was particularly coordinated. Game B Black also found effective tactics—17...Nc5, 18...Ncxb3 and 25...Bxf1—and finished accurately with 30...Rc1+. However, the repeated early bishop moves (...Bd7–f5–d7–g4–e6) suggest less efficient opening handling. Both benefited from substantial White errors, so the preference for A is moderate.", "confidence_valid": true }, { "pair": "pair_13", "game_a": "game_22", "color_a": "white", "game_b": "game_04", "color_b": "white", "rating_a": 717, "rating_b": 2508, "gap": 1791, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.94, "correct": 1, "brier": null, "old_estimate_a": 900, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B White shows substantially more coherent development and planning: 4.Bd3, 6.Ne2 and 7.O-O, followed by the coordinated queenside expansion 14.b4, 15.Nb3 and 18.Nc5. White ultimately overlooks the mating combination beginning 24...Ne2+, but the preceding play suggests stronger established understanding. Game A White’s 2.Qf3, 3.g4 and 4.g5 neglect development and king safety; 6.h3 allows ...Nxf2, forcing an exposed king walk. A’s win relies heavily on Black’s poor attack and especially 22...Qxg1+, surrendering the queen for a rook, rather than demonstrating comparable overall strength.", "confidence_valid": false }, { "pair": "pair_14", "game_a": "game_18", "color_a": "black", "game_b": "game_14", "color_b": "white", "rating_a": 2641, "rating_b": 1249, "gap": 1392, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.8, "correct": 1, "brier": 0.03999999999999998, "old_estimate_a": 1750, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black showed more coherent tactical play: 14...h4 and 16...Bh3 opened attacking lines, while 29...Re8, 30...Re2+, and 31...Rh2 coordinated the remaining heavy pieces against White’s king. After 32.Qxd5, Black continued with forcing checks rather than abandoning the attack. Game B White developed reasonably and gained central space, but the knight excursion 17.Nd6–18.Ne4 achieved little. More importantly, 30.Rg6 overlooked the direct capture 30...Rxd3, losing a bishop without evident compensation. A’s sustained attacking coordination is stronger evidence than B’s otherwise plausible buildup.", "confidence_valid": true }, { "pair": "pair_15", "game_a": "game_05", "color_a": "white", "game_b": "game_20", "color_b": "white", "rating_a": 1120, "rating_b": 1932, "gap": 812, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.95, "correct": 1, "brier": null, "old_estimate_a": 1200, "old_estimate_b": 2000, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B White showed a coherent attacking plan: queenside castling, h-pawn pressure, 22.Rdh1 and 23.Qh2, followed by the forcing 26.Rxh7–31.Rxd8 sequence. White then coordinated both knights and the passed pawn with 33.Nf5, 36.Ng5 and 38.Nh6#. Game A White spent several opening tempi relocating the bishop (2.Bc4, 5.Bd3, 6.Be2), then 16.Be4 allowed ...Nxg2+ and ...fxe4, losing a bishop for pawns. Later, 22.Bc3 and 23.Nd4 permitted ...Bxe4 and ...Bxh1. Although A faced stronger resistance, B’s sustained coordination and tactical execution provide substantially stronger evidence of established playing strength.", "confidence_valid": false }, { "pair": "pair_16", "game_a": "game_11", "color_a": "black", "game_b": "game_02", "color_b": "white", "rating_a": 1273, "rating_b": 1575, "gap": 302, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.79, "correct": 0, "brier": 0.6241000000000001, "old_estimate_a": 1550, "old_estimate_b": 1250, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A Black showed more sustained middlegame competence: 23...e4 and 24...d5 gained central space, 28...Nd3 established an outpost, and 33...Rd8 followed by ...Rdd6 and ...Rc6 activated both rooks. The decisive 40...e3 overlooked the back-rank mating threat, but followed substantial purposeful play. Game B White developed reasonably, then 18.h4 and 19.Nxh4 led to a pawn-down queen trade without a convincing attack. White allowed ...Re2–...Rxb2–...Rb3–...Rxh3 and finally missed the immediate mate with 25.Bg5. A’s broader positional coordination is stronger evidence despite both targets losing.", "confidence_valid": true }, { "pair": "pair_17", "game_a": "game_13", "color_a": "white", "game_b": "game_16", "color_b": "white", "rating_a": 1126, "rating_b": 1752, "gap": 626, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.95, "correct": 1, "brier": null, "old_estimate_a": 1100, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White repeatedly spent tempi on queen and rook excursions (10.Qa1, 13.Ra3, 14.Rc3), then left the attacked rook hanging with 18.Qb2 bxc5. Despite later tactical recovery, 35.Nxb5 allowed immediate mate with ...Rc1#. Game B White handled Black’s speculative sacrifice coherently: 7.Kxf2 followed by 8.Rf1 and 9.Kg1 secured the king, while 12.h3 and 14.Qxe2 emerged with extra material. Subsequent play—17.Rae1, 22.Nb3, 23.Nxd4, and 25.Qe2—showed purposeful coordination and simplification. B’s opponent played unsoundly, but B’s much cleaner execution strongly favors the higher established rating.", "confidence_valid": false }, { "pair": "pair_18", "game_a": "game_24", "color_a": "black", "game_b": "game_12", "color_b": "white", "rating_a": 765, "rating_b": 1877, "gap": 1112, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.03, "correct": 1, "brier": 0.0009, "old_estimate_a": 1150, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black neglects development and king safety with 7...f6, 8...h5 and 9...g5. Although 13...Nd4 shows some positional awareness, Black allows the rook fork 20.Nb6 and then simply hangs the queen with 22...Qd8 23.Bxd8. Game B White demonstrates substantially better tactical coordination: 12.Bb5+ followed by 13.Qxd4 recovers material, and 22.Bf4+ gains time before 23.Bxc4. Most impressively, 27.Bxh8 allows 27...Nxe3 because 28.Qg8+ forces mate after any interposition. B's coherent development and concrete calculation strongly outweigh A's play.", "confidence_valid": true }, { "pair": "pair_19", "game_a": "game_17", "color_a": "white", "game_b": "game_10", "color_b": "white", "rating_a": 1514, "rating_b": 2689, "gap": 1175, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.61, "correct": 1, "brier": null, "old_estimate_a": 1700, "old_estimate_b": 1900, "baseline_choice": "B", "baseline_credit": 1, "explanation": "B White showed more coherent opening and positional play: 10.Nc3, 11.Rc1, 13.Nd4 and 17.Rfd1 coordinated development and central pressure. However, 24.Nxa7 followed by 26.Be5 overlooked Black’s ...Ba6–...Bxf1 tactic, losing the exchange. A White’s 11.g4 weakened the king, while 12.Neg5 h6 13.Ne4 wasted tempi; 15.Kg2 compounded the awkward king placement. A subsequently played reasonably with 18.c3 and 24.Rd5, but the recorded win after ...c6 does not demonstrate a decisive advantage. B’s stronger positional foundation narrowly outweighs the tactical lapse.", "confidence_valid": false }, { "pair": "pair_20", "game_a": "game_11", "color_a": "white", "game_b": "game_23", "color_b": "white", "rating_a": 1228, "rating_b": 557, "gap": 671, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "B", "probability_a": 0.6, "correct": 0, "brier": null, "old_estimate_a": 1650, "old_estimate_b": 1550, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game B White showed cleaner tactical accounting: 14.gxh3, 15.Nxf5 and 16.Nxh4 converted Black’s unsound attack into a material advantage. White then handled queen pressure coherently with 20.Qe2, 21.Qh2 and 23.Rg1, offering exchanges while ahead. Game A White found useful active play with 30.f6 and 38.Rb5, then spotted 41.Rd8+ and mate. However, 19.Qh4 allowed damaged kingside pawns, and the subsequent play conceded central activity and the c4 pawn. B’s opponent was much more reckless, so the inference is modest rather than decisive.", "confidence_valid": false }, { "pair": "pair_21", "game_a": "game_21", "color_a": "white", "game_b": "game_15", "color_b": "white", "rating_a": 739, "rating_b": 2157, "gap": 1418, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.98, "correct": 1, "brier": null, "old_estimate_a": 1100, "old_estimate_b": 1900, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B White shows substantially more coherent development, castling and coordination. The sequence 24.Rd1, 25.Qxc8+ and 26.Rxd5 navigates the tactical exchanges effectively; later, 53.Kf1–55.Kh1 secures the drawing corner against Black’s rook pawn. Game A White opens with queen excursions and f-pawn moves, then allows Black’s knight raid with 19.a3 and 20.Kd1, losing the f1 bishop. White also permits 29...Nf2 followed by 30...Nxh1, and simply hangs a knight with 39.Ne5+ Rxe5. Despite imperfect play in Game B, the gap in tactical reliability strongly favors B.", "confidence_valid": false }, { "pair": "pair_22", "game_a": "game_04", "color_a": "black", "game_b": "game_13", "color_b": "black", "rating_a": 2718, "rating_b": 1551, "gap": 1167, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.86, "correct": 1, "brier": 0.019600000000000003, "old_estimate_a": 2000, "old_estimate_b": 1350, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black showed coherent development and coordinated attacking play: 16...Ne4, 19...g5 and 22...Nd4 established active central knights, culminating in the accurately calculated 24...Ne2+ 25...Qg1+ 26...Nf2# combination. Game B Black also recognized a mating opportunity with 34...Rc7 and 35...Rc1#, but benefited substantially from White's rook wandering and 18.Qb2, which allowed ...bxc5. More tellingly, B's 28...Re6 allowed 29.Nc7 to fork both rooks, followed by 30.Nxe6 and 31.Rxc6, surrendering much of the earlier material advantage. A's more consistent coordination and tactical accuracy suggest the higher established strength.", "confidence_valid": true }, { "pair": "pair_23", "game_a": "game_25", "color_a": "white", "game_b": "game_18", "color_b": "white", "rating_a": 738, "rating_b": 2426, "gap": 1688, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.96, "correct": 1, "brier": null, "old_estimate_a": 1150, "old_estimate_b": 1550, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B White shows substantially stronger opening and middlegame fundamentals: castling, coordinated development, 10.Re1+ forcing Black's king to f8, and sensible central exchanges. White later underestimated Black's attack with 30.Rd1 and especially 32.Qxd5, allowing decisive checks. Game A White instead played 3.Qf3 and repeated flank-pawn pushes while leaving the king and pieces undeveloped; 13.Nh3 allowed 13...Nc2+ and the loss of the a1 rook. A's eventual tactical success—33.Ra8 followed by 34.Rxc8 and a mating sequence—came against highly cooperative opposition and does not outweigh the much weaker earlier play.", "confidence_valid": false }, { "pair": "pair_24", "game_a": "game_03", "color_a": "white", "game_b": "game_15", "color_b": "black", "rating_a": 1782, "rating_b": 2154, "gap": 372, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.61, "correct": 0, "brier": 0.3721, "old_estimate_a": 1800, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White showed sustained tactical coordination: 16.Bxd6 exploited the potential Ne7+ queen fork, while 34.Ne7+ followed by 35.Nxg8 won material. White then used 37.Rd8 and finished accurately with 46.Re8#. Game B Black demonstrated competence with the liquidation 24...Nxb7 25...Nxc8 26...exd5 and active king play. However, 44...gxh4 and 45...h5 simplified the remaining chances into an ultimately drawn rook-pawn ending. A’s stronger tactical execution gives White a modest edge, tempered by Game A Black’s weaker resistance.", "confidence_valid": true }, { "pair": "pair_25", "game_a": "game_19", "color_a": "black", "game_b": "game_07", "color_b": "white", "rating_a": 2474, "rating_b": 1862, "gap": 612, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.65, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1800, "old_estimate_b": 1650, "baseline_choice": "A", "baseline_credit": 1, "explanation": "A Black showed more sustained positional and endgame competence: 22...c5 and 23...c4+ created counterplay, while ...Nf8–e6 and ...Nd8–b7–d6 improved the knight. However, 34...Nxf5 allowed 35.Ra8+, losing the rook and permitting promotion. B White developed sensibly and pursued a coherent minority attack with b4–b5, but mishandled the tactical threat after 21...Bc2: 22.Bh5 Bxd1 23.Bxg6 Bxb3 traded queens while losing a rook. Both made serious errors, but A’s longer stretch of purposeful play provides somewhat stronger evidence of established strength.", "confidence_valid": true } ] }, { "model": "openai/gpt-5.6-luna", "correct": 18, "n": 25, "accuracy": 72.0, "baseline_accuracy": 78.0, "invalid_confidence": 5, "consistent_reversals": 4, "reversals": [ { "pair": "pair_09_reverse", "original": "pair_09", "prediction": { "stronger": "B", "probability_a_stronger": 0.2, "explanation": "Game B’s black showed more coherent development and active play: castling long on move 14, centralizing with ...Rd4 and ...Rd2, and using ...Nxc4 and ...Nxe3 to win material. The decisive error was tactical, overlooking 28.Qg8+ after 27...Nxe3. Game A’s black displayed more persistent weaknesses: 16...Kf7 and 17...Kg8 exposed the king, 19...gxf6 damaged the structure, and 23...Nd7 allowed 23.Nxd5. Black then failed to contain the queen invasion with 27.Qxc5 and ultimately allowed the forced mating attack ending 46.Re8#. Thus B appears more likely to have had the higher established rapid strength." }, "consistent": false }, { "pair": "pair_15_reverse", "original": "pair_15", "prediction": { "stronger": "A", "probability_a_stronger": 0.9, "explanation": "Game A White demonstrated a coherent attacking plan with 9.g4, 11.h4, and 12.h5, then accurately exploited the kingside weaknesses. The sequence 26.Rxh7 Nxh7 27.Qxh7 Rh8 28.fxg6+ and 29.Qxh8+ shows useful tactical calculation, followed by a clean mating finish with 37.g7+ and 38.Nh6#. Game B White played less accurately: 16...Nxg2+ led to king displacement, and after 22...Bxe4, 23.Nd4? Bxh1 simply lost the rook. White then failed to contain the attack, allowing 25...Nh3+ and 28...Bxd3+, followed by 29...Qxf3#. On these games, A’s target White appears substantially stronger." }, "consistent": true }, { "pair": "pair_13_reverse", "original": "pair_13", "prediction": { "stronger": "A", "probability_a_stronger": 0.68, "explanation": "Game A’s White used a generally coherent opening and development, including 4 Bd3, 7 O-O, and queenside space with 14 b4 and 20 a5. The loss came from a specific tactical oversight: after 22 b5 Nd4, White allowed 24...Ne2+ and the mating sequence 25...Qg1+ 26 Rxg1 Nf2#. Game B’s White showed tactical resourcefulness by exploiting Black’s loose play with 13 Qxb5, 17 Qxc5, and 18 Qxd5, but the opening (2 Qf3, 3 g4, 6 h3) and resulting king exposure were highly unsound. Thus A more likely had the higher established rapid strength despite losing." }, "consistent": true }, { "pair": "pair_12_reverse", "original": "pair_12", "prediction": { "stronger": "B", "probability_a_stronger": 0.38, "explanation": "Game B’s Black showed the more coherent tactical calculation: 16...Nxg2+ forced the king away, 17...fxe4 won the attacked bishop, and 22...Bxe4 followed by 23...Bxh1 captured a rook. The subsequent 26...Bxf3 and 28...Bxd3+ maintained a forcing attack ending in mate. Game A’s Black also found strong tactics, especially 18...Ncxb3 and 19...Nxa1, then 25...Bxf1 and 27...Qxf6, but the game was somewhat more dependent on White’s exposed, inaccurate play. Thus B’s target appears more likely to have had the higher established rapid strength." }, "consistent": true }, { "pair": "pair_18_reverse", "original": "pair_18", "prediction": { "stronger": "A", "probability_a_stronger": 0.92, "explanation": "Game A’s White showed substantially better tactical awareness and conversion: after 10...Nxd4 11.Nxd4 Bxd4, White correctly played 13.Qxd4, winning material, then developed an effective attack with 22.Bf4+, 23.Bxc4, and 27.Bxh8. Game B’s Black made several serious strategic and tactical errors, including the weakening 7...f6, 8...h5, and 9...g5, then allowed 20.Nb6 and 21.Nxa8 to win a rook. Most decisively, Black failed to prevent 22.Bd5 followed by 23.Bxd8, losing the queen. Although Game A’s White also benefited from Black’s mistakes, the overall play strongly favors A." }, "consistent": true } ], "rows": [ { "pair": "pair_01", "game_a": "game_16", "color_a": "black", "game_b": "game_19", "color_b": "white", "rating_a": 1569, "rating_b": 2668, "gap": 1099, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.82, "correct": 1, "brier": null, "old_estimate_a": 1100, "old_estimate_b": 1700, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s White showed substantially better practical play: after the unconventional 12.Kd2, White coordinated effectively with 20.Rhe1, exchanged favorably, created a dangerous h-pawn, and converted it with 33.Re7+, 34.Ra7, and 36.Rxh8. White also promptly exploited Black’s 34...Nxf5. In Game A, Black’s opening became strategically and materially dubious: 6...Bxf2+ sacrificed a bishop for a pawn, and 12...Bxf3 13.hxg4 further left Black’s position vulnerable. Black never generated adequate compensation and allowed 27.Re6. Although Game B’s opponent made larger errors, its target demonstrated much stronger conversion and coordination than Game A’s target.", "confidence_valid": false }, { "pair": "pair_02", "game_a": "game_01", "color_a": "black", "game_b": "game_24", "color_b": "white", "rating_a": 1159, "rating_b": 762, "gap": 397, "gap_band": "200-499", "low_band": "one<1000", "gold": "A", "choice": "B", "probability_a": 0.35, "correct": 0, "brier": 0.42250000000000004, "old_estimate_a": 1450, "old_estimate_b": 1500, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game B’s White showed more reliable tactical control: after 20 Nb6, White captured the rook with 21 Nxa8 and then exploited 22...Qd8 with 23 Bxd8, winning decisively without an evident major error. Game A’s Black eventually found useful tactics, especially 40...Rxb8 followed by 41...Rbxe8, but 15...Nxg5 allowed 16 Nf6+ and 17 Nxd7, losing the queen for a minor piece. Black’s eventual win therefore depended heavily on White’s later tactical mistakes, whereas Game B’s White more cleanly converted the opponent’s errors.", "confidence_valid": true }, { "pair": "pair_03", "game_a": "game_01", "color_a": "white", "game_b": "game_06", "color_b": "white", "rating_a": 1363, "rating_b": 1112, "gap": 251, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.72, "correct": 1, "brier": 0.07840000000000001, "old_estimate_a": 1500, "old_estimate_b": 1200, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A’s White showed a significant tactical resource with 16.Nf6+ followed by 17.Nxd7, winning Black’s queen, and later advanced a dangerous passed e-pawn. However, White then overlooked 30...Bxg3 and 31...Bxe1, losing a rook, and the eventual promotion tactic failed. Game B’s White made more fundamental oversights: 19.f5 allowed ...Nxa1 to win the rook, 25.Rf1 allowed ...Bxf1, and 27.Ke1 permitted ...Qxf6 before the forced mate 31...Qf2#. Thus A’s play contained stronger tactical ideas despite serious later inaccuracies.", "confidence_valid": true }, { "pair": "pair_04", "game_a": "game_22", "color_a": "black", "game_b": "game_08", "color_b": "black", "rating_a": 557, "rating_b": 1791, "gap": 1234, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.08, "correct": 1, "brier": 0.0064, "old_estimate_a": 950, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s Black played a coherent, sound game: rapid development with ...O-O, ...Re8, and ...c5, then accurately recovered the sacrificed bishop after 15.Bxc5 with 15...Ndxc5. The sequence 16...Qb6 and 18...axb6 led to a favorable endgame, and 20...Bg4, 22...Bxf3+, and 24...Nf3+ showed consistent tactical control. Game A’s Black chose a dubious tactical line with 2...e5 and 3...Nc6, allowed the king to be driven around after 7...h6 and 8...Qf6+, and then lost substantial material to 13.Qxb5, 17.Qxc5, and 18.Qxd5. Although White in Game A also played poorly, Black’s handling was much less reliable.", "confidence_valid": true }, { "pair": "pair_05", "game_a": "game_09", "color_a": "white", "game_b": "game_07", "color_b": "black", "rating_a": 2193, "rating_b": 1593, "gap": 600, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.65, "correct": 0, "brier": null, "old_estimate_a": 1450, "old_estimate_b": 1500, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game B’s target Black showed the clearer tactical understanding: after 21 Bf3, ...Bc2 attacked both the queen on d1 and rook on b3, and Black correctly collected them with 22...Bxd1 and 23...Bxb3. The opening play was also coherent. Game A’s target White played actively, including 28 e5 and 32 Rxe7, but allowed Black’s queenside pawn gains with 22...Qxa3 and 23...Qxc3, then was overwhelmed by the coordinated rooks after 37...Kg7. Thus B has the stronger evidence, though the short game and White’s major blunder make the comparison uncertain.", "confidence_valid": false }, { "pair": "pair_06", "game_a": "game_17", "color_a": "black", "game_b": "game_25", "color_b": "black", "rating_a": 1639, "rating_b": 838, "gap": 801, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.8, "correct": 1, "brier": 0.03999999999999998, "old_estimate_a": 1600, "old_estimate_b": 1400, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A’s Black played a coherent, principled opening: 6...d5, castling, active piece placement with ...Nf4, and pressure on the e-file with ...Rfe8 and ...Rxe4. The later 18...g5 and 24...c6 may be inaccurate, but the game shows substantially more positional discipline. Game B’s Black chose a highly dubious setup with 2...Bd7, 3...c6, and 4...b5, then allowed major tactical complications. Although 13...Nc2+ and 14...Nxa1 won material, Black’s subsequent 20...Rxd4+ and 21...Rxc4 led to tactical exchanges, and the king was eventually mated after 39.Rxd8#. Overall, A’s Black more likely had the higher established rapid strength.", "confidence_valid": true }, { "pair": "pair_07", "game_a": "game_14", "color_a": "black", "game_b": "game_08", "color_b": "white", "rating_a": 1188, "rating_b": 1310, "gap": 122, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.68, "correct": 0, "brier": 0.4624000000000001, "old_estimate_a": 1700, "old_estimate_b": 1400, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A’s Black showed more purposeful coordination and tactical awareness: after 9...Nd5, 10...Bxh4 exploited the cleared diagonal, and the rooks became very active with 21...Rd5, 29...Rd4, and 30...Rxd3. Game B’s White developed passively with 1.Nc3 and 2.d3, then allowed Black’s initiative to grow; 22.Rd4 permitted 22...Bxf3+, followed by the decisive-looking 24...Nf3+. Although Game B’s White made some useful tactical exchanges, Game A’s Black appeared more consistently effective, so A is more likely to have had the higher established rapid strength.", "confidence_valid": true }, { "pair": "pair_08", "game_a": "game_02", "color_a": "black", "game_b": "game_20", "color_b": "black", "rating_a": 2410, "rating_b": 1903, "gap": 507, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.8, "correct": 1, "brier": 0.03999999999999998, "old_estimate_a": 1750, "old_estimate_b": 1600, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A’s Black showed stronger tactical execution: 9...Nxe4 and 10...exd4 exploited White’s center, while 18...Bxh4! 19 Nxh4 Qxh4 removed the key defender. The follow-up 21...Re2 and 22...Rxb2 created a coherent rook attack, culminating in 25...Rh1#. Game B’s Black developed sensibly, but the kingside defense deteriorated: after White’s aggressive 24 f5, Black allowed 26 Rxh7 and 27 Qxh7, then was unable to stop the advancing g-pawn and was mated by 38 Nh6#. Although Game A’s opponent made larger errors, Black’s tactical recognition and conversion provide stronger evidence of established strength.", "confidence_valid": true }, { "pair": "pair_09", "game_a": "game_12", "color_a": "black", "game_b": "game_03", "color_b": "black", "rating_a": 1811, "rating_b": 1901, "gap": 90, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.35, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1200, "old_estimate_b": 1350, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s Black showed somewhat more coherent development and defensive play: castling on move 7, central counterplay with ...c5 and ...d5, and active rook moves (...Rc4 and ...Rc7). However, Black still missed serious tactical issues, notably 16...Kf7 followed by 18...Re8 19.Bxf6, and was eventually mated after 38.Rxg8+. Game A’s Black appears weaker: 10...Nxd4 11.Nxd4 Bxd4 12.Bb5+ c6 13.Qxd4 loses two minor pieces for one, and 26...Nxc4 allows 27.Bxh8 to win a rook. Thus B is more likely to have had the higher established rapid strength, though the evidence is limited.", "confidence_valid": true }, { "pair": "pair_10", "game_a": "game_23", "color_a": "black", "game_b": "game_09", "color_b": "black", "rating_a": 564, "rating_b": 2177, "gap": 1613, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.04, "correct": 1, "brier": 0.0016, "old_estimate_a": 1250, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s Black shows substantially stronger strategic and tactical play: the queenside expansion with 10...b5, active maneuvering via 15...Nc4 and 19...Nd4, and the sequence 20...Nxe2 21...Bxc3 won material cleanly. Black then coordinated the rooks effectively with ...Rb2, ...Rd8, and ...Rd2, finishing with a forcing attack. Game A’s Black played 12...Be7 and then sacrificed both bishops with 13...Bxh3 and 15...Bh4, apparently without sufficient compensation; White simply took them with 14 gxh3 and 16 Nxh4. The ensuing queen attack was energetic but failed to produce a result, indicating much weaker judgment than Game B’s Black.", "confidence_valid": true }, { "pair": "pair_11", "game_a": "game_21", "color_a": "black", "game_b": "game_10", "color_b": "black", "rating_a": 743, "rating_b": 2501, "gap": 1758, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.68, "correct": 1, "brier": null, "old_estimate_a": 1500, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s black showed cleaner, more coherent opening development and strong tactical awareness: after 22 Rxc8, Bxc8 preserved activity, and 24 Nxa7 Qc2 followed by 25...Ba6 and 26...Bxf1 won White’s rook through a clear tactical sequence. Game A’s black was resourceful and ultimately converted actively, including 7...Qb4+ and 21...Nxf1+, but the opening with 2...e5, 4...f6, and allowing 5 Qxd5 and 7 Qxb7 was strategically risky. The long game also featured considerable maneuvering and depended on substantial white inaccuracies. B therefore more likely had the higher established rapid strength.", "confidence_valid": false }, { "pair": "pair_12", "game_a": "game_05", "color_a": "black", "game_b": "game_06", "color_b": "black", "rating_a": 1867, "rating_b": 1382, "gap": 485, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.58, "correct": 1, "brier": 0.17640000000000003, "old_estimate_a": 1750, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A’s Black showed a more coherent attacking plan and tactical coordination: 16...Nxg2+ followed by 17...fxe4, 21...Nbd3, and 23...Bxh1 created sustained pressure and a forced mating net. Game B’s Black also exploited major oversights, especially 18...Ncxb3 and 19...Nxa1, then 25...Bxf1 and 27...Qxf6. However, the repeated bishop retreats in the opening (3...Bd7, 9...Bf5, 10...Bd7) and dependence on White’s loose play make B’s performance look slightly less polished. A therefore appears marginally more likely to have had the higher established rapid strength.", "confidence_valid": true }, { "pair": "pair_13", "game_a": "game_22", "color_a": "white", "game_b": "game_04", "color_b": "white", "rating_a": 717, "rating_b": 2508, "gap": 1791, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.35, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1200, "old_estimate_b": 1500, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s White shows a much more coherent opening and positional plan: 1.e4 c6, 4.Bd3, 7.O-O, and the queenside expansion with 14.b4 and 20.a5. However, the game ends with a severe tactical oversight: 24.Bb4? Ne2+ 25.Kh1 Qg1+ 26.Rxg1 Nf2#. Game A’s White plays very dubious early moves, especially 2.Qf3 and 3.g4, and exposes the king with 7.Kxf2 and 9.Ke3, but consistently exploits Black’s hanging pieces, notably 13.Qxb5 and 17.Qxc5. Overall B appears stronger despite the decisive blunder.", "confidence_valid": true }, { "pair": "pair_14", "game_a": "game_18", "color_a": "black", "game_b": "game_14", "color_b": "white", "rating_a": 2641, "rating_b": 1249, "gap": 1392, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.74, "correct": 1, "brier": 0.06760000000000001, "old_estimate_a": 1850, "old_estimate_b": 1500, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A’s black showed the more convincing tactical and strategic play. After the awkward-looking 10...Kf8, Black built a coherent kingside attack with ...h5, ...h4, ...Bh3, and the rook maneuvers ...Re2+ and ...Rh2. The final 32...Qh3+ and 33...Qxg3+ demonstrated accurate attacking calculation. Game B’s white played actively but less consistently: 8 Nh4 and 9 Qf3 allowed Black comfortable development, while 24 f4 and 27 Rg3 left the position vulnerable. After 28 Qe5 Qxe5 29 fxe5, White failed to prevent 30...Rxd3, losing the bishop and ending the game. Thus A’s target more likely had the higher established rapid strength.", "confidence_valid": true }, { "pair": "pair_15", "game_a": "game_05", "color_a": "white", "game_b": "game_20", "color_b": "white", "rating_a": 1120, "rating_b": 1932, "gap": 812, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.9, "correct": 1, "brier": null, "old_estimate_a": 1300, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s White showed a coherent attacking plan and strong tactical execution: the kingside expansion with 9 g4, 11 h4, and 12 h5 was followed by 18 Rxh6, 22 Rdh1, and the combination 26 Rxh7 Nxh7 27 Qxh7, winning decisive material. The finish 29 Qxh8+ Bxh8 30 Rxh8+ and 38 Nh6# was accurate. Game A’s White played an unfocused opening and missed major tactical threats: after 16...Nxg2+ and 17...fxe4, the attack collapsed, and 23 Nd4?? allowed ...Bxh1, losing a rook. White then permitted the forced mating sequence ending 29...Qxf3#.", "confidence_valid": false }, { "pair": "pair_16", "game_a": "game_11", "color_a": "black", "game_b": "game_02", "color_b": "white", "rating_a": 1273, "rating_b": 1575, "gap": 302, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.78, "correct": 0, "brier": 0.6084, "old_estimate_a": 1600, "old_estimate_b": 1200, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Target A (Black) showed more sustained, purposeful play: after developing, Black generated active counterplay with 23...e4 and 24...d5, then used the rooks actively on the c-file. The late 40...e3 and failure to handle 41 Rd8+ led to 42 Rxe8#, but much of the game was competently conducted. Target B (White) produced a plausible opening, yet the attack beginning with 18 h4 was poorly judged; after 24 Bc2? Rxh3, 25 Bg5?? allowed the immediate 25...Rh1#. B’s failure to address the rook invasion and mating threat suggests weaker established rapid strength.", "confidence_valid": true }, { "pair": "pair_17", "game_a": "game_13", "color_a": "white", "game_b": "game_16", "color_b": "white", "rating_a": 1126, "rating_b": 1752, "gap": 626, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.15, "correct": 1, "brier": 0.0225, "old_estimate_a": 1250, "old_estimate_b": 1500, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s White showed stronger tactical and positional control: after 6...Bxf2+ 7.Kxf2, White stabilized the exposed king, then used 12.h3 and 13.hxg4 to eliminate Black’s active knight. White coordinated the rooks with 17.Rae1 and 18.Rxe1, improved the position, and invaded with 27.Re6. In Game A, White’s 18.c3?? allowed 18...bxc5, losing the rook on c5 outright. Although White later found 30.Nxe6 and 31.Rxc6, the game ended with the basic mating oversight 35...Rc1#. These games favor B as the more established player.", "confidence_valid": true }, { "pair": "pair_18", "game_a": "game_24", "color_a": "black", "game_b": "game_12", "color_b": "white", "rating_a": 765, "rating_b": 1877, "gap": 1112, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.08, "correct": 1, "brier": 0.0064, "old_estimate_a": 1250, "old_estimate_b": 1450, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s White showed substantially better tactical awareness and coordination. White correctly exploited the sequence 10...Nxd4 11 Nxd4 Bxd4 13 Qxd4, emerging with material advantage, then developed a coherent attack with 17 b4, 20 bxc5, and 22 Bf4+. The decisive continuation 23 Bxc4, 24 Rfe1, 25 Be5, and 27 Bxh8 demonstrated effective calculation. Game A’s Black played an unsound-looking kingside expansion with 7...f6, 8...h5, and 9...g5, then missed the simple tactical threat after 20 Nb6: 21 Nxa8 won the rook, and 22...Qd8 allowed 23 Bxd8 to take the queen. Thus B is much more likely the stronger established player.", "confidence_valid": true }, { "pair": "pair_19", "game_a": "game_17", "color_a": "white", "game_b": "game_10", "color_b": "white", "rating_a": 1514, "rating_b": 2689, "gap": 1175, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.64, "correct": 0, "brier": 0.4096, "old_estimate_a": 1500, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A White played more forcefully and tactically, with 11.g4, 14.Bxf4, and the active 16.Re1 and 20.Bc2. The sequence 20...Rxe4 21.dxe4 kept the initiative, and Black resigned after 24...c6. Game B White showed reasonable positional development, but the tactical sequence 25...Ba6 26.Be5? Bxf1 allowed Black to take the rook, with only 27.Bxf1 recovering the bishop. That substantial one-move oversight makes Game B White look less established, despite the otherwise coherent opening and middlegame.", "confidence_valid": true }, { "pair": "pair_20", "game_a": "game_11", "color_a": "white", "game_b": "game_23", "color_b": "white", "rating_a": 1228, "rating_b": 557, "gap": 671, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.65, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1700, "old_estimate_b": 1500, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A White appears more likely to have the higher established strength. White handled the opening actively with 7.Ne4 and 8.Nxc5, then coordinated an effective kingside/central attack with 22.Bg5, 23.b3, 25.f5, and 30.f6. The rook activity and finish, 38.Rb5 through 42.Rxe8#, show sustained tactical awareness, despite some loose play such as 20.gxf3. Game B White defended well after Black’s unsound 13...Bxh3, notably with 14.gxh3, 17.f3, 18.Kh1, and 21.Qh2, but the game was short and Black’s sacrifice and subsequent attack were highly questionable. Thus B’s evidence is less informative about established strength.", "confidence_valid": true }, { "pair": "pair_21", "game_a": "game_21", "color_a": "white", "game_b": "game_15", "color_b": "white", "rating_a": 739, "rating_b": 2157, "gap": 1418, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.93, "correct": 1, "brier": null, "old_estimate_a": 1250, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B shows substantially more controlled play: White develops naturally, castles, coordinates rooks, and executes the concrete sequence 25 Qxc8+ Nxc8 26 Rxd5 exd5, simplifying into a sound minor-piece ending. White then handles the endgame actively with 35 Ne5+ and 39 Nc5+. In Game A, White’s early queen raid (2 Qd3, 5 Qxd5, 7 Qxb7) is speculative, and the resulting play is poorly coordinated: the rook is lost on 15...Kxd8, another rook is left vulnerable to 29...Nf2 and 30...Nxh1, and 39 Ne5+ allows 39...Rxe5. These concrete oversights strongly favor B.", "confidence_valid": false }, { "pair": "pair_22", "game_a": "game_04", "color_a": "black", "game_b": "game_13", "color_b": "black", "rating_a": 2718, "rating_b": 1551, "gap": 1167, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.43, "correct": 0, "brier": 0.3249000000000001, "old_estimate_a": 1750, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game B’s black handled a more complicated tactical battle and still converted after White’s active sequence 28.Ne8, 30.Nxe6, and 31.Rxc6. The queen excursion with 6...Qxb2–8...Qa5 was risky, but Black consistently found practical resources and finished with 35...Rc1#. Game A’s black played a coherent attack, especially 23...Qxc5, 24...Ne2+, and 25...Qg1+, but White’s 22.b5 and the resulting mating net were comparatively straightforward errors. Thus B appears slightly more likely to have had the higher established strength.", "confidence_valid": true }, { "pair": "pair_23", "game_a": "game_25", "color_a": "white", "game_b": "game_18", "color_b": "white", "rating_a": 738, "rating_b": 2426, "gap": 1688, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.3, "correct": 1, "brier": 0.09, "old_estimate_a": 1500, "old_estimate_b": 1700, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s White showed more coherent opening and positional play: 1.e4, 7.O-O, 9.Bb2, 10.Re1+, and 12.Nxc6 develop with clear purpose. However, White ultimately missed the danger of Black’s attack, especially 30...Re2+ followed by 32...Qh3+ and 33...Qxg3+. Game A’s White found a strong tactical attack and mating net, but the opening was extremely loose (2.e3, 3.Qf3, 4.g4), and White allowed 13...Nc2+ and 14...Nxa1, losing a rook. The eventual win depended heavily on Black’s inaccurate defense, including 24...Rxc3+. Overall, B’s White appears more established despite losing.", "confidence_valid": true }, { "pair": "pair_24", "game_a": "game_03", "color_a": "white", "game_b": "game_15", "color_b": "black", "rating_a": 1782, "rating_b": 2154, "gap": 372, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.55, "correct": 0, "brier": 0.30250000000000005, "old_estimate_a": 1750, "old_estimate_b": 1750, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "Game A’s white showed more convincing tactical initiative and conversion: 15.Ng6, 18.Be7, and 19.Bxf6 created lasting pressure, followed by 23.Nxd5 and 27.Qxc5. The attack was converted accurately with 35.Nxg8, 38.Rxg8+, and the mating finish 46.Re8#. Game B’s black played solidly and found a useful simplification with 24...Nxb7, 25...Nxc8, and 26...exd5, then handled the pawn ending competently. However, that game provides mostly evidence of resilience and endgame technique rather than the stronger player creating winning chances. Thus A is favored, though only slightly because Game A’s opponent made substantial defensive errors.", "confidence_valid": true }, { "pair": "pair_25", "game_a": "game_19", "color_a": "black", "game_b": "game_07", "color_b": "white", "rating_a": 2474, "rating_b": 1862, "gap": 612, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.82, "correct": 1, "brier": 0.03240000000000002, "old_estimate_a": 1550, "old_estimate_b": 1250, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A’s black player showed a coherent strategic plan: developing actively with ...Bf5, ...b5, ...Rac8, then using ...c5 and ...c4+ to create counterplay. Although Black later missed the rook-saving resource after 34.Ra7, the game remained competitive and featured meaningful endgame play. Game B’s white player made a much more decisive tactical oversight: after 21.Bf3 Bc2, 22.Bh5?? allowed ...Bxd1, losing the queen, and 23.Bxg6 was followed by ...Bxb3, losing another major piece. This indicates substantially weaker tactical awareness than Game A’s black player.", "confidence_valid": true } ] }, { "model": "openai/gpt-6-astra", "correct": 18, "n": 25, "accuracy": 72.0, "baseline_accuracy": 80.0, "invalid_confidence": 12, "consistent_reversals": 5, "reversals": [ { "pair": "pair_09_reverse", "original": "pair_09", "prediction": { "stronger": "B", "probability_a_stronger": 0.6, "explanation": "B’s Black showed more purposeful counterplay: 17...Nd5, 18...Nb6, and the rook invasion 23...Rd4–25...Rd2, followed by 26...Nxc4, created concrete threats. However, 11...Bxd4 overlooked 12.Bb5+ and 13.Qxd4, while 27...Nxe3 allowed the decisive back-rank attack. A’s Black also lost a piece in the sequence 16.Bxd6 through 19.Bxf6, then allowed 34.Ne7+ to win further material. Both games contain substantial tactical errors, but B’s coordinated middlegame activity gives B a slight edge." }, "consistent": true }, { "pair": "pair_15_reverse", "original": "pair_15", "prediction": { "stronger": "A", "probability_a_stronger": 0.98, "explanation": "Game A White demonstrated coherent attacking play: 14.Nce2–15.Ng3 improved the knight, 22.Rdh1 and 23.Qh2 coordinated the heavy pieces, and 26.Rxh7! initiated a forcing sequence culminating in 31.Rxd8. The finish with 35.g6, 36.Ng5 and 38.Nh6# showed accurate mating coordination. Game B White repeatedly lost opening tempi with the bishop and underestimated concrete threats: 16.Be4 allowed ...Nxg2+ followed by ...fxe4, while 22.Bc3 and 23.Nd4 permitted ...Bxe4 and ...Bxh1. Although B faced more effective opposition, the contrast in planning, tactical awareness, and execution strongly favors A." }, "consistent": true }, { "pair": "pair_13_reverse", "original": "pair_13", "prediction": { "stronger": "A", "probability_a_stronger": 0.96, "explanation": "Game A White shows coherent development and a purposeful queenside attack against opposite-side castling: 14.b4, 15.Nb3, 18.Nc5 and 20.a5. White ultimately misses Black’s forcing 24...Ne2+ and 25...Qg1+ mating combination, but the preceding play suggests substantially stronger foundations. Game B White’s 2.Qf3, 3.g4 and 6.h3 neglect development and expose the king, which wanders to e3 by move nine. B’s victory relies heavily on Black’s material giveaways, including the knight captured by 13.Qxb5 and the queen exchange blunder 22...Qxg1+. The result is much less persuasive than A’s overall organization." }, "consistent": true }, { "pair": "pair_12_reverse", "original": "pair_12", "prediction": { "stronger": "B", "probability_a_stronger": 0.72, "explanation": "Game B Black showed more coherent development and coordinated tactics: 15...Nf4, 16...Nxg2+ and 17...fxe4 won material, while 21...Nbd3 and 22...Bxe4 intensified the pressure. The finishing 27...Be4+ combined a discovered rook check with an attack on White’s queen. Game A Black also found effective tactics with 17...Nc5 and 18...Ncxb3, but shuffled the bishop through d7–f5–d7–g4–e6 and benefited from particularly accommodating opposition, including 25.Rf1 allowing ...Bxf1. Both performances were competent, but B provides stronger evidence of established tactical and positional fluency." }, "consistent": true }, { "pair": "pair_18_reverse", "original": "pair_18", "prediction": { "stronger": "A", "probability_a_stronger": 0.98, "explanation": "Game A’s White showed coherent development and tactical awareness: 12.Bb5+ enabled 13.Qxd4, while 22.Bf4+ answered the pawn fork with an intermediate check before 23.Bxc4. The finishing 28.Qg8+ creates a forced mating attack despite Black’s material captures. Game B’s Black instead spent many opening tempi on weakening pawn advances, notably 7...f6, 8...h5 and 9...g5, while leaving the king exposed. Black then allowed Nb6–Nxa8 and, most decisively, played 22...Qd8, hanging the queen directly to the bishop on g5. These differences strongly favor A’s established strength." }, "consistent": true } ], "rows": [ { "pair": "pair_01", "game_a": "game_16", "color_a": "black", "game_b": "game_19", "color_b": "white", "rating_a": 1569, "rating_b": 2668, "gap": 1099, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.95, "correct": 1, "brier": null, "old_estimate_a": 1450, "old_estimate_b": 1900, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B White showed coherent positional and endgame play: 8.Bxf6 followed by 9.Qxf6 damaged Black’s structure, 12.Kd2 activated the king, and 22.Nxh5 followed by the h-pawn advance created a dangerous passer. The rook invasion 33.Re7+ and 35.Ra8+ exploited Black’s exposed rook decisively. Game A Black developed actively, but 6...Bxf2+ was an unsupported sacrifice; 11...Ng4 and the ensuing exchanges left Black a bishop down without convincing compensation. The queen exchange with 25...Qxe2 further reduced any attacking chances. B’s sustained coordination is substantially stronger evidence of established playing strength.", "confidence_valid": false }, { "pair": "pair_02", "game_a": "game_01", "color_a": "black", "game_b": "game_24", "color_b": "white", "rating_a": 1159, "rating_b": 762, "gap": 397, "gap_band": "200-499", "low_band": "one<1000", "gold": "A", "choice": "B", "probability_a": 0.86, "correct": 0, "brier": null, "old_estimate_a": 1400, "old_estimate_b": 1350, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game B White showed better tactical awareness: 17.Nxf6+ captured with tempo, 20.Nb6 forked queen and rook, and 22.Bd5 attacked the queen before 23.Bxd8 punished its retreat. Black’s weak play limits how much this proves, but White exploited the opportunities coherently. Game A Black’s 15...Nxg5 overlooked 16.Nf6+ and 17.Nxd7, losing the queen for inadequate compensation. Black later found useful counterplay with 27...e4 and 30...Bxg3, but the eventual victory depended heavily on White’s errors, especially 40.Qb8+. B’s play is stronger evidence of established strength.", "confidence_valid": false }, { "pair": "pair_03", "game_a": "game_01", "color_a": "white", "game_b": "game_06", "color_b": "white", "rating_a": 1363, "rating_b": 1112, "gap": 251, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.81, "correct": 1, "brier": 0.03609999999999998, "old_estimate_a": 1350, "old_estimate_b": 1250, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A White shows better tactical awareness and coordination: 15.Nd5 followed by 16.Nf6+ and 17.Nxd7 exploits Black’s queen placement, while castling and centralizing the rooks provide a coherent foundation. A nevertheless makes serious errors, notably 31.e5 allowing ...Bxe1 and the losing liquidation beginning 40.Qb8+. Game B White spends several opening tempi on Bd3–b1 and pawn moves, then allows ...Nc5–b3–a1 to win a rook. Later, 25.Rf1 simply exposes another rook to ...Bxf1, and 27.Ke1 leaves the knight on f6 undefended. Despite both players’ blunders, A demonstrates the stronger overall baseline.", "confidence_valid": true }, { "pair": "pair_04", "game_a": "game_22", "color_a": "black", "game_b": "game_08", "color_b": "black", "rating_a": 557, "rating_b": 1791, "gap": 1234, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.99, "correct": 1, "brier": null, "old_estimate_a": 850, "old_estimate_b": 1700, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B Black shows coherent development and tactical awareness: 14...c5 attacks White’s queen and bishop, 15...Ndxc5 collects the bishop, and 19...f6 gains time against the knight. The finish 22...Bxf3+ followed by 23...Be4 and 24...Nf3+ creates a king-and-rooks fork. Game A Black attacks White’s exposed king but repeatedly sacrifices material without adequate compensation: 6...Nxf2, 11...Nb5+ allowing Qxb5, and ultimately 22...Qxg1+ surrendering the queen for a rook. Despite Game A White’s highly vulnerable opening, Black cannot exploit it and displays much weaker material judgment.", "confidence_valid": false }, { "pair": "pair_05", "game_a": "game_09", "color_a": "white", "game_b": "game_07", "color_b": "black", "rating_a": 2193, "rating_b": 1593, "gap": 600, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.59, "correct": 0, "brier": null, "old_estimate_a": 1900, "old_estimate_b": 1750, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game B Black developed coherently, activated the queen with 18...Qe6 and 20...Qg6, and exploited the queen–rook fork with 21...Bc2. After 22.Bh5, Black correctly played 22...Bxd1 and 23...Bxb3, winning a rook through the queen exchange. Game A White showed more ambitious play, including 28.e5 and 30.f5, but allowed damaging simplification after 20.Qd3 and ultimately missed the decisive mating attack with 39.R8xc5? Rh2+. B gets a modest edge for tactical control, although B’s opponent provided a much easier opportunity than A’s.", "confidence_valid": false }, { "pair": "pair_06", "game_a": "game_17", "color_a": "black", "game_b": "game_25", "color_b": "black", "rating_a": 1639, "rating_b": 838, "gap": 801, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.95, "correct": 1, "brier": 0.0025000000000000044, "old_estimate_a": 1800, "old_estimate_b": 1100, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black shows coherent development and positional play: 6...d5 challenges the center, 13...Nf4 establishes an active knight, and 19...Rfe8 followed by 21...Rd8 activates the rooks. The final position after 24...c6 does not explain the recorded loss. Game B Black exploits White’s weak play with 13...Nc2+ and the rook-winning fork, but 2...Bd7 and 7...Rd8 reflect awkward development while the kingside remains undeveloped. Later, 32...Nxf5 permits Ra8, and 33...Bd8 allows 34.Rxc8, losing the queen for a rook. A’s consistently purposeful play is substantially stronger evidence of established strength.", "confidence_valid": true }, { "pair": "pair_07", "game_a": "game_14", "color_a": "black", "game_b": "game_08", "color_b": "white", "rating_a": 1188, "rating_b": 1310, "gap": 122, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.86, "correct": 0, "brier": 0.7395999999999999, "old_estimate_a": 1550, "old_estimate_b": 1350, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A Black played a slow, somewhat aimless opening, but showed tactical awareness with 10...Bxh4, developed both rooks, and used 28...Qxe5 followed by 29...Rd4 to exploit White’s position, taking the loose bishop with 30...Rxd3. Game B White made more serious material errors: 15.Bxc5? Ndxc5 surrendered a bishop for a pawn, while 22.Rd4? allowed ...Bxf3+, losing another minor piece. Finally, 24.h4 permitted ...Nf3+, forking the king and both rooks. Despite A benefiting from weak opposition, B’s repeated failures to track threats make A the likelier stronger established player.", "confidence_valid": true }, { "pair": "pair_08", "game_a": "game_02", "color_a": "black", "game_b": "game_20", "color_b": "black", "rating_a": 2410, "rating_b": 1903, "gap": 507, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.66, "correct": 1, "brier": 0.11559999999999998, "old_estimate_a": 1650, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game A Black played more consistently: 9...Nxe4 exploited White’s center, 13...Na5–14...Nc4 improved the knight, and 18...Bxh4 won another pawn through a favorable exchange. After trading queens, 21...Re2 and 23...Rb3 showed effective rook activity, culminating in 25...Rh1#. Game B Black showed purposeful ideas with ...Nxh6, ...Nf8 and ...Ba6, against a more coherent attacking opponent. However, 25...Kf7 and 27...Rh8 allowed the forcing 28.fxg6+ through 31.Rxd8 sequence. A’s cleaner tactical judgment gives a moderate edge, tempered by White’s substantially weaker resistance in Game A.", "confidence_valid": true }, { "pair": "pair_09", "game_a": "game_12", "color_a": "black", "game_b": "game_03", "color_b": "black", "rating_a": 1811, "rating_b": 1901, "gap": 90, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.86, "correct": 0, "brier": 0.7395999999999999, "old_estimate_a": 1650, "old_estimate_b": 1650, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "Game A Black showed more purposeful coordination: 15...c5, 17...Nd5, 18...Nb6 and 21...c4 generated queenside counterplay, followed by active rook penetration with 23...Rd4 and 25...Rd2. A still made serious tactical errors, notably 10...Nxd4 and allowing 28.Qg8+. Game B Black displayed more persistent weaknesses: 14...Bd6 left the bishop vulnerable to Bxd6, 16...Kf7–17...Kg8 wasted time while exposed, and 33...Nc8 allowed 34.Ne7+ followed by Nxg8. Both games contain substantial mistakes, but A’s coherent counterplay suggests the higher established strength.", "confidence_valid": true }, { "pair": "pair_10", "game_a": "game_23", "color_a": "black", "game_b": "game_09", "color_b": "black", "rating_a": 564, "rating_b": 2177, "gap": 1613, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.98, "correct": 1, "brier": null, "old_estimate_a": 1300, "old_estimate_b": 2100, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B Black showed coherent Benoni counterplay with 10...b5 and 12...Rb8, purposeful knight maneuvering through ...Ng4–e5–c4, and 19...Nd4 followed by exchanges that exposed queenside pawns. The invasion 25...Rb2 and coordinated finish 38...Rdd2, 40...Rbg2+, 41...f3 demonstrate strong tactical and positional control. Game A Black developed reasonably but played the speculative 13...Bxh3 followed by 14...f5, leaving inadequate compensation after 15.Nxf5. Subsequent queen checks never established a convincing attack. B’s sustained coordination and conversion are substantially stronger evidence than A’s unsupported sacrifice.", "confidence_valid": false }, { "pair": "pair_11", "game_a": "game_21", "color_a": "black", "game_b": "game_10", "color_b": "black", "rating_a": 743, "rating_b": 2501, "gap": 1758, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.97, "correct": 1, "brier": null, "old_estimate_a": 1250, "old_estimate_b": 2000, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B Black shows coherent development and purposeful coordination: 14...Rfe8, 17...Bd6 and 18...Rf8 build activity, while 24...Qc2 and 25...Ba6 exploit White’s exposed rook to win the exchange with 26...Bxf1. This comes against substantially more organized opposition. Game A Black finds the useful 19...Nc2+–21...Nxf1 maneuver, but later simply loses a bishop with 25...Bxa4 26.bxa4. The eventual victory depends heavily on White’s further errors, notably 39.Ne5+ allowing ...Rxe5. B’s play is much stronger evidence of established playing strength.", "confidence_valid": false }, { "pair": "pair_12", "game_a": "game_05", "color_a": "black", "game_b": "game_06", "color_b": "black", "rating_a": 1867, "rating_b": 1382, "gap": 485, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.73, "correct": 1, "brier": 0.0729, "old_estimate_a": 1650, "old_estimate_b": 1600, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black showed more coherent development and sustained tactical coordination: 15...Nf4, 16...Nxg2+, and 17...fxe4 won material, while ...Nb4–d3 intensified the attack. The finish 27...Be4+ cleverly combined a discovered rook check with an attack on White’s queen. Game B Black also found effective tactics, notably 17...Nc5 and ...Ncxb3–a1, but the opening featured repeated bishop moves (...Bd7–f5–d7–g4–e6) and less economical development. Both opponents supplied substantial errors; B’s opponent additionally donated a rook with 25.Rf1. A’s more consistent coordination makes A the likelier stronger established player.", "confidence_valid": true }, { "pair": "pair_13", "game_a": "game_22", "color_a": "white", "game_b": "game_04", "color_b": "white", "rating_a": 717, "rating_b": 2508, "gap": 1791, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.03, "correct": 1, "brier": 0.0009, "old_estimate_a": 1000, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B White showed substantially stronger opening principles and planning: 4.Bd3, 5.c3, 6.Ne2 and 7.O-O developed coherently, followed by queenside expansion against Black’s long-castled king and the thematic 18.Nc5. White ultimately missed the mating combination 24...Ne2+ 25.Kh1 Qg1+ 26.Rxg1 Nf2#, but Black’s coordinated attack was considerably more demanding. Game A White’s 2.Qf3, 3.g4 and 4.g5 neglected development and exposed the king, which reached e3 by move nine. A’s eventual win involved collecting loose pieces and benefited from Black’s much weaker play, including 22...Qxg1+ allowing 23.Nxg1.", "confidence_valid": true }, { "pair": "pair_14", "game_a": "game_18", "color_a": "black", "game_b": "game_14", "color_b": "white", "rating_a": 2641, "rating_b": 1249, "gap": 1392, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.85, "correct": 1, "brier": 0.022500000000000006, "old_estimate_a": 1800, "old_estimate_b": 1500, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black showed coherent attacking play: 11...h5 and 14...h4 opened the h-file, while 29...Re8, 30...Re2+ and 31...Rh2 coordinated the heavy pieces against White’s king. After 32.Qxd5, Black exploited the exposed king with ...Qh3+ and ...Qxg3+. Game B White developed actively and found useful moves such as 17.Nd6, but 8.Nh4 spent time without a clear payoff, and 30.Rg6 simply left the bishop on d3 hanging to ...Rxd3. A’s play was not flawless and benefited from White’s mistakes, but its sustained coordination suggests greater established strength than B’s attack and elementary final oversight.", "confidence_valid": true }, { "pair": "pair_15", "game_a": "game_05", "color_a": "white", "game_b": "game_20", "color_b": "white", "rating_a": 1120, "rating_b": 1932, "gap": 812, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.98, "correct": 1, "brier": null, "old_estimate_a": 1300, "old_estimate_b": 2050, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B White shows coherent opening preparation and attacking coordination: queenside castling, the h-pawn advance, and 22.Rdh1 followed by 23.Qh2. The forcing sequence 26.Rxh7 through 31.Rxd8 converts the attack into material gain, while 35.g6 and 36.Ng5 create a precise mating finish. Game A White repeatedly spends tempi moving the same bishop, then allows 15...Nf4 and 16...Nxg2+, losing a bishop after ...fxe4. Later, 23.Nd4 permits ...Bxh1, and 27.Qxd3 allows the discovered check ...Be4+. Although A faced more effective opposition, B demonstrates substantially stronger planning and tactical awareness.", "confidence_valid": false }, { "pair": "pair_16", "game_a": "game_11", "color_a": "black", "game_b": "game_02", "color_b": "white", "rating_a": 1273, "rating_b": 1575, "gap": 302, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.8, "correct": 0, "brier": 0.6400000000000001, "old_estimate_a": 1550, "old_estimate_b": 1350, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A Black showed more coherent middlegame and endgame play: 23...e4 and 24...d5 established central control, 28...Nd3 created an active outpost, and 33...Rd8 followed by rook activity won material. However, 40...e3 overlooked the decisive 41.Rd8+. Game B White developed reasonably but lost direction: 18.h4 allowed ...Bxh4, while 20.Qh3 abandoned attacking prospects for a vulnerable, pawn-down ending. White then allowed ...Re2 and ...Rxb2, and 25.Bg5 missed mate in one. A's sustained coordination outweighs the final tactical failure.", "confidence_valid": true }, { "pair": "pair_17", "game_a": "game_13", "color_a": "white", "game_b": "game_16", "color_b": "white", "rating_a": 1126, "rating_b": 1752, "gap": 626, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.95, "correct": 1, "brier": null, "old_estimate_a": 1300, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B White showed coherent development and king safety: 7.Kxf2 accepted Black’s speculative sacrifice, followed by 8.Rf1, 9.Kg1 and 10.d4. White then coordinated with 17.Rae1 and simplified through 25.Qe2, retaining the extra piece. Game A White spent many opening tempi moving the same rook (Rb1–b3–a3–c3–c5), and 18.Qb2 allowed ...bxc5, conceding the exchange. Although 28.Ne8 and 29.Nc7 showed tactical awareness, 35.Nxb5 overlooked the immediate ...Rc1#. B’s opponent supplied an early advantage, but B’s subsequent handling was substantially more controlled.", "confidence_valid": false }, { "pair": "pair_18", "game_a": "game_24", "color_a": "black", "game_b": "game_12", "color_b": "white", "rating_a": 765, "rating_b": 1877, "gap": 1112, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.98, "correct": 1, "brier": null, "old_estimate_a": 1100, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black spends many opening moves on pawns (7...f6, 8...h5, 9...g5), leaving the king exposed and development neglected. Black then allows 20.Nb6, forking queen and rook, and outright hangs the queen with 22...Qd8 23.Bxd8. Game B White develops coherently and finds 12.Bb5+ followed by 13.Qxd4, exploiting Black’s capture sequence to win material. White also coordinates an attack against the castled king with 17.b4 and 22.Bf4+, finishing with the decisive 28.Qg8+. Despite some loose play, B shows substantially stronger tactical awareness and positional organization.", "confidence_valid": false }, { "pair": "pair_19", "game_a": "game_17", "color_a": "white", "game_b": "game_10", "color_b": "white", "rating_a": 1514, "rating_b": 2689, "gap": 1175, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.36, "correct": 1, "brier": 0.1296, "old_estimate_a": 1750, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "B’s White showed more coherent positional development: 9.d4, 11.Rc1, 13.Nd4 and 17.Rfd1 coordinated the pieces and contested the center. The later 24.Nxa7 and especially 26.Be5, allowing ...Bxf1, were serious tactical lapses. A’s White avoided comparable material losses, but 6.a3 and the sequence 11.g4, 12.Neg5, 13.Ne4 spent time and weakened the kingside without a clear payoff. A’s 23.Rd1 and 24.Rd5 were purposeful, though the unexplained ending after 24...c6 supplies no evidence of a winning position. B’s stronger opening organization narrowly outweighs the later mistakes.", "confidence_valid": true }, { "pair": "pair_20", "game_a": "game_11", "color_a": "white", "game_b": "game_23", "color_b": "white", "rating_a": 1228, "rating_b": 557, "gap": 671, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "B", "probability_a": 0.62, "correct": 0, "brier": null, "old_estimate_a": 1500, "old_estimate_b": 1550, "baseline_choice": "B", "baseline_credit": 0, "explanation": "B’s White showed more consistent coordination: 13.Nd4 centralized effectively, and after Black’s dubious 13...Bxh3, White defended with 19.Re1, 20.Qe2 and 21.Qh2, then sought queen exchanges with 22.Qe5 and 24.Qh2. A’s White found useful counterplay with 25.f5 and 30.f6, but 14.Bc4 followed by 15.Be2 wasted time, and the later bishop maneuver allowed Black to collect c4. A’s mating finish was alert, though it depended on Black ignoring the threat with 40...e3. B faced less convincing opposition, so the preference is modest.", "confidence_valid": false }, { "pair": "pair_21", "game_a": "game_21", "color_a": "white", "game_b": "game_15", "color_b": "white", "rating_a": 739, "rating_b": 2157, "gap": 1418, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.97, "correct": 1, "brier": null, "old_estimate_a": 1100, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "B’s White shows much more coherent development and tactical calculation: 24.Rd1 followed by 25.Qxc8+ and 26.Rxd5 liquidates accurately into a balanced minor-piece ending. Later, 49.e5 helps eliminate Black’s central pawns, and White reaches a drawn rook-pawn ending. A’s White opens with premature queen adventures (2.Qd3, 5.Qxd5, 7.Qxb7), leaves the kingside undeveloped, and allows 19...Nc2+ followed by ...Ne3+ and ...Nxf1. Most tellingly, 39.Ne5+ simply hangs a knight to 39...Rxe5. Both opponents make mistakes, but B demonstrates substantially stronger coordination and tactical awareness.", "confidence_valid": false }, { "pair": "pair_22", "game_a": "game_04", "color_a": "black", "game_b": "game_13", "color_b": "black", "rating_a": 2718, "rating_b": 1551, "gap": 1167, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.9, "correct": 1, "brier": 0.009999999999999995, "old_estimate_a": 2000, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black showed coherent development and attacking coordination: 16...Ne4, 17...f5 and 19...g5 established central control and kingside pressure. The finish 22...Nd4 followed by 24...Ne2+ and 25...Qg1+ demonstrated precise tactical vision. Game B Black exploited White’s loose rook with 17...b6 and found 35...Rc1#, but played less consistently. Repeated early queen moves delayed development, and 28...Re6 allowed 29.Nc7, forking the rooks and forcing material concessions. Both opponents supplied opportunities, but A’s sustained coordination is stronger evidence of established playing strength than B’s recovery against erratic opposition.", "confidence_valid": true }, { "pair": "pair_23", "game_a": "game_25", "color_a": "white", "game_b": "game_18", "color_b": "white", "rating_a": 738, "rating_b": 2426, "gap": 1688, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.96, "correct": 1, "brier": null, "old_estimate_a": 1050, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B White shows much stronger fundamentals: 7.O-O, 9.Bb2 and 10.Re1+ coordinate development, while 19.Rh1 sensibly exchanges Black’s attacking rook. The intermediate 27.Bxc5+ before 28.Qxc3 also shows tactical awareness. White later allows a decisive attack with 32.Qxd5 Qh3+. Game A White instead plays 3.Qf3 and repeated g- and h-pawn pushes while leaving almost everything undeveloped; 13.Nh3 allows 13...Nc2+ and the loss of the a1-rook. A’s later rook invasion and mating finish are creditable, but arise against highly cooperative opposition and do not outweigh the much weaker opening and king safety.", "confidence_valid": false }, { "pair": "pair_24", "game_a": "game_03", "color_a": "white", "game_b": "game_15", "color_b": "black", "rating_a": 1782, "rating_b": 2154, "gap": 372, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.59, "correct": 0, "brier": 0.34809999999999997, "old_estimate_a": 1850, "old_estimate_b": 1850, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "A’s White showed sustained tactical awareness: 16.Bxd6 exploited the potential Ne7+ queen fork, 23.Nxd5 increased central pressure, and 34.Ne7+ followed by 37.Rd8 decisively converted the attack. The early h4–Ng5–Nh3 maneuver was less convincing. B’s Black demonstrated good coordination with 22...c5 and the liquidation beginning 24...Nxb7, then activated the king effectively. However, 47...d3+ and the subsequent exchanges allowed White’s e5 resource and a drawn rook-pawn ending. A’s cleaner tactical execution gives a slight edge, tempered by the weaker defensive resistance in Game A.", "confidence_valid": true }, { "pair": "pair_25", "game_a": "game_19", "color_a": "black", "game_b": "game_07", "color_b": "white", "rating_a": 2474, "rating_b": 1862, "gap": 612, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.64, "correct": 1, "brier": 0.1296, "old_estimate_a": 1750, "old_estimate_b": 1550, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black showed more sustained endgame coordination: 14...Ke7 and 19...Kd6 activated the king, while 22...c5 and 23...c4+ generated counterplay. Black nevertheless mishandled White’s passed h-pawn, and 34...Nxf5 allowed 35.Ra8+ and the loss of the rook. Game B White developed reasonably and pursued a coherent queenside expansion, but 21.Bf3 allowed 21...Bc2, attacking the queen and skewering the rook on b3. The resource 22.Bh5 avoided losing a queen outright but still conceded an exchange after 23...Bxb3. A’s longer stretch of purposeful play gives Black a modest edge, despite the eventual collapse.", "confidence_valid": true } ] }, { "model": "z-ai/glm-5.3-flash", "correct": 18, "n": 25, "accuracy": 72.0, "baseline_accuracy": 86.0, "invalid_confidence": 4, "consistent_reversals": 4, "reversals": [ { "pair": "pair_09_reverse", "original": "pair_09", "prediction": { "stronger": "A", "probability_a_stronger": 0.65, "explanation": "Game B's Black commits a basic piece blunder: 10...Nxd4 11.Nxd4 Bxd4 allows 12.Bb5+ c6 13.Qxd4 winning the bishop outright, then loses more material (23...Rd4 25...Rd2 26...Nxc4??). Game A's Black plays a coherent Stonewall (8...d6, 13...d5), trades accurately (19...gxf6, 24...Bxd5), and its only real error, 22...e5 allowing 23.Nxd5, comes under pressure after White's earlier speculative 8.h4/9.Ng6. Overall A's Black showed more developed understanding and a less elementary mistake, suggesting the higher established rapid rating." }, "consistent": false }, { "pair": "pair_15_reverse", "original": "pair_15", "prediction": { "stronger": "A", "probability_a_stronger": 0.8, "explanation": "White A played a coherent Modern attack (h4-h5, g5, Rh6, clean h7 sacrifice 26.Rxh7 leading to mate 38.Nh6#), showing typical strong rapid/150-attack technique. White B misplayed from the opening: 12.Bd3 invited 11...f5 ideas, blundered with 16.Be4 Nxg2+ 17.Kf1 fxe4 losing material, then allowed 21...Nbd3 and 22...Bxe4 with further losses, being mated after 26...Bxf3 27.Qxd3 Be4+ 28.Nf3 Bxd3#. B's errors are elementary; A's play is smooth. So A is likely higher rated." }, "consistent": true }, { "pair": "pair_13_reverse", "original": "pair_13", "prediction": { "stronger": "A", "probability_a_stronger": 0.62, "explanation": "Target A (Game A White) played a coherent Panov-style game: reasonable development, c4-c5 space play (16.a4, 17...f5 answered by 18.Nc5), only failing late with 22.b5?? and 23.Qb2 allowing 24...Ne2+ and smothered mate. Their opponent was solid (sound Caro structure, ...g5 counterplay). Target B (Game B White) won only because the opponent blundered catastrophically (1...Nf6?! 2.Qf3, 6...Nxf2?? 7.Kxf2 h6? 8.gxh6, 11...Nb5+ dropping queen to 13.Qxb5). B's own moves (2.Qf3?!, 4.g5, greedy queen grabs) suggest a much weaker pool. A faced tougher opposition and played saner chess, so A is likely higher-rated." }, "consistent": true }, { "pair": "pair_12_reverse", "original": "pair_12", "prediction": { "stronger": "B", "probability_a_stronger": 0.65, "explanation": "Both Black targets won cleanly as Black. Game B's Black showed sharper tactical skill: 11...f5! and 15...Nf4 exploiting White's weak e4/pinned f2, then 16...Nxg2+ 17.Kf1 fxe4 winning a piece, followed by accurate regrouping (18...Nf4, 20...Nb4, 21...Nbd3) and a clean mating attack 25...Nh3+ 26...Bxf3 27...Be4+ 29...Qxf3#. Game A's Black also played well (13...Nd4, 17...Nc5, 18...Ncxb3 winning material), though slightly slower, with 19...Nxa1 allowing White counterplay before converting efficiently (27...Qxf6, 31...Qf2#). B's tactical sharpness and clean material win suggest slightly higher strength." }, "consistent": true }, { "pair": "pair_18_reverse", "original": "pair_18", "prediction": { "stronger": "A", "probability_a_stronger": 0.9, "explanation": "Game A's White played coherently: sound development (7.h3, 8.Nxf3), accurate central exchanges (10...Nxd4 11.Nxd4), good prophylaxis (16.Qa4, 17.b4), and clean tactics (23.Bxc4, 26.Re3, 27.Bxh8, 28.Qg8+). Errors were only minor. Game B's Black played very weakly: 9...g5 and 10...h4 wrecked the kingside, 15...Be6/16...Bg4 ignored the f6 square, and 17...Nxf6 was refuted by the fork 18.Nxf6+ winning material; everything collapsed (20.Nb6, 21.Nxa8). Such planless king-walking style suggests a far weaker player. Even discounting opponent strength, White in A is clearly stronger." }, "consistent": true } ], "rows": [ { "pair": "pair_01", "game_a": "game_16", "color_a": "black", "game_b": "game_19", "color_b": "white", "rating_a": 1569, "rating_b": 2668, "gap": 1099, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.3, "correct": 1, "brier": 0.09, "old_estimate_a": 1650, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B's White plays a clean, methodical game: timely queen trade (9.Qxf6), purposeful h4-h5 expansion, good rook placement (Rac1, Re1), and accurate endgame technique (Kd3, Re1-Ra7 winning the exchange, h7). Game A's Black plays erratically: the speculative 6...Bxf2+ gives up material without clear compensation, and 27...Re6 blunders the pinned rook with Nc7 while White threatens the e-file, losing immediately. Black's compensation attempt (17...Rxe1, 18...Nd5) shows some ideas but the overall play is less coherent and collapses faster. B's White shows stronger positional understanding and endgame technique, suggesting the higher rapid rating.", "confidence_valid": true }, { "pair": "pair_02", "game_a": "game_01", "color_a": "black", "game_b": "game_24", "color_b": "white", "rating_a": 1159, "rating_b": 762, "gap": 397, "gap_band": "200-499", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.65, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1750, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A's Black plays a coherent strategic game: 9...g5-11...g4 secures a strong pawn shield, 15...Nxg5/16...Nf6+ wins material via a calculated knight fork line (15.Nd5?? blunder punished), then Black converts cleanly in the endgame (34-41 precise checks, 38...h3, 39...e7) to promote and force resignation. Game B's White wins mainly because Black self-destructs: 17...Nxf6 allowing Nxf6+, 20...Qc6? losing the rook to 21.Nxa8. White's play (12...Nc4-to-13...Nd4 setup is awkward) shows reasonable tactics but less sustained quality. A's demonstrated technique and punishment of a smaller error suggests higher strength.", "confidence_valid": true }, { "pair": "pair_03", "game_a": "game_01", "color_a": "white", "game_b": "game_06", "color_b": "white", "rating_a": 1363, "rating_b": 1112, "gap": 251, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.9, "correct": 1, "brier": 0.009999999999999995, "old_estimate_a": 1650, "old_estimate_b": 1350, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A's White plays a coherent strategic game: regrouping Nf6+/Nxd7 tricks (15-17), trading into a defensible endgame, and only slipping late (30.dxe4? Bxg3 dropping a rook after 31...Bxe1). Game B's White self-destructs: 13.Ng5? allows ...Nd4 pressuring f3, then 16.f4? Bxg5 loses material, 18.Qe3? runs into 18...Ncxb3, and 19.f5? Nxa1 simply drops a rook; Black converts with 24...Qxf6 and a clean 31...Qf2#. White's handling in B (open king, aimless wing play 8.h4) is far weaker than A's, who fought a sound attacking plan.", "confidence_valid": true }, { "pair": "pair_04", "game_a": "game_22", "color_a": "black", "game_b": "game_08", "color_b": "black", "rating_a": 557, "rating_b": 1791, "gap": 1234, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.3, "correct": 1, "brier": 0.09, "old_estimate_a": 1350, "old_estimate_b": 1900, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A's Black found the strong 5...Ng4 and 6...Nxf2 sacrifice, but then erred badly: 12...Qf6 allowed 13.Qxb5, and after winning material Black drifted (19...Kf6, 20...Qf4, 21...Qg5) and lost the queen with 21...Qg5 22.Rg1 Qxg1+??, resigning from a winning position. Game B's Black played coherently throughout: logical development, 9...e4 seizing the center, accurate tactics 16...Qb6 forcing exchanges, and a clean conversion ending with 22...Bxf3+ 23...Be4 and 24...Nf3+ winning the knight. Consistent sound play and a won-game finish suggest B is the stronger player.", "confidence_valid": true }, { "pair": "pair_05", "game_a": "game_09", "color_a": "white", "game_b": "game_07", "color_b": "black", "rating_a": 2193, "rating_b": 1593, "gap": 600, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.62, "correct": 0, "brier": null, "old_estimate_a": 1600, "old_estimate_b": 1550, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game B's Black plays a clean, forcing tactic: 20...Qg6 21...Bc2! 22...Bxd1 23...Bxb3 nets a full rook with back-rank themes, executed instantly in rapid time. Game A's White shows decent Benoni handling (12.Bd2, 19.f4) but repeatedly errs: 15.Kh2 lets the knight fork, 27.Rg1 paralyzes the rook, and 35.Rc1/36.Bxd5 drops the bishop, with passive defense throughout. Black B's tactical accuracy under time pressure suggests the higher established strength, while White A's errors look more typical of a lower-rated rapid player.", "confidence_valid": false }, { "pair": "pair_06", "game_a": "game_17", "color_a": "black", "game_b": "game_25", "color_b": "black", "rating_a": 1639, "rating_b": 838, "gap": 801, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.75, "correct": 1, "brier": 0.0625, "old_estimate_a": 1650, "old_estimate_b": 1150, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A's Black plays a coherent Italian: 8...Bg4 active development, 13...Nf4 hitting the bishop, 16...Bxe4/17...Qf6 clean trades, then 22...Ne5 and 23...Rd6 with a solid, equal game; White resigned at move 24 with 24.Rd5? meeting 24...c6. Game B's Black plays erratically: 2...Bd7 and 4...b5/5...Na6 are aimless, 13...Nc2+ 14...Nxa1 trades a knight for a rook-corner trap but abandons queenside, then loses material (20...Rxd4+ 21...Rxc4 losing the rook for bishop) and gets mated. Black B's planless aggressive pawn pushes (g4?!) by White aside, Black A shows far better positional understanding and technique, suggesting the higher rating.", "confidence_valid": true }, { "pair": "pair_07", "game_a": "game_14", "color_a": "black", "game_b": "game_08", "color_b": "white", "rating_a": 1188, "rating_b": 1310, "gap": 122, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.72, "correct": 0, "brier": 0.5184, "old_estimate_a": 1600, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A's Black played coherently: good trades (10...Bxh4, 12...Qxg5), sensible regrouping with ...Na6, ...Bd7, and careful maneuvering (17.Ne4... wait, Black's ...f5 and ...Qf6 redeployment), then converted a small edge cleanly, with accurate rook play 29...Rxd4 30...Rxd3 winning a pawn. Game B's White blundered chronically: 14.Bxc5 allowed ...Ndxc5 regaining, and after 22.Rd4 Bxf3+ 23.Kg1 Be4 24.h4?? White lost to ...Nf3+ with the rook on d4 hanging and king unsafe—decisive carelessness. Black also erred little. A's target showed better technique and fewer mistakes, so A was likely higher-rated.", "confidence_valid": true }, { "pair": "pair_08", "game_a": "game_02", "color_a": "black", "game_b": "game_20", "color_b": "black", "rating_a": 2410, "rating_b": 1903, "gap": 507, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.75, "correct": 1, "brier": 0.0625, "old_estimate_a": 1700, "old_estimate_b": 1500, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A's Black played a principled Ruy Lopez, grabbing a pawn with 9...Nxe4, regrouping (13...Na5, 14...Nc4), and executing a clean tactical sequence: 18...Bxh4! 19.Nxh4 Qxh4, then converting accurately to mate (21...Re2, 24...Rxh3, 25...Rh1#). Game B's Black chose a passive kingside setup (...Re8 6..., 7...Bh8) and drifted: 23...Bg7? allowed 24.Rxh7/26.Rxh7, and after 26...Nxh7 27.Qxh7 the defense collapsed, leading to a forced mating net (38.Nh6#). A's Black showed coherent strategy and precise calculation; B's Black was outplayed systematically.", "confidence_valid": true }, { "pair": "pair_09", "game_a": "game_12", "color_a": "black", "game_b": "game_03", "color_b": "black", "rating_a": 1811, "rating_b": 1901, "gap": 90, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.7, "correct": 0, "brier": 0.48999999999999994, "old_estimate_a": 1500, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Black in A plays a coherent game: good tactics (8...exd4, 10...Nxd4 trading into a clean position), active queenside play with 15...O-O-O and 17...Nd5, and only succumbs to White's late mating attack (28.Qg8+) after material was even. Black in B handles the Stonewall structure passively: 9...Qc8 and 10...h6 misplace pieces, 15...Rd8/16...Kf7 mishandle the Bxd6 trade, and repeated tactical lapses lose material (22...e5 allowing 23.Nxd5, then 23...Nd7?? hanging pieces to 27.Qxc5). Black B's errors look much more amateur than Black A's solid play.", "confidence_valid": true }, { "pair": "pair_10", "game_a": "game_23", "color_a": "black", "game_b": "game_09", "color_b": "black", "rating_a": 564, "rating_b": 2177, "gap": 1613, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.25, "correct": 1, "brier": 0.0625, "old_estimate_a": 1450, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black loses material cheaply: 13...Bxh3? 14.gxh3 then 14...f5? 15.Nxf5 Bxf5? loses a piece for little, and the attack fizzles while White simply consolidates. Game B Black plays a coherent Benko/Benoni-style game: timely ...b5 and knight redeployment, regains material (21...Bxc3, 22...Qxa3), grabs queenside pawns (23...Qxc3, 25...Rxa7), keeps coordination with ...Rb2 infiltration, and converts to a clean win despite complex defense. The difference in positional understanding and conversion skill favors B clearly.", "confidence_valid": true }, { "pair": "pair_11", "game_a": "game_21", "color_a": "black", "game_b": "game_10", "color_b": "black", "rating_a": 743, "rating_b": 2501, "gap": 1758, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "A", "probability_a": 0.75, "correct": 0, "brier": 0.5625, "old_estimate_a": 1450, "old_estimate_b": 1700, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A's Black won material early with 5...fxe5/6...Bf5, exploited tactics (17...Nb4, 19...Nc2+ winning the knight via fork ideas), then converted a long R+N vs N endgame with solid technique: king activity (44...Kb7-a6), the...", "confidence_valid": true }, { "pair": "pair_12", "game_a": "game_05", "color_a": "black", "game_b": "game_06", "color_b": "black", "rating_a": 1867, "rating_b": 1382, "gap": 485, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.68, "correct": 1, "brier": 0.10239999999999996, "old_estimate_a": 1750, "old_estimate_b": 1650, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Black in A played an active, coherent game: queenside space (4...a6/4...b5), the tactical shots 15...Nf4 and 16...Nxg2+!, then 21...Nbd3 forking, winning material cleanly and concluding with a forced mating net (25...Nh3+, 26...Bxf3, 27...Be4+, 29...Qxf3#). Black in B also won but largely by punishing obvious White errors (13.Ng5?, 18...Ncxb3/19...Nxa1 exploiting loose moves), and needed 30 moves against passive play; A's attack was more precise and self-created, suggesting the higher rating.", "confidence_valid": true }, { "pair": "pair_13", "game_a": "game_22", "color_a": "white", "game_b": "game_04", "color_b": "white", "rating_a": 717, "rating_b": 2508, "gap": 1791, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.6, "correct": 1, "brier": null, "old_estimate_a": 1450, "old_estimate_b": 1450, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "Game B's White showed coherent positional chess: normal Caro-Kann structure, safe O-O, planned a4/b5 expansion, and sensible trades (11.Qxe2, 18.Nc5). The loss came mainly from one late blunder 24.Bb4?? overlooking the smothered mate after 24...Ne2+ 25.Kh1 Qg1+ 26.Rxg1 Nf2#. Game A's White played chaotic opening moves (2.Qf3, 4.g4) and merely profited from Black's absurd blunders (6...Nxf2?, 7...h6? allowing gxh6/Qf6+), with the win largely gifted. B's target demonstrated stronger fundamental play, so likely higher rating despite losing.", "confidence_valid": false }, { "pair": "pair_14", "game_a": "game_18", "color_a": "black", "game_b": "game_14", "color_b": "white", "rating_a": 2641, "rating_b": 1249, "gap": 1392, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.75, "correct": 1, "brier": 0.0625, "old_estimate_a": 1750, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A's Black played forcefully and accurately: 11...h5!, 14...h4! gaining attack, clean rook doubling (16...Bh3, 19...Rxh1), and a precise tactical finish 26...Bxc3, 30...Re2+ 31...Rh2, 32...Qxd5! winning material decisively. Game B's White erred early with 8.Nh4?! (allowing ...g5 ideas) and 9...Nd5 10...Bxh4 regained a piece, later drifting passively (17.Nd6/19.Ne4 shuffling) before 29...Rd4 and 30...Rxd3 collapsed tactically. Black's sharp, coherent attacking play and conversion in A exceeds White's solid but blunder-prone effort in B.", "confidence_valid": true }, { "pair": "pair_15", "game_a": "game_05", "color_a": "white", "game_b": "game_20", "color_b": "white", "rating_a": 1120, "rating_b": 1932, "gap": 812, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.12, "correct": 1, "brier": 0.0144, "old_estimate_a": 1300, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A's White plays weakly: 15.Be4 allows 15...Nxg2+ winning a piece (16.Kf1 fxe4 traps the bishop and 18.Nxe4 Nf4 keeps material), then 23.Nd4?? loses the exchange to 23...Bxh1. Black methodically converts, and White resigns mated — White looks ~1400-1600 level. Game B's White handles a sharp Saemisch KID: solid setup, timely 17.f4 and 13.hxg6, accurate attack culminating in 26.Rxh7! 27.Qxh7 Rh8 28.fxg6+ and clean queen sacrifice 29.Qxh8+, finishing with a model mate 38.Nh6#. B is clearly the stronger player.", "confidence_valid": true }, { "pair": "pair_16", "game_a": "game_11", "color_a": "black", "game_b": "game_02", "color_b": "white", "rating_a": 1273, "rating_b": 1575, "gap": 302, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.65, "correct": 0, "brier": 0.42250000000000004, "old_estimate_a": 1650, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A's Black plays coherently in a tricky line: 8.Nxc5 Qxc5 keeps material even, and Black trades actively (16...Bg4, 19...Bxf3) and creates counterplay with 23...e4 and 24...d5. The loss comes late (40...e3 41.Rd8+) in a bad endgame, suggesting weaker endgame technique but decent opening/middlegame understanding. Game B's White blunders immediately: 18.h4?? allows 18...Bxh4! 19.Nxh4 Qxh4 winning a piece, then drifts to 21...Re2, 22...Rxb2 and gets mated (25...Rh1#). Conceding a piece to a primitive tactic on move 18 is a clear sign of lower strength.", "confidence_valid": true }, { "pair": "pair_17", "game_a": "game_13", "color_a": "white", "game_b": "game_16", "color_b": "white", "rating_a": 1126, "rating_b": 1752, "gap": 626, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.75, "correct": 1, "brier": null, "old_estimate_a": 1350, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "White in Game A shows poor technique: grabbing ...Qxb2 (6...), then after luck (Black's 18...bxc5?? dropping a rook) still collapses with 23.Nd6??/24.Ba4 weakening, allowing Black's smooth conversion and mate (35...Rc1#). White in Game B handles a tricky Scandinavian exchange calmly: 7.Kxf2 safely, solid development, 12.hxg4 winning a pawn, careful king safety (Kg1, Rf1), and converts cleanly with 25.Qe2 Qxe2 26.Rxe2 and 27.Re6 exploiting the pin. Black in B was slightly more solid than Black in A, making B's win more convincing. B's target likely had the higher rating.", "confidence_valid": false }, { "pair": "pair_18", "game_a": "game_24", "color_a": "black", "game_b": "game_12", "color_b": "white", "rating_a": 765, "rating_b": 1877, "gap": 1112, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.12, "correct": 1, "brier": 0.0144, "old_estimate_a": 1050, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B's White plays a clean strategic-tactical game: punishing ...Nd5? (17...Nb6 retreat, 26.Re3-xe3 fork trick winning material), building pressure with a4-b4, and finishing with 28.Qg8+ mate. Black's errors were punished efficiently. Game A's Black played weakly throughout: the bizarre 4...b5/6...b4, 9...g5 and 10...h4 pawn storms against his own king, allowing 17.Nxf6+ and losing material (18...gxh4, then the queen-trap 22...Qd8 losing the queen to 23.Bxd8). Black A's play was unsound beginner-level aggression; White B showed solid technique. Player B is clearly the stronger.", "confidence_valid": true }, { "pair": "pair_19", "game_a": "game_17", "color_a": "white", "game_b": "game_10", "color_b": "white", "rating_a": 1514, "rating_b": 2689, "gap": 1175, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.4, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 1800, "old_estimate_b": 1600, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game B's White plays a coherent English/Reversed-Sicilian structure, trades well, and only loses after 24.Nxa7 grabs a poisoned pawn into Black's organized counterplay (24...Qc2, 25...Ba6, 26...Bxf1), suggesting Black was genuinely strong rather than White weak. Game A's White wins mainly because Black self-destructs: 17...Qf6 allows Rxe4 pressure, 18...g5 weakens the king, and 24...c6 simply drops the pawn/position. A's own play (12.Neg5 then retreating 13.Ne4, passive 21.Qe2) is unremarkable. B's positional handling looks stronger overall.", "confidence_valid": true }, { "pair": "pair_20", "game_a": "game_11", "color_a": "white", "game_b": "game_23", "color_b": "white", "rating_a": 1228, "rating_b": 557, "gap": 671, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "B", "probability_a": 0.45, "correct": 0, "brier": 0.30250000000000005, "old_estimate_a": 1700, "old_estimate_b": 1550, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Both whites won against weak opposition, but Black in B was catastrophically weak (13...Bxh3?? and 15...Bh4?? giving away two pieces). White B nonetheless played crisply, punishing accurately (14.gxh3, 15.Nxf5, 16.Nxh4) and safely neutralizing checks. White A also played reasonably but had slack moments (24.f4 instead of fxe4, 27.g4 weakening), and his opponent erred mainly in mutual complicity (23...e4?, 38...Rc1?). B's clean, accurate exploitation against even weaker resistance slightly suggests the stronger established player, though the evidence is thin.", "confidence_valid": true }, { "pair": "pair_21", "game_a": "game_21", "color_a": "white", "game_b": "game_15", "color_b": "white", "rating_a": 739, "rating_b": 2157, "gap": 1418, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.15, "correct": 1, "brier": 0.0225, "old_estimate_a": 1350, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B's White plays a coherent Semi-Slav setup (Nc3, Bxc4, regulated queenside maneuvers like 15.Nxa5 and 19.Qc2), trades into a safe rook endgame and holds a draw with accurate king play (46.Kd2, 57.Kh2). Game A's White opens with the weak 2.Qd3, loses material via 5.Qxd5 and the forced 7.Qxb7, and needs Black's own errors to reach an endgame, which White then misplayes (44.a4, 45.Kd5 letting the king through) and loses. B's technique and sound opening play indicate a clearly higher rating.", "confidence_valid": true }, { "pair": "pair_22", "game_a": "game_04", "color_a": "black", "game_b": "game_13", "color_b": "black", "rating_a": 2718, "rating_b": 1551, "gap": 1167, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.35, "correct": 0, "brier": 0.42250000000000004, "old_estimate_a": 1600, "old_estimate_b": 1400, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Both Black targets beat weaker-seeming White opposition, but B's play looks more sustained. In Game B, Black punished White's queen retreats (6...Qxb2, 8...Qa5) and found the in-between tactic 18...bxc5! netting a piece, then converted cleanly: 20...O-O, trades with 30...dxe6, and finished with 34...Bb5 and 35...Rc1#, showing endgame technique. In Game A, Black played well (22...Nd4 opening lines, 24...Ne2+ leading to the smothered mate 26...Nf2#), but the mate relied on White's cooperative 25.Kh1 Qg1+ allowing Rxg1, and earlier Black moves (17...f5, 19...gxf4) were more speculative. B's longer, controlled domination suggests a slightly higher rating.", "confidence_valid": true }, { "pair": "pair_23", "game_a": "game_25", "color_a": "white", "game_b": "game_18", "color_b": "white", "rating_a": 738, "rating_b": 2426, "gap": 1688, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.4, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 1350, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 1, "explanation": "White in Game B plays a principled Sicilian/Taimanov setup, trades sensibly (11.Nxd4, 12.Nxc6), maneuvers well (21.cxd5, 22.Be3), and only loses after Black finds strong rook infiltrations (29...Re8, 30...Re2+); White's final blunder 32.Qxd5 walking into ...Qh3+ costs the game. White in Game A benefits from a catastrophically weak opponent (7...Rd8??, 13...Nc2+ giving a knight for nothing) and plays erratic moves himself (2...Bd7 response aside, 4.g4?!, 13.Nh3 with king marching). A's win is more about opponent collapse than skill; B shows more coherent technique despite losing.", "confidence_valid": true }, { "pair": "pair_24", "game_a": "game_03", "color_a": "white", "game_b": "game_15", "color_b": "black", "rating_a": 1782, "rating_b": 2154, "gap": 372, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.6, "correct": 1, "brier": null, "old_estimate_a": 1650, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B's Black navigated a tricky Semi-Slav line (15...Qd5, 16...Rxa4 counting out the knight fork ideas), simplified correctly (24...Nxb7, 25...Nxc8), and reached a dead-drawn pawn endgame — solid, theory-aware play with clean technique (42...d4, 47...d3+ showing accuracy). Game A's White won but with less convincing chess: 8.h4 and 9.Ng5 are crude attacking setups, though 21.Bh5 and the mating net (34.Ne7+, 45.Qxg6) show tactical alertness. Black's opposition in A was notably passive (10...h6, 12...bxc5 opening lines against himself), inflating A's result. B's clean handling of concrete lines suggests the higher established strength.", "confidence_valid": false }, { "pair": "pair_25", "game_a": "game_19", "color_a": "black", "game_b": "game_07", "color_b": "white", "rating_a": 2474, "rating_b": 1862, "gap": 612, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.72, "correct": 1, "brier": 0.07840000000000001, "old_estimate_a": 1650, "old_estimate_b": 1500, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A's Black handled the semi-Slav structure reasonably, exploiting White's 9.Qxf6?? blunder, showed decent endgame technique (22...c5, 23...c4+ creating counterplay), and only erred late (34...Nxf5 allowing 35.Ra8+ and 37.h7). Game B's White played weakly: 21.Bf3 allowed 21...Bc2, then 22.Bh5 Bxd1 23.Bxg6 Bxb3 simply dropped a full bishop, prompting resignation despite no catastrophic earlier mistake. Black's clean tactical conversion (Bc2/Bxd1/Bxb3) and White's piece-loss blunder suggest B's target is weaker than A's, who competed for ~37 moves against competent opposition.", "confidence_valid": true } ] } ], "pairs": [ { "id": "pair_07", "predictions": [ { "model": "anthropic/claude-opus-5.5", "pair": "pair_07", "game_a": "game_14", "color_a": "black", "game_b": "game_08", "color_b": "white", "rating_a": 1188, "rating_b": 1310, "gap": 122, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.6, "correct": 0, "brier": 0.36, "old_estimate_a": 1350, "old_estimate_b": 1300, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A Black played an odd opening (1...d6 2...e6 3...h6) but punished White's 10.Bxd5?? with 10...Bxh4, winning a piece. Afterward Black consolidated reasonably, and 24...Re8 and 26...Rad8 led to 29...Rd4 and 30...Rxd3, picking up the bishop. Game B White played a passive setup and then blundered: 15.Bxc5?? simply dropped a piece. That was followed by 21.Kg2?, allowing 22...Bxf3+, and then 23.Kg1, allowing the fork 24...Nf3+. That is repeated tactical blindness. Both opponents were weak, so the evidence is limited. Still, A's target showed better tactical awareness, while B's target made several unforced blunders.", "confidence_valid": true }, { "model": "anthropic/claude-sonnet-5.5", "pair": "pair_07", "game_a": "game_14", "color_a": "black", "game_b": "game_08", "color_b": "white", "rating_a": 1188, "rating_b": 1310, "gap": 122, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.64, "correct": 0, "brier": 0.4096, "old_estimate_a": 1550, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A, Black: answered the early e5 with ...c6 and ...Be7 and developed sensibly. Played ...Nd5 and ...Bxh4 to win the knight pair, then ...Qf5 and ...Qg6 to improve the queen. ...f5 and ...Rd5 were active, and ...Rd4 followed by ...Rxd3 won a piece after White's Rg6 blunder. This was a solid, purposeful game against a mediocre opponent. Game B, White: played the opening passively and allowed ...e4 with tempo. 15.Bxc5 and 16.Ne5 were questionable, 18.Qxb6 axb6 left a worse structure, and 21.Kg2 walked into ...Ng5. 22.Rd4 and 24.h4 lost to ...Nf3+ smothered-style tactics. White's play had more inaccuracies and a tactical oversight, while Black in Game A was steadier. The evidence is modest, and both opponents seem to be around the same level.", "confidence_valid": true }, { "model": "google/gemini-3.8-flash", "pair": "pair_07", "game_a": "game_14", "color_a": "black", "game_b": "game_08", "color_b": "white", "rating_a": 1188, "rating_b": 1310, "gap": 122, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.28, "correct": 1, "brier": 0.07840000000000001, "old_estimate_a": 1450, "old_estimate_b": 1250, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A features severe tactical oversights by both players characteristic of lower ratings. In Game A, Black (Target A) missed winning a free knight with 15...Qxe4 (playing 15...Rd8 instead), and on move 17 played 17...Qf6, completely hanging the undefended rook on e8 (which White inexplicably failed to capture). In Game B, White (Target B) played a somewhat passive opening and lost a piece to an alert fork (14...c5), but Black in Game B demonstrated strong tactical pressure (20...Bg4 and 21...Ng5). Target B's play was flawed, but avoiding basic one-move blunders suggests a higher rating than Target A.", "confidence_valid": true }, { "model": "google/gemini-3.1-pro-preview", "pair": "pair_07", "game_a": "game_14", "color_a": "black", "game_b": "game_08", "color_b": "white", "rating_a": 1188, "rating_b": 1310, "gap": 122, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.65, "correct": 0, "brier": 0.42250000000000004, "old_estimate_a": 1150, "old_estimate_b": 1350, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Target A (Black) played a somewhat passive but solid game, avoiding major tactical errors before eventually capitalizing on White's blatant blunder (30...Rxd3). Target B (White) demonstrated a very passive and disjointed opening setup (1.Nc3, 2.d3, 3.e3, 5.g3), fell for a simple pawn fork (14...c5), and later allowed a devastating knight fork (24.h4 Nf3+). Target A's ability to maintain a solid position and avoid basic tactical oversights suggests a higher rating than Target B's error-prone and excessively passive play.", "confidence_valid": true }, { "model": "google/gemini-3.5-flash", "pair": "pair_07", "game_a": "game_14", "color_a": "black", "game_b": "game_08", "color_b": "white", "rating_a": 1188, "rating_b": 1310, "gap": 122, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.75, "correct": 0, "brier": 0.5625, "old_estimate_a": 1200, "old_estimate_b": 1200, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "In Game A, Black (Target A) displayed reasonable tactical awareness, capitalizing on White's mistake to win a knight with 10... Bxh4 and safely converting the material advantage to a win. In contrast, in Game B, White (Target B) played a very passive opening and exhibited poor tactical vision. Target B overlooked a simple pawn fork, losing a piece to 14... c5, and later allowed a devastating pin with 20. Nf3 Bg4 and 21... Ng5, leading to a quick collapse. Target A's play was significantly more solid and tactically alert.", "confidence_valid": true }, { "model": "google/gemini-3.7-flash", "pair": "pair_07", "game_a": "game_14", "color_a": "black", "game_b": "game_08", "color_b": "white", "rating_a": 1188, "rating_b": 1310, "gap": 122, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.72, "correct": 0, "brier": 0.5184, "old_estimate_a": 1500, "old_estimate_b": 1350, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A Black demonstrated solid positional fundamentals, defending their center, contesting the d-file, and cleanly punishing White's final blunder with 30...Rxd3 (though missing the free knight earlier on move 15). In contrast, Game B White played an extremely passive and disjointed opening setup (Nc3, d3, e3, g3), walked into a simple pawn fork with 14...c5 losing a piece on c5 without compensation, and subsequently collapsed to the pin on f3 with 20...Bg4 and 21...Ng5. Target A displayed much better board vision and coordination.", "confidence_valid": true }, { "model": "qwen/qwen3.8-max-prime", "pair": "pair_07", "game_a": "game_14", "color_a": "black", "game_b": "game_08", "color_b": "white", "rating_a": 1188, "rating_b": 1310, "gap": 122, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.42, "correct": 1, "brier": 0.17639999999999997, "old_estimate_a": 1400, "old_estimate_b": 1350, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A Black wins mostly from White blundering away a rook: 29...Rd4 30.Rg6?? Rxd3, with Black's play (16...Na6, 18...Qg6, 22...Rd8) competent but unexceptional. Game B White, despite losing, played 16.Ne5 and 17.Bxe4, creating real complications, and only lost to a slick finish 22...Bxf3+ 23.Kg1 Be4 24...Nf3+. B's active middlegame ideas and tighter calculation against stronger opposition suggest a slightly higher rating.", "confidence_valid": true }, { "model": "anthropic/claude-fable-5.1", "pair": "pair_07", "game_a": "game_14", "color_a": "black", "game_b": "game_08", "color_b": "white", "rating_a": 1188, "rating_b": 1310, "gap": 122, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.55, "correct": 0, "brier": 0.30250000000000005, "old_estimate_a": 1300, "old_estimate_b": 1250, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A Black played a passive setup (1...d6 2...e6 3...h6, 5...Bxd6 6...Be7 retreat) but found reasonable simplifications (10...Bxh4, 11...Bg5) and won material after White's 29.fxe5?? 30.Rg6?? blunder (30...Rxd3). Black also made odd shuffles (15...Rd8, 21...Rd5, 22...Rd8). Game B White built a solid Van Geet/fianchetto setup, but 14.Rfe1? allowed 14...c5 winning the Bb4 and a piece via 15.Bxc5 Ndxc5, then 20.Nf3 Bg4 21.Kg2 Ng5 22.Rd4?? Bxf3+ dropped more material. White B's errors were unforced piece-losing blunders against modest opposition; Black A's play, while passive, contained fewer outright material losses. Slight edge to A.", "confidence_valid": true }, { "model": "google/gemini-3-flash-preview", "pair": "pair_07", "game_a": "game_14", "color_a": "black", "game_b": "game_08", "color_b": "white", "rating_a": 1188, "rating_b": 1310, "gap": 122, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.65, "correct": 0, "brier": 0.42250000000000004, "old_estimate_a": 1300, "old_estimate_b": 1300, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "Target A (Black) demonstrated solid fundamentals, winning material early (10... Bxh4) and maintaining a clear advantage through the endgame. While Black missed a winning tactical opportunity on move 15, their play remained cohesive and focused. Conversely, Target B (White) displayed poor tactical awareness, blundering a full piece to a basic pawn fork (14... c5). Target B subsequently failed to manage the pin on the f3-knight, allowing the decisive fork 24. Nf3+. The disparity in tactical oversight suggests Target A is the stronger player; Target B's loss of material to a simple one-move threat indicates a lower level of established play compared to Target A's overall stability.", "confidence_valid": true }, { "model": "google/gemini-3.6-flash", "pair": "pair_07", "game_a": "game_14", "color_a": "black", "game_b": "game_08", "color_b": "white", "rating_a": 1188, "rating_b": 1310, "gap": 122, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.8, "correct": 0, "brier": 0.6400000000000001, "old_estimate_a": 1300, "old_estimate_b": 1200, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Target A (Black) played a solid positional game despite a passive opening (6...Be7) and missing a free knight with 15...Qxe4. However, A maintained structural integrity, defended well, and accurately punished White's 30. Rg6 blunder with 30...Rxd3. In contrast, Target B (White) demonstrated severe tactical oversights early in Game B. B allowed a troubling pin with 9...e4, missed the basic 14...c5 fork that lost a bishop, and later allowed 20...Bg4 and 21...Ng5, walking into a fatal knight fork on f3. Target A showed far better fundamental tactical awareness and stability.", "confidence_valid": true }, { "model": "openai/gpt-5.6-sol", "pair": "pair_07", "game_a": "game_14", "color_a": "black", "game_b": "game_08", "color_b": "white", "rating_a": 1188, "rating_b": 1310, "gap": 122, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.78, "correct": 0, "brier": 0.6084, "old_estimate_a": 1550, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A’s Black played more consistently and purposefully: 9...Nd5 answered White’s pressure, 19...f5 challenged the centralized knight, and 21...Rd5 followed by ...Rd8/...Rad8 coordinated both rooks before 30...Rxd3 punished the loose bishop. Game B’s White showed reasonable development and simplification with 17.Bxe4 and 18.Qxb6, but the passive opening conceded initiative, and 22.Rd4? overlooked 22...Bxf3+, where the bishop could escape because ...Ng5 protected f3. Then 24.h4? allowed 24...Nf3+, also forking the rook on d4. These tactical oversights make A more likely to have had the higher established rapid strength.", "confidence_valid": true }, { "model": "openai/gpt-6-sol", "pair": "pair_07", "game_a": "game_14", "color_a": "black", "game_b": "game_08", "color_b": "white", "rating_a": 1188, "rating_b": 1310, "gap": 122, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.95, "correct": 0, "brier": 0.9025, "old_estimate_a": 1650, "old_estimate_b": 1300, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A’s Black handled the exchanges coherently with 20...Nxc5 and 28...Qxe5, then exploited White’s exposed bishop with 29...Rd4 and 30...Rxd3. Game B’s White made more consequential errors: 15.Bxc5 gave up a bishop for a pawn after 15...Ndxc5, and 17.Bxe4 Nxe4 lost the other bishop. Black then won White’s remaining knight with 22...Bxf3+. A looks substantially stronger despite an unpolished opening.", "confidence_valid": true }, { "model": "openai/gpt-6.1-sol", "pair": "pair_07", "game_a": "game_14", "color_a": "black", "game_b": "game_08", "color_b": "white", "rating_a": 1188, "rating_b": 1310, "gap": 122, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.87, "correct": 0, "brier": 0.7569, "old_estimate_a": 1550, "old_estimate_b": 1250, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A Black showed better tactical awareness: 10...Bxh4 answered White’s capture on d5 by recovering a knight, and 20...Nxc5 removed White’s advanced knight. Black then coordinated the rooks with 26...Rad8 and 29...Rd4, exploiting 30.Rg6 with 30...Rxd3. The opening was slow, notably ...h6 and the bishop retreat to e7, but subsequent play was coherent. Game B White developed reasonably, yet 15.Bxc5 surrendered a bishop for a pawn without adequate compensation. Later, 22.Rd4 left the knight on f3 available to ...Bxf3+, and 24.h4 allowed ...Nf3+, forking both rooks. These repeated material oversights favor A.", "confidence_valid": true }, { "model": "openai/gpt-5.6-luna", "pair": "pair_07", "game_a": "game_14", "color_a": "black", "game_b": "game_08", "color_b": "white", "rating_a": 1188, "rating_b": 1310, "gap": 122, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.68, "correct": 0, "brier": 0.4624000000000001, "old_estimate_a": 1700, "old_estimate_b": 1400, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A’s Black showed more purposeful coordination and tactical awareness: after 9...Nd5, 10...Bxh4 exploited the cleared diagonal, and the rooks became very active with 21...Rd5, 29...Rd4, and 30...Rxd3. Game B’s White developed passively with 1.Nc3 and 2.d3, then allowed Black’s initiative to grow; 22.Rd4 permitted 22...Bxf3+, followed by the decisive-looking 24...Nf3+. Although Game B’s White made some useful tactical exchanges, Game A’s Black appeared more consistently effective, so A is more likely to have had the higher established rapid strength.", "confidence_valid": true }, { "model": "openai/gpt-6-astra", "pair": "pair_07", "game_a": "game_14", "color_a": "black", "game_b": "game_08", "color_b": "white", "rating_a": 1188, "rating_b": 1310, "gap": 122, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.86, "correct": 0, "brier": 0.7395999999999999, "old_estimate_a": 1550, "old_estimate_b": 1350, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A Black played a slow, somewhat aimless opening, but showed tactical awareness with 10...Bxh4, developed both rooks, and used 28...Qxe5 followed by 29...Rd4 to exploit White’s position, taking the loose bishop with 30...Rxd3. Game B White made more serious material errors: 15.Bxc5? Ndxc5 surrendered a bishop for a pawn, while 22.Rd4? allowed ...Bxf3+, losing another minor piece. Finally, 24.h4 permitted ...Nf3+, forking the king and both rooks. Despite A benefiting from weak opposition, B’s repeated failures to track threats make A the likelier stronger established player.", "confidence_valid": true }, { "model": "z-ai/glm-5.3-flash", "pair": "pair_07", "game_a": "game_14", "color_a": "black", "game_b": "game_08", "color_b": "white", "rating_a": 1188, "rating_b": 1310, "gap": 122, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.72, "correct": 0, "brier": 0.5184, "old_estimate_a": 1600, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A's Black played coherently: good trades (10...Bxh4, 12...Qxg5), sensible regrouping with ...Na6, ...Bd7, and careful maneuvering (17.Ne4... wait, Black's ...f5 and ...Qf6 redeployment), then converted a small edge cleanly, with accurate rook play 29...Rxd4 30...Rxd3 winning a pawn. Game B's White blundered chronically: 14.Bxc5 allowed ...Ndxc5 regaining, and after 22.Rd4 Bxf3+ 23.Kg1 Be4 24.h4?? White lost to ...Nf3+ with the rook on d4 hanging and king unsafe—decisive carelessness. Black also erred little. A's target showed better technique and fewer mistakes, so A was likely higher-rated.", "confidence_valid": true } ], "a": { "game": "game_14", "color": "black", "rating": 1188, "url": "https://lichess.org/tqOYQRH5", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"0-1\"]\n[TimeControl \"600+0\"]\n\n1. e4 d6 2. Bc4 e6 3. Nf3 h6 4. e5 c6 5. exd6 Bxd6 6. O-O Be7 7. d4 Nf6 8. Nh4\nO-O 9. Qf3 Nd5 10. Bxd5 Bxh4 11. Bc4 Bg5 12. Bxg5 Qxg5 13. Nc3 Re8 14. Ne4 Qf5\n15. Qg3 Rd8 16. Bd3 Na6 17. Nd6 Qf6 18. Ne4 Qg6 19. Qe3 f5 20. Nc5 Nxc5 21.\ndxc5 Rd5 22. c4 Rd8 23. Rae1 Bd7 24. f4 Re8 25. Rf3 Qf6 26. b3 Rad8 27. Rg3 Bc8\n28. Qe5 Qxe5 29. fxe5 Rd4 30. Rg6 Rxd3 0-1" }, "b": { "game": "game_08", "color": "white", "rating": 1310, "url": "https://lichess.org/5z2eLce5", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"0-1\"]\n[TimeControl \"600+0\"]\n\n1. Nc3 e5 2. d3 d5 3. e3 Bb4 4. Bd2 Nf6 5. g3 O-O 6. Bg2 c6 7. Nf3 Bg4 8. O-O\nBxc3 9. Bxc3 e4 10. dxe4 Nxe4 11. Bb4 Re8 12. Qd4 Be6 13. Rad1 Nd7 14. Rfe1 c5\n15. Bxc5 Ndxc5 16. Ne5 Qb6 17. Bxe4 Nxe4 18. Qxb6 axb6 19. a3 f6 20. Nf3 Bg4\n21. Kg2 Ng5 22. Rd4 Bxf3+ 23. Kg1 Be4 24. h4 Nf3+ 0-1" }, "wrong": 14, "gap": 122 }, { "id": "pair_05", "predictions": [ { "model": "anthropic/claude-opus-5.5", "pair": "pair_05", "game_a": "game_09", "color_a": "white", "game_b": "game_07", "color_b": "black", "rating_a": 2193, "rating_b": 1593, "gap": 600, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.6, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 1700, "old_estimate_b": 1350, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A White handled a Benoni-type opening soundly (6.g3, 9.Nge2, 10.O-O) against a clearly competent opponent, but was outplayed tactically: 21...Bxc3 22.bxc3 Qxa3 cost two pawns, and the later 32.Rxe7 Qf4+ allowed a winning endgame for Black. Still, White kept active play (28.e5, 30.f5) against strong opposition. Game B Black played a solid Slav and found 21...Bc2, forking queen and rook, but this was against a very weak opponent: 21.Bf3?? and 22.Bh5?? simply dropped the queen. That gives little evidence about Black's real strength. The tougher opposition in Game A slightly favors White there.", "confidence_valid": true }, { "model": "anthropic/claude-sonnet-5.5", "pair": "pair_05", "game_a": "game_09", "color_a": "white", "game_b": "game_07", "color_b": "black", "rating_a": 2193, "rating_b": 1593, "gap": 600, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.6, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 1750, "old_estimate_b": 1550, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A (White) played a reasonable King's Indian/Benoni setup with 9.Nge2, 10.O-O and 12.Bd2. It was then outplayed: 13.Qc1 and 14.h3 allowed ...Ng4-e5-c4 and ...Qxa3 (22...Qxa3 won a pawn), and 19.f4 allowed 19...Nd4. White still fought on with activity (25.Rxa7, 32.Rxe7), so this looks like a solid mid-level player with some tactical lapses. Game B (White) made a serious blunder, 21.Bf3 followed by 22.Bh5, which lost the queen for a bishop and the exchange. The earlier play was passive, and 13.Ba3 and 17.b4 were not harmful but gave Black easy equality. Black in Game B played accurately, but that player is not the target. Target B (Black) played a standard Slav/QGD structure well, winning 12...Bxe4 and 21...Bc2 tactically, though against a weak opponent. Evidence for B is limited because the opponent was weak, while Game A's Black was stronger. Slight lean to A.", "confidence_valid": true }, { "model": "google/gemini-3.8-flash", "pair": "pair_05", "game_a": "game_09", "color_a": "white", "game_b": "game_07", "color_b": "black", "rating_a": 2193, "rating_b": 1593, "gap": 600, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.35, "correct": 0, "brier": 0.42250000000000004, "old_estimate_a": 1900, "old_estimate_b": 1650, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game B Black showed excellent positional handling of the Slav/London structure, trading pieces harmoniously and setting up a decisive tactical strike with 20...Qg6! (threatening mate on g2) followed by 21...Bc2! to cleanly net a rook. In Game A, White handled the Benoni setup sluggishly (13. Qc1, 14. h3, 15. Kh2), allowing Black free rein on the queenside. White then blundered a pawn with 21. Qxe2? and drifted into a completely passive, dominated position along the b-file and second rank. Target B exhibited cleaner tactical precision and overall coordination than Target A.", "confidence_valid": true }, { "model": "google/gemini-3.1-pro-preview", "pair": "pair_05", "game_a": "game_09", "color_a": "white", "game_b": "game_07", "color_b": "black", "rating_a": 2193, "rating_b": 1593, "gap": 600, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.15, "correct": 0, "brier": 0.7224999999999999, "old_estimate_a": 1950, "old_estimate_b": 1550, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Target A (White in Game A) plays a passive and slightly awkward opening (8. a3, 12. Bd2, 13. Qc1) and is outplayed positionally before blundering material with 21. Qxe2 Bxc3. They are eventually ground down in a lost endgame. Target B (Black in Game B) plays a solid and very well-developed Slav Defense, easily equalizing. They demonstrate excellent tactical awareness by exploiting White's inaccuracies, culminating in the sharp sequence 20... Qg6 and 21... Bc2!, which cleanly wins a rook through an intermediate exchange of queens. Target B's clean, mistake-free play strongly suggests a higher rating.", "confidence_valid": true }, { "model": "google/gemini-3.5-flash", "pair": "pair_05", "game_a": "game_09", "color_a": "white", "game_b": "game_07", "color_b": "black", "rating_a": 2193, "rating_b": 1593, "gap": 600, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.2, "correct": 0, "brier": 0.6400000000000001, "old_estimate_a": 1550, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game B Black showed excellent positional play and tactical alertness in a Slav Defense. Black equalized smoothly, then launched a decisive tactical sequence with 20...Qg6 (threatening mate) and 21...Bc2, utilizing a fork on White's queen and rook to win a full rook. Conversely, Game A White played passively in the opening and middlegame (such as 15. Kh2 and 16. Rb1), allowed Black to dominate the queenside, blundered a pawn on a3, and defended poorly, ultimately getting mated on the back rank.", "confidence_valid": true }, { "model": "google/gemini-3.7-flash", "pair": "pair_05", "game_a": "game_09", "color_a": "white", "game_b": "game_07", "color_b": "black", "rating_a": 2193, "rating_b": 1593, "gap": 600, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.28, "correct": 0, "brier": 0.5184, "old_estimate_a": 1850, "old_estimate_b": 1700, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Target B (Black) demonstrated clean, mature handling of the Slav Defense, effortlessly completing development, coordinating pieces toward White's kingside, and instantly exploiting tactical weaknesses with 20...Qg6! threatening g2 and setting up the decisive 21...Bc2 fork winning a rook. Target A (White) played rather passively against the Benoni, lost queenside control, blundered a pawn via 21.Qxe2? Bxc3 22.bxc3 Qxa3, and allowed Black's heavy pieces to completely dominate the 2nd rank and finish the game.", "confidence_valid": true }, { "model": "qwen/qwen3.8-max-prime", "pair": "pair_05", "game_a": "game_09", "color_a": "white", "game_b": "game_07", "color_b": "black", "rating_a": 2193, "rating_b": 1593, "gap": 600, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.72, "correct": 1, "brier": 0.07840000000000001, "old_estimate_a": 1450, "old_estimate_b": 1600, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game A White handled a Modern Benoni with reasonable theory (6.g3, 9.Nge2) but collapsed tactically: 15.Kh2? allowed ...Nc4, and 19.f4? lost the d4-knight sequence, leaving Black a clear pawn up. Despite losing, White found the resourceful 28.e5 and kept resistance into a complex rook ending. Game B Black was outplayed positionally; 21...Bc2?? hangs material to 22.Bh5! and Black resigned after 22...Bxd1 23.Bxg6 Bxb3 24.Bd3, losing the exchange and position outright. A's errors were subtle; B blundered a full exchange in a quiet position, indicating lower strength.", "confidence_valid": true }, { "model": "anthropic/claude-fable-5.1", "pair": "pair_05", "game_a": "game_09", "color_a": "white", "game_b": "game_07", "color_b": "black", "rating_a": 2193, "rating_b": 1593, "gap": 600, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.4, "correct": 0, "brier": 0.36, "old_estimate_a": 1500, "old_estimate_b": 1400, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A White played a reasonable Benoni setup but drifted: 12.Bd2/13.Qc1 was passive, 15.Kh2 and 16.Rb1 allowed Nxd2 and the collapse of the queenside pawns (22...Qxa3, 23...Qxc3), then 28.e5/29.g4 opened the king and 32.Rxe7 lost to 32...Qf4+ trading into a hopeless rook ending. Game B Black played solid Slav development, correctly recaptured 8...exd5, equalized cleanly, and spotted the tactic 21...Bc2 winning the exchange; White's 22.Bh5?? blundered the queen, which Black handled accurately. Black in B showed sounder positional play against a fairly competent opponent; White in A was outplayed throughout.", "confidence_valid": true }, { "model": "google/gemini-3-flash-preview", "pair": "pair_05", "game_a": "game_09", "color_a": "white", "game_b": "game_07", "color_b": "black", "rating_a": 2193, "rating_b": 1593, "gap": 600, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.3, "correct": 0, "brier": 0.48999999999999994, "old_estimate_a": 1800, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Target B showed superior tactical vision and efficiency. The maneuver 18... Qe6 and 20... Qg6 prepared a clever tactical shot, 21... Bc2, which won the exchange and forced resignation. B's play was solid and purposeful throughout the Slav structure. Target A, playing White, struggled with coordination in a Benoni. Allowing the loss of the bishop pair with 16. Rb1 and failing to handle the pressure on the kingside (culminating in the 39-41 sequence) suggests a lower level of tactical and positional awareness compared to Target B. A's 13. Qc1 and 19. f4 were also somewhat imprecise, leading to a disadvantageous position.", "confidence_valid": true }, { "model": "google/gemini-3.6-flash", "pair": "pair_05", "game_a": "game_09", "color_a": "white", "game_b": "game_07", "color_b": "black", "rating_a": 2193, "rating_b": 1593, "gap": 600, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.2, "correct": 0, "brier": 0.6400000000000001, "old_estimate_a": 1850, "old_estimate_b": 1550, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A White played passively out of the opening against a Benoni setup with an early 6. g3. White struggled for a plan, conceded the queenside initiative, lost a pawn with 22. bxc3, and allowed Black's pieces to dominate the position down to a hopeless rook endgame. In contrast, Game B Black played a very solid and classical Slav Defense structure, maneuvered smoothly in the middlegame, and showed sharp tactical alertness by punishing White's blunder (21. Bf3) with 21...Bc2!, winning the queen after 22. Bh5 Bxd1. Game B Black displayed much cleaner overall technique and tactical awareness.", "confidence_valid": true }, { "model": "openai/gpt-5.6-sol", "pair": "pair_05", "game_a": "game_09", "color_a": "white", "game_b": "game_07", "color_b": "black", "rating_a": 2193, "rating_b": 1593, "gap": 600, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.38, "correct": 0, "brier": 0.3844, "old_estimate_a": 1900, "old_estimate_b": 1650, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game B’s Black played a coherent Slav setup and showed clear tactical awareness with 20...Qg6 and 21...Bc2, exploiting White’s misplaced rook and queen; after 22.Bh5, 22...Bxd1 23.Bxg6 Bxb3 converted the sequence cleanly. Game A’s White showed good opening familiarity and active ideas such as 25.Rxa7 and 28.e5, but 19.f4 allowed ...Nd4 followed by exchanges that exposed the queenside, while 22...Qxa3 and 23...Qxc3 won pawns. Later, White’s rooks became vulnerable to the coordinated invasion 38...Rdd2, 39...Rh2+, and 40...Rbg2+. B’s game was shorter and opponent-assisted, but B displayed fewer weaknesses.", "confidence_valid": true }, { "model": "openai/gpt-6-sol", "pair": "pair_05", "game_a": "game_09", "color_a": "white", "game_b": "game_07", "color_b": "black", "rating_a": 2193, "rating_b": 1593, "gap": 600, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.25, "correct": 0, "brier": 0.5625, "old_estimate_a": 1800, "old_estimate_b": 1800, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "Game B’s Black played a sound opening and found the tactical 21...Bc2, attacking White’s queen and rook; after 22.Bh5, Black collected both with 22...Bxd1 and 23...Bxb3. Game A’s White fought back actively with 25.Rxa7 and 32.Rxe7, but allowed Black’s queen to take pawns on a3 and c3 and later struggled against active rooks. B’s play looks more controlled, though White’s abrupt collapse in Game B limits confidence.", "confidence_valid": true }, { "model": "openai/gpt-6.1-sol", "pair": "pair_05", "game_a": "game_09", "color_a": "white", "game_b": "game_07", "color_b": "black", "rating_a": 2193, "rating_b": 1593, "gap": 600, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.64, "correct": 0, "brier": null, "old_estimate_a": 1900, "old_estimate_b": 1800, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game B Black showed coordinated tactical play: 18...Qe6 and 20...Qg6 created kingside threats, while 21...Bc2 forked White’s queen and rook. After 22.Bh5, Black correctly chose 22...Bxd1 and 23...Bxb3, emerging a rook ahead despite losing the queen. Game A White handled a more demanding position and generated activity with 25.Rxa7 and 28.e5, but allowed the sequence 19...Nd4 through 23...Qxc3 to cost two pawns, then failed to contain Black’s second-rank rooks and mating net. B’s opponent supplied a major tactical opportunity, so the comparison remains uncertain, but B’s cleaner execution slightly outweighs A’s greater complexity.", "confidence_valid": false }, { "model": "openai/gpt-5.6-luna", "pair": "pair_05", "game_a": "game_09", "color_a": "white", "game_b": "game_07", "color_b": "black", "rating_a": 2193, "rating_b": 1593, "gap": 600, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.65, "correct": 0, "brier": null, "old_estimate_a": 1450, "old_estimate_b": 1500, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game B’s target Black showed the clearer tactical understanding: after 21 Bf3, ...Bc2 attacked both the queen on d1 and rook on b3, and Black correctly collected them with 22...Bxd1 and 23...Bxb3. The opening play was also coherent. Game A’s target White played actively, including 28 e5 and 32 Rxe7, but allowed Black’s queenside pawn gains with 22...Qxa3 and 23...Qxc3, then was overwhelmed by the coordinated rooks after 37...Kg7. Thus B has the stronger evidence, though the short game and White’s major blunder make the comparison uncertain.", "confidence_valid": false }, { "model": "openai/gpt-6-astra", "pair": "pair_05", "game_a": "game_09", "color_a": "white", "game_b": "game_07", "color_b": "black", "rating_a": 2193, "rating_b": 1593, "gap": 600, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.59, "correct": 0, "brier": null, "old_estimate_a": 1900, "old_estimate_b": 1750, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game B Black developed coherently, activated the queen with 18...Qe6 and 20...Qg6, and exploited the queen–rook fork with 21...Bc2. After 22.Bh5, Black correctly played 22...Bxd1 and 23...Bxb3, winning a rook through the queen exchange. Game A White showed more ambitious play, including 28.e5 and 30.f5, but allowed damaging simplification after 20.Qd3 and ultimately missed the decisive mating attack with 39.R8xc5? Rh2+. B gets a modest edge for tactical control, although B’s opponent provided a much easier opportunity than A’s.", "confidence_valid": false }, { "model": "z-ai/glm-5.3-flash", "pair": "pair_05", "game_a": "game_09", "color_a": "white", "game_b": "game_07", "color_b": "black", "rating_a": 2193, "rating_b": 1593, "gap": 600, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.62, "correct": 0, "brier": null, "old_estimate_a": 1600, "old_estimate_b": 1550, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game B's Black plays a clean, forcing tactic: 20...Qg6 21...Bc2! 22...Bxd1 23...Bxb3 nets a full rook with back-rank themes, executed instantly in rapid time. Game A's White shows decent Benoni handling (12.Bd2, 19.f4) but repeatedly errs: 15.Kh2 lets the knight fork, 27.Rg1 paralyzes the rook, and 35.Rc1/36.Bxd5 drops the bishop, with passive defense throughout. Black B's tactical accuracy under time pressure suggests the higher established strength, while White A's errors look more typical of a lower-rated rapid player.", "confidence_valid": false } ], "a": { "game": "game_09", "color": "white", "rating": 2193, "url": "https://lichess.org/tGYghpt5", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"0-1\"]\n[TimeControl \"600+0\"]\n\n1. d4 Nf6 2. c4 c5 3. d5 d6 4. Nc3 g6 5. e4 Bg7 6. g3 O-O 7. Bg2 Na6 8. a3 Bd7\n9. Nge2 Nc7 10. O-O b5 11. cxb5 Nxb5 12. Bd2 Rb8 13. Qc1 Ng4 14. h3 Ne5 15. Kh2\nNc4 16. Rb1 Nxd2 17. Qxd2 Qa5 18. Qc2 Rfc8 19. f4 Nd4 20. Qd3 Nxe2 21. Qxe2\nBxc3 22. bxc3 Qxa3 23. Ra1 Qxc3 24. Rfc1 Qd4 25. Rxa7 Rb2 26. Qf3 Qd2 27. Rg1\nRd8 28. e5 Bf5 29. g4 Bb1 30. f5 dxe5 31. fxg6 hxg6 32. Rxe7 Qf4+ 33. Qxf4 exf4\n34. Kh1 Ba2 35. Rc1 Bxd5 36. Bxd5 Rxd5 37. Re8+ Kg7 38. Rc8 Rdd2 39. R8xc5 Rh2+\n40. Kg1 Rbg2+ 41. Kf1 f3 0-1" }, "b": { "game": "game_07", "color": "black", "rating": 1593, "url": "https://lichess.org/qtL5x6uE", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"0-1\"]\n[TimeControl \"600+0\"]\n\n1. d4 d5 2. c4 c6 3. e3 Bf5 4. Nf3 Nf6 5. Nbd2 h6 6. Be2 Nbd7 7. O-O e6 8. cxd5\nexd5 9. h3 Bd6 10. b3 O-O 11. a4 Ne4 12. Nxe4 Bxe4 13. Ba3 Bxa3 14. Rxa3 Nf6\n15. Ne5 Nd7 16. Nxd7 Qxd7 17. b4 a6 18. Rb3 Qe6 19. b5 axb5 20. axb5 Qg6 21.\nBf3 Bc2 22. Bh5 Bxd1 23. Bxg6 Bxb3 24. Bd3 0-1" }, "wrong": 13, "gap": 600 }, { "id": "pair_02", "predictions": [ { "model": "anthropic/claude-opus-5.5", "pair": "pair_02", "game_a": "game_01", "color_a": "black", "game_b": "game_24", "color_b": "white", "rating_a": 1159, "rating_b": 762, "gap": 397, "gap_band": "200-499", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.55, "correct": 1, "brier": 0.20249999999999996, "old_estimate_a": 1200, "old_estimate_b": 1250, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Both games are low-level. In Game A, Black allowed a fork chain with 15.Nd5 and 16.Nf6+, where 7...Qd7 left the queen exposed, giving up the queen for minor pieces. Afterwards Black played reasonably: 19...Nd4, 20...gxf3 and 30...Bxg3 picked up material, and Black converted after White's 40.Qb8+?? blunder. In Game B, White faced a very weak opponent (5...a5, 7...f6, 8...h5) but played dubiously, with 15.g3 and 16.gxh4. White also gave up a knight with 17.Nxf6+ for little, then won because Black blundered the queen with 22...Qd8?? into 23.Bxd8. Game A's Black showed somewhat more tactical resourcefulness against a stronger opponent, so I lean slightly to A.", "confidence_valid": true }, { "model": "anthropic/claude-sonnet-5.5", "pair": "pair_02", "game_a": "game_01", "color_a": "black", "game_b": "game_24", "color_b": "white", "rating_a": 1159, "rating_b": 762, "gap": 397, "gap_band": "200-499", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.62, "correct": 1, "brier": 0.1444, "old_estimate_a": 1600, "old_estimate_b": 1550, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Both targets are fairly weak, but Black in Game A played more purposefully. In Game A, Black's 15...Nxg5 and 18...Bb7 were reasonable, and 19...Nd4 and 30...Bxg3 showed tactical awareness. Black then converted the endgame with 38...h3 and 40...Rxb8. Black did make loose moves such as 2...d5 and 12...h5, and White was also weak (17.Nxd7 was a blunder-ish trade). In Game B, White won only because Black blundered with 20...Qc6 and 22...Qd8. White's own play was sloppy: 16.gxh4 and 17.Nxf6+ were fine, but 11.Nh5 and 20.Nb6 were speculative. Black in Game B played extremely poorly, which suggests a low-rated pool, and White's win says little about their strength. Game A's opponent was somewhat more competent and Black still outplayed them, which points to A being stronger.", "confidence_valid": true }, { "model": "google/gemini-3.8-flash", "pair": "pair_02", "game_a": "game_01", "color_a": "black", "game_b": "game_24", "color_b": "white", "rating_a": 1159, "rating_b": 762, "gap": 397, "gap_band": "200-499", "low_band": "one<1000", "gold": "A", "choice": "B", "probability_a": 0.22, "correct": 0, "brier": 0.6084, "old_estimate_a": 1250, "old_estimate_b": 1250, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "Player B showed much sounder opening fundamentals (developing pieces naturally with Nf3, Nc3, Bc4, and castling) compared to Black in Game A, who pushed kingside pawns recklessly (4...f5, 8...h6, 9...g5). Furthermore, Black in Game A committed an elementary tactical oversight with 15...Nxg5??, directly walking into the 16. Nf6+ fork that dropped their queen for just a minor piece, only winning because White completely hung their queen on move 40. In contrast, White in Game B maintained solid piece coordination, navigated Black's flank aggression, and capitalized on tactics with 20. Nb6 and 22. Bd5 to decisively win material.", "confidence_valid": true }, { "model": "google/gemini-3.1-pro-preview", "pair": "pair_02", "game_a": "game_01", "color_a": "black", "game_b": "game_24", "color_b": "white", "rating_a": 1159, "rating_b": 762, "gap": 397, "gap_band": "200-499", "low_band": "one<1000", "gold": "A", "choice": "B", "probability_a": 0.15, "correct": 0, "brier": 0.7224999999999999, "old_estimate_a": 1200, "old_estimate_b": 900, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Black in Game A demonstrates very low tactical awareness, blundering their queen to a basic one-move knight fork with 15... Nxg5?? 16. Nf6+. Black only wins because White commits an equally egregious queen blunder (40. Qb8+??). White in Game B, despite some awkward maneuvering like 11. Nh5, shows much better tactical vision by finding resources like 17. Nxf6+ to escape a trap and 20. Nb6 to win material. White B's ability to spot and execute these tactics suggests a higher rating than Black A's beginner-level blunders.", "confidence_valid": true }, { "model": "google/gemini-3.5-flash", "pair": "pair_02", "game_a": "game_01", "color_a": "black", "game_b": "game_24", "color_b": "white", "rating_a": 1159, "rating_b": 762, "gap": 397, "gap_band": "200-499", "low_band": "one<1000", "gold": "A", "choice": "B", "probability_a": 0.25, "correct": 0, "brier": 0.5625, "old_estimate_a": 1150, "old_estimate_b": 1250, "baseline_choice": "B", "baseline_credit": 0, "explanation": "In Game A, Black (Target A) committed a severe tactical oversight on move 15, playing 15... Nxg5, which completely ignored the threat on f6 and allowed the simple knight fork 16. Nf6+ winning the queen. In Game B, White (Target B) displayed far better tactical awareness and solid positional understanding. White successfully navigated Black's kingside threats, found a resource with 17. Nxf6+ to save the queen, executed a strong fork with 20. Nb6, and promptly punished Black's queen blunder with 23. Bxd8. Target B's play suggests a much higher rating.", "confidence_valid": true }, { "model": "google/gemini-3.7-flash", "pair": "pair_02", "game_a": "game_01", "color_a": "black", "game_b": "game_24", "color_b": "white", "rating_a": 1159, "rating_b": 762, "gap": 397, "gap_band": "200-499", "low_band": "one<1000", "gold": "A", "choice": "B", "probability_a": 0.18, "correct": 0, "brier": 0.6724000000000001, "old_estimate_a": 1250, "old_estimate_b": 1300, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Player A (Black) exhibited serious tactical oversights, notably blundering their queen on move 15 with 15...Nxg5?? allowing the elementary fork 16. Nf6+, and only winning after White threw away a winning position with 40. Qb8+??. In contrast, Player B (White) played a much more coherent opening, handled Black's kingside thrust reasonably, and seized practical chances with tactical awareness (20. Nb6 and 22. Bd5 winning material before capturing Black's blundered queen on 23. Bxd8).", "confidence_valid": true }, { "model": "qwen/qwen3.8-max-prime", "pair": "pair_02", "game_a": "game_01", "color_a": "black", "game_b": "game_24", "color_b": "white", "rating_a": 1159, "rating_b": 762, "gap": 397, "gap_band": "200-499", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.68, "correct": 1, "brier": 0.10239999999999996, "old_estimate_a": 1300, "old_estimate_b": 1300, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "Target A (Black, Game A) handled a chaotic middlegame competently, finding accurate trades (15...Nxg5, 17...Kxd7) and converting a winning endgame. Target B (White, Game B) won only through Black's severe blunder 22...Qxd8??, losing the queen; before that, Black had outplayed White with 13...Nd4 and 16...Bg4. Black's 8...Qd7 and 26...a5 in Game A show coherent plans, while White in Game B played passively (7.Ne2, 10.Nd2) and relied on an opponent's one-move oversight. A's opponent offered a tougher fight; A's play against it was stronger.", "confidence_valid": true }, { "model": "anthropic/claude-fable-5.1", "pair": "pair_02", "game_a": "game_01", "color_a": "black", "game_b": "game_24", "color_b": "white", "rating_a": 1159, "rating_b": 762, "gap": 397, "gap_band": "200-499", "low_band": "one<1000", "gold": "A", "choice": "B", "probability_a": 0.3, "correct": 0, "brier": 0.48999999999999994, "old_estimate_a": 1200, "old_estimate_b": 1250, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game A Black played a dubious setup (2...d5, 4...f5, 7...Qd7, 12...h5) and allowed 15.Nd5 Nxg5 16.Nf6+ losing the queen for two minor pieces; White then blundered badly (26.Qc6?, 27.Re1?? allowing e4 and Bxg3-Bxe1, and finally 40.Qb8+??), so Black's win reflects a weak opponent more than skill. Game B White handled a bizarre Black setup (3...c6, 4...b5, 7...f6, 8...h5, 9...g5) soundly, developed, castled, and punished tactically with 17.Nxf6+, 20.Nb6 winning the exchange and 22.Bd5 trapping/winning the queen. White in B showed better fundamentals and tactical alertness; Black in A showed poor opening understanding and missed basic tactics.", "confidence_valid": true }, { "model": "google/gemini-3-flash-preview", "pair": "pair_02", "game_a": "game_01", "color_a": "black", "game_b": "game_24", "color_b": "white", "rating_a": 1159, "rating_b": 762, "gap": 397, "gap_band": "200-499", "low_band": "one<1000", "gold": "A", "choice": "B", "probability_a": 0.35, "correct": 0, "brier": 0.42250000000000004, "old_estimate_a": 1400, "old_estimate_b": 1300, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Target B shows better fundamental understanding and tactical alertness. In Game B, White develops logically (4. Bc4, 13. O-O) and exploits Black's blunders, such as the fork with 20. Nb6 and the hanging queen 23. Bxd8. In contrast, Target A (Black) in Game A commits a severe tactical oversight by allowing the simple fork 16. Nf6+, losing the queen. Although Black eventually wins, it is primarily due to White's even more egregious blunder (40. Qb8+), which hangs the queen for nothing. Target B's play is significantly more stable and demonstrates a higher floor of tactical competence than Target A.", "confidence_valid": true }, { "model": "google/gemini-3.6-flash", "pair": "pair_02", "game_a": "game_01", "color_a": "black", "game_b": "game_24", "color_b": "white", "rating_a": 1159, "rating_b": 762, "gap": 397, "gap_band": "200-499", "low_band": "one<1000", "gold": "A", "choice": "B", "probability_a": 0.2, "correct": 0, "brier": 0.6400000000000001, "old_estimate_a": 1350, "old_estimate_b": 1150, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black played overly aggressive pawn pushes early on (4...f5, 9...g5) and committed a major tactical oversight with 15...Nxg5, allowing White's 16. Nf6+ fork to win Black's queen for a knight. Black only won because White blundered back heavily later (including hanging the queen with 40. Qb8+). In contrast, Game B White played a solid opening (1. e4, 2. Nf3, 4. Bc4) and demonstrated far superior tactical alertness, successfully capturing Black's hanging rook with 21. Nxa8 and taking Black's queen after 22...Qd8 23. Bxd8 to seal a clean victory without committing gross blunders.", "confidence_valid": true }, { "model": "openai/gpt-5.6-sol", "pair": "pair_02", "game_a": "game_01", "color_a": "black", "game_b": "game_24", "color_b": "white", "rating_a": 1159, "rating_b": 762, "gap": 397, "gap_band": "200-499", "low_band": "one<1000", "gold": "A", "choice": "B", "probability_a": 0.27, "correct": 0, "brier": 0.5328999999999999, "old_estimate_a": 1450, "old_estimate_b": 1450, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "Game A Black showed fighting resourcefulness with 19...Nd4, 27...e4 and the promotion defense 40...Rxb8 41...Rbxe8, but the elementary queen loss after 15...Nxg5? 16.Nf6+ Ke7 17.Nxd7 is strong evidence against established strength; victory depended on White’s later collapse. Game B White handled an eccentric attack calmly, castled, and exploited tactics accurately: 17.Nxf6+ cleared lines, 20.Nb6 drove the queen while threatening the rook, 21.Nxa8 won material, and 23.Bxd8 punished Black’s final blunder. Although Game B’s opponent was very weak, White made fewer serious errors and displayed clearer tactical awareness.", "confidence_valid": true }, { "model": "openai/gpt-6-sol", "pair": "pair_02", "game_a": "game_01", "color_a": "black", "game_b": "game_24", "color_b": "white", "rating_a": 1159, "rating_b": 762, "gap": 397, "gap_band": "200-499", "low_band": "one<1000", "gold": "A", "choice": "B", "probability_a": 0.16, "correct": 0, "brier": 0.7055999999999999, "old_estimate_a": 1400, "old_estimate_b": 1500, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game A’s Black played actively, but 15...Nxg5 allowed 16.Nf6+ and 17.Nxd7, losing the queen. Black’s eventual win depended heavily on White’s later errors, including 40.Qb8+??. Game B’s White handled the tactics more reliably: 20.Nb6 attacked Black’s queen while threatening the a8 rook, and 22.Bd5 kept pressure on the queen before 23.Bxd8 won it. B therefore looks likelier to have been the stronger established player.", "confidence_valid": true }, { "model": "openai/gpt-6.1-sol", "pair": "pair_02", "game_a": "game_01", "color_a": "black", "game_b": "game_24", "color_b": "white", "rating_a": 1159, "rating_b": 762, "gap": 397, "gap_band": "200-499", "low_band": "one<1000", "gold": "A", "choice": "B", "probability_a": 0.79, "correct": 0, "brier": null, "old_estimate_a": 1300, "old_estimate_b": 1500, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game B White developed more coherently and spotted 20.Nb6, forking Black’s queen and rook, followed by 21.Nxa8. White also exploited 22...Qd8 with 23.Bxd8, though 15.g3 and the subsequent knight sacrifice show imperfect play. Game A Black weakened the position with early pawn pushes and, crucially, allowed 15...Nxg5 16.Nf6+ followed by 17.Nxd7, losing the queen to a straightforward fork. Black showed resourcefulness with 27...e4 and 30...Bxg3–31...Bxe1, but the eventual win depended heavily on White’s errors, especially 40.Qb8+. B’s play provides stronger evidence of established playing strength.", "confidence_valid": false }, { "model": "openai/gpt-5.6-luna", "pair": "pair_02", "game_a": "game_01", "color_a": "black", "game_b": "game_24", "color_b": "white", "rating_a": 1159, "rating_b": 762, "gap": 397, "gap_band": "200-499", "low_band": "one<1000", "gold": "A", "choice": "B", "probability_a": 0.35, "correct": 0, "brier": 0.42250000000000004, "old_estimate_a": 1450, "old_estimate_b": 1500, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game B’s White showed more reliable tactical control: after 20 Nb6, White captured the rook with 21 Nxa8 and then exploited 22...Qd8 with 23 Bxd8, winning decisively without an evident major error. Game A’s Black eventually found useful tactics, especially 40...Rxb8 followed by 41...Rbxe8, but 15...Nxg5 allowed 16 Nf6+ and 17 Nxd7, losing the queen for a minor piece. Black’s eventual win therefore depended heavily on White’s later tactical mistakes, whereas Game B’s White more cleanly converted the opponent’s errors.", "confidence_valid": true }, { "model": "openai/gpt-6-astra", "pair": "pair_02", "game_a": "game_01", "color_a": "black", "game_b": "game_24", "color_b": "white", "rating_a": 1159, "rating_b": 762, "gap": 397, "gap_band": "200-499", "low_band": "one<1000", "gold": "A", "choice": "B", "probability_a": 0.86, "correct": 0, "brier": null, "old_estimate_a": 1400, "old_estimate_b": 1350, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game B White showed better tactical awareness: 17.Nxf6+ captured with tempo, 20.Nb6 forked queen and rook, and 22.Bd5 attacked the queen before 23.Bxd8 punished its retreat. Black’s weak play limits how much this proves, but White exploited the opportunities coherently. Game A Black’s 15...Nxg5 overlooked 16.Nf6+ and 17.Nxd7, losing the queen for inadequate compensation. Black later found useful counterplay with 27...e4 and 30...Bxg3, but the eventual victory depended heavily on White’s errors, especially 40.Qb8+. B’s play is stronger evidence of established strength.", "confidence_valid": false }, { "model": "z-ai/glm-5.3-flash", "pair": "pair_02", "game_a": "game_01", "color_a": "black", "game_b": "game_24", "color_b": "white", "rating_a": 1159, "rating_b": 762, "gap": 397, "gap_band": "200-499", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.65, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1750, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A's Black plays a coherent strategic game: 9...g5-11...g4 secures a strong pawn shield, 15...Nxg5/16...Nf6+ wins material via a calculated knight fork line (15.Nd5?? blunder punished), then Black converts cleanly in the endgame (34-41 precise checks, 38...h3, 39...e7) to promote and force resignation. Game B's White wins mainly because Black self-destructs: 17...Nxf6 allowing Nxf6+, 20...Qc6? losing the rook to 21.Nxa8. White's play (12...Nc4-to-13...Nd4 setup is awkward) shows reasonable tactics but less sustained quality. A's demonstrated technique and punishment of a smaller error suggests higher strength.", "confidence_valid": true } ], "a": { "game": "game_01", "color": "black", "rating": 1159, "url": "https://lichess.org/n6H9m2oi", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"0-1\"]\n[TimeControl \"600+0\"]\n\n1. e4 e5 2. d3 d5 3. Nc3 dxe4 4. Nxe4 f5 5. Nc3 Nc6 6. Nf3 Bd6 7. Bg5 Qd7 8.\nQe2 h6 9. Bd2 g5 10. O-O-O Nf6 11. h4 g4 12. Nh2 h5 13. g3 b6 14. Bg5 Nh7 15.\nNd5 Nxg5 16. Nf6+ Ke7 17. Nxd7 Kxd7 18. hxg5 Bb7 19. f3 Nd4 20. Qf2 gxf3 21.\nNxf3 Nxf3 22. Bg2 Rab8 23. Bxf3 Bxf3 24. Qxf3 Ke6 25. Rdf1 Rbf8 26. Qc6 a5 27.\nRe1 e4 28. Qc4+ Kd7 29. Qa4+ c6 30. dxe4 Bxg3 31. e5 Bxe1 32. Rxe1 h4 33. Qd4+\nKc7 34. Qd6+ Kb7 35. Qd7+ Ka6 36. Qd3+ b5 37. Qd6 Kb6 38. e6 h3 39. e7 Re8 40.\nQb8+ Rxb8 41. e8=Q Rbxe8 42. Rxe8 Rxe8 0-1" }, "b": { "game": "game_24", "color": "white", "rating": 762, "url": "https://lichess.org/S3xZRfct", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"1-0\"]\n[TimeControl \"600+0\"]\n\n1. e4 e5 2. Nf3 d6 3. Nc3 c6 4. Bc4 b5 5. Bb3 a5 6. a4 b4 7. Ne2 f6 8. Ng3 h5\n9. d3 g5 10. Nd2 h4 11. Nh5 c5 12. Nc4 Nc6 13. O-O Nd4 14. Ba2 Be6 15. g3 Qd7\n16. gxh4 Bg4 17. Nxf6+ Nxf6 18. Qe1 gxh4 19. Bg5 Nh7 20. Nb6 Qc6 21. Nxa8 Qxa8\n22. Bd5 Qd8 23. Bxd8 1-0" }, "wrong": 12, "gap": 397 }, { "id": "pair_16", "predictions": [ { "model": "anthropic/claude-opus-5.5", "pair": "pair_16", "game_a": "game_11", "color_a": "black", "game_b": "game_02", "color_b": "white", "rating_a": 1273, "rating_b": 1575, "gap": 302, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.68, "correct": 0, "brier": 0.4624000000000001, "old_estimate_a": 1400, "old_estimate_b": 1200, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A Black chose an odd opening (2...Bc5, 3...Qf6) but then played coherently: 10...Nxd5 won a pawn, 15...Nxe2+ took the bishop, and the endgame was handled actively with 31...dxc4 and 36...Rxc4. Black only collapsed at the end with 40...e3??, allowing the back-rank mate Rd8+. Game B White made more basic errors. 8.Nd5 followed by 9.c3 let Black play 9...Nxe4 for a free pawn. 18.h4?? dropped another pawn, and 20.Qh3 traded queens into a wrecked structure. White then let a rook into e2 and b2, and 25.Bg5?? allowed Rh1#. Black B's poorer play overall points to A being the stronger target.", "confidence_valid": true }, { "model": "anthropic/claude-sonnet-5.5", "pair": "pair_16", "game_a": "game_11", "color_a": "black", "game_b": "game_02", "color_b": "white", "rating_a": 1273, "rating_b": 1575, "gap": 302, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.4, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 1600, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A, Black: played an early Qf6/Qb6 opening and lost the queen trade with 8...Qxc5 fine, but then 23...e4 and 24...d5 let White gain f-pawn space. Black allowed 25.f5 and 30.f6, then 36...Rxc4 and 38...Rc1 lost to a back-rank mate after 41...Re8. Black showed decent tactics like 15...Nxe2+ but was passive and missed the mating net. Game B, White: knew the Ruy Lopez to move 7, but 8.Nd5 and 9.c3 lost a pawn to 9...Nxe4. Then 16.Bh6, 18.h4 and 20.Qh3 were poor, allowing 18...Bxh4 and 21...Re2, and White lost badly with a hung b2 pawn. White in Game B played worse, with more blunders, while Black in Game A was only moderately weak. Slight edge to B's target being weaker, so A is slightly favored, but A's Black also lost. The choice is close.", "confidence_valid": true }, { "model": "google/gemini-3.8-flash", "pair": "pair_16", "game_a": "game_11", "color_a": "black", "game_b": "game_02", "color_b": "white", "rating_a": 1273, "rating_b": 1575, "gap": 302, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.22, "correct": 1, "brier": 0.0484, "old_estimate_a": 1250, "old_estimate_b": 1450, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Player B displays far superior foundational opening and strategic understanding. Playing a standard Ruy Lopez, Game B White followed sensible development schemes (8.Nd5, 10.d4, 15.Qd3) before collapsing in tactical defense and suffering 25...Rh1#. In contrast, Game A Black plays an unprincipled early queen sortie with Scholar's Mate aspirations (3...Qf6), misses White's hanging piece after 34.Bb6? by retreating with 34...Rdd6, and finally commits a catastrophic back-rank blunder with 40...e3?? walking directly into mate in two (41.Rd8+ Re8 42.Rxe8#). These errors are hallmarks of a much lower rating pool.", "confidence_valid": true }, { "model": "google/gemini-3.1-pro-preview", "pair": "pair_16", "game_a": "game_11", "color_a": "black", "game_b": "game_02", "color_b": "white", "rating_a": 1273, "rating_b": 1575, "gap": 302, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.4, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 1150, "old_estimate_b": 1300, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Black in Game A plays a very dubious opening (1...e5, 3...Qf6) and ultimately blunders a simple back-rank mate (40...e3??). White in Game B plays a standard Ruy Lopez but hangs pawns (9. c3, 18. h4) and completely misses a mating net (24. Bc2??). Both players show significant tactical weaknesses, but White in Game B exhibits slightly better opening principles than Black's early queen sorties in Game A, suggesting B might be slightly stronger.", "confidence_valid": true }, { "model": "google/gemini-3.5-flash", "pair": "pair_16", "game_a": "game_11", "color_a": "black", "game_b": "game_02", "color_b": "white", "rating_a": 1273, "rating_b": 1575, "gap": 302, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.4, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 1150, "old_estimate_b": 1350, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black plays the Englund Gambit (1...e5) and follows with naive, beginner-like early queen development (3...Qf6), which strongly suggests a lower established rating. Although Game A Black later coordinates their pieces reasonably well, they blunder a basic mate in two with 40...e3??. Game B White plays a highly standard and theoretical Ruy Lopez opening up to move 8, suggesting a more established understanding of chess principles, even though they suffer a tactical collapse, blundering the e4-pawn on move 9, the h4-pawn on move 18, and mate in one with 25. Bg5??. Game B White's opening competence makes them likely the higher-rated player overall.", "confidence_valid": true }, { "model": "google/gemini-3.7-flash", "pair": "pair_16", "game_a": "game_11", "color_a": "black", "game_b": "game_02", "color_b": "white", "rating_a": 1273, "rating_b": 1575, "gap": 302, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.28, "correct": 1, "brier": 0.07840000000000001, "old_estimate_a": 1400, "old_estimate_b": 1500, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Player B demonstrates standard Ruy Lopez opening principles and reasonable piece coordination before collapsing to tactical oversights in the endgame (allowing 24...Rxh3 and mate). In contrast, Player A plays the suspect 1...e5 against 1.d4 followed by an early queen sortie (3...Qf6), fails to capture a hanging bishop (34...Rdd6 instead of 34...Rxb6), and blunders into an elementary back-rank checkmate on move 40 with 40...e3?? allowing 41.Rd8+.", "confidence_valid": true }, { "model": "qwen/qwen3.8-max-prime", "pair": "pair_16", "game_a": "game_11", "color_a": "black", "game_b": "game_02", "color_b": "white", "rating_a": 1273, "rating_b": 1575, "gap": 302, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.22, "correct": 1, "brier": 0.0484, "old_estimate_a": 1300, "old_estimate_b": 1200, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Target A (Black, Game A) mishandled a dubious opening (1...e5 2...Bc5 3...Qf6), allowed White a comfortable edge after 8.Nxc5, and later blundered into 42.Rxe8#. Target B (White, Game B) chose the passive 8.Nd5, lost a pawn after 9...Nxe4, then allowed 18...Bxh4 winning material, and finally permitted 25...Rh1# with the rook on h1. Both lost, but A's opening was objectively poor and the mate resulted from a clear oversight in a simplified position, whereas B's errors were more tactical in a complex middlegame. B's play suggests a somewhat higher baseline.", "confidence_valid": true }, { "model": "anthropic/claude-fable-5.1", "pair": "pair_16", "game_a": "game_11", "color_a": "black", "game_b": "game_02", "color_b": "white", "rating_a": 1273, "rating_b": 1575, "gap": 302, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.4, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 1150, "old_estimate_b": 1250, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black opened with 1...e5 2...Bc5 3...Qf6 (dubious, queen out early), then shuffled the queen (Qb6, Qc5, Qa5+, Qc7) and later allowed an elementary back-rank mate with 41...Re8 42.Rxe8#, having also lost material to tactics like 36...Rxc4 37.Rab1. Game B White played a coherent Ruy Lopez with Nd5/c3/d4 ideas, but blundered a pawn to 9...Nxe4 and then hung material with 18.h4?? Bxh4, collapsing afterward. Both are weak, but Game B White's opening knowledge and structure look more developed than Game A Black's queen-wandering and back-rank oversight; slight edge to B.", "confidence_valid": true }, { "model": "google/gemini-3-flash-preview", "pair": "pair_16", "game_a": "game_11", "color_a": "black", "game_b": "game_02", "color_b": "white", "rating_a": 1273, "rating_b": 1575, "gap": 302, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.4, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 1400, "old_estimate_b": 1600, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Target B (White) demonstrates a more mature understanding of opening principles, navigating a main-line Ruy Lopez (5. Nc3, 7. O-O) and coordinating pieces logically. Despite the catastrophic one-move blunder (25. Bg5?? allowing 25... Rh1#), the prior play was coherent. In contrast, Target A (Black) chose a dubious opening (1... e5 vs 1. d4) and an early queen development (3... Qf6) that typically characterizes lower-rated play. Target A also blundered a back-rank mate (41... Re8#) in a relatively simple endgame. Target B’s sounder development and opening knowledge suggest a higher established rating.", "confidence_valid": true }, { "model": "google/gemini-3.6-flash", "pair": "pair_16", "game_a": "game_11", "color_a": "black", "game_b": "game_02", "color_b": "white", "rating_a": 1273, "rating_b": 1575, "gap": 302, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.25, "correct": 1, "brier": 0.0625, "old_estimate_a": 1350, "old_estimate_b": 1350, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "Target A (Game A Black) plays the dubious Englund Gambit (1...e5), misses a clean piece win on move 14 (14...Nf4 instead of 14...Qxc4 winning the bishop on c4), and ultimately blunders a simple back-rank checkmate with 41...Re8??. Target B (Game B White) demonstrates solid theoretical opening understanding in the Ruy Lopez through move 17. Although Target B collapses quickly after 18. h4? and gets mated on move 25, the overall quality of opening play and lack of gross early tactical blunders suggest Target B has a higher established rating than Target A.", "confidence_valid": true }, { "model": "openai/gpt-5.6-sol", "pair": "pair_16", "game_a": "game_11", "color_a": "black", "game_b": "game_02", "color_b": "white", "rating_a": 1273, "rating_b": 1575, "gap": 302, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.72, "correct": 0, "brier": 0.5184, "old_estimate_a": 1600, "old_estimate_b": 1400, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A Black showed more sustained competence: despite the dubious 1...e5 and early queen moves, they completed development, castled, found useful play with 23...e4, 24...d5, 28...Nd3, and won the c4-pawn with 36...Rxc4. The final 40...e3? overlooked the back-rank mating sequence 41.Rd8+ Re8 42.Rxe8#, but it was one late tactical failure. Game B White played a coherent opening but deteriorated sharply: 18.h4?! exposed the king, 20.Qh3 invited a damaging queen trade, and 25.Bg5?? allowed the immediate 25...Rh1#. Missing such a direct mate, after already permitting ...Re2 and ...Rxh3, is stronger evidence of lower established strength.", "confidence_valid": true }, { "model": "openai/gpt-6-sol", "pair": "pair_16", "game_a": "game_11", "color_a": "black", "game_b": "game_02", "color_b": "white", "rating_a": 1273, "rating_b": 1575, "gap": 302, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.88, "correct": 0, "brier": 0.7744, "old_estimate_a": 1550, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A’s Black played an eccentric opening but then coordinated well: 10...Nxd5 recovered a pawn, 23...e4 challenged the center, and 25...Qxg3+ simplified. The late 40...e3?? allowed a back-rank mate. Game B’s White made more basic errors: 9.c3 allowed ...Nxe4, 18.h4 led to the loss of a knight after 18...Bxh4 19.Nxh4 Qxh4, and 24.Bc2 failed to address the rook’s attack on h3. Despite A’s final blunder, A appears more likely to have been the stronger established player.", "confidence_valid": true }, { "model": "openai/gpt-6.1-sol", "pair": "pair_16", "game_a": "game_11", "color_a": "black", "game_b": "game_02", "color_b": "white", "rating_a": 1273, "rating_b": 1575, "gap": 302, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.79, "correct": 0, "brier": 0.6241000000000001, "old_estimate_a": 1550, "old_estimate_b": 1250, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A Black showed more sustained middlegame competence: 23...e4 and 24...d5 gained central space, 28...Nd3 established an outpost, and 33...Rd8 followed by ...Rdd6 and ...Rc6 activated both rooks. The decisive 40...e3 overlooked the back-rank mating threat, but followed substantial purposeful play. Game B White developed reasonably, then 18.h4 and 19.Nxh4 led to a pawn-down queen trade without a convincing attack. White allowed ...Re2–...Rxb2–...Rb3–...Rxh3 and finally missed the immediate mate with 25.Bg5. A’s broader positional coordination is stronger evidence despite both targets losing.", "confidence_valid": true }, { "model": "openai/gpt-5.6-luna", "pair": "pair_16", "game_a": "game_11", "color_a": "black", "game_b": "game_02", "color_b": "white", "rating_a": 1273, "rating_b": 1575, "gap": 302, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.78, "correct": 0, "brier": 0.6084, "old_estimate_a": 1600, "old_estimate_b": 1200, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Target A (Black) showed more sustained, purposeful play: after developing, Black generated active counterplay with 23...e4 and 24...d5, then used the rooks actively on the c-file. The late 40...e3 and failure to handle 41 Rd8+ led to 42 Rxe8#, but much of the game was competently conducted. Target B (White) produced a plausible opening, yet the attack beginning with 18 h4 was poorly judged; after 24 Bc2? Rxh3, 25 Bg5?? allowed the immediate 25...Rh1#. B’s failure to address the rook invasion and mating threat suggests weaker established rapid strength.", "confidence_valid": true }, { "model": "openai/gpt-6-astra", "pair": "pair_16", "game_a": "game_11", "color_a": "black", "game_b": "game_02", "color_b": "white", "rating_a": 1273, "rating_b": 1575, "gap": 302, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.8, "correct": 0, "brier": 0.6400000000000001, "old_estimate_a": 1550, "old_estimate_b": 1350, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A Black showed more coherent middlegame and endgame play: 23...e4 and 24...d5 established central control, 28...Nd3 created an active outpost, and 33...Rd8 followed by rook activity won material. However, 40...e3 overlooked the decisive 41.Rd8+. Game B White developed reasonably but lost direction: 18.h4 allowed ...Bxh4, while 20.Qh3 abandoned attacking prospects for a vulnerable, pawn-down ending. White then allowed ...Re2 and ...Rxb2, and 25.Bg5 missed mate in one. A's sustained coordination outweighs the final tactical failure.", "confidence_valid": true }, { "model": "z-ai/glm-5.3-flash", "pair": "pair_16", "game_a": "game_11", "color_a": "black", "game_b": "game_02", "color_b": "white", "rating_a": 1273, "rating_b": 1575, "gap": 302, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.65, "correct": 0, "brier": 0.42250000000000004, "old_estimate_a": 1650, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A's Black plays coherently in a tricky line: 8.Nxc5 Qxc5 keeps material even, and Black trades actively (16...Bg4, 19...Bxf3) and creates counterplay with 23...e4 and 24...d5. The loss comes late (40...e3 41.Rd8+) in a bad endgame, suggesting weaker endgame technique but decent opening/middlegame understanding. Game B's White blunders immediately: 18.h4?? allows 18...Bxh4! 19.Nxh4 Qxh4 winning a piece, then drifts to 21...Re2, 22...Rxb2 and gets mated (25...Rh1#). Conceding a piece to a primitive tactic on move 18 is a clear sign of lower strength.", "confidence_valid": true } ], "a": { "game": "game_11", "color": "black", "rating": 1273, "url": "https://lichess.org/GRATE7vg", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"1-0\"]\n[TimeControl \"600+0\"]\n\n1. d4 e5 2. d5 Bc5 3. e4 Qf6 4. Nf3 c6 5. Nc3 cxd5 6. exd5 Ne7 7. Ne4 Qb6 8.\nNxc5 Qxc5 9. Be3 Qa5+ 10. c3 Nxd5 11. Bd2 Qc7 12. Bd3 O-O 13. O-O d6 14. Bc4\nNf4 15. Be2 Nxe2+ 16. Qxe2 Bg4 17. h3 Bh5 18. Qe4 Nd7 19. Qh4 Bxf3 20. gxf3\nRfe8 21. Qg3 g6 22. Bg5 Nc5 23. b3 e4 24. f4 d5 25. f5 Qxg3+ 26. fxg3 Re5 27.\ng4 a5 28. Be3 Nd3 29. Bd4 Re7 30. f6 Re6 31. c4 dxc4 32. bxc4 a4 33. a3 Rd8 34.\nBb6 Rdd6 35. Bc7 Rc6 36. Bh2 Rxc4 37. Rab1 b6 38. Rb5 Rc1 39. Rxc1 Nxc1 40. Rd5\ne3 41. Rd8+ Re8 42. Rxe8# 1-0" }, "b": { "game": "game_02", "color": "white", "rating": 1575, "url": "https://lichess.org/8foldxhD", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"0-1\"]\n[TimeControl \"600+0\"]\n\n1. e4 e5 2. Nf3 Nc6 3. Bb5 a6 4. Ba4 Nf6 5. Nc3 b5 6. Bb3 Be7 7. O-O O-O 8. Nd5\nd6 9. c3 Nxe4 10. d4 exd4 11. cxd4 Nf6 12. Nxf6+ Bxf6 13. Be3 Na5 14. Bc2 Nc4\n15. Qd3 g6 16. Bh6 Re8 17. Qb3 Bb7 18. h4 Bxh4 19. Nxh4 Qxh4 20. Qh3 Qxh3 21.\ngxh3 Re2 22. Bd3 Rxb2 23. a3 Rb3 24. Bc2 Rxh3 25. Bg5 Rh1# 0-1" }, "wrong": 7, "gap": 302 }, { "id": "pair_20", "predictions": [ { "model": "anthropic/claude-opus-5.5", "pair": "pair_20", "game_a": "game_11", "color_a": "white", "game_b": "game_23", "color_b": "white", "rating_a": 1228, "rating_b": 557, "gap": 671, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.57, "correct": 1, "brier": 0.18490000000000004, "old_estimate_a": 1450, "old_estimate_b": 1350, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A White handled a messy game soundly. After Black's odd 2...Bc5 and 3...Qf6 he won the bishop pair with 8.Nxc5, though 15.Be2 (allowing Nxe2+) and 19.Qh4 (allowing Bxf3 to wreck his pawns) were loose. In the endgame he played actively with 30.f6, 34.Bb6, 38.Rb5 and 40.Rd5, then finished with the back-rank mate 41.Rd8+. Game B White played aimlessly early: 2.Bf4, 5.Bb5, 6.Bg5 moves the bishop around. Weakening moves like 17.f3 let Black's queen roam. He mainly won material because Black gave pieces away (13...Bxh3, 15...Bh4). Neither player is convincing, but A showed somewhat more technique.", "confidence_valid": true }, { "model": "anthropic/claude-sonnet-5.5", "pair": "pair_20", "game_a": "game_11", "color_a": "white", "game_b": "game_23", "color_b": "white", "rating_a": 1228, "rating_b": 557, "gap": 671, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.62, "correct": 1, "brier": 0.1444, "old_estimate_a": 1600, "old_estimate_b": 1700, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game A White (A) played a long, coherent game. They traded the dark-squared bishop with 8.Nxc5 and handled the queen trade on move 25 well. They pushed f6 (30.f6) and the passed pawns, then found the finishing 40.Rd5 and 42.Rxe8#. Black blundered the endgame, though White did make a few inaccuracies, such as 17.h3 and the weakened structure after 19...Bxf3 20.gxf3. Game B White (B) played the slow, passive-looking 3.h3 and 5.Bb5, and 10.Bxc6 gave up the bishop pair. They then allowed 13...Bxh3 14.gxh3 f5, which exposed their king. White won only because Black blundered with 14...f5 15.Nxf5 and 15...Bh4, and the opponent's play was weak throughout (16...Qxh4 and the later queen moves). B's win was largely a gift, and A's performance was more consistent and showed better technique, so A is slightly more likely the stronger player.", "confidence_valid": true }, { "model": "google/gemini-3.8-flash", "pair": "pair_20", "game_a": "game_11", "color_a": "white", "game_b": "game_23", "color_b": "white", "rating_a": 1228, "rating_b": 557, "gap": 671, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.88, "correct": 1, "brier": 0.0144, "old_estimate_a": 1300, "old_estimate_b": 1300, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "Player A demonstrates vastly superior tactical awareness and board vision compared to Player B. In Game A, White develops logically, navigates the opening soundly, and finishes cleanly with an alert mating net (40. Rd5! e3 41. Rd8+). In contrast, Game B features severe tactical blunders by both players. White in Game B hung a piece with 14. Nxf5? (walked into bishop coverage from e7) and later inexplicably hung their queen in broad daylight with 22. Qe5?? (which Black failed to capture with 22...Qg6). The density of basic, one-move blunders in Game B is characteristic of a much lower rating tier.", "confidence_valid": true }, { "model": "google/gemini-3.1-pro-preview", "pair": "pair_20", "game_a": "game_11", "color_a": "white", "game_b": "game_23", "color_b": "white", "rating_a": 1228, "rating_b": 557, "gap": 671, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.85, "correct": 1, "brier": 0.022500000000000006, "old_estimate_a": 1200, "old_estimate_b": 1100, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Target A demonstrates a better understanding of development and tactics, finding nice moves like 7. Ne4 to exploit the opponent's early Queen sortie, though they do hang a pawn later with 10. c3. Target B plays the opening decently but then makes several massive blunders that both players fail to notice. For instance, in Game B, White plays 15. Nxf5 hanging the knight to Black's rook on f8, and later 22. Qe5?? hanging the Queen outright to Black's Queen on e6, which Black completely ignores with 22... Qg6. This mutual blindness to hanging pieces suggests Target B is a beginner, making Target A the stronger player.", "confidence_valid": true }, { "model": "google/gemini-3.5-flash", "pair": "pair_20", "game_a": "game_11", "color_a": "white", "game_b": "game_23", "color_b": "white", "rating_a": 1228, "rating_b": 557, "gap": 671, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.65, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1200, "old_estimate_b": 1250, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game A White showed better overall coordination and successfully executed a back-rank mate with 41. Rd8+ and 42. Rxe8#. Although White made errors like hanging a bishop with 14. Bc4 and 34. Bb6, the general structure of the game was reasonable. In contrast, Game B White committed an egregious one-move blunder by playing 22. Qe5, which hung the queen directly to Black's queen on e6 (which Black incredibly missed with 22... Qg6). This level of basic tactical oversight suggests Game B is of a lower rating band than Game A.", "confidence_valid": true }, { "model": "google/gemini-3.7-flash", "pair": "pair_20", "game_a": "game_11", "color_a": "white", "game_b": "game_23", "color_b": "white", "rating_a": 1228, "rating_b": 557, "gap": 671, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "B", "probability_a": 0.22, "correct": 0, "brier": 0.6084, "old_estimate_a": 1450, "old_estimate_b": 1350, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game B White demonstrates much better tactical awareness and defensive control. In Game B, White played a coherent opening, cleanly won material after Black's positional errors (11. dxe5 and 14. gxh3), and calmly diffused Black's kingside counterplay to force resignation (20. Qe2 and 24. Qh2). In contrast, Game A White made egregious one-move tactical blunders, outright hanging a bishop on move 14 with 14. Bc4 (which Black failed to capture with 14...Qxc4) and again on move 34 with 34. Bb6 (allowing 34...Rxb6). Game A White only won because Black overlooked a basic back-rank mate (41. Rd8+).", "confidence_valid": true }, { "model": "qwen/qwen3.8-max-prime", "pair": "pair_20", "game_a": "game_11", "color_a": "white", "game_b": "game_23", "color_b": "white", "rating_a": 1228, "rating_b": 557, "gap": 671, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.72, "correct": 1, "brier": 0.07840000000000001, "old_estimate_a": 1500, "old_estimate_b": 1050, "baseline_choice": "A", "baseline_credit": 1, "explanation": "In Game A, White shows solid middlegame understanding: developing naturally, exploiting the bishop pair, and converting with precise tactics like 30.f6 and 41.Rd8+ Re8 42.Rxe8#. In Game B, White plays passively (Bb5 then Bxc6 voluntarily, 6.Bg5 awkwardly), allows 13...Bxh3! winning a pawn with attacking chances, and only wins because Black blunders repeatedly (15...Bh4?, 20...Qxh3+? losing the queen after 21.Qh2). Game A's White demonstrates clearer strategic control and cleaner conversion than Game B's White, who relied on opponent errors.", "confidence_valid": true }, { "model": "anthropic/claude-fable-5.1", "pair": "pair_20", "game_a": "game_11", "color_a": "white", "game_b": "game_23", "color_b": "white", "rating_a": 1228, "rating_b": 557, "gap": 671, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "B", "probability_a": 0.4, "correct": 0, "brier": 0.36, "old_estimate_a": 1250, "old_estimate_b": 1450, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game A White faced a weak opponent (1...e5 2.d5 Bc5, Qf6/Qb6 wandering) and played loosely: 7.Ne4 lost the d5 pawn, 19.Qh4 allowed Bxf3 wrecking the kingside, 34.Bb6?/35.Bc7? dropped the c4 pawn, and the win came only via Black's blunder 41...Re8?? allowing Rxe8#. Game B White played a coherent London setup, calmly refuted Black's 13...Bxh3 sacrifice with 15.Nxf5 and 16.Nxh4 winning two pieces, then consolidated (19.Re1, 20.Qe2, 23.Rg1) forcing resignation. Game B's opponent also played more reasonably (Nf6, Bf5, castling), suggesting a higher rating pool. B shows sounder technique.", "confidence_valid": true }, { "model": "google/gemini-3-flash-preview", "pair": "pair_20", "game_a": "game_11", "color_a": "white", "game_b": "game_23", "color_b": "white", "rating_a": 1228, "rating_b": 557, "gap": 671, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "B", "probability_a": 0.35, "correct": 0, "brier": 0.42250000000000004, "old_estimate_a": 1400, "old_estimate_b": 1250, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Target B demonstrates a more solid and principled understanding of the opening (London System) and tactical consolidation. 13. Nd4 in Game B is a strong centralizing move that exploits Black's weaknesses (c6 and f5). Target B also showed good technique by offering a queen trade with 22. Qe5 while a piece up. In contrast, Target A's play was more erratic. Move 14. Bc4 in Game A was a wasted tempo that allowed Black to trade off an influential knight, and White’s kingside structure was unnecessarily shattered (20. gxf3). Target A was technically losing the endgame and only won due to Black’s major back-rank oversight (41. Rd8+), whereas Target B maintained control throughout.", "confidence_valid": true }, { "model": "google/gemini-3.6-flash", "pair": "pair_20", "game_a": "game_11", "color_a": "white", "game_b": "game_23", "color_b": "white", "rating_a": 1228, "rating_b": 557, "gap": 671, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.65, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1400, "old_estimate_b": 1300, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Player A demonstrates superior positional structure and tactical awareness compared to Player B. In Game A, White quickly seized an advantage against the Englund Gambit with 7. Ne4! and maintained good piece activity, eventually capitalizing cleanly on Black's back-rank oversight with 41. Rd8+. While White blundered a bishop with 14. Bc4 (which Black missed), Player A's general play was much more cohesive. In Game B, White played passive early moves (3. h3, 7. Bxf6) and blundered a full queen with 22. Qe5, which Black miraculously failed to take with 22... Qxe5. Game B exhibits a significantly lower level of tactical oversight.", "confidence_valid": true }, { "model": "openai/gpt-5.6-sol", "pair": "pair_20", "game_a": "game_11", "color_a": "white", "game_b": "game_23", "color_b": "white", "rating_a": 1228, "rating_b": 557, "gap": 671, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.76, "correct": 1, "brier": 0.0576, "old_estimate_a": 1700, "old_estimate_b": 1600, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A White showed more coherent strategic and tactical play: 19.Qh4 followed by f4-f5, g4, f6 created a dangerous kingside bind, while 31.c4 and the rook activation 37.Rab1–38.Rb5–40.Rd5 culminated accurately in 41.Rd8+ and 42.Rxe8#. Game B White correctly accepted Black’s unsound 13...Bxh3 sacrifice and found 15.Nxf5 and 16.Nxh4, but the opening plan with early h3, Bb5, Bg5 and repeated exchanges was less convincing, and 17.f3 exposed the king unnecessarily. Game B’s result depended heavily on Black’s crude sacrifice and premature resignation, whereas A demonstrated stronger sustained planning.", "confidence_valid": true }, { "model": "openai/gpt-6-sol", "pair": "pair_20", "game_a": "game_11", "color_a": "white", "game_b": "game_23", "color_b": "white", "rating_a": 1228, "rating_b": 557, "gap": 671, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.77, "correct": 1, "brier": 0.05289999999999999, "old_estimate_a": 1650, "old_estimate_b": 1600, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A White showed more sustained planning against a relatively resilient opponent: 25.f5 secured a queen trade, 30.f6 restricted Black, and 38.Rb5 followed by 40.Rd5 created the decisive rook threat. Game B White correctly accepted Black’s unsound bishop play with 14.gxh3 and 16.Nxh4, then handled the queen checks, but much of that advantage came from Black giving up material. A’s play across the longer, contested game is stronger evidence of established strength.", "confidence_valid": true }, { "model": "openai/gpt-6.1-sol", "pair": "pair_20", "game_a": "game_11", "color_a": "white", "game_b": "game_23", "color_b": "white", "rating_a": 1228, "rating_b": 557, "gap": 671, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "B", "probability_a": 0.6, "correct": 0, "brier": null, "old_estimate_a": 1650, "old_estimate_b": 1550, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game B White showed cleaner tactical accounting: 14.gxh3, 15.Nxf5 and 16.Nxh4 converted Black’s unsound attack into a material advantage. White then handled queen pressure coherently with 20.Qe2, 21.Qh2 and 23.Rg1, offering exchanges while ahead. Game A White found useful active play with 30.f6 and 38.Rb5, then spotted 41.Rd8+ and mate. However, 19.Qh4 allowed damaged kingside pawns, and the subsequent play conceded central activity and the c4 pawn. B’s opponent was much more reckless, so the inference is modest rather than decisive.", "confidence_valid": false }, { "model": "openai/gpt-5.6-luna", "pair": "pair_20", "game_a": "game_11", "color_a": "white", "game_b": "game_23", "color_b": "white", "rating_a": 1228, "rating_b": 557, "gap": 671, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.65, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1700, "old_estimate_b": 1500, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A White appears more likely to have the higher established strength. White handled the opening actively with 7.Ne4 and 8.Nxc5, then coordinated an effective kingside/central attack with 22.Bg5, 23.b3, 25.f5, and 30.f6. The rook activity and finish, 38.Rb5 through 42.Rxe8#, show sustained tactical awareness, despite some loose play such as 20.gxf3. Game B White defended well after Black’s unsound 13...Bxh3, notably with 14.gxh3, 17.f3, 18.Kh1, and 21.Qh2, but the game was short and Black’s sacrifice and subsequent attack were highly questionable. Thus B’s evidence is less informative about established strength.", "confidence_valid": true }, { "model": "openai/gpt-6-astra", "pair": "pair_20", "game_a": "game_11", "color_a": "white", "game_b": "game_23", "color_b": "white", "rating_a": 1228, "rating_b": 557, "gap": 671, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "B", "probability_a": 0.62, "correct": 0, "brier": null, "old_estimate_a": 1500, "old_estimate_b": 1550, "baseline_choice": "B", "baseline_credit": 0, "explanation": "B’s White showed more consistent coordination: 13.Nd4 centralized effectively, and after Black’s dubious 13...Bxh3, White defended with 19.Re1, 20.Qe2 and 21.Qh2, then sought queen exchanges with 22.Qe5 and 24.Qh2. A’s White found useful counterplay with 25.f5 and 30.f6, but 14.Bc4 followed by 15.Be2 wasted time, and the later bishop maneuver allowed Black to collect c4. A’s mating finish was alert, though it depended on Black ignoring the threat with 40...e3. B faced less convincing opposition, so the preference is modest.", "confidence_valid": false }, { "model": "z-ai/glm-5.3-flash", "pair": "pair_20", "game_a": "game_11", "color_a": "white", "game_b": "game_23", "color_b": "white", "rating_a": 1228, "rating_b": 557, "gap": 671, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "B", "probability_a": 0.45, "correct": 0, "brier": 0.30250000000000005, "old_estimate_a": 1700, "old_estimate_b": 1550, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Both whites won against weak opposition, but Black in B was catastrophically weak (13...Bxh3?? and 15...Bh4?? giving away two pieces). White B nonetheless played crisply, punishing accurately (14.gxh3, 15.Nxf5, 16.Nxh4) and safely neutralizing checks. White A also played reasonably but had slack moments (24.f4 instead of fxe4, 27.g4 weakening), and his opponent erred mainly in mutual complicity (23...e4?, 38...Rc1?). B's clean, accurate exploitation against even weaker resistance slightly suggests the stronger established player, though the evidence is thin.", "confidence_valid": true } ], "a": { "game": "game_11", "color": "white", "rating": 1228, "url": "https://lichess.org/GRATE7vg", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"1-0\"]\n[TimeControl \"600+0\"]\n\n1. d4 e5 2. d5 Bc5 3. e4 Qf6 4. Nf3 c6 5. Nc3 cxd5 6. exd5 Ne7 7. Ne4 Qb6 8.\nNxc5 Qxc5 9. Be3 Qa5+ 10. c3 Nxd5 11. Bd2 Qc7 12. Bd3 O-O 13. O-O d6 14. Bc4\nNf4 15. Be2 Nxe2+ 16. Qxe2 Bg4 17. h3 Bh5 18. Qe4 Nd7 19. Qh4 Bxf3 20. gxf3\nRfe8 21. Qg3 g6 22. Bg5 Nc5 23. b3 e4 24. f4 d5 25. f5 Qxg3+ 26. fxg3 Re5 27.\ng4 a5 28. Be3 Nd3 29. Bd4 Re7 30. f6 Re6 31. c4 dxc4 32. bxc4 a4 33. a3 Rd8 34.\nBb6 Rdd6 35. Bc7 Rc6 36. Bh2 Rxc4 37. Rab1 b6 38. Rb5 Rc1 39. Rxc1 Nxc1 40. Rd5\ne3 41. Rd8+ Re8 42. Rxe8# 1-0" }, "b": { "game": "game_23", "color": "white", "rating": 557, "url": "https://lichess.org/LDkelAog", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"1-0\"]\n[TimeControl \"600+0\"]\n\n1. d4 d5 2. Bf4 Nf6 3. h3 Nc6 4. e3 Bf5 5. Bb5 e6 6. Bg5 Be7 7. Bxf6 Bxf6 8.\nNf3 O-O 9. O-O a6 10. Bxc6 bxc6 11. Nbd2 e5 12. dxe5 Be7 13. Nd4 Bxh3 14. gxh3\nf5 15. Nxf5 Bh4 16. Nxh4 Qxh4 17. f3 Qg5+ 18. Kh1 Qxe5 19. Re1 Qg3 20. Qe2\nQxh3+ 21. Qh2 Qe6 22. Qe5 Qg6 23. Rg1 Qh6+ 24. Qh2 1-0" }, "wrong": 6, "gap": 671 }, { "id": "pair_09", "predictions": [ { "model": "anthropic/claude-opus-5.5", "pair": "pair_09", "game_a": "game_12", "color_a": "black", "game_b": "game_03", "color_b": "black", "rating_a": 1811, "rating_b": 1901, "gap": 90, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.45, "correct": 1, "brier": 0.2025, "old_estimate_a": 1350, "old_estimate_b": 1300, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Both targets are weak, so this is close to a guess. Game A Black's 10...Nxd4 combination fails to 12.Bb5+, and 13.Qxd4 wins a piece. Black then missed 13...cxb5, which would have limited the damage, and later ignored his back rank: 26...Nxc4 allowed 27.Bxh8 and 28.Qg8+. Game B Black blundered with 14...Bd6 into the 15.Ng6 fork, losing material, and later dropped the exchange to 34.Ne7+. Still, he defended longer and more coherently against a cleaner opponent, with ideas like 22...e5 and 24...Bxd5. That gives a slight edge to Game B's target.", "confidence_valid": true }, { "model": "anthropic/claude-sonnet-5.5", "pair": "pair_09", "game_a": "game_12", "color_a": "black", "game_b": "game_03", "color_b": "black", "rating_a": 1811, "rating_b": 1901, "gap": 90, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.35, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1500, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black (Black) played a Modern setup and traded well early, but 12...c6 and 14...O-O-O left the king exposed. 16...Kb8 was passive, and 19...f5 and 20...dxc5 let White's pieces dominate. 23...Rd4 and 24...Qc5 allowed 25.Be5 with a decisive attack. Black blundered into 28.Qg8+ by playing 26...Nxc4, which lost quickly. Game B Black (Black) played a Stonewall-like Dutch with 1...f5 and 8...d6. It was passive early, but Black defended tenaciously for 46 moves, with 22...e5 and 24...Bxd5, and survived into a long endgame. Black blundered on 33...Nc8 and 34.Ne7+, yet resisted longer than Game A's Black and kept a more coherent structure. Game A's collapse in 28 moves suggests weaker play, though both players are probably similar in strength.", "confidence_valid": true }, { "model": "google/gemini-3.8-flash", "pair": "pair_09", "game_a": "game_12", "color_a": "black", "game_b": "game_03", "color_b": "black", "rating_a": 1811, "rating_b": 1901, "gap": 90, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.35, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1500, "old_estimate_b": 1600, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black exhibits severe tactical blind spots typical of lower ratings. After falling for the elementary deflection tactic 10...Nxd4? 11.Nxd4 Bxd4 12.Bb5+!, Black orchestrated a dynamic counterattack but suffered complete tactical tunnel vision at the climax: Black missed a mate-in-two (27...Qxf2+ 28.Kh1 Qxg2#) and instead played 27...Nxe3??, walking straight into 28.Qg8# on the back rank. Game B Black played a coherent Dutch setup; although Black overlooked the tactical hook 15...Rd8? permitting 16.Bxd6, Black alertly spotted the fork threat and avoided immediate disaster with 16...Kf7. Overall, Game B Black demonstrates better strategic coherence and board awareness than Game A Black.", "confidence_valid": true }, { "model": "google/gemini-3.1-pro-preview", "pair": "pair_09", "game_a": "game_12", "color_a": "black", "game_b": "game_03", "color_b": "black", "rating_a": 1811, "rating_b": 1901, "gap": 90, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.15, "correct": 1, "brier": 0.0225, "old_estimate_a": 1800, "old_estimate_b": 1500, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Black in Game A blunders a full piece early to a basic tactic (12... c6 13. Qxd4) and later walks into a simple back-rank mate (27... Nxe3 28. Qg8+). Black in Game B displays a better positional understanding in the Dutch Defense, completing development and castling before breaking down to a complex knight maneuver by White. Although Black B also drops a piece with 16... Kf7 (missing 16... Rxd6), the overall opening play and resistance suggest a more experienced player than Black A's basic tactical blindness.", "confidence_valid": true }, { "model": "google/gemini-3.5-flash", "pair": "pair_09", "game_a": "game_12", "color_a": "black", "game_b": "game_03", "color_b": "black", "rating_a": 1811, "rating_b": 1901, "gap": 90, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.3, "correct": 1, "brier": 0.09, "old_estimate_a": 1250, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black exhibits weak tactical vision. They blunder a piece early to a simple two-ply combination (10... Nxd4? 11. Nxd4 Bxd4 12. Bb5+ and 13. Qxd4) and later succumb to a basic back-rank mate (26... Nxc4? 27. Bxh8 Nxe3 28. Qg8+). In contrast, Game B Black demonstrates superior opening knowledge and positional understanding in a complex Dutch Defense setup. Although Game B Black also makes a tactical oversight on move 15 (15... Rd8? allowing 16. Bxd6 with a Ne7+ fork threat), their overall strategic play and standard developmental plans suggest a significantly higher established rating than Game A Black.", "confidence_valid": true }, { "model": "google/gemini-3.7-flash", "pair": "pair_09", "game_a": "game_12", "color_a": "black", "game_b": "game_03", "color_b": "black", "rating_a": 1811, "rating_b": 1901, "gap": 90, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.22, "correct": 1, "brier": 0.0484, "old_estimate_a": 1550, "old_estimate_b": 1500, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A Black displayed low tactical awareness early on, blundering a full piece to a simple deflection motif on move 10 (10...Nxd4? 11.Nxd4 Bxd4 12.Bb5+ c6 13.Qxd4) and later allowing a direct back-rank mate with 27...Nxe3? 28.Qg8#. In contrast, Game B Black demonstrated a solid understanding of Dutch Defense structures (1...f5 through 7...O-O) and coordinated play. While Game B Black was outplayed by White's active knights starting around 15...Rd8, they showed better positional understanding, calculation, and board vision than Game A Black.", "confidence_valid": true }, { "model": "qwen/qwen3.8-max-prime", "pair": "pair_09", "game_a": "game_12", "color_a": "black", "game_b": "game_03", "color_b": "black", "rating_a": 1811, "rating_b": 1901, "gap": 90, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.35, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1250, "old_estimate_b": 1200, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A Black played dubious opening moves like 8...exd4 and 9...Qe7+, and blundered material at 25...Nxc4, but stayed competitively active until then. Game B Black committed severe strategic and tactical errors: 16...Kf7?? allowed 17.Ne5+ forking king and queen, 23...Nd7?? dropped the queen, 31...f4?? and 36...Rg7?? lost more, and ultimately got checkmated. B Black's repeated blunders and king exposure indicate a substantially weaker player than A Black.", "confidence_valid": true }, { "model": "anthropic/claude-fable-5.1", "pair": "pair_09", "game_a": "game_12", "color_a": "black", "game_b": "game_03", "color_b": "black", "rating_a": 1811, "rating_b": 1901, "gap": 90, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.42, "correct": 1, "brier": 0.17639999999999997, "old_estimate_a": 1300, "old_estimate_b": 1200, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A Black played a coherent Modern setup and won a pawn with 10...Nxd4, but then collapsed: 14...O-O-O into a half-open b/a file position, 19...f5 and 21...c4 lost material to 22.Bf4+ and 23.Bxc4, and 24...Qc5?? allowed 25.Be5 forking the rooks. Game B Black (Dutch) made positional errors (13...d5 allowing 14.Ne5, 15...Rd8? dropping the bishop to 16.Bxd6, 18...Re8 losing the exchange after Bxf6/Ng6) and lost badly, but the losses were gradual and the opponent played competently; Game A's White was also weak (5.a3, 17.b4), yet Black still fell for simple tactics. Both look low-rated; slight edge to B for longer resistance and more normal piece development.", "confidence_valid": true }, { "model": "google/gemini-3-flash-preview", "pair": "pair_09", "game_a": "game_12", "color_a": "black", "game_b": "game_03", "color_b": "black", "rating_a": 1811, "rating_b": 1901, "gap": 90, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.58, "correct": 0, "brier": 0.3364, "old_estimate_a": 1900, "old_estimate_b": 1700, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Player A (Black) shows better tactical resourcefulness and coordination despite losing a piece to a common discovery (12. Bb5+). Their counterplay, particularly with 23... Rd4 and 25... Rd2, forced White to find precise defensive moves. In contrast, Player B (Black) mismanaged the pressure in the Dutch Defense, committing a severe positional and safety error with 16... Kf7, which led to a passive and structural collapse. While both players suffered from tactical oversights (the back-rank mate in Game A and the exchange loss in Game B), Player A's ability to generate complications while down material indicates a slightly higher level of established strength compared to Player B's more passive deterioration.", "confidence_valid": true }, { "model": "google/gemini-3.6-flash", "pair": "pair_09", "game_a": "game_12", "color_a": "black", "game_b": "game_03", "color_b": "black", "rating_a": 1811, "rating_b": 1901, "gap": 90, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.65, "correct": 0, "brier": 0.42250000000000004, "old_estimate_a": 1400, "old_estimate_b": 1300, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Player A played a coherent opening (Modern Defense) and reached a reasonable position before falling victim to a two-move tactical trick with 10...Nxd4, missing White's 12. Bb5+ intermediate check which won the d4-bishop. In contrast, Player B displayed much worse tactical vision in Game B: after White played 16. Bxd6, Black played 16...Kf7??, failing to even recapture the bishop on d6 and giving away a full piece for free. Player B continued to make severe one-move blunders such as 22...e5, hanging the d5-pawn. Player A's mistake was a classic intermediate check oversight, while Player B demonstrated a lack of basic board awareness.", "confidence_valid": true }, { "model": "openai/gpt-5.6-sol", "pair": "pair_09", "game_a": "game_12", "color_a": "black", "game_b": "game_03", "color_b": "black", "rating_a": 1811, "rating_b": 1901, "gap": 90, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.78, "correct": 0, "brier": 0.6084, "old_estimate_a": 1600, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A Black showed more coherent development and active counterplay with 14...O-O-O, 21...c4, and the rook invasion 23...Rd4–25...Rd2. They did blunder a piece through 10...Nxd4?! 11.Nxd4 Bxd4 12.Bb5+ c6 13.Qxd4 and later missed 28.Qg8+. Game B Black’s tactical awareness looked substantially weaker: 15...Rd8 ignored the attack on f8, 16...Kf7 failed to address the lost bishop and rook, and 18...Re8 allowed 19.Bxf6. Further material losses followed with little compensation. Thus A is more likely to have had the higher established rapid strength.", "confidence_valid": true }, { "model": "openai/gpt-6-sol", "pair": "pair_09", "game_a": "game_12", "color_a": "black", "game_b": "game_03", "color_b": "black", "rating_a": 1811, "rating_b": 1901, "gap": 90, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.82, "correct": 0, "brier": 0.6723999999999999, "old_estimate_a": 1600, "old_estimate_b": 1600, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "Both Black players made serious tactical mistakes, but A showed more purposeful counterplay after losing material to White’s 12.Bb5+ and 13.Qxd4, using ...c5, ...c4 and ...Rd4 to create threats. B’s 16...Kf7 left White’s bishop on d6 unchallenged; it then took the knight with 19.Bxf6. Later, B allowed 34.Ne7+ to win a rook and 38.Rxg8+ to win the queen. A’s play looks flawed, but B’s repeated missed threats suggest lower established strength.", "confidence_valid": true }, { "model": "openai/gpt-6.1-sol", "pair": "pair_09", "game_a": "game_12", "color_a": "black", "game_b": "game_03", "color_b": "black", "rating_a": 1811, "rating_b": 1901, "gap": 90, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.66, "correct": 1, "brier": null, "old_estimate_a": 1600, "old_estimate_b": 1500, "baseline_choice": "A", "baseline_credit": 0, "explanation": "B looks somewhat stronger despite both losses. Game A Black’s 10...Nxd4 and 11...Bxd4 overlooked 12.Bb5+ followed by 13.Qxd4, losing a piece. Later, 27...Nxe3 allowed the decisive back-rank check 28.Qg8+. Game B Black showed more coherent central play with 11...c5 and 13...d5, then purposeful rook activity through 27...Rac8, 28...Rc4 and 29...Rg8. B’s 16...Kf7 and subsequent king exposure were serious weaknesses, but the overall development and counterplay suggest a somewhat higher established strength than A’s repeated tactical oversights.", "confidence_valid": false }, { "model": "openai/gpt-5.6-luna", "pair": "pair_09", "game_a": "game_12", "color_a": "black", "game_b": "game_03", "color_b": "black", "rating_a": 1811, "rating_b": 1901, "gap": 90, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.35, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1200, "old_estimate_b": 1350, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s Black showed somewhat more coherent development and defensive play: castling on move 7, central counterplay with ...c5 and ...d5, and active rook moves (...Rc4 and ...Rc7). However, Black still missed serious tactical issues, notably 16...Kf7 followed by 18...Re8 19.Bxf6, and was eventually mated after 38.Rxg8+. Game A’s Black appears weaker: 10...Nxd4 11.Nxd4 Bxd4 12.Bb5+ c6 13.Qxd4 loses two minor pieces for one, and 26...Nxc4 allows 27.Bxh8 to win a rook. Thus B is more likely to have had the higher established rapid strength, though the evidence is limited.", "confidence_valid": true }, { "model": "openai/gpt-6-astra", "pair": "pair_09", "game_a": "game_12", "color_a": "black", "game_b": "game_03", "color_b": "black", "rating_a": 1811, "rating_b": 1901, "gap": 90, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.86, "correct": 0, "brier": 0.7395999999999999, "old_estimate_a": 1650, "old_estimate_b": 1650, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "Game A Black showed more purposeful coordination: 15...c5, 17...Nd5, 18...Nb6 and 21...c4 generated queenside counterplay, followed by active rook penetration with 23...Rd4 and 25...Rd2. A still made serious tactical errors, notably 10...Nxd4 and allowing 28.Qg8+. Game B Black displayed more persistent weaknesses: 14...Bd6 left the bishop vulnerable to Bxd6, 16...Kf7–17...Kg8 wasted time while exposed, and 33...Nc8 allowed 34.Ne7+ followed by Nxg8. Both games contain substantial mistakes, but A’s coherent counterplay suggests the higher established strength.", "confidence_valid": true }, { "model": "z-ai/glm-5.3-flash", "pair": "pair_09", "game_a": "game_12", "color_a": "black", "game_b": "game_03", "color_b": "black", "rating_a": 1811, "rating_b": 1901, "gap": 90, "gap_band": "<200", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.7, "correct": 0, "brier": 0.48999999999999994, "old_estimate_a": 1500, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Black in A plays a coherent game: good tactics (8...exd4, 10...Nxd4 trading into a clean position), active queenside play with 15...O-O-O and 17...Nd5, and only succumbs to White's late mating attack (28.Qg8+) after material was even. Black in B handles the Stonewall structure passively: 9...Qc8 and 10...h6 misplace pieces, 15...Rd8/16...Kf7 mishandle the Bxd6 trade, and repeated tactical lapses lose material (22...e5 allowing 23.Nxd5, then 23...Nd7?? hanging pieces to 27.Qxc5). Black B's errors look much more amateur than Black A's solid play.", "confidence_valid": true } ], "a": { "game": "game_12", "color": "black", "rating": 1811, "url": "https://lichess.org/BdIeZ73n", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"1-0\"]\n[TimeControl \"600+0\"]\n\n1. d4 g6 2. Nf3 Bg7 3. e3 d6 4. Bd3 Nc6 5. a3 Bg4 6. Nbd2 e5 7. h3 Bxf3 8. Nxf3\nexd4 9. exd4 Qe7+ 10. Be3 Nxd4 11. Nxd4 Bxd4 12. Bb5+ c6 13. Qxd4 Nf6 14. Bd3\nO-O-O 15. O-O c5 16. Qa4 Kb8 17. b4 Nd5 18. Bd2 Nb6 19. Qb3 f5 20. bxc5 dxc5\n21. a4 c4 22. Bf4+ Ka8 23. Bxc4 Rd4 24. Rfe1 Qc5 25. Be5 Rd2 26. Re3 Nxc4 27.\nBxh8 Nxe3 28. Qg8+ 1-0" }, "b": { "game": "game_03", "color": "black", "rating": 1901, "url": "https://lichess.org/iSAnxtl2", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"1-0\"]\n[TimeControl \"600+0\"]\n\n1. d4 f5 2. Bf4 Nf6 3. e3 e6 4. Nd2 b6 5. Ngf3 Bb7 6. c3 Be7 7. Be2 O-O 8. h4\nd6 9. Ng5 Qc8 10. O-O h6 11. Nh3 c5 12. dxc5 bxc5 13. Nc4 d5 14. Ne5 Bd6 15.\nNg6 Rd8 16. Bxd6 Kf7 17. Ne5+ Kg8 18. Be7 Re8 19. Bxf6 gxf6 20. Ng6 Qd7 21. Bh5\nQg7 22. Nhf4 e5 23. Nxd5 Nd7 24. Qb3 Bxd5 25. Qxd5+ Kh7 26. Rfd1 Nb6 27. Qxc5\nRac8 28. Qb4 Rc4 29. Qb5 Rg8 30. b3 Rc7 31. c4 f4 32. exf4 exf4 33. Qf5 Nc8 34.\nNe7+ Kh8 35. Nxg8 Qxg8 36. Qxf6+ Rg7 37. Rd8 Ne7 38. Rxg8+ Nxg8 39. Qd4 Kh7 40.\nQxf4 Ne7 41. Re1 Ng6 42. Qe4 a5 43. Bxg6+ Rxg6 44. h5 Kh8 45. Qxg6 a4 46. Re8#\n1-0" }, "wrong": 6, "gap": 90 }, { "id": "pair_24", "predictions": [ { "model": "anthropic/claude-opus-5.5", "pair": "pair_24", "game_a": "game_03", "color_a": "white", "game_b": "game_15", "color_b": "black", "rating_a": 1782, "rating_b": 2154, "gap": 372, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.66, "correct": 0, "brier": 0.43560000000000004, "old_estimate_a": 1550, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "In Game A, White played a coherent London setup and punished Black's loosening play with clean tactics: 15.Ng6, 16.Bxd6, 23.Nxd5 winning material, and 34.Ne7+ with 35.Nxg8. White then converted smoothly, finishing with 43.Bxg6+ and 46.Re8#. In Game B, Black's play was error-prone. 15...Qd5 allowed White's knights to invade, and 24...Nxb7?? opened the c-file for 25.Qxc8+, dropping a rook to a simple tactic. Black only drew because White also misplayed. The king-and-pawn ending was also handled imprecisely, since the rook pawn let White's king reach the drawing corner. Overall, A's target looks more tactically reliable.", "confidence_valid": true }, { "model": "anthropic/claude-sonnet-5.5", "pair": "pair_24", "game_a": "game_03", "color_a": "white", "game_b": "game_15", "color_b": "black", "rating_a": 1782, "rating_b": 2154, "gap": 372, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.35, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1700, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White (Stronger? unclear): played a Stonewall-like Bf4 setup with an early h4 and Ng5. The play was aggressive but mostly sound, and Black's defence was loose (e.g. 9...Qc8, 20...Qd7, 23...Nd7 allowing Nxd5). White won with Nxd5 and Qxc5, then cleanly finished with Rd8 and Re8#. Overall this is a club-level performance with tactical alertness. Game B Black: handled a Slav and Queen's Gambit Accepted line with ...b5/...b4 accurately, and found ...Qd5 and ...Rxa4. Black navigated the tactical sequence (15.Ncxa5 through 26.Rxd5 exd5) and held an ending a pawn down against a stronger-looking White; the 41...Kxc5 pawn race was precise, and the draw was secure. Black's play was more accurate and consistent against a competent opponent, so B is slightly more likely to be stronger.", "confidence_valid": true }, { "model": "google/gemini-3.8-flash", "pair": "pair_24", "game_a": "game_03", "color_a": "white", "game_b": "game_15", "color_b": "black", "rating_a": 1782, "rating_b": 2154, "gap": 372, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.22, "correct": 1, "brier": 0.0484, "old_estimate_a": 1800, "old_estimate_b": 2200, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Player B exhibits substantially higher chess understanding and tactical refinement. In an advanced Slav/a4-b4 line, Black handled Queen's-side piece coordination cleanly and executed the tactical liquidation 24...Nxb7! 25.Qxc8+ Nxc8 26.Rxd5 exd5, transitioning into a slightly worse but holdable endgame that was drawn with textbook king activity and pawn-play technique. In contrast, White in Game A plays an aggressive early h4/Ng5 London attack against a severely underperforming opponent who collapsed with 15...Rd8? and 16...Kf7?. While White efficiently converted the material advantage, the level of opposition and strategic complexity in Game B reflects a significantly higher established rating.", "confidence_valid": true }, { "model": "google/gemini-3.1-pro-preview", "pair": "pair_24", "game_a": "game_03", "color_a": "white", "game_b": "game_15", "color_b": "black", "rating_a": 1782, "rating_b": 2154, "gap": 372, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.35, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1800, "old_estimate_b": 2250, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Target B (Black) demonstrates deep positional and tactical understanding in the Slav Defense, navigating a highly complex sequence from moves 22-26 that results in an equal endgame. The thematic 22...c5 break and the ensuing mass exchanges (24...Nxb7 25. Qxc8+ Nxc8 26. Rxd5 exd5) show precise calculation. Target A (White) plays a decent attacking game in the Dutch, punishing Black's blunders (such as 15...Rd8? allowing 16. Bxd6), but early moves like 8.h4 and 11.Nh3 are somewhat committal and less refined than B's play. B's ability to maintain equality in a sharp theoretical variation suggests a higher rating.", "confidence_valid": true }, { "model": "google/gemini-3.5-flash", "pair": "pair_24", "game_a": "game_03", "color_a": "white", "game_b": "game_15", "color_b": "black", "rating_a": 1782, "rating_b": 2154, "gap": 372, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.15, "correct": 1, "brier": 0.0225, "old_estimate_a": 1850, "old_estimate_b": 2000, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B (Black) demonstrates a much higher level of chess understanding and technical skill. Black navigates a sharp, theoretical Slav Defense (3...dxc4, 5...b4), coordinates pieces accurately in a queenless middlegame (15...Qd5, 16...Rxa4), and calculates a pawn-down minor piece endgame into a precise drawn king-and-pawn endgame (39...Bxc5, 48...Kd4). In contrast, Game A (White) plays a superficial opening with early flank attacks (8. h4, 11. Nh3) and wins primarily due to Black's major tactical blunder (16...Kf7?? allowing 17. Ne5+). Game B's positional accuracy and endgame precision suggest a significantly higher established rating.", "confidence_valid": true }, { "model": "google/gemini-3.7-flash", "pair": "pair_24", "game_a": "game_03", "color_a": "white", "game_b": "game_15", "color_b": "black", "rating_a": 1782, "rating_b": 2154, "gap": 372, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.22, "correct": 1, "brier": 0.0484, "old_estimate_a": 1700, "old_estimate_b": 2150, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B demonstrates much higher-level chess understanding and precision from both players. Target B navigates a complex Slav structure smoothly, executing accurate piece coordination (14...Nb6, 15...Qd5) and sharp liquidating tactics (22...c5! through 26...exd5). Target B then displays flawless endgame calculation to secure a pawn-ending draw (42...d4!, 47...d3+). In contrast, Game A features cruder opening play with an aggressive flank thrust (8. h4, 9. Ng5), where White capitalized easily on Black's severe tactical blunders (such as 16...Kf7 and passive piece placement) without facing substantial resistance.", "confidence_valid": true }, { "model": "qwen/qwen3.8-max-prime", "pair": "pair_24", "game_a": "game_03", "color_a": "white", "game_b": "game_15", "color_b": "black", "rating_a": 1782, "rating_b": 2154, "gap": 372, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.72, "correct": 0, "brier": 0.5184, "old_estimate_a": 1500, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White shows strong tactical awareness: 16.Bxd6 wins material, 19.Bxf6 shatters Black's kingside, and 23.Nxd5 exploits the pin, converting to a clean win. Game B Black plays the Slav solidly but drifts passively; after 15...Qd5 and 16...Rxa4, Black trades into a simplified endgame (24...Nxb7 25.Qxc8+) without generating winning chances, ultimately drawing a king-and-pawn race. White's active piece coordination and decisive finishing in Game A suggest a higher established rating than Black's competent but reactive play in Game B.", "confidence_valid": true }, { "model": "anthropic/claude-fable-5.1", "pair": "pair_24", "game_a": "game_03", "color_a": "white", "game_b": "game_15", "color_b": "black", "rating_a": 1782, "rating_b": 2154, "gap": 372, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.4, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 1450, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White plays a loose London setup with speculative 8.h4 and 9.Ng5/11.Nh3 shuffling, then 14.Ne5/15.Ng6 tactics work only because Black (Game A) blundered with 14...Bd6?? and 16...Kf7; White's conversion (24.Qb3, 37.Rd8) is competent but the opponent's level was low. Game B Black handles a theoretical Slav line (3...dxc4, 4...b5, 8...a5, 11...Ba6), keeps material balance through complex trades (15...Qd5, 16...Rxa4, 24...Nxb7 25.Qxc8+ Nxc8), and plays a long, accurate king-and-pawn endgame, though 56...h4?? spoiled a win with ...Kf2/...h3 ideas earlier. Overall Black in B shows more structured, sound chess against a stronger opponent.", "confidence_valid": true }, { "model": "google/gemini-3-flash-preview", "pair": "pair_24", "game_a": "game_03", "color_a": "white", "game_b": "game_15", "color_b": "black", "rating_a": 1782, "rating_b": 2154, "gap": 372, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.2, "correct": 1, "brier": 0.04000000000000001, "old_estimate_a": 1900, "old_estimate_b": 2150, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Target B shows superior technical skill, navigating a complex Slav Defense mainline (4... b5, 5... b4). Black's maneuvering (14... Nb6) and the thematic 22... c5 break indicate deep theoretical knowledge. The tactical sequence 24... Nxb7 and the flawless transition to a drawn pawn endgame (39... Bxc5, 48... Kd4) suggest a high rating. Conversely, Target A wins against a weak Dutch Defense (1... f5) where Black blunders early (9... Qc8, 15... Rd8). While Target A efficiently capitalizes with 15. Ng6 and 24. Qb3, the overall quality of opposition and the lack of complex positional requirements in Game A make Target B the likely stronger player.", "confidence_valid": true }, { "model": "google/gemini-3.6-flash", "pair": "pair_24", "game_a": "game_03", "color_a": "white", "game_b": "game_15", "color_b": "black", "rating_a": 1782, "rating_b": 2154, "gap": 372, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.35, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1550, "old_estimate_b": 2200, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B's target (Black) demonstrates high positional understanding and theoretical fluency in a sharp Slav Defense line. Black executes the tactical break 22...c5! smoothly, leading to a forcing sequence (24...Nxb7! 25.Qxc8+ Nxc8 26.Rxd5 exd5) that transforms into a balanced endgame, which Black defends flawlessly to a draw. In contrast, Game A's target (White) plays a somewhat premature early flank advance with 8. h4 and 11. Nh3, though White effectively capitalizes on Black's major blunder (16...Kf7??). Overall, Black in Game B shows superior overall chess culture, calculation depth, and endgame technique.", "confidence_valid": true }, { "model": "openai/gpt-5.6-sol", "pair": "pair_24", "game_a": "game_03", "color_a": "white", "game_b": "game_15", "color_b": "black", "rating_a": 1782, "rating_b": 2154, "gap": 372, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.2, "correct": 1, "brier": 0.04000000000000001, "old_estimate_a": 1800, "old_estimate_b": 2050, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s Black showed stronger positional and tactical control. The sequence 24...Nxb7! 25.Qxc8+ Nxc8 26.Rxd5 exd5 deliberately simplified into a sound endgame, followed by active king play with 32...Ke8, 33...Kd7 and the well-timed 42...d4. Game A’s White spotted several clear opportunities—16.Bxd6, 23.Nxd5 and 34.Ne7+—and converted decisively, but these arose largely from Black’s loose play, especially 14...Bd6, 15...Rd8 and the exposed king. White A’s early 8.h4 and repeated knight maneuvers were effective but less convincing as evidence of established strength than Black B’s sustained, coherent play.", "confidence_valid": true }, { "model": "openai/gpt-6-sol", "pair": "pair_24", "game_a": "game_03", "color_a": "white", "game_b": "game_15", "color_b": "black", "rating_a": 1782, "rating_b": 2154, "gap": 372, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.34, "correct": 1, "brier": 0.11560000000000002, "old_estimate_a": 1800, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s Black shows more sustained positional control: 16...Rxa4 finds counterplay, 21...Ne4 and 22...c5 challenge White’s center, and 39...Bxc5 leads to an active king-and-pawn ending. Game A’s White exploits Black’s errors well, notably with 17.Ne5+ and 34.Ne7+, but the early 8.h4, 9.Ng5, 11.Nh3 sequence spends several tempi on the same knight. A’s tactical win is impressive, but B’s play against a more resistant opponent is slightly stronger evidence of established strength.", "confidence_valid": true }, { "model": "openai/gpt-6.1-sol", "pair": "pair_24", "game_a": "game_03", "color_a": "white", "game_b": "game_15", "color_b": "black", "rating_a": 1782, "rating_b": 2154, "gap": 372, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.61, "correct": 0, "brier": 0.3721, "old_estimate_a": 1800, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White showed sustained tactical coordination: 16.Bxd6 exploited the potential Ne7+ queen fork, while 34.Ne7+ followed by 35.Nxg8 won material. White then used 37.Rd8 and finished accurately with 46.Re8#. Game B Black demonstrated competence with the liquidation 24...Nxb7 25...Nxc8 26...exd5 and active king play. However, 44...gxh4 and 45...h5 simplified the remaining chances into an ultimately drawn rook-pawn ending. A’s stronger tactical execution gives White a modest edge, tempered by Game A Black’s weaker resistance.", "confidence_valid": true }, { "model": "openai/gpt-5.6-luna", "pair": "pair_24", "game_a": "game_03", "color_a": "white", "game_b": "game_15", "color_b": "black", "rating_a": 1782, "rating_b": 2154, "gap": 372, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.55, "correct": 0, "brier": 0.30250000000000005, "old_estimate_a": 1750, "old_estimate_b": 1750, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "Game A’s white showed more convincing tactical initiative and conversion: 15.Ng6, 18.Be7, and 19.Bxf6 created lasting pressure, followed by 23.Nxd5 and 27.Qxc5. The attack was converted accurately with 35.Nxg8, 38.Rxg8+, and the mating finish 46.Re8#. Game B’s black played solidly and found a useful simplification with 24...Nxb7, 25...Nxc8, and 26...exd5, then handled the pawn ending competently. However, that game provides mostly evidence of resilience and endgame technique rather than the stronger player creating winning chances. Thus A is favored, though only slightly because Game A’s opponent made substantial defensive errors.", "confidence_valid": true }, { "model": "openai/gpt-6-astra", "pair": "pair_24", "game_a": "game_03", "color_a": "white", "game_b": "game_15", "color_b": "black", "rating_a": 1782, "rating_b": 2154, "gap": 372, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.59, "correct": 0, "brier": 0.34809999999999997, "old_estimate_a": 1850, "old_estimate_b": 1850, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "A’s White showed sustained tactical awareness: 16.Bxd6 exploited the potential Ne7+ queen fork, 23.Nxd5 increased central pressure, and 34.Ne7+ followed by 37.Rd8 decisively converted the attack. The early h4–Ng5–Nh3 maneuver was less convincing. B’s Black demonstrated good coordination with 22...c5 and the liquidation beginning 24...Nxb7, then activated the king effectively. However, 47...d3+ and the subsequent exchanges allowed White’s e5 resource and a drawn rook-pawn ending. A’s cleaner tactical execution gives a slight edge, tempered by the weaker defensive resistance in Game A.", "confidence_valid": true }, { "model": "z-ai/glm-5.3-flash", "pair": "pair_24", "game_a": "game_03", "color_a": "white", "game_b": "game_15", "color_b": "black", "rating_a": 1782, "rating_b": 2154, "gap": 372, "gap_band": "200-499", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.6, "correct": 1, "brier": null, "old_estimate_a": 1650, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B's Black navigated a tricky Semi-Slav line (15...Qd5, 16...Rxa4 counting out the knight fork ideas), simplified correctly (24...Nxb7, 25...Nxc8), and reached a dead-drawn pawn endgame — solid, theory-aware play with clean technique (42...d4, 47...d3+ showing accuracy). Game A's White won but with less convincing chess: 8.h4 and 9.Ng5 are crude attacking setups, though 21.Bh5 and the mating net (34.Ne7+, 45.Qxg6) show tactical alertness. Black's opposition in A was notably passive (10...h6, 12...bxc5 opening lines against himself), inflating A's result. B's clean handling of concrete lines suggests the higher established strength.", "confidence_valid": false } ], "a": { "game": "game_03", "color": "white", "rating": 1782, "url": "https://lichess.org/iSAnxtl2", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"1-0\"]\n[TimeControl \"600+0\"]\n\n1. d4 f5 2. Bf4 Nf6 3. e3 e6 4. Nd2 b6 5. Ngf3 Bb7 6. c3 Be7 7. Be2 O-O 8. h4\nd6 9. Ng5 Qc8 10. O-O h6 11. Nh3 c5 12. dxc5 bxc5 13. Nc4 d5 14. Ne5 Bd6 15.\nNg6 Rd8 16. Bxd6 Kf7 17. Ne5+ Kg8 18. Be7 Re8 19. Bxf6 gxf6 20. Ng6 Qd7 21. Bh5\nQg7 22. Nhf4 e5 23. Nxd5 Nd7 24. Qb3 Bxd5 25. Qxd5+ Kh7 26. Rfd1 Nb6 27. Qxc5\nRac8 28. Qb4 Rc4 29. Qb5 Rg8 30. b3 Rc7 31. c4 f4 32. exf4 exf4 33. Qf5 Nc8 34.\nNe7+ Kh8 35. Nxg8 Qxg8 36. Qxf6+ Rg7 37. Rd8 Ne7 38. Rxg8+ Nxg8 39. Qd4 Kh7 40.\nQxf4 Ne7 41. Re1 Ng6 42. Qe4 a5 43. Bxg6+ Rxg6 44. h5 Kh8 45. Qxg6 a4 46. Re8#\n1-0" }, "b": { "game": "game_15", "color": "black", "rating": 2154, "url": "https://lichess.org/aBhzitCG", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"1/2-1/2\"]\n[TimeControl \"600+0\"]\n\n1. d4 d5 2. c4 c6 3. Nc3 dxc4 4. e3 b5 5. a4 b4 6. Nb1 Nf6 7. Bxc4 e6 8. Nf3 a5\n9. Nbd2 Be7 10. O-O O-O 11. Nb3 Ba6 12. Nfd2 Nbd7 13. Bxa6 Rxa6 14. Nc4 Nb6 15.\nNcxa5 Qd5 16. Nb7 Rxa4 17. Rxa4 Nxa4 18. N3a5 Nb6 19. Qc2 Rc8 20. Bd2 h6 21.\nRc1 Ne4 22. Be1 c5 23. dxc5 Nxc5 24. Rd1 Nxb7 25. Qxc8+ Nxc8 26. Rxd5 exd5 27.\nNxb7 Nb6 28. b3 Kf8 29. Na5 Nd7 30. Nc6 Nc5 31. Bxb4 Bd6 32. Kf1 Ke8 33. Ke2\nKd7 34. Bxc5 Bxc5 35. Ne5+ Ke6 36. Nd3 Bd6 37. g3 g5 38. b4 Kd7 39. Nc5+ Bxc5\n40. bxc5 Kc6 41. Kd3 Kxc5 42. e4 d4 43. f4 f6 44. h4 gxh4 45. gxh4 h5 46. Kd2\nKc4 47. Ke2 d3+ 48. Kd2 Kd4 49. e5 fxe5 50. fxe5 Kxe5 51. Kxd3 Kf4 52. Ke2 Kg3\n53. Kf1 Kxh4 54. Kg1 Kh3 55. Kh1 Kg3 56. Kg1 h4 57. Kh1 Kf2 58. Kh2 1/2-1/2" }, "wrong": 5, "gap": 372 }, { "id": "pair_19", "predictions": [ { "model": "anthropic/claude-opus-5.5", "pair": "pair_19", "game_a": "game_17", "color_a": "white", "game_b": "game_10", "color_b": "white", "rating_a": 1514, "rating_b": 2689, "gap": 1175, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.45, "correct": 1, "brier": 0.2025, "old_estimate_a": 1550, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A (White) played a sound but simple Italian. However, 11.g4 loosened the kingside, and the win came abruptly after 24.Rd5 c6 (likely a flag or resignation), not from clear technique. Game B (White) handled a double-fianchetto English/Catalan structure competently: 12.dxc5, 13.Nd4 and 16.Nxe6 created targets, and 20.Nb5 with 24.Nxa7 won material. White then erred with 25.Rf1? allowing ...Ba6 to win the exchange, and the queen wanderings (Qf4-h4-f4) were imprecise. Game B's opening and middlegame understanding look somewhat more sophisticated, though both players show errors. The evidence is close, so this is a slight lean toward B.", "confidence_valid": true }, { "model": "anthropic/claude-sonnet-5.5", "pair": "pair_19", "game_a": "game_17", "color_a": "white", "game_b": "game_10", "color_b": "white", "rating_a": 1514, "rating_b": 2689, "gap": 1175, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.35, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1650, "old_estimate_b": 1700, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White played an Italian-style opening but allowed 11...Bg6 and 13...Nf4 trades. The 24 moves are not very accurate: 11.g4 loosened the king, and 15.Kg2 left it exposed. White then won against a weak Black who dropped material (24...c6 ended the game, probably after a blunder). Black's play (18...g5, 22...Ne5) was loose, suggesting a modest opponent. Game B White played a coherent English with 3.g3, Bb2, Rc1, and 13.Nd4, so the structure and development look sound. But 20.Nb5 and 24.Nxa7 grabbed a pawn and left White's pieces offside and exposed to 24...Qc2 and 26...Bxf1, and White lost. Black in Game B played well and punished White's mistakes, which suggests a stronger opponent. Overall B's play was more theoretically sound but A's opponent was weaker. I lean slightly to B.", "confidence_valid": true }, { "model": "google/gemini-3.8-flash", "pair": "pair_19", "game_a": "game_17", "color_a": "white", "game_b": "game_10", "color_b": "white", "rating_a": 1514, "rating_b": 2689, "gap": 1175, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.23, "correct": 1, "brier": 0.0529, "old_estimate_a": 1700, "old_estimate_b": 2050, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B exhibits high-level strategic comprehension from both sides in a sophisticated English Opening. White coordinates pieces harmoniously with 11. Rc1, 13. Nd4, and 17. Rfd1, maintaining strong pressure until a tactical oversight with 24. Nxa7? allowed 24...Qc2!. In contrast, Game A features more rudimentary play: White unnecessarily weakens the kingside with 11. g4 and 12. Neg5, misplacing pieces and leaving holes that Black could have exploited. Even though White won Game A (likely by an early resignation or time-out on move 24), Player B's opening sophistication and positional maneuvering reflect a considerably higher established rating.", "confidence_valid": true }, { "model": "google/gemini-3.1-pro-preview", "pair": "pair_19", "game_a": "game_17", "color_a": "white", "game_b": "game_10", "color_b": "white", "rating_a": 1514, "rating_b": 2689, "gap": 1175, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.2, "correct": 1, "brier": 0.04000000000000001, "old_estimate_a": 1800, "old_estimate_b": 2000, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B White demonstrates a sophisticated understanding of positional play in a Reti/Catalan system, developing harmoniously and maneuvering well (13. Nd4, 15. Qf4) before a tactical oversight at the end (24. Nxa7 allowing Qc2). In contrast, Game A White plays a less refined opening, pointlessly wasting tempi (12. Neg5 h6 13. Ne4) and unnecessarily weakening the kingside with 11. g4 to break a pin. Therefore, Game B White likely has a higher rating despite losing.", "confidence_valid": true }, { "model": "google/gemini-3.5-flash", "pair": "pair_19", "game_a": "game_17", "color_a": "white", "game_b": "game_10", "color_b": "white", "rating_a": 1514, "rating_b": 2689, "gap": 1175, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.58, "correct": 0, "brier": 0.3364, "old_estimate_a": 1700, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White played a solid, blunder-free game. While the opening was relatively simple, White maintained active pieces, coordinated well, and capitalized on Black's late-game errors to secure a win. In contrast, Game B White displayed good positional understanding in an English Opening but suffered a severe tactical collapse. 24. Nxa7?? was a decisive blunder that overlooked 24...Qc2, which forks the b2-bishop and f1-rook and immediately loses the game. Player A's tactical consistency and lack of critical errors suggest a more reliable established playing strength.", "confidence_valid": true }, { "model": "google/gemini-3.7-flash", "pair": "pair_19", "game_a": "game_17", "color_a": "white", "game_b": "game_10", "color_b": "white", "rating_a": 1514, "rating_b": 2689, "gap": 1175, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.32, "correct": 1, "brier": 0.1024, "old_estimate_a": 1750, "old_estimate_b": 2050, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B features much deeper opening understanding and strategic sophistication in a standard English/Reti structure (1. c4 through 18. Qh4) before White tactical slip with 24. Nxa7? allowing 24...Qc2. In contrast, Game A displays significantly lower-tier play, with White playing crude pawn-pushing and awkwardly placed knights (11. g4, 12. Neg5 h6 13. Ne4), facing an opponent who resigned prematurely after 24. Rd5. The overall structural and positional competence in Game B strongly points to a higher established rating.", "confidence_valid": true }, { "model": "qwen/qwen3.8-max-prime", "pair": "pair_19", "game_a": "game_17", "color_a": "white", "game_b": "game_10", "color_b": "white", "rating_a": 1514, "rating_b": 2689, "gap": 1175, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.32, "correct": 1, "brier": 0.1024, "old_estimate_a": 1250, "old_estimate_b": 1200, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A's White played solid, accurate moves—central control, timely exchanges (16...Rxe4 20...Rxe4), and a clean rook lift ending with 24.Rd5, earning a win with minimal errors. Game B's White collapsed badly: 24.Nxa7?? hangs material and structure, allowing 24...Qc2, then 26.Bxf1?? loses to 26...Ne8, resigning immediately. Despite losing, B's opening (Catalan setup, 12...Nxc5, 15...Ne6) showed higher-level strategic understanding; the blunder was time-pressure induced. A won but with less demanding play and a simpler structure.", "confidence_valid": true }, { "model": "anthropic/claude-fable-5.1", "pair": "pair_19", "game_a": "game_17", "color_a": "white", "game_b": "game_10", "color_b": "white", "rating_a": 1514, "rating_b": 2689, "gap": 1175, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.38, "correct": 1, "brier": 0.1444, "old_estimate_a": 1500, "old_estimate_b": 1500, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "Game A White played 2.Bc4 then 6.a3 and the aimless 10.h3/11.g4/12.Neg5/13.Ne4 knight shuffle, and 24.Rd5 (Black can take with ...cxd5) looks like a blunder; the win came from Black resigning abruptly, not strong play. Game B White handled a Catalan/Reti structure with coherent plans (8.cxd5, 12.dxc5, 13.Nd4, 15.Qf4 pressuring d5), only collapsing after 23.Qc7?/24.Nxa7 allowed 24...Qc2 and 25...Ba6 winning the exchange. Game B's positional understanding and opponent quality suggest a higher rating despite the loss.", "confidence_valid": true }, { "model": "google/gemini-3-flash-preview", "pair": "pair_19", "game_a": "game_17", "color_a": "white", "game_b": "game_10", "color_b": "white", "rating_a": 1514, "rating_b": 2689, "gap": 1175, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.35, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1850, "old_estimate_b": 2150, "baseline_choice": "B", "baseline_credit": 1, "explanation": "White B demonstrates sophisticated positional play in an English/Catalan structure (1. c4, 3. g3, 9. d4), showing high-level opening knowledge. The coordination and maneuvering (13. Nd4, 15. Qf4, 20. Nb5) suggest a strong established rating. White A plays a standard Italian with a somewhat aggressive but loosening kingside expansion (11. g4). While White A capitalized on Black's late blunder (24... c6) to win, the strategic depth in Game B is more characteristic of a higher-rated player. Even with White B's tactical oversight (24. Nxa7), the overall quality of move selection in the first 20 moves points towards B having a higher established Lichess rapid rating.", "confidence_valid": true }, { "model": "google/gemini-3.6-flash", "pair": "pair_19", "game_a": "game_17", "color_a": "white", "game_b": "game_10", "color_b": "white", "rating_a": 1514, "rating_b": 2689, "gap": 1175, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.65, "correct": 0, "brier": 0.42250000000000004, "old_estimate_a": 1700, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Player A played a solid, clean positional game without major mistakes, smoothly consolidating after 11. g4 with 15. Kg2 and maintaining strong central/d-file control (21. dxe4, 23. Rd1, 24. Rd5), ultimately forcing Black to resign in a lost position. In contrast, Player B built a nice English opening setup but suffered a catastrophic tactical oversight with 24. Nxa7??, missing Black's double attack 24...Qc2! which attacked both the bishop on b2 and the rook on f1. Player B failed to defend the tactics, losing material rapidly (26...Bxf1) and collapsing immediately.", "confidence_valid": true }, { "model": "openai/gpt-5.6-sol", "pair": "pair_19", "game_a": "game_17", "color_a": "white", "game_b": "game_10", "color_b": "white", "rating_a": 1514, "rating_b": 2689, "gap": 1175, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.35, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1750, "old_estimate_b": 1750, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "Game B White showed stronger established opening and positional habits: 6.b3/7.Bb2, 11.Rc1, 14.Qd2, and 17.Rfd1 formed a coherent setup, while 20.Nb5 created active pressure. The decisive error was concrete but isolated: after saving the rook from 24...Qc2 with 25.Rf1, White overlooked 25...Ba6 and allowed 26...Bxf1. Game A White remained tactically sound, but the plan with 6.a3, 11.g4, and the 12.Neg5–13.Ne4 retreat was comparatively loose and inefficient. Game A’s unexplained finish after 24...c6 also provides little evidence of conversion skill. Thus B looks likelier to have had the higher established rapid strength despite losing.", "confidence_valid": true }, { "model": "openai/gpt-6-sol", "pair": "pair_19", "game_a": "game_17", "color_a": "white", "game_b": "game_10", "color_b": "white", "rating_a": 1514, "rating_b": 2689, "gap": 1175, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.18, "correct": 1, "brier": 0.0324, "old_estimate_a": 1550, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s White shows more coherent development and planning: 7.Bb2, 11.Rc1, 17.Rfd1 and 22.Rxc8. The later 25.Rf1? allowed 25...Ba6 and 26...Bxf1, but that oversight does not erase the stronger earlier play. Game A’s White spent several moves retreating or repositioning the same pieces—12.Neg5 13.Ne4 and 19.Bb3 20.Bc2—after weakening the kingside with 11.g4. Game A’s result alone is not persuasive evidence of greater established strength.", "confidence_valid": true }, { "model": "openai/gpt-6.1-sol", "pair": "pair_19", "game_a": "game_17", "color_a": "white", "game_b": "game_10", "color_b": "white", "rating_a": 1514, "rating_b": 2689, "gap": 1175, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.61, "correct": 1, "brier": null, "old_estimate_a": 1700, "old_estimate_b": 1900, "baseline_choice": "B", "baseline_credit": 1, "explanation": "B White showed more coherent opening and positional play: 10.Nc3, 11.Rc1, 13.Nd4 and 17.Rfd1 coordinated development and central pressure. However, 24.Nxa7 followed by 26.Be5 overlooked Black’s ...Ba6–...Bxf1 tactic, losing the exchange. A White’s 11.g4 weakened the king, while 12.Neg5 h6 13.Ne4 wasted tempi; 15.Kg2 compounded the awkward king placement. A subsequently played reasonably with 18.c3 and 24.Rd5, but the recorded win after ...c6 does not demonstrate a decisive advantage. B’s stronger positional foundation narrowly outweighs the tactical lapse.", "confidence_valid": false }, { "model": "openai/gpt-5.6-luna", "pair": "pair_19", "game_a": "game_17", "color_a": "white", "game_b": "game_10", "color_b": "white", "rating_a": 1514, "rating_b": 2689, "gap": 1175, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.64, "correct": 0, "brier": 0.4096, "old_estimate_a": 1500, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A White played more forcefully and tactically, with 11.g4, 14.Bxf4, and the active 16.Re1 and 20.Bc2. The sequence 20...Rxe4 21.dxe4 kept the initiative, and Black resigned after 24...c6. Game B White showed reasonable positional development, but the tactical sequence 25...Ba6 26.Be5? Bxf1 allowed Black to take the rook, with only 27.Bxf1 recovering the bishop. That substantial one-move oversight makes Game B White look less established, despite the otherwise coherent opening and middlegame.", "confidence_valid": true }, { "model": "openai/gpt-6-astra", "pair": "pair_19", "game_a": "game_17", "color_a": "white", "game_b": "game_10", "color_b": "white", "rating_a": 1514, "rating_b": 2689, "gap": 1175, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.36, "correct": 1, "brier": 0.1296, "old_estimate_a": 1750, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "B’s White showed more coherent positional development: 9.d4, 11.Rc1, 13.Nd4 and 17.Rfd1 coordinated the pieces and contested the center. The later 24.Nxa7 and especially 26.Be5, allowing ...Bxf1, were serious tactical lapses. A’s White avoided comparable material losses, but 6.a3 and the sequence 11.g4, 12.Neg5, 13.Ne4 spent time and weakened the kingside without a clear payoff. A’s 23.Rd1 and 24.Rd5 were purposeful, though the unexplained ending after 24...c6 supplies no evidence of a winning position. B’s stronger opening organization narrowly outweighs the later mistakes.", "confidence_valid": true }, { "model": "z-ai/glm-5.3-flash", "pair": "pair_19", "game_a": "game_17", "color_a": "white", "game_b": "game_10", "color_b": "white", "rating_a": 1514, "rating_b": 2689, "gap": 1175, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.4, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 1800, "old_estimate_b": 1600, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game B's White plays a coherent English/Reversed-Sicilian structure, trades well, and only loses after 24.Nxa7 grabs a poisoned pawn into Black's organized counterplay (24...Qc2, 25...Ba6, 26...Bxf1), suggesting Black was genuinely strong rather than White weak. Game A's White wins mainly because Black self-destructs: 17...Qf6 allows Rxe4 pressure, 18...g5 weakens the king, and 24...c6 simply drops the pawn/position. A's own play (12.Neg5 then retreating 13.Ne4, passive 21.Qe2) is unremarkable. B's positional handling looks stronger overall.", "confidence_valid": true } ], "a": { "game": "game_17", "color": "white", "rating": 1514, "url": "https://lichess.org/KH0akECy", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"1-0\"]\n[TimeControl \"600+0\"]\n\n1. e4 e5 2. Bc4 Nf6 3. Nf3 Bc5 4. d3 O-O 5. O-O Nc6 6. a3 d5 7. exd5 Nxd5 8.\nNbd2 Bg4 9. Ne4 Bb6 10. h3 Bh5 11. g4 Bg6 12. Neg5 h6 13. Ne4 Nf4 14. Bxf4 exf4\n15. Kg2 Qe7 16. Re1 Bxe4 17. Rxe4 Qf6 18. c3 g5 19. Bb3 Rfe8 20. Bc2 Rxe4 21.\ndxe4 Rd8 22. Qe2 Ne5 23. Rd1 Rd6 24. Rd5 c6 1-0" }, "b": { "game": "game_10", "color": "white", "rating": 2689, "url": "https://lichess.org/ghN8Hbdl", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"0-1\"]\n[TimeControl \"600+0\"]\n\n1. c4 e6 2. Nf3 d5 3. g3 Nf6 4. Bg2 Nbd7 5. O-O Be7 6. b3 O-O 7. Bb2 c5 8. cxd5\nexd5 9. d4 b6 10. Nc3 Bb7 11. Rc1 Rc8 12. dxc5 Nxc5 13. Nd4 Qd7 14. Qd2 Rfe8\n15. Qf4 Ne6 16. Nxe6 fxe6 17. Rfd1 Bd6 18. Qh4 Rf8 19. e3 Qf7 20. Nb5 Be7 21.\nQf4 Qg6 22. Rxc8 Bxc8 23. Qc7 Bc5 24. Nxa7 Qc2 25. Rf1 Ba6 26. Be5 Bxf1 27.\nBxf1 Ne8 0-1" }, "wrong": 3, "gap": 1175 }, { "id": "pair_12", "predictions": [ { "model": "anthropic/claude-opus-5.5", "pair": "pair_12", "game_a": "game_05", "color_a": "black", "game_b": "game_06", "color_b": "black", "rating_a": 1867, "rating_b": 1382, "gap": 485, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.58, "correct": 1, "brier": 0.17640000000000003, "old_estimate_a": 1400, "old_estimate_b": 1450, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game A Black played purposeful moves: the knight raid ...Nf4, ...Nxg2+ winning material, ...fxe4, then a coordinated attack with ...Nbd3, ...Bxh1, ...Nh3+ and a clean finish with ...Qxf3#. The opponent was weak (Bc4-d3-e2 wandering, 16.Be4 allowing Nxg2+). Game B Black played more passively and loosely, with odd bishop shuffles (...Bd7-f5-d7-g4-e6) and 16...Bxg5 giving up the bishop. Black still won material via ...Ncxb3 and ...Nxa1 and later mated with ...Qf2#, but White blundered heavily (Rf1?). Both targets exploited weak opponents, but Game A's attack looked more coherent, so A is slightly favoured.", "confidence_valid": true }, { "model": "anthropic/claude-sonnet-5.5", "pair": "pair_12", "game_a": "game_05", "color_a": "black", "game_b": "game_06", "color_b": "black", "rating_a": 1867, "rating_b": 1382, "gap": 485, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.4, "correct": 0, "brier": 0.36, "old_estimate_a": 1600, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game A (Black): White played poorly, with 10.e5 and 11.Be4 allowing 11...f5, then 15.c4 and 16.Be4. Black found 16...Nxg2+ and later 23...Bxh1, 25...Nh3+ and 29...Qxf3#, a strong attack. But White's opening and defense were weak, so the opponent looks low-rated, and Black's 2...e6 and 4...b5 are standard Sicilian-gambit-like play at club level. Game B (Black): Black played a solid King's Indian/Philidor setup and took the chance after 13.Ng5 with 13...Nd4, 16...Bxg5 and 18...Ncxb3, winning material. White blundered with 14.Qd3 and 19.f5, and Black then converted accurately with 25...Bxf1, 27...Qxf6 and 31...Qf2#. Black's 30...Rc1+ was a precise finish. Both opponents were weak and both targets played tactically well, so it is close. Black in B showed slightly more positional understanding and calculation in a more complex game, so I lean B.", "confidence_valid": true }, { "model": "google/gemini-3.8-flash", "pair": "pair_12", "game_a": "game_05", "color_a": "black", "game_b": "game_06", "color_b": "black", "rating_a": 1867, "rating_b": 1382, "gap": 485, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.42, "correct": 0, "brier": 0.3364000000000001, "old_estimate_a": 1550, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Both target players won cleanly with Black, but Game B faced a more coherent kingside setup and demonstrated superior coordination. Game B Black neutralized White's h-pawn thrust, found energetic knight outposts (13...Nd4, 17...Nc5), and executed a dominant queenside infiltration (18...Ncxb3, 19...Nxa1). The conversion was completely clinical through 29...Rac8 and mate. In Game A, White wasted several tempi shuffling the bishop (Bd3-Be2, Be4-Bd3-Be4) and blundered outright with 16. Be4?? allowing 16...Nxg2+, leaving Black with an easier tactical task to finish the game.", "confidence_valid": true }, { "model": "google/gemini-3.1-pro-preview", "pair": "pair_12", "game_a": "game_05", "color_a": "black", "game_b": "game_06", "color_b": "black", "rating_a": 1867, "rating_b": 1382, "gap": 485, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.6, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 1550, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Both targets played excellent attacking games against opponents who misplayed the opening. In Game A, Black was extremely sharp, finding tactical shots like 15... Nf4, 16... Nxg2+, and the crushing 24... Qb7! to create mating nets. In Game B, Black also punished White's bizarre setup (6. Bb1) efficiently with knights maneuvering to d4 and c5. However, Game A Black's tactical execution was slightly crisper and more complex, suggesting a higher level of rating.", "confidence_valid": true }, { "model": "google/gemini-3.5-flash", "pair": "pair_12", "game_a": "game_05", "color_a": "black", "game_b": "game_06", "color_b": "black", "rating_a": 1867, "rating_b": 1382, "gap": 485, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.7, "correct": 1, "brier": 0.09000000000000002, "old_estimate_a": 1450, "old_estimate_b": 1450, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "In Game A, Black demonstrates excellent opening understanding with an active Sicilian setup, followed by precise, energetic tactical calculation. Moves like 15...Nf4, the tactical sequence 16...Nxg2+ and 17...fxe4, followed by 20...Nb4! show high-level tactical vision and a direct, punishing attack. In Game B, Black plays a somewhat passive and disjointed opening, including the repetitive maneuver 3...Bd7, 9...Bf5, and 10...Bd7. While Game B's Black successfully exploits White's passive play later on, the overall cohesion, opening preparation, and tactical sharpness of Game A's Black suggest a significantly higher established rating.", "confidence_valid": true }, { "model": "google/gemini-3.7-flash", "pair": "pair_12", "game_a": "game_05", "color_a": "black", "game_b": "game_06", "color_b": "black", "rating_a": 1867, "rating_b": 1382, "gap": 485, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.64, "correct": 1, "brier": 0.1296, "old_estimate_a": 1600, "old_estimate_b": 1550, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black displayed excellent opening understanding and tactical sharpness against White's unorthodox Sicilian play. Moves like 4...b5, 5...c4, 11...f5, and the precise tactical combination starting with 16...Nxg2+ followed by 17...fxe4 and 20...Nb4/21...Nbd3 demonstrated high-level positional control and tactical accuracy. In Game B, both players looked less seasoned; Black's passive 3...Bd7 and retreating tempo loss 9...Bf5 and 10...Bd7 signaled less sophisticated opening knowledge, even though Black converted well later with 13...Nd4 and 17...Nc5. Game A Black demonstrated higher overall proficiency.", "confidence_valid": true }, { "model": "qwen/qwen3.8-max-prime", "pair": "pair_12", "game_a": "game_05", "color_a": "black", "game_b": "game_06", "color_b": "black", "rating_a": 1867, "rating_b": 1382, "gap": 485, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.68, "correct": 1, "brier": 0.10239999999999996, "old_estimate_a": 1650, "old_estimate_b": 1500, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black shows sharper tactical vision: 15...Nf4! sets up 16...Nxg2+!, and after 17.Kf1 the sequence 21...Nbd3, 22...Bxe4, 23...Bxh1 wins decisively, finishing with 29...Qxf3#. Game B Black plays solidly but relies heavily on opponent blunders: White's 13.Ng5?, 17.Qe3?, and especially 24.Nf6+? sacrifice material gratuitously. B's 18...Ncxb3 is good but less forcing than A's coordinated knight-bishop attack. A's opponent resisted longer and made fewer outright hangs, so A's win required more accurate, creative play.", "confidence_valid": true }, { "model": "anthropic/claude-fable-5.1", "pair": "pair_12", "game_a": "game_05", "color_a": "black", "game_b": "game_06", "color_b": "black", "rating_a": 1867, "rating_b": 1382, "gap": 485, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.42, "correct": 0, "brier": 0.3364000000000001, "old_estimate_a": 1250, "old_estimate_b": 1050, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black played a coherent Sicilian with ...b5/...c4 harassing the bishop, then exploited White's loose play with Nf4/Nxg2+ tactics and a clean mating attack; but White in A was very weak (Bc4/Bd3/Be2 shuffles, Be4 hanging to Nxg2+). Game B Black faced a slightly more organized opponent (Nc3, Qc2, Bb2 setup) and converted via ...Bxg5, ...Ncxb3 winning the exchange, ...Bxc4, and the final ...Rc1+ ...Qf2# mating net, though ...Bf5/Bd7/Bg4/Be6 bishop wandering and ...Bxg5 were loose. Both are similar club level; B's calculation of multiple material-winning tactics against a less cooperative opponent gives a slight edge.", "confidence_valid": true }, { "model": "google/gemini-3-flash-preview", "pair": "pair_12", "game_a": "game_05", "color_a": "black", "game_b": "game_06", "color_b": "black", "rating_a": 1867, "rating_b": 1382, "gap": 485, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.65, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1600, "old_estimate_b": 1700, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Black in Game A demonstrates superior opening knowledge and tactical precision. Black plays a thematic Sicilian setup and immediately punishes White's inaccuracies (13. b3 and 16. Be4) with the powerful 15... Nf4 and 16... Nxg2+. The subsequent coordination of the knights (20... Nb4, 21... Nbd3) and the clinical finish (25... Nh3+ and 27... Be4+) show a high level of proficiency. In Game B, Black plays more passively with 3... Bd7 and 10... Bd7. While Black B exploits White's eccentric play (6. Bb1, 8. h4) well with 13... Nd4, the overall level of theoretical accuracy and calculated aggression in Game A suggests a stronger established player.", "confidence_valid": true }, { "model": "google/gemini-3.6-flash", "pair": "pair_12", "game_a": "game_05", "color_a": "black", "game_b": "game_06", "color_b": "black", "rating_a": 1867, "rating_b": 1382, "gap": 485, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.58, "correct": 1, "brier": 0.17640000000000003, "old_estimate_a": 1800, "old_estimate_b": 1550, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Both players capitalized well on unorthodox opening play by their opponents, but Black in Game A displayed superior dynamic precision and tactical awareness. Game A Black launched a energetic Sicilian expansion with 4...b5 and 5...c4, then efficiently punished White's clumsy piece coordination (13. b3?) with a series of sharp tactical blows starting with 15...Nf4!, 16...Nxg2+!, 17...fxe4!, and 20...Nb4!. In Game B, Black played slightly passively early on (3...Bd7) before taking advantage of White's poor setup (4. Bd3, 6. Bb1) to win material. Game A Black showed higher tactical sharpness and dynamic initiative overall.", "confidence_valid": true }, { "model": "openai/gpt-5.6-sol", "pair": "pair_12", "game_a": "game_05", "color_a": "black", "game_b": "game_06", "color_b": "black", "rating_a": 1867, "rating_b": 1382, "gap": 485, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.64, "correct": 1, "brier": 0.1296, "old_estimate_a": 1850, "old_estimate_b": 1600, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black showed more purposeful calculation: 16...Nxg2+! 17.Kf1 fxe4 won the bishop while preserving the knight, and 22...Bxe4 23...Bxh1 followed by ...Nh3+ and the forced mating sequence was precise. Game B Black also converted well, especially with 18...Ncxb3 19...Nxa1 and 25...Bxf1, but the opening involved several inefficient bishop moves (...Bd7-f5-d7), and much of the win followed White’s conspicuous tactical errors. A’s play looks more consistently coordinated and tactically sophisticated.", "confidence_valid": true }, { "model": "openai/gpt-6-sol", "pair": "pair_12", "game_a": "game_05", "color_a": "black", "game_b": "game_06", "color_b": "black", "rating_a": 1867, "rating_b": 1382, "gap": 485, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.68, "correct": 1, "brier": 0.10239999999999996, "old_estimate_a": 1700, "old_estimate_b": 1600, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A’s Black played a more coherent game: 11...f5 drove back White’s bishop, 16...Nxg2+ set up 17...fxe4, and 20...Nb4 followed by 21...Nbd3 supported the decisive 22...Bxe4. Game B’s Black also spotted strong tactics, notably 18...Ncxb3 and 25...Bxf1, but the early 9...Bf5 10...Bd7 lost time. Both opponents made serious mistakes, so the comparison is uncertain.", "confidence_valid": true }, { "model": "openai/gpt-6.1-sol", "pair": "pair_12", "game_a": "game_05", "color_a": "black", "game_b": "game_06", "color_b": "black", "rating_a": 1867, "rating_b": 1382, "gap": 485, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.67, "correct": 1, "brier": 0.10889999999999997, "old_estimate_a": 1650, "old_estimate_b": 1550, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black showed more coherent development and sustained tactical coordination: 15...Nf4, 16...Nxg2+ and 17...fxe4 exploited White’s bishop placement, while 21...Nbd3 and 22...Bxe4 led to winning the rook. The finish with 24...Qb7, 25...Nh3+ and 27...Be4+ was particularly coordinated. Game B Black also found effective tactics—17...Nc5, 18...Ncxb3 and 25...Bxf1—and finished accurately with 30...Rc1+. However, the repeated early bishop moves (...Bd7–f5–d7–g4–e6) suggest less efficient opening handling. Both benefited from substantial White errors, so the preference for A is moderate.", "confidence_valid": true }, { "model": "openai/gpt-5.6-luna", "pair": "pair_12", "game_a": "game_05", "color_a": "black", "game_b": "game_06", "color_b": "black", "rating_a": 1867, "rating_b": 1382, "gap": 485, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.58, "correct": 1, "brier": 0.17640000000000003, "old_estimate_a": 1750, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A’s Black showed a more coherent attacking plan and tactical coordination: 16...Nxg2+ followed by 17...fxe4, 21...Nbd3, and 23...Bxh1 created sustained pressure and a forced mating net. Game B’s Black also exploited major oversights, especially 18...Ncxb3 and 19...Nxa1, then 25...Bxf1 and 27...Qxf6. However, the repeated bishop retreats in the opening (3...Bd7, 9...Bf5, 10...Bd7) and dependence on White’s loose play make B’s performance look slightly less polished. A therefore appears marginally more likely to have had the higher established rapid strength.", "confidence_valid": true }, { "model": "openai/gpt-6-astra", "pair": "pair_12", "game_a": "game_05", "color_a": "black", "game_b": "game_06", "color_b": "black", "rating_a": 1867, "rating_b": 1382, "gap": 485, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.73, "correct": 1, "brier": 0.0729, "old_estimate_a": 1650, "old_estimate_b": 1600, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black showed more coherent development and sustained tactical coordination: 15...Nf4, 16...Nxg2+, and 17...fxe4 won material, while ...Nb4–d3 intensified the attack. The finish 27...Be4+ cleverly combined a discovered rook check with an attack on White’s queen. Game B Black also found effective tactics, notably 17...Nc5 and ...Ncxb3–a1, but the opening featured repeated bishop moves (...Bd7–f5–d7–g4–e6) and less economical development. Both opponents supplied substantial errors; B’s opponent additionally donated a rook with 25.Rf1. A’s more consistent coordination makes A the likelier stronger established player.", "confidence_valid": true }, { "model": "z-ai/glm-5.3-flash", "pair": "pair_12", "game_a": "game_05", "color_a": "black", "game_b": "game_06", "color_b": "black", "rating_a": 1867, "rating_b": 1382, "gap": 485, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.68, "correct": 1, "brier": 0.10239999999999996, "old_estimate_a": 1750, "old_estimate_b": 1650, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Black in A played an active, coherent game: queenside space (4...a6/4...b5), the tactical shots 15...Nf4 and 16...Nxg2+!, then 21...Nbd3 forking, winning material cleanly and concluding with a forced mating net (25...Nh3+, 26...Bxf3, 27...Be4+, 29...Qxf3#). Black in B also won but largely by punishing obvious White errors (13.Ng5?, 18...Ncxb3/19...Nxa1 exploiting loose moves), and needed 30 moves against passive play; A's attack was more precise and self-created, suggesting the higher rating.", "confidence_valid": true } ], "a": { "game": "game_05", "color": "black", "rating": 1867, "url": "https://lichess.org/mCkfc84Y", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"0-1\"]\n[TimeControl \"600+0\"]\n\n1. e4 c5 2. Bc4 e6 3. Nf3 a6 4. c3 b5 5. Bd3 c4 6. Be2 Bb7 7. d3 cxd3 8. Bxd3\nNf6 9. Nbd2 Be7 10. e5 Nd5 11. Be4 f5 12. Bd3 O-O 13. b3 Nc6 14. Bb2 Qc7 15. c4\nNf4 16. Be4 Nxg2+ 17. Kf1 fxe4 18. Nxe4 Nf4 19. Nfg5 h6 20. Nf3 Nb4 21. a3 Nbd3\n22. Bc3 Bxe4 23. Nd4 Bxh1 24. Kg1 Qb7 25. f3 Nh3+ 26. Kf1 Bxf3 27. Qxd3 Be4+\n28. Nf3 Bxd3+ 29. Kg2 Qxf3# 0-1" }, "b": { "game": "game_06", "color": "black", "rating": 1382, "url": "https://lichess.org/wLJhUOmU", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"0-1\"]\n[TimeControl \"600+0\"]\n\n1. c4 d6 2. Nc3 Nf6 3. e3 Bd7 4. Bd3 e5 5. a3 Be7 6. Bb1 O-O 7. Qc2 g6 8. h4 h5\n9. b3 Bf5 10. e4 Bd7 11. Bb2 Nc6 12. Nf3 Bg4 13. Ng5 Nd4 14. Qd3 Nd7 15. f3 Be6\n16. f4 Bxg5 17. hxg5 Nc5 18. Qe3 Ncxb3 19. f5 Nxa1 20. Bxa1 Bxc4 21. fxg6 fxg6\n22. Nd5 Ne6 23. Qc3 Ba6 24. Nf6+ Kg7 25. Rf1 Bxf1 26. Kxf1 Nxg5 27. Ke1 Qxf6\n28. Qxc7+ Kh6 29. Bd3 Rac8 30. Qxb7 Rc1+ 31. Ke2 Qf2# 0-1" }, "wrong": 3, "gap": 485 }, { "id": "pair_22", "predictions": [ { "model": "anthropic/claude-opus-5.5", "pair": "pair_22", "game_a": "game_04", "color_a": "black", "game_b": "game_13", "color_b": "black", "rating_a": 2718, "rating_b": 1551, "gap": 1167, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.68, "correct": 1, "brier": 0.10239999999999996, "old_estimate_a": 1550, "old_estimate_b": 1300, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black played a coherent Caro-Kann exchange plan (…Bg4xe2, …O-O-O, …Ne4, …f5, …g5) and finished with the sharp queen sacrifice 25…Qg1+ 26.Rxg1 Nf2#, a smothered mate. Game B Black won mainly because White blundered repeatedly (8.Rb3, 16.Rc5 losing the exchange, 35.Nxb5?? allowing …Rc1#). Black's own play there was loose: early queen wandering with 6…Qxb2, and the odd 26…Rf6/28…Re6 that dropped the exchange to Nc7. That suggests a weaker player benefiting from a weak opponent. A's opponent was more competent, and A's tactical finish was cleaner.", "confidence_valid": true }, { "model": "anthropic/claude-sonnet-5.5", "pair": "pair_22", "game_a": "game_04", "color_a": "black", "game_b": "game_13", "color_b": "black", "rating_a": 2718, "rating_b": 1551, "gap": 1167, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.6, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 1600, "old_estimate_b": 1550, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black played the Caro-Kann sensibly (4...Nc6, 6...Bg4, 10...Bxe2) and then punished White's weak kingside play. 18...Bxc5 and 19...g5 were active, 21...gxf4 opened lines, and 22...Nd4 followed by 23...Qxc5 and 24...Ne2+ led to a clean smothered-style mate with 26...Nf2#. The opponent was weak, so the win is only modest evidence. Game B Black grabbed the b2 pawn with 6...Qxb2 and won, but the opponent was also weak: White's 16.Rc5 lost an exchange, and 28.Ne8 and 30.Nxe6 were poor. Black's play was solid, yet 21...Qe5 and 26...Rf6 were fairly ordinary, and the endgame mate was handed over. Both opponents look similar in strength. A's attack was slightly more convincing, so I lean to A.", "confidence_valid": true }, { "model": "google/gemini-3.8-flash", "pair": "pair_22", "game_a": "game_04", "color_a": "black", "game_b": "game_13", "color_b": "black", "rating_a": 2718, "rating_b": 1551, "gap": 1167, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.88, "correct": 1, "brier": 0.0144, "old_estimate_a": 1950, "old_estimate_b": 1250, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Player A demonstrates sound opening knowledge in the Caro-Kann Exchange Variation and excellent strategic understanding (16...Ne4, 17...f5, 19...g5), culminating in a flawlessly executed smothered mate combination starting with 24...Ne2+. In contrast, Player B exhibits low-tier opening habits with early, risky queen sorties (3...Qa5+, 5...Qb5, 6...Qxb2) and poor tactical awareness, allowing the devastating 28. Ne8 and 29. Nc7 fork to drop an entire rook. Player B only won because White committed an elementary back-rank blunder on move 35. Player A is far more polished and established.", "confidence_valid": true }, { "model": "google/gemini-3.1-pro-preview", "pair": "pair_22", "game_a": "game_04", "color_a": "black", "game_b": "game_13", "color_b": "black", "rating_a": 2718, "rating_b": 1551, "gap": 1167, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.95, "correct": 1, "brier": 0.0025000000000000044, "old_estimate_a": 2150, "old_estimate_b": 1150, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Target A played a principled Caro-Kann, castled queenside, and executed a beautiful smothered mate sequence starting with 22... Nd4! Target A demonstrated strong positional understanding and advanced tactical vision. In contrast, Target B played a dubious opening (3... Qa5+), went pawn-grabbing early (6... Qxb2), and blundered material (28... Re6? leading to 31. Rxc6). Target B only won because White blundered a basic back-rank mate at the end (35. Nxb5?? Rc1#). Target A's cohesive, high-level play heavily outclasses Target B's erratic and opportunistic style.", "confidence_valid": true }, { "model": "google/gemini-3.5-flash", "pair": "pair_22", "game_a": "game_04", "color_a": "black", "game_b": "game_13", "color_b": "black", "rating_a": 2718, "rating_b": 1551, "gap": 1167, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.95, "correct": 1, "brier": 0.0025000000000000044, "old_estimate_a": 1900, "old_estimate_b": 1050, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black demonstrates excellent positional understanding and high tactical sharpness, playing a solid Caro-Kann and executing a flawless smothered mate sequence (24...Ne2+ 25.Kh1 Qg1+ 26.Rxg1 Nf2#). In contrast, Game B Black plays an erratic opening with early queen moves (3...Qa5+, 5...Qb5) and exhibits weak tactical vision, missing a simple winning queen fork with 17...Nc2+ after 17.c3, instead playing 17...b6. Game B only ends because White blunders into a simple back-rank mate with 35.Nxb5?? Rc1#.", "confidence_valid": true }, { "model": "google/gemini-3.7-flash", "pair": "pair_22", "game_a": "game_04", "color_a": "black", "game_b": "game_13", "color_b": "black", "rating_a": 2718, "rating_b": 1551, "gap": 1167, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.85, "correct": 1, "brier": 0.022500000000000006, "old_estimate_a": 1950, "old_estimate_b": 1300, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black displayed high-level Caro-Kann opening understanding, purposeful strategic play on both flanks, and flawless tactical calculation, concluding with a textbook smothered mate starting with 24...Ne2+ and 25...Qg1+. In contrast, Game B featured sloppy opening play with risky queen excursions (5...Qb5, 6...Qxb2). Game B Black later missed a tactical sequence, falling into a knight fork and dropping an exchange on move 29 (28.Ne8 Re6 29.Nc7), only winning due to White blundering back-rank mate with 35.Nxb5??", "confidence_valid": true }, { "model": "qwen/qwen3.8-max-prime", "pair": "pair_22", "game_a": "game_04", "color_a": "black", "game_b": "game_13", "color_b": "black", "rating_a": 2718, "rating_b": 1551, "gap": 1167, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.72, "correct": 1, "brier": 0.07840000000000001, "old_estimate_a": 1550, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black plays a coherent Caro-Kann, coordinates queenside castling with ...O-O-O and ...Kb8, then executes a clean tactical finish: 21...gxf4, 22...Nd4, 23...Qxc5, and the forcing 24...Ne2+ 25...Qg1+ 26...Nf2#. Game B Black's opening is awkward: the queen raid 3...Qa5+, 5...Qb5, 6...Qxb2 wastes tempos, and later ...b6/...bxc5 leaves a clumsy structure; Black wins mainly because White self-destructs with 28.Ne8? and 30.Nxe6?. A's active piece play and precise smothered-mate finish show stronger established technique.", "confidence_valid": true }, { "model": "anthropic/claude-fable-5.1", "pair": "pair_22", "game_a": "game_04", "color_a": "black", "game_b": "game_13", "color_b": "black", "rating_a": 2718, "rating_b": 1551, "gap": 1167, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.6, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 1600, "old_estimate_b": 1300, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black played a coherent Caro-Kann exchange setup (6...Bg4, 10...Bxe2 trading the bishop, 16...Ne4, 17...f5) and finished with a clean 25...Qg1+ 26.Rxg1 Nf2# smothered mate; White's play (Qe1, Bg5-h4-e1) was weak but not absurd. Game B Black grabbed material with 6...Qxb2 and won, but allowed a repeatedly harassed queen and lost an exchange to 28.Ne8/29.Nc7 (28...Re6?), winning only because White blundered terribly (Rc5?, Nxb5?? allowing Rc1#). White in Game B was very weak (Qa1, Rb3-a3-c3-c5), suggesting a lower-rated pairing. Game A's opposition and Black's technique look somewhat higher-level.", "confidence_valid": true }, { "model": "google/gemini-3-flash-preview", "pair": "pair_22", "game_a": "game_04", "color_a": "black", "game_b": "game_13", "color_b": "black", "rating_a": 2718, "rating_b": 1551, "gap": 1167, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.85, "correct": 1, "brier": 0.022500000000000006, "old_estimate_a": 2050, "old_estimate_b": 1250, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Target A (Black) exhibits a sophisticated understanding of the Caro-Kann, coordinating a complex kingside attack while managing queenside pressure (16...Ne4, 19...g5). The execution of the classic smothered mate (24...Ne2+ through 26...Nf2#) demonstrates high tactical awareness. Target B (Black) plays a suspicious opening, making multiple early queen moves (3...Qa5+, 5...Qb5, 6...Qxb2) to grab a pawn, a strategy often punished at higher levels. While Target B won via a back-rank mate, it was facilitated by a significant oversight by White (34. Nc3??). Target A’s strategic cohesion and tactical finish suggest a higher established rating.", "confidence_valid": true }, { "model": "google/gemini-3.6-flash", "pair": "pair_22", "game_a": "game_04", "color_a": "black", "game_b": "game_13", "color_b": "black", "rating_a": 2718, "rating_b": 1551, "gap": 1167, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.88, "correct": 1, "brier": 0.0144, "old_estimate_a": 1850, "old_estimate_b": 1300, "baseline_choice": "A", "baseline_credit": 1, "explanation": "In Game A, Black played a very high-quality Caro-Kann defense, castling queenside and accurately handling White's pawn storm before executing a brilliant smothered mate combination (24...Ne2+ 25. Kh1 Qg1+ 26. Rxg1 Nf2#). In contrast, Game B's Black displayed poor tactical awareness, blundering material with 28...Re6 29. Nc7! and losing a piece on c6 (31. Rxc6). Game B Black was heavily losing and only won because White committed a fatal back-rank blunder with 35. Nxb5??, allowing 35...Rc1#. Game A Black demonstrated vastly superior tactical calculation and positional judgment.", "confidence_valid": true }, { "model": "openai/gpt-5.6-sol", "pair": "pair_22", "game_a": "game_04", "color_a": "black", "game_b": "game_13", "color_b": "black", "rating_a": 2718, "rating_b": 1551, "gap": 1167, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.82, "correct": 1, "brier": 0.03240000000000002, "old_estimate_a": 1950, "old_estimate_b": 1550, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black showed a coherent plan: 13...O-O-O, 15...Rc8, 16...Ne4 and 17...f5 coordinated the pieces before the kingside pawn attack. The finish 24...Ne2+ 25.Kh1 Qg1+ 26.Rxg1 Nf2# was a precise forced mating combination. Game B Black found useful tactics with 17...b6 and ultimately 35...Rc1#, but the early queen expedition 5...Qb5 6...Qxb2 cost many tempi, and later 28...Re6 29.Nc7 Ra7 30.Nxe6 allowed White to recover material and gain the initiative. B’s win depended more heavily on White’s final mating blunder.", "confidence_valid": true }, { "model": "openai/gpt-6-sol", "pair": "pair_22", "game_a": "game_04", "color_a": "black", "game_b": "game_13", "color_b": "black", "rating_a": 2718, "rating_b": 1551, "gap": 1167, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.88, "correct": 1, "brier": 0.0144, "old_estimate_a": 1800, "old_estimate_b": 1500, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A’s Black built a coherent attack with 16...Ne4, 17...f5 and 22...Nd4, then finished with the calculated 24...Ne2+ 25...Qg1+ 26...Nf2#. Game B’s Black won after White ignored the threat to the rook on c5, but the early queen excursion with 5...Qb5 and 6...Qxb2 was risky, and 28...Re6 allowed 29.Nc7 to fork the rooks. A’s play is more convincing evidence of established strength.", "confidence_valid": true }, { "model": "openai/gpt-6.1-sol", "pair": "pair_22", "game_a": "game_04", "color_a": "black", "game_b": "game_13", "color_b": "black", "rating_a": 2718, "rating_b": 1551, "gap": 1167, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.86, "correct": 1, "brier": 0.019600000000000003, "old_estimate_a": 2000, "old_estimate_b": 1350, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black showed coherent development and coordinated attacking play: 16...Ne4, 19...g5 and 22...Nd4 established active central knights, culminating in the accurately calculated 24...Ne2+ 25...Qg1+ 26...Nf2# combination. Game B Black also recognized a mating opportunity with 34...Rc7 and 35...Rc1#, but benefited substantially from White's rook wandering and 18.Qb2, which allowed ...bxc5. More tellingly, B's 28...Re6 allowed 29.Nc7 to fork both rooks, followed by 30.Nxe6 and 31.Rxc6, surrendering much of the earlier material advantage. A's more consistent coordination and tactical accuracy suggest the higher established strength.", "confidence_valid": true }, { "model": "openai/gpt-5.6-luna", "pair": "pair_22", "game_a": "game_04", "color_a": "black", "game_b": "game_13", "color_b": "black", "rating_a": 2718, "rating_b": 1551, "gap": 1167, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.43, "correct": 0, "brier": 0.3249000000000001, "old_estimate_a": 1750, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game B’s black handled a more complicated tactical battle and still converted after White’s active sequence 28.Ne8, 30.Nxe6, and 31.Rxc6. The queen excursion with 6...Qxb2–8...Qa5 was risky, but Black consistently found practical resources and finished with 35...Rc1#. Game A’s black played a coherent attack, especially 23...Qxc5, 24...Ne2+, and 25...Qg1+, but White’s 22.b5 and the resulting mating net were comparatively straightforward errors. Thus B appears slightly more likely to have had the higher established strength.", "confidence_valid": true }, { "model": "openai/gpt-6-astra", "pair": "pair_22", "game_a": "game_04", "color_a": "black", "game_b": "game_13", "color_b": "black", "rating_a": 2718, "rating_b": 1551, "gap": 1167, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.9, "correct": 1, "brier": 0.009999999999999995, "old_estimate_a": 2000, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black showed coherent development and attacking coordination: 16...Ne4, 17...f5 and 19...g5 established central control and kingside pressure. The finish 22...Nd4 followed by 24...Ne2+ and 25...Qg1+ demonstrated precise tactical vision. Game B Black exploited White’s loose rook with 17...b6 and found 35...Rc1#, but played less consistently. Repeated early queen moves delayed development, and 28...Re6 allowed 29.Nc7, forking the rooks and forcing material concessions. Both opponents supplied opportunities, but A’s sustained coordination is stronger evidence of established playing strength than B’s recovery against erratic opposition.", "confidence_valid": true }, { "model": "z-ai/glm-5.3-flash", "pair": "pair_22", "game_a": "game_04", "color_a": "black", "game_b": "game_13", "color_b": "black", "rating_a": 2718, "rating_b": 1551, "gap": 1167, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.35, "correct": 0, "brier": 0.42250000000000004, "old_estimate_a": 1600, "old_estimate_b": 1400, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Both Black targets beat weaker-seeming White opposition, but B's play looks more sustained. In Game B, Black punished White's queen retreats (6...Qxb2, 8...Qa5) and found the in-between tactic 18...bxc5! netting a piece, then converted cleanly: 20...O-O, trades with 30...dxe6, and finished with 34...Bb5 and 35...Rc1#, showing endgame technique. In Game A, Black played well (22...Nd4 opening lines, 24...Ne2+ leading to the smothered mate 26...Nf2#), but the mate relied on White's cooperative 25.Kh1 Qg1+ allowing Rxg1, and earlier Black moves (17...f5, 19...gxf4) were more speculative. B's longer, controlled domination suggests a slightly higher rating.", "confidence_valid": true } ], "a": { "game": "game_04", "color": "black", "rating": 2718, "url": "https://lichess.org/At0NmiOd", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"0-1\"]\n[TimeControl \"600+0\"]\n\n1. e4 c6 2. d4 d5 3. exd5 cxd5 4. Bd3 Nc6 5. c3 Qc7 6. Ne2 Bg4 7. O-O e6 8. Qe1\nNf6 9. Bg5 Bh5 10. f4 Bxe2 11. Qxe2 Be7 12. Nd2 h6 13. Bh4 O-O-O 14. b4 Kb8 15.\nNb3 Rc8 16. a4 Ne4 17. Be1 f5 18. Nc5 Bxc5 19. dxc5 g5 20. a5 a6 21. c4 gxf4\n22. b5 Nd4 23. Qb2 Qxc5 24. Bb4 Ne2+ 25. Kh1 Qg1+ 26. Rxg1 Nf2# 0-1" }, "b": { "game": "game_13", "color": "black", "rating": 1551, "url": "https://lichess.org/GLVVikmb", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"0-1\"]\n[TimeControl \"600+0\"]\n\n1. d4 e6 2. Nf3 c5 3. dxc5 Qa5+ 4. Bd2 Qxc5 5. Be3 Qb5 6. Nc3 Qxb2 7. Rb1 Qa3\n8. Rb3 Qa5 9. Bd2 Bb4 10. Qa1 Ne7 11. Nb1 Bxd2+ 12. Nfxd2 f6 13. Ra3 Qc7 14.\nRc3 Nbc6 15. e3 Nd5 16. Rc5 Ndb4 17. c3 b6 18. Qb2 bxc5 19. cxb4 cxb4 20. Ne4\nO-O 21. Bb5 Qe5 22. Qxe5 Nxe5 23. Nd6 a6 24. Ba4 Nc6 25. O-O f5 26. Re1 Rf6 27.\nRc1 e5 28. Ne8 Re6 29. Nc7 Ra7 30. Nxe6 dxe6 31. Rxc6 Bd7 32. Rc4 Bxa4 33. Rxb4\nBb5 34. Nc3 Rc7 35. Nxb5 Rc1# 0-1" }, "wrong": 2, "gap": 1167 }, { "id": "pair_25", "predictions": [ { "model": "anthropic/claude-opus-5.5", "pair": "pair_25", "game_a": "game_19", "color_a": "black", "game_b": "game_07", "color_b": "white", "rating_a": 2474, "rating_b": 1862, "gap": 612, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.75, "correct": 1, "brier": 0.0625, "old_estimate_a": 1550, "old_estimate_b": 1200, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black played a coherent QGD Exchange with ...Bf5-g6 and ...h5 into an endgame. The errors came late: 22...c5 allowed Nxh5 to win a pawn, and 33...Kd8 and 34...Nxf5 let the h-pawn promote. Overall this was reasonable play against a competent opponent. Game B White made gross blunders. 21.Bf3?? allowed 21...Bc2, trapping the queen, and 22.Bh5 made it worse, losing the queen for little. Earlier play such as 18.Rb3 and 19.b5 was also aimless. Hanging the queen to a simple bishop tactic suggests a weaker player than A's endgame inaccuracies do.", "confidence_valid": true }, { "model": "anthropic/claude-sonnet-5.5", "pair": "pair_25", "game_a": "game_19", "color_a": "black", "game_b": "game_07", "color_b": "white", "rating_a": 2474, "rating_b": 1862, "gap": 612, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.4, "correct": 0, "brier": 0.36, "old_estimate_a": 1600, "old_estimate_b": 1500, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A, Black: allowed a queen trade and doubled f-pawns, then played passively. 13...b5 and 14...Ke7 were okay, but 22...c5 and 23...c4+ let White build a kingside attack. Black blundered with 36...b4, overlooking the h-pawn advance; 34...Nxf5 gave up the knight and the rook trade was poor, so the h-pawn was decisive. Black's play looks around club level. Game B, White: played a standard Slav/Queen's Gambit structure sensibly through move 16, but 21.Bf3 allowed 21...Bc2, and 22.Bh5 Bxd1 23.Bxg6 Bxb3 lost the exchange and more, a major tactical oversight (the queen was en prise to a deflection). Both are weak; White in B blundered a queen-trade sequence, whereas Black in A held a roughly equal game for a long time. Slight lean to B being stronger because B's opening play was more structured, though the evidence is thin.", "confidence_valid": true }, { "model": "google/gemini-3.8-flash", "pair": "pair_25", "game_a": "game_19", "color_a": "black", "game_b": "game_07", "color_b": "white", "rating_a": 2474, "rating_b": 1862, "gap": 612, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.72, "correct": 1, "brier": 0.07840000000000001, "old_estimate_a": 1750, "old_estimate_b": 1550, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Player A (Black) demonstrated a solid grasp of structural and positional nuances in an offbeat Queen's Gambit Declined endgame, handling doubled f-pawns and coordinating pieces reasonably well until collapsing under heavy pressure late in the game (33...Kd8? and 34...Nxf5?). In contrast, Player B (White) adopted a timid, passive setup with little initiative and committed a severe one-move tactical blunder with 21. Bf3?, completely missing the direct fork 21...Bc2, which immediately dropped a full exchange and prompted resignation on move 24.", "confidence_valid": true }, { "model": "google/gemini-3.1-pro-preview", "pair": "pair_25", "game_a": "game_19", "color_a": "black", "game_b": "game_07", "color_b": "white", "rating_a": 2474, "rating_b": 1862, "gap": 612, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.75, "correct": 1, "brier": 0.0625, "old_estimate_a": 1850, "old_estimate_b": 1400, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Target A (Black) played a solid opening and navigated a complex endgame reasonably well until falling for a multi-step tactical trap (33... Kd8 34. Ra7 Nxf5 35. Ra8+) which lost the rook. Target B (White) played a rather passive opening (3. e3, 5. Nbd2) and then completely missed a basic one-move double attack with 21. Bf3? Bc2, forking the Queen and Rook. Target A demonstrated deeper strategic understanding and positional play compared to Target B's passive play and elementary tactical oversight.", "confidence_valid": true }, { "model": "google/gemini-3.5-flash", "pair": "pair_25", "game_a": "game_19", "color_a": "black", "game_b": "game_07", "color_b": "white", "rating_a": 2474, "rating_b": 1862, "gap": 612, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.75, "correct": 1, "brier": 0.0625, "old_estimate_a": 1650, "old_estimate_b": 1600, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Player A (Black) demonstrates a much higher baseline of chess understanding than Player B (White). In Game A, Black navigates a highly theoretical Queen's Gambit Declined Exchange line, maintaining a reasonable position through the middlegame (e.g., 10... h5, 12... Nd7) before succumbing to White's strong endgame play. In contrast, Player B plays very passively in the opening (3. e3, 5. Nbd2) and exhibits poor tactical vision. After Black's 20... Qg6, which threatens mate on g2, White plays 21. Bf3? completely overlooking the basic fork 21... Bc2, which wins the exchange and the game on the spot. Player A is clearly the stronger, more established player.", "confidence_valid": true }, { "model": "google/gemini-3.7-flash", "pair": "pair_25", "game_a": "game_19", "color_a": "black", "game_b": "game_07", "color_b": "white", "rating_a": 2474, "rating_b": 1862, "gap": 612, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.72, "correct": 1, "brier": 0.07840000000000001, "old_estimate_a": 1850, "old_estimate_b": 1650, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Player A (Black) played a complex endgame against a strong opponent, demonstrating reasonable strategic ideas (queenside pawn expansion, king activity) before succumbing to tactical tricks on move 34 (34...Nxf5? allowing 35. Ra8+). Player B (White) played passively with unambitious developing moves and committed a basic tactical oversight with 21. Bf3??, walking straight into the 21...Bc2 fork winning the rook on b3 and leading to an immediate resignation.", "confidence_valid": true }, { "model": "qwen/qwen3.8-max-prime", "pair": "pair_25", "game_a": "game_19", "color_a": "black", "game_b": "game_07", "color_b": "white", "rating_a": 2474, "rating_b": 1862, "gap": 612, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.68, "correct": 1, "brier": 0.10239999999999996, "old_estimate_a": 1300, "old_estimate_b": 1400, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game A Black defended the Exchange Caro-Kann solidly; the loss resulted from a concrete tactical slip (30...dxc4 allowing 31.d5 and the rook raid 33.Re7+/37.h7), not systemic errors. Game B White played passively (b3, a4), lost time after 12.Nxe4, then blundered a whole queen with 22.Bh5??, ignoring that 22...Bxd1 captures the queen. Game B White's queen oversight indicates weaker tactical alertness than Game A Black's eventual endgame lapse, so A was likely the higher-rated player.", "confidence_valid": true }, { "model": "anthropic/claude-fable-5.1", "pair": "pair_25", "game_a": "game_19", "color_a": "black", "game_b": "game_07", "color_b": "white", "rating_a": 2474, "rating_b": 1862, "gap": 612, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.6, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 1400, "old_estimate_b": 1250, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game B White blundered a full exchange tactically with 21. Bf3?? allowing 21...Bc2 forking queen and rook, then resigned early after 23...Bxb3 — a clear tactical oversight by a lower-level player, though earlier play (8.cxd5, 10.b3, 17.b4 minority plan) was coherent. Game A Black lost a long endgame after a sound opening (6...Bf5 7...Bg6, trading into a doubled-pawn structure), with errors like 22...c5 and 29...Nd8 letting the h-pawn and d5 break decide; no single piece-losing blunder. Both appear modest, but Game A Black's play was more consistent and lacked a one-move tactical collapse.", "confidence_valid": true }, { "model": "google/gemini-3-flash-preview", "pair": "pair_25", "game_a": "game_19", "color_a": "black", "game_b": "game_07", "color_b": "white", "rating_a": 2474, "rating_b": 1862, "gap": 612, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.6, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 2100, "old_estimate_b": 1750, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black demonstrates a reasonable understanding of the QGD Exchange structure, employing active king centralization (19... Kd6) and seeking counterplay in a strategically difficult endgame. Although Black ultimately loses to a skewer (35. Ra8+), the game reached a high degree of technical complexity. In contrast, Game B White plays a very passive opening and commits a basic tactical oversight by missing the 21... Bc2 fork, which immediately loses the exchange in a stable, quiet position. The nature of White's blunder in Game B, occurring without significant pressure, suggests a lower established rating compared to the more sophisticated defensive effort shown by Black in Game A.", "confidence_valid": true }, { "model": "google/gemini-3.6-flash", "pair": "pair_25", "game_a": "game_19", "color_a": "black", "game_b": "game_07", "color_b": "white", "rating_a": 2474, "rating_b": 1862, "gap": 612, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.65, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1750, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Player A (Black) demonstrates good opening familiarity in a Carlsbad Queen's Gambit Declined (6...Bf5) and plays logically against White's flank pawn storm, only collapsing in a difficult, already-losing endgame (34...Nxf5?). In contrast, Player B (White) plays passively in a symmetrical Slav structure and commits a basic tactical oversight in a balanced middlegame with 21. Bf3??, completely missing Black's 21...Bc2 which forks the queen and rook and immediately loses significant material. Player A shows significantly better baseline positional understanding and tactical awareness throughout the main phase of the game.", "confidence_valid": true }, { "model": "openai/gpt-5.6-sol", "pair": "pair_25", "game_a": "game_19", "color_a": "black", "game_b": "game_07", "color_b": "white", "rating_a": 2474, "rating_b": 1862, "gap": 612, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.62, "correct": 1, "brier": 0.1444, "old_estimate_a": 1800, "old_estimate_b": 1500, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black showed more sustained positional competence: 13...b5, 16...Rhg8, king centralization through ...Ke7-d6, and the timely 22...c5 created active counterplay in the queenless middlegame. Black ultimately mishandled White’s h-pawn and blundered with 34...Nxf5?, allowing 35.Ra8+ and 36.Rxh8. Game B White also played a coherent Slav structure, but 21.Bf3? permitted 21...Bc2, a straightforward double attack on Qd1 and Rb3; 22.Bh5 Bxd1 23.Bxg6 Bxb3 then left White decisively down material. A’s longer stretch of purposeful play slightly outweighs the comparable final tactical oversight.", "confidence_valid": true }, { "model": "openai/gpt-6-sol", "pair": "pair_25", "game_a": "game_19", "color_a": "black", "game_b": "game_07", "color_b": "white", "rating_a": 2474, "rating_b": 1862, "gap": 612, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.27, "correct": 0, "brier": 0.5328999999999999, "old_estimate_a": 1700, "old_estimate_b": 1650, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game B’s White developed coherently and pursued a clear queenside plan with 17.b4 and 19.b5, though 21.Bf3 and 22.Bh5 missed that ...Bc2 would let Black take the queen and then the rook on b3. Game A’s Black also played a reasonable opening, but allowed 22.Nxh5, made little progress against White’s advancing h-pawn, and overlooked 35.Ra8+ followed by 36.Rxh8. B’s play before the tactical mistake looks more convincing.", "confidence_valid": true }, { "model": "openai/gpt-6.1-sol", "pair": "pair_25", "game_a": "game_19", "color_a": "black", "game_b": "game_07", "color_b": "white", "rating_a": 2474, "rating_b": 1862, "gap": 612, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.65, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1800, "old_estimate_b": 1650, "baseline_choice": "A", "baseline_credit": 1, "explanation": "A Black showed more sustained positional and endgame competence: 22...c5 and 23...c4+ created counterplay, while ...Nf8–e6 and ...Nd8–b7–d6 improved the knight. However, 34...Nxf5 allowed 35.Ra8+, losing the rook and permitting promotion. B White developed sensibly and pursued a coherent minority attack with b4–b5, but mishandled the tactical threat after 21...Bc2: 22.Bh5 Bxd1 23.Bxg6 Bxb3 traded queens while losing a rook. Both made serious errors, but A’s longer stretch of purposeful play provides somewhat stronger evidence of established strength.", "confidence_valid": true }, { "model": "openai/gpt-5.6-luna", "pair": "pair_25", "game_a": "game_19", "color_a": "black", "game_b": "game_07", "color_b": "white", "rating_a": 2474, "rating_b": 1862, "gap": 612, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.82, "correct": 1, "brier": 0.03240000000000002, "old_estimate_a": 1550, "old_estimate_b": 1250, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A’s black player showed a coherent strategic plan: developing actively with ...Bf5, ...b5, ...Rac8, then using ...c5 and ...c4+ to create counterplay. Although Black later missed the rook-saving resource after 34.Ra7, the game remained competitive and featured meaningful endgame play. Game B’s white player made a much more decisive tactical oversight: after 21.Bf3 Bc2, 22.Bh5?? allowed ...Bxd1, losing the queen, and 23.Bxg6 was followed by ...Bxb3, losing another major piece. This indicates substantially weaker tactical awareness than Game A’s black player.", "confidence_valid": true }, { "model": "openai/gpt-6-astra", "pair": "pair_25", "game_a": "game_19", "color_a": "black", "game_b": "game_07", "color_b": "white", "rating_a": 2474, "rating_b": 1862, "gap": 612, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.64, "correct": 1, "brier": 0.1296, "old_estimate_a": 1750, "old_estimate_b": 1550, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black showed more sustained endgame coordination: 14...Ke7 and 19...Kd6 activated the king, while 22...c5 and 23...c4+ generated counterplay. Black nevertheless mishandled White’s passed h-pawn, and 34...Nxf5 allowed 35.Ra8+ and the loss of the rook. Game B White developed reasonably and pursued a coherent queenside expansion, but 21.Bf3 allowed 21...Bc2, attacking the queen and skewering the rook on b3. The resource 22.Bh5 avoided losing a queen outright but still conceded an exchange after 23...Bxb3. A’s longer stretch of purposeful play gives Black a modest edge, despite the eventual collapse.", "confidence_valid": true }, { "model": "z-ai/glm-5.3-flash", "pair": "pair_25", "game_a": "game_19", "color_a": "black", "game_b": "game_07", "color_b": "white", "rating_a": 2474, "rating_b": 1862, "gap": 612, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.72, "correct": 1, "brier": 0.07840000000000001, "old_estimate_a": 1650, "old_estimate_b": 1500, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A's Black handled the semi-Slav structure reasonably, exploiting White's 9.Qxf6?? blunder, showed decent endgame technique (22...c5, 23...c4+ creating counterplay), and only erred late (34...Nxf5 allowing 35.Ra8+ and 37.h7). Game B's White played weakly: 21.Bf3 allowed 21...Bc2, then 22.Bh5 Bxd1 23.Bxg6 Bxb3 simply dropped a full bishop, prompting resignation despite no catastrophic earlier mistake. Black's clean tactical conversion (Bc2/Bxd1/Bxb3) and White's piece-loss blunder suggest B's target is weaker than A's, who competed for ~37 moves against competent opposition.", "confidence_valid": true } ], "a": { "game": "game_19", "color": "black", "rating": 2474, "url": "https://lichess.org/ZAG2anr0", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"1-0\"]\n[TimeControl \"600+0\"]\n\n1. d4 Nf6 2. c4 e6 3. Nc3 d5 4. cxd5 exd5 5. Bg5 c6 6. e3 Bf5 7. Qf3 Bg6 8.\nBxf6 Qxf6 9. Qxf6 gxf6 10. h4 h5 11. Nh3 Bd6 12. Kd2 Nd7 13. Bd3 b5 14. Rac1\nKe7 15. Ne2 Rac8 16. Nhf4 Rhg8 17. Ng3 Bxf4 18. exf4 Bxd3 19. Kxd3 Kd6 20. Rhe1\nRce8 21. Rxe8 Rxe8 22. Nxh5 c5 23. Ng3 c4+ 24. Kd2 Nf8 25. h5 Ne6 26. Nf5+ Kd7\n27. g3 Rh8 28. h6 a5 29. b3 Nd8 30. bxc4 dxc4 31. d5 Nb7 32. Re1 Nd6 33. Re7+\nKd8 34. Ra7 Nxf5 35. Ra8+ Kd7 36. Rxh8 b4 37. h7 1-0" }, "b": { "game": "game_07", "color": "white", "rating": 1862, "url": "https://lichess.org/qtL5x6uE", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"0-1\"]\n[TimeControl \"600+0\"]\n\n1. d4 d5 2. c4 c6 3. e3 Bf5 4. Nf3 Nf6 5. Nbd2 h6 6. Be2 Nbd7 7. O-O e6 8. cxd5\nexd5 9. h3 Bd6 10. b3 O-O 11. a4 Ne4 12. Nxe4 Bxe4 13. Ba3 Bxa3 14. Rxa3 Nf6\n15. Ne5 Nd7 16. Nxd7 Qxd7 17. b4 a6 18. Rb3 Qe6 19. b5 axb5 20. axb5 Qg6 21.\nBf3 Bc2 22. Bh5 Bxd1 23. Bxg6 Bxb3 24. Bd3 0-1" }, "wrong": 2, "gap": 612 }, { "id": "pair_13", "predictions": [ { "model": "anthropic/claude-opus-5.5", "pair": "pair_13", "game_a": "game_22", "color_a": "white", "game_b": "game_04", "color_b": "white", "rating_a": 717, "rating_b": 2508, "gap": 1791, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.35, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1200, "old_estimate_b": 1450, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White opens with the unorthodox 2.Qf3 and 3.g4. It wins mainly because Black collapses: 6...Nxf2?? is unsound, and 22...Qxg1+?? gives away the queen. White's own play (13.Qxb5, 17.Qxc5, 21.Qxb7) just collects loose material, which says little about skill against a weak opponent. Game B White plays a coherent Exchange Caro-Kann setup (4.Bd3, 5.c3, Ne2, O-O) and a sensible queenside pawn storm (14.b4, 16.a4, 20.a5) against a competent opponent. It does err: 23.Qb2 lets Black win c5, and 24.Bb4? allows 24...Ne2+ 25.Kh1 Qg1+ 26.Rxg1 Nf2#, a smothered mate. Even so, B's overall understanding and the stronger opposition point to B being the stronger target.", "confidence_valid": true }, { "model": "anthropic/claude-sonnet-5.5", "pair": "pair_13", "game_a": "game_22", "color_a": "white", "game_b": "game_04", "color_b": "white", "rating_a": 717, "rating_b": 2508, "gap": 1791, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.4, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 1450, "old_estimate_b": 1500, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A: White opened 2.Qf3 and 3.g4, which are dubious. Black's 6...Nxf2 sacrifice and 8...Qf6+ gave Black a strong attack, yet White survived. Black then blundered with 13...Qf2+ and 16...cxd5, and White won with 17.Qxc5 and 21.Qxb7. This points to a weak game from both sides, and White's opening moves were poor. Game B: White played a reasonable Caro-Kann Exchange with solid development (Bd3, c3, Ne2, Qe1), but 10.f4 and 18.Nc5 gave Black chances. The decisive error was 21.c4 and 22.b5, which ignored Black's threats, and White lost to a knight-and-queen-sacrifice mate. White in B played more principled chess for most of the game, while White in A relied on opponent errors. B is slightly more likely stronger, but with low confidence.", "confidence_valid": true }, { "model": "google/gemini-3.8-flash", "pair": "pair_13", "game_a": "game_22", "color_a": "white", "game_b": "game_04", "color_b": "white", "rating_a": 717, "rating_b": 2508, "gap": 1791, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.08, "correct": 1, "brier": 0.0064, "old_estimate_a": 850, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A features very low-level, chaotic play: White plays dubious early queen moves like 2. Qf3 and 3. g4, allowing their king to wander to e3 on move 9, while both sides repeatedly hang material. In contrast, Game B shows solid positional opening knowledge in the Caro-Kann Exchange Variation (3. exd5, 4. Bd3, 5. c3) and coherent opposite-side castling plans. Although White in Game B blundered into a classic smothered mate tactic starting with 24. Bb4? allowing 24...Ne2+, their strategic understanding and move quality across the middlegame reflect a vastly higher established rating than White in Game A.", "confidence_valid": true }, { "model": "google/gemini-3.1-pro-preview", "pair": "pair_13", "game_a": "game_22", "color_a": "white", "game_b": "game_04", "color_b": "white", "rating_a": 717, "rating_b": 2508, "gap": 1791, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 950, "old_estimate_b": 1950, "baseline_choice": "B", "baseline_credit": 1, "explanation": "White A plays a nonsensical opening (2. Qf3, 3. g4) and exposes their king (9. Ke3), surviving only because Black inexplicably hangs pieces (13. Qxb5, 17. Qxc5). This indicates a beginner level. In contrast, White B plays a theoretical Caro-Kann Exchange, develops logically, and launches a credible queenside attack (14. b4, 20. a5). Although White B falls for a classic smothered mate trap (24...Ne2+ 25.Kh1 Qg1+ 26.Rxg1 Nf2#), their overall positional understanding and structural play are vastly superior to White A.", "confidence_valid": true }, { "model": "google/gemini-3.5-flash", "pair": "pair_13", "game_a": "game_22", "color_a": "white", "game_b": "game_04", "color_b": "white", "rating_a": 717, "rating_b": 2508, "gap": 1791, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.03, "correct": 1, "brier": 0.0009, "old_estimate_a": 1050, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White plays highly irregular, low-level chess, with an early queen excursion (2. Qf3), a premature pawn storm (3. g4), and walking their king to e3 by move 9. Both players commit severe errors. Conversely, Game B White demonstrates solid opening and positional understanding in the Caro-Kann Exchange, developing logically (4. Bd3, 6. Ne2) and organizing a queenside attack (14. b4, 20. a5). Although Game B White ultimately misses a tactical smothered mate sequence starting with 24...Ne2+, their level of play prior to this is vastly superior, indicating a much higher established rating.", "confidence_valid": true }, { "model": "google/gemini-3.7-flash", "pair": "pair_13", "game_a": "game_22", "color_a": "white", "game_b": "game_04", "color_b": "white", "rating_a": 717, "rating_b": 2508, "gap": 1791, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.08, "correct": 1, "brier": 0.0064, "old_estimate_a": 900, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A features chaotic, low-level opening play with 2. Qf3 and 3. g4 by White, followed by reckless piece sacrifices and mutual tactical blunders throughout a wild king hunt. In contrast, Game B displays coherent positional understanding in a Caro-Kann Exchange Variation, with White executing standard strategic concepts like queenside expansion (14. b4, 15. Nb3, 16. a4). Although White in Game B blundered into a classic smothered mate sequence starting with 24. Bb4?? Ne2+, their overall opening preparation, pawn structure handling, and strategic coordination indicate a much higher established rating than Game A's White.", "confidence_valid": true }, { "model": "qwen/qwen3.8-max-prime", "pair": "pair_13", "game_a": "game_22", "color_a": "white", "game_b": "game_04", "color_b": "white", "rating_a": 717, "rating_b": 2508, "gap": 1791, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "A", "probability_a": 0.66, "correct": 0, "brier": 0.43560000000000004, "old_estimate_a": 1100, "old_estimate_b": 1300, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White survived a chaotic opening (6...Nxf2 7.Kxf2) and converted material with accurate tactics: 13.Qxb5, 17.Qxc5, 22.Rg1 repulsing 21...Qg5. The win against speculative Black shows practical strength. Game B White chose a passive Exchange Caro-Kann (4.Bd3, 5.c3, 8.Qe1), drifted after 13...O-O-O, and lost to a standard smothered pattern (24...Ne2+ 25.Kh1 Qg1+ 26.Rxg1 Nf2#) without counterplay. A's active resourcefulness exceeds B's generic play, though A's 2.Qf3/3.g4 opening was also dubious.", "confidence_valid": true }, { "model": "anthropic/claude-fable-5.1", "pair": "pair_13", "game_a": "game_22", "color_a": "white", "game_b": "game_04", "color_b": "white", "rating_a": 717, "rating_b": 2508, "gap": 1791, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.3, "correct": 1, "brier": 0.09, "old_estimate_a": 1000, "old_estimate_b": 1450, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White played 2.Qf3 and 3.g4 against the Alekhine, then walked the king to e3 after 6...Nxf2; the win came only because Black (Game A) blundered pieces with 11...Nb5+?, 13...Qf2+ and 21...Qg5?? dropping the queen. Both A players look weak. Game B White played a coherent Caro-Kann Exchange setup (4.Bd3, 5.c3, 6.Ne2, 7.O-O), a sensible queenside pawn storm (14.b4, 16.a4, 20.a5, 22.b5), and lost to a sharp smothered-mate combination (24...Ne2+, 25...Qg1+) by a competent opponent. Game B White's structured play and stronger opposition suggest a higher rating than A's chaotic opening.", "confidence_valid": true }, { "model": "google/gemini-3-flash-preview", "pair": "pair_13", "game_a": "game_22", "color_a": "white", "game_b": "game_04", "color_b": "white", "rating_a": 717, "rating_b": 2508, "gap": 1791, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.15, "correct": 1, "brier": 0.0225, "old_estimate_a": 900, "old_estimate_b": 1950, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White shows a lack of fundamental principles, developing the queen prematurely (2. Qf3), pushing flank pawns (3. g4), and walking the king to the center early (9. Ke3) in a chaotic sequence. In contrast, Game B White plays a standard Exchange Caro-Kann with logical development (4. Bd3, 6. Ne2, 12. Nd2) and positional awareness, establishing a strong outpost with 18. Nc5. Although Game B White loses to a classic smothered mate tactic (24. Bb4?? Ne2+), their overall opening knowledge and structural play suggest a significantly higher established rating than Game A White.", "confidence_valid": true }, { "model": "google/gemini-3.6-flash", "pair": "pair_13", "game_a": "game_22", "color_a": "white", "game_b": "game_04", "color_b": "white", "rating_a": 717, "rating_b": 2508, "gap": 1791, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1100, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White displays a clear lack of basic opening and tactical principles, playing dubious moves like 2. Qf3, 3. g4, and moving the king to e3 on move 9. Game A White wins only because Black blunders the queen multiple times (17...Bf5 missing 17...Qxc5, and 22...Qxg1+). In contrast, Game B White demonstrates solid positional understanding in a standard Caro-Kann Exchange Variation, executing a coherent queenside pawn storm with 14. b4, 15. Nb3, and 16. a4. Although Game B White falls for a classic smothered mate tactic starting with 24...Ne2+, Game B White shows vastly superior established playing strength compared to Game A White.", "confidence_valid": true }, { "model": "openai/gpt-5.6-sol", "pair": "pair_13", "game_a": "game_22", "color_a": "white", "game_b": "game_04", "color_b": "white", "rating_a": 717, "rating_b": 2508, "gap": 1791, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.08, "correct": 1, "brier": 0.0064, "old_estimate_a": 1200, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B White showed much more coherent opening and strategic play: normal development, castling, and a purposeful queenside expansion with 14.b4, 16.a4, 18.Nc5, and 22.b5. The decisive error was tactical—24.Bb4? allowed 24...Ne2+ and the forced mating pattern. Game A White played extremely unsoundly with 2.Qf3, 3.g4, 6.h3, and an exposed king walk to e3/d1, succeeding mainly because Black repeatedly hung material, notably 13...Qf2+ allowing 17.Qxc5 and 22...Qxg1+ losing the queen to 23.Nxg1. B is therefore much likelier to have had the stronger established rapid level.", "confidence_valid": true }, { "model": "openai/gpt-6-sol", "pair": "pair_13", "game_a": "game_22", "color_a": "white", "game_b": "game_04", "color_b": "white", "rating_a": 717, "rating_b": 2508, "gap": 1791, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.2, "correct": 1, "brier": 0.04000000000000001, "old_estimate_a": 1350, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s White showed more coherent play: 7.O-O completed development, and 14.b4 and 20.a5 pursued a queenside plan. White then missed the mating threat with 24.Bb4, but that single oversight is less telling than Game A’s White playing 2.Qf3 and 3.g4, then exposing the king with 9.Ke3. Game A’s win also relied heavily on Black leaving pieces available to 13.Qxb5 and 17.Qxc5.", "confidence_valid": true }, { "model": "openai/gpt-6.1-sol", "pair": "pair_13", "game_a": "game_22", "color_a": "white", "game_b": "game_04", "color_b": "white", "rating_a": 717, "rating_b": 2508, "gap": 1791, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.94, "correct": 1, "brier": null, "old_estimate_a": 900, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B White shows substantially more coherent development and planning: 4.Bd3, 6.Ne2 and 7.O-O, followed by the coordinated queenside expansion 14.b4, 15.Nb3 and 18.Nc5. White ultimately overlooks the mating combination beginning 24...Ne2+, but the preceding play suggests stronger established understanding. Game A White’s 2.Qf3, 3.g4 and 4.g5 neglect development and king safety; 6.h3 allows ...Nxf2, forcing an exposed king walk. A’s win relies heavily on Black’s poor attack and especially 22...Qxg1+, surrendering the queen for a rook, rather than demonstrating comparable overall strength.", "confidence_valid": false }, { "model": "openai/gpt-5.6-luna", "pair": "pair_13", "game_a": "game_22", "color_a": "white", "game_b": "game_04", "color_b": "white", "rating_a": 717, "rating_b": 2508, "gap": 1791, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.35, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1200, "old_estimate_b": 1500, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s White shows a much more coherent opening and positional plan: 1.e4 c6, 4.Bd3, 7.O-O, and the queenside expansion with 14.b4 and 20.a5. However, the game ends with a severe tactical oversight: 24.Bb4? Ne2+ 25.Kh1 Qg1+ 26.Rxg1 Nf2#. Game A’s White plays very dubious early moves, especially 2.Qf3 and 3.g4, and exposes the king with 7.Kxf2 and 9.Ke3, but consistently exploits Black’s hanging pieces, notably 13.Qxb5 and 17.Qxc5. Overall B appears stronger despite the decisive blunder.", "confidence_valid": true }, { "model": "openai/gpt-6-astra", "pair": "pair_13", "game_a": "game_22", "color_a": "white", "game_b": "game_04", "color_b": "white", "rating_a": 717, "rating_b": 2508, "gap": 1791, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.03, "correct": 1, "brier": 0.0009, "old_estimate_a": 1000, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B White showed substantially stronger opening principles and planning: 4.Bd3, 5.c3, 6.Ne2 and 7.O-O developed coherently, followed by queenside expansion against Black’s long-castled king and the thematic 18.Nc5. White ultimately missed the mating combination 24...Ne2+ 25.Kh1 Qg1+ 26.Rxg1 Nf2#, but Black’s coordinated attack was considerably more demanding. Game A White’s 2.Qf3, 3.g4 and 4.g5 neglected development and exposed the king, which reached e3 by move nine. A’s eventual win involved collecting loose pieces and benefited from Black’s much weaker play, including 22...Qxg1+ allowing 23.Nxg1.", "confidence_valid": true }, { "model": "z-ai/glm-5.3-flash", "pair": "pair_13", "game_a": "game_22", "color_a": "white", "game_b": "game_04", "color_b": "white", "rating_a": 717, "rating_b": 2508, "gap": 1791, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.6, "correct": 1, "brier": null, "old_estimate_a": 1450, "old_estimate_b": 1450, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "Game B's White showed coherent positional chess: normal Caro-Kann structure, safe O-O, planned a4/b5 expansion, and sensible trades (11.Qxe2, 18.Nc5). The loss came mainly from one late blunder 24.Bb4?? overlooking the smothered mate after 24...Ne2+ 25.Kh1 Qg1+ 26.Rxg1 Nf2#. Game A's White played chaotic opening moves (2.Qf3, 4.g4) and merely profited from Black's absurd blunders (6...Nxf2?, 7...h6? allowing gxh6/Qf6+), with the win largely gifted. B's target demonstrated stronger fundamental play, so likely higher rating despite losing.", "confidence_valid": false } ], "a": { "game": "game_22", "color": "white", "rating": 717, "url": "https://lichess.org/5mUXnOAF", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"1-0\"]\n[TimeControl \"600+0\"]\n\n1. e4 Nf6 2. Qf3 e5 3. g4 Nc6 4. g5 Nd4 5. Qd3 Ng4 6. h3 Nxf2 7. Kxf2 h6 8.\ngxh6 Qf6+ 9. Ke3 Bc5 10. hxg7 Qxg7 11. c3 Nb5+ 12. Ke2 Qf6 13. Qxb5 Qf2+ 14.\nKd1 c6 15. Qc4 d5 16. exd5 cxd5 17. Qxc5 Bf5 18. Qxd5 Ke7 19. Bb5 Kf6 20. Ne2\nQf4 21. Qxb7 Qg5 22. Rg1 Qxg1+ 23. Nxg1 1-0" }, "b": { "game": "game_04", "color": "white", "rating": 2508, "url": "https://lichess.org/At0NmiOd", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"0-1\"]\n[TimeControl \"600+0\"]\n\n1. e4 c6 2. d4 d5 3. exd5 cxd5 4. Bd3 Nc6 5. c3 Qc7 6. Ne2 Bg4 7. O-O e6 8. Qe1\nNf6 9. Bg5 Bh5 10. f4 Bxe2 11. Qxe2 Be7 12. Nd2 h6 13. Bh4 O-O-O 14. b4 Kb8 15.\nNb3 Rc8 16. a4 Ne4 17. Be1 f5 18. Nc5 Bxc5 19. dxc5 g5 20. a5 a6 21. c4 gxf4\n22. b5 Nd4 23. Qb2 Qxc5 24. Bb4 Ne2+ 25. Kh1 Qg1+ 26. Rxg1 Nf2# 0-1" }, "wrong": 1, "gap": 1791 }, { "id": "pair_11", "predictions": [ { "model": "anthropic/claude-opus-5.5", "pair": "pair_11", "game_a": "game_21", "color_a": "black", "game_b": "game_10", "color_b": "black", "rating_a": 743, "rating_b": 2501, "gap": 1758, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.25, "correct": 1, "brier": 0.0625, "old_estimate_a": 1300, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black played a dubious opening that dropped pawns: 3...Qe7 and 4...f6 allowed 5.Qxd5 and 7.Qxb7. Black then let a knight go with 26...Nd5 and 28...Ng4, allowing 29.exd5. The rook-versus-knight ending was also handled slowly, and White blundered constantly, so the result says little. Game B Black played a coherent Queen's Gambit-style setup with ...Nbd7, ...c5 and ...b6/Bb7. Black kept the structure sound and found active ideas: 24...Qc2 hit the rook and 25...Ba6 won the exchange (26...Bxf1). This shows tactical awareness against a competent opponent. B's play looks considerably more controlled.", "confidence_valid": true }, { "model": "anthropic/claude-sonnet-5.5", "pair": "pair_11", "game_a": "game_21", "color_a": "black", "game_b": "game_10", "color_b": "black", "rating_a": 743, "rating_b": 2501, "gap": 1758, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.35, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1500, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black met 2.Qd3 with 2...e5 and 4...f6, which left Black down a pawn after 7.Qxb7. Black then found 7...Qb4+ and 8...Bxb4+, followed by 10...Bxc2, which won back material and gave Black activity. Black won a long endgame. White's play was weak, and Black's conversion was uneven, with a long rook-and-knight ending that took until move 69. Game B Black played a solid, principled Queen's Gambit-style setup (Nbd7, Be7, ...c5, ...b6, ...Bb7). Black played 12...Nxc5, 15...Ne6, and 17...Bd6 sensibly. Black then tactically punished White's 24.Nxa7 with 24...Qc2 and 26...Bxf1, winning quickly. Black in B looks more coherent and cleaner, though White in B also blundered. Game A's opponent also played poorly, and Black's play there was shakier. I lean B.", "confidence_valid": true }, { "model": "google/gemini-3.8-flash", "pair": "pair_11", "game_a": "game_21", "color_a": "black", "game_b": "game_10", "color_b": "black", "rating_a": 743, "rating_b": 2501, "gap": 1758, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.04, "correct": 1, "brier": 0.0016, "old_estimate_a": 1300, "old_estimate_b": 2150, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B demonstrates high-level, mature positional chess and sharp tactics. Black in Game B navigated a classical Catalan/English structure cleanly, coordinated pieces purposefully, and executed a decisive tactical finish starting with 24...Qc2!, culminating in 27...Ne8!, leaving White completely lost. In stark contrast, Game A features chaotic, low-rated beginner/intermediate play: Black played the unsound 2...e5, 3...Qe7, and 4...f6, blundering early on (allowing 7. Qxb7), followed by both sides missing simple tactical themes (e.g., White dropping the h1-rook to ...Nxh1). Black in Game B plays with vastly superior chess understanding.", "confidence_valid": true }, { "model": "google/gemini-3.1-pro-preview", "pair": "pair_11", "game_a": "game_21", "color_a": "black", "game_b": "game_10", "color_b": "black", "rating_a": 743, "rating_b": 2501, "gap": 1758, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1100, "old_estimate_b": 2150, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B Black demonstrates a solid understanding of positional chess, smoothly navigating a complex Reti/Tarrasch structure and executing a sharp tactical sequence (24... Qc2!). In contrast, Game A Black plays dubiously in the opening (2... e5?!), blunders a pawn with 6... Bf5 allowing 7. Qxb7, and relies on basic knight forks against an equally weak opponent in a chaotic, low-level game. Target B is significantly stronger.", "confidence_valid": true }, { "model": "google/gemini-3.5-flash", "pair": "pair_11", "game_a": "game_21", "color_a": "black", "game_b": "game_10", "color_b": "black", "rating_a": 743, "rating_b": 2501, "gap": 1758, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1200, "old_estimate_b": 2000, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A features a chaotic, lower-rated game. Black plays a highly questionable opening with 2...e5 and 3...Qe7, which blocks development, and the game proceeds with numerous tactical and positional inconsistencies from both sides. In contrast, Game B shows a high level of theoretical understanding and positional control from Black. Black navigates a standard English Defence setup flawlessly, developing logically, maintaining a solid pawn structure, and capitalizing on White's inaccuracies with sharp tactical precision starting with 24...Qc2, 25...Ba6, and 27...Ne8, forcing an immediate resignation. Player B is clearly the more established and stronger player.", "confidence_valid": true }, { "model": "google/gemini-3.7-flash", "pair": "pair_11", "game_a": "game_21", "color_a": "black", "game_b": "game_10", "color_b": "black", "rating_a": 743, "rating_b": 2501, "gap": 1758, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.08, "correct": 1, "brier": 0.0064, "old_estimate_a": 1200, "old_estimate_b": 2150, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B Black demonstrates significantly higher chess understanding, playing a classical, principled setup against White's English Opening (1...e6, 4...Nbd7, 7...c5). Black coordinated their pieces harmoniously in the middlegame and executed a decisive tactical sequence with 24...Qc2!, exploiting the fork on d1 and b2, followed cleanly by 25...Ba6 and 27...Ne8. In contrast, Game A Black plays chaotic and suboptimal chess from the opening (2...e5, 3...Qe7, 4...f6), dropping pawns and allowing Queen tactics (7.Qxb7), surviving only through White's blunders in an erratic low-level endgame.", "confidence_valid": true }, { "model": "qwen/qwen3.8-max-prime", "pair": "pair_11", "game_a": "game_21", "color_a": "black", "game_b": "game_10", "color_b": "black", "rating_a": 743, "rating_b": 2501, "gap": 1758, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.18, "correct": 1, "brier": 0.0324, "old_estimate_a": 1150, "old_estimate_b": 1400, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A's Black wins but against a very weak opponent (White's 2.Qd3, 4.f4, and 10.Rxc1 blundering Bxc2). Black plays solidly but nothing exceptional; the win stems from White's opening disaster. Game B's Black demonstrates stronger strategic play: 12...Nxc5 recapture, 15...Ne6 redeployment, 18...Rf8 prophylaxis, and the decisive 23...Qc2 and 26...Bxf1 tactics exploiting White's overextension (24.Nxa7). Game B's opponent played a coherent opening and middlegame, making Black's clean conversion more impressive evidence of higher established strength.", "confidence_valid": true }, { "model": "anthropic/claude-fable-5.1", "pair": "pair_11", "game_a": "game_21", "color_a": "black", "game_b": "game_10", "color_b": "black", "rating_a": 743, "rating_b": 2501, "gap": 1758, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.3, "correct": 1, "brier": 0.09, "old_estimate_a": 1000, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black won a very weak opponent's gambit (2.Qd3, 5.Qxd5 losing material) but then played erratically: 23...Nxg2 and 24...Nf4 followed by 25...Bxa4?? and 27...Nh6 lost pieces, and the knight on h1 was stranded for 30+ moves in a sloppy endgame. Game B Black handled a mainstream Catalan/Réti setup soundly (7...c5, 11...Rc8, 15...Ne6), then found 23...Bc5 and 24...Qc2 25...Ba6 winning the exchange with a crushing attack, forcing resignation after 27...Ne8. The opponent in B also played a coherent, higher-level game. B's play is markedly more consistent and tactically sharp.", "confidence_valid": true }, { "model": "google/gemini-3-flash-preview", "pair": "pair_11", "game_a": "game_21", "color_a": "black", "game_b": "game_10", "color_b": "black", "rating_a": 743, "rating_b": 2501, "gap": 1758, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.15, "correct": 1, "brier": 0.0225, "old_estimate_a": 1300, "old_estimate_b": 2300, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B Black demonstrates significantly higher skill, playing a sophisticated English defense with sound development (10... Bb7, 14... Rfe8). Black's tactical execution is precise, notably 24... Qc2! which wins material by targeting the loose rook and bishop. In contrast, Game A Black plays an erratic opening (3... Qe7, 4... f6) and succeeds largely because White blunders a piece (21. Kd2? allowing Nxf1+). The quality of calculation and opening knowledge in Game B suggests a much higher established rating than the scrappy, blunder-prone play seen in Game A.", "confidence_valid": true }, { "model": "google/gemini-3.6-flash", "pair": "pair_11", "game_a": "game_21", "color_a": "black", "game_b": "game_10", "color_b": "black", "rating_a": 743, "rating_b": 2501, "gap": 1758, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1300, "old_estimate_b": 1950, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Player B demonstrates high positional understanding and opening knowledge in a standard English/Catalan setup (1...e6, 2...d5, 7...c5, 10...Bb7). Player B executed a decisive tactical sequence starting with 24...Qc2!, exploiting White's back rank and bishop on b2, leading to a quick victory. In contrast, Player A played very low-level opening moves (2...e5, 3...Qe7, 4...f6), giving away pawns early for no compensation against White's eccentric 2. Qd3. Although Player A eventually won after White blundered a rook and knight, Player B's clean, high-level tactical and strategic play indicates a much higher established rating.", "confidence_valid": true }, { "model": "openai/gpt-5.6-sol", "pair": "pair_11", "game_a": "game_21", "color_a": "black", "game_b": "game_10", "color_b": "black", "rating_a": 743, "rating_b": 2501, "gap": 1758, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.08, "correct": 1, "brier": 0.0064, "old_estimate_a": 1450, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s Black showed much stronger positional coherence: sound development through 14...Rfe8, purposeful pressure with 18...Rf8 and 19...Qf7, then accurate tactics after 22.Rxc8 Bxc8. The sequence 24...Qc2, 25...Ba6 and 26...Bxf1 decisively exploited White’s loose back rank. Game A’s Black found good ideas such as 7...Qb4+ and the knight incursion 19...Nc2+–23...Nxg2, but the opening was highly irregular and the large advantage required a long, meandering conversion, with the knight stranded on h1 and many inefficient rook maneuvers. B looks substantially more established.", "confidence_valid": true }, { "model": "openai/gpt-6-sol", "pair": "pair_11", "game_a": "game_21", "color_a": "black", "game_b": "game_10", "color_b": "black", "rating_a": 743, "rating_b": 2501, "gap": 1758, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.04, "correct": 1, "brier": 0.0016, "old_estimate_a": 1450, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s Black played a coherent opening against a normally developing opponent, then found active moves: 24...Qc2, 25...Ba6 and 26...Bxf1 won a rook, while 27...Ne8 attacked White’s queen. Game A’s Black exploited major errors by an opponent who brought the queen out on move 2, but chose a risky early ...f6 and later allowed 29.exd5 to take a knight. B’s play is stronger evidence of established rapid strength.", "confidence_valid": true }, { "model": "openai/gpt-6.1-sol", "pair": "pair_11", "game_a": "game_21", "color_a": "black", "game_b": "game_10", "color_b": "black", "rating_a": 743, "rating_b": 2501, "gap": 1758, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.96, "correct": 1, "brier": null, "old_estimate_a": 1250, "old_estimate_b": 2050, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B Black shows coherent development and purposeful coordination: 14...Rfe8, 17...Bd6 and 19...Qf7 improve the position, while 23...Bc5, 24...Qc2 and 25...Ba6 exploit White’s pawn grab to win the exchange with 26...Bxf1. Game A Black’s opening, especially 2...e5 and 3...Qe7, allows substantial pawn losses without convincing compensation. Although 19...Nc2+ and 20...Ne3+ exploit White’s mistakes effectively, 25...Bxa4 simply permits 26.bxa4, surrendering a bishop. A’s eventual victory relies much more heavily on the opponent’s repeated tactical errors; B’s play is substantially more consistent and sophisticated.", "confidence_valid": false }, { "model": "openai/gpt-5.6-luna", "pair": "pair_11", "game_a": "game_21", "color_a": "black", "game_b": "game_10", "color_b": "black", "rating_a": 743, "rating_b": 2501, "gap": 1758, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.68, "correct": 1, "brier": null, "old_estimate_a": 1500, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s black showed cleaner, more coherent opening development and strong tactical awareness: after 22 Rxc8, Bxc8 preserved activity, and 24 Nxa7 Qc2 followed by 25...Ba6 and 26...Bxf1 won White’s rook through a clear tactical sequence. Game A’s black was resourceful and ultimately converted actively, including 7...Qb4+ and 21...Nxf1+, but the opening with 2...e5, 4...f6, and allowing 5 Qxd5 and 7 Qxb7 was strategically risky. The long game also featured considerable maneuvering and depended on substantial white inaccuracies. B therefore more likely had the higher established rapid strength.", "confidence_valid": false }, { "model": "openai/gpt-6-astra", "pair": "pair_11", "game_a": "game_21", "color_a": "black", "game_b": "game_10", "color_b": "black", "rating_a": 743, "rating_b": 2501, "gap": 1758, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.97, "correct": 1, "brier": null, "old_estimate_a": 1250, "old_estimate_b": 2000, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B Black shows coherent development and purposeful coordination: 14...Rfe8, 17...Bd6 and 18...Rf8 build activity, while 24...Qc2 and 25...Ba6 exploit White’s exposed rook to win the exchange with 26...Bxf1. This comes against substantially more organized opposition. Game A Black finds the useful 19...Nc2+–21...Nxf1 maneuver, but later simply loses a bishop with 25...Bxa4 26.bxa4. The eventual victory depends heavily on White’s further errors, notably 39.Ne5+ allowing ...Rxe5. B’s play is much stronger evidence of established playing strength.", "confidence_valid": false }, { "model": "z-ai/glm-5.3-flash", "pair": "pair_11", "game_a": "game_21", "color_a": "black", "game_b": "game_10", "color_b": "black", "rating_a": 743, "rating_b": 2501, "gap": 1758, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "A", "probability_a": 0.75, "correct": 0, "brier": 0.5625, "old_estimate_a": 1450, "old_estimate_b": 1700, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A's Black won material early with 5...fxe5/6...Bf5, exploited tactics (17...Nb4, 19...Nc2+ winning the knight via fork ideas), then converted a long R+N vs N endgame with solid technique: king activity (44...Kb7-a6), the...", "confidence_valid": true } ], "a": { "game": "game_21", "color": "black", "rating": 743, "url": "https://lichess.org/NxjNr7LI", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"0-1\"]\n[TimeControl \"600+0\"]\n\n1. d4 d5 2. Qd3 e5 3. dxe5 Qe7 4. f4 f6 5. Qxd5 fxe5 6. fxe5 Bf5 7. Qxb7 Qb4+\n8. Qxb4 Bxb4+ 9. Bd2 Bxd2+ 10. Nxd2 Bxc2 11. Rc1 Ba4 12. Rxc7 Na6 13. Rc4 Bb5\n14. Rd4 Rd8 15. Rxd8+ Kxd8 16. Nb3 Ba4 17. Nc1 Nb4 18. b3 Bb5 19. a3 Nc2+ 20.\nKd1 Ne3+ 21. Kd2 Nxf1+ 22. Ke1 Ne3 23. Kd2 Nxg2 24. a4 Nf4 25. e3 Bxa4 26. bxa4\nNd5 27. Nb3 Nh6 28. e4 Ng4 29. exd5 Nf2 30. Ke2 Nxh1 31. Nf3 Re8 32. Ng5 h6 33.\nNf7+ Ke7 34. Nd6 Rd8 35. Nb5 Rxd5 36. Nxa7 Rxe5+ 37. Kd3 Rh5 38. Nc6+ Kd7 39.\nNe5+ Rxe5 40. Kd4 Re2 41. h4 Rh2 42. Nc5+ Kc6 43. Ne4 Rxh4 44. a5 Kb7 45. Kd5\nKa6 46. Nc5+ Kxa5 47. Nd7 Rh2 48. Ne5 Rd2+ 49. Kc4 Ka4 50. Nf7 Rc2+ 51. Kd3 Kb3\n52. Ke4 Rc3 53. Kf5 Re3 54. Kg6 h5 55. Kxg7 Rg3+ 56. Kh6 Rh3 57. Ne5 Rh2 58.\nNf3 Rh3 59. Ng5 Rh2 60. Nf3 Rf2 61. Ng5 Ng3 62. Nh3 Rh2 63. Nf4 h4 64. Kg5 h3\n65. Kg4 Ne4 66. Nxh3 Nf2+ 67. Kg3 Rxh3+ 68. Kxf2 Rd3 69. Ke2 Kc2 0-1" }, "b": { "game": "game_10", "color": "black", "rating": 2501, "url": "https://lichess.org/ghN8Hbdl", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"0-1\"]\n[TimeControl \"600+0\"]\n\n1. c4 e6 2. Nf3 d5 3. g3 Nf6 4. Bg2 Nbd7 5. O-O Be7 6. b3 O-O 7. Bb2 c5 8. cxd5\nexd5 9. d4 b6 10. Nc3 Bb7 11. Rc1 Rc8 12. dxc5 Nxc5 13. Nd4 Qd7 14. Qd2 Rfe8\n15. Qf4 Ne6 16. Nxe6 fxe6 17. Rfd1 Bd6 18. Qh4 Rf8 19. e3 Qf7 20. Nb5 Be7 21.\nQf4 Qg6 22. Rxc8 Bxc8 23. Qc7 Bc5 24. Nxa7 Qc2 25. Rf1 Ba6 26. Be5 Bxf1 27.\nBxf1 Ne8 0-1" }, "wrong": 1, "gap": 1758 }, { "id": "pair_23", "predictions": [ { "model": "anthropic/claude-opus-5.5", "pair": "pair_23", "game_a": "game_25", "color_a": "white", "game_b": "game_18", "color_b": "white", "rating_a": 738, "rating_b": 2426, "gap": 1688, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.3, "correct": 1, "brier": 0.09, "old_estimate_a": 1100, "old_estimate_b": 1450, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A is very low-level on both sides. White's 3.Qf3, 4.g4 and 6.h4 neglect development, and 13.Nh3 allowed 13...Nc2+ forking king and rook. White won mainly because Black blundered repeatedly, e.g. 16...Bxh3 and the later collapse of Black's king. Game B's White played a coherent double-fianchetto setup, including 8.d4, 9.Bb2 and 13.Bd4, and handled a sharp middlegame reasonably. White then collapsed with 32.Qxd5?? into ...Qh3+ and ...Qxg3+, a serious error. Even so, Game B's overall structure and the opponent's quality suggest a stronger level of play than Game A's chaotic game.", "confidence_valid": true }, { "model": "anthropic/claude-sonnet-5.5", "pair": "pair_23", "game_a": "game_25", "color_a": "white", "game_b": "game_18", "color_b": "white", "rating_a": 738, "rating_b": 2426, "gap": 1688, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.35, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1550, "old_estimate_b": 1550, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "Game A: White played the opening poorly (2.e3 and 4.g4 gambits, 5.h3 and 6.h4 pawn-pushing, 3.Qf3 early queen). Black's 13...Nc2+ and 16...Bxh3 were strong, and White was probably losing around move 20. White won only because Black blundered, e.g. 23...Qc8 and 24...Rxc3+. The result suggests a low-rated, chaotic game. Game B: White played a solid Sicilian setup with 3.b3, 4.g3 and 7.O-O, and the moves were reasonable, though 10.Re1+ and 14.c4 led to a worse position. The loss came from tactics: 31.Kf1? and 32.Qxd5? allowed 32...Qh3+ and 33...Qxg3+. White's opening play was more principled in B, so B is likely stronger, but the evidence is weak and the margin small.", "confidence_valid": true }, { "model": "google/gemini-3.8-flash", "pair": "pair_23", "game_a": "game_25", "color_a": "white", "game_b": "game_18", "color_b": "white", "rating_a": 738, "rating_b": 2426, "gap": 1688, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1350, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Player B exhibits solid positional understanding and coherent opening play (2. Nf3, 3. b3, 4. g3, 7. O-O), followed by purposeful piece activity and central play, only collapsing tactically late with 30. Rd1? allowing the decisive ...Re2+. In contrast, Player A plays like a beginner: violating opening principles with 3. Qf3 and an absurd flurry of flank pawn pushes (4. g4, 5. h3, 6. h4, 7. g5, 8. h5), while completely neglecting development and overlooking the elementary fork 13...Nc2+ dropping a full rook. Despite Black later blundering in Game A, Player A's overall grasp of chess fundamentals is vastly inferior to Player B's.", "confidence_valid": true }, { "model": "google/gemini-3.1-pro-preview", "pair": "pair_23", "game_a": "game_25", "color_a": "white", "game_b": "game_18", "color_b": "white", "rating_a": 738, "rating_b": 2426, "gap": 1688, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1100, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White shows a lack of opening fundamentals with bizarre early moves like 3. Qf3, 4. g4, and pushing flank pawns to h5 and g5, later falling for a simple knight fork with 13... Nc2+. In contrast, Game B White demonstrates a solid understanding of opening principles, piece development, and structural play against the Sicilian Defense (e.g., fianchetto, challenging the center with d4). Although Game B White blunders at the end with 32. Qxd5 allowing a mating net, their overall positional understanding and tactical awareness throughout the game indicate a significantly higher rating than Game A White.", "confidence_valid": true }, { "model": "google/gemini-3.5-flash", "pair": "pair_23", "game_a": "game_25", "color_a": "white", "game_b": "game_18", "color_b": "white", "rating_a": 738, "rating_b": 2426, "gap": 1688, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1100, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "In Game A, White's opening play is very weak, including premature queen moves (3. Qf3) and random pawn pushes (4. g4, 6. h4), quickly blundering a rook to a basic fork with 13. Nh3 Nc2+. In contrast, Game B's White plays a coherent and logical game, employing a recognized sideline (3. b3) against the Sicilian, developing pieces harmoniously (7. O-O, 10. Re1+), and executing sound plans. While Game B's White eventually blunders the king's safety with 30. Rd1?, their overall strategic and tactical competence is vastly superior to the beginner-level play shown by White in Game A.", "confidence_valid": true }, { "model": "google/gemini-3.7-flash", "pair": "pair_23", "game_a": "game_25", "color_a": "white", "game_b": "game_18", "color_b": "white", "rating_a": 738, "rating_b": 2426, "gap": 1688, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.15, "correct": 1, "brier": 0.0225, "old_estimate_a": 1200, "old_estimate_b": 1950, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White shows weak opening fundamentals with random flank pawn pushes (3. Qf3, 4. g4, 6. h4, 8. h5) and outright blundered a full rook on move 13 (13. Nh3 Nc2+), winning only because Black repeatedly blundered back. In contrast, Game B White plays a coherent opening with standard piece development and central control (3. b3, 5. Bg2, 7. O-O, 8. d4). Even though Game B White eventually lost due to tactical oversights in time trouble (30. Rd1 Re2+ 31. Kf1 Rh2), the overall strategic and tactical comprehension is significantly higher than Game A White's.", "confidence_valid": true }, { "model": "qwen/qwen3.8-max-prime", "pair": "pair_23", "game_a": "game_25", "color_a": "white", "game_b": "game_18", "color_b": "white", "rating_a": 738, "rating_b": 2426, "gap": 1688, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "A", "probability_a": 0.82, "correct": 0, "brier": 0.6723999999999999, "old_estimate_a": 1350, "old_estimate_b": 1300, "baseline_choice": "A", "baseline_credit": 0, "explanation": "Game A's White faced absurd openings (2…Bd7, 3…c6, 4…b5) yet played coherently, winning material cleanly (18.Bxa1, 20.Bxc4) and converting with 30.Rd7+, 38.Qd5+, 39.Rxd8#. Game B's White made real errors: 11.g4? weakening the kingside, 16.hxg3 allowing 16…Bh3, and losing after 30…Re2+ 31.Kf1 Rh2 32.Qxd5 Qh3+ 33.Ke1 Qxg3#. B's White collapsed against a much more competent opponent, suggesting a substantially lower established rating than A's White.", "confidence_valid": true }, { "model": "anthropic/claude-fable-5.1", "pair": "pair_23", "game_a": "game_25", "color_a": "white", "game_b": "game_18", "color_b": "white", "rating_a": 738, "rating_b": 2426, "gap": 1688, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.3, "correct": 1, "brier": 0.09, "old_estimate_a": 1100, "old_estimate_b": 1550, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White played a bizarre, unsound opening (2.e3, 3.Qf3, 4.g4, 5.h3, 6.h4, 7.g5, 8.h5), lost the exchange and rook on a1 to 13...Nc2+, and only won because Black blundered badly (23...Qc8?? losing the queen and later Rxc3+ for nothing). The opponent's play (2...Bd7, 3...c6, 4...b5, 5...Na6) indicates a very low-rated pairing. Game B White showed coherent development (b3/g3 fianchetto, 10.Re1+ Kf8, 11.Nxd4, 14.c4, 21.cxd5 c5) and was even better before 29.Qd4?? allowed 29...Re8 and 30...Re2+ with a losing attack. Despite losing, Game B White's overall move quality and opponent's Sicilian handling suggest a higher established rating than Game A's target.", "confidence_valid": true }, { "model": "google/gemini-3-flash-preview", "pair": "pair_23", "game_a": "game_25", "color_a": "white", "game_b": "game_18", "color_b": "white", "rating_a": 738, "rating_b": 2426, "gap": 1688, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1100, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White plays a primitive opening, making five pawn moves and an early queen sortie (3. Qf3) before developing any minor pieces. White misses a basic fork (13. Nh3 Nc2+), losing a full rook, and only wins because Black fails to convert. Conversely, Game B White exhibits sophisticated positional play in a 3. b3 Sicilian, prioritizing development (9. Bb2, 10. Re1+) and central control. Game B involves high-level themes like the h-pawn thrust and piece maneuvering. Although Game B White blunders at the end (32. Qxd5??), the technical proficiency and opening knowledge displayed are significantly higher than the disorganized play seen from Game A White.", "confidence_valid": true }, { "model": "google/gemini-3.6-flash", "pair": "pair_23", "game_a": "game_25", "color_a": "white", "game_b": "game_18", "color_b": "white", "rating_a": 738, "rating_b": 2426, "gap": 1688, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.15, "correct": 1, "brier": 0.0225, "old_estimate_a": 1150, "old_estimate_b": 1700, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White shows extremely weak opening fundamentals, pushing flank pawns recklessly with 4. g4, 6. h4, 7. g5, and 8. h5 while neglecting piece development, and blundering a full rook with 13. Nh3 Nc2+. Although White eventually won after Black blundered back, the initial quality of play indicates a much lower rating. Game B White plays a sound, disciplined opening (Snyder Sicilian), castles early, and manages a complex positional battle logically up to move 29. Even though Game B White loses to a late-game tactical miscalculation (30. Rd1 Re2+), their overall chess understanding and move quality are vastly superior to Game A White.", "confidence_valid": true }, { "model": "openai/gpt-5.6-sol", "pair": "pair_23", "game_a": "game_25", "color_a": "white", "game_b": "game_18", "color_b": "white", "rating_a": 738, "rating_b": 2426, "gap": 1688, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.1, "correct": 1, "brier": 0.010000000000000002, "old_estimate_a": 1300, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B White showed substantially more coherent development and positional play: 7.O-O, 8.d4, 9.Bb2, 14.c4, and 23.Rc1 followed recognizable plans, while 27.Bxc5+ and 28.Qxc3 handled tactics sensibly. The late 32.Qxd5? allowed ...Qh3+ and ...Qxg3+, but it was one tactical failure. Game A White’s 3.Qf3, 4.g4, repeated h-pawn moves, and 13.Nh3? allowed ...Nc2+ and ...Nxa1, indicating much weaker fundamentals. A’s eventual mating attack, including 30.Rd7+ and 34.Rxc8, was resourceful but depended heavily on Black’s repeated defensive errors.", "confidence_valid": true }, { "model": "openai/gpt-6-sol", "pair": "pair_23", "game_a": "game_25", "color_a": "white", "game_b": "game_18", "color_b": "white", "rating_a": 738, "rating_b": 2426, "gap": 1688, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.08, "correct": 1, "brier": 0.0064, "old_estimate_a": 1200, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s White developed and contested the center with 7.O-O, 8.d4 and 11.Nxd4, then navigated exchanges with 27.Bxc5+ and 28.Qxc3. The late 32.Qxd5? was a serious mistake, but Game A’s White showed more persistent problems: 3.Qf3 and 4.g4 neglected development, and 13.Nh3 allowed 13...Nc2+ 14.Kd2 Nxa1. A’s win also depended on Black’s 33...Bd8, which allowed 34.Rxc8. B looks more likely to have had the higher established strength despite losing.", "confidence_valid": true }, { "model": "openai/gpt-6.1-sol", "pair": "pair_23", "game_a": "game_25", "color_a": "white", "game_b": "game_18", "color_b": "white", "rating_a": 738, "rating_b": 2426, "gap": 1688, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.96, "correct": 1, "brier": null, "old_estimate_a": 1150, "old_estimate_b": 1550, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B White shows substantially stronger opening and middlegame fundamentals: castling, coordinated development, 10.Re1+ forcing Black's king to f8, and sensible central exchanges. White later underestimated Black's attack with 30.Rd1 and especially 32.Qxd5, allowing decisive checks. Game A White instead played 3.Qf3 and repeated flank-pawn pushes while leaving the king and pieces undeveloped; 13.Nh3 allowed 13...Nc2+ and the loss of the a1 rook. A's eventual tactical success—33.Ra8 followed by 34.Rxc8 and a mating sequence—came against highly cooperative opposition and does not outweigh the much weaker earlier play.", "confidence_valid": false }, { "model": "openai/gpt-5.6-luna", "pair": "pair_23", "game_a": "game_25", "color_a": "white", "game_b": "game_18", "color_b": "white", "rating_a": 738, "rating_b": 2426, "gap": 1688, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.3, "correct": 1, "brier": 0.09, "old_estimate_a": 1500, "old_estimate_b": 1700, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s White showed more coherent opening and positional play: 1.e4, 7.O-O, 9.Bb2, 10.Re1+, and 12.Nxc6 develop with clear purpose. However, White ultimately missed the danger of Black’s attack, especially 30...Re2+ followed by 32...Qh3+ and 33...Qxg3+. Game A’s White found a strong tactical attack and mating net, but the opening was extremely loose (2.e3, 3.Qf3, 4.g4), and White allowed 13...Nc2+ and 14...Nxa1, losing a rook. The eventual win depended heavily on Black’s inaccurate defense, including 24...Rxc3+. Overall, B’s White appears more established despite losing.", "confidence_valid": true }, { "model": "openai/gpt-6-astra", "pair": "pair_23", "game_a": "game_25", "color_a": "white", "game_b": "game_18", "color_b": "white", "rating_a": 738, "rating_b": 2426, "gap": 1688, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.96, "correct": 1, "brier": null, "old_estimate_a": 1050, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B White shows much stronger fundamentals: 7.O-O, 9.Bb2 and 10.Re1+ coordinate development, while 19.Rh1 sensibly exchanges Black’s attacking rook. The intermediate 27.Bxc5+ before 28.Qxc3 also shows tactical awareness. White later allows a decisive attack with 32.Qxd5 Qh3+. Game A White instead plays 3.Qf3 and repeated g- and h-pawn pushes while leaving almost everything undeveloped; 13.Nh3 allows 13...Nc2+ and the loss of the a1-rook. A’s later rook invasion and mating finish are creditable, but arise against highly cooperative opposition and do not outweigh the much weaker opening and king safety.", "confidence_valid": false }, { "model": "z-ai/glm-5.3-flash", "pair": "pair_23", "game_a": "game_25", "color_a": "white", "game_b": "game_18", "color_b": "white", "rating_a": 738, "rating_b": 2426, "gap": 1688, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.4, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 1350, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 1, "explanation": "White in Game B plays a principled Sicilian/Taimanov setup, trades sensibly (11.Nxd4, 12.Nxc6), maneuvers well (21.cxd5, 22.Be3), and only loses after Black finds strong rook infiltrations (29...Re8, 30...Re2+); White's final blunder 32.Qxd5 walking into ...Qh3+ costs the game. White in Game A benefits from a catastrophically weak opponent (7...Rd8??, 13...Nc2+ giving a knight for nothing) and plays erratic moves himself (2...Bd7 response aside, 4.g4?!, 13.Nh3 with king marching). A's win is more about opponent collapse than skill; B shows more coherent technique despite losing.", "confidence_valid": true } ], "a": { "game": "game_25", "color": "white", "rating": 738, "url": "https://lichess.org/ryvfsxd9", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"1-0\"]\n[TimeControl \"600+0\"]\n\n1. d4 d5 2. e3 Bd7 3. Qf3 c6 4. g4 b5 5. h3 Na6 6. h4 Qc7 7. g5 Rd8 8. h5 c5 9.\nc3 b4 10. b3 c4 11. bxc4 dxc4 12. cxb4 Nxb4 13. Nh3 Nc2+ 14. Kd2 Nxa1 15. Bb2\nQa5+ 16. Nc3 Bxh3 17. Rxh3 Qxg5 18. Bxa1 Qg1 19. Rh1 Qg5 20. Bxc4 Rxd4+ 21. Kc2\nRxc4 22. Rd1 Qc5 23. Qa8+ Qc8 24. Qd5 Rxc3+ 25. Bxc3 f6 26. f4 e6 27. Qb5+ Kf7\n28. e4 e5 29. f5 Nh6 30. Rd7+ Be7 31. Qd5+ Ke8 32. Rxa7 Nxf5 33. Ra8 Bd8 34.\nRxc8 Ke7 35. Qb7+ Kd6 36. Qc6+ Ke7 37. Bb4+ Kf7 38. Qd5+ Ke8 39. Rxd8# 1-0" }, "b": { "game": "game_18", "color": "white", "rating": 2426, "url": "https://lichess.org/ZPxafolO", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"0-1\"]\n[TimeControl \"600+0\"]\n\n1. e4 c5 2. Nf3 e6 3. b3 a6 4. g3 Nc6 5. Bg2 d5 6. exd5 exd5 7. O-O Nf6 8. d4\ncxd4 9. Bb2 Bc5 10. Re1+ Kf8 11. Nxd4 h5 12. Nxc6 bxc6 13. Bd4 Bd6 14. c4 h4\n15. Nc3 hxg3 16. hxg3 Bh3 17. Bxh3 Rxh3 18. Kg2 Rh6 19. Rh1 Rxh1 20. Qxh1 Ng8\n21. cxd5 c5 22. Be3 Be5 23. Rc1 Bd4 24. Qe1 Nf6 25. f3 Qd7 26. Qd2 Bxc3 27.\nBxc5+ Kg8 28. Qxc3 Nxd5 29. Qd4 Re8 30. Rd1 Re2+ 31. Kf1 Rh2 32. Qxd5 Qh3+ 33.\nKe1 Qxg3+ 0-1" }, "wrong": 1, "gap": 1688 }, { "id": "pair_01", "predictions": [ { "model": "anthropic/claude-opus-5.5", "pair": "pair_01", "game_a": "game_16", "color_a": "black", "game_b": "game_19", "color_b": "white", "rating_a": 1569, "rating_b": 2668, "gap": 1099, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.25, "correct": 1, "brier": 0.0625, "old_estimate_a": 1100, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black played a dubious gambit (2...e6) and then the unsound 6...Bxf2+, ending down material for pawns. 11...Ng4 12.h3 Bxf3 13.hxg4 Bxe2 only traded pieces without recovering the deficit. Black then drifted into a worse endgame and resigned after 27.Re6, showing little compensation or resourcefulness. Game B White handled a QGD Exchange competently: 8.Bxf6 led to a queen trade, 22.Nxh5 won a pawn, and 25.h5 with 28.h6 pushed the passed pawn. White then used the rook actively with 33.Re7+ and 34.Ra7, and punished 34...Nxf5 with 35.Ra8+ and 36.Rxh8. Game B's play is more coherent and technically sound.", "confidence_valid": true }, { "model": "anthropic/claude-sonnet-5.5", "pair": "pair_01", "game_a": "game_16", "color_a": "black", "game_b": "game_19", "color_b": "white", "rating_a": 1569, "rating_b": 2668, "gap": 1099, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.4, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 1500, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A, Black: Black played a speculative 6...Bxf2+ sacrifice that gave up a piece for a pawn, then played reasonably (12...Bxf3, 17...Rxe1). The game ended abruptly after 27.Re6 Nc7, and Black was worse by then. The opening sacrifice suggests an intermediate-level player. Game B, White: White played a standard Queen's Gambit Exchange with Bg5, traded queens, and played a coherent plan (h4-h5, Rac1, Ne2-g3). Its 22.Nxh5 won a pawn, and it converted the endgame cleanly with 33.Re7+ and 36.Rxh8. Black in Game B also played reasonably but slipped. White in B looks somewhat more solid and technically sound than Black in A, though both are probably club-level players, so the evidence is modest.", "confidence_valid": true }, { "model": "google/gemini-3.8-flash", "pair": "pair_01", "game_a": "game_16", "color_a": "black", "game_b": "game_19", "color_b": "white", "rating_a": 1569, "rating_b": 2668, "gap": 1099, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.03, "correct": 1, "brier": 0.0009, "old_estimate_a": 1450, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black played an unsound opening line and immediately sacrificed a full minor piece for zero compensation with 6...Bxf2+ 7.Kxf2, after which White smoothly consolidated with Rf1 and Kg1, leaving Black a piece down without counterplay. In contrast, Game B White demonstrated solid positional understanding and endgame technique. White strategically compromised Black's pawn structure (8.Bxf6 and 9.Qxf6), systematically exploited Black's weaknesses on the kingside (16.Nhf4, 22.Nxh5), created an unstoppable passed h-pawn, and executed a decisive tactical wrap-up (34.Ra7 and 35.Ra8+). Game B White shows vastly superior chess strength.", "confidence_valid": true }, { "model": "google/gemini-3.1-pro-preview", "pair": "pair_01", "game_a": "game_16", "color_a": "black", "game_b": "game_19", "color_b": "white", "rating_a": 1569, "rating_b": 2668, "gap": 1099, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1100, "old_estimate_b": 2000, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Black in Game A plays a highly dubious opening with 2... e6 and unprovokedly sacrifices a bishop for no compensation with 6... Bxf2+, showing a lack of basic tactical and positional understanding. In contrast, White in Game B demonstrates high-level positional play in a QGD Exchange, strategically damaging Black's pawn structure, maneuvering knights effectively (Nh3-f4-g3), and executing a precise endgame conversion with the tactical sequence 34. Ra7 and 35. Ra8+. White B is clearly the stronger, more advanced player.", "confidence_valid": true }, { "model": "google/gemini-3.5-flash", "pair": "pair_01", "game_a": "game_16", "color_a": "black", "game_b": "game_19", "color_b": "white", "rating_a": 1569, "rating_b": 2668, "gap": 1099, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.03, "correct": 1, "brier": 0.0009, "old_estimate_a": 1300, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "In Game A, Black plays a dubious opening gambit (2... e6) and an unsound piece sacrifice (6... Bxf2+?) that leaves them down a piece for no compensation. Black's play lacks tactical and positional foundation. In contrast, Game B (White) demonstrates high-level positional and endgame play. White expertly exploits Black's pawn weaknesses, maneuvers the knight efficiently (Nh3-f4-g3-f5), and executes a precise rook endgame. White secures the win with a neat tactical sequence starting with 34. Ra7 and 35. Ra8+, winning Black's rook. Target B is vastly more accomplished.", "confidence_valid": true }, { "model": "google/gemini-3.7-flash", "pair": "pair_01", "game_a": "game_16", "color_a": "black", "game_b": "game_19", "color_b": "white", "rating_a": 1569, "rating_b": 2668, "gap": 1099, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1300, "old_estimate_b": 2000, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Target B (White in Game B) demonstrated high-level strategic and endgame skill. After doubling Black's pawns with 8. Bxf6 and trading queens, White accurately navigated the endgame with Kd2, Nh3-f4, exploited Black's kingside weaknesses to win pawns (22. Nxh5), and converted effortlessly with 32. Re1 and 35. Ra8+. In contrast, Target A (Black in Game A) played an unsound gambit and blundered a full piece with 6...Bxf2+? without any compensation, subsequently losing without mounting any real resistance.", "confidence_valid": true }, { "model": "qwen/qwen3.8-max-prime", "pair": "pair_01", "game_a": "game_16", "color_a": "black", "game_b": "game_19", "color_b": "white", "rating_a": 1569, "rating_b": 2668, "gap": 1099, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "A", "probability_a": 0.6, "correct": 0, "brier": 0.36, "old_estimate_a": 1200, "old_estimate_b": 1500, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black sacrifices the bishop unsoundly (6...Bxf2+), but recovers with sound positional moves: 12...Bxf3, 17...Rxe1, 22...cxd4, and the simplifying 25...Qxe2, reaching a roughly defensible rook ending before a late collapse. Game B White plays more passively and strategically shaky: 7.Qf3 invites trades, 9.Qxf6 gxf6 improves Black's structure, 10.h4 and 22.Nxh5 chase loose pawns while Black equalizes; the win owes more to opponent errors after 30...dxc4. A's middlegame resourcefulness under a bad opening suggests a slightly stronger established player.", "confidence_valid": true }, { "model": "anthropic/claude-fable-5.1", "pair": "pair_01", "game_a": "game_16", "color_a": "black", "game_b": "game_19", "color_b": "white", "rating_a": 1569, "rating_b": 2668, "gap": 1099, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.2, "correct": 1, "brier": 0.04000000000000001, "old_estimate_a": 1100, "old_estimate_b": 1550, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black plays a speculative Scandinavian gambit with 3...Bxe6 and the unsound 6...Bxf2+ sacrifice, then 11...Ng4?? losing a piece outright (12.h3 Bxf3 13.hxg4), and resigns a pawn down in a lost ending after 27.Re6. That is a beginner-level piece blunder. Game B White conducts a solid Exchange QGD, trades sensibly (8.Bxf6, 9.Qxf6), centralizes the king for the queenless middlegame (12.Kd2, 19.Kxd3), wins a pawn with 22.Nxh5, and converts with the h-pawn and 33.Re7+ 34.Ra7 35.Ra8+. Technique is clean and purposeful; Black's defence in B was also reasonable. White in B shows clearly higher established strength.", "confidence_valid": true }, { "model": "google/gemini-3-flash-preview", "pair": "pair_01", "game_a": "game_16", "color_a": "black", "game_b": "game_19", "color_b": "white", "rating_a": 1569, "rating_b": 2668, "gap": 1099, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.1, "correct": 1, "brier": 0.010000000000000002, "old_estimate_a": 1400, "old_estimate_b": 2200, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Target B demonstrates a much higher level of positional understanding and endgame technique. In Game B, White navigates a Queen's Gambit Exchange structure expertly, utilizing the h-pawn and knight maneuvers (11. Nh3, 16. Nhf4) to exploit Black's doubled pawns. White converts the technical endgame efficiently with 32. Re1 and 37. h7. In contrast, Target A (Black) plays a dubious opening (2... e6) and an unsound piece sacrifice (6... Bxf2+). After the initial aggression fails, Black lacks positional compensation and eventually collapses, ending with the blunder 27... Nc7 which loses the knight on d5 to the rook. The disparity in strategic depth and tactical stability clearly favors Target B.", "confidence_valid": true }, { "model": "google/gemini-3.6-flash", "pair": "pair_01", "game_a": "game_16", "color_a": "black", "game_b": "game_19", "color_b": "white", "rating_a": 1569, "rating_b": 2668, "gap": 1099, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.1, "correct": 1, "brier": 0.010000000000000002, "old_estimate_a": 1400, "old_estimate_b": 1900, "baseline_choice": "B", "baseline_credit": 1, "explanation": "In Game A, Black (Target A) played an unsound piece sacrifice with 6...Bxf2+? for minimal compensation and later blundered with 27...Nc7, hanging the knight. In contrast, Target B (White in Game B) demonstrated strong positional understanding in a standard QGD Carlsbad structure. White systematically exploited Black's weak f- and h-pawns through precise knight maneuvering (11.Nh3, 16.Nhf4, 22.Nxh5). White then cleanly converted the endgame with active piece play (33.Re7+, 34.Ra7) to create a decisive passed pawn (37.h7). Target B displayed much higher strategic depth and tactical soundness than Target A.", "confidence_valid": true }, { "model": "openai/gpt-5.6-sol", "pair": "pair_01", "game_a": "game_16", "color_a": "black", "game_b": "game_19", "color_b": "white", "rating_a": 1569, "rating_b": 2668, "gap": 1099, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.07, "correct": 1, "brier": 0.004900000000000001, "old_estimate_a": 1600, "old_estimate_b": 1950, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s White showed substantially stronger, coherent play: the queen trade with 8.Bxf6 and 9.Qxf6, king centralization, the maneuver Nh3-f4-g3-h5, and 22.Nxh5 followed by the passed h-pawn were purposeful. White also found the clean tactical finish 33.Re7+ and 35.Ra8+, winning the h8-rook. Game A’s Black chose the highly unsound 6...Bxf2+, leaving White materially ahead despite the resource 12...Bxf3 13.hxg4 Bxe2. Black then traded queens while a piece down and resigned after 27...Nc7. B is much more likely to have had the higher established rapid strength.", "confidence_valid": true }, { "model": "openai/gpt-6-sol", "pair": "pair_01", "game_a": "game_16", "color_a": "black", "game_b": "game_19", "color_b": "white", "rating_a": 1569, "rating_b": 2668, "gap": 1099, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.04, "correct": 1, "brier": 0.0016, "old_estimate_a": 1450, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s White played a coherent endgame: 22.Nxh5 won a pawn, 25.h5 and 28.h6 created a dangerous passer, and 34.Ra7 followed by 35.Ra8+ exploited Black’s knight capture. Game A’s Black chose the unsound-looking 6...Bxf2+, then lost another minor piece in the sequence 11...Ng4 12.h3 Bxf3 13.hxg4 Bxe2 14.Qxe2, leaving White with a substantial material advantage. B’s play is stronger evidence of established skill, regardless of the results.", "confidence_valid": true }, { "model": "openai/gpt-6.1-sol", "pair": "pair_01", "game_a": "game_16", "color_a": "black", "game_b": "game_19", "color_b": "white", "rating_a": 1569, "rating_b": 2668, "gap": 1099, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.93, "correct": 1, "brier": null, "old_estimate_a": 1550, "old_estimate_b": 1900, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A’s Black played the speculative 6...Bxf2+, sacrificing a bishop without a convincing continuation. The sequence 12...Bxf3 13.hxg4 Bxe2 left Black a minor piece down, and subsequent exchanges offered little compensation. Game B’s White showed substantially more coherent planning: 12.Kd2 activated the king after the queen trade, 22.Nxh5 won a pawn, and h5–h6 created a dangerous passer. White then combined central pressure with seventh-rank activity through 31.d5, 32.Re1 and 33.Re7+, followed by 35.Ra8+ and 36.Rxh8. Black’s cooperation in Game B limits certainty, but White’s sustained coordination is much stronger evidence of established playing strength.", "confidence_valid": false }, { "model": "openai/gpt-5.6-luna", "pair": "pair_01", "game_a": "game_16", "color_a": "black", "game_b": "game_19", "color_b": "white", "rating_a": 1569, "rating_b": 2668, "gap": 1099, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.82, "correct": 1, "brier": null, "old_estimate_a": 1100, "old_estimate_b": 1700, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s White showed substantially better practical play: after the unconventional 12.Kd2, White coordinated effectively with 20.Rhe1, exchanged favorably, created a dangerous h-pawn, and converted it with 33.Re7+, 34.Ra7, and 36.Rxh8. White also promptly exploited Black’s 34...Nxf5. In Game A, Black’s opening became strategically and materially dubious: 6...Bxf2+ sacrificed a bishop for a pawn, and 12...Bxf3 13.hxg4 further left Black’s position vulnerable. Black never generated adequate compensation and allowed 27.Re6. Although Game B’s opponent made larger errors, its target demonstrated much stronger conversion and coordination than Game A’s target.", "confidence_valid": false }, { "model": "openai/gpt-6-astra", "pair": "pair_01", "game_a": "game_16", "color_a": "black", "game_b": "game_19", "color_b": "white", "rating_a": 1569, "rating_b": 2668, "gap": 1099, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.95, "correct": 1, "brier": null, "old_estimate_a": 1450, "old_estimate_b": 1900, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B White showed coherent positional and endgame play: 8.Bxf6 followed by 9.Qxf6 damaged Black’s structure, 12.Kd2 activated the king, and 22.Nxh5 followed by the h-pawn advance created a dangerous passer. The rook invasion 33.Re7+ and 35.Ra8+ exploited Black’s exposed rook decisively. Game A Black developed actively, but 6...Bxf2+ was an unsupported sacrifice; 11...Ng4 and the ensuing exchanges left Black a bishop down without convincing compensation. The queen exchange with 25...Qxe2 further reduced any attacking chances. B’s sustained coordination is substantially stronger evidence of established playing strength.", "confidence_valid": false }, { "model": "z-ai/glm-5.3-flash", "pair": "pair_01", "game_a": "game_16", "color_a": "black", "game_b": "game_19", "color_b": "white", "rating_a": 1569, "rating_b": 2668, "gap": 1099, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.3, "correct": 1, "brier": 0.09, "old_estimate_a": 1650, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B's White plays a clean, methodical game: timely queen trade (9.Qxf6), purposeful h4-h5 expansion, good rook placement (Rac1, Re1), and accurate endgame technique (Kd3, Re1-Ra7 winning the exchange, h7). Game A's Black plays erratically: the speculative 6...Bxf2+ gives up material without clear compensation, and 27...Re6 blunders the pinned rook with Nc7 while White threatens the e-file, losing immediately. Black's compensation attempt (17...Rxe1, 18...Nd5) shows some ideas but the overall play is less coherent and collapses faster. B's White shows stronger positional understanding and endgame technique, suggesting the higher rapid rating.", "confidence_valid": true } ], "a": { "game": "game_16", "color": "black", "rating": 1569, "url": "https://lichess.org/HCYL2QiH", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"1-0\"]\n[TimeControl \"600+0\"]\n\n1. e4 d5 2. exd5 e6 3. dxe6 Bxe6 4. Nf3 Bc5 5. c3 Qe7 6. Be2 Bxf2+ 7. Kxf2 Nf6\n8. Rf1 O-O 9. Kg1 Bd5 10. d4 Qd6 11. Bg5 Ng4 12. h3 Bxf3 13. hxg4 Bxe2 14. Qxe2\nNd7 15. Nd2 Rae8 16. Qf2 Nb6 17. Rae1 Rxe1 18. Rxe1 Nd5 19. Qf3 f6 20. Bh4 c5\n21. Bg3 Qc6 22. Nb3 cxd4 23. Nxd4 Qc4 24. a3 a5 25. Qe2 Qxe2 26. Rxe2 Rd8 27.\nRe6 Nc7 1-0" }, "b": { "game": "game_19", "color": "white", "rating": 2668, "url": "https://lichess.org/ZAG2anr0", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"1-0\"]\n[TimeControl \"600+0\"]\n\n1. d4 Nf6 2. c4 e6 3. Nc3 d5 4. cxd5 exd5 5. Bg5 c6 6. e3 Bf5 7. Qf3 Bg6 8.\nBxf6 Qxf6 9. Qxf6 gxf6 10. h4 h5 11. Nh3 Bd6 12. Kd2 Nd7 13. Bd3 b5 14. Rac1\nKe7 15. Ne2 Rac8 16. Nhf4 Rhg8 17. Ng3 Bxf4 18. exf4 Bxd3 19. Kxd3 Kd6 20. Rhe1\nRce8 21. Rxe8 Rxe8 22. Nxh5 c5 23. Ng3 c4+ 24. Kd2 Nf8 25. h5 Ne6 26. Nf5+ Kd7\n27. g3 Rh8 28. h6 a5 29. b3 Nd8 30. bxc4 dxc4 31. d5 Nb7 32. Re1 Nd6 33. Re7+\nKd8 34. Ra7 Nxf5 35. Ra8+ Kd7 36. Rxh8 b4 37. h7 1-0" }, "wrong": 1, "gap": 1099 }, { "id": "pair_08", "predictions": [ { "model": "anthropic/claude-opus-5.5", "pair": "pair_08", "game_a": "game_02", "color_a": "black", "game_b": "game_20", "color_b": "black", "rating_a": 2410, "rating_b": 1903, "gap": 507, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.68, "correct": 1, "brier": 0.10239999999999996, "old_estimate_a": 1350, "old_estimate_b": 1450, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game A Black punished errors cleanly. 9...Nxe4 took a free pawn after 8.Nd5, and 18...Bxh4 exploited 18.h4. After the queen trade, 21...Re2 led to 22...Rxb2, 24...Rxh3 and the mating 25...Rh1#, showing efficient, accurate conversion. Admittedly White there was weak. Game B Black played poorly against a standard kingside attack. 6...Re8 and 7...Bh8 wasted tempi and dropped the dark-squared bishop's defensive role. 13...fxg6 opened lines, and 17...Nxh6 handed White the h-file. Black's queenside play (b4, a4, b3) was far too slow, and the king collapsed after 26.Rxh7. Black B's opponent attacked competently, but B's opening and defensive understanding looks weaker than A's tactical alertness.", "confidence_valid": true }, { "model": "anthropic/claude-sonnet-5.5", "pair": "pair_08", "game_a": "game_02", "color_a": "black", "game_b": "game_20", "color_b": "black", "rating_a": 2410, "rating_b": 1903, "gap": 507, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.66, "correct": 1, "brier": 0.11559999999999998, "old_estimate_a": 1550, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game A, Black: plays the Open Spanish normally. After White's weak 8.Nd5, Black takes 9...Nxe4 and 10...exd4, then 14...Nc4 and 18...Bxh4 win material and tempo. Black then finishes cleanly with 21...Re2, 22...Rxb2 and 25...Rh1#. White blundered, but Black punished each error efficiently. Game B, Black: plays a King's Indian/Modern setup but loses a pawn structure battle. 7...Bh8 is odd, and 17...Nxh6 followed by 18.Rxh6 lets White's attack run. 24...b3 and 25...Kf7 are dubious, and 26...Nxh7 loses material. Black loses to a mating attack with lots of errors, including 28...Kf8. Black in A looks a bit stronger, though A's opponent was very weak, which tempers the evidence.", "confidence_valid": true }, { "model": "google/gemini-3.8-flash", "pair": "pair_08", "game_a": "game_02", "color_a": "black", "game_b": "game_20", "color_b": "black", "rating_a": 2410, "rating_b": 1903, "gap": 507, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.68, "correct": 1, "brier": 0.10239999999999996, "old_estimate_a": 1650, "old_estimate_b": 1650, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "Game A's Black demonstrated clean, disciplined positional and tactical play throughout. In the Spanish opening, Black capitalized cleanly on White's pawn sacrifice with 9...Nxe4, consolidated comfortably with 13...Na5 and 15...g6, and methodically punished White's overextension on the kingside with 18...Bxh4 and 21...Re2, executing a swift finish. Game B's Black mishandled the defense against White's standard kingside storm in the Modern/Pirc, spending too many tempi shuffling knights (14...Ne5, 15...Nf7, 16...Nd7, 18...Nf8) while queenside counterplay lagged behind. This passive defense allowed White a decisive breakthrough via 23. Qh2 and 26. Rxh7.", "confidence_valid": true }, { "model": "google/gemini-3.1-pro-preview", "pair": "pair_08", "game_a": "game_02", "color_a": "black", "game_b": "game_20", "color_b": "black", "rating_a": 2410, "rating_b": 1903, "gap": 507, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "B", "probability_a": 0.15, "correct": 0, "brier": 0.7224999999999999, "old_estimate_a": 1650, "old_estimate_b": 2000, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Black B demonstrates advanced opening knowledge (such as the thematic 7...Bh8 to preserve the bishop) and executes a coherent queenside counterattack in a highly complex position, ultimately falling to a brilliant combination by a strong opponent. In contrast, Black A plays a solid game but simply capitalizes on basic blunders from a much weaker White opponent, such as dropping a pawn with 9. c3 Nxe4 and walking into a simple mate-in-one with 25. Bg5. Black B shows a significantly higher level of strategic understanding.", "confidence_valid": true }, { "model": "google/gemini-3.5-flash", "pair": "pair_08", "game_a": "game_02", "color_a": "black", "game_b": "game_20", "color_b": "black", "rating_a": 2410, "rating_b": 1903, "gap": 507, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.65, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1500, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 0, "explanation": "In Game A, Black plays a highly logical Ruy Lopez, quickly punishing White's inaccuracy. Black coordinates pieces excellently with 13...Na5 and 14...Nc4, wins a pawn cleanly with 18...Bxh4, and executes a swift mating attack with 24...Rxh3 and 25...Rh1#. In Game B, Black's passive Modern Defense setup (such as 6...Re8) allows White to build a quick kingside storm. Black fails to generate effective counterplay on the queenside, defensive maneuvers like 18...Nf8 are too slow, and Black ultimately collapses to a straightforward sacrifice on h7. Target A exhibits superior tactical alertness and opening preparation.", "confidence_valid": true }, { "model": "google/gemini-3.7-flash", "pair": "pair_08", "game_a": "game_02", "color_a": "black", "game_b": "game_20", "color_b": "black", "rating_a": 2410, "rating_b": 1903, "gap": 507, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.78, "correct": 1, "brier": 0.04839999999999999, "old_estimate_a": 1650, "old_estimate_b": 1700, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Target A (Game A Black) demonstrated solid opening understanding and sharp tactical awareness. After White's inaccurate gambit play, Black smoothly collected material (9...Nxe4), activated pieces dynamically (13...Na5, 14...Nc4, 21...Re2), and delivered a flawless finish with 24...Rxh3 and 25...Rh1#. Conversely, Target B (Game B Black) defended passively against a standard Yugoslav-style kingside pawn storm, misplaying the kingside defense with 23...Bg7? and 25...Kf7?, allowing White a crushing breakthrough (26. Rxh7) and a clean tactical finish.", "confidence_valid": true }, { "model": "qwen/qwen3.8-max-prime", "pair": "pair_08", "game_a": "game_02", "color_a": "black", "game_b": "game_20", "color_b": "black", "rating_a": 2410, "rating_b": 1903, "gap": 507, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.68, "correct": 1, "brier": 0.10239999999999996, "old_estimate_a": 1350, "old_estimate_b": 1550, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game A Black handled the Ruy Lopez solidly, exploiting White's weak Nd5 (Game A 9…Nxe4) and later capitalizing on White's flawed 18.h4 to win a pawn and deliver a clean mate (Game A 25…Rh1#). Game B Black, in the Pirc (Game B 1…g6, 7…Bh8), was repeatedly passive; 25…Kf7 and 27…Rh8 invited disaster, and Black failed to counter White's g-file attack, leading to mate on move 38. Game B Black showed weaker prophylaxis and middlegame judgment than Game A Black.", "confidence_valid": true }, { "model": "anthropic/claude-fable-5.1", "pair": "pair_08", "game_a": "game_02", "color_a": "black", "game_b": "game_20", "color_b": "black", "rating_a": 2410, "rating_b": 1903, "gap": 507, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.62, "correct": 1, "brier": 0.1444, "old_estimate_a": 1450, "old_estimate_b": 1350, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black played a sound Ruy Lopez, exploited 9...Nxe4 after White's inaccurate 8.Nd5/9.c3, then found the tactical sequence 18...Bxh4! 19...Qxh4 and the rook invasion 21...Re2 to 25...Rh1#, a crisp conversion against a weak White. Game B Black played a passive Pirc setup (7...Bh8, 15...Nf7, 16...Nd7) and allowed a known mating pattern; 25...Kf7?? and 26...Nxh7 walked into Rxh7/Qxh7 and total collapse, and the queenside counterplay (24...b3, 26...a4) was far too slow. Game B's White showed decent attacking technique, but Black's defensive understanding was weak. Game A Black looks the more competent player.", "confidence_valid": true }, { "model": "google/gemini-3-flash-preview", "pair": "pair_08", "game_a": "game_02", "color_a": "black", "game_b": "game_20", "color_b": "black", "rating_a": 2410, "rating_b": 1903, "gap": 507, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.65, "correct": 1, "brier": 0.12249999999999998, "old_estimate_a": 1800, "old_estimate_b": 1950, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Player A (Black) played a precise and disciplined game, efficiently punishing White's inaccuracies. Moves like 9...Nxe4 and 14...Nc4 demonstrate strong tactical awareness, while the infiltration with 21...Re2 and 23...Rb3 showed high technique in converting an advantage. In contrast, Player B (Black) struggled to handle White’s aggressive kingside storm in a Modern Defense. Allowing the h-file to open with 17...Nxh6 and failing to generate sufficient counter-pressure led to a rapid collapse. While Player B faced a strong attack, Player A's clean execution and tactical alertness suggest a more established skill level.", "confidence_valid": true }, { "model": "google/gemini-3.6-flash", "pair": "pair_08", "game_a": "game_02", "color_a": "black", "game_b": "game_20", "color_b": "black", "rating_a": 2410, "rating_b": 1903, "gap": 507, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.88, "correct": 1, "brier": 0.0144, "old_estimate_a": 1650, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game A Black displayed excellent tactical precision and solid opening knowledge in the Ruy Lopez. Black accurately seized central control (14...Nc4), punished White's premature flank push 18.h4 with 18...Bxh4! 19.Nxh4 Qxh4, and executed a swift, decisive conversion ending in 25...Rh1#. In contrast, Game B Black defended poorly against White's kingside storm in a Pirc structure. Facing severe pressure on the h-file, Game B Black ignored White's threats with 24...b3? and collapsed to the blunder 25...Kf7??, allowing 26.Rxh7! which won White a queen and rook. Game A Black demonstrated significantly higher tactical awareness and positional understanding.", "confidence_valid": true }, { "model": "openai/gpt-5.6-sol", "pair": "pair_08", "game_a": "game_02", "color_a": "black", "game_b": "game_20", "color_b": "black", "rating_a": 2410, "rating_b": 1903, "gap": 507, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.88, "correct": 1, "brier": 0.0144, "old_estimate_a": 1650, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game A Black played a coherent tactical game: 9...Nxe4 exploited the center, 18...Bxh4! converted White’s kingside aggression into material and initiative, and 21...Re2 followed by ...Rxb2/...Rxh3 produced the precise 25...Rh1# finish. Game B Black’s queenside counterplay was too slow while the king came under a direct attack; 24...b3 and especially 25...Kf7 failed to address White’s threats, and 26...Nxh7 allowed the forcing sequence 27.Qxh7 Rh8 28.fxg6+ through 31.Rxd8, leaving Black decisively down. A showed stronger tactical awareness and defensive judgment.", "confidence_valid": true }, { "model": "openai/gpt-6-sol", "pair": "pair_08", "game_a": "game_02", "color_a": "black", "game_b": "game_20", "color_b": "black", "rating_a": 2410, "rating_b": 1903, "gap": 507, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.98, "correct": 1, "brier": 0.0004000000000000007, "old_estimate_a": 1650, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game A’s Black played a coherent game: 18...Bxh4 met White’s advance, and 21...Re2 followed by 22...Rxb2 and 24...Rxh3 turned the open lines into a mating attack. Game B’s Black pursued queenside counterplay with 24...b3, but missed a decisive danger: after 27...Rh8 and 28...Kf8, White’s 29.Qxh8+ Bxh8 30.Rxh8+ led to 31.Rxd8, winning Black’s queen. A’s play looks substantially more controlled.", "confidence_valid": true }, { "model": "openai/gpt-6.1-sol", "pair": "pair_08", "game_a": "game_02", "color_a": "black", "game_b": "game_20", "color_b": "black", "rating_a": 2410, "rating_b": 1903, "gap": 507, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.76, "correct": 1, "brier": 0.0576, "old_estimate_a": 1650, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game A Black played more coherently: 9...Nxe4 exploited the center, 18...Bxh4 won another pawn through 19.Nxh4 Qxh4, and 21...Re2 converted the queen trade into active rook play. The finish 24...Rxh3 and 25...Rh1# correctly used the b7 bishop. Game B Black showed sensible queenside counterplay, but the repeated knight maneuvers consumed time while White built a clear kingside attack. Black's 25...Kf7 allowed the forcing sequence beginning 26.Rxh7, ultimately losing the queen on d8; 36...c4 also ignored the mating threat. A's opposition was weaker, which limits confidence, but A demonstrated better tactical control.", "confidence_valid": true }, { "model": "openai/gpt-5.6-luna", "pair": "pair_08", "game_a": "game_02", "color_a": "black", "game_b": "game_20", "color_b": "black", "rating_a": 2410, "rating_b": 1903, "gap": 507, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.8, "correct": 1, "brier": 0.03999999999999998, "old_estimate_a": 1750, "old_estimate_b": 1600, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A’s Black showed stronger tactical execution: 9...Nxe4 and 10...exd4 exploited White’s center, while 18...Bxh4! 19 Nxh4 Qxh4 removed the key defender. The follow-up 21...Re2 and 22...Rxb2 created a coherent rook attack, culminating in 25...Rh1#. Game B’s Black developed sensibly, but the kingside defense deteriorated: after White’s aggressive 24 f5, Black allowed 26 Rxh7 and 27 Qxh7, then was unable to stop the advancing g-pawn and was mated by 38 Nh6#. Although Game A’s opponent made larger errors, Black’s tactical recognition and conversion provide stronger evidence of established strength.", "confidence_valid": true }, { "model": "openai/gpt-6-astra", "pair": "pair_08", "game_a": "game_02", "color_a": "black", "game_b": "game_20", "color_b": "black", "rating_a": 2410, "rating_b": 1903, "gap": 507, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.66, "correct": 1, "brier": 0.11559999999999998, "old_estimate_a": 1650, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game A Black played more consistently: 9...Nxe4 exploited White’s center, 13...Na5–14...Nc4 improved the knight, and 18...Bxh4 won another pawn through a favorable exchange. After trading queens, 21...Re2 and 23...Rb3 showed effective rook activity, culminating in 25...Rh1#. Game B Black showed purposeful ideas with ...Nxh6, ...Nf8 and ...Ba6, against a more coherent attacking opponent. However, 25...Kf7 and 27...Rh8 allowed the forcing 28.fxg6+ through 31.Rxd8 sequence. A’s cleaner tactical judgment gives a moderate edge, tempered by White’s substantially weaker resistance in Game A.", "confidence_valid": true }, { "model": "z-ai/glm-5.3-flash", "pair": "pair_08", "game_a": "game_02", "color_a": "black", "game_b": "game_20", "color_b": "black", "rating_a": 2410, "rating_b": 1903, "gap": 507, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.75, "correct": 1, "brier": 0.0625, "old_estimate_a": 1700, "old_estimate_b": 1500, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A's Black played a principled Ruy Lopez, grabbing a pawn with 9...Nxe4, regrouping (13...Na5, 14...Nc4), and executing a clean tactical sequence: 18...Bxh4! 19.Nxh4 Qxh4, then converting accurately to mate (21...Re2, 24...Rxh3, 25...Rh1#). Game B's Black chose a passive kingside setup (...Re8 6..., 7...Bh8) and drifted: 23...Bg7? allowed 24.Rxh7/26.Rxh7, and after 26...Nxh7 27.Qxh7 the defense collapsed, leading to a forced mating net (38.Nh6#). A's Black showed coherent strategy and precise calculation; B's Black was outplayed systematically.", "confidence_valid": true } ], "a": { "game": "game_02", "color": "black", "rating": 2410, "url": "https://lichess.org/8foldxhD", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"0-1\"]\n[TimeControl \"600+0\"]\n\n1. e4 e5 2. Nf3 Nc6 3. Bb5 a6 4. Ba4 Nf6 5. Nc3 b5 6. Bb3 Be7 7. O-O O-O 8. Nd5\nd6 9. c3 Nxe4 10. d4 exd4 11. cxd4 Nf6 12. Nxf6+ Bxf6 13. Be3 Na5 14. Bc2 Nc4\n15. Qd3 g6 16. Bh6 Re8 17. Qb3 Bb7 18. h4 Bxh4 19. Nxh4 Qxh4 20. Qh3 Qxh3 21.\ngxh3 Re2 22. Bd3 Rxb2 23. a3 Rb3 24. Bc2 Rxh3 25. Bg5 Rh1# 0-1" }, "b": { "game": "game_20", "color": "black", "rating": 1903, "url": "https://lichess.org/jLDNTu60", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"1-0\"]\n[TimeControl \"600+0\"]\n\n1. e4 g6 2. Nc3 Bg7 3. d4 d6 4. Be3 Nf6 5. Qd2 O-O 6. f3 Re8 7. Bh6 Bh8 8.\nO-O-O Nbd7 9. g4 c5 10. d5 a6 11. h4 b5 12. h5 b4 13. hxg6 fxg6 14. Nce2 Ne5\n15. Ng3 Nf7 16. g5 Nd7 17. f4 Nxh6 18. Rxh6 Nf8 19. Bc4 a5 20. Nf3 Ba6 21. Bxa6\nRxa6 22. Rdh1 a4 23. Qh2 Bg7 24. f5 b3 25. a3 Kf7 26. Rxh7 Nxh7 27. Qxh7 Rh8\n28. fxg6+ Kf8 29. Qxh8+ Bxh8 30. Rxh8+ Kg7 31. Rxd8 Kxg6 32. Rd7 Kf7 33. Nf5\nbxc2 34. Rxe7+ Kf8 35. g6 Ra8 36. Ng5 c4 37. g7+ Kg8 38. Nh6# 1-0" }, "wrong": 1, "gap": 507 }, { "id": "pair_10", "predictions": [ { "model": "anthropic/claude-opus-5.5", "pair": "pair_10", "game_a": "game_23", "color_a": "black", "game_b": "game_09", "color_b": "black", "rating_a": 564, "rating_b": 2177, "gap": 1613, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.2, "correct": 1, "brier": 0.04000000000000001, "old_estimate_a": 1200, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black blundered badly with 13...Bxh3?, an unsound sacrifice. After 14.gxh3 f5 15.Nxf5 they were simply down material, then wasted tempi with the queen (19...Qg3, 21...Qe6, 22...Qg6) and resigned early. Game B Black played a coherent Benoni/Benko-style plan with ...Na6-c7, ...b5, ...Rb8 and ...Ng4-e5-c4. They won material with 21...Bxc3 and 22...Qxa3, kept active rooks with 25...Rb2 and 26...Qd2, and converted cleanly with 35...Bxd5, ...Rdd2 and 41...f3 mating ideas. B shows much stronger understanding and technique.", "confidence_valid": true }, { "model": "anthropic/claude-sonnet-5.5", "pair": "pair_10", "game_a": "game_23", "color_a": "black", "game_b": "game_09", "color_b": "black", "rating_a": 564, "rating_b": 2177, "gap": 1613, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.3, "correct": 1, "brier": 0.09, "old_estimate_a": 1600, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black played the opening reasonably (4...Bf5, 8...O-O) but erred with 13...Bxh3 14.gxh3 f5 15.Nxf5, which dropped material; 15...Bh4 and 16...Qxh4 then lost a piece. Later 18...Qxe5 and 20...Qxh3+ grabbed pawns, but White had a strong attack, and Black lost quickly in 24 moves. Game B Black handled a King's Indian-style Benoni well: 11...Nxb5, 13...Ng4 and 15...Nc4 were active, 21...Bxc3 and 22...Qxa3 won material, and the tactical finish with 39...Rh2+ and 41...f3 was accurate. Black still allowed some complications (25.Rxa7), but overall played at a higher and more consistent level than A Black.", "confidence_valid": true }, { "model": "google/gemini-3.8-flash", "pair": "pair_10", "game_a": "game_23", "color_a": "black", "game_b": "game_09", "color_b": "black", "rating_a": 564, "rating_b": 2177, "gap": 1613, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.12, "correct": 1, "brier": 0.0144, "old_estimate_a": 1200, "old_estimate_b": 2050, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B Black demonstrated strong strategic mastery in a Benoni setup, methodically executing the typical queenside expansion (...b5, ...Nc7, ...Rb8), winning a pawn cleanly with 21...Bxc3 and 22...Qxa3, and converting decisively with active rook play (...Rb2, ...Rdd2) to weave a mating net. In contrast, Game A Black blundered a pawn immediately with 11...e5?, followed up with an unsound piece sacrifice 13...Bxh3, and never achieved real compensation before losing quickly. Player B's understanding of opening plans, piece coordination, and technical execution reflects a considerably higher playing strength.", "confidence_valid": true }, { "model": "google/gemini-3.1-pro-preview", "pair": "pair_10", "game_a": "game_23", "color_a": "black", "game_b": "game_09", "color_b": "black", "rating_a": 564, "rating_b": 2177, "gap": 1613, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1050, "old_estimate_b": 2250, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Target B demonstrates a sophisticated understanding of the Benoni structure, executing typical queenside expansion (10... b5) and excellent knight maneuvers (14... Ne5, 15... Nc4). Target B seamlessly converts a tactical advantage into a won endgame with precise technique (32... Qf4+). In contrast, Target A plays a chaotic and unsound game, sacrificing a piece dubiously with 13... Bxh3, and blundering another piece outright with 15... Bh4??. Target B's positional depth and tactical precision heavily outweigh Target A's poor tactical vision.", "confidence_valid": true }, { "model": "google/gemini-3.5-flash", "pair": "pair_10", "game_a": "game_23", "color_a": "black", "game_b": "game_09", "color_b": "black", "rating_a": 564, "rating_b": 2177, "gap": 1613, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1150, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "In Game A, Black shows poor tactical and positional awareness, dropping a pawn with 11...e5? and then sacrificing a bishop unsoundly with 13...Bxh3?. In contrast, Game B's Black plays a highly coherent Benoni setup, executing a thematic queenside expansion (10...b5). Black shows great positional understanding, wins a pawn cleanly with 21...Bxc3 and 22...Qxa3, and converts the endgame flawlessly to create a mating net with 41...f3. Game B's Black is clearly the more established and stronger player.", "confidence_valid": true }, { "model": "google/gemini-3.7-flash", "pair": "pair_10", "game_a": "game_23", "color_a": "black", "game_b": "game_09", "color_b": "black", "rating_a": 564, "rating_b": 2177, "gap": 1613, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.12, "correct": 1, "brier": 0.0144, "old_estimate_a": 1250, "old_estimate_b": 1950, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Player B displays a sophisticated understanding of opening strategy and positional play in the Benoni/KID setup, executing a classic queenside expansion (...Na6-c7, ...b5) and active piece maneuvers (14...Ne5, 15...Nc4). Player B converted smoothly with 21...Bxc3, heavy-piece infiltration (...Rb2, ...Qd2), and precise technique in the endgame. In contrast, Player A demonstrated poor positional and tactical awareness, dropping a central pawn with 11...e5? and following up with an unsound piece sacrifice (13...Bxh3?) and weak defense that led to an early resignation after 24.Qh2.", "confidence_valid": true }, { "model": "qwen/qwen3.8-max-prime", "pair": "pair_10", "game_a": "game_23", "color_a": "black", "game_b": "game_09", "color_b": "black", "rating_a": 564, "rating_b": 2177, "gap": 1613, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.15, "correct": 1, "brier": 0.0225, "old_estimate_a": 1100, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "In Game A, Black plays a dubious 13...Bxh3 sacrifice that loses material after 14.gxh3 and then wastes moves with a wandering queen (17...Qg5+, 19...Qg3, 21...Qe6, 22...Qg6, 23...Qh6+) before resigning down a full piece. In Game B, Black executes a coherent Benko-style plan with 10...b5, 13...Ng4-e5-c4, and 19...Nd4, winning cleanly via 21...Bxc3 and a decisive attack ending with 39...Rdd2 and 41...f3. Game B's Black shows clearly superior strategic and tactical play.", "confidence_valid": true }, { "model": "anthropic/claude-fable-5.1", "pair": "pair_10", "game_a": "game_23", "color_a": "black", "game_b": "game_09", "color_b": "black", "rating_a": 564, "rating_b": 2177, "gap": 1613, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.12, "correct": 1, "brier": 0.0144, "old_estimate_a": 1300, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black played a speculative, unsound 13...Bxh3 sacrifice after 12...Be7 (allowing the pawn loss), then dropped more material with 15...Bh4 and resigned by move 24 with a hopeless position — typical of a lower-rated player chasing cheap tactics. Game B Black handled a Benoni-style structure with purpose (7...Na6-c7, 10...b5, 13...Ng4-e5-c4 to win the bishop pair), then converted with precise tactics: 19...Nd4, 22...Qxa3, the 32...Qf4+ trade into a won ending, and a clean mating net with 38...Rdd2, 39...Rh2+, 41...f3. Game B's Black shows far superior strategic and tactical competence.", "confidence_valid": true }, { "model": "google/gemini-3-flash-preview", "pair": "pair_10", "game_a": "game_23", "color_a": "black", "game_b": "game_09", "color_b": "black", "rating_a": 564, "rating_b": 2177, "gap": 1613, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.1, "correct": 1, "brier": 0.010000000000000002, "old_estimate_a": 1100, "old_estimate_b": 2050, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Target B demonstrates a clear understanding of Benoni structures, utilizing effective maneuvering (9... Nc7, 14... Ne5, 15... Nc4) and tactical alertness to secure a positional advantage. The conversion of the endgame was precise, culminating in a clean mating net (41... f3). In contrast, Target A displayed significant tactical weaknesses in Game A. They hung a pawn with 11... e5 and failed to capitalize on White's blunder 15. Nxf5, instead playing 15... Bh4?? which allowed White to save the knight. Target A's speculative sacrifice 13... Bxh3 was unsound and lacked follow-through, indicating a much lower level of tactical and positional awareness compared to Target B.", "confidence_valid": true }, { "model": "google/gemini-3.6-flash", "pair": "pair_10", "game_a": "game_23", "color_a": "black", "game_b": "game_09", "color_b": "black", "rating_a": 564, "rating_b": 2177, "gap": 1613, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1250, "old_estimate_b": 1950, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Player B demonstrates a far superior positional and tactical understanding. In Game B, Black plays a well-executed Benoni structure with 10...b5, coordinates pieces purposefully (13...Ng4, 15...Nc4, 19...Nd4), wins a pawn with 21...Bxc3, and executes a dominant endgame conversion with double rooks (38...Rdd2 and 41...f3 mate threat). In contrast, Player A plays poorly in Game A, hanging a pawn with 11...e5 and blundering a full bishop with 13...Bxh3 14.gxh3 without adequate compensation, leading to a swift defeat. Player B is clearly the stronger player.", "confidence_valid": true }, { "model": "openai/gpt-5.6-sol", "pair": "pair_10", "game_a": "game_23", "color_a": "black", "game_b": "game_09", "color_b": "black", "rating_a": 564, "rating_b": 2177, "gap": 1613, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.03, "correct": 1, "brier": 0.0009, "old_estimate_a": 1350, "old_estimate_b": 2050, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s Black showed substantially stronger strategic and tactical control: 19...Nd4!, 20...Nxe2 and 21...Bxc3 dismantled White’s queenside, while 25...Rb2 and 32...Qf4+ transitioned cleanly into a winning rook invasion. Game A’s Black instead launched the unsound 13...Bxh3? and compounded it with 15...Bh4, surrendering both bishops for insufficient material. The ensuing queen checks never produced a decisive attack, leaving Black down a minor piece and resigning after 24.Qh2.", "confidence_valid": true }, { "model": "openai/gpt-6-sol", "pair": "pair_10", "game_a": "game_23", "color_a": "black", "game_b": "game_09", "color_b": "black", "rating_a": 564, "rating_b": 2177, "gap": 1613, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.02, "correct": 1, "brier": 0.0004, "old_estimate_a": 1450, "old_estimate_b": 1950, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s Black played a coherent queenside plan with 10...b5 and 12...Rb8, then used 19...Nd4 and 21...Bxc3 to win material before converting with active rooks. Game A’s Black made a speculative 13...Bxh3 sacrifice and followed it with 15...Bh4, allowing 16.Nxh4 and leaving White with an extra knight. That large tactical oversight makes B the much likelier stronger player, despite the uncertainty of judging one game each.", "confidence_valid": true }, { "model": "openai/gpt-6.1-sol", "pair": "pair_10", "game_a": "game_23", "color_a": "black", "game_b": "game_09", "color_b": "black", "rating_a": 564, "rating_b": 2177, "gap": 1613, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.98, "correct": 1, "brier": null, "old_estimate_a": 1300, "old_estimate_b": 2100, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B Black shows coherent Benoni play: 10...b5, the knight maneuver ...Ng4–e5–c4, and 19...Nd4 followed by ...Nxe2 and ...Bxc3 simplify effectively and win queenside pawns. Black then coordinates the rooks with 25...Rb2 and finishes with 38...Rdd2, 39...Rh2+ and 40...Rbg2+, creating a mating net. Game A Black’s 13...Bxh3, 14...f5 and 15...Bh4 sacrifice substantial material without a convincing continuation; the subsequent queen checks produce little. B’s sustained positional planning and tactical execution strongly suggest the higher established rapid strength.", "confidence_valid": false }, { "model": "openai/gpt-5.6-luna", "pair": "pair_10", "game_a": "game_23", "color_a": "black", "game_b": "game_09", "color_b": "black", "rating_a": 564, "rating_b": 2177, "gap": 1613, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.04, "correct": 1, "brier": 0.0016, "old_estimate_a": 1250, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s Black shows substantially stronger strategic and tactical play: the queenside expansion with 10...b5, active maneuvering via 15...Nc4 and 19...Nd4, and the sequence 20...Nxe2 21...Bxc3 won material cleanly. Black then coordinated the rooks effectively with ...Rb2, ...Rd8, and ...Rd2, finishing with a forcing attack. Game A’s Black played 12...Be7 and then sacrificed both bishops with 13...Bxh3 and 15...Bh4, apparently without sufficient compensation; White simply took them with 14 gxh3 and 16 Nxh4. The ensuing queen attack was energetic but failed to produce a result, indicating much weaker judgment than Game B’s Black.", "confidence_valid": true }, { "model": "openai/gpt-6-astra", "pair": "pair_10", "game_a": "game_23", "color_a": "black", "game_b": "game_09", "color_b": "black", "rating_a": 564, "rating_b": 2177, "gap": 1613, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.98, "correct": 1, "brier": null, "old_estimate_a": 1300, "old_estimate_b": 2100, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B Black showed coherent Benoni counterplay with 10...b5 and 12...Rb8, purposeful knight maneuvering through ...Ng4–e5–c4, and 19...Nd4 followed by exchanges that exposed queenside pawns. The invasion 25...Rb2 and coordinated finish 38...Rdd2, 40...Rbg2+, 41...f3 demonstrate strong tactical and positional control. Game A Black developed reasonably but played the speculative 13...Bxh3 followed by 14...f5, leaving inadequate compensation after 15.Nxf5. Subsequent queen checks never established a convincing attack. B’s sustained coordination and conversion are substantially stronger evidence than A’s unsupported sacrifice.", "confidence_valid": false }, { "model": "z-ai/glm-5.3-flash", "pair": "pair_10", "game_a": "game_23", "color_a": "black", "game_b": "game_09", "color_b": "black", "rating_a": 564, "rating_b": 2177, "gap": 1613, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.25, "correct": 1, "brier": 0.0625, "old_estimate_a": 1450, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black loses material cheaply: 13...Bxh3? 14.gxh3 then 14...f5? 15.Nxf5 Bxf5? loses a piece for little, and the attack fizzles while White simply consolidates. Game B Black plays a coherent Benko/Benoni-style game: timely ...b5 and knight redeployment, regains material (21...Bxc3, 22...Qxa3), grabs queenside pawns (23...Qxc3, 25...Rxa7), keeps coordination with ...Rb2 infiltration, and converts to a clean win despite complex defense. The difference in positional understanding and conversion skill favors B clearly.", "confidence_valid": true } ], "a": { "game": "game_23", "color": "black", "rating": 564, "url": "https://lichess.org/LDkelAog", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"1-0\"]\n[TimeControl \"600+0\"]\n\n1. d4 d5 2. Bf4 Nf6 3. h3 Nc6 4. e3 Bf5 5. Bb5 e6 6. Bg5 Be7 7. Bxf6 Bxf6 8.\nNf3 O-O 9. O-O a6 10. Bxc6 bxc6 11. Nbd2 e5 12. dxe5 Be7 13. Nd4 Bxh3 14. gxh3\nf5 15. Nxf5 Bh4 16. Nxh4 Qxh4 17. f3 Qg5+ 18. Kh1 Qxe5 19. Re1 Qg3 20. Qe2\nQxh3+ 21. Qh2 Qe6 22. Qe5 Qg6 23. Rg1 Qh6+ 24. Qh2 1-0" }, "b": { "game": "game_09", "color": "black", "rating": 2177, "url": "https://lichess.org/tGYghpt5", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"0-1\"]\n[TimeControl \"600+0\"]\n\n1. d4 Nf6 2. c4 c5 3. d5 d6 4. Nc3 g6 5. e4 Bg7 6. g3 O-O 7. Bg2 Na6 8. a3 Bd7\n9. Nge2 Nc7 10. O-O b5 11. cxb5 Nxb5 12. Bd2 Rb8 13. Qc1 Ng4 14. h3 Ne5 15. Kh2\nNc4 16. Rb1 Nxd2 17. Qxd2 Qa5 18. Qc2 Rfc8 19. f4 Nd4 20. Qd3 Nxe2 21. Qxe2\nBxc3 22. bxc3 Qxa3 23. Ra1 Qxc3 24. Rfc1 Qd4 25. Rxa7 Rb2 26. Qf3 Qd2 27. Rg1\nRd8 28. e5 Bf5 29. g4 Bb1 30. f5 dxe5 31. fxg6 hxg6 32. Rxe7 Qf4+ 33. Qxf4 exf4\n34. Kh1 Ba2 35. Rc1 Bxd5 36. Bxd5 Rxd5 37. Re8+ Kg7 38. Rc8 Rdd2 39. R8xc5 Rh2+\n40. Kg1 Rbg2+ 41. Kf1 f3 0-1" }, "wrong": 0, "gap": 1613 }, { "id": "pair_21", "predictions": [ { "model": "anthropic/claude-opus-5.5", "pair": "pair_21", "game_a": "game_21", "color_a": "white", "game_b": "game_15", "color_b": "white", "rating_a": 739, "rating_b": 2157, "gap": 1418, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.2, "correct": 1, "brier": 0.04000000000000001, "old_estimate_a": 1250, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White plays an unsound opening (2.Qd3, 4.f4) and later blunders badly: 11.Rc1/12.Rxc7 lets Black's bishop roam, 19.a3? allows ...Nc2+ and ...Nxf1+, and 24.a4 drops more material. White did fight back, as Black blundered too (26...Nd5, 30...Nxh1), but later 39.Ne5+?? gives a knight away and 66.Nxh3?? loses to the fork ...Nf2+. Game B White plays a coherent Slav, wins the b-pawn and a piece via 24.Rd1/25.Qxc8+ tactics, and handles the minor-piece endgame reasonably. White then lost the king-and-pawn ending's winning chances but still held a correct draw (57.Kh1, 58.Kh2). Game B's play is clearly more competent.", "confidence_valid": true }, { "model": "anthropic/claude-sonnet-5.5", "pair": "pair_21", "game_a": "game_21", "color_a": "white", "game_b": "game_15", "color_b": "white", "rating_a": 739, "rating_b": 2157, "gap": 1418, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.25, "correct": 1, "brier": 0.0625, "old_estimate_a": 1450, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White played 2.Qd3, an early queen sortie, then 4.f4 and 5.Qxd5. Grabbing 7.Qxb7 allowed 7...Qb4+ and an immediate queen trade, with Black's rook and bishop active. It later lost the h1 rook to 30...Nxh1 and lost the endgame. These are weak, beginner-like choices. Game B White followed standard Queen's Gambit Accepted theory and developed sensibly. It kept the structure sound with 9.Nbd2, 10.O-O and 13.Bxa6. It traded into an even endgame with 25.Qxc8+ and 26.Rxd5. It held a draw accurately against a decent opponent with Black's active king play. B looks clearly stronger and more consistent, though the sample is small.", "confidence_valid": true }, { "model": "google/gemini-3.8-flash", "pair": "pair_21", "game_a": "game_21", "color_a": "white", "game_b": "game_15", "color_b": "white", "rating_a": 739, "rating_b": 2157, "gap": 1418, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.02, "correct": 1, "brier": 0.0004, "old_estimate_a": 1250, "old_estimate_b": 2200, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White demonstrates beginner-level play, starting with the awkward 2. Qd3 and repeatedly missing simple tactical combinations, allowing 20...Ne3+ winning a bishop on f1 and later blundering an entire rook via 28...Ng4 and 29...Nf2. In contrast, Game B White shows advanced positional and tactical understanding in a theoretical Slav Defense, executing a sharp queen-sac sequence (23. dxc5 Nxc5 24. Rd1 Nxb7 25. Qxc8+!) to reach a favorable endgame, which was then converted into a theoretically precise draw. Target B is far stronger.", "confidence_valid": true }, { "model": "google/gemini-3.1-pro-preview", "pair": "pair_21", "game_a": "game_21", "color_a": "white", "game_b": "game_15", "color_b": "white", "rating_a": 739, "rating_b": 2157, "gap": 1418, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.02, "correct": 1, "brier": 0.0004, "old_estimate_a": 1000, "old_estimate_b": 2250, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Target A (White) played a highly irregular, beginner-level opening with 2. Qd3 and later completely collapsed tactically, blundering a knight fork (19. a3 Nc2+) and a full rook (29. exd5 Nf2). In contrast, Target B (White) navigated a theoretical Slav Defense (6. Nb1, 7. Bxc4) with deep positional understanding, solid piece coordination, and competent endgame technique to secure a draw in a pawn endgame. Target B's play demonstrates a vastly higher level of tactical and strategic maturity.", "confidence_valid": true }, { "model": "google/gemini-3.5-flash", "pair": "pair_21", "game_a": "game_21", "color_a": "white", "game_b": "game_15", "color_b": "white", "rating_a": 739, "rating_b": 2157, "gap": 1418, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.02, "correct": 1, "brier": 0.0004, "old_estimate_a": 1100, "old_estimate_b": 2000, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White shows low-level play, starting with the bizarre 2. Qd3, and subsequently blundering a full piece with 21. Kd2? allowing 21... Nxf1+. In contrast, Game B White exhibits sophisticated positional understanding in a main-line Slav Defense, managing complex queenless middlegame maneuvering (e.g., 14. Nc4, 18. N3a5) and demonstrating precise endgame technique to secure a draw. Game B White is vastly superior in tactical awareness and strategic planning.", "confidence_valid": true }, { "model": "google/gemini-3.7-flash", "pair": "pair_21", "game_a": "game_21", "color_a": "white", "game_b": "game_15", "color_b": "white", "rating_a": 739, "rating_b": 2157, "gap": 1418, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.03, "correct": 1, "brier": 0.0009, "old_estimate_a": 1150, "old_estimate_b": 2150, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White opens with the dubious 2. Qd3 and demonstrates poor tactical awareness throughout the game, allowing Black to win a rook via ...Nf2 (moves 28-30) after previously missing multiple knight forks and hanging pawns. In contrast, Game B White exhibits sound positional understanding in the Slav Defense (4. e3, 5. a4), executes clean tactical simplifications (23. dxc5 through 27. Nxb7), and plays a technically solid knight and pawn endgame leading into a straightforward drawn king-and-pawn finish. Player B demonstrates much higher tactical and endgame competence.", "confidence_valid": true }, { "model": "qwen/qwen3.8-max-prime", "pair": "pair_21", "game_a": "game_21", "color_a": "white", "game_b": "game_15", "color_b": "white", "rating_a": 739, "rating_b": 2157, "gap": 1418, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.22, "correct": 1, "brier": 0.0484, "old_estimate_a": 800, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White blunders the opening with 2.Qd3, loses material through 5.Qxd5/7.Qxb7, and never recovers. Game B White plays a solid Slav (5.a4, 7.Bxc4, 11.Nb3), reaches an equal middlegame, and navigates the complex knight maneuvers (15.Nxa5, 17.Rxa4, 25.Qxc8+) into a drawn king-and-pawn ending with accurate opposition technique. B's opening knowledge and endgame precision far exceed A's.", "confidence_valid": true }, { "model": "anthropic/claude-fable-5.1", "pair": "pair_21", "game_a": "game_21", "color_a": "white", "game_b": "game_15", "color_b": "white", "rating_a": 739, "rating_b": 2157, "gap": 1418, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.1, "correct": 1, "brier": 0.010000000000000002, "old_estimate_a": 850, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White played 2.Qd3 and 5.Qxd5?? 6.fxe5?? 7.Qxb7?? grabbing pawns while undeveloped, then lost the whole queenside and later a rook to knight forks; endgame technique was aimless (Nf3/Ng5 shuffling) and the opponent also blundered badly (e.g. 8.Qxb4 trade allowed, missed wins), suggesting a low-rated pairing. Game B White handled a Slav structure competently, won material with 15.Ncxa5 and 16.Nb7, reached a favorable ending with 25.Qxc8+ 26.Rxd5 liquidation, and even when outplayed in the king-pawn ending found the drawing resource 49.e5 and held a defensive stalemate pattern. Game B White is clearly the stronger player.", "confidence_valid": true }, { "model": "google/gemini-3-flash-preview", "pair": "pair_21", "game_a": "game_21", "color_a": "white", "game_b": "game_15", "color_b": "white", "rating_a": 739, "rating_b": 2157, "gap": 1418, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1100, "old_estimate_b": 2150, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Target A demonstrates a lack of basic opening principles and tactical awareness, playing 2. Qd3 and later hanging a piece to a simple sequence (20. Kd1 Ne3+ 21. Kd2 Nxf1+). The endgame play in Game A is erratic, with multiple blunders from both sides. In contrast, Target B shows a solid grasp of opening theory in a complex Slav line (3. Nc3 dxc4 4. e3 b5 5. a4). Target B handles a delicate positional struggle, accurately navigates a queen trade at move 25, and demonstrates precise knowledge of the drawn king-and-pawn endgame. The difference in technical proficiency between the two players is substantial.", "confidence_valid": true }, { "model": "google/gemini-3.6-flash", "pair": "pair_21", "game_a": "game_21", "color_a": "white", "game_b": "game_15", "color_b": "white", "rating_a": 739, "rating_b": 2157, "gap": 1418, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1150, "old_estimate_b": 2150, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White demonstrates weak opening knowledge with 2. Qd3 and poor tactical vision, blundering material with 21. Kd2 (allowing 21...Nxf1+) and failing to see 29...Nf2 winning the rook on h1. In contrast, Game B White plays sound, standard opening theory in the Slav Defense (1. d4, 2. c4, 3. Nc3), shows good positional understanding, cleanly wins a pawn with 15. Ncxa5, and maintains strong tactical stability throughout the game into a classic drawn pawn endgame. Game B White exhibits vastly superior opening preparation, tactical awareness, and overall playing strength.", "confidence_valid": true }, { "model": "openai/gpt-5.6-sol", "pair": "pair_21", "game_a": "game_21", "color_a": "white", "game_b": "game_15", "color_b": "white", "rating_a": 739, "rating_b": 2157, "gap": 1418, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.06, "correct": 1, "brier": 0.0036, "old_estimate_a": 1350, "old_estimate_b": 2100, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B White showed much stronger opening discipline and calculation: standard Slav development, then the forcing sequence 24.Rd1 Nxb7 25.Qxc8+ Nxc8 26.Rxd5 exd5 27.Nxb7, followed by purposeful simplification with 31.Bxb4 and 34.Bxc5. Game A White’s 2.Qd3 and repeated pawn-grabbing with 5.Qxd5 and 7.Qxb7 neglected development. More seriously, 19.a3? allowed 19...Nc2+ and the knight’s checking route to 21...Nxf1+, costing a rook and forcing prolonged king exposure. Although A resisted for many moves, B’s play was substantially more coherent and tactically controlled.", "confidence_valid": true }, { "model": "openai/gpt-6-sol", "pair": "pair_21", "game_a": "game_21", "color_a": "white", "game_b": "game_15", "color_b": "white", "rating_a": 739, "rating_b": 2157, "gap": 1418, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.04, "correct": 1, "brier": 0.0016, "old_estimate_a": 1250, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s White developed and castled normally, then handled the tactical sequence 24.Rd1 Nxb7 25.Qxc8+ Nxc8 26.Rxd5 to reach a manageable endgame. Game A’s White spent the opening moving the queen with 2.Qd3, 5.Qxd5 and 7.Qxb7 while leaving pieces undeveloped, then allowed 19...Nc2+ and 21...Nxf1+ to cost material. B’s play is much stronger evidence of established rapid strength, despite the different results.", "confidence_valid": true }, { "model": "openai/gpt-6.1-sol", "pair": "pair_21", "game_a": "game_21", "color_a": "white", "game_b": "game_15", "color_b": "white", "rating_a": 739, "rating_b": 2157, "gap": 1418, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.98, "correct": 1, "brier": null, "old_estimate_a": 1100, "old_estimate_b": 1900, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B White shows substantially more coherent development, castling and coordination. The sequence 24.Rd1, 25.Qxc8+ and 26.Rxd5 navigates the tactical exchanges effectively; later, 53.Kf1–55.Kh1 secures the drawing corner against Black’s rook pawn. Game A White opens with queen excursions and f-pawn moves, then allows Black’s knight raid with 19.a3 and 20.Kd1, losing the f1 bishop. White also permits 29...Nf2 followed by 30...Nxh1, and simply hangs a knight with 39.Ne5+ Rxe5. Despite imperfect play in Game B, the gap in tactical reliability strongly favors B.", "confidence_valid": false }, { "model": "openai/gpt-5.6-luna", "pair": "pair_21", "game_a": "game_21", "color_a": "white", "game_b": "game_15", "color_b": "white", "rating_a": 739, "rating_b": 2157, "gap": 1418, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.93, "correct": 1, "brier": null, "old_estimate_a": 1250, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B shows substantially more controlled play: White develops naturally, castles, coordinates rooks, and executes the concrete sequence 25 Qxc8+ Nxc8 26 Rxd5 exd5, simplifying into a sound minor-piece ending. White then handles the endgame actively with 35 Ne5+ and 39 Nc5+. In Game A, White’s early queen raid (2 Qd3, 5 Qxd5, 7 Qxb7) is speculative, and the resulting play is poorly coordinated: the rook is lost on 15...Kxd8, another rook is left vulnerable to 29...Nf2 and 30...Nxh1, and 39 Ne5+ allows 39...Rxe5. These concrete oversights strongly favor B.", "confidence_valid": false }, { "model": "openai/gpt-6-astra", "pair": "pair_21", "game_a": "game_21", "color_a": "white", "game_b": "game_15", "color_b": "white", "rating_a": 739, "rating_b": 2157, "gap": 1418, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.97, "correct": 1, "brier": null, "old_estimate_a": 1100, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "B’s White shows much more coherent development and tactical calculation: 24.Rd1 followed by 25.Qxc8+ and 26.Rxd5 liquidates accurately into a balanced minor-piece ending. Later, 49.e5 helps eliminate Black’s central pawns, and White reaches a drawn rook-pawn ending. A’s White opens with premature queen adventures (2.Qd3, 5.Qxd5, 7.Qxb7), leaves the kingside undeveloped, and allows 19...Nc2+ followed by ...Ne3+ and ...Nxf1. Most tellingly, 39.Ne5+ simply hangs a knight to 39...Rxe5. Both opponents make mistakes, but B demonstrates substantially stronger coordination and tactical awareness.", "confidence_valid": false }, { "model": "z-ai/glm-5.3-flash", "pair": "pair_21", "game_a": "game_21", "color_a": "white", "game_b": "game_15", "color_b": "white", "rating_a": 739, "rating_b": 2157, "gap": 1418, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.15, "correct": 1, "brier": 0.0225, "old_estimate_a": 1350, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B's White plays a coherent Semi-Slav setup (Nc3, Bxc4, regulated queenside maneuvers like 15.Nxa5 and 19.Qc2), trades into a safe rook endgame and holds a draw with accurate king play (46.Kd2, 57.Kh2). Game A's White opens with the weak 2.Qd3, loses material via 5.Qxd5 and the forced 7.Qxb7, and needs Black's own errors to reach an endgame, which White then misplayes (44.a4, 45.Kd5 letting the king through) and loses. B's technique and sound opening play indicate a clearly higher rating.", "confidence_valid": true } ], "a": { "game": "game_21", "color": "white", "rating": 739, "url": "https://lichess.org/NxjNr7LI", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"0-1\"]\n[TimeControl \"600+0\"]\n\n1. d4 d5 2. Qd3 e5 3. dxe5 Qe7 4. f4 f6 5. Qxd5 fxe5 6. fxe5 Bf5 7. Qxb7 Qb4+\n8. Qxb4 Bxb4+ 9. Bd2 Bxd2+ 10. Nxd2 Bxc2 11. Rc1 Ba4 12. Rxc7 Na6 13. Rc4 Bb5\n14. Rd4 Rd8 15. Rxd8+ Kxd8 16. Nb3 Ba4 17. Nc1 Nb4 18. b3 Bb5 19. a3 Nc2+ 20.\nKd1 Ne3+ 21. Kd2 Nxf1+ 22. Ke1 Ne3 23. Kd2 Nxg2 24. a4 Nf4 25. e3 Bxa4 26. bxa4\nNd5 27. Nb3 Nh6 28. e4 Ng4 29. exd5 Nf2 30. Ke2 Nxh1 31. Nf3 Re8 32. Ng5 h6 33.\nNf7+ Ke7 34. Nd6 Rd8 35. Nb5 Rxd5 36. Nxa7 Rxe5+ 37. Kd3 Rh5 38. Nc6+ Kd7 39.\nNe5+ Rxe5 40. Kd4 Re2 41. h4 Rh2 42. Nc5+ Kc6 43. Ne4 Rxh4 44. a5 Kb7 45. Kd5\nKa6 46. Nc5+ Kxa5 47. Nd7 Rh2 48. Ne5 Rd2+ 49. Kc4 Ka4 50. Nf7 Rc2+ 51. Kd3 Kb3\n52. Ke4 Rc3 53. Kf5 Re3 54. Kg6 h5 55. Kxg7 Rg3+ 56. Kh6 Rh3 57. Ne5 Rh2 58.\nNf3 Rh3 59. Ng5 Rh2 60. Nf3 Rf2 61. Ng5 Ng3 62. Nh3 Rh2 63. Nf4 h4 64. Kg5 h3\n65. Kg4 Ne4 66. Nxh3 Nf2+ 67. Kg3 Rxh3+ 68. Kxf2 Rd3 69. Ke2 Kc2 0-1" }, "b": { "game": "game_15", "color": "white", "rating": 2157, "url": "https://lichess.org/aBhzitCG", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"1/2-1/2\"]\n[TimeControl \"600+0\"]\n\n1. d4 d5 2. c4 c6 3. Nc3 dxc4 4. e3 b5 5. a4 b4 6. Nb1 Nf6 7. Bxc4 e6 8. Nf3 a5\n9. Nbd2 Be7 10. O-O O-O 11. Nb3 Ba6 12. Nfd2 Nbd7 13. Bxa6 Rxa6 14. Nc4 Nb6 15.\nNcxa5 Qd5 16. Nb7 Rxa4 17. Rxa4 Nxa4 18. N3a5 Nb6 19. Qc2 Rc8 20. Bd2 h6 21.\nRc1 Ne4 22. Be1 c5 23. dxc5 Nxc5 24. Rd1 Nxb7 25. Qxc8+ Nxc8 26. Rxd5 exd5 27.\nNxb7 Nb6 28. b3 Kf8 29. Na5 Nd7 30. Nc6 Nc5 31. Bxb4 Bd6 32. Kf1 Ke8 33. Ke2\nKd7 34. Bxc5 Bxc5 35. Ne5+ Ke6 36. Nd3 Bd6 37. g3 g5 38. b4 Kd7 39. Nc5+ Bxc5\n40. bxc5 Kc6 41. Kd3 Kxc5 42. e4 d4 43. f4 f6 44. h4 gxh4 45. gxh4 h5 46. Kd2\nKc4 47. Ke2 d3+ 48. Kd2 Kd4 49. e5 fxe5 50. fxe5 Kxe5 51. Kxd3 Kf4 52. Ke2 Kg3\n53. Kf1 Kxh4 54. Kg1 Kh3 55. Kh1 Kg3 56. Kg1 h4 57. Kh1 Kf2 58. Kh2 1/2-1/2" }, "wrong": 0, "gap": 1418 }, { "id": "pair_14", "predictions": [ { "model": "anthropic/claude-opus-5.5", "pair": "pair_14", "game_a": "game_18", "color_a": "black", "game_b": "game_14", "color_b": "white", "rating_a": 2641, "rating_b": 1249, "gap": 1392, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.78, "correct": 1, "brier": 0.04839999999999999, "old_estimate_a": 1600, "old_estimate_b": 1250, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black played a coherent game. After the slightly odd 10...Kf8, the plan of ...h5-h4 opened the h-file, and 15...hxg3 and 16...Bh3 traded off White's key bishop. Black finished tactically with 30...Re2+, 31...Rh2 and 33...Qxg3+. Game B White blundered badly. 8.Nh4 followed by 9.Qf3 let 9...Nd5 and 10...Bxh4 win a knight, so 10.Bxd5 simply lost a piece. Later 30.Rg6?? left the d3 bishop hanging to 30...Rxd3. These are basic tactical oversights, while Black in Game A showed better understanding and calculation.", "confidence_valid": true }, { "model": "anthropic/claude-sonnet-5.5", "pair": "pair_14", "game_a": "game_18", "color_a": "black", "game_b": "game_14", "color_b": "white", "rating_a": 2641, "rating_b": 1249, "gap": 1392, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.66, "correct": 1, "brier": 0.11559999999999998, "old_estimate_a": 1600, "old_estimate_b": 1500, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A, Black: answered the Sicilian setup well with ...d5 and ...Nf6, and took the h-file with ...h5-h4 and ...hxg3. Black won with 26...Bxc3 and 28...Nxd5, and finished with ...Qh3+ and ...Qxg3+. ...Kf8 and ...h5 were a bit loose, but White's play was also weak (Bxc5+ and Qxd5 did not hold). Game B, White: opened with 2.Bc4 and 4.e5, and the 8.Nh4 and 9.Qf3 plan lost a piece after 10...Bxh4. Black's play was fairly accurate there, but White's blunders were basic. White also dropped a bishop with 30.Rg6 Rxd3. Both games show club-level play, but White in B made the larger errors, so target A looks somewhat stronger.", "confidence_valid": true }, { "model": "google/gemini-3.8-flash", "pair": "pair_14", "game_a": "game_18", "color_a": "black", "game_b": "game_14", "color_b": "white", "rating_a": 2641, "rating_b": 1249, "gap": 1392, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.83, "correct": 1, "brier": 0.028900000000000012, "old_estimate_a": 1850, "old_estimate_b": 1350, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Player A (Black) conducted an energetic, purposeful kingside attack featuring early dynamic play with ...h5-h4. When White miscalculated with 30. Rd1, Black punished it crisply with 30...Re2+! followed by 31...Rh2! and 32...Qh3+, forcing resignation due to unstoppable mate. In contrast, Player B (White) displayed weak positional understanding with moves like 8. Nh4, outright hung a knight on e4 at move 15 (which Black missed), and eventually resigned after blatantly hanging the d3-bishop in one move with 30. Rg6??. Target A demonstrates far superior board vision and tactical sharpness.", "confidence_valid": true }, { "model": "google/gemini-3.1-pro-preview", "pair": "pair_14", "game_a": "game_18", "color_a": "black", "game_b": "game_14", "color_b": "white", "rating_a": 2641, "rating_b": 1249, "gap": 1392, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.95, "correct": 1, "brier": 0.0025000000000000044, "old_estimate_a": 2000, "old_estimate_b": 1100, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Black in Game A demonstrated strong tactical awareness and attacking play, launching a successful kingside attack with moves like 11...h5 and finishing with the crushing sequence starting with 31...Rh2!. In contrast, White in Game B showed a lack of basic board awareness, hanging a piece in the opening with 10. Bxd5 Bxh4 (leaving the h4 knight unprotected) and blundering a bishop at the end with 30. Rg6 Rxd3. Target A is clearly the stronger player.", "confidence_valid": true }, { "model": "google/gemini-3.5-flash", "pair": "pair_14", "game_a": "game_18", "color_a": "black", "game_b": "game_14", "color_b": "white", "rating_a": 2641, "rating_b": 1249, "gap": 1392, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.95, "correct": 1, "brier": 0.0025000000000000044, "old_estimate_a": 1850, "old_estimate_b": 1150, "baseline_choice": "A", "baseline_credit": 1, "explanation": "In Game A, Black demonstrated high-level play, employing a strong kingside attack (11...h5, 14...h4, 16...Bh3) and maintaining excellent piece activity despite losing castling rights. Black's tactical execution was sharp, culminating in a decisive mating net (29...Re8, 30...Re2+, and 32...Qh3+). In contrast, Game B White showed poor tactical awareness, misplacing a knight with 8. Nh4, which allowed 10...Bxh4, and later hanging a bishop with 30. Rg6?? Rxd3. Black in Game A is clearly the much stronger player.", "confidence_valid": true }, { "model": "google/gemini-3.7-flash", "pair": "pair_14", "game_a": "game_18", "color_a": "black", "game_b": "game_14", "color_b": "white", "rating_a": 2641, "rating_b": 1249, "gap": 1392, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.82, "correct": 1, "brier": 0.03240000000000002, "old_estimate_a": 2050, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black displayed strong tactical and positional awareness, conducting a purposeful kingside assault starting with 11...h5, coordinating pieces effectively, and executing a decisive mating sequence with 30...Re2+ and 31...Rh2. In contrast, Game B White played passively and aimlessly (e.g., 8. Nh4, allowing 15...Qxe4 which Black missed), before abruptly blundering a full minor piece with 30. Rg6?? Rxd3. Target A demonstrated much higher tactical vigilance and overall strategic cohesion.", "confidence_valid": true }, { "model": "qwen/qwen3.8-max-prime", "pair": "pair_14", "game_a": "game_18", "color_a": "black", "game_b": "game_14", "color_b": "white", "rating_a": 2641, "rating_b": 1249, "gap": 1392, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.72, "correct": 1, "brier": 0.07840000000000001, "old_estimate_a": 1350, "old_estimate_b": 1200, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black shows coherent Sicilian plans: ...h5-hxg3 opening the h-file, the strong 20...Ng8 redeployment, and the clean finish 30...Re2+, 32...Qh3+, 33...Qxg3+. Errors are modest (10...Kf8). Game B White plays dubious moves: 8.Nh4 wasting time, the unnecessary 10.Bxd5 giving up the bishop pair, awkward 17.Nd6, passive 21.c4, and the losing tactical oversight allowing 30...Rxd3. White's positional drift and final collapse indicate noticeably weaker play than Black's purposeful, accurate attacking execution.", "confidence_valid": true }, { "model": "anthropic/claude-fable-5.1", "pair": "pair_14", "game_a": "game_18", "color_a": "black", "game_b": "game_14", "color_b": "white", "rating_a": 2641, "rating_b": 1249, "gap": 1392, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.6, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 1650, "old_estimate_b": 1250, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black played a reasonable Sicilian setup, handled 10.Re1+ Kf8 by going for a kingside attack (11...h5, 14...h4, 16...Bh3) and converted tactically with 29...Re8, 30...Re2+ and 32...Qh3+ winning material, though 20...Ng8 and 22...Be5 were loose. Game B White's play was weaker: 2.Bc4 and 4.e5 sideline, 8.Nh4?/9.Qf3 lost a piece to 9...Nd5 10.Bxd5 Bxh4, then aimless knight shuffling (17.Nd6 18.Ne4) and finally 30.Rg6?? dropping the bishop to 30...Rxd3. Opponents in both games were modest, but A's target showed better tactical awareness.", "confidence_valid": true }, { "model": "google/gemini-3-flash-preview", "pair": "pair_14", "game_a": "game_18", "color_a": "black", "game_b": "game_14", "color_b": "white", "rating_a": 2641, "rating_b": 1249, "gap": 1392, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.85, "correct": 1, "brier": 0.022500000000000006, "old_estimate_a": 1900, "old_estimate_b": 1250, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Player A (Black) demonstrates sophisticated tactical and attacking skills in a sharp Sicilian. Navigating a king-in-center position after 10...Kf8, they correctly prioritized a kingside initiative (11...h5, 14...h4). The concluding sequence (30...Re2+ to 33...Qxg3+) shows precise calculation. In contrast, Player B (White) played a more aimless middlegame, exemplified by the unnecessary 10.Bxd5. Crucially, Player B committed a primitive tactical error with 30.Rg6, simply leaving the d3-bishop en prise to 30...Rxd3. The difference in tactical alertness and strategic depth between the two players is significant.", "confidence_valid": true }, { "model": "google/gemini-3.6-flash", "pair": "pair_14", "game_a": "game_18", "color_a": "black", "game_b": "game_14", "color_b": "white", "rating_a": 2641, "rating_b": 1249, "gap": 1392, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.85, "correct": 1, "brier": 0.022500000000000006, "old_estimate_a": 1900, "old_estimate_b": 1150, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black played a remarkably sharp and accurate tactical game despite losing castling rights early (10...Kf8). Black organized a powerful Kingside attack with 14...h4, 16...Bh3, and finished with a decisive mating sequence involving 30...Re2+, 31...Rh2, and 33...Qxg3+. In contrast, Game B White displayed poor opening understanding with 8. Nh4, allowing Black to win a piece for nothing with 9...Nd5 10. Bxd5 Bxh4. Game B White later blundered a full piece outright with 30. Rg6 Rxd3. Game A Black demonstrated far superior tactical alertness and calculation.", "confidence_valid": true }, { "model": "openai/gpt-5.6-sol", "pair": "pair_14", "game_a": "game_18", "color_a": "black", "game_b": "game_14", "color_b": "white", "rating_a": 2641, "rating_b": 1249, "gap": 1392, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.9, "correct": 1, "brier": 0.009999999999999995, "old_estimate_a": 1800, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black showed coherent opening play and strong tactical conversion: 29...Re8, 30...Re2+ and 31...Rh2 created a decisive attack, while 32...Qh3+ punished White’s greedy 32.Qxd5. Game B White used several inefficient moves, notably 8.Nh4 and repeated queen/knight maneuvers, then apparently resigned after 30...Rxd3 despite the immediate 31.Rxe8+, which wins Black’s hanging rook with check. Missing that elementary resource at the game’s conclusion is much stronger evidence of lower established strength than Game A Black’s occasional awkwardness such as 20...Ng8.", "confidence_valid": true }, { "model": "openai/gpt-6-sol", "pair": "pair_14", "game_a": "game_18", "color_a": "black", "game_b": "game_14", "color_b": "white", "rating_a": 2641, "rating_b": 1249, "gap": 1392, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.91, "correct": 1, "brier": 0.008099999999999994, "old_estimate_a": 1750, "old_estimate_b": 1550, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A’s Black coordinated a kingside attack with 14...h4, 16...Bh3, and the decisive 30...Re2+ 31...Rh2 32...Qh3+. Game B’s White developed some active ideas, but after 29...Rd4 attacked the bishop on d3, 30.Rg6 allowed 30...Rxd3, losing it. A’s sustained tactical play is stronger evidence of established strength, despite the risk of 10...Kf8.", "confidence_valid": true }, { "model": "openai/gpt-6.1-sol", "pair": "pair_14", "game_a": "game_18", "color_a": "black", "game_b": "game_14", "color_b": "white", "rating_a": 2641, "rating_b": 1249, "gap": 1392, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.8, "correct": 1, "brier": 0.03999999999999998, "old_estimate_a": 1750, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black showed more coherent tactical play: 14...h4 and 16...Bh3 opened attacking lines, while 29...Re8, 30...Re2+, and 31...Rh2 coordinated the remaining heavy pieces against White’s king. After 32.Qxd5, Black continued with forcing checks rather than abandoning the attack. Game B White developed reasonably and gained central space, but the knight excursion 17.Nd6–18.Ne4 achieved little. More importantly, 30.Rg6 overlooked the direct capture 30...Rxd3, losing a bishop without evident compensation. A’s sustained attacking coordination is stronger evidence than B’s otherwise plausible buildup.", "confidence_valid": true }, { "model": "openai/gpt-5.6-luna", "pair": "pair_14", "game_a": "game_18", "color_a": "black", "game_b": "game_14", "color_b": "white", "rating_a": 2641, "rating_b": 1249, "gap": 1392, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.74, "correct": 1, "brier": 0.06760000000000001, "old_estimate_a": 1850, "old_estimate_b": 1500, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A’s black showed the more convincing tactical and strategic play. After the awkward-looking 10...Kf8, Black built a coherent kingside attack with ...h5, ...h4, ...Bh3, and the rook maneuvers ...Re2+ and ...Rh2. The final 32...Qh3+ and 33...Qxg3+ demonstrated accurate attacking calculation. Game B’s white played actively but less consistently: 8 Nh4 and 9 Qf3 allowed Black comfortable development, while 24 f4 and 27 Rg3 left the position vulnerable. After 28 Qe5 Qxe5 29 fxe5, White failed to prevent 30...Rxd3, losing the bishop and ending the game. Thus A’s target more likely had the higher established rapid strength.", "confidence_valid": true }, { "model": "openai/gpt-6-astra", "pair": "pair_14", "game_a": "game_18", "color_a": "black", "game_b": "game_14", "color_b": "white", "rating_a": 2641, "rating_b": 1249, "gap": 1392, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.85, "correct": 1, "brier": 0.022500000000000006, "old_estimate_a": 1800, "old_estimate_b": 1500, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black showed coherent attacking play: 11...h5 and 14...h4 opened the h-file, while 29...Re8, 30...Re2+ and 31...Rh2 coordinated the heavy pieces against White’s king. After 32.Qxd5, Black exploited the exposed king with ...Qh3+ and ...Qxg3+. Game B White developed actively and found useful moves such as 17.Nd6, but 8.Nh4 spent time without a clear payoff, and 30.Rg6 simply left the bishop on d3 hanging to ...Rxd3. A’s play was not flawless and benefited from White’s mistakes, but its sustained coordination suggests greater established strength than B’s attack and elementary final oversight.", "confidence_valid": true }, { "model": "z-ai/glm-5.3-flash", "pair": "pair_14", "game_a": "game_18", "color_a": "black", "game_b": "game_14", "color_b": "white", "rating_a": 2641, "rating_b": 1249, "gap": 1392, "gap_band": ">=500", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.75, "correct": 1, "brier": 0.0625, "old_estimate_a": 1750, "old_estimate_b": 1450, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A's Black played forcefully and accurately: 11...h5!, 14...h4! gaining attack, clean rook doubling (16...Bh3, 19...Rxh1), and a precise tactical finish 26...Bxc3, 30...Re2+ 31...Rh2, 32...Qxd5! winning material decisively. Game B's White erred early with 8.Nh4?! (allowing ...g5 ideas) and 9...Nd5 10...Bxh4 regained a piece, later drifting passively (17.Nd6/19.Ne4 shuffling) before 29...Rd4 and 30...Rxd3 collapsed tactically. Black's sharp, coherent attacking play and conversion in A exceeds White's solid but blunder-prone effort in B.", "confidence_valid": true } ], "a": { "game": "game_18", "color": "black", "rating": 2641, "url": "https://lichess.org/ZPxafolO", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"0-1\"]\n[TimeControl \"600+0\"]\n\n1. e4 c5 2. Nf3 e6 3. b3 a6 4. g3 Nc6 5. Bg2 d5 6. exd5 exd5 7. O-O Nf6 8. d4\ncxd4 9. Bb2 Bc5 10. Re1+ Kf8 11. Nxd4 h5 12. Nxc6 bxc6 13. Bd4 Bd6 14. c4 h4\n15. Nc3 hxg3 16. hxg3 Bh3 17. Bxh3 Rxh3 18. Kg2 Rh6 19. Rh1 Rxh1 20. Qxh1 Ng8\n21. cxd5 c5 22. Be3 Be5 23. Rc1 Bd4 24. Qe1 Nf6 25. f3 Qd7 26. Qd2 Bxc3 27.\nBxc5+ Kg8 28. Qxc3 Nxd5 29. Qd4 Re8 30. Rd1 Re2+ 31. Kf1 Rh2 32. Qxd5 Qh3+ 33.\nKe1 Qxg3+ 0-1" }, "b": { "game": "game_14", "color": "white", "rating": 1249, "url": "https://lichess.org/tqOYQRH5", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"0-1\"]\n[TimeControl \"600+0\"]\n\n1. e4 d6 2. Bc4 e6 3. Nf3 h6 4. e5 c6 5. exd6 Bxd6 6. O-O Be7 7. d4 Nf6 8. Nh4\nO-O 9. Qf3 Nd5 10. Bxd5 Bxh4 11. Bc4 Bg5 12. Bxg5 Qxg5 13. Nc3 Re8 14. Ne4 Qf5\n15. Qg3 Rd8 16. Bd3 Na6 17. Nd6 Qf6 18. Ne4 Qg6 19. Qe3 f5 20. Nc5 Nxc5 21.\ndxc5 Rd5 22. c4 Rd8 23. Rae1 Bd7 24. f4 Re8 25. Rf3 Qf6 26. b3 Rad8 27. Rg3 Bc8\n28. Qe5 Qxe5 29. fxe5 Rd4 30. Rg6 Rxd3 0-1" }, "wrong": 0, "gap": 1392 }, { "id": "pair_04", "predictions": [ { "model": "anthropic/claude-opus-5.5", "pair": "pair_04", "game_a": "game_22", "color_a": "black", "game_b": "game_08", "color_b": "black", "rating_a": 557, "rating_b": 1791, "gap": 1234, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.15, "correct": 1, "brier": 0.0225, "old_estimate_a": 1000, "old_estimate_b": 1450, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black played poorly: 6...Nxf2?! was a dubious sacrifice, and 11...Nb5+? simply hung the knight to 13.Qxb5. Black then dropped more material and finished with 22...Qxg1+??, giving away the queen. The opponent's 2.Qf3 and 3.g4 suggest low-level play. Game B Black played a sound opening (...e5, ...d5, ...Bb4, ...O-O) and won with tactics: 15...Ndxc5 recovered material, and 22...Bxf3+ followed by 24...Nf3+ won the exchange and more. The opponent did blunder with 15.Bxc5 and 22.Rd4, but Black's play was coherent and accurate. B looks clearly stronger.", "confidence_valid": true }, { "model": "anthropic/claude-sonnet-5.5", "pair": "pair_04", "game_a": "game_22", "color_a": "black", "game_b": "game_08", "color_b": "black", "rating_a": 557, "rating_b": 1791, "gap": 1234, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.3, "correct": 1, "brier": 0.09, "old_estimate_a": 1450, "old_estimate_b": 1550, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black met 2.Qf3 with decent development, but 5...Ng4 and 6...Nxf2 was an unsound sacrifice. Black followed with 9...Bc5 and 13...Qf2+, yet lost material. Black then blundered with 20...Qf4 and 22...Qxg1+, which gave up the queen, so the game was messy and error-prone. Game B Black played a solid opening with 3...Bb4, 5...O-O and 7...Bg4. The play was coherent: 9...e4 and 10...Nxe4 won a pawn, 14...c5 was a good central break, and 22...Bxf3+ followed by 24...Nf3+ finished cleanly. White in Game B was passive. Black in B appears stronger and more consistent.", "confidence_valid": true }, { "model": "google/gemini-3.8-flash", "pair": "pair_04", "game_a": "game_22", "color_a": "black", "game_b": "game_08", "color_b": "black", "rating_a": 557, "rating_b": 1791, "gap": 1234, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.08, "correct": 1, "brier": 0.0064, "old_estimate_a": 800, "old_estimate_b": 1500, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B's Black played sound, disciplined chess, exploiting the pin on f3 with 9...e4 and winning material cleanly with the 14...c5 fork. Black then executed a completely winning endgame sequence with 20...Bg4 and 21...Ng5 to force White's resignation. In contrast, Game A's Black played an unsound, chaotic attack (6...Nxf2), missed capturing White's queen on move 17 (17...Bf5 instead of 17...Qxc5), and blundered away their queen multiple times (20...Qf4 and 22...Qxg1+), showing very low tactical awareness.", "confidence_valid": true }, { "model": "google/gemini-3.1-pro-preview", "pair": "pair_04", "game_a": "game_22", "color_a": "black", "game_b": "game_08", "color_b": "black", "rating_a": 557, "rating_b": 1791, "gap": 1234, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.02, "correct": 1, "brier": 0.0004, "old_estimate_a": 900, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Black in Game B demonstrates solid, principled play, controlling the center, developing logically, and executing strong tactics such as the 14... c5 fork and the game-winning 24... Nf3+. In stark contrast, Black in Game A plays recklessly and blunders multiple pieces outright, such as hanging a knight with 11... Nb5+ (13. Qxb5) and a bishop with 16... cxd5 (17. Qxc5), before eventually blundering their queen as well. Target B is clearly a much more experienced and competent player.", "confidence_valid": true }, { "model": "google/gemini-3.5-flash", "pair": "pair_04", "game_a": "game_22", "color_a": "black", "game_b": "game_08", "color_b": "black", "rating_a": 557, "rating_b": 1791, "gap": 1234, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1000, "old_estimate_b": 1450, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B Black demonstrates solid positional play, sound development, and sharp tactical awareness. Black exploits a pin with 7... Bg4, spots the tactical piece win 14... c5, and cleanly converts the advantage, finishing with the fork 24... Nf3+. In contrast, Game A Black plays an chaotic game, starting with an unsound sacrifice 6... Nxf2, followed by numerous positional errors, and concluding with a catastrophic blunder of the queen with 22... Qxg1+.", "confidence_valid": true }, { "model": "google/gemini-3.7-flash", "pair": "pair_04", "game_a": "game_22", "color_a": "black", "game_b": "game_08", "color_b": "black", "rating_a": 557, "rating_b": 1791, "gap": 1234, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 850, "old_estimate_b": 1550, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Target A plays a chaotic, low-accuracy game characterized by unsound piece sacrifices (6...Nxf2), huge tactical oversights (failing to capture White's Queen on move 17 with 17...Bf5 instead of 17...Qxc5), and ultimately blundering the Queen entirely (22...Qxg1+). In contrast, Target B demonstrates solid opening principles, spots a clean tactical fork to win material (14...c5!), and smoothly converts the advantage with accurate tactical strikes (21...Ng5 pinning the knight and 24...Nf3+ winning the rook). Target B shows vastly superior tactical vision and positional understanding.", "confidence_valid": true }, { "model": "qwen/qwen3.8-max-prime", "pair": "pair_04", "game_a": "game_22", "color_a": "black", "game_b": "game_08", "color_b": "black", "rating_a": 557, "rating_b": 1791, "gap": 1234, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.3, "correct": 1, "brier": 0.09, "old_estimate_a": 800, "old_estimate_b": 1550, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black (target) collapses tactically: after 5...Ng4 6.h3 Nxf2 7.Kxf2, Black plays 8...Qf6+ 9.Ke3, then later loses material with 11...Nb5+ 12.Ke2 Qf6 13.Qxb5, dropping a knight for nothing. Black's king safety and coordination are poor throughout, and White wins comfortably. Game B Black plays solidly: ...e4 9...Nxe4 gains a central foothold, ...c5 15...Ndxc5 regains the pawn, and the endgame technique is clean—21...Ng5, 22...Bxf3+ 23...Be4, 24...Nf3+ forcing mate threats. Black's play is coherent and decisive, indicating a higher established strength than Game A Black.", "confidence_valid": true }, { "model": "anthropic/claude-fable-5.1", "pair": "pair_04", "game_a": "game_22", "color_a": "black", "game_b": "game_08", "color_b": "black", "rating_a": 557, "rating_b": 1791, "gap": 1234, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.2, "correct": 1, "brier": 0.04000000000000001, "old_estimate_a": 800, "old_estimate_b": 1400, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black played a speculative Alekhine against a weak Qf3/g4 setup but then sacrificed unsoundly (6...Nxf2, 11...Nb5+) and hung the bishop and knight, then blundered the queen with 22...Qxg1+ into a hopeless position; the whole game shows poor calculation and piece-hanging. Game B Black played coherent developing moves (3...Bb4, 8...Bxc3, 9...e4), kept pieces coordinated, exploited White's errors (15...Ndxc5, 17...Nxe4) and finished with a clean tactical sequence 21...Ng5, 22...Bxf3+, 24...Nf3+ winning material. Game B's opponent was also weak, but Black B's play is markedly more sound than Black A's.", "confidence_valid": true }, { "model": "google/gemini-3-flash-preview", "pair": "pair_04", "game_a": "game_22", "color_a": "black", "game_b": "game_08", "color_b": "black", "rating_a": 557, "rating_b": 1791, "gap": 1234, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 850, "old_estimate_b": 1600, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black displays extremely low-level play, losing two minor pieces to simple captures (13. Qxb5 and 17. Qxc5) and hanging their queen with 22... Qxg1+. In contrast, Game B Black follows sound opening principles (2... d5, 5... O-O) and demonstrates tactical competence. Black exploits the pin on the f3 knight with 9... e4 and 21... Ng5, wins a piece with the pawn fork 14... c5, and finishes the game with the fork 24... Nf3+. Game B Black's play is consistent with a much higher rating, whereas Game A Black lacks basic board vision and coordination.", "confidence_valid": true }, { "model": "google/gemini-3.6-flash", "pair": "pair_04", "game_a": "game_22", "color_a": "black", "game_b": "game_08", "color_b": "black", "rating_a": 557, "rating_b": 1791, "gap": 1234, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1100, "old_estimate_b": 1500, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black played wildly and inaccurately, sacrificing material unsoundly with 6...Nxf2, missing basic tactical defenses, and ultimately blundering a queen outright with 22...Qxg1+. In contrast, Game B Black demonstrated solid positional understanding, tactical awareness, and smooth execution. Game B Black effectively capitalized on White's passivity, finding the winning tactic 14...c5 to pick up a piece, followed by a crisp sequence with 20...Bg4, 21...Ng5, and 24...Nf3+ to force resignation. Game B Black clearly displays a much higher established rating.", "confidence_valid": true }, { "model": "openai/gpt-5.6-sol", "pair": "pair_04", "game_a": "game_22", "color_a": "black", "game_b": "game_08", "color_b": "black", "rating_a": 557, "rating_b": 1791, "gap": 1234, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.04, "correct": 1, "brier": 0.0016, "old_estimate_a": 1000, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s Black played a coherent, controlled game: 5...O-O completed king safety, 9...e4 and 14...c5 challenged the center, and 15...Ndxc5 correctly recovered the bishop. After the queen trade, 20...Bg4, 21...Ng5, 22...Bxf3+, and 24...Nf3+ formed a purposeful winning sequence. Game A’s Black instead chose the highly speculative 6...Nxf2, then lost the knight after 11...Nb5+ 13.Qxb5 and a bishop after 15...d5 17.Qxc5. The final 22...Qxg1+ simply allowed 23.Nxg1. B showed substantially stronger coordination, calculation, and risk control.", "confidence_valid": true }, { "model": "openai/gpt-6-sol", "pair": "pair_04", "game_a": "game_22", "color_a": "black", "game_b": "game_08", "color_b": "black", "rating_a": 557, "rating_b": 1791, "gap": 1234, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.01, "correct": 1, "brier": 0.0001, "old_estimate_a": 1150, "old_estimate_b": 1600, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A’s Black played ambitiously, but 6...Nxf2 sacrificed a knight without sufficient compensation. After 11...Nb5+ 12.Ke2 Qf6, White simply took that knight with 13.Qxb5; Black then also allowed 17.Qxc5. Game B’s Black played much more coherently: 14...c5 challenged White’s center, 15...Ndxc5 recaptured the bishop, and 20...Bg4 followed by 22...Bxf3+ and 24...Nf3+ showed coordinated tactics.", "confidence_valid": true }, { "model": "openai/gpt-6.1-sol", "pair": "pair_04", "game_a": "game_22", "color_a": "black", "game_b": "game_08", "color_b": "black", "rating_a": 557, "rating_b": 1791, "gap": 1234, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.97, "correct": 1, "brier": null, "old_estimate_a": 800, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B Black develops coherently with ...O-O, ...Re8 and ...Nd7, then exploits White’s mistakes: 14...c5 15.Bxc5 Ndxc5 wins a bishop, while 22...Bxf3+ wins another piece and 24...Nf3+ forks the king and both rooks. Game A Black attacks an exposed king but repeatedly sacrifices material without sufficient compensation: 6...Nxf2, 11...Nb5+ allowing Qxb5, and ultimately 22...Qxg1+ allowing Nxg1, exchanging the queen for a rook. Although both opponents offer opportunities, B shows much stronger development, tactical execution and material judgment.", "confidence_valid": false }, { "model": "openai/gpt-5.6-luna", "pair": "pair_04", "game_a": "game_22", "color_a": "black", "game_b": "game_08", "color_b": "black", "rating_a": 557, "rating_b": 1791, "gap": 1234, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.08, "correct": 1, "brier": 0.0064, "old_estimate_a": 950, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s Black played a coherent, sound game: rapid development with ...O-O, ...Re8, and ...c5, then accurately recovered the sacrificed bishop after 15.Bxc5 with 15...Ndxc5. The sequence 16...Qb6 and 18...axb6 led to a favorable endgame, and 20...Bg4, 22...Bxf3+, and 24...Nf3+ showed consistent tactical control. Game A’s Black chose a dubious tactical line with 2...e5 and 3...Nc6, allowed the king to be driven around after 7...h6 and 8...Qf6+, and then lost substantial material to 13.Qxb5, 17.Qxc5, and 18.Qxd5. Although White in Game A also played poorly, Black’s handling was much less reliable.", "confidence_valid": true }, { "model": "openai/gpt-6-astra", "pair": "pair_04", "game_a": "game_22", "color_a": "black", "game_b": "game_08", "color_b": "black", "rating_a": 557, "rating_b": 1791, "gap": 1234, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.99, "correct": 1, "brier": null, "old_estimate_a": 850, "old_estimate_b": 1700, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B Black shows coherent development and tactical awareness: 14...c5 attacks White’s queen and bishop, 15...Ndxc5 collects the bishop, and 19...f6 gains time against the knight. The finish 22...Bxf3+ followed by 23...Be4 and 24...Nf3+ creates a king-and-rooks fork. Game A Black attacks White’s exposed king but repeatedly sacrifices material without adequate compensation: 6...Nxf2, 11...Nb5+ allowing Qxb5, and ultimately 22...Qxg1+ surrendering the queen for a rook. Despite Game A White’s highly vulnerable opening, Black cannot exploit it and displays much weaker material judgment.", "confidence_valid": false }, { "model": "z-ai/glm-5.3-flash", "pair": "pair_04", "game_a": "game_22", "color_a": "black", "game_b": "game_08", "color_b": "black", "rating_a": 557, "rating_b": 1791, "gap": 1234, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.3, "correct": 1, "brier": 0.09, "old_estimate_a": 1350, "old_estimate_b": 1900, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A's Black found the strong 5...Ng4 and 6...Nxf2 sacrifice, but then erred badly: 12...Qf6 allowed 13.Qxb5, and after winning material Black drifted (19...Kf6, 20...Qf4, 21...Qg5) and lost the queen with 21...Qg5 22.Rg1 Qxg1+??, resigning from a winning position. Game B's Black played coherently throughout: logical development, 9...e4 seizing the center, accurate tactics 16...Qb6 forcing exchanges, and a clean conversion ending with 22...Bxf3+ 23...Be4 and 24...Nf3+ winning the knight. Consistent sound play and a won-game finish suggest B is the stronger player.", "confidence_valid": true } ], "a": { "game": "game_22", "color": "black", "rating": 557, "url": "https://lichess.org/5mUXnOAF", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"1-0\"]\n[TimeControl \"600+0\"]\n\n1. e4 Nf6 2. Qf3 e5 3. g4 Nc6 4. g5 Nd4 5. Qd3 Ng4 6. h3 Nxf2 7. Kxf2 h6 8.\ngxh6 Qf6+ 9. Ke3 Bc5 10. hxg7 Qxg7 11. c3 Nb5+ 12. Ke2 Qf6 13. Qxb5 Qf2+ 14.\nKd1 c6 15. Qc4 d5 16. exd5 cxd5 17. Qxc5 Bf5 18. Qxd5 Ke7 19. Bb5 Kf6 20. Ne2\nQf4 21. Qxb7 Qg5 22. Rg1 Qxg1+ 23. Nxg1 1-0" }, "b": { "game": "game_08", "color": "black", "rating": 1791, "url": "https://lichess.org/5z2eLce5", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"0-1\"]\n[TimeControl \"600+0\"]\n\n1. Nc3 e5 2. d3 d5 3. e3 Bb4 4. Bd2 Nf6 5. g3 O-O 6. Bg2 c6 7. Nf3 Bg4 8. O-O\nBxc3 9. Bxc3 e4 10. dxe4 Nxe4 11. Bb4 Re8 12. Qd4 Be6 13. Rad1 Nd7 14. Rfe1 c5\n15. Bxc5 Ndxc5 16. Ne5 Qb6 17. Bxe4 Nxe4 18. Qxb6 axb6 19. a3 f6 20. Nf3 Bg4\n21. Kg2 Ng5 22. Rd4 Bxf3+ 23. Kg1 Be4 24. h4 Nf3+ 0-1" }, "wrong": 0, "gap": 1234 }, { "id": "pair_18", "predictions": [ { "model": "anthropic/claude-opus-5.5", "pair": "pair_18", "game_a": "game_24", "color_a": "black", "game_b": "game_12", "color_b": "white", "rating_a": 765, "rating_b": 1877, "gap": 1112, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.25, "correct": 1, "brier": 0.0625, "old_estimate_a": 1050, "old_estimate_b": 1550, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black plays a chaotic opening with many flank pawn moves (4...b5, 5...a5, 8...h5, 9...g5), which leaves the kingside weak. Black then allows 17.Nxf6+ and 20.Nb6 forking the rook. The decisive error is 22...Qd8??, which simply loses the queen to 23.Bxd8. Game B White is imprecise early: 10.Be3 lets Black win the d4 pawn. Afterward, though, White plays coherently. 12.Bb5+ forces 12...c6, then 13.Qxd4 regains the piece, followed by queenside pressure with 17.b4 and 20.bxc5. White finishes with active tactics (22.Bf4+, 25.Be5, 27.Bxh8) and 28.Qg8+. Game B White looks stronger.", "confidence_valid": true }, { "model": "anthropic/claude-sonnet-5.5", "pair": "pair_18", "game_a": "game_24", "color_a": "black", "game_b": "game_12", "color_b": "white", "rating_a": 765, "rating_b": 1877, "gap": 1112, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.3, "correct": 1, "brier": 0.09, "old_estimate_a": 1450, "old_estimate_b": 1550, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black played weakly: 2...d6 and 3...c6 with the pawn pushes 4...b5, 5...a5, 6...b4, 8...h5 and 9...g5 ignored development and weakened the king. 11...c5 and 13...Nd4 left material hanging, and 16...Bg4 allowed 17.Nxf6+. 20...Qc6 then 21...Qxa8 lost to 22.Bd5 and 23.Bxd8, a queen trap-style blunder. White in A was also loose but won tactically. Game B White played a sound opening (Bd3, Nbd2, Be3) and punished 7...Bxf3 and 10...Nxd4 with solid moves like 12.Bb5+, 13.Qxd4 and 17.b4. It then found 22.Bf4+, 23.Bxc4 and 25.Be5, picking up material and finishing with 28.Qg8+. B's opponent was also inaccurate, but White in B played cleanly, while A's Black blundered repeatedly. Black in A looks weaker than White in B.", "confidence_valid": true }, { "model": "google/gemini-3.8-flash", "pair": "pair_18", "game_a": "game_24", "color_a": "black", "game_b": "game_12", "color_b": "white", "rating_a": 765, "rating_b": 1877, "gap": 1112, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.08, "correct": 1, "brier": 0.0064, "old_estimate_a": 1150, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Player B demonstrates far superior tactical vision and positional understanding compared to Player A. In Game B, White cleverly won material with 12. Bb5+ followed by 13. Qxd4, then converted with active piece play (22. Bf4+, 25. Be5, 28. Qg8#). In contrast, Player A pushed pawns recklessly on the flanks leaving their king completely vulnerable (7...f6, 8...h5, 9...g5), and finally blundered their queen outright in one move with 22...Qd8?? allowing 23. Bxd8.", "confidence_valid": true }, { "model": "google/gemini-3.1-pro-preview", "pair": "pair_18", "game_a": "game_24", "color_a": "black", "game_b": "game_12", "color_b": "white", "rating_a": 765, "rating_b": 1877, "gap": 1112, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 850, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Target A (Black in Game A) demonstrates extremely poor fundamentals, making numerous weakening pawn moves in the opening (7...f6, 8...h5, 9...g5) while neglecting development, and ultimately blunders their queen with 22...Qd8??. Target B (White in Game B) plays logically and demonstrates strong tactical awareness, such as recovering a piece with 12. Bb5+ and 13. Qxd4, creating attacking chances with 16. Qa4, exploiting a fork with 25. Be5, and finding a decisive back-rank combination with 28. Qg8+. Target B's play is vastly superior.", "confidence_valid": true }, { "model": "google/gemini-3.5-flash", "pair": "pair_18", "game_a": "game_24", "color_a": "black", "game_b": "game_12", "color_b": "white", "rating_a": 765, "rating_b": 1877, "gap": 1112, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.1, "correct": 1, "brier": 0.010000000000000002, "old_estimate_a": 1150, "old_estimate_b": 1450, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Target B (Game B, White) demonstrated solid tactical awareness. They quickly capitalized on Black's mistake with the combination 12. Bb5+ and 13. Qxd4 to win a piece, and converted the game cleanly, finding the mating sequence starting with 28. Qg8+. Conversely, Target A (Game A, Black) exhibited poor board vision. They missed a simple, winning royal fork with 18...Nf3+ (which would have won White's queen) and later blundered their own queen in a single move with 22...Qd8??. Target B is clearly the stronger, more established player.", "confidence_valid": true }, { "model": "google/gemini-3.7-flash", "pair": "pair_18", "game_a": "game_24", "color_a": "black", "game_b": "game_12", "color_b": "white", "rating_a": 765, "rating_b": 1877, "gap": 1112, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1250, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Target B (White in Game B) demonstrated solid tactical vision and strategic competence. After Black grabbed a central pawn with 10...Nxd4 and 11...Bxd4, White immediately punished the tactic with the intermediate check 12.Bb5+! winning a piece (13.Qxd4). White then converted the advantage cleanly, concluding with 28.Qg8+. In contrast, Target A (Black in Game A) played an opening with excessive weakening pawn pushes (7...f6, 8...h5, 9...g5, 11...c5) leaving the king vulnerable, missed multiple tactics, and ultimately hung the queen outright with 22...Qd8?? allowing 23.Bxd8.", "confidence_valid": true }, { "model": "qwen/qwen3.8-max-prime", "pair": "pair_18", "game_a": "game_24", "color_a": "black", "game_b": "game_12", "color_b": "white", "rating_a": 765, "rating_b": 1877, "gap": 1112, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.12, "correct": 1, "brier": 0.0144, "old_estimate_a": 1100, "old_estimate_b": 1300, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A target (Black) played bizarrely with 3...c6, 4...b5, 7...f6, and 8...h5, weakening the kingside and allowing White a winning attack; after 21.Nxa8 Black resigned at 23.Bxd8, showing weak strategic understanding. Game B target (White) played solidly with d4/Nf3/e3, handled Black's tactics calmly after 10...Nxd4, and converted with accurate moves like 25.Be5, 27.Bxh8, and the mating finish 28.Qg8+, demonstrating clearly superior technique and composure.", "confidence_valid": true }, { "model": "anthropic/claude-fable-5.1", "pair": "pair_18", "game_a": "game_24", "color_a": "black", "game_b": "game_12", "color_b": "white", "rating_a": 765, "rating_b": 1877, "gap": 1112, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.15, "correct": 1, "brier": 0.0225, "old_estimate_a": 900, "old_estimate_b": 1450, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black played a chaotic, non-developing opening (3...c6, 4...b5, 5...a5, 7...f6, 8...h5, 9...g5) and then hung a knight with 16...Bg4 (17.Nxf6+), lost a rook to 20.Nb6 and finally 22...Qd8?? dropping the queen — very weak, beginner-level play. Game B White showed decent fundamentals: 7.h3 provoking Bxf3, recognizing 10.Be3 was fine after Nxd4 since 12.Bb5+ regained material, then exploited Black's loose king with 20.bxc5, 22.Bf4+, 25.Be5 and finished with 28.Qg8+ winning. White in B made some inaccuracies but clearly played coherent chess, far above A's target.", "confidence_valid": true }, { "model": "google/gemini-3-flash-preview", "pair": "pair_18", "game_a": "game_24", "color_a": "black", "game_b": "game_12", "color_b": "white", "rating_a": 765, "rating_b": 1877, "gap": 1112, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.15, "correct": 1, "brier": 0.0225, "old_estimate_a": 1150, "old_estimate_b": 2200, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Target B (White) demonstrates superior tactical awareness and positional stability. In Game B, White calculates a favorable exchange sequence starting with 12. Bb5+ and finishes the game with a clean back-rank mate (28. Qg8+). Target A (Black), conversely, plays an extremely loose and weakening opening with 7...f6 and 9...g5, creating numerous holes in their position. Furthermore, Target A blunders their queen in a single move with 22...Qd8, allowing 23. Bxd8. While Target A shows some tactical aggression (16...Bg4), their lack of basic safety and the catastrophic blunder at the end suggest a lower established rating compared to Target B's coordinated play.", "confidence_valid": true }, { "model": "google/gemini-3.6-flash", "pair": "pair_18", "game_a": "game_24", "color_a": "black", "game_b": "game_12", "color_b": "white", "rating_a": 765, "rating_b": 1877, "gap": 1112, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.08, "correct": 1, "brier": 0.0064, "old_estimate_a": 1050, "old_estimate_b": 1550, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A's target (Black) played an excessively dubious opening, severely weakening the kingside with 7...f6, 8...h5, and 9...g5 while leaving the king uncastled. Black capped off a chaotic performance by hanging their queen to a simple bishop attack with 22...Qd8 23.Bxd8. Conversely, Game B's target (White) displayed strong positional awareness and sharp tactical execution. White utilized neat tactical finesses such as 12.Bb5+ c6 13.Qxd4 to regain material, launched a decisive queenside expansion with 17.b4 and 21.a4, and spotted the powerful tactical strike 25.Be5, converting smoothly into a back-rank mate with 28.Qg8+.", "confidence_valid": true }, { "model": "openai/gpt-5.6-sol", "pair": "pair_18", "game_a": "game_24", "color_a": "black", "game_b": "game_12", "color_b": "white", "rating_a": 765, "rating_b": 1877, "gap": 1112, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1050, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black showed repeated strategic and tactical weakness: 7...f6, 8...h5, and 9...g5 badly loosened the position, while 20...Qc6 allowed 21.Nxa8 to win the rook. Most decisively, 22...Qd8 simply placed the queen where White’s bishop could take it with 23.Bxd8. Game B White played much more coherently and tactically: 12.Bb5+ was a strong zwischenzug enabling 13.Qxd4, the queenside expansion with 17.b4 was purposeful, and 25.Be5 followed by 28.Qg8+ demonstrated awareness of the mating attack despite material complications.", "confidence_valid": true }, { "model": "openai/gpt-6-sol", "pair": "pair_18", "game_a": "game_24", "color_a": "black", "game_b": "game_12", "color_b": "white", "rating_a": 765, "rating_b": 1877, "gap": 1112, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.02, "correct": 1, "brier": 0.0004, "old_estimate_a": 1250, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s White played a coherent opening and found tactical resources: 12.Bb5+ followed by 13.Qxd4 recovered the bishop, while 22.Bf4+ and 27.Bxh8 led to a decisive attack. Game A’s Black pushed several kingside pawns without developing or securing the king, then allowed 20.Nb6 and 21.Nxa8 to lose a rook. Black also missed 22.Bd5, which attacked the queen and led to 23.Bxd8. Despite some imperfect play by B’s White, B shows substantially stronger tactical awareness.", "confidence_valid": true }, { "model": "openai/gpt-6.1-sol", "pair": "pair_18", "game_a": "game_24", "color_a": "black", "game_b": "game_12", "color_b": "white", "rating_a": 765, "rating_b": 1877, "gap": 1112, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.03, "correct": 1, "brier": 0.0009, "old_estimate_a": 1150, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black neglects development and king safety with 7...f6, 8...h5 and 9...g5. Although 13...Nd4 shows some positional awareness, Black allows the rook fork 20.Nb6 and then simply hangs the queen with 22...Qd8 23.Bxd8. Game B White demonstrates substantially better tactical coordination: 12.Bb5+ followed by 13.Qxd4 recovers material, and 22.Bf4+ gains time before 23.Bxc4. Most impressively, 27.Bxh8 allows 27...Nxe3 because 28.Qg8+ forces mate after any interposition. B's coherent development and concrete calculation strongly outweigh A's play.", "confidence_valid": true }, { "model": "openai/gpt-5.6-luna", "pair": "pair_18", "game_a": "game_24", "color_a": "black", "game_b": "game_12", "color_b": "white", "rating_a": 765, "rating_b": 1877, "gap": 1112, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.08, "correct": 1, "brier": 0.0064, "old_estimate_a": 1250, "old_estimate_b": 1450, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s White showed substantially better tactical awareness and coordination. White correctly exploited the sequence 10...Nxd4 11 Nxd4 Bxd4 13 Qxd4, emerging with material advantage, then developed a coherent attack with 17 b4, 20 bxc5, and 22 Bf4+. The decisive continuation 23 Bxc4, 24 Rfe1, 25 Be5, and 27 Bxh8 demonstrated effective calculation. Game A’s Black played an unsound-looking kingside expansion with 7...f6, 8...h5, and 9...g5, then missed the simple tactical threat after 20 Nb6: 21 Nxa8 won the rook, and 22...Qd8 allowed 23 Bxd8 to take the queen. Thus B is much more likely the stronger established player.", "confidence_valid": true }, { "model": "openai/gpt-6-astra", "pair": "pair_18", "game_a": "game_24", "color_a": "black", "game_b": "game_12", "color_b": "white", "rating_a": 765, "rating_b": 1877, "gap": 1112, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.98, "correct": 1, "brier": null, "old_estimate_a": 1100, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A Black spends many opening moves on pawns (7...f6, 8...h5, 9...g5), leaving the king exposed and development neglected. Black then allows 20.Nb6, forking queen and rook, and outright hangs the queen with 22...Qd8 23.Bxd8. Game B White develops coherently and finds 12.Bb5+ followed by 13.Qxd4, exploiting Black’s capture sequence to win material. White also coordinates an attack against the castled king with 17.b4 and 22.Bf4+, finishing with the decisive 28.Qg8+. Despite some loose play, B shows substantially stronger tactical awareness and positional organization.", "confidence_valid": false }, { "model": "z-ai/glm-5.3-flash", "pair": "pair_18", "game_a": "game_24", "color_a": "black", "game_b": "game_12", "color_b": "white", "rating_a": 765, "rating_b": 1877, "gap": 1112, "gap_band": ">=500", "low_band": "one<1000", "gold": "B", "choice": "B", "probability_a": 0.12, "correct": 1, "brier": 0.0144, "old_estimate_a": 1050, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B's White plays a clean strategic-tactical game: punishing ...Nd5? (17...Nb6 retreat, 26.Re3-xe3 fork trick winning material), building pressure with a4-b4, and finishing with 28.Qg8+ mate. Black's errors were punished efficiently. Game A's Black played weakly throughout: the bizarre 4...b5/6...b4, 9...g5 and 10...h4 pawn storms against his own king, allowing 17.Nxf6+ and losing material (18...gxh4, then the queen-trap 22...Qd8 losing the queen to 23.Bxd8). Black A's play was unsound beginner-level aggression; White B showed solid technique. Player B is clearly the stronger.", "confidence_valid": true } ], "a": { "game": "game_24", "color": "black", "rating": 765, "url": "https://lichess.org/S3xZRfct", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"1-0\"]\n[TimeControl \"600+0\"]\n\n1. e4 e5 2. Nf3 d6 3. Nc3 c6 4. Bc4 b5 5. Bb3 a5 6. a4 b4 7. Ne2 f6 8. Ng3 h5\n9. d3 g5 10. Nd2 h4 11. Nh5 c5 12. Nc4 Nc6 13. O-O Nd4 14. Ba2 Be6 15. g3 Qd7\n16. gxh4 Bg4 17. Nxf6+ Nxf6 18. Qe1 gxh4 19. Bg5 Nh7 20. Nb6 Qc6 21. Nxa8 Qxa8\n22. Bd5 Qd8 23. Bxd8 1-0" }, "b": { "game": "game_12", "color": "white", "rating": 1877, "url": "https://lichess.org/BdIeZ73n", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"1-0\"]\n[TimeControl \"600+0\"]\n\n1. d4 g6 2. Nf3 Bg7 3. e3 d6 4. Bd3 Nc6 5. a3 Bg4 6. Nbd2 e5 7. h3 Bxf3 8. Nxf3\nexd4 9. exd4 Qe7+ 10. Be3 Nxd4 11. Nxd4 Bxd4 12. Bb5+ c6 13. Qxd4 Nf6 14. Bd3\nO-O-O 15. O-O c5 16. Qa4 Kb8 17. b4 Nd5 18. Bd2 Nb6 19. Qb3 f5 20. bxc5 dxc5\n21. a4 c4 22. Bf4+ Ka8 23. Bxc4 Rd4 24. Rfe1 Qc5 25. Be5 Rd2 26. Re3 Nxc4 27.\nBxh8 Nxe3 28. Qg8+ 1-0" }, "wrong": 0, "gap": 1112 }, { "id": "pair_15", "predictions": [ { "model": "anthropic/claude-opus-5.5", "pair": "pair_15", "game_a": "game_05", "color_a": "white", "game_b": "game_20", "color_b": "white", "rating_a": 1120, "rating_b": 1932, "gap": 812, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.22, "correct": 1, "brier": 0.0484, "old_estimate_a": 1150, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White played a wandering opening (2.Bc4, 5.Bd3, 6.Be2, 11.Be4, 12.Bd3) and then blundered badly: 16.Be4?? allowed Nxg2+, and 22.Bc3/23.Nd4 dropped the h1 rook. This is a collapse to simple tactics. Game B White ran a coherent Pirc/150 attack: Be3, Qd2, f3, O-O-O, g4, h4, h5 and hxg6, then Rxh6. The 26.Rxh7 sacrifice and the precise finish ending in 38.Nh6# show real attacking skill. Black in B did misplay with 7...Bh8 and 23...Bg7, but B's play is still clearly more competent.", "confidence_valid": true }, { "model": "anthropic/claude-sonnet-5.5", "pair": "pair_15", "game_a": "game_05", "color_a": "white", "game_b": "game_20", "color_b": "white", "rating_a": 1120, "rating_b": 1932, "gap": 812, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.3, "correct": 1, "brier": 0.09, "old_estimate_a": 1450, "old_estimate_b": 1700, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White played a Bc4/c3 setup against the Sicilian, then went wrong with 10.e5 and 11.Be4, which allowed 11...f5. After 15.c4 Nf4 and 16.Be4?, Black's 16...Nxg2+ won material with a strong attack, and White never recovered. Overall that was a loss of control in the opening and middlegame, with White missing the tactics. Game B White played a coherent Pirc attack with Be3, Qd2, Bh6 and a pawn storm (g4, h4-h5). White found the 26.Rxh7 combination, then 29.Qxh8+ and the 33.Nf5 and 38.Nh6# finish, which converted accurately. Black's defence was weak, but White's play was more purposeful and tactically sound, so B looks stronger.", "confidence_valid": true }, { "model": "google/gemini-3.8-flash", "pair": "pair_15", "game_a": "game_05", "color_a": "white", "game_b": "game_20", "color_b": "white", "rating_a": 1120, "rating_b": 1932, "gap": 812, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.03, "correct": 1, "brier": 0.0009, "old_estimate_a": 1200, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White shows poor opening understanding, moving the light-squared bishop five times in the first twelve moves (Bc4-d3-e2-d3-e4-d3) before blundering tactically with 16. Be4?? (allowing 16...Nxg2+ and dropping decisive material). In contrast, Game B White executes a textbook Yugoslav-style kingside storm against the Pirc/Modern. Game B White methodically opens the h-file with 11. h4, 12. h5, and 22. Rdh1, followed by a crisp tactical breakthrough with 26. Rxh7! and 28. fxg6+, efficiently winning Black's queen and converting the ending directly into a forced checkmate.", "confidence_valid": true }, { "model": "google/gemini-3.1-pro-preview", "pair": "pair_15", "game_a": "game_05", "color_a": "white", "game_b": "game_20", "color_b": "white", "rating_a": 1120, "rating_b": 1932, "gap": 812, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1250, "old_estimate_b": 2150, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Target B (White in Game B) demonstrated a much higher level of play than Target A. White in Game B executed a thematic and well-calculated kingside attack against the Pirc/Modern defense, complete with long castling, a pawn storm (h4-h5), and brilliant sacrifices like 26. Rxh7 and the queen sacrifice 29. Qxh8+, leading to a beautiful checkmate. In contrast, White in Game A struggled with basic opening principles (2. Bc4, 5. Bd3) and blundered a piece to a simple tactic (16. Be4?? Nxg2+), getting completely crushed. The difference in tactical and positional understanding is vast.", "confidence_valid": true }, { "model": "google/gemini-3.5-flash", "pair": "pair_15", "game_a": "game_05", "color_a": "white", "game_b": "game_20", "color_b": "white", "rating_a": 1120, "rating_b": 1932, "gap": 812, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.01, "correct": 1, "brier": 0.0001, "old_estimate_a": 1150, "old_estimate_b": 1950, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White shows a low level of play, making severe tactical blunders such as 16. Be4? (allowing 16... Nxg2+ and losing central control), 23. Nd4? (hanging the h1-rook), and 27. Qxd3?? (hanging the queen to a simple discovery). In contrast, Game B White exhibits advanced positional and attacking understanding in a Pirc/Modern structure, orchestrating a high-level kingside assault (7. Bh6, 11. h4, 22. Qh2). Game B White executes a beautiful, calculation-heavy combination starting with 25. Rxh7! and the queen sacrifice 28. Qxh8+! to transition into a winning endgame, finishing with an elegant mating net.", "confidence_valid": true }, { "model": "google/gemini-3.7-flash", "pair": "pair_15", "game_a": "game_05", "color_a": "white", "game_b": "game_20", "color_b": "white", "rating_a": 1120, "rating_b": 1932, "gap": 812, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.04, "correct": 1, "brier": 0.0016, "old_estimate_a": 1400, "old_estimate_b": 1850, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Player B demonstrates a high level of strategic and tactical competence against the Modern Defense. White in Game B executes a cohesive kingside attack (g4, h4-h5, Qh2, f4-f5), culminating in a precise queen sacrifice and checkmate (26. Rxh7! through 38. Nh6#). In contrast, Player A shows poor opening handling and low tactical awareness, repeatedly shuffling the light-squared bishop (Bc4-d3-e2-d3-e4), blundering a piece with 16. Be4? fxe4, and collapsing rapidly to a decisive mating attack on f3.", "confidence_valid": true }, { "model": "qwen/qwen3.8-max-prime", "pair": "pair_15", "game_a": "game_05", "color_a": "white", "game_b": "game_20", "color_b": "white", "rating_a": 1120, "rating_b": 1932, "gap": 812, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.08, "correct": 1, "brier": 0.0064, "old_estimate_a": 1350, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "In Game A, White plays passively and blunders: 16.Kf1 allows 16...Nxg2+ losing the h1-rook and king safety, and White's pieces are quickly overwhelmed (23...Bxh1). In Game B, White executes a coherent, aggressive plan (7.Bh6, 11.h5, 16.g5), keeps pressure, and delivers a precise mating finish (31.Rxd8, 37.g7+, 38.Nh6#). Game A's target is outplayed from the opening and loses badly, while Game B's target dominates with strong attacking technique. The evidence strongly favors B as the higher-rated player.", "confidence_valid": true }, { "model": "anthropic/claude-fable-5.1", "pair": "pair_15", "game_a": "game_05", "color_a": "white", "game_b": "game_20", "color_b": "white", "rating_a": 1120, "rating_b": 1932, "gap": 812, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.12, "correct": 1, "brier": 0.0144, "old_estimate_a": 1000, "old_estimate_b": 1600, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White played an incoherent opening (2.Bc4, 5.Bd3, 6.Be2, 7.d3 losing tempi), then 15.c4?? and 16.Be4?? allowed 16...Nxg2+ and a collapsing position, finishing with 27.Qxd3 walking into mate by 29...Qxf3#. Game B White executed a thematic Pirc/150 Attack plan: 7.Bh6, 9.g4, 11.h4, 12.h5, 13.hxg6, then 17.f4 Nxh6 18.Rxh6, doubled rooks 22.Rdh1, and converted cleanly with 26.Rxh7 Nxh7 27.Qxh7 and a precise mating net ending in 38.Nh6#. Game B White showed clear opening knowledge and tactical calculation; Game A White showed neither.", "confidence_valid": true }, { "model": "google/gemini-3-flash-preview", "pair": "pair_15", "game_a": "game_05", "color_a": "white", "game_b": "game_20", "color_b": "white", "rating_a": 1120, "rating_b": 1932, "gap": 812, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1250, "old_estimate_b": 2050, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Player A demonstrates poor opening fundamentals, wasting tempi with 2. Bc4 and 5. Bd3, and then commits a massive tactical blunder with 16. Be4?, losing a piece to 16... Nxg2+ and 17... fxe4. Player A's defense is non-existent, leading to a quick mate. In contrast, Player B plays a sophisticated, thematic kingside attack against the Pirc Defense. White correctly executes the h-pawn storm (11. h4), coordinates pieces well (14. Nce2), and finds a decisive exchange sacrifice (26. Rxh7!). Player B's ability to simplify into a won endgame (29. Qxh8+) and finish with a clean mate (38. Nh6#) indicates a significantly higher established playing strength.", "confidence_valid": true }, { "model": "google/gemini-3.6-flash", "pair": "pair_15", "game_a": "game_05", "color_a": "white", "game_b": "game_20", "color_b": "white", "rating_a": 1120, "rating_b": 1932, "gap": 812, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.05, "correct": 1, "brier": 0.0025000000000000005, "old_estimate_a": 1400, "old_estimate_b": 1900, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White played very weakly, opening with the passive 2. Bc4, misplacing pieces with 5. Bd3, and collapsing rapidly after tactical blunders like 13. b3 (hanging the c3 pawn) and 16. Be4?? (allowing 16...Nxg2+ and losing a piece). White in Game A offered no resistance and was mated on move 29. In contrast, Game B White demonstrated clear strategic understanding of the Yugoslav-style attack against the Modern Defense with 8. O-O-O, 11. h4, and 15. g5. Game B White conducted a powerful kingside storm, brilliantly sacrificed with 25. Rxh7! to dismantle Black's defense, and executed a clean tactical win ending in checkmate with 38. Nh6#.", "confidence_valid": true }, { "model": "openai/gpt-5.6-sol", "pair": "pair_15", "game_a": "game_05", "color_a": "white", "game_b": "game_20", "color_b": "white", "rating_a": 1120, "rating_b": 1932, "gap": 812, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.06, "correct": 1, "brier": 0.0036, "old_estimate_a": 1450, "old_estimate_b": 1900, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B White showed a coherent attacking plan: opposite-side castling followed by g4, h4-h5, Rdh1 and Qh2, then calculated 26.Rxh7! Nxh7 27.Qxh7, converting into a decisive material attack and finishing accurately with 38.Nh6#. Game A White spent many tempi retreating the same bishop and overlooked elementary tactics: 16.Be4? allowed 16...Nxg2+ 17.Kf1 fxe4, losing the bishop, while 22.Bc3? Bxe4 and 23.Nd4? Bxh1 led to collapse. B’s target displayed substantially stronger planning and tactical awareness.", "confidence_valid": true }, { "model": "openai/gpt-6-sol", "pair": "pair_15", "game_a": "game_05", "color_a": "white", "game_b": "game_20", "color_b": "white", "rating_a": 1120, "rating_b": 1932, "gap": 812, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.01, "correct": 1, "brier": 0.0001, "old_estimate_a": 1300, "old_estimate_b": 1900, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s White built a coherent kingside attack with 8.O-O-O, 11.h4 and 22.Rdh1, then calculated 26.Rxh7 through 31.Rxd8, winning Black’s queen. Game A’s White repeatedly retreated the bishop and lost it after 16.Be4 Nxg2+ 17.Kf1 fxe4; later, 27.Qxd3 allowed 27...Be4+ and the queen was lost. Despite the limited evidence of one game each, B’s play shows substantially stronger planning and tactical awareness.", "confidence_valid": true }, { "model": "openai/gpt-6.1-sol", "pair": "pair_15", "game_a": "game_05", "color_a": "white", "game_b": "game_20", "color_b": "white", "rating_a": 1120, "rating_b": 1932, "gap": 812, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.95, "correct": 1, "brier": null, "old_estimate_a": 1200, "old_estimate_b": 2000, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B White showed a coherent attacking plan: queenside castling, h-pawn pressure, 22.Rdh1 and 23.Qh2, followed by the forcing 26.Rxh7–31.Rxd8 sequence. White then coordinated both knights and the passed pawn with 33.Nf5, 36.Ng5 and 38.Nh6#. Game A White spent several opening tempi relocating the bishop (2.Bc4, 5.Bd3, 6.Be2), then 16.Be4 allowed ...Nxg2+ and ...fxe4, losing a bishop for pawns. Later, 22.Bc3 and 23.Nd4 permitted ...Bxe4 and ...Bxh1. Although A faced stronger resistance, B’s sustained coordination and tactical execution provide substantially stronger evidence of established playing strength.", "confidence_valid": false }, { "model": "openai/gpt-5.6-luna", "pair": "pair_15", "game_a": "game_05", "color_a": "white", "game_b": "game_20", "color_b": "white", "rating_a": 1120, "rating_b": 1932, "gap": 812, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.9, "correct": 1, "brier": null, "old_estimate_a": 1300, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s White showed a coherent attacking plan and strong tactical execution: the kingside expansion with 9 g4, 11 h4, and 12 h5 was followed by 18 Rxh6, 22 Rdh1, and the combination 26 Rxh7 Nxh7 27 Qxh7, winning decisive material. The finish 29 Qxh8+ Bxh8 30 Rxh8+ and 38 Nh6# was accurate. Game A’s White played an unfocused opening and missed major tactical threats: after 16...Nxg2+ and 17...fxe4, the attack collapsed, and 23 Nd4?? allowed ...Bxh1, losing a rook. White then permitted the forced mating sequence ending 29...Qxf3#.", "confidence_valid": false }, { "model": "openai/gpt-6-astra", "pair": "pair_15", "game_a": "game_05", "color_a": "white", "game_b": "game_20", "color_b": "white", "rating_a": 1120, "rating_b": 1932, "gap": 812, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.98, "correct": 1, "brier": null, "old_estimate_a": 1300, "old_estimate_b": 2050, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B White shows coherent opening preparation and attacking coordination: queenside castling, the h-pawn advance, and 22.Rdh1 followed by 23.Qh2. The forcing sequence 26.Rxh7 through 31.Rxd8 converts the attack into material gain, while 35.g6 and 36.Ng5 create a precise mating finish. Game A White repeatedly spends tempi moving the same bishop, then allows 15...Nf4 and 16...Nxg2+, losing a bishop after ...fxe4. Later, 23.Nd4 permits ...Bxh1, and 27.Qxd3 allows the discovered check ...Be4+. Although A faced more effective opposition, B demonstrates substantially stronger planning and tactical awareness.", "confidence_valid": false }, { "model": "z-ai/glm-5.3-flash", "pair": "pair_15", "game_a": "game_05", "color_a": "white", "game_b": "game_20", "color_b": "white", "rating_a": 1120, "rating_b": 1932, "gap": 812, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.12, "correct": 1, "brier": 0.0144, "old_estimate_a": 1300, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A's White plays weakly: 15.Be4 allows 15...Nxg2+ winning a piece (16.Kf1 fxe4 traps the bishop and 18.Nxe4 Nf4 keeps material), then 23.Nd4?? loses the exchange to 23...Bxh1. Black methodically converts, and White resigns mated — White looks ~1400-1600 level. Game B's White handles a sharp Saemisch KID: solid setup, timely 17.f4 and 13.hxg6, accurate attack culminating in 26.Rxh7! 27.Qxh7 Rh8 28.fxg6+ and clean queen sacrifice 29.Qxh8+, finishing with a model mate 38.Nh6#. B is clearly the stronger player.", "confidence_valid": true } ], "a": { "game": "game_05", "color": "white", "rating": 1120, "url": "https://lichess.org/mCkfc84Y", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"0-1\"]\n[TimeControl \"600+0\"]\n\n1. e4 c5 2. Bc4 e6 3. Nf3 a6 4. c3 b5 5. Bd3 c4 6. Be2 Bb7 7. d3 cxd3 8. Bxd3\nNf6 9. Nbd2 Be7 10. e5 Nd5 11. Be4 f5 12. Bd3 O-O 13. b3 Nc6 14. Bb2 Qc7 15. c4\nNf4 16. Be4 Nxg2+ 17. Kf1 fxe4 18. Nxe4 Nf4 19. Nfg5 h6 20. Nf3 Nb4 21. a3 Nbd3\n22. Bc3 Bxe4 23. Nd4 Bxh1 24. Kg1 Qb7 25. f3 Nh3+ 26. Kf1 Bxf3 27. Qxd3 Be4+\n28. Nf3 Bxd3+ 29. Kg2 Qxf3# 0-1" }, "b": { "game": "game_20", "color": "white", "rating": 1932, "url": "https://lichess.org/jLDNTu60", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"1-0\"]\n[TimeControl \"600+0\"]\n\n1. e4 g6 2. Nc3 Bg7 3. d4 d6 4. Be3 Nf6 5. Qd2 O-O 6. f3 Re8 7. Bh6 Bh8 8.\nO-O-O Nbd7 9. g4 c5 10. d5 a6 11. h4 b5 12. h5 b4 13. hxg6 fxg6 14. Nce2 Ne5\n15. Ng3 Nf7 16. g5 Nd7 17. f4 Nxh6 18. Rxh6 Nf8 19. Bc4 a5 20. Nf3 Ba6 21. Bxa6\nRxa6 22. Rdh1 a4 23. Qh2 Bg7 24. f5 b3 25. a3 Kf7 26. Rxh7 Nxh7 27. Qxh7 Rh8\n28. fxg6+ Kf8 29. Qxh8+ Bxh8 30. Rxh8+ Kg7 31. Rxd8 Kxg6 32. Rd7 Kf7 33. Nf5\nbxc2 34. Rxe7+ Kf8 35. g6 Ra8 36. Ng5 c4 37. g7+ Kg8 38. Nh6# 1-0" }, "wrong": 0, "gap": 812 }, { "id": "pair_06", "predictions": [ { "model": "anthropic/claude-opus-5.5", "pair": "pair_06", "game_a": "game_17", "color_a": "black", "game_b": "game_25", "color_b": "black", "rating_a": 1639, "rating_b": 838, "gap": 801, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.72, "correct": 1, "brier": 0.07840000000000001, "old_estimate_a": 1550, "old_estimate_b": 950, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black plays a coherent Italian with sensible developing moves (…Bc5, …O-O, …d5, …Bg4). The resignation at move 24 likely followed a pawn loss or tactical issue after 24.Rd5, but the play up to that point is solid and positional. Game B Black plays a chaotic opening: 2…Bd7, 3…c6, 4…b5, 5…Na6. Black does win the a1 rook, but then blunders. After 25.Bxc3, Black is down material, and Black later allows mate with 39.Rxd8#. Game B's opponent was also weak (3.Qf3, 4.g4). On balance, A's target shows more competent play.", "confidence_valid": true }, { "model": "anthropic/claude-sonnet-5.5", "pair": "pair_06", "game_a": "game_17", "color_a": "black", "game_b": "game_25", "color_b": "black", "rating_a": 1639, "rating_b": 838, "gap": 801, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.66, "correct": 1, "brier": 0.11559999999999998, "old_estimate_a": 1600, "old_estimate_b": 1500, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black played a standard Italian setup with sensible development (4...O-O, 6...d5, 8...Bg4, 9...Bb6). Black traded pieces and kept a fairly solid position. The 12...h6 and 13...Nf4 moves were reasonable, though 18...g5 weakened the king and Black eventually lost slowly. Game B Black played a bizarre opening (2...Bd7, 3...c6, 4...b5, 5...Na6) and left the king exposed. Black did win material with tactics (13...Nc2+, 14...Nxa1, 16...Bxh3), but later blundered with 20...Rxd4+ and 21...Rxc4, then dropped the queen with 23...Qc8 and 24...Rxc3+. Overall Game B Black looks weaker and more erratic, so A is more likely stronger.", "confidence_valid": true }, { "model": "google/gemini-3.8-flash", "pair": "pair_06", "game_a": "game_17", "color_a": "black", "game_b": "game_25", "color_b": "black", "rating_a": 1639, "rating_b": 838, "gap": 801, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.88, "correct": 1, "brier": 0.0144, "old_estimate_a": 1650, "old_estimate_b": 1300, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Player A demonstrates sound opening principles and structured positional understanding in a Giuoco Pianissimo, coordinating pieces logically (5...Nc6, 6...d5, 8...Bg4) and managing trades cleanly until 24...c6. In contrast, Player B plays an extremely low-level game characterized by bizarre early pawn thrusts and awkward setups (2...Bd7, 4...b5, 7...Rd8). Despite winning a rook with 13...Nc2+, Black in Game B displayed severe tactical blindness, needlessly sacrificing material with 24...Rxc3+ and passively walking into back-rank and pin disasters (30...Be7?, 32...Nxf5?) culminating in 39. Rxd8#.", "confidence_valid": true }, { "model": "google/gemini-3.1-pro-preview", "pair": "pair_06", "game_a": "game_17", "color_a": "black", "game_b": "game_25", "color_b": "black", "rating_a": 1639, "rating_b": 838, "gap": 801, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.85, "correct": 1, "brier": 0.022500000000000006, "old_estimate_a": 1750, "old_estimate_b": 1150, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Target A (Black in Game A) played a standard Italian Game, demonstrating solid opening principles, logical piece development (...Bg4, ...Nf4), and a good understanding of pawn structure. The game ended abruptly, likely due to a timeout or disconnect, but Black's play was positionally sound. In contrast, Target B (Black in Game B) played a chaotic, beginner-level opening (...Bd7, ...b5, ...Na6, ...Rd8) against an opponent who recklessly pushed kingside pawns early on. Although Black B found some basic tactical tricks like 20...Rxd4+, their overall strategic understanding was poor, culminating in severe blunders and getting checkmated. Target A's solid positional play strongly suggests a higher rating.", "confidence_valid": true }, { "model": "google/gemini-3.5-flash", "pair": "pair_06", "game_a": "game_17", "color_a": "black", "game_b": "game_25", "color_b": "black", "rating_a": 1639, "rating_b": 838, "gap": 801, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.85, "correct": 1, "brier": 0.022500000000000006, "old_estimate_a": 1650, "old_estimate_b": 1050, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Black in Game A plays standard, principled opening moves, castling early and maintaining a solid positional structure before a final blunder with 24...c6, which hangs the e5-knight. In contrast, Black in Game B plays extremely erratically from the start, choosing bizarre moves like 2...Bd7 and 7...Rd8. Despite capturing a rook after 13...Nc2+, Black displays poor board vision, blundering back material with 24...Rxc3+ and 32...Nxf5, leading to a quick checkmate. Black in Game A shows much higher baseline chess competence.", "confidence_valid": true }, { "model": "google/gemini-3.7-flash", "pair": "pair_06", "game_a": "game_17", "color_a": "black", "game_b": "game_25", "color_b": "black", "rating_a": 1639, "rating_b": 838, "gap": 801, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.88, "correct": 1, "brier": 0.0144, "old_estimate_a": 1700, "old_estimate_b": 1200, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black demonstrates solid fundamental chess understanding, navigating an open game with sound opening principles (6...d5, 8...Bg4), active piece placement (13...Nf4), and classical central play (21...Rd8, 22...Ne5). Although Black resigned or flagged after 24...c6, their position remained completely playable and logical throughout. In contrast, Game B Black plays an erratic, passive opening (2...Bd7, 7...Rd8) and, despite winning a trapped rook on a1, displays terrible king safety and coordination. Game B Black completely mishandles White's uncoordinated attack, missing major tactical threats (22...Qc5 allowing 23.Qa8+) and collapsing rapidly into checkmate (39.Rxd8#).", "confidence_valid": true }, { "model": "qwen/qwen3.8-max-prime", "pair": "pair_06", "game_a": "game_17", "color_a": "black", "game_b": "game_25", "color_b": "black", "rating_a": 1639, "rating_b": 838, "gap": 801, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.87, "correct": 1, "brier": 0.016900000000000002, "old_estimate_a": 1050, "old_estimate_b": 1200, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Game A Black played solid, standard rapid lines: …Bc5, …d5 counter-strike, …Nxd5, …Rfe8, and only resigned after 24.Rd5. Game B Black made bizarre, non-developing moves: 2…Bd7, 4…b5, 6…Qc7, 7…Rd8, then won material but collapsed tactically after 20…Rxd4+, allowing the forced mate 39.Rxd8#. B's opening was incoherent and defense hopeless; A showed ordinary club competence.", "confidence_valid": true }, { "model": "anthropic/claude-fable-5.1", "pair": "pair_06", "game_a": "game_17", "color_a": "black", "game_b": "game_25", "color_b": "black", "rating_a": 1639, "rating_b": 838, "gap": 801, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.8, "correct": 1, "brier": 0.03999999999999998, "old_estimate_a": 1450, "old_estimate_b": 950, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black played a coherent Italian: ...d5 break, ...Bg4, ...Nf4 exchange, ...Rxe4 sac for play; the game ended early (likely resignation/flag) without gross blunders shown. Game B Black played bizarre, incoherent moves from the start (2...Bd7, 4...b5, 5...Na6, 7...Rd8), then after winning material with ...Nc2+ and ...Nxa1 repeatedly misplayed: 19...Qg5 wasted tempi, 24...Rxc3+ gave back a rook, 25...f6/28...e5 weakened the king, and allowed a mating net. Game B White was also very weak (2.e3 Qf3 g4 h3), indicating a low-rated pairing. Black in Game A shows clearly more structured understanding.", "confidence_valid": true }, { "model": "google/gemini-3-flash-preview", "pair": "pair_06", "game_a": "game_17", "color_a": "black", "game_b": "game_25", "color_b": "black", "rating_a": 1639, "rating_b": 838, "gap": 801, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.9, "correct": 1, "brier": 0.009999999999999995, "old_estimate_a": 1800, "old_estimate_b": 1050, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black demonstrates sound opening principles (3... Bc5, 6... d5) and maintains a balanced, coordinated position throughout the middlegame. In contrast, Game B Black plays an erratic opening (2... Bd7, 5... Na6) and shows poor tactical awareness. Despite winning a rook early, Game B Black collapses with unnecessary exchanges (24... Rxc3+) and fails to defend a simple mating threat, ultimately allowing 39. Rxd8#. Player A's play shows a significantly higher level of foundational chess knowledge and calculation compared to Player B's chaotic performance, which is characterized by uncoordinated development and severe blunders under pressure.", "confidence_valid": true }, { "model": "google/gemini-3.6-flash", "pair": "pair_06", "game_a": "game_17", "color_a": "black", "game_b": "game_25", "color_b": "black", "rating_a": 1639, "rating_b": 838, "gap": 801, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.85, "correct": 1, "brier": 0.022500000000000006, "old_estimate_a": 1650, "old_estimate_b": 1100, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black demonstrates solid opening knowledge and positional principles in an Italian Game (6...d5, active piece placement). Black maintained a balanced position until a single tactical oversight on move 24 (24...c6, allowing 25.Rxe5). In contrast, Game B Black plays unprovoked passive/erratic moves early on (2...Bd7, 7...Rd8). Despite winning a full rook via 13...Nc2+, Game B Black showed poor board vision, voluntarily sacrificing material back (24...Rxc3+) and allowing White's rook and queen to infiltrate for a decisive mate on d8. Game A Black shows far higher fundamental chess competence.", "confidence_valid": true }, { "model": "openai/gpt-5.6-sol", "pair": "pair_06", "game_a": "game_17", "color_a": "black", "game_b": "game_25", "color_b": "black", "rating_a": 1639, "rating_b": 838, "gap": 801, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.84, "correct": 1, "brier": 0.02560000000000001, "old_estimate_a": 1850, "old_estimate_b": 1250, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black played a coherent, principled game: 6...d5 challenged the center, 13...Nf4 created activity, and 16...Bxe4 followed by 19...Rfe8 and 21...Rd8 showed purposeful coordination. The unexplained ending after 24...c6 does not reflect an obvious collapse. Game B Black found the strong fork 13...Nc2+ and tactical 20...Rxd4+, but the opening setup (...Bd7, ...b5, ...Na6, ...Rd8) was awkward, and 24...Rxc3+ gave up a rook without sufficient return. Most decisively, 32...Nxf5 allowed 33.Ra8 and 34.Rxc8, losing the queen and leading to mate. A appears more consistently skilled.", "confidence_valid": true }, { "model": "openai/gpt-6-sol", "pair": "pair_06", "game_a": "game_17", "color_a": "black", "game_b": "game_25", "color_b": "black", "rating_a": 1639, "rating_b": 838, "gap": 801, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.97, "correct": 1, "brier": 0.0009000000000000016, "old_estimate_a": 1600, "old_estimate_b": 1250, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A’s Black played a coherent opening with 6...d5, then coordinated exchanges and central pressure with 20...Rxe4, 21...Rd8, and 24...c6. Game B’s Black spotted tactics against White’s exposed king, notably 13...Nc2+ and 20...Rxd4+, but developed awkwardly and ultimately left the queen on c8 to be taken after 33...Bd8 34.Rxc8. A’s play looks substantially more reliable, regardless of the recorded results.", "confidence_valid": true }, { "model": "openai/gpt-6.1-sol", "pair": "pair_06", "game_a": "game_17", "color_a": "black", "game_b": "game_25", "color_b": "black", "rating_a": 1639, "rating_b": 838, "gap": 801, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.96, "correct": 1, "brier": 0.001600000000000003, "old_estimate_a": 1750, "old_estimate_b": 1050, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black showed coherent development and central play with 6...d5, 8...Bg4, and 13...Nf4, then coordinated the rooks through 19...Rfe8 and 21...Rd8. The final position does not indicate an obvious decisive blunder. Game B Black found tactics such as 13...Nc2+ and 20...Rxd4+, but against highly unconventional opposition. Black’s early ...Bd7, ...Na6, and ...Rd8 neglected kingside development, while 32...Nxf5 and 33...Bd8 allowed 34.Rxc8, losing the queen. A’s play is substantially more consistent with established strength.", "confidence_valid": true }, { "model": "openai/gpt-5.6-luna", "pair": "pair_06", "game_a": "game_17", "color_a": "black", "game_b": "game_25", "color_b": "black", "rating_a": 1639, "rating_b": 838, "gap": 801, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.8, "correct": 1, "brier": 0.03999999999999998, "old_estimate_a": 1600, "old_estimate_b": 1400, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A’s Black played a coherent, principled opening: 6...d5, castling, active piece placement with ...Nf4, and pressure on the e-file with ...Rfe8 and ...Rxe4. The later 18...g5 and 24...c6 may be inaccurate, but the game shows substantially more positional discipline. Game B’s Black chose a highly dubious setup with 2...Bd7, 3...c6, and 4...b5, then allowed major tactical complications. Although 13...Nc2+ and 14...Nxa1 won material, Black’s subsequent 20...Rxd4+ and 21...Rxc4 led to tactical exchanges, and the king was eventually mated after 39.Rxd8#. Overall, A’s Black more likely had the higher established rapid strength.", "confidence_valid": true }, { "model": "openai/gpt-6-astra", "pair": "pair_06", "game_a": "game_17", "color_a": "black", "game_b": "game_25", "color_b": "black", "rating_a": 1639, "rating_b": 838, "gap": 801, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.95, "correct": 1, "brier": 0.0025000000000000044, "old_estimate_a": 1800, "old_estimate_b": 1100, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A Black shows coherent development and positional play: 6...d5 challenges the center, 13...Nf4 establishes an active knight, and 19...Rfe8 followed by 21...Rd8 activates the rooks. The final position after 24...c6 does not explain the recorded loss. Game B Black exploits White’s weak play with 13...Nc2+ and the rook-winning fork, but 2...Bd7 and 7...Rd8 reflect awkward development while the kingside remains undeveloped. Later, 32...Nxf5 permits Ra8, and 33...Bd8 allows 34.Rxc8, losing the queen for a rook. A’s consistently purposeful play is substantially stronger evidence of established strength.", "confidence_valid": true }, { "model": "z-ai/glm-5.3-flash", "pair": "pair_06", "game_a": "game_17", "color_a": "black", "game_b": "game_25", "color_b": "black", "rating_a": 1639, "rating_b": 838, "gap": 801, "gap_band": ">=500", "low_band": "one<1000", "gold": "A", "choice": "A", "probability_a": 0.75, "correct": 1, "brier": 0.0625, "old_estimate_a": 1650, "old_estimate_b": 1150, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A's Black plays a coherent Italian: 8...Bg4 active development, 13...Nf4 hitting the bishop, 16...Bxe4/17...Qf6 clean trades, then 22...Ne5 and 23...Rd6 with a solid, equal game; White resigned at move 24 with 24.Rd5? meeting 24...c6. Game B's Black plays erratically: 2...Bd7 and 4...b5/5...Na6 are aimless, 13...Nc2+ 14...Nxa1 trades a knight for a rook-corner trap but abandons queenside, then loses material (20...Rxd4+ 21...Rxc4 losing the rook for bishop) and gets mated. Black B's planless aggressive pawn pushes (g4?!) by White aside, Black A shows far better positional understanding and technique, suggesting the higher rating.", "confidence_valid": true } ], "a": { "game": "game_17", "color": "black", "rating": 1639, "url": "https://lichess.org/KH0akECy", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"1-0\"]\n[TimeControl \"600+0\"]\n\n1. e4 e5 2. Bc4 Nf6 3. Nf3 Bc5 4. d3 O-O 5. O-O Nc6 6. a3 d5 7. exd5 Nxd5 8.\nNbd2 Bg4 9. Ne4 Bb6 10. h3 Bh5 11. g4 Bg6 12. Neg5 h6 13. Ne4 Nf4 14. Bxf4 exf4\n15. Kg2 Qe7 16. Re1 Bxe4 17. Rxe4 Qf6 18. c3 g5 19. Bb3 Rfe8 20. Bc2 Rxe4 21.\ndxe4 Rd8 22. Qe2 Ne5 23. Rd1 Rd6 24. Rd5 c6 1-0" }, "b": { "game": "game_25", "color": "black", "rating": 838, "url": "https://lichess.org/ryvfsxd9", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"1-0\"]\n[TimeControl \"600+0\"]\n\n1. d4 d5 2. e3 Bd7 3. Qf3 c6 4. g4 b5 5. h3 Na6 6. h4 Qc7 7. g5 Rd8 8. h5 c5 9.\nc3 b4 10. b3 c4 11. bxc4 dxc4 12. cxb4 Nxb4 13. Nh3 Nc2+ 14. Kd2 Nxa1 15. Bb2\nQa5+ 16. Nc3 Bxh3 17. Rxh3 Qxg5 18. Bxa1 Qg1 19. Rh1 Qg5 20. Bxc4 Rxd4+ 21. Kc2\nRxc4 22. Rd1 Qc5 23. Qa8+ Qc8 24. Qd5 Rxc3+ 25. Bxc3 f6 26. f4 e6 27. Qb5+ Kf7\n28. e4 e5 29. f5 Nh6 30. Rd7+ Be7 31. Qd5+ Ke8 32. Rxa7 Nxf5 33. Ra8 Bd8 34.\nRxc8 Ke7 35. Qb7+ Kd6 36. Qc6+ Ke7 37. Bb4+ Kf7 38. Qd5+ Ke8 39. Rxd8# 1-0" }, "wrong": 0, "gap": 801 }, { "id": "pair_17", "predictions": [ { "model": "anthropic/claude-opus-5.5", "pair": "pair_17", "game_a": "game_13", "color_a": "white", "game_b": "game_16", "color_b": "white", "rating_a": 1126, "rating_b": 1752, "gap": 626, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.3, "correct": 1, "brier": 0.09, "old_estimate_a": 1250, "old_estimate_b": 1350, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White played oddly throughout: 5.Be3 then 9.Bd2, 10.Qa1 and 11.Nb1 undeveloping, and 16.Rc5?? dropping the exchange to ...b6. Despite later gaining material after Black's errors (28.Ne8 forking/winning the exchange), White ignored back-rank danger and allowed 35...Rc1#. Game B White handled a sloppy opponent sensibly: accepted 6...Bxf2+ (a bad sacrifice), 13.hxg4 kept the extra piece, and the follow-up was solid with central control (10.d4, 17.Rae1). White also traded queens with 25.Qe2 to convert. Black's 27...Nc7?? hangs f6 or leads to resignation-level loss. B's play shows cleaner technique, so B is likely stronger.", "confidence_valid": true }, { "model": "anthropic/claude-sonnet-5.5", "pair": "pair_17", "game_a": "game_13", "color_a": "white", "game_b": "game_16", "color_b": "white", "rating_a": 1126, "rating_b": 1752, "gap": 626, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.3, "correct": 1, "brier": 0.09, "old_estimate_a": 1450, "old_estimate_b": 1550, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White played an odd, passive opening: 3.dxc5 Qa5+ 4.Bd2 Qxc5 5.Be3 Qb5 6.Nc3 Qxb2 lost a pawn and allowed the queen raid. Later 7.Rb1 Qa3 8.Rb3 and 16.Rc5 were dubious rook moves. 20.Ne4 and 23.Nd6 were also shaky, and 28.Ne8 and 29.Nc7 lost material. White then missed the final back-rank mate (35.Nxb5 Rc1#). Game B White took a pawn after 2.exd5, and Black's 6...Bxf2+ was an unsound sacrifice. White handled it cleanly with 7.Kxf2, 10.d4, 11.Bg5 and 13.hxg4, then traded into a won endgame. Black's play was also weak, but White was accurate and reasonable. Game B White looks somewhat stronger, though both are modest in level.", "confidence_valid": true }, { "model": "google/gemini-3.8-flash", "pair": "pair_17", "game_a": "game_13", "color_a": "white", "game_b": "game_16", "color_b": "white", "rating_a": 1126, "rating_b": 1752, "gap": 626, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.15, "correct": 1, "brier": 0.0225, "old_estimate_a": 1200, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Player B exhibits solid fundamentals, comfortably defending against Black's unsound sacrifice 6...Bxf2+ by safely repositioning the king with 8.Rf1 and 9.Kg1. From there, B smoothly coordinates, develops pieces, controls central files, and converts the material advantage cleanly without any tactical oversights. In contrast, Player A plays haphazardly, needlessly giving up pawn structure and falling into a cheap exchange tactic with 17.c3? b6, and later blunders a simple back-rank checkmate in one move with 35.Nxb5?? Rc1#.", "confidence_valid": true }, { "model": "google/gemini-3.1-pro-preview", "pair": "pair_17", "game_a": "game_13", "color_a": "white", "game_b": "game_16", "color_b": "white", "rating_a": 1126, "rating_b": 1752, "gap": 626, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.1, "correct": 1, "brier": 0.010000000000000002, "old_estimate_a": 1050, "old_estimate_b": 1400, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White demonstrates very low-level play, with bizarre maneuvers like 10. Qa1 and 11. Nb1, dropping an exchange with 16. Rc5 Ndb4 17. c3 b6, and eventually blundering a simple back-rank mate with 35. Nxb5 Rc1#. Game B White, facing an early speculative sacrifice (6...Bxf2+), calmly defends by castling by hand (7. Kxf2, 8. Rf1, 9. Kg1) and efficiently trades down to consolidate the extra piece into a winning endgame. White B's play is fundamentally sound and coordinated, whereas White A plays erratically and blunders severely.", "confidence_valid": true }, { "model": "google/gemini-3.5-flash", "pair": "pair_17", "game_a": "game_13", "color_a": "white", "game_b": "game_16", "color_b": "white", "rating_a": 1126, "rating_b": 1752, "gap": 626, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.15, "correct": 1, "brier": 0.0225, "old_estimate_a": 1000, "old_estimate_b": 1500, "baseline_choice": "B", "baseline_credit": 1, "explanation": "White in Game B demonstrated significantly better tactical and positional awareness than White in Game A. In Game A, White committed several large tactical errors, including 17. c3? (which allowed a knight fork) and the final blunder 35. Nxb5??, which completely overlooked a simple back-rank checkmate. In contrast, White in Game B responded calmly to an early sacrifice on f2, consolidated their material advantage, and converted the game smoothly without giving Black any counterplay.", "confidence_valid": true }, { "model": "google/gemini-3.7-flash", "pair": "pair_17", "game_a": "game_13", "color_a": "white", "game_b": "game_16", "color_b": "white", "rating_a": 1126, "rating_b": 1752, "gap": 626, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.12, "correct": 1, "brier": 0.0144, "old_estimate_a": 1250, "old_estimate_b": 1600, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Player B shows far superior tactical and strategic discipline. Against Black's speculative piece sacrifice (6...Bxf2+), Player B cleanly executed artificial castling (8.Rf1, 9.Kg1), consolidated the extra piece smoothly, and converted the advantage without any concessions. In contrast, Player A in Game A played awkwardly in the opening (10.Qa1), permitted tactical vulnerabilities like 17.c3 (allowing 17...Nc2+), and ultimately blundered a basic back-rank mate with 35.Nxb5?? Rc1#.", "confidence_valid": true }, { "model": "qwen/qwen3.8-max-prime", "pair": "pair_17", "game_a": "game_13", "color_a": "white", "game_b": "game_16", "color_b": "white", "rating_a": 1126, "rating_b": 1752, "gap": 626, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.34, "correct": 1, "brier": 0.11560000000000002, "old_estimate_a": 1300, "old_estimate_b": 1400, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White played several dubious moves: 10.Qa1 was awkward, 18.Qb2 dropped the c5 rook to 18...bxc5, and 28.Ne8/29.Nc7 were poorly timed, losing material and being checkmated. Game B White, though winning partly due to Black's blunder 27...Nc7 (allowing the winning tactic), generally kept control after absorbing the Bxf2+ sac (7.Kxf2), coordinated well with 11.Bg5 and 23.Nxd4, and maintained a solid edge into the endgame. White B showed better overall piece play; White A made more elementary errors and lost convincingly.", "confidence_valid": true }, { "model": "anthropic/claude-fable-5.1", "pair": "pair_17", "game_a": "game_13", "color_a": "white", "game_b": "game_16", "color_b": "white", "rating_a": 1126, "rating_b": 1752, "gap": 626, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.25, "correct": 1, "brier": 0.0625, "old_estimate_a": 1150, "old_estimate_b": 1450, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White played very oddly: 3.dxc5 then 4.Bd2 5.Be3 wasting tempi, allowed 6...Qxb2, then 8.Rb3, 10.Qa1, 10.Nb1 and 13.Ra3/14.Rc3/16.Rc5?? dropping an exchange, and finally allowed 35...Rc1# back-rank mate. Black in A was also weak (early queen hunting). Game B White handled the Bxf2+ sacrifice calmly (7.Kxf2, 8.Rf1, 9.Kg1), won material cleanly with 12.h3/13.hxg4, consolidated with Qf2/Rae1 and converted via 25.Qe2 trade and 27.Re6, with the opponent resigning. Game B White showed solid, error-free play; Game A White showed basic tactical blindness.", "confidence_valid": true }, { "model": "google/gemini-3-flash-preview", "pair": "pair_17", "game_a": "game_13", "color_a": "white", "game_b": "game_16", "color_b": "white", "rating_a": 1126, "rating_b": 1752, "gap": 626, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.15, "correct": 1, "brier": 0.0225, "old_estimate_a": 1050, "old_estimate_b": 1600, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Target B shows significantly better tactical awareness and fundamental coordination. In Game A, White loses a pawn early (6... Qxb2), allows unnecessary pressure, and ultimately blunders a simple back-rank mate with 35. Nxb5??. In contrast, Target B handles an unsound sacrifice (6. Be2 Bxf2+) with composure, maintains the material advantage through clean development (17. Rae1, 26. Rxe2), and forces a resignation through solid positional play. Target A's failure to see the back-rank threat is a strong indicator of a lower rating compared to Target B's steady conversion.", "confidence_valid": true }, { "model": "google/gemini-3.6-flash", "pair": "pair_17", "game_a": "game_13", "color_a": "white", "game_b": "game_16", "color_b": "white", "rating_a": 1126, "rating_b": 1752, "gap": 626, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.15, "correct": 1, "brier": 0.0225, "old_estimate_a": 1150, "old_estimate_b": 1650, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B White played cleanly and displayed solid tactical awareness. After Black's unsound piece sacrifice with 6...Bxf2+ 7.Kxf2, Game B White consolidated comfortably, castled by hand, smoothly centralized forces, simplified the position, and maintained a clear winning advantage throughout without any major flaws. In contrast, Game A White showed weak tactical oversight, hanging the b2-pawn early with 6.Nc3, allowing tactical vulnerabilities like 17.c3? (permitting 17...Nc2+ fork), and ultimately blundering a back-rank checkmate in one with 35.Nxb5?? Rc1#. Game B White demonstrates significantly higher established playing strength.", "confidence_valid": true }, { "model": "openai/gpt-5.6-sol", "pair": "pair_17", "game_a": "game_13", "color_a": "white", "game_b": "game_16", "color_b": "white", "rating_a": 1126, "rating_b": 1752, "gap": 626, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.18, "correct": 1, "brier": 0.0324, "old_estimate_a": 1500, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B White responded calmly to Black’s dubious 6...Bxf2+: 7.Kxf2, 8.Rf1 and 9.Kg1 restored king safety. The tactical 12.h3 correctly met ...Bxf3 with 13.hxg4, winning the g4-knight, and White then simplified sensibly with 25.Qe2 Qxe2 26.Rxe2. Game A White showed some tactical awareness with 28.Ne8–30.Nxe6 and 31.Rxc6, but the opening rook excursions (8.Rb3, 13.Ra3, 14.Rc3, 16.Rc5) culminated in losing that rook after 18.Qb2? bxc5. Most decisively, 35.Nxb5?? overlooked the immediate 35...Rc1#.", "confidence_valid": true }, { "model": "openai/gpt-6-sol", "pair": "pair_17", "game_a": "game_13", "color_a": "white", "game_b": "game_16", "color_b": "white", "rating_a": 1126, "rating_b": 1752, "gap": 626, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.08, "correct": 1, "brier": 0.0064, "old_estimate_a": 1350, "old_estimate_b": 1600, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s White handled Black’s unsound 6...Bxf2+ calmly with 7.Kxf2, 8.Rf1 and 9.Kg1, then developed and simplified with 25.Qe2 while ahead. Game A’s White found useful tactics with 30.Nxe6 and 31.Rxc6, but spent many opening moves shuffling the rook, lost it to 18...bxc5, and missed the mating threat before 35...Rc1#. B’s play looks more consistently controlled.", "confidence_valid": true }, { "model": "openai/gpt-6.1-sol", "pair": "pair_17", "game_a": "game_13", "color_a": "white", "game_b": "game_16", "color_b": "white", "rating_a": 1126, "rating_b": 1752, "gap": 626, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.95, "correct": 1, "brier": null, "old_estimate_a": 1100, "old_estimate_b": 1800, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game A White repeatedly spent tempi on queen and rook excursions (10.Qa1, 13.Ra3, 14.Rc3), then left the attacked rook hanging with 18.Qb2 bxc5. Despite later tactical recovery, 35.Nxb5 allowed immediate mate with ...Rc1#. Game B White handled Black’s speculative sacrifice coherently: 7.Kxf2 followed by 8.Rf1 and 9.Kg1 secured the king, while 12.h3 and 14.Qxe2 emerged with extra material. Subsequent play—17.Rae1, 22.Nb3, 23.Nxd4, and 25.Qe2—showed purposeful coordination and simplification. B’s opponent played unsoundly, but B’s much cleaner execution strongly favors the higher established rating.", "confidence_valid": false }, { "model": "openai/gpt-5.6-luna", "pair": "pair_17", "game_a": "game_13", "color_a": "white", "game_b": "game_16", "color_b": "white", "rating_a": 1126, "rating_b": 1752, "gap": 626, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.15, "correct": 1, "brier": 0.0225, "old_estimate_a": 1250, "old_estimate_b": 1500, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B’s White showed stronger tactical and positional control: after 6...Bxf2+ 7.Kxf2, White stabilized the exposed king, then used 12.h3 and 13.hxg4 to eliminate Black’s active knight. White coordinated the rooks with 17.Rae1 and 18.Rxe1, improved the position, and invaded with 27.Re6. In Game A, White’s 18.c3?? allowed 18...bxc5, losing the rook on c5 outright. Although White later found 30.Nxe6 and 31.Rxc6, the game ended with the basic mating oversight 35...Rc1#. These games favor B as the more established player.", "confidence_valid": true }, { "model": "openai/gpt-6-astra", "pair": "pair_17", "game_a": "game_13", "color_a": "white", "game_b": "game_16", "color_b": "white", "rating_a": 1126, "rating_b": 1752, "gap": 626, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.95, "correct": 1, "brier": null, "old_estimate_a": 1300, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "Game B White showed coherent development and king safety: 7.Kxf2 accepted Black’s speculative sacrifice, followed by 8.Rf1, 9.Kg1 and 10.d4. White then coordinated with 17.Rae1 and simplified through 25.Qe2, retaining the extra piece. Game A White spent many opening tempi moving the same rook (Rb1–b3–a3–c3–c5), and 18.Qb2 allowed ...bxc5, conceding the exchange. Although 28.Ne8 and 29.Nc7 showed tactical awareness, 35.Nxb5 overlooked the immediate ...Rc1#. B’s opponent supplied an early advantage, but B’s subsequent handling was substantially more controlled.", "confidence_valid": false }, { "model": "z-ai/glm-5.3-flash", "pair": "pair_17", "game_a": "game_13", "color_a": "white", "game_b": "game_16", "color_b": "white", "rating_a": 1126, "rating_b": 1752, "gap": 626, "gap_band": ">=500", "low_band": "neither<1000", "gold": "B", "choice": "B", "probability_a": 0.75, "correct": 1, "brier": null, "old_estimate_a": 1350, "old_estimate_b": 1750, "baseline_choice": "B", "baseline_credit": 1, "explanation": "White in Game A shows poor technique: grabbing ...Qxb2 (6...), then after luck (Black's 18...bxc5?? dropping a rook) still collapses with 23.Nd6??/24.Ba4 weakening, allowing Black's smooth conversion and mate (35...Rc1#). White in Game B handles a tricky Scandinavian exchange calmly: 7.Kxf2 safely, solid development, 12.hxg4 winning a pawn, careful king safety (Kg1, Rf1), and converts cleanly with 25.Qe2 Qxe2 26.Rxe2 and 27.Re6 exploiting the pin. Black in B was slightly more solid than Black in A, making B's win more convincing. B's target likely had the higher rating.", "confidence_valid": false } ], "a": { "game": "game_13", "color": "white", "rating": 1126, "url": "https://lichess.org/GLVVikmb", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"0-1\"]\n[TimeControl \"600+0\"]\n\n1. d4 e6 2. Nf3 c5 3. dxc5 Qa5+ 4. Bd2 Qxc5 5. Be3 Qb5 6. Nc3 Qxb2 7. Rb1 Qa3\n8. Rb3 Qa5 9. Bd2 Bb4 10. Qa1 Ne7 11. Nb1 Bxd2+ 12. Nfxd2 f6 13. Ra3 Qc7 14.\nRc3 Nbc6 15. e3 Nd5 16. Rc5 Ndb4 17. c3 b6 18. Qb2 bxc5 19. cxb4 cxb4 20. Ne4\nO-O 21. Bb5 Qe5 22. Qxe5 Nxe5 23. Nd6 a6 24. Ba4 Nc6 25. O-O f5 26. Re1 Rf6 27.\nRc1 e5 28. Ne8 Re6 29. Nc7 Ra7 30. Nxe6 dxe6 31. Rxc6 Bd7 32. Rc4 Bxa4 33. Rxb4\nBb5 34. Nc3 Rc7 35. Nxb5 Rc1# 0-1" }, "b": { "game": "game_16", "color": "white", "rating": 1752, "url": "https://lichess.org/HCYL2QiH", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"1-0\"]\n[TimeControl \"600+0\"]\n\n1. e4 d5 2. exd5 e6 3. dxe6 Bxe6 4. Nf3 Bc5 5. c3 Qe7 6. Be2 Bxf2+ 7. Kxf2 Nf6\n8. Rf1 O-O 9. Kg1 Bd5 10. d4 Qd6 11. Bg5 Ng4 12. h3 Bxf3 13. hxg4 Bxe2 14. Qxe2\nNd7 15. Nd2 Rae8 16. Qf2 Nb6 17. Rae1 Rxe1 18. Rxe1 Nd5 19. Qf3 f6 20. Bh4 c5\n21. Bg3 Qc6 22. Nb3 cxd4 23. Nxd4 Qc4 24. a3 a5 25. Qe2 Qxe2 26. Rxe2 Rd8 27.\nRe6 Nc7 1-0" }, "wrong": 0, "gap": 626 }, { "id": "pair_03", "predictions": [ { "model": "anthropic/claude-opus-5.5", "pair": "pair_03", "game_a": "game_01", "color_a": "white", "game_b": "game_06", "color_b": "white", "rating_a": 1363, "rating_b": 1112, "gap": 251, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.58, "correct": 1, "brier": 0.17640000000000003, "old_estimate_a": 1250, "old_estimate_b": 1250, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "Both targets look weak. Game A White found 15.Nd5 and 16.Nf6+, forking king and queen to win the queen and reach a winning position. Its later collapse was dramatic: 40.Qb8+?? gave away the queen, and 42.Rxe8 left a lost ending. Game B White played an aimless opening (6.Bb1, 8.h4, 9.b3) and allowed 18...Ncxb3, losing the exchange. 25.Rf1?? dropped another exchange, and 27.Ke1?? hung the f6 knight to Qxf6. White was then quickly mated. Game B White shows more repeated basic tactical oversights, while Game A White showed some tactical vision before blundering. I lean slightly to A.", "confidence_valid": true }, { "model": "anthropic/claude-sonnet-5.5", "pair": "pair_03", "game_a": "game_01", "color_a": "white", "game_b": "game_06", "color_b": "white", "rating_a": 1363, "rating_b": 1112, "gap": 251, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.6, "correct": 1, "brier": 0.16000000000000003, "old_estimate_a": 1550, "old_estimate_b": 1550, "baseline_choice": "tie", "baseline_credit": 0.5, "explanation": "Both White players lost. In Game A, White lasted 42 moves in a messy game. Opening with 2.d3 and 3.Nc3 was passive, but 10.O-O-O and 11.h4 showed some attacking intent. 15.Nd5 and 17.Nxd7 were reasonable, and White kept pace until Black's passed e-pawn and 30...Bxg3 decided it. Black also played loosely, so White's resistance was fairly normal for the level. In Game B, White's play was much weaker: 8.h4 and 9.b3 loosened the position, 13.Ng5 and 14.Qd3 drifted, and 18.Qe3 allowed 18...Ncxb3 and 19...Nxa1. After 24.Nf6+ Kg7 25.Rf1, White hung material and was mated by 31...Qf2#, having ignored threats with 30.Qxb7. Game B's White therefore looks weaker.", "confidence_valid": true }, { "model": "google/gemini-3.8-flash", "pair": "pair_03", "game_a": "game_01", "color_a": "white", "game_b": "game_06", "color_b": "white", "rating_a": 1363, "rating_b": 1112, "gap": 251, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.78, "correct": 1, "brier": 0.04839999999999999, "old_estimate_a": 1200, "old_estimate_b": 1250, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Target A demonstrated good tactical alertness with 16. Nf6+, winning Black's queen, and built a completely winning position before suffering an apparent mouse-slip or massive lapse with 40. Qb8+. Up to that point, White in Game A navigated tactical complications and pushed pawns purposefully. In contrast, Target B played a very passive and awkward opening (4. Bd3, 6. Bb1) and was outplayed step-by-step. Black quickly dominated the board with active knights and bishops (18... Ncxb3), while Game B White shed material easily (20. Bxa1, 25. Rf1) and succumbed swiftly to 31... Qf2#.", "confidence_valid": true }, { "model": "google/gemini-3.1-pro-preview", "pair": "pair_03", "game_a": "game_01", "color_a": "white", "game_b": "game_06", "color_b": "white", "rating_a": 1363, "rating_b": 1112, "gap": 251, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.7, "correct": 1, "brier": 0.09000000000000002, "old_estimate_a": 1250, "old_estimate_b": 1200, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Target A played a somewhat standard opening and spotted a nice tactical sequence (15. Nd5 followed by the 16. Nf6+ royal fork) to win the queen, even though they later self-destructed with a horrific endgame blunder (40. Qb8+??). Target B played an extremely bizarre and unprincipled opening setup (4. Bd3, 6. Bb1, 7. Qc2) and was easily outplayed tactically by Black's knights, dropping an exchange to a simple fork (18... Ncxb3) and getting checkmated. A's grasp of opening fundamentals and early tactical awareness suggest a higher rating than B's beginner-like maneuvering.", "confidence_valid": true }, { "model": "google/gemini-3.5-flash", "pair": "pair_03", "game_a": "game_01", "color_a": "white", "game_b": "game_06", "color_b": "white", "rating_a": 1363, "rating_b": 1112, "gap": 251, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.75, "correct": 1, "brier": 0.0625, "old_estimate_a": 1250, "old_estimate_b": 1150, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A White showed reasonable tactical awareness, capitalizing on Black's mistake with the fork 16. Nf6+ to win the queen. Despite throwing away a completely winning position with the colossal blunder 40. Qb8+?? (likely a mouse slip or time-trouble hallucination), their play prior was competent. In contrast, Game B White showed poor fundamental understanding, wasting tempi with passive moves like 4. Bd3 and 6. Bb1. Game B White failed to develop effectively, lost material easily, and was quickly mated on f2 with no counterplay, suggesting a lower playing strength.", "confidence_valid": true }, { "model": "google/gemini-3.7-flash", "pair": "pair_03", "game_a": "game_01", "color_a": "white", "game_b": "game_06", "color_b": "white", "rating_a": 1363, "rating_b": 1112, "gap": 251, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.62, "correct": 1, "brier": 0.1444, "old_estimate_a": 1300, "old_estimate_b": 1350, "baseline_choice": "B", "baseline_credit": 0, "explanation": "White in Game A demonstrated solid tactical awareness by spotting the intermediate fork 16.Nf6+, winning Black's queen. White then navigated the complications well and pushed a dangerous passed pawn to e7 before suffering a catastrophic, single-move blunder with 40.Qb8+??. In contrast, White in Game B played an awkward opening setup (4.Bd3 and 6.Bb1), lost control of the dark squares and key outposts (13...Nd4, 18...Ncxb3), and showed little resistance as Black quickly broke through to deliver checkmate on move 31. Despite the endgame blunder, Player A displayed superior tactical vision and positional competence.", "confidence_valid": true }, { "model": "qwen/qwen3.8-max-prime", "pair": "pair_03", "game_a": "game_01", "color_a": "white", "game_b": "game_06", "color_b": "white", "rating_a": 1363, "rating_b": 1112, "gap": 251, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.62, "correct": 1, "brier": 0.1444, "old_estimate_a": 1150, "old_estimate_b": 1400, "baseline_choice": "B", "baseline_credit": 0, "explanation": "Both whites lost, but Game B's collapse was far more one-sided. B's White played passively (6.Bb1, 7.Qc2), then after 17...Nc5 and 18...Ncxb3 lost material with no compensation; 24.Nf6+ Kg7 simply hung into 25...Bxf1, and Black mated with 31...Qf2#. Game A's White built a playable kingside attack (11.h4, 14.Bg5, 16.Nf6+), creating real tactical problems, and only lost after a long, complex endgame where 30.dxe4 allowed 30...Bxg3. A showed more competence in a richer middlegame.", "confidence_valid": true }, { "model": "anthropic/claude-fable-5.1", "pair": "pair_03", "game_a": "game_01", "color_a": "white", "game_b": "game_06", "color_b": "white", "rating_a": 1363, "rating_b": 1112, "gap": 251, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.58, "correct": 1, "brier": 0.17640000000000003, "old_estimate_a": 1300, "old_estimate_b": 900, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A White played a coherent if passive setup, found the tactical shot 15.Nd5/16.Nf6+ winning the queen for two pieces, then blundered a rook with 27.Re1?? allowing 27...e4 and later 40.Qb8+?? hallucinating a promotion trick; still, the earlier play showed genuine tactical awareness. Game B White drifted badly: 6.Bb1 and 7.Qc2 wasted tempi, 9.b3 and 13.Ng5 allowed ...Nd4 and ...Ncxb3 forking queen and rook, dropping the a1 rook and then c4, Rf1 for free, and finally walked into 31...Qf2# after grabbing pawns. Black in A also played weakly, making both opponents comparable. A's White seems slightly stronger.", "confidence_valid": true }, { "model": "google/gemini-3-flash-preview", "pair": "pair_03", "game_a": "game_01", "color_a": "white", "game_b": "game_06", "color_b": "white", "rating_a": 1363, "rating_b": 1112, "gap": 251, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.7, "correct": 1, "brier": 0.09000000000000002, "old_estimate_a": 1350, "old_estimate_b": 1250, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Player A shows superior tactical awareness and opening knowledge compared to Player B. In Game A, White executes a strong tactical sequence beginning with 15. Nd5 and 16. Nf6+ to win the Queen for two pieces. While Player A ultimately collapses in the endgame, blundering the Queen back with 40. Qb8+, their play was coherent until the time pressure phase. Player B, by contrast, plays a nonsensical opening (4. Bd3, 6. Bb1) and shows poor tactical vision, allowing a simple knight fork on a1 (19. f5) and failing to defend against a basic mating sequence. Player A’s peak level within the game is significantly higher.", "confidence_valid": true }, { "model": "google/gemini-3.6-flash", "pair": "pair_03", "game_a": "game_01", "color_a": "white", "game_b": "game_06", "color_b": "white", "rating_a": 1363, "rating_b": 1112, "gap": 251, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.78, "correct": 1, "brier": 0.04839999999999999, "old_estimate_a": 1400, "old_estimate_b": 1300, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A White demonstrated good tactical vision by finding 16. Nf6+ to win Black's queen, subsequently building a completely dominant position before suffering a catastrophic late blunder (40. Qb8+). Game B White showed poor opening planning (4. Bd3, 6. Bb1), lost central control, allowed Black's knights to infiltrate unpunished (13...Nd4, 17...Nc5), and quickly collapsed under pressure (25. Rf1, 26...Nxg5), leading to mate on move 31. Despite Game A White's single-move blunder, their overall tactical awareness and positional play were significantly superior to Game B White.", "confidence_valid": true }, { "model": "openai/gpt-5.6-sol", "pair": "pair_03", "game_a": "game_01", "color_a": "white", "game_b": "game_06", "color_b": "white", "rating_a": 1363, "rating_b": 1112, "gap": 251, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.86, "correct": 1, "brier": 0.019600000000000003, "old_estimate_a": 1450, "old_estimate_b": 1400, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A White showed substantially better tactical awareness and coordination: 15.Nd5 followed by 16.Nf6+ and 17.Nxd7 won Black’s queen, while 26.Qc6 and the later checks kept pressure. The conversion was poor—especially 40.Qb8+?, which allowed ...Rxb8 and liquidation—but the preceding play displayed real calculation. Game B White spent many opening tempi on Bd3-b1, a3, h4, and b3, then overextended with 15.f3 and 16.f4. More seriously, White allowed 18...Ncxb3 and 19...Nxa1, played 25.Rf1? Bxf1, and failed to address the mating attack ending in 31...Qf2#. A therefore looks more established despite also losing.", "confidence_valid": true }, { "model": "openai/gpt-6-sol", "pair": "pair_03", "game_a": "game_01", "color_a": "white", "game_b": "game_06", "color_b": "white", "rating_a": 1363, "rating_b": 1112, "gap": 251, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.88, "correct": 1, "brier": 0.0144, "old_estimate_a": 1400, "old_estimate_b": 1300, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A’s White spotted a concrete tactic with 15.Nd5 and 16.Nf6+, winning Black’s queen with 17.Nxd7. Although 40.Qb8+ was a decisive blunder, White had pursued a dangerous passed pawn with 38.e6 and 39.e7. Game B’s White showed less consistent awareness of threats: 19.f5 allowed ...Nxa1 to take a rook, 25.Rf1 allowed ...Bxf1 to take the other, and 30.Qxb7 overlooked the mating attack. The contrast favors A despite both losses.", "confidence_valid": true }, { "model": "openai/gpt-6.1-sol", "pair": "pair_03", "game_a": "game_01", "color_a": "white", "game_b": "game_06", "color_b": "white", "rating_a": 1363, "rating_b": 1112, "gap": 251, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.79, "correct": 1, "brier": 0.044099999999999986, "old_estimate_a": 1300, "old_estimate_b": 1200, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A White showed more coherent development, castled early, and exploited Black’s queen placement with 15.Nd5 and 16.Nf6+, winning the queen on d7. White later made serious errors: 31.e5 allowed ...Bxe1, and 40.Qb8+ overlooked Black’s second rook in the promotion combination. Game B White spent opening tempi on Bd3–b1 and f3–f4 while leaving the king uncastled. After 18...Ncxb3, 19.f5 allowed ...Nxa1; later 25.Rf1 simply permitted ...Bxf1, followed by losing the knight to ...Qxf6. Despite facing a more coherent opponent, B’s repeated development and material-safety failures suggest lower established strength.", "confidence_valid": true }, { "model": "openai/gpt-5.6-luna", "pair": "pair_03", "game_a": "game_01", "color_a": "white", "game_b": "game_06", "color_b": "white", "rating_a": 1363, "rating_b": 1112, "gap": 251, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.72, "correct": 1, "brier": 0.07840000000000001, "old_estimate_a": 1500, "old_estimate_b": 1200, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A’s White showed a significant tactical resource with 16.Nf6+ followed by 17.Nxd7, winning Black’s queen, and later advanced a dangerous passed e-pawn. However, White then overlooked 30...Bxg3 and 31...Bxe1, losing a rook, and the eventual promotion tactic failed. Game B’s White made more fundamental oversights: 19.f5 allowed ...Nxa1 to win the rook, 25.Rf1 allowed ...Bxf1, and 27.Ke1 permitted ...Qxf6 before the forced mate 31...Qf2#. Thus A’s play contained stronger tactical ideas despite serious later inaccuracies.", "confidence_valid": true }, { "model": "openai/gpt-6-astra", "pair": "pair_03", "game_a": "game_01", "color_a": "white", "game_b": "game_06", "color_b": "white", "rating_a": 1363, "rating_b": 1112, "gap": 251, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.81, "correct": 1, "brier": 0.03609999999999998, "old_estimate_a": 1350, "old_estimate_b": 1250, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A White shows better tactical awareness and coordination: 15.Nd5 followed by 16.Nf6+ and 17.Nxd7 exploits Black’s queen placement, while castling and centralizing the rooks provide a coherent foundation. A nevertheless makes serious errors, notably 31.e5 allowing ...Bxe1 and the losing liquidation beginning 40.Qb8+. Game B White spends several opening tempi on Bd3–b1 and pawn moves, then allows ...Nc5–b3–a1 to win a rook. Later, 25.Rf1 simply exposes another rook to ...Bxf1, and 27.Ke1 leaves the knight on f6 undefended. Despite both players’ blunders, A demonstrates the stronger overall baseline.", "confidence_valid": true }, { "model": "z-ai/glm-5.3-flash", "pair": "pair_03", "game_a": "game_01", "color_a": "white", "game_b": "game_06", "color_b": "white", "rating_a": 1363, "rating_b": 1112, "gap": 251, "gap_band": "200-499", "low_band": "neither<1000", "gold": "A", "choice": "A", "probability_a": 0.9, "correct": 1, "brier": 0.009999999999999995, "old_estimate_a": 1650, "old_estimate_b": 1350, "baseline_choice": "A", "baseline_credit": 1, "explanation": "Game A's White plays a coherent strategic game: regrouping Nf6+/Nxd7 tricks (15-17), trading into a defensible endgame, and only slipping late (30.dxe4? Bxg3 dropping a rook after 31...Bxe1). Game B's White self-destructs: 13.Ng5? allows ...Nd4 pressuring f3, then 16.f4? Bxg5 loses material, 18.Qe3? runs into 18...Ncxb3, and 19.f5? Nxa1 simply drops a rook; Black converts with 24...Qxf6 and a clean 31...Qf2#. White's handling in B (open king, aimless wing play 8.h4) is far weaker than A's, who fought a sound attacking plan.", "confidence_valid": true } ], "a": { "game": "game_01", "color": "white", "rating": 1363, "url": "https://lichess.org/n6H9m2oi", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"0-1\"]\n[TimeControl \"600+0\"]\n\n1. e4 e5 2. d3 d5 3. Nc3 dxe4 4. Nxe4 f5 5. Nc3 Nc6 6. Nf3 Bd6 7. Bg5 Qd7 8.\nQe2 h6 9. Bd2 g5 10. O-O-O Nf6 11. h4 g4 12. Nh2 h5 13. g3 b6 14. Bg5 Nh7 15.\nNd5 Nxg5 16. Nf6+ Ke7 17. Nxd7 Kxd7 18. hxg5 Bb7 19. f3 Nd4 20. Qf2 gxf3 21.\nNxf3 Nxf3 22. Bg2 Rab8 23. Bxf3 Bxf3 24. Qxf3 Ke6 25. Rdf1 Rbf8 26. Qc6 a5 27.\nRe1 e4 28. Qc4+ Kd7 29. Qa4+ c6 30. dxe4 Bxg3 31. e5 Bxe1 32. Rxe1 h4 33. Qd4+\nKc7 34. Qd6+ Kb7 35. Qd7+ Ka6 36. Qd3+ b5 37. Qd6 Kb6 38. e6 h3 39. e7 Re8 40.\nQb8+ Rxb8 41. e8=Q Rbxe8 42. Rxe8 Rxe8 0-1" }, "b": { "game": "game_06", "color": "white", "rating": 1112, "url": "https://lichess.org/wLJhUOmU", "pgn": "[White \"White\"]\n[Black \"Black\"]\n[Result \"0-1\"]\n[TimeControl \"600+0\"]\n\n1. c4 d6 2. Nc3 Nf6 3. e3 Bd7 4. Bd3 e5 5. a3 Be7 6. Bb1 O-O 7. Qc2 g6 8. h4 h5\n9. b3 Bf5 10. e4 Bd7 11. Bb2 Nc6 12. Nf3 Bg4 13. Ng5 Nd4 14. Qd3 Nd7 15. f3 Be6\n16. f4 Bxg5 17. hxg5 Nc5 18. Qe3 Ncxb3 19. f5 Nxa1 20. Bxa1 Bxc4 21. fxg6 fxg6\n22. Nd5 Ne6 23. Qc3 Ba6 24. Nf6+ Kg7 25. Rf1 Bxf1 26. Kxf1 Nxg5 27. Ke1 Qxf6\n28. Qxc7+ Kh6 29. Bd3 Rac8 30. Qxb7 Rc1+ 31. Ke2 Qf2# 0-1" }, "wrong": 0, "gap": 251 } ], "prompt_template": "Compare the established playing strength of two chess players from one complete game each. Both games are Lichess rapid games.\n\nThe target in Game A is WHITE/BLACK as specified below. The target in Game B is separately specified. Judge only these two target players, using the moves and their opponents' play as evidence. Infer which target player likely had the higher established Lichess rapid rating before these games. A single game's winner need not be the stronger player across games.\n\nChoose A or B even if uncertain. Give the probability that target A was stronger, from 0 to 1, consistent with your choice (0.5 is allowed for a forced guess). Give a concise explanation, at most 120 words, citing concrete moves with game and color. Do not provide numerical rating estimates. Use only the supplied games; no external tools.\n\nGAME A — target player: {color_a}\n{pgn_a}\n\nGAME B — target player: {color_b}\n{pgn_b}\n\nReturn JSON with exactly: stronger (\"A\" or \"B\"), probability_a_stronger (number), explanation (string).\n", "primary_pairs": 25, "diagnostic_reversals": 5, "cost_usd": 6.1433541126, "run_start": "2026-10-13", "run_end": "2026-10-13", "amendment": { "recorded_at": "2026-10-13T00:38:42.754928+00:00", "reason": "Four early responses gave an unambiguous A/B choice and explanation but a probability for the opposite choice. Strict validation quarantined these models, although the primary categorical outcome was usable. This is an outcome-independent field-validation change after these responses were seen, before resuming affected models.", "change": "Accept the existing stronger field unchanged even when its probability contradicts it. Mark confidence invalid; exclude those values from Brier and probability-shift summaries. Preserve raw probabilities, choices, explanations, original ledger snapshot and validation errors. No probability inversion, choice correction or call retry. Continue only unstarted requests. All other validation and failure rules remain.", "unchanged": "Same pairs, sixteen models, prompts, payloads, primary outcome, baselines, price ceilings and cumulative $20 budget. Invalid confidence is reported per model; the primary leaderboard includes every valid categorical response regardless of correctness.", "execution": "Let original workers finish first. Acquire same exclusive lock, snapshot strict ledger, reclassify only errors exactly matching Choice contradicts probability after raw identity/billing checks, then resume still-unstarted frozen requests.", "replay": "python amendment.py --score" } }