From 3bceafbc12ea2ac64cb1cfce9986d0e30f92a444 Mon Sep 17 00:00:00 2001 From: Ilia Denisov Date: Mon, 15 Jun 2026 21:37:23 +0200 Subject: [PATCH] feat(robot): occasional off-strategy deviation, strict in the endgame MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The robot followed its per-game playToWin/lose intent on every move, which made the outcome too predictable. It now flips that intent for a single move on ~20% of opening/midgame turns (a winning robot eases off, a losing one surges ahead), so the chosen strategy may not pan out — which favours the human. The chance tapers linearly to 0 over the last 14 tiles in the bag and is 0 once the bag is empty, so the endgame follows the chosen strategy strictly. The decision is deterministic from the seed (mix(seed,"deviate",moveCount)) and applies to both robot paths via the shared selectMove; the per-game play-to-win intent the admin card shows is unchanged. Adds deviateProb/deviates helpers and unit tests (taper bounds + monotonicity, never-in-endgame, determinism, ~20% distribution); bakes the behaviour into ARCHITECTURE §7, FUNCTIONAL (+_ru), backend/README, PRERELEASE and PLAN Stage 5. --- PLAN.md | 5 +- PRERELEASE.md | 8 +++ backend/README.md | 3 +- backend/internal/robot/driver.go | 17 ++++-- backend/internal/robot/strategy.go | 37 +++++++++++++ backend/internal/robot/strategy_test.go | 72 +++++++++++++++++++++++++ docs/ARCHITECTURE.md | 9 +++- docs/FUNCTIONAL.md | 11 ++-- docs/FUNCTIONAL_ru.md | 10 ++-- 9 files changed, 156 insertions(+), 16 deletions(-) diff --git a/PLAN.md b/PLAN.md index 115de83..c4214b1 100644 --- a/PLAN.md +++ b/PLAN.md @@ -648,7 +648,10 @@ cannot submit; three-way admin filter. - **Margin** (interview): pick the candidate whose resulting margin (own+move−opp) is closest to **[1,30]** when playing to win / **[−30,−1]** when playing to lose, tie-broken toward the conservative edge; no legal play → exchange the full rack - when the bag can refill it, else pass. + when the bag can refill it, else pass. *(Later refined — separate PR: on ≈20% of + opening/midgame moves the robot flips its intent for that one move, the chance + tapering to 0 over the last 14 bag tiles and 0 at an empty bag, so the endgame + stays strict; deterministic from the seed.)* - **Substitution** (interview): a matchmaker **reaper** (`Reap`/`RunReaper`) substitutes a pooled robot after a **10 s** wait (`BACKEND_LOBBY_ROBOT_WAIT`), `NewMatchmaker` now takes a `RobotProvider`. A waiter learns of a match — human diff --git a/PRERELEASE.md b/PRERELEASE.md index d0b81dc..1d4544a 100644 --- a/PRERELEASE.md +++ b/PRERELEASE.md @@ -511,3 +511,11 @@ Then Stage 18. (`GameRow`/`GameDetailView` gain `VsAI`); (4) `games_started_total` / `games_abandoned_total` gain a **`vs_ai`** attribute and the Grafana *Game domain* dashboard splits started/abandoned into **human** and **AI** panels. + - **Follow-up (separate PR — strategy deviation):** the robot now plays **≈20%** of opening/midgame moves + *against* its per-game `playToWin` intent (toward the opposite margin band — a winning robot eases off, a + losing one surges ahead), tapering linearly to **0 over the last 14 bag tiles** and **0 once the bag is + empty**, so the endgame follows the chosen strategy strictly while earlier outcomes can swing the human's + way. Deterministic from the seed (`mix(seed,"deviate",moveCount)`), applied to **both** robot paths via + the shared `selectMove`; the per-game intent (and the admin card) is unchanged. Tests: `robot` unit + (taper bounds + monotonicity, never-in-endgame, determinism, ~20% distribution). Bake-back: + `docs/ARCHITECTURE.md` §7, `docs/FUNCTIONAL.md` (+`_ru`), `backend/README.md`, `PLAN.md` Stage 5. diff --git a/backend/README.md b/backend/README.md index 849da24..663260d 100644 --- a/backend/README.md +++ b/backend/README.md @@ -50,7 +50,8 @@ each a `kind='robot'` identity, provisioned at startup with chat and friend requests blocked — backs human-like, per-language composed names. A background driver plays the robot's moves through the public game API as an ordinary seated player (so only `internal/engine` imports the solver): it decides once per game whether to play to -win (≈ 40%), targets a small score margin, and times its moves with a move-number-aware +win (≈ 40%), targets a small score margin — with an occasional off-strategy move that tapers to +none as the bag empties — and times its moves with a move-number-aware right-skewed delay (quick openings, long endgames), a night-sleep window anchored to the opponent's timezone, and nudge behaviour — all derived deterministically from the game seed, so it keeps no extra state. A background **reaper** seats a pooled robot (matching the game's language) in any open diff --git a/backend/internal/robot/driver.go b/backend/internal/robot/driver.go index 711d73c..edac356 100644 --- a/backend/internal/robot/driver.go +++ b/backend/internal/robot/driver.go @@ -67,7 +67,8 @@ func (s *Service) handle(ctx context.Context, rt game.RobotTurn, now time.Time) // Honest-AI game: the robot moves the instant it is its turn — no sleep window and // no proactive nudge (chat and nudge are disabled in these games, and the player // chose an opponent that "moves at once"). It still plays to the same per-game - // strength (playToWin) and margin band as the human-mimicry path. + // strength (playToWin), margin band and occasional off-strategy deviation as the + // human-mimicry path. if rt.VsAI { if rt.ToMove == rt.RobotSeat { return s.act(ctx, rt, now) @@ -164,9 +165,11 @@ func (s *Service) maybeNudge(ctx context.Context, rt game.RobotTurn, now time.Ti return nil } -// act reads the live turn, chooses a move by margin and submits it. State that -// has moved on since the scan (a finished game, a turn that is no longer the -// robot's) surfaces as a benign error and is skipped. +// act reads the live turn, chooses a move by margin — usually toward the robot's +// per-game intent, but with an occasional off-strategy deviation that fades to none +// as the bag empties — and submits it. State that has moved on since the scan (a +// finished game, a turn that is no longer the robot's) surfaces as a benign error +// and is skipped. func (s *Service) act(ctx context.Context, rt game.RobotTurn, now time.Time) error { st, err := s.games.GameState(ctx, rt.GameID, rt.RobotID) if err != nil { @@ -179,7 +182,11 @@ func (s *Service) act(ctx context.Context, rt game.RobotTurn, now time.Time) err myScore := st.Game.Seats[st.Seat].Score oppScore := bestOpponentScore(st.Game.Seats, st.Seat) - d := selectMove(cands, myScore, oppScore, playToWin(rt.Seed), defaultBand, st.Rack, st.BagLen) + win := playToWin(rt.Seed) + if deviates(rt.Seed, rt.MoveCount, st.BagLen) { + win = !win // an occasional off-strategy move; never once the bag is empty + } + d := selectMove(cands, myScore, oppScore, win, defaultBand, st.Rack, st.BagLen) var res game.MoveResult switch d.kind { diff --git a/backend/internal/robot/strategy.go b/backend/internal/robot/strategy.go index 5ac917a..a6c9736 100644 --- a/backend/internal/robot/strategy.go +++ b/backend/internal/robot/strategy.go @@ -23,6 +23,15 @@ const ( // human wins about 60% of games (docs/ARCHITECTURE.md §7). playToWinPercent = 40 + // The robot occasionally plays a single move against its per-game win/lose + // intent (an off-strategy "wobble"), so the chosen strategy may not pan out — + // which favours the human. deviateMaxProb is the peak probability of that, held + // through the opening and midgame; it tapers linearly to 0 over the last + // deviateTaperTiles tiles left in the bag, reaching 0 once the bag is empty, so + // the endgame follows the chosen strategy strictly (docs/ARCHITECTURE.md §7). + deviateMaxProb = 0.20 + deviateTaperTiles = 14 + // The robot's think time depends on how far the game has progressed: early moves // are quick and late moves can be long (endgame deliberation). The delay is drawn // from a band that interpolates with the move count from [delayEarlyLoMinutes, @@ -129,6 +138,34 @@ func PlayToWin(seed int64) bool { return playToWin(seed) } // win in any given game (the admin console shows it alongside the per-game decision). const PlayToWinTargetPercent = playToWinPercent +// deviateProb is the probability that the robot plays a single move against its +// per-game win/lose intent, given the number of tiles left in the bag. It is +// deviateMaxProb through the opening and midgame, then tapers linearly to 0 over +// the last deviateTaperTiles tiles, reaching 0 once the bag is empty so the endgame +// follows the chosen strategy strictly. +func deviateProb(bagLen int) float64 { + switch { + case bagLen <= 0: + return 0 + case bagLen >= deviateTaperTiles: + return deviateMaxProb + default: + return deviateMaxProb * float64(bagLen) / float64(deviateTaperTiles) + } +} + +// deviates reports whether the robot deviates from its per-game win/lose intent on +// the move at moveCount: a deterministic per-turn draw (mix/unitFloat, like the +// think-time sampling) against deviateProb(bagLen), so it is reproducible across +// restarts and never fires once the bag is empty. +func deviates(seed int64, moveCount, bagLen int) bool { + p := deviateProb(bagLen) + if p <= 0 { + return false + } + return unitFloat(mix(seed, "deviate", moveCount)) < p +} + // NextMoveAt is the deterministic instant the robot is scheduled to play the move at // moveCount, given when the turn started and the opponent's timezone (which anchors the // robot's sleep window). It is the sampled think-time delay, deferred to the end of the diff --git a/backend/internal/robot/strategy_test.go b/backend/internal/robot/strategy_test.go index f1d2cbe..84ff7fd 100644 --- a/backend/internal/robot/strategy_test.go +++ b/backend/internal/robot/strategy_test.go @@ -1,6 +1,7 @@ package robot import ( + "math" "sort" "testing" "time" @@ -238,6 +239,77 @@ func TestPlayToWinExport(t *testing.T) { } } +// TestDeviateProbTaper checks the deviation probability is deviateMaxProb while the +// bag holds at least deviateTaperTiles tiles, halves at the taper midpoint, is 0 +// once the bag is empty, and stays within [0, deviateMaxProb] and non-decreasing. +func TestDeviateProbTaper(t *testing.T) { + if p := deviateProb(0); p != 0 { + t.Errorf("deviateProb(0) = %v, want 0 (strict endgame)", p) + } + if p := deviateProb(deviateTaperTiles); p != deviateMaxProb { + t.Errorf("deviateProb(%d) = %v, want %v", deviateTaperTiles, p, deviateMaxProb) + } + if p := deviateProb(deviateTaperTiles + 50); p != deviateMaxProb { + t.Errorf("deviateProb above the taper = %v, want %v (capped)", p, deviateMaxProb) + } + if p := deviateProb(deviateTaperTiles / 2); math.Abs(p-deviateMaxProb/2) > 1e-9 { + t.Errorf("deviateProb at half taper = %v, want ~%v", p, deviateMaxProb/2) + } + prev := -1.0 + for bag := 0; bag <= deviateTaperTiles+5; bag++ { + p := deviateProb(bag) + if p < 0 || p > deviateMaxProb { + t.Fatalf("deviateProb(%d) = %v out of [0,%v]", bag, p, deviateMaxProb) + } + if p < prev { + t.Fatalf("deviateProb not non-decreasing: bag %d gives %v after %v", bag, p, prev) + } + prev = p + } +} + +// TestDeviatesNeverInEndgame checks the robot never deviates once the bag is empty, +// for every seed and move count, so the endgame follows the chosen strategy strictly. +func TestDeviatesNeverInEndgame(t *testing.T) { + for seed := int64(1); seed <= 5000; seed++ { + for mc := 0; mc < 40; mc++ { + if deviates(seed, mc, 0) { + t.Fatalf("deviates with an empty bag for seed=%d mc=%d", seed, mc) + } + } + } +} + +// TestDeviatesDeterministic checks the per-turn deviation draw is reproducible for a +// (seed, moveCount, bagLen), so the driver recomputes the same decision on every scan. +func TestDeviatesDeterministic(t *testing.T) { + for seed := int64(1); seed <= 500; seed++ { + for mc := 0; mc < 30; mc++ { + got := deviates(seed, mc, deviateTaperTiles) + if deviates(seed, mc, deviateTaperTiles) != got { + t.Fatalf("deviates not deterministic for seed=%d mc=%d", seed, mc) + } + } + } +} + +// TestDeviatesDistribution checks the deviation rate over many games lands near +// deviateMaxProb while the bag is full (above the taper), at a fixed move count. +func TestDeviatesDistribution(t *testing.T) { + const n = 20000 + hits := 0 + for seed := int64(1); seed <= n; seed++ { + if deviates(seed, 3, deviateTaperTiles+20) { + hits++ + } + } + pct := float64(hits) / float64(n) * 100 + want := deviateMaxProb * 100 + if pct < want-2 || pct > want+2 { + t.Errorf("deviation rate = %.1f%%, want ~%.0f%% (±2)", pct, want) + } +} + // TestProactiveNudgeGap checks the proactive-nudge schedule: the first gap (refIdle 0) is // ~60-90 min, every gap stays within [60 min, 6 h] and is deterministic, and the gap lengthens // as the idle grows (the median at 12 h idle exceeds the median at the start). diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index aede476..4b6fab0 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -407,7 +407,14 @@ English game the Latin pool. (`engine.Candidates`) the move whose resulting lead (playing to win) or deficit (playing to lose) is closest to a small band (**1–30 points**), rather than always the maximum; with no legal play it exchanges a full rack when the bag can - refill it, else passes. + refill it, else passes. On **≈20%** of moves through the opening and midgame it + **deviates** — playing that single move toward the *opposite* band (a winning + robot eases off, a losing one surges ahead), so the chosen strategy may not pan + out, which favours the human; the deviation chance tapers linearly to **0 over + the last 14 tiles in the bag** and is **0 once the bag is empty**, so the endgame + follows the per-game intent strictly. It is **deterministic from the seed** + (`mix(seed,"deviate",moveCount)`), a per-move wobble that leaves the per-game + play-to-win intent (and the admin card) unchanged. - **Timing**: the per-move delay is **move-number-aware** — a right-skewed sample (exponent k=4, short delays frequent) from a band that interpolates from **[3, 10] min** at the first move to **[10, 90] min** by ~28 moves, so openings are diff --git a/docs/FUNCTIONAL.md b/docs/FUNCTIONAL.md index 05c78b3..f83986f 100644 --- a/docs/FUNCTIONAL.md +++ b/docs/FUNCTIONAL.md @@ -141,8 +141,10 @@ When auto-match finds no human within the wait window (1.5–3 minutes), a robot takes the empty seat of the game you are already waiting in. It is meant to feel like a person: it decides once per game whether to play to win (about 40% of the time, so the human wins most games), aims for a close score rather than crushing or throwing the game, -and plays at a human pace — short thinking times for most moves, the occasional long -one, and a night-time pause that tracks the player's own day. It answers a nudge +now and then plays a single move against that plan — a surprise lead or a slack move — +yet holds to the plan once the bag empties, and plays at a human pace — short thinking +times for most moves, the occasional long one, and a night-time pause that tracks the +player's own day. It answers a nudge within a few minutes and nudges back when the player has been away a long time. It carries a human-like, language-appropriate name (a Russian game draws mostly Russian names); it does not chat, and **silently ignores friend requests** — a request to a @@ -150,8 +152,9 @@ robot stays pending and expires, exactly like a human who never responds. The same robot also backs the **honest-AI quick game** the player chooses directly (above). There it makes no pretence: it is shown as **🤖** everywhere, joins and moves at once (no thinking time -or night pause), keeps the same strength (it still plays to win only about 40% of the time), and -chat, nudge and add-friend are off. AI games are **practice** — they never count toward a player's +or night pause), keeps the same strength (it still plays to win only about 40% of the time, with +the same occasional move against its plan that fades out by the endgame), and chat, nudge and +add-friend are off. AI games are **practice** — they never count toward a player's statistics. ### Social: friends, block, chat, nudge diff --git a/docs/FUNCTIONAL_ru.md b/docs/FUNCTIONAL_ru.md index d77f430..060b2c4 100644 --- a/docs/FUNCTIONAL_ru.md +++ b/docs/FUNCTIONAL_ru.md @@ -146,8 +146,10 @@ nudge) приходят от бота **этой партии** — по язы в которой вы уже ждёте, занимает робот-соперник. Он задуман неотличимым от человека: один раз за партию решает, играть ли на победу (примерно в 40% случаев, так что человек выигрывает большинство партий), целится в близкий счёт, а не в разгром или -поддавки, и ходит с человеческим темпом — чаще короткие раздумья, изредка долгие, и -ночная пауза, подстроенная под день игрока. На nudge отвечает за несколько минут и +поддавки, время от времени делает один ход вопреки этому плану — неожиданный отрыв или +слабый ход, — но к концу партии (когда мешок пуст) держится плана, и ходит с +человеческим темпом — чаще короткие раздумья, изредка долгие, и ночная пауза, +подстроенная под день игрока. На nudge отвечает за несколько минут и сам шлёт nudge, когда игрок надолго пропал. Носит человекоподобное имя, подходящее языку партии (в русской партии — в основном русские имена); не общается в чате и **молча игнорирует заявки в друзья** — заявка роботу остаётся в ожидании и истекает, @@ -155,8 +157,8 @@ nudge) приходят от бота **этой партии** — по язы Тот же робот стоит и за **честной игрой против ИИ**, которую игрок выбирает напрямую (выше). Там он не притворяется: везде показан как **🤖**, присоединяется и ходит сразу (без раздумий и ночной -паузы), сохраняет ту же силу (по-прежнему играет на победу лишь примерно в 40% партий), а чат, nudge -и «добавить в друзья» выключены. Партии с ИИ — это **тренировка**: они не идут в статистику игрока. +паузы), сохраняет ту же силу (по-прежнему играет на победу лишь примерно в 40% партий, с теми же +редкими ходами вопреки плану, затухающими к эндшпилю), а чат, nudge и «добавить в друзья» выключены. Партии с ИИ — это **тренировка**: они не идут в статистику игрока. ### Социальное: друзья, блок, чат, nudge Подружиться можно двумя способами: погасить **одноразовый код**, который выпускает -- 2.52.0