diff --git a/PLAN.md b/PLAN.md index 115de83..c4214b1 100644 --- a/PLAN.md +++ b/PLAN.md @@ -648,7 +648,10 @@ cannot submit; three-way admin filter. - **Margin** (interview): pick the candidate whose resulting margin (own+move−opp) is closest to **[1,30]** when playing to win / **[−30,−1]** when playing to lose, tie-broken toward the conservative edge; no legal play → exchange the full rack - when the bag can refill it, else pass. + when the bag can refill it, else pass. *(Later refined — separate PR: on ≈20% of + opening/midgame moves the robot flips its intent for that one move, the chance + tapering to 0 over the last 14 bag tiles and 0 at an empty bag, so the endgame + stays strict; deterministic from the seed.)* - **Substitution** (interview): a matchmaker **reaper** (`Reap`/`RunReaper`) substitutes a pooled robot after a **10 s** wait (`BACKEND_LOBBY_ROBOT_WAIT`), `NewMatchmaker` now takes a `RobotProvider`. A waiter learns of a match — human diff --git a/PRERELEASE.md b/PRERELEASE.md index d0b81dc..1d4544a 100644 --- a/PRERELEASE.md +++ b/PRERELEASE.md @@ -511,3 +511,11 @@ Then Stage 18. (`GameRow`/`GameDetailView` gain `VsAI`); (4) `games_started_total` / `games_abandoned_total` gain a **`vs_ai`** attribute and the Grafana *Game domain* dashboard splits started/abandoned into **human** and **AI** panels. + - **Follow-up (separate PR — strategy deviation):** the robot now plays **≈20%** of opening/midgame moves + *against* its per-game `playToWin` intent (toward the opposite margin band — a winning robot eases off, a + losing one surges ahead), tapering linearly to **0 over the last 14 bag tiles** and **0 once the bag is + empty**, so the endgame follows the chosen strategy strictly while earlier outcomes can swing the human's + way. Deterministic from the seed (`mix(seed,"deviate",moveCount)`), applied to **both** robot paths via + the shared `selectMove`; the per-game intent (and the admin card) is unchanged. Tests: `robot` unit + (taper bounds + monotonicity, never-in-endgame, determinism, ~20% distribution). Bake-back: + `docs/ARCHITECTURE.md` §7, `docs/FUNCTIONAL.md` (+`_ru`), `backend/README.md`, `PLAN.md` Stage 5. diff --git a/backend/README.md b/backend/README.md index 849da24..663260d 100644 --- a/backend/README.md +++ b/backend/README.md @@ -50,7 +50,8 @@ each a `kind='robot'` identity, provisioned at startup with chat and friend requests blocked — backs human-like, per-language composed names. A background driver plays the robot's moves through the public game API as an ordinary seated player (so only `internal/engine` imports the solver): it decides once per game whether to play to -win (≈ 40%), targets a small score margin, and times its moves with a move-number-aware +win (≈ 40%), targets a small score margin — with an occasional off-strategy move that tapers to +none as the bag empties — and times its moves with a move-number-aware right-skewed delay (quick openings, long endgames), a night-sleep window anchored to the opponent's timezone, and nudge behaviour — all derived deterministically from the game seed, so it keeps no extra state. A background **reaper** seats a pooled robot (matching the game's language) in any open diff --git a/backend/internal/robot/driver.go b/backend/internal/robot/driver.go index 711d73c..edac356 100644 --- a/backend/internal/robot/driver.go +++ b/backend/internal/robot/driver.go @@ -67,7 +67,8 @@ func (s *Service) handle(ctx context.Context, rt game.RobotTurn, now time.Time) // Honest-AI game: the robot moves the instant it is its turn — no sleep window and // no proactive nudge (chat and nudge are disabled in these games, and the player // chose an opponent that "moves at once"). It still plays to the same per-game - // strength (playToWin) and margin band as the human-mimicry path. + // strength (playToWin), margin band and occasional off-strategy deviation as the + // human-mimicry path. if rt.VsAI { if rt.ToMove == rt.RobotSeat { return s.act(ctx, rt, now) @@ -164,9 +165,11 @@ func (s *Service) maybeNudge(ctx context.Context, rt game.RobotTurn, now time.Ti return nil } -// act reads the live turn, chooses a move by margin and submits it. State that -// has moved on since the scan (a finished game, a turn that is no longer the -// robot's) surfaces as a benign error and is skipped. +// act reads the live turn, chooses a move by margin — usually toward the robot's +// per-game intent, but with an occasional off-strategy deviation that fades to none +// as the bag empties — and submits it. State that has moved on since the scan (a +// finished game, a turn that is no longer the robot's) surfaces as a benign error +// and is skipped. func (s *Service) act(ctx context.Context, rt game.RobotTurn, now time.Time) error { st, err := s.games.GameState(ctx, rt.GameID, rt.RobotID) if err != nil { @@ -179,7 +182,11 @@ func (s *Service) act(ctx context.Context, rt game.RobotTurn, now time.Time) err myScore := st.Game.Seats[st.Seat].Score oppScore := bestOpponentScore(st.Game.Seats, st.Seat) - d := selectMove(cands, myScore, oppScore, playToWin(rt.Seed), defaultBand, st.Rack, st.BagLen) + win := playToWin(rt.Seed) + if deviates(rt.Seed, rt.MoveCount, st.BagLen) { + win = !win // an occasional off-strategy move; never once the bag is empty + } + d := selectMove(cands, myScore, oppScore, win, defaultBand, st.Rack, st.BagLen) var res game.MoveResult switch d.kind { diff --git a/backend/internal/robot/strategy.go b/backend/internal/robot/strategy.go index 5ac917a..a6c9736 100644 --- a/backend/internal/robot/strategy.go +++ b/backend/internal/robot/strategy.go @@ -23,6 +23,15 @@ const ( // human wins about 60% of games (docs/ARCHITECTURE.md §7). playToWinPercent = 40 + // The robot occasionally plays a single move against its per-game win/lose + // intent (an off-strategy "wobble"), so the chosen strategy may not pan out — + // which favours the human. deviateMaxProb is the peak probability of that, held + // through the opening and midgame; it tapers linearly to 0 over the last + // deviateTaperTiles tiles left in the bag, reaching 0 once the bag is empty, so + // the endgame follows the chosen strategy strictly (docs/ARCHITECTURE.md §7). + deviateMaxProb = 0.20 + deviateTaperTiles = 14 + // The robot's think time depends on how far the game has progressed: early moves // are quick and late moves can be long (endgame deliberation). The delay is drawn // from a band that interpolates with the move count from [delayEarlyLoMinutes, @@ -129,6 +138,34 @@ func PlayToWin(seed int64) bool { return playToWin(seed) } // win in any given game (the admin console shows it alongside the per-game decision). const PlayToWinTargetPercent = playToWinPercent +// deviateProb is the probability that the robot plays a single move against its +// per-game win/lose intent, given the number of tiles left in the bag. It is +// deviateMaxProb through the opening and midgame, then tapers linearly to 0 over +// the last deviateTaperTiles tiles, reaching 0 once the bag is empty so the endgame +// follows the chosen strategy strictly. +func deviateProb(bagLen int) float64 { + switch { + case bagLen <= 0: + return 0 + case bagLen >= deviateTaperTiles: + return deviateMaxProb + default: + return deviateMaxProb * float64(bagLen) / float64(deviateTaperTiles) + } +} + +// deviates reports whether the robot deviates from its per-game win/lose intent on +// the move at moveCount: a deterministic per-turn draw (mix/unitFloat, like the +// think-time sampling) against deviateProb(bagLen), so it is reproducible across +// restarts and never fires once the bag is empty. +func deviates(seed int64, moveCount, bagLen int) bool { + p := deviateProb(bagLen) + if p <= 0 { + return false + } + return unitFloat(mix(seed, "deviate", moveCount)) < p +} + // NextMoveAt is the deterministic instant the robot is scheduled to play the move at // moveCount, given when the turn started and the opponent's timezone (which anchors the // robot's sleep window). It is the sampled think-time delay, deferred to the end of the diff --git a/backend/internal/robot/strategy_test.go b/backend/internal/robot/strategy_test.go index f1d2cbe..84ff7fd 100644 --- a/backend/internal/robot/strategy_test.go +++ b/backend/internal/robot/strategy_test.go @@ -1,6 +1,7 @@ package robot import ( + "math" "sort" "testing" "time" @@ -238,6 +239,77 @@ func TestPlayToWinExport(t *testing.T) { } } +// TestDeviateProbTaper checks the deviation probability is deviateMaxProb while the +// bag holds at least deviateTaperTiles tiles, halves at the taper midpoint, is 0 +// once the bag is empty, and stays within [0, deviateMaxProb] and non-decreasing. +func TestDeviateProbTaper(t *testing.T) { + if p := deviateProb(0); p != 0 { + t.Errorf("deviateProb(0) = %v, want 0 (strict endgame)", p) + } + if p := deviateProb(deviateTaperTiles); p != deviateMaxProb { + t.Errorf("deviateProb(%d) = %v, want %v", deviateTaperTiles, p, deviateMaxProb) + } + if p := deviateProb(deviateTaperTiles + 50); p != deviateMaxProb { + t.Errorf("deviateProb above the taper = %v, want %v (capped)", p, deviateMaxProb) + } + if p := deviateProb(deviateTaperTiles / 2); math.Abs(p-deviateMaxProb/2) > 1e-9 { + t.Errorf("deviateProb at half taper = %v, want ~%v", p, deviateMaxProb/2) + } + prev := -1.0 + for bag := 0; bag <= deviateTaperTiles+5; bag++ { + p := deviateProb(bag) + if p < 0 || p > deviateMaxProb { + t.Fatalf("deviateProb(%d) = %v out of [0,%v]", bag, p, deviateMaxProb) + } + if p < prev { + t.Fatalf("deviateProb not non-decreasing: bag %d gives %v after %v", bag, p, prev) + } + prev = p + } +} + +// TestDeviatesNeverInEndgame checks the robot never deviates once the bag is empty, +// for every seed and move count, so the endgame follows the chosen strategy strictly. +func TestDeviatesNeverInEndgame(t *testing.T) { + for seed := int64(1); seed <= 5000; seed++ { + for mc := 0; mc < 40; mc++ { + if deviates(seed, mc, 0) { + t.Fatalf("deviates with an empty bag for seed=%d mc=%d", seed, mc) + } + } + } +} + +// TestDeviatesDeterministic checks the per-turn deviation draw is reproducible for a +// (seed, moveCount, bagLen), so the driver recomputes the same decision on every scan. +func TestDeviatesDeterministic(t *testing.T) { + for seed := int64(1); seed <= 500; seed++ { + for mc := 0; mc < 30; mc++ { + got := deviates(seed, mc, deviateTaperTiles) + if deviates(seed, mc, deviateTaperTiles) != got { + t.Fatalf("deviates not deterministic for seed=%d mc=%d", seed, mc) + } + } + } +} + +// TestDeviatesDistribution checks the deviation rate over many games lands near +// deviateMaxProb while the bag is full (above the taper), at a fixed move count. +func TestDeviatesDistribution(t *testing.T) { + const n = 20000 + hits := 0 + for seed := int64(1); seed <= n; seed++ { + if deviates(seed, 3, deviateTaperTiles+20) { + hits++ + } + } + pct := float64(hits) / float64(n) * 100 + want := deviateMaxProb * 100 + if pct < want-2 || pct > want+2 { + t.Errorf("deviation rate = %.1f%%, want ~%.0f%% (±2)", pct, want) + } +} + // TestProactiveNudgeGap checks the proactive-nudge schedule: the first gap (refIdle 0) is // ~60-90 min, every gap stays within [60 min, 6 h] and is deterministic, and the gap lengthens // as the idle grows (the median at 12 h idle exceeds the median at the start). diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index aede476..4b6fab0 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -407,7 +407,14 @@ English game the Latin pool. (`engine.Candidates`) the move whose resulting lead (playing to win) or deficit (playing to lose) is closest to a small band (**1–30 points**), rather than always the maximum; with no legal play it exchanges a full rack when the bag can - refill it, else passes. + refill it, else passes. On **≈20%** of moves through the opening and midgame it + **deviates** — playing that single move toward the *opposite* band (a winning + robot eases off, a losing one surges ahead), so the chosen strategy may not pan + out, which favours the human; the deviation chance tapers linearly to **0 over + the last 14 tiles in the bag** and is **0 once the bag is empty**, so the endgame + follows the per-game intent strictly. It is **deterministic from the seed** + (`mix(seed,"deviate",moveCount)`), a per-move wobble that leaves the per-game + play-to-win intent (and the admin card) unchanged. - **Timing**: the per-move delay is **move-number-aware** — a right-skewed sample (exponent k=4, short delays frequent) from a band that interpolates from **[3, 10] min** at the first move to **[10, 90] min** by ~28 moves, so openings are diff --git a/docs/FUNCTIONAL.md b/docs/FUNCTIONAL.md index 05c78b3..f83986f 100644 --- a/docs/FUNCTIONAL.md +++ b/docs/FUNCTIONAL.md @@ -141,8 +141,10 @@ When auto-match finds no human within the wait window (1.5–3 minutes), a robot takes the empty seat of the game you are already waiting in. It is meant to feel like a person: it decides once per game whether to play to win (about 40% of the time, so the human wins most games), aims for a close score rather than crushing or throwing the game, -and plays at a human pace — short thinking times for most moves, the occasional long -one, and a night-time pause that tracks the player's own day. It answers a nudge +now and then plays a single move against that plan — a surprise lead or a slack move — +yet holds to the plan once the bag empties, and plays at a human pace — short thinking +times for most moves, the occasional long one, and a night-time pause that tracks the +player's own day. It answers a nudge within a few minutes and nudges back when the player has been away a long time. It carries a human-like, language-appropriate name (a Russian game draws mostly Russian names); it does not chat, and **silently ignores friend requests** — a request to a @@ -150,8 +152,9 @@ robot stays pending and expires, exactly like a human who never responds. The same robot also backs the **honest-AI quick game** the player chooses directly (above). There it makes no pretence: it is shown as **🤖** everywhere, joins and moves at once (no thinking time -or night pause), keeps the same strength (it still plays to win only about 40% of the time), and -chat, nudge and add-friend are off. AI games are **practice** — they never count toward a player's +or night pause), keeps the same strength (it still plays to win only about 40% of the time, with +the same occasional move against its plan that fades out by the endgame), and chat, nudge and +add-friend are off. AI games are **practice** — they never count toward a player's statistics. ### Social: friends, block, chat, nudge diff --git a/docs/FUNCTIONAL_ru.md b/docs/FUNCTIONAL_ru.md index d77f430..060b2c4 100644 --- a/docs/FUNCTIONAL_ru.md +++ b/docs/FUNCTIONAL_ru.md @@ -146,8 +146,10 @@ nudge) приходят от бота **этой партии** — по язы в которой вы уже ждёте, занимает робот-соперник. Он задуман неотличимым от человека: один раз за партию решает, играть ли на победу (примерно в 40% случаев, так что человек выигрывает большинство партий), целится в близкий счёт, а не в разгром или -поддавки, и ходит с человеческим темпом — чаще короткие раздумья, изредка долгие, и -ночная пауза, подстроенная под день игрока. На nudge отвечает за несколько минут и +поддавки, время от времени делает один ход вопреки этому плану — неожиданный отрыв или +слабый ход, — но к концу партии (когда мешок пуст) держится плана, и ходит с +человеческим темпом — чаще короткие раздумья, изредка долгие, и ночная пауза, +подстроенная под день игрока. На nudge отвечает за несколько минут и сам шлёт nudge, когда игрок надолго пропал. Носит человекоподобное имя, подходящее языку партии (в русской партии — в основном русские имена); не общается в чате и **молча игнорирует заявки в друзья** — заявка роботу остаётся в ожидании и истекает, @@ -155,8 +157,8 @@ nudge) приходят от бота **этой партии** — по язы Тот же робот стоит и за **честной игрой против ИИ**, которую игрок выбирает напрямую (выше). Там он не притворяется: везде показан как **🤖**, присоединяется и ходит сразу (без раздумий и ночной -паузы), сохраняет ту же силу (по-прежнему играет на победу лишь примерно в 40% партий), а чат, nudge -и «добавить в друзья» выключены. Партии с ИИ — это **тренировка**: они не идут в статистику игрока. +паузы), сохраняет ту же силу (по-прежнему играет на победу лишь примерно в 40% партий, с теми же +редкими ходами вопреки плану, затухающими к эндшпилю), а чат, nudge и «добавить в друзья» выключены. Партии с ИИ — это **тренировка**: они не идут в статистику игрока. ### Социальное: друзья, блок, чат, nudge Подружиться можно двумя способами: погасить **одноразовый код**, который выпускает