feat(robot): occasional off-strategy deviation, strict in the endgame #69

Merged
developer merged 1 commits from feature/ai-strategy-deviation into development 2026-06-15 19:49:28 +00:00
9 changed files with 156 additions and 16 deletions
Showing only changes of commit 3bceafbc12 - Show all commits
+4 -1
View File
@@ -648,7 +648,10 @@ cannot submit; three-way admin filter.
- **Margin** (interview): pick the candidate whose resulting margin (own+moveopp)
is closest to **[1,30]** when playing to win / **[30,1]** when playing to lose,
tie-broken toward the conservative edge; no legal play → exchange the full rack
when the bag can refill it, else pass.
when the bag can refill it, else pass. *(Later refined — separate PR: on ≈20% of
opening/midgame moves the robot flips its intent for that one move, the chance
tapering to 0 over the last 14 bag tiles and 0 at an empty bag, so the endgame
stays strict; deterministic from the seed.)*
- **Substitution** (interview): a matchmaker **reaper** (`Reap`/`RunReaper`)
substitutes a pooled robot after a **10 s** wait (`BACKEND_LOBBY_ROBOT_WAIT`),
`NewMatchmaker` now takes a `RobotProvider`. A waiter learns of a match — human
+8
View File
@@ -511,3 +511,11 @@ Then Stage 18.
(`GameRow`/`GameDetailView` gain `VsAI`); (4) `games_started_total` / `games_abandoned_total` gain a
**`vs_ai`** attribute and the Grafana *Game domain* dashboard splits started/abandoned into **human**
and **AI** panels.
- **Follow-up (separate PR — strategy deviation):** the robot now plays **≈20%** of opening/midgame moves
*against* its per-game `playToWin` intent (toward the opposite margin band — a winning robot eases off, a
losing one surges ahead), tapering linearly to **0 over the last 14 bag tiles** and **0 once the bag is
empty**, so the endgame follows the chosen strategy strictly while earlier outcomes can swing the human's
way. Deterministic from the seed (`mix(seed,"deviate",moveCount)`), applied to **both** robot paths via
the shared `selectMove`; the per-game intent (and the admin card) is unchanged. Tests: `robot` unit
(taper bounds + monotonicity, never-in-endgame, determinism, ~20% distribution). Bake-back:
`docs/ARCHITECTURE.md` §7, `docs/FUNCTIONAL.md` (+`_ru`), `backend/README.md`, `PLAN.md` Stage 5.
+2 -1
View File
@@ -50,7 +50,8 @@ each a `kind='robot'` identity, provisioned at startup with chat and friend
requests blocked — backs human-like, per-language composed names. A background driver plays the
robot's moves through the public game API as an ordinary seated player (so only
`internal/engine` imports the solver): it decides once per game whether to play to
win (≈ 40%), targets a small score margin, and times its moves with a move-number-aware
win (≈ 40%), targets a small score margin — with an occasional off-strategy move that tapers to
none as the bag empties — and times its moves with a move-number-aware
right-skewed delay (quick openings, long endgames), a night-sleep window anchored to the opponent's timezone, and nudge
behaviour — all derived deterministically from the game seed, so it keeps no extra
state. A background **reaper** seats a pooled robot (matching the game's language) in any open
+12 -5
View File
@@ -67,7 +67,8 @@ func (s *Service) handle(ctx context.Context, rt game.RobotTurn, now time.Time)
// Honest-AI game: the robot moves the instant it is its turn — no sleep window and
// no proactive nudge (chat and nudge are disabled in these games, and the player
// chose an opponent that "moves at once"). It still plays to the same per-game
// strength (playToWin) and margin band as the human-mimicry path.
// strength (playToWin), margin band and occasional off-strategy deviation as the
// human-mimicry path.
if rt.VsAI {
if rt.ToMove == rt.RobotSeat {
return s.act(ctx, rt, now)
@@ -164,9 +165,11 @@ func (s *Service) maybeNudge(ctx context.Context, rt game.RobotTurn, now time.Ti
return nil
}
// act reads the live turn, chooses a move by margin and submits it. State that
// has moved on since the scan (a finished game, a turn that is no longer the
// robot's) surfaces as a benign error and is skipped.
// act reads the live turn, chooses a move by margin — usually toward the robot's
// per-game intent, but with an occasional off-strategy deviation that fades to none
// as the bag empties — and submits it. State that has moved on since the scan (a
// finished game, a turn that is no longer the robot's) surfaces as a benign error
// and is skipped.
func (s *Service) act(ctx context.Context, rt game.RobotTurn, now time.Time) error {
st, err := s.games.GameState(ctx, rt.GameID, rt.RobotID)
if err != nil {
@@ -179,7 +182,11 @@ func (s *Service) act(ctx context.Context, rt game.RobotTurn, now time.Time) err
myScore := st.Game.Seats[st.Seat].Score
oppScore := bestOpponentScore(st.Game.Seats, st.Seat)
d := selectMove(cands, myScore, oppScore, playToWin(rt.Seed), defaultBand, st.Rack, st.BagLen)
win := playToWin(rt.Seed)
if deviates(rt.Seed, rt.MoveCount, st.BagLen) {
win = !win // an occasional off-strategy move; never once the bag is empty
}
d := selectMove(cands, myScore, oppScore, win, defaultBand, st.Rack, st.BagLen)
var res game.MoveResult
switch d.kind {
+37
View File
@@ -23,6 +23,15 @@ const (
// human wins about 60% of games (docs/ARCHITECTURE.md §7).
playToWinPercent = 40
// The robot occasionally plays a single move against its per-game win/lose
// intent (an off-strategy "wobble"), so the chosen strategy may not pan out —
// which favours the human. deviateMaxProb is the peak probability of that, held
// through the opening and midgame; it tapers linearly to 0 over the last
// deviateTaperTiles tiles left in the bag, reaching 0 once the bag is empty, so
// the endgame follows the chosen strategy strictly (docs/ARCHITECTURE.md §7).
deviateMaxProb = 0.20
deviateTaperTiles = 14
// The robot's think time depends on how far the game has progressed: early moves
// are quick and late moves can be long (endgame deliberation). The delay is drawn
// from a band that interpolates with the move count from [delayEarlyLoMinutes,
@@ -129,6 +138,34 @@ func PlayToWin(seed int64) bool { return playToWin(seed) }
// win in any given game (the admin console shows it alongside the per-game decision).
const PlayToWinTargetPercent = playToWinPercent
// deviateProb is the probability that the robot plays a single move against its
// per-game win/lose intent, given the number of tiles left in the bag. It is
// deviateMaxProb through the opening and midgame, then tapers linearly to 0 over
// the last deviateTaperTiles tiles, reaching 0 once the bag is empty so the endgame
// follows the chosen strategy strictly.
func deviateProb(bagLen int) float64 {
switch {
case bagLen <= 0:
return 0
case bagLen >= deviateTaperTiles:
return deviateMaxProb
default:
return deviateMaxProb * float64(bagLen) / float64(deviateTaperTiles)
}
}
// deviates reports whether the robot deviates from its per-game win/lose intent on
// the move at moveCount: a deterministic per-turn draw (mix/unitFloat, like the
// think-time sampling) against deviateProb(bagLen), so it is reproducible across
// restarts and never fires once the bag is empty.
func deviates(seed int64, moveCount, bagLen int) bool {
p := deviateProb(bagLen)
if p <= 0 {
return false
}
return unitFloat(mix(seed, "deviate", moveCount)) < p
}
// NextMoveAt is the deterministic instant the robot is scheduled to play the move at
// moveCount, given when the turn started and the opponent's timezone (which anchors the
// robot's sleep window). It is the sampled think-time delay, deferred to the end of the
+72
View File
@@ -1,6 +1,7 @@
package robot
import (
"math"
"sort"
"testing"
"time"
@@ -238,6 +239,77 @@ func TestPlayToWinExport(t *testing.T) {
}
}
// TestDeviateProbTaper checks the deviation probability is deviateMaxProb while the
// bag holds at least deviateTaperTiles tiles, halves at the taper midpoint, is 0
// once the bag is empty, and stays within [0, deviateMaxProb] and non-decreasing.
func TestDeviateProbTaper(t *testing.T) {
if p := deviateProb(0); p != 0 {
t.Errorf("deviateProb(0) = %v, want 0 (strict endgame)", p)
}
if p := deviateProb(deviateTaperTiles); p != deviateMaxProb {
t.Errorf("deviateProb(%d) = %v, want %v", deviateTaperTiles, p, deviateMaxProb)
}
if p := deviateProb(deviateTaperTiles + 50); p != deviateMaxProb {
t.Errorf("deviateProb above the taper = %v, want %v (capped)", p, deviateMaxProb)
}
if p := deviateProb(deviateTaperTiles / 2); math.Abs(p-deviateMaxProb/2) > 1e-9 {
t.Errorf("deviateProb at half taper = %v, want ~%v", p, deviateMaxProb/2)
}
prev := -1.0
for bag := 0; bag <= deviateTaperTiles+5; bag++ {
p := deviateProb(bag)
if p < 0 || p > deviateMaxProb {
t.Fatalf("deviateProb(%d) = %v out of [0,%v]", bag, p, deviateMaxProb)
}
if p < prev {
t.Fatalf("deviateProb not non-decreasing: bag %d gives %v after %v", bag, p, prev)
}
prev = p
}
}
// TestDeviatesNeverInEndgame checks the robot never deviates once the bag is empty,
// for every seed and move count, so the endgame follows the chosen strategy strictly.
func TestDeviatesNeverInEndgame(t *testing.T) {
for seed := int64(1); seed <= 5000; seed++ {
for mc := 0; mc < 40; mc++ {
if deviates(seed, mc, 0) {
t.Fatalf("deviates with an empty bag for seed=%d mc=%d", seed, mc)
}
}
}
}
// TestDeviatesDeterministic checks the per-turn deviation draw is reproducible for a
// (seed, moveCount, bagLen), so the driver recomputes the same decision on every scan.
func TestDeviatesDeterministic(t *testing.T) {
for seed := int64(1); seed <= 500; seed++ {
for mc := 0; mc < 30; mc++ {
got := deviates(seed, mc, deviateTaperTiles)
if deviates(seed, mc, deviateTaperTiles) != got {
t.Fatalf("deviates not deterministic for seed=%d mc=%d", seed, mc)
}
}
}
}
// TestDeviatesDistribution checks the deviation rate over many games lands near
// deviateMaxProb while the bag is full (above the taper), at a fixed move count.
func TestDeviatesDistribution(t *testing.T) {
const n = 20000
hits := 0
for seed := int64(1); seed <= n; seed++ {
if deviates(seed, 3, deviateTaperTiles+20) {
hits++
}
}
pct := float64(hits) / float64(n) * 100
want := deviateMaxProb * 100
if pct < want-2 || pct > want+2 {
t.Errorf("deviation rate = %.1f%%, want ~%.0f%% (±2)", pct, want)
}
}
// TestProactiveNudgeGap checks the proactive-nudge schedule: the first gap (refIdle 0) is
// ~60-90 min, every gap stays within [60 min, 6 h] and is deterministic, and the gap lengthens
// as the idle grows (the median at 12 h idle exceeds the median at the start).
+8 -1
View File
@@ -407,7 +407,14 @@ English game the Latin pool.
(`engine.Candidates`) the move whose resulting lead (playing to win) or deficit
(playing to lose) is closest to a small band (**130 points**), rather than
always the maximum; with no legal play it exchanges a full rack when the bag can
refill it, else passes.
refill it, else passes. On **≈20%** of moves through the opening and midgame it
**deviates** — playing that single move toward the *opposite* band (a winning
robot eases off, a losing one surges ahead), so the chosen strategy may not pan
out, which favours the human; the deviation chance tapers linearly to **0 over
the last 14 tiles in the bag** and is **0 once the bag is empty**, so the endgame
follows the per-game intent strictly. It is **deterministic from the seed**
(`mix(seed,"deviate",moveCount)`), a per-move wobble that leaves the per-game
play-to-win intent (and the admin card) unchanged.
- **Timing**: the per-move delay is **move-number-aware** — a right-skewed sample
(exponent k=4, short delays frequent) from a band that interpolates from
**[3, 10] min** at the first move to **[10, 90] min** by ~28 moves, so openings are
+7 -4
View File
@@ -141,8 +141,10 @@ When auto-match finds no human within the wait window (1.53 minutes), a robot
takes the empty seat of the game you are already waiting in. It is meant to feel like a person: it
decides once per game whether to play to win (about 40% of the time, so the human
wins most games), aims for a close score rather than crushing or throwing the game,
and plays at a human pace — short thinking times for most moves, the occasional long
one, and a night-time pause that tracks the player's own day. It answers a nudge
now and then plays a single move against that plan — a surprise lead or a slack move —
yet holds to the plan once the bag empties, and plays at a human pace — short thinking
times for most moves, the occasional long one, and a night-time pause that tracks the
player's own day. It answers a nudge
within a few minutes and nudges back when the player has been away a long time. It
carries a human-like, language-appropriate name (a Russian game draws mostly Russian
names); it does not chat, and **silently ignores friend requests** — a request to a
@@ -150,8 +152,9 @@ robot stays pending and expires, exactly like a human who never responds.
The same robot also backs the **honest-AI quick game** the player chooses directly (above). There
it makes no pretence: it is shown as **🤖** everywhere, joins and moves at once (no thinking time
or night pause), keeps the same strength (it still plays to win only about 40% of the time), and
chat, nudge and add-friend are off. AI games are **practice** — they never count toward a player's
or night pause), keeps the same strength (it still plays to win only about 40% of the time, with
the same occasional move against its plan that fades out by the endgame), and chat, nudge and
add-friend are off. AI games are **practice** — they never count toward a player's
statistics.
### Social: friends, block, chat, nudge
+6 -4
View File
@@ -146,8 +146,10 @@ nudge) приходят от бота **этой партии** — по язы
в которой вы уже ждёте, занимает робот-соперник. Он задуман неотличимым от человека:
один раз за партию решает, играть ли на победу (примерно в 40% случаев, так что
человек выигрывает большинство партий), целится в близкий счёт, а не в разгром или
поддавки, и ходит с человеческим темпом — чаще короткие раздумья, изредка долгие, и
ночная пауза, подстроенная под день игрока. На nudge отвечает за несколько минут и
поддавки, время от времени делает один ход вопреки этому плану — неожиданный отрыв или
слабый ход, — но к концу партии (когда мешок пуст) держится плана, и ходит с
человеческим темпом — чаще короткие раздумья, изредка долгие, и ночная пауза,
подстроенная под день игрока. На nudge отвечает за несколько минут и
сам шлёт nudge, когда игрок надолго пропал. Носит человекоподобное имя, подходящее
языку партии (в русской партии — в основном русские имена); не общается в чате и
**молча игнорирует заявки в друзья** — заявка роботу остаётся в ожидании и истекает,
@@ -155,8 +157,8 @@ nudge) приходят от бота **этой партии** — по язы
Тот же робот стоит и за **честной игрой против ИИ**, которую игрок выбирает напрямую (выше). Там он
не притворяется: везде показан как **🤖**, присоединяется и ходит сразу (без раздумий и ночной
паузы), сохраняет ту же силу (по-прежнему играет на победу лишь примерно в 40% партий), а чат, nudge
и «добавить в друзья» выключены. Партии с ИИ — это **тренировка**: они не идут в статистику игрока.
паузы), сохраняет ту же силу (по-прежнему играет на победу лишь примерно в 40% партий, с теми же
редкими ходами вопреки плану, затухающими к эндшпилю), а чат, nudge и «добавить в друзья» выключены. Партии с ИИ — это **тренировка**: они не идут в статистику игрока.
### Социальное: друзья, блок, чат, nudge
Подружиться можно двумя способами: погасить **одноразовый код**, который выпускает