OpenCorpora tags indeclinable abbreviations as common nouns (Abbr+Fixd), and
Stage 2 seeded the result with its whole noun lexicon, so non-words leaked into
scrabble.txt: ндс, ст, ср, кпд, чп, гибдд, днк, …
ru_stage2.py now drops Abbr+Fixd nouns from the OpenCorpora seed and lets the
orthographic dictionary decide instead: a lowercase РАН headword whose note is a
noun is kept (the lexicalised сельпо, под, ска, роно, врио, тв, фио, суперэвм),
everything else is dropped. Function words (CONJ/PRCL, e.g. зато) and OpenCorpora
inflected forms (ан = род. мн. от «ана»; proper «Ан») are excluded too.
--trace WORD now prints a detailed per-signal audit (all.txt, OpenCorpora POS /
Abbr / Fixd / CONJ/PRCL, libmorph, РАН note + classify, outcome) for auditing a
word or repeating the analysis; tools/README.md documents the rule and method.
Net: 179 words removed from scrabble.txt (83385 -> 83206) and erudit.txt
(83343 -> 83164); verified to remove exactly those 179 and add none. DAWGs
rebuild clean.
The dict pipeline was moved out of scrabble-solver (256999b) into this repo,
but the initial import (d04470b) dropped three primary sources, and the
"Tidy sources" reorg (dd61ff1) git-ignored them while tools/README.md still
documented them as the committed source of truth — so the word-provenance
analysis could no longer run.
Recover them byte-identically from scrabble-solver's history (reachable from
its v1.0.0 tag):
- tools/orfo_dict_2025.pdf primary source (RAS orthographic dictionary PDF)
- sources/scrabble_ru/orfo_dict_2025.txt pdftotext output, pipeline source of truth
- sources/scrabble_ru/all.txt Stage-1 base, input to Stage 2
Stop ignoring them in .gitignore and fix the contradictory note in
sources/scrabble_ru/README.md (only the debug dumps are regenerated locally).
Consolidate the scattered build inputs (dictionaries/english/, dictprep/russian/)
into one sources/ tree keyed by the variant labels (scrabble_en/scrabble_ru/
erudit_ru), and move the Russian prep pipeline to tools/. The dawg outputs and
their filenames are unchanged — rebuilt byte-identical (en_sowpods/ru_scrabble/
ru_erudit) — so the release artifact and the backend are unaffected.
ru_stage2.py OUT_DIR and the ruwords flag defaults are repointed to
sources/scrabble_ru/; Makefile / CI / cmd/builddict default / README updated;
pipeline intermediates git-ignored. Verified: make dawg byte-identical to the
committed baseline, py_compile + go vet of the moved tools. The full Russian
regeneration pipeline (pymorphy3/libmorph/orfo PDF) was not run here.
- build.yaml dropped the release step whose ${{ github.* }} contexts failed the Gitea
workflow compile (the run produced 0 jobs); it now inlines go run (no make dependency)
and reads the committed dictprep/russian/erudit.txt (no python dependency).
- erudit.txt is scrabble.txt with Ё→Е folded (dictprep/fold_yo.py); it reproduces the
canonical ru_erudit.dawg byte-for-byte. Release artifacts are published manually for now
(see README).
- builddict drives the de-internalized scrabble-solver dictdawg/wordlist builders
(pinned v1.0.0) to produce the three DAWGs (en_sowpods, ru_scrabble, ru_erudit),
byte-identical to the solver's committed fixtures (same dafsa/alphabet v1.1.0 -> no
index drift with the running backend).
- Sources: english/sowpods.txt vendored from kamilmielnik/scrabble-dictionaries;
russian/scrabble.txt + the dictprep tooling moved out of scrabble-solver.
- CI builds the DAWGs on push/PR and, on a vX.Y.Z tag, packages them flat into
scrabble-dawg-<tag>.tar.gz and attaches it to the Gitea release.
Ilia Denisov
developer