Ilia Denisov
7f8165f16d
build / dawg (pull_request) Successful in 24s
Remove the 11 obscene nouns built on the хуй/еб·ёб/пизд/бляд roots (ахуй, блядь, выблядок, ебальник, еблан, охуение, пизда, пиздец, пиздобол, хуй, хуйня) from the Russian Scrabble/Эрудит dictionary. These are morphologically valid common nouns, so the pipeline admitted them like any other word — OpenCorpora's tagset exposes no obscenity grammeme to filter on. They are vetoed as policy via manual_reject.txt, which is subtracted last in ru_stage2.build(), so no pipeline logic changes: scrabble.txt is the deterministic result of that subtraction and erudit.txt is its Ё→Е fold, regenerated with tools/fold_yo.py. The coarser-but-not-mat муд* (мудак, мудила) and legitimate homographs (очко, сука, моча, дрочёна) are deliberately kept. scrabble.txt 83027 -> 83016, erudit.txt 82985 -> 82974.
erudit_ru source
erudit.txt — the Эрудит word list: the Ё→Е folded and de-duplicated form of
../scrabble_ru/scrabble.txt, produced by tools/fold_yo.py
(the Эрудит ruleset has no Ё tile and treats Е/Ё as one letter). Built to dawg/ru_erudit.dawg
(make dawg-erudit).