Commit Graph

3 Commits

Author SHA1 Message Date
Ilia Denisov 74ff3aefb6 feat(ru): drop abbreviations from the Russian noun list
build / dawg (pull_request) Successful in 6m4s
The list leaked abbreviations two ways: an OpenCorpora noun tagged Abbr without
Fixd (the declinable acronym ОГВЗ) slipped past the Abbr+Fixd seed filter, and the
РАН «(сокр.)» words (вуз, нэп, колхоз, гост, …) were kept on purpose by the old
policy.

Drop every abbreviation instead. A final pass in ru_stage2.py removes a word when
the orthographic note marks it «(сокр.)» (is_abbrev_note) or when every OpenCorpora
noun reading carries Abbr (oc_abbr_only — also the declinable огвз/заз the seed
filter missed). Homographs with a plain-noun reading survive (ваза «ВАЗ», рис).
Lexicalised words are rescued via manual_confirm.txt (колхоз, совхоз, рация,
спецназ, эсминец, дзот, дот, прораб, токамак, тэн, … and под — a real noun OC knows
only as ПОД); the bound prefix гос is vetoed in manual_reject.txt.

108 words removed from ru_scrabble / ru_erudit; en_sowpods unchanged. tools/README
documents the policy and that libmorph is required for a faithful rebuild.
2026-06-22 14:16:55 +02:00
Ilia Denisov 01821afc9a Reject 9 more adjective-primary false nouns (consistency pass)
build / dawg (pull_request) Successful in 6m52s
Follow-up to v1.2.0: reviewing the kept low-confidence noun verdicts surfaced
adjective-primary words admitted only by colloquial ellipsis, while their twins
were already rejected — notably большой (kept) vs маленький (dropped).

Adds to manual_reject.txt: большой, старшенький, горбатый, городской, одинокий,
вольный, каторжный, припадочный, сподручный. Their real nouns are different
words (горбун, горожанин, одиночка, каторжник) or the sense is marginal/устар.

scrabble.txt -9 (83144->83135); erudit.txt re-folded.
2026-06-20 17:39:09 +02:00
Ilia Denisov e17e945b41 Drop substantivized-adjective false nouns via manual_reject.txt
build / dawg (pull_request) Successful in 1m33s
OpenCorpora/libmorph hand a noun reading to Stage 2 above the РАН note, so
substantivized-adjective false nouns a dictionary misreads (нёбный, акцизный,
велярный, …) reached scrabble.txt with no veto path — the earlier abbreviation
filter only carved out Abbr+Fixd.

ru_stage2.py now subtracts sources/scrabble_ru/manual_reject.txt last, after
every admission path (OC seed / libmorph / note / manual_confirm / variant),
mirroring manual_confirm.txt. The list seeds 62 hand-reviewed words — 43 pure
adjectives plus 19 marginal (slang/archaic/jargon) substantivizations; genuine
substantivized nouns (больной, знакомый, учёный, участковый, …) stay.

scrabble.txt -62 (83206->83144); erudit.txt re-folded -62. The DAWGs are
gitignored and rebuild from these lists in CI.
2026-06-20 17:09:02 +02:00