Home /Modules/languages/FRUCHST_STATUS.md

FRUCHST_STATUS.md

languages/FRUCHST_STATUS.md · 153 line(s) · markdown

FRUCHST status: words and English/French/Spanish definitions

Snapshot as of 2026-11-16. Counts come from the committed data in datasets/frequency-candidates/, datasets/definition-review/ and datasets/definition-pivot/. No definition is approved yet, so no fruchst.<code>.bin has been built.

Summary

MeasureCount
Profiles with a 1,000-word list51 of 80 (51,000 of 80,000 words)
Words with an English definition28,111
Words without English that have a close-variant English definition5,277 of 22,889 (23.1%)
Words with English, own or close variant33,388 (65.5%; 64.8% without accent-stripped matches)
Words with a French definition or equivalent25,010
Words with a Spanish definition or equivalent22,485
Words with all three (direct Wiktionary only)4,462
Words with all three (direct or through English)22,344
Of those, part of speech matched21,357
Approved definitions0
Profiles with all 1,000 words fully defined0

A word counts as having all three when it has an English Wiktionary definition and a French and a Spanish definition or equivalent. French and Spanish come either directly from French/Spanish Wiktionary or through English: the English word from the definition is looked up in English Wiktionary translation tables and in French/Spanish Wiktionary (tools/fruchst_english_pivot.py, see datasets/frequency-candidates/DEFINITION-PIPELINE.md). "Part of speech matched" means the English word was found with the same part of speech as the original sense.

Close-variant English definitions

tools/fruchst_close_variants.py (output in datasets/definition-variants/) finds an English definition for words that have none by matching a close variant in English Wiktionary. Counts by closest method:

MethodWords
Inflected form listed in another entry's inflection table3,922
Case variant490
Orthographic variant (optional marks only)424
Accent/tone marks removed (flagged with a caution)328
Uzbek Cyrillic-to-Latin transliteration111
Arabic clitic removed2

Highest gains: Arabic 523 of 569 words without English, Uzbek 505 of 961, Persian 335 of 486, Mandarin 291 of 492. The per-profile table below still counts only each word's own English definition. Close definitions are not yet passed through the French/Spanish English pivot.

English round

tools/fruchst_english_round.py (output in datasets/definition-english/, one <stem>.english.json per profile) collects every English definition that English Wiktionary supplies for each headword and ranks them. The review-queue glosses are unchanged, and every review sense appears in the English file. Run of 2026-11-16 on the 51 staged profiles (51,000 words):

MeasureCount
Words with any English definition34,242 (67.1%), up from 28,018 with their own English sense
Words gaining an English definition6,224
Words whose best definition is their own substantive sense23,283
Words whose best definition is a resolved form-of lemma ("plural of mtu" → "person")4,495
Words with only a pointer gloss whose lemma is not in Wiktionary240
Words whose best definition is a close variant5,277
Words whose only English is a translation-table equivalent947
Pointer senses resolved to lemma definitions / unresolved11,503 / 993
Words whose lead candidate is corroborated by a translation table15,657
Words with usage examples translated into English12,754
Words with a romanization12,356
Glosses with wiki markup cleaned for display (original kept)431
Words still without any English definition16,758
Approved definitions0

Largest gains in words with English: Uzbek 39 → 621, Arabic 431 → 959, Persian 514 → 866, Mandarin 508 → 810, Kannada 146 → 373, Yoruba 356 → 610, German 716 → 980. Form-of resolution matters most where the existing English was mostly pointers: Polish has 523 words, Russian 481, Hindi 351, Portuguese 317 and Romanian 304 whose best definition is now the lemma's meaning.

Known limits: the lead is a ranked suggestion, not a selection. Homograph lemmas can resolve to the wrong entry. For example, Swahili ya resolves to the verb suffix -a instead of the possessive -a ("of"). Function words with many senses still need a reviewer to choose the sense the frequency list means. The lowest-coverage profiles (Magahi 15, Nigerian Pidgin 44, Maithili 102, Sinhala 150) have little English Wiktionary content, so they need another licensed source.

Per profile

RankProfileTagWordsENFRESAll 3All 3, POS matchedWord list / blocker
1englishen1,0001000970962960884wordhoard-en.json
2mandarin-chinesezh-Hans1,000508737492492278frequencywords-zh-Hans.json
3hindihi1,000911578535533518frequencywords-hi.json
4spanishes1,000935924931920915wordhoard-es.json
5standard-arabicar1,000431374371366357frequencywords-ar.json
6frenchfr1,000904910880878876wordhoard-fr.json
7bengalibn1,000540443443441398frequencywords-bn.json
8portuguesept1,000973950669668663frequencywords-pt.json
9russianru1,000983698475473453frequencywords-ru.json
10indonesianid1,000820802719717697frequencywords-id.json
11urduur1,000597494494493478frequencywords-ur.json
12standard-germande1,000716723680677670wordhoard-de.json
13japaneseja1,000896901836834772frequencywords-ja.json
14nigerian-pidginpcm1,0004135333333unilex-pcm.json
15egyptian-arabicarz——————No licensed frequency source found
16marathimr1,000403358357356337unilex-mr.json
17vietnamesevi1,000977940887887874frequencywords-vi.json
18telugute1,000399254261254243frequencywords-te.json
19hausaha1,000208343192182172unilex-ha.json
20turkishtr1,000739531494493455frequencywords-tr.json
21western-punjabipnb-Arab——————No licensed frequency source found
22swahilisw1,000846594538535521unilex-sw.json
23tagalogtl1,000626533525524511frequencywords-tl.json
24tamilta1,000481411389389372frequencywords-ta.json
25yue-chineseyue——————No licensed frequency source found
26wu-chinesewuu——————No licensed frequency source found
27iranian-persianfa1,000514456445444436frequencywords-fa.json
28koreanko1,000423379349349323frequencywords-ko.json
29thaith1,000257232225225213frequencywords-th.json
30javanesejv1,00031294979490unilex-jv.json
31italianit1,000960920655649645frequencywords-it.json
32gujaratigu1,000312279276276258unilex-gu.json
33levantine-arabicapc——————No licensed frequency source found
34amharicam1,000265258259253247unilex-am.json
35kannadakn1,000146137137137133unilex-kn.json
36bhojpuribho——————No licensed frequency source found
37sudanese-arabicapd——————No licensed frequency source found
38yorubayo1,000356335339324315unilex-yo.json
39jinyu-chinesecjy——————No licensed frequency source found
40min-nan-chinesenan——————No licensed frequency source found
41polishpl1,000981586445441411frequencywords-pl.json
42burmesemy1,000800759752752730unilex-my.json
43algerian-arabicarq——————No licensed frequency source found
44hakka-chinesehak——————No licensed frequency source found
45lingalaln——————No licensed frequency source found
46moroccan-arabicary——————No licensed frequency source found
47odiaor1,000209204206204188unilex-or.json
48ukrainianuk1,000642523328327312frequencywords-uk.json
49xiang-chinesehsn——————No licensed frequency source found
50malayalamml1,000351304301301269frequencywords-ml.json
51sindhisd——————No licensed frequency source found
52eastern-punjabipa-Guru1,000473360362357328unilex-pa-Guru.json
53sundasu1,000231217216214208unilex-su.json
54igboig——————Too few forms (unilex: 693)
55dariprs1,000758687670670659unilex-prs.json
56nepaline——————No licensed frequency source found
57malayms1,000778743715713690frequencywords-ms.json
58northern-uzbekuz1,000392322unilex-uz.json
59zuluzu——————No licensed frequency source found
60saidi-arabicaec——————No licensed frequency source found
61northern-pashtopbu——————Only macrolanguage Pashto (ps) available
62west-central-oromogaz——————No licensed frequency source found
63saraikiskr——————No licensed frequency source found
64dutchnl1,000978891726726709frequencywords-nl.json
65somaliso1,000119112115106104unilex-so.json
66romanianro1,000814603501486472frequencywords-ro.json
67assameseas——————No licensed frequency source found
68gan-chinesegan——————No licensed frequency source found
69southern-pashtopst——————Only macrolanguage Pashto (ps) available
70cebuanoceb1,000400371360360349unilex-ceb.json
71kazakhkk1,000373374315315297frequencywords-kk.json
72magahimag1,00044444unilex-mag.json
73najdi-arabicars——————No licensed frequency source found
74sinhalasi1,0001281001009997frequencywords-si.json
75mesopotamian-arabicacm——————No licensed frequency source found
76xhosaxh——————No licensed frequency source found
77khmerkm1,000900861849847831unilex-km.json
78afrikaansaf1,000626698555547544frequencywords-af.json
79nigerian-fulfuldefuv——————Too few forms (unilex: 749)
80maithilimai1,0002818171716unilex-mai.json

Blockers