zimankit guide

zimankit 0.1.0. Everything here also works in the playground, without installing anything.

Install

zimankit needs Python 3.11 or newer. The skills use only the standard library. Download every file from the latest release.

Optional extras: [data] (mdict-utils, libzim) to read .mdx dictionaries and .zim wiki dumps, [kaggle] (pyarrow) for the Parquet export, [dev] for the tests. Each release also has a SHA256SUMS file: sha256sum -c SHA256SUMS --ignore-missing.

The wordlist pipeline, the docs and the tutorials are in the repository, not in the wheel: use the source archive or a clone for them.

Getting started

1. Check the install: zimankit --version prints zimankit 0.1.0.
2. List the commands: zimankit --help, and zimankit <command> --help for each one.
3. Try a word: zimankit analyze کتێبەکەم jinekê.
4. Pipe text in: every command that takes text reads its arguments, or standard input line by line when there are none: cat text.txt | zimankit normalize.

Without installing, run the package from a clone as python -m zimankit ….

Choosing the language

zimankit decides from the script: text in the Arabic-based script is treated as Sorani (ckb), text in Latin letters as Kurmanji (kmr). Override it with --lang ckb or --lang kmr. To find out which language a text is in, use lid.

The zimankit command

normalize, validate

Folds Arabic and Persian look-alike letters (ك ي ى ھ ة) to the Kurdish ones, repairs the legacy ه + ZWNJ spelling of ە, writes word-initial ڕ, and cleans invisible characters. In Kurmanji it folds look-alikes such as é ü ı ș to ê û i ş.

$ zimankit normalize كوردي Kurdî
کوردی
Kurdî

$ zimankit validate كتێب
كتێب	کتێب	OK

validate prints the input, the normalised word, and OK or the problems found.

tokenize

$ zimankit tokenize "کتێبەکەم، خوێندەوە."
کتێبەکەم ، خوێندەوە .

With --json each token comes with its kind (word, number, punct) and its start and end offsets.

lid

Identifies Central Kurdish (ckb), Northern Kurdish (kmr), Zazaki (zza), Gorani / Hawrami (hac), Persian (fas), Arabic (arb), Turkish (tur) and English (eng) from function words, diagnostic letters and morphology. It prints the language, the confidence and the start of the text.

$ zimankit lid "ئەمڕۆ هەوا خۆشە" "Îro hewa xweş e" "Ma yew roje şima ra vat ke ziwanê ma zaf rind o"
ckb	1.0	ئەمڕۆ هەوا خۆشە
kmr	1.0	Îro hewa xweş e
zza	0.995	Ma yew roje şima ra vat ke ziwanê ma zaf rind o

ipa

A phonemic transcription. For Sorani it re-inserts the unwritten vowel (bizroke), marked in brackets, and resolves و and ی as consonant or vowel. --syllables puts dots between syllables, --narrow adds Kurmanji aspiration and ŋ, and --plain leaves the Sorani bizroke unmarked.

$ zimankit ipa پیاوەکە کتێبەکەم jinekê dîtin
pjaːwækæ
k(ɯ)teːbækæm
ʒɪnɛkeː
diːtɯn

$ zimankit ipa --syllables کتێبەکەم jinekê
k(ɯ).teː.bæ.kæm
ʒɪ.nɛ.keː

translit

$ zimankit translit "ئەمڕۆ هەوا خۆشە" "Îro hewa xweş e"
emrro hewa xoşe
ئیرۆ هەوا خوەش ئە

analyze

For each word: the word, lemma, part of speech, segmentation, gloss, features and score. --top N shows the N best analyses, --json prints one JSON object per word.

$ zimankit analyze کتێبەکەم jinekê
کتێبەکەم	کتێب	NOUN	کتێب+ەکە+م	STEM+DEF.SG+1SG	{"definite": "def", "number": "sg", "enclitic": "1sg"}	-2.8
jinekê	jin	NOUN	jin+ekê	STEM+INDEF.OBL.F	{"definite": "ind", "number": "sg", "case": "obl", "gender": "f"}	-2.5

$ zimankit analyze xwendiye
xwendiye	xwendin	VERB	xwendiye	VERB	{"tense": "present_perfect", "person": "3sg", "agreement": "object", "alignment": "ergative", …}	7.0

conjugate

Every form of a verb, one per line, with its features. Verbs in the reference list are conjugated directly. For others, zimankit builds the paradigm from the infinitive; give the present stem with --present and the transitivity with --transitive when it cannot guess them.

$ zimankit conjugate کردن | head -4
دەکەم	tense=present,mood=indicative,person=1sg
ناکەم	tense=present,mood=indicative,person=1sg,polarity=neg
بکەم	tense=present,mood=subjunctive,person=1sg
نەکەم	tense=present,mood=subjunctive,person=1sg,polarity=neg

$ zimankit conjugate kirin | head -2
dikim	tense=present,mood=indicative,person=1sg,agreement=subject
nakim	tense=present,mood=indicative,person=1sg,polarity=neg,agreement=subject

ergativity

Kurmanji is split-ergative: in the present the verb agrees with the subject, but in past transitive clauses the agent takes the oblique case and the verb agrees with the object. ergativity finds the verb, agent and object and checks the agreement.

$ echo "Min tu dîtim." | zimankit ergativity
{"sentence": "Min tu dîtim.", "verb": "dîtim", "lemma": "dîtin", "tense_stem": "past", "transitive": true,
 "alignment": "ergative", "agent": {"token": "Min", "case": "obl", "person": "1sg", …},
 "object": {"token": "tu", "case": "dir", "person": "2sg", …}, "verb_person": "1sg", "controller": "object",
 "agreement_ok": false, "notes": ["verb ending marks 1sg but object 'tu' is 2sg"]}

The correct sentence is Min tu dîtî (“I saw you”), where the verb agrees with tu.

rules

$ zimankit rules --lang kmr --out grammar_rules_kmr.json

Writes the grammar rules zimankit uses as JSON, for use in other tools: alphabet, orthography, phonology, morphology, verbs, syntax and the regular expressions.

Python API

from zimankit import analyze, normalize, to_ipa, identify, detect_script, __version__

normalize("كوردي")                        # 'کوردی'
to_ipa("jinekê")                          # 'ʒɪnɛkeː'
best = analyze("کتێبەکەم")[0]
best.lemma, best.pos, best.segmentation   # ('کتێب', 'NOUN', 'کتێب+ەکە+م')
best.features                             # {'definite': 'def', 'number': 'sg', 'enclitic': '1sg'}
r = identify("Îro hewa xweş e")
r.lang, r.confidence                      # ('kmr', 1.0)

Every function takes an optional lang="ckb" or lang="kmr". The full skills are in the subpackages:

from zimankit.skills_kmr.syntax import detect_alignment
detect_alignment("Min tu dîtî.").to_dict()["agreement_ok"]   # True

from zimankit.skills_ckb import verbs
idx = verbs.default_index()
forms = list(verbs.conjugate(idx.verbs["کردن"]))
forms[0].form, forms[0].features                           # ('دەکەم', {'tense': 'present', …})

Wordlist pipeline

From a clone, run_pipeline.py builds clean wordlists from your own data: .mdx dictionaries, .zim wiki dumps (Wikipedia, Wiktionary) and plain-text corpora.

pip install -e ".[data]"
cp config/sources.example.toml config/sources.local.toml   # set the paths; the file is git-ignored
python run_pipeline.py                 # all stages; cached stages are reused
python run_pipeline.py --stages build  # only rebuild the outputs
python run_pipeline.py --force         # recompute every stage

Stages: lexicon, dictionaries, corpus, build, rules, export. The outputs in output/ckb/ and output/kmr/ are clean_wordlist_<lang>.csv (grapheme, lemma, pos, ipa, morph_meta, translit, frequency, sources, confidence), lemma tables, spelling-variant maps and grammar_rules_<lang>.json. See chapter 16.

Kaggle dataset

The export stage flattens the outputs into a dataset with one row per word and 59 columns (features, IPA, syllables, both scripts, definitions, frequency), as CSV, Parquet and SQLite, with a data card and cover, in output/kaggle/:

pip install -e ".[kaggle]"
python run_pipeline.py --stages export

The published dataset is Kurdish Lexicon: Sorani & Kurmanji Word Database on Kaggle (CC BY-SA 4.0).

Tutorials and docs

• Tutorials: two complete courses, Sorani and Kurmanji, 23 lessons each from the alphabet to ergativity, tagged beginner, intermediate and advanced. Their tables are generated by zimankit and tested.
• Developer documentation: 21 chapters on data ingestion, Unicode, language identification, tokenisation, the orthography, phonology, morphology and verbs of each variety, transliteration, the pipeline, a regex cookbook, testing, a glossary and references.

Agent skills

The skills/ folder has ten self-contained skills (SKILL.md with front matter) that an AI agent can load to work with Kurdish: orthography, IPA, morphology and verbs for each variety, language identification and the wordlist builder. They call the zimankit package.

Other languages

examples/ implements the same portable core (normalisation and suffix segmentation) in shell, Python, TypeScript, Rust, Go, Java and C#, with the same expected output, so you can port the rules to any stack.

Versions and releases

zimankit follows Semantic Versioning; changes are listed in the changelog. Development follows git flow: features are merged into develop, and each release is merged into main and tagged vX.Y.Z. The tag builds and publishes the wheel, sdist, .deb and source archives. See CONTRIBUTING.md.

Troubleshooting

“zimankit: command not found”

pip installed the command into a directory that is not on your PATH (often ~/.local/bin). Add it to PATH, use pipx, or run python -m zimankit.

“requires a different Python”

zimankit needs Python 3.11 or newer (python3 --version).

The wrong language is used

The language is chosen from the script. Mixed or transliterated text may need --lang ckb or --lang kmr.

Arabic-script text shows in the wrong order in the terminal

Many terminals do not support right-to-left text. The output itself is correct: redirect it to a file, or use a terminal with bidi support.

Report a problem

Open an issue with the command, the input and the output you expected.