Juurio

Open-vocabulary Finnish morphology over raw text. Give it a sentence and an operation (past, passive, plural, …) and it returns the transformed sentence or an explicit abstention. It also inflects out-of-vocabulary and loan words that a closed-lexicon morphology cannot (striimata → striimannee, striimaaja → striimaajillanikin).

from juurio import transform, realize_np, realize_verbform

transform("past",      "Koira haukkuu pihalla.").text   # 'Koira haukkui pihalla.'
transform("passive",   "Mies osti talon.").text         # 'Ostettiin talo.'
transform("pluperfect","Hän lukee kirjan.").text        # 'Hän oli lukenut kirjan.'  (compound tense)
transform("neg_imperative","Osta talo.").text           # 'Älä osta taloa.'  (imperative/2nd-person source)
transform("potential", "Nuori striimaa peliä.", lemma="striimata").text   # 'Nuori striimannee peliä.'  (OOV verb)
transform("plural+poss_1sg+clitic_kin", "Koira haukkuu pihalla.").text    # 'Koiranikin haukkuvat pihalla.'

# phrase-level morphology utilities
realize_np("talo", case="Ine", number="Plur", px="1Sg").text                       # 'taloissani'
realize_np("striimaaja", case="Ade", number="Plur", px="1Sg", clitic="kin").text   # 'striimaajillanikin' (OOV)
realize_verbform("mennä", form="inf3", case="Ade").text                            # 'menemällä'
realize_verbform("striimata", form="va").text                                      # 'striimaava' (OOV participle)

Every call returns a Result (ok, text, reason, trace, warnings). Input that is out of scope or that the operation does not apply to abstains, with ok=False and a documented reason. Forms produced by a neural model rather than the exception lexicon are listed in trace["neural_unverified"] and in warnings.

Install

pip install https://huggingface.co/samuli/juurio/resolve/main/juurio-0.1.0-py3-none-any.whl
juurio setup         # download and verify the model checkpoints and data

Python 3.12 or newer. The install pulls in torch, transformers, numpy and huggingface-hub.

On a machine with a GPU, that installs the CUDA build of PyTorch. On a CPU-only machine the default PyTorch wheel also bundles CUDA, a download of several GB you can't use. To get the small CPU-only build instead, install it first:

pip install torch --index-url https://download.pytorch.org/whl/cpu
pip install https://huggingface.co/samuli/juurio/resolve/main/juurio-0.1.0-py3-none-any.whl

The wheel contains only the code. juurio setup downloads the model checkpoints and data from this repository (about 190 MB), checks their sha256 hashes and caches them. Without it, they are downloaded on first use. juurio doctor shows what is present and what is missing.

Background

Finnish is richly inflecting, and the standard open-source tools for it are excellent but closed-vocabulary and heavy at runtime. A finite-state morphology (Giella fin and Omorfi, also the engine behind Voikko) inflects only words already in its lexicon, and a neural dependency parser (Stanza) brings a large dependency stack. A loan or slang word the lexicon has never seen (striimata, striimaaja) cannot be inflected.

Juurio distills those tools into small neural models and a compact exception lexicon, so at runtime it needs only PyTorch and it can inflect words the finite-state morphology does not know. The heavy tools were used only to build the models and data: Giella fin is the morphology the neural models learned from, Stanza produced silver parses for the parser, and Voikko was used to check generated forms.

Architecture

concern	component
structure and scope (which clause, subject, object)	neural UD dependency parser
morphology of known words	exception lexicon (the forms the neural realizers get wrong)
open-vocabulary morphology (OOV and loan words)	small neural realizers: nominal, verb-form and finite-verb

The rule engine handles all non-local structure (agreement, object case, subject drop). The neural realizers are called per word, only when the lexicon has no entry for the form.

Entry points

• transform(op, text, lemma=None): finite-clause transforms that preserve grammar, such as negation, tense and mood (including the potential and the compound tenses), passive, question, the full imperative paradigm (with the negative älä series), number with agreement, degree, enclitics, possessives and +-composed nominal morphology.
• realize_np(np, …): phrase-level nominal morphology, with the full case system, possessive suffixes, clitics and modifier agreement. Arbitrary case lives here because it is syntactically governed.
• realize_verbform(verb, …): phrase-level non-finite verb morphology. Participles (VA, NUT, TAVA, TU and the agent participle MA) and infinitives, inflected in case and number with an optional possessive suffix.
• inflect(word, …), analyze(text) and lemmatize(text): single-word inflection, analysis and lemmatization.

Anything semantic, such as existential framing, is deliberately out of scope.

CLI

Every library function has a command of the same name. The CLI is a thin wrapper over the same Transformer.

# one command per library function
juurio transform <op> "<text>" [--lemma L] [--trace] [--json]      # transform()
juurio inflect "<word>" [spec…] [--case C] [--number N] [--degree D] [--px P] [--clitic K] [--pos P] [--like W] [--proper] [--json]   # inflect()
juurio analyze "<text>" [--json]                                   # analyze()
juurio lemmatize "<text>" [--json]                                 # lemmatize()
juurio realize-np "<np>" [--case Ine] [--number Plur] [--px 1Sg] [--clitic kin]   # realize_np()
juurio realize-verbform "<verb>" [--form va|nut|tava|tu|inf2|inf2_pass|inf3|ma] [--case Ine] [--px 3]   # realize_verbform()

# infrastructure (no library counterpart)
juurio doctor                 # read-only environment diagnostics (dependencies, checkpoints + sha256)
juurio setup                  # download and verify the checkpoints and data
juurio list-ops               # documented operation catalog
juurio batch <op> < lines     # or per-line "op<TAB>text[<TAB>lemma]"
juurio demo | bench | repl

inflect takes a case, number and degree phrase positionally (juurio inflect talo mon gen → talojen) or the equivalent --case, --number and --degree flags.

Exit status is 0 on success, 2 if any transform abstained (useful for scripting) and 1 on setup failure.

Files

path	what
juurio-0.1.0-py3-none-any.whl	the Python package
juurio/, pyproject.toml	source code of the package
docs/api.md	API and CLI contract, scope boundary, reason codes
checkpoints/	parser, analyser and realizer models with their tokenizer and vocabularies
data/	exception lexicon, morphology tag table and vocabularies
ATTRIBUTION.md	licences and sources, file by file

Training data

• The realizers learned from inflected forms generated with the Giella fin transducer, for lemmas found in a Finnish web corpus.
• The exception lexicon holds the Giella forms the realizers get wrong.
• The parser was trained on UD_Finnish-TDT and on silver parses of Finnish web text made with Stanza.
• The morphology tag table lists Giella's tags for the 100,000 most frequent web-text word forms where the neural analyser disagrees with Giella. E-mail addresses, @handles, URLs, phone numbers and hashtags were removed from it.

Limitations

• transform handles one finite main clause in the active indicative, within limits of about twelve words and one finite verb. Subordinate and coordinated clauses, already negated clauses and multi-verb fragments abstain.
• Forms from the neural realizers are model guesses. They are flagged in warnings but not verified, and some are wrong: realize_np("bugisovellus", case="Ine", px="1Pl", clitic="kin") gives bugisovelludessammekin instead of bugisovelluksessammekin.
• The parser makes mistakes, and a wrong lemma or analysis can lead to a wrong form or an abstention. The lemma= argument overrides the lemma of the main verb.

License

Three parts with separate licences (details in ATTRIBUTION.md):

part	licence
code (juurio/, the wheel)	MIT
models (checkpoints/, data/char_vocab.json)	CC BY 4.0
data derived from Giella fin (data/wordlist.json, data/fst_table.jsonl, data/fst_tag_vocab.json)	GPL-3.0, the licence of GiellaLT lang-fin (© UiT The Arctic University of Norway / Giellatekno)