MMSEG(1)

📦
version 3.0.0~rc2 · 9 versions
, in ubuntu noble, ubuntu questing

💅 MMSEG(1) — like, the mmseg manual but make it make sense 🎀

okay so bestie, welcome to mmseg, a user-contributed Perl situation, and honestly? she's iconic. let's get into it ✨

✨ NAME ✨

mmseg — maximum matching segment Chinese text. that's it, that's the whole vibe. she takes Chinese text and chops it into words, no cap 🌸

🦋 SYNOPSIS 🦋

mmseg -d dict_file [option]... [corpus_file]...

like… you need the dict, the options are optional (as options are), and the corpus files are whatever you feel like throwing at her 💅

🎀 DESCRIPTION 🎀

mmseg is a tool for segmenting Chinese text into words using the maximum matching algorithm. periodt.

she segments corpus_file, or standard input if you didn't name a file (she's not picky, she's flexible 🫶), and writes the segmented result to standard output. in, chopped up, out. main character energy from start to finish 👑

💖 OPTIONS 💖

  • -d dict_file — use dict_file as the lexicon, bestie. and if you don't have one handy, there's a default lexicon chillin' at /usr/share/sunpinyin-slm/dict.utf8 — she's already there waiting for you 🥰
  • -f, --format (text|bin) — output format, and it can be 'text' or 'bin'. default is 'bin'. so like, normally: in text mode the actual word text gets output, while in binary mode a binary short integer of the word-ids gets written to stdout. two totally different aesthetics, both valid 💫
  • -s, --stok STOK_ID — sentence token id. default 10. this lil cutie gets written to output in binary mode after every sentence, like a little period but make it numeric 🎯
  • -i, --show-id — show id info. under text output format mode, she attaches the id after known words; if you're under binary mode, she prints the id(s) in text. either way you're getting the receipts 🔍
  • -a, --ambiguious-id AMBI-ID — okay so "ambiguious" (yes, that's how it's spelled, we love her for it 💀) means ABC => A BC or AB C. if you specify it (AMBI-ID != 0), the sequence ABC will not be segmented. in binary mode the AMBI-ID is written out; in text mode <ambi>ABC</ambi> gets output. default is 0, i.e. she minding her own business 🙈

🧚 NOTES 🧚

under binary mode, consecutive ids of 0 are merged into one 0. under text mode, no spaces are inserted between unknown-words. that's just how she rolls, don't @ her 💀

🌈 AUTHOR 🌈

originally written by Phill.Zhang <phill.zhang@sun.com>. currently maintained by Kov.Chai <tchaikov@gmail.com>. legends, both of them 👑💜

⭐ SEE ALSO ⭐

slmseg(1), ids2ngram (1) — go read those too, iykyk 📚✨

okay that's the whole page, you're officially educated 🎓💖 this lil number was brought to you by perl v5.36.0 and dated 2023-08-08 — mwah, go segment something gorgeous 🍓🦋

📦 where this page comes from