SLMSEG(1)

📦
version 3.0.0~rc2 · 9 versions
, in ubuntu noble, ubuntu questing

✨ slmseg — maximum matching segment Chinese text ✨

okay bestie, so this is slmseg(1) from the User Contributed Perl Documentation section, and it's genuinely one of those little tools that just does the thing. let's get into it 💅

🌸 SYNOPSIS 🌸

slmseg -d dict_file [option]... [corpus_file]...

🦋 DESCRIPTION 🦋

so slmseg is the tool for segmenting Chinese text into words, and it does it using the maximum matching algorithm. that's the whole vibe. 🫶

it segments corpus_file, or standard input if you don't give it a filename, and writes the segmented result to standard output. no cap, that's it. main character energy tbh.

💖 OPTIONS 💖

  • -d dict_file — use dict_file as the lexicon. a default lexicon can be found at /usr/share/sunpinyin-slm/dict.utf8. literally just point it at your dictionary and go ✨
  • -f,--format (text|bin) — the output format, and it can be 'text' or 'bin'. default is 'bin'. normally, in text mode, word text is output, while in binary mode, binary short integer of the word-ids are written to stdout. so like… readable words vs. compact ids, pick your fighter 👑
  • -s, --stok STOK_ID — sentence token id. default 10. it will be written to output in binary mode after every sentence. cute little sentence marker fr fr 🎀
  • -i, --show-id — show Id info. under text output format mode, attach id after known words. if under binary mode, print id(s) in text. iykyk 🙈
  • -m, --model language-model-file — specify the language model file. this file is always generated by slmthread. always. no "i made it myself" energy allowed 💀

⭐ NOTES ⭐

under binary mode, consecutive id of 0 are merged into one 0. under text mode, no space are inserted between unknown-words. that's just how it is bestie, don't @ me 🌈

lowkey important: those two behaviours are not configurable. they're just the rules. remember them before you go debugging something that was never broken 💜

🌼 AUTHOR 🌼

originally written by Phill.Zhang <phill.zhang@sun.com>. currently maintained by Kov.Chai <tchaikov@gmail.com>. icons, both of them 🫡

💫 SEE ALSO 💫

and that's the whole page, bestie 🍓 sign-off time: this one's from perl v5.36.0, dated 2023-08-08, filed under SLMSEG(1) — go segment some text and slay 💅✨

📦 where this page comes from