GETWORDFREQ(1)

📦
version 3.0.0~rc2 · 9 versions
, in ubuntu noble, ubuntu questing

💅 GETWORDFREQ 💅

okay so bestie, this is a Perl script from the sunpinyin corner of the universe — getwordfreq — and it's literally just here to spill the tea on your language model's vocabulary 🫶

✨ NAME ✨

getwordfreq — prints word freq info straight out of a language model, no cap 🔥

🌸 SYNOPSIS 🌸

getwordfreq [option]... -m slm-file -l lexicon

like, that's the whole vibe check: some options, an -m slm-file, and an -l lexicon. periodt ✨

🦋 DESCRIPTION 🦋

So what does it actually do? It prints out the word string and its freq for all the words in a language model 💖 — every single one, one per line, word + freq. It's giving "let me read the whole dictionary out loud" but make it statistical. Iconic, honestly.

🎀 OPTIONS 🎀

  • -s corpus-size — tells it how big the training corpus was. If you don't pass it, the default corpus-size is 300000000. yes, three hundred million, she's not being shy about it 💀
  • -v — be verbose! After the word and freq on each line, it also outputs other information. extra lore, we love to see it 🍓
  • -e — give format for ervin. that's it, that's the whole description, iykyk 👑
  • -m slm-file — specify the language model file. the main character of this whole operation, ngl 💜
  • -l lexicon — specify the lexicon file. and if you forget, there's a default lexicon waiting for you at /usr/share/sunpinyin-slm/dict.utf8 🌈 (literally just sitting there, ready to be your bestie)

🧚 AUTHOR 🧚

Originally written by Phill.Zhang <phill.zhang@sun.com>. Currently maintained by Kov.Chai <tchaikov@gmail.com>. legends, both of them 🙈💅

⭐ SEE ALSO ⭐

  • slmthread(1) — go check on her too, she's part of the fam
okay quick reminder bestie: -m and -l are in the synopsis for a reason — don't come crying to me if you leave them out 😭

and that's the page! 🎀 brought to you by Perl v5.36.0, dated 2023-08-08 — mwah, slay, bye 💖✨

📦 where this page comes from