NGRAMSYMBOLS(1)

📦
version 1.3.17 · 3 versions
, in fedora 44

🌸 NAME 🌸

ngramsymbols — bestie, this is the manual page for ngramsymbols version 1.3.17, and it's lowkey about deriving a symbol table from a corpus. 💅

✨ DESCRIPTION ✨

okay so picture this: you hand it a corpus, and it derives a symbol table from it. that's the whole vibe. it's giving "I make symbol tables out of text files" and honestly? iconic. 👑

the usage is exactly this, no notes:

Usage: ngramsymbols [--options] [in.txt [out.txt]]

so you can pass an input file, and optionally an output file too, plus whatever --options you want. slay. 🌈

🎀 PROGRAM FLAGS 🎀

these are the flags that belong to the program itself, its own little personality traits fr fr:

  • --OOV_symbol — type = std::string, default = "<UNK>". this is the class label for OOV symbols. out-of-vocabulary besties get this name and that's just how it is 💖
  • --epsilon_symbol — type = std::string, default = "<epsilon>". label for epsilon. yes, epsilon gets its own little nametag 🦋

📚 LIBRARY FLAGS 📚

these ones come from the library side of things, still important, still eating 💅

🌟 Flags from: flags.cc

  • --help — type = bool, default = false. show usage information. the classic. the moment. use it when you're lost 🙈
  • --helpshort — type = bool, default = false. show brief usage information. same energy, shorter, for when you're in a rush ⭐
  • --tmpdir — type = std::string, default = "/tmp". temporary directory. where the temp files go to hang out 🍓
  • --v — type = int32_t, default = 0. verbosity level. crank it up if you want it to yap more 📢

🌟 Flags from: fst.cc

  • --fst_align — type = bool, default = false. write FST data aligned where appropriate. alignment matters, even for data 🧚
  • --fst_default_cache_gc — type = bool, default = true. enable garbage collection of cache. it's cleaning up after itself, we love a self-care queen ✨
  • --fst_default_cache_gc_limit — type = int64_t, default = 1048576. the cache byte size that triggers garbage collection. hit that number and the GC clock starts ticking 👑
  • --fst_read_mode — type = std::string, default = "read". default file reading mode for mappable files 🫶
  • --fst_verify_properties — type = bool, default = false. verify FST properties queried by TestProperties. double-checking the vibe, honestly valid 💜
  • --save_relabel_ipairs — type = std::string, default = "". save input relabel pairs to file. note the default is empty, so nothing gets saved unless you say so 🌸
  • --save_relabel_opairs — type = std::string, default = "". save output relabel pairs to file. same deal, empty default, your call 💖

🌟 Flags from: ngram-output.cc

  • --end_symbol — type = std::string, default = "</s>". class label for sentence end. the periodt of the symbol world 💅
  • --start_symbol — type = std::string, default = "<s>". class label for sentence start. main character energy, obviously it goes first 👑

🌟 Flags from: symbol-table.cc

  • --fst_compat_symbols — type = bool, default = true. require symbol tables to match when appropriate. they gotta match, no cap 🤝
  • --fst_field_separator — type = std::string, default = " ". set of characters used as a separator between printed fields. yeah bestie that default is literally a bunch of spaces, copy it exactly, don't get creative 😭

🌟 Flags from: util.cc

  • --fst_error_fatal — type = bool, default = true. FST errors are fatal; o.w. return objects flagged as bad: e.g., FSTs: kError property set, FST weights: not a Member(). so like, by default an FST error will absolutely end you 💀
  • --ngram_error_fatal — type = bool, default = true. NGram errors are fatal if true; otherwise returns objects flagged as bad: e.g., NGramModel::Error() is true. same dramatic energy, but for NGram errors 🔥

🌟 Flags from: weight.cc

  • --fst_weight_parentheses — type = std::string, default = "". characters enclosing the first weight of a printed composite weight (e.g., pair weight, tuple weight and derived classes) to ensure proper I/O of nested composite weights; must have size 0 (none) or 2 (open and close parenthesis). so it's a size-0-or-2 situation, no in-between, periodt ⚠️
  • --fst_weight_separator — type = std::string, default = ",". character separator between printed composite weights; must be a single character. one character only bestie, don't try to be extra 🌈
💌 quick vibe check: all those type = ... and default = ... bits are copy-pasted exactly as they come, so if you're scripting against them, you're safe. no guessing, no improvising, iykyk.

and that's the whole page!! this was ngramsymbols version 1.3.17, dated February 2022, and honestly it ate 💖🦋✨ ily, go derive some symbol tables bestie 💅👑

📦 where this page comes from