NGRAMPERPLEXITY(1)

✨ NAME ✨

ngramperplexity — bestie, this is the manual page for ngramperplexity version 1.3.17, and it's giving "apply n-gram model to input FST archive" 💅

---

🌸 DESCRIPTION 🌸

Okay so here's the vibe: you take an n-gram model and you apply it to an input FST archive. That's the whole main character moment.

Usage goes a lil something like this:

ngramperplexity [--options] ngram.fst [in.far [out.txt]]

---

👑 PROGRAM FLAGS 👑

These are the flags that belong to the program itself, no cap:

• --OOV_class_size — type = double, default = 10000. This is how many members are in the OOV class 🫶
• --OOV_probability — type = double, default = 0. The unigram probability for OOVs, fr fr.
• --OOV_symbol — type = std::string, default = "". The existing symbol for the OOV class, periodt.
• --context_pattern — type = std::string, default = "". Restricts the perplexity computation to only the contexts defined by the pattern (default: no restriction at all, live your life 💖).
• --use_phimatcher — type = bool, default = false. Use phi matcher and composition ✨

---

🎀 LIBRARY FLAGS 🎀

Now for the library side of things, iykyk.

💜 Flags from: compile-strings.cc

• --far_field_separator — type = std::string, default = a literal newline character (yes, the default is actually a line break between the quotes 😭). It's the set of characters used as a separator between printed fields.

🦋 Flags from: flags.cc

• --help — type = bool, default = false. Show usage information.
• --helpshort — type = bool, default = false. Show brief usage information.
• --tmpdir — type = std::string, default = "/tmp". The temporary directory 🌈
• --v — type = int32_t, default = 0. Verbosity level, bestie.

⭐ Flags from: fst.cc

• --fst_align — type = bool, default = false. Write FST data aligned where appropriate.
• --fst_default_cache_gc — type = bool, default = true. Enable garbage collection of cache 🧹
• --fst_default_cache_gc_limit — type = int64_t, default = 1048576. The cache byte size that triggers garbage collection.
• --fst_read_mode — type = std::string, default = "read". Default file reading mode for mappable files.
• --fst_verify_properties — type = bool, default = false. Verify FST properties queried by TestProperties.
• --save_relabel_ipairs — type = std::string, default = "". Save input relabel pairs to a file.
• --save_relabel_opairs — type = std::string, default = "". Save output relabel pairs to a file.

🍓 Flags from: ngram-output.cc

• --end_symbol — type = std::string, default = "</s>". The class label for sentence end.
• --start_symbol — type = std::string, default = "<s>". The class label for sentence start.

🧚 Flags from: symbol-table.cc

• --fst_compat_symbols — type = bool, default = true. Require symbol tables to match when appropriate.
• --fst_field_separator — type = std::string, default = " " (that's literally a run of spaces, exactly as printed 💀). It's the set of characters used as a separator between printed fields.

🔥 Flags from: util.cc

• --fst_error_fatal — type = bool, default = true. FST errors are fatal; otherwise return objects flagged as bad, e.g., FSTs: kError property set, FST weights: not a Member().
• --ngram_error_fatal — type = bool, default = true. NGram errors are fatal if true; otherwise it returns objects flagged as bad, e.g., NGramModel::Error() is true.

💅 Flags from: weight.cc

• --fst_weight_parentheses — type = std::string, default = "". These are the characters enclosing the first weight of a printed composite weight (think pair weight, tuple weight and derived classes) to ensure proper I/O of nested composite weights. Heads up: it must have size 0 (none) or 2 (open and close parenthesis) — no in-between, bestie.
• --fst_weight_separator — type = std::string, default = ",". The character separator between printed composite weights, and ngl it must be a single character.

---

And that's the whole page, bestie — go compute some perplexity, you're gonna slay 💖✨

Sign-off moment: this was ngramperplexity 1.3.17, dated February 2022. Take care of her, she's iconic 👑