NGRAMCOUNT(1)

💖 NGRAMCOUNT 💖

okay bestie, gather round, because this is ngramcount — the manual page for ngramcount version 1.3.17. she counts n-grams. that's the whole thing. she's an icon, she's efficient, she's literally just vibing and tallying up n-grams from an input file. no cap.

---

✨ DESCRIPTION ✨

Count n-grams from input file. yes ma'am, that's the job. that's what we do here. 🎀

the usage, verbatim, because you will be typing this:

Usage: ngramcount [--options] [in.far [out.fst]]

so you hand her an optional in.far, and optionally an out.fst, and you get counts. gorgeous.

---

👑 PROGRAM FLAGS 👑

these are the main character flags, the ones specific to ngramcount herself. every single one, no skipping, i promised:

• --add_to_symbol_unigram_count — type = double, default = 0. adds this amount to the unigram count of every single word in the symbol table. little boost for everybody, we love to see it.
• --alpha — type = double, default = 1. the weight for the first FST.
• --backoff_label — type = int64_t, default = 0. the backoff label. simple as that.
• --beta — type = double, default = 1. weight for the second (and subsequent) FST(s). so if you've got a whole lineup of FSTs, alpha takes the first, beta takes the rest of the squad.
• --check_consistency — type = bool, default = false. checks model consistency. she's doing a vibe check on your model.
• --context_pattern — type = std::string, default = "". pattern of contexts to count.
• --epsilon_as_backoff — type = bool, default = false. treats epsilon in the input Fsts as backoff. (capital-F "Fsts" — that's how the manpage says it, i'm not gonna lie to you 💅)
• --method — type = std::string, default = "counts". one of: "counts", "histograms", "count_of_counts", "count_of_histograms". pick your fighter.
• --norm_eps — type = double, default = 0.001. the normalization check epsilon.
• --normalize — type = bool, default = false. normalizes the resulting model. glow-up for your output.
• --order — type = int64_t, default = 3. sets the maximal order of ngrams to be counted. default three, because trigrams are that girl.
• --output_fst — type = bool, default = true. outputs counts as fst — otherwise strings. so if you turn this off you're getting strings, bestie, fair warning.
• --require_symbols — type = bool, default = true. require symbol tables? (default: yes) — the manpage literally answers its own question here, obsessed.
• --round_to_int — type = bool, default = false. rounds all counts to integers. no more decimals, no more crumbs.

---

🦋 LIBRARY FLAGS 🦋

these are the shared ones, grouped by which file they came from. keeping them all in their little friend groups because that's how the original does it and we respect the source material. periodt.

flags from: compile-strings.cc

• --far_field_separator — type = std::string, default:

"
"

(set of characters used as a separator between printed fields — and yes, that default really is a literal newline chillin' inside the quotes. she contains multitudes. 😭)

flags from: flags.cc

• --help — type = bool, default = false. show usage information.
• --helpshort — type = bool, default = false. show brief usage information. the tl;dr version, for when you're in a rush.
• --tmpdir — type = std::string, default = "/tmp". temporary directory.
• --v — type = int32_t, default = 0. verbosity level. crank it up if you want her to talk more.

flags from: fst.cc

• --fst_align — type = bool, default = false. write FST data aligned where appropriate.
• --fst_default_cache_gc — type = bool, default = true. enable garbage collection of cache.
• --fst_default_cache_gc_limit — type = int64_t, default = 1048576. the cache byte size that triggers garbage collection. that's 1048576 bytes exactly, don't @ me.
• --fst_read_mode — type = std::string, default = "read". default file reading mode for mappable files.
• --fst_verify_properties — type = bool, default = false. verify FST properties queried by TestProperties.
• --save_relabel_ipairs — type = std::string, default = "". save input relabel pairs to file.
• --save_relabel_opairs — type = std::string, default = "". save output relabel pairs to file.

flags from: ngram-output.cc

• --end_symbol — type = std::string, default = "</s>". class label for sentence end.
• --start_symbol — type = std::string, default = "<s>". class label for sentence start.

flags from: symbol-table.cc

• --fst_compat_symbols — type = bool, default = true. require symbol tables to match when appropriate.
• --fst_field_separator — type = std::string, default = " ". set of characters used as a separator between printed fields.

flags from: util.cc

• --fst_error_fatal — type = bool, default = true. FST errors are fatal; o.w. return objects flagged as bad: e.g., FSTs: kError property set, FST weights: not a Member().
• --ngram_error_fatal — type = bool, default = true. NGram errors are fatal if true; otherwise returns objects flagged as bad: e.g., NGramModel::Error() is true.

flags from: weight.cc

• --fst_weight_parentheses — type = std::string, default = "". characters enclosing the first weight of a printed composite weight (e.g., pair weight, tuple weight and derived classes) to ensure proper I/O of nested composite weights; must have size 0 (none) or 2 (open and close parenthesis). so it's nothing or it's a matched pair, no in-between, she does not do half measures. 💀
• --fst_weight_separator — type = std::string, default = ",". character separator between printed composite weights; must be a single character. one character, bestie. one. that's the rule.

---

💜 real talk: --fst_error_fatal and --ngram_error_fatal both default to true, which means errors are fatal unless you flip them. so if things go boom, that's not a bug, that's the default configuration doing exactly what it promised. adjust accordingly.

---

and that's the whole page, start to finish, nothing skipped. 🫶

sign-off time: this is ngramcount version 1.3.17, dated February 2022. she's out here being a User Command and she ate, fr fr. go count some n-grams, bestie. 💖✨🎀🦋