NGRAMRANDTEST(1)

📦
version 1.3.17 · 3 versions
, in fedora 44

💖 NAME 💖

ngramrandtest — manual page for ngramrandtest 1.3.17, living in section 1 (User Commands) of the manual, bestie 💅

✨ DESCRIPTION ✨

Okay so this one generates random data/models. That's the whole vibe. It's giving "chaos, but make it structured" 🦋

Usage:

ngramrandtest [--options]

🎀 PROGRAM FLAGS 🎀

These are the ones that actually shape what gets generated. Pay attention, this is the main character section 👑

  • --directory — type = std::string, default = ".". The directory where files will be placed. It just wants to know where to put your stuff 🏠
  • --max_length — type = int32_t, default = 1000. Maximum sentence length. Cap it, queen.
  • --mean_length — type = int32_t, default = 100. Maximum mean string length.
  • --ngram_max — type = int32_t, default = 3. Maximum n-gram order size.
  • --sample_max — type = int32_t, default = 10000. Maximum sample corpus size.
  • --seed — type = int32_t, default = 1768688633. Randomization seed. Yes it has a default seed, and yes it's that specific number, no I will not round it, periodt 💀
  • --thresh_max — type = double, default = 3. Maximum threshold size.
  • --vars — type = std::string, default = "". File name for outputting variable values.
  • --vocabulary_max — type = int32_t, default = 5000. Maximum vocabulary size.

📚 LIBRARY FLAGS 📚

These are the borrowed-from-the-library flags, sorted by which file they came from. Not me organising them by source file like a little librarian 🧚

Flags from: flags.cc

  • --help — type = bool, default = false. Show usage information. Your bestie when you're lost 🫶
  • --helpshort — type = bool, default = false. Show brief usage information.
  • --tmpdir — type = std::string, default = "/tmp". Temporary directory.
  • --v — type = int32_t, default = 0. Verbosity level.

Flags from: fst.cc

  • --fst_align — type = bool, default = false. Write FST data aligned where appropriate.
  • --fst_default_cache_gc — type = bool, default = true. Enable garbage collection of cache. It's cleaning up after itself, we love a self-care icon ✨
  • --fst_default_cache_gc_limit — type = int64_t, default = 1048576. Cache byte size that triggers garbage collection.
  • --fst_read_mode — type = std::string, default = "read". Default file reading mode for mappable files.
  • --fst_verify_properties — type = bool, default = false. Verify FST properties queried by TestProperties.
  • --save_relabel_ipairs — type = std::string, default = "". Save input relabel pairs to file.
  • --save_relabel_opairs — type = std::string, default = "". Save output relabel pairs to file.

Flags from: ngram-output.cc

  • --end_symbol — type = std::string, default = "</s>". Class label for sentence end. Goodbye symbol, slay 🌈
  • --start_symbol — type = std::string, default = "<s>". Class label for sentence start. Hello symbol 💜

Flags from: symbol-table.cc

  • --fst_compat_symbols — type = bool, default = true. Require symbol tables to match when appropriate.
  • --fst_field_separator — type = std::string, default = " ". Set of characters used as a separator between printed fields. (Yes, that default is literally just a run of spaces. It said what it said. 🙈)

Flags from: util.cc

  • --fst_error_fatal — type = bool, default = true. FST errors are fatal; o.w. return objects flagged as bad: e.g., FSTs: kError property set, FST weights: not a Member(). So like, when this is on, an FST error is over — no recovery arc 😭
  • --ngram_error_fatal — type = bool, default = true. NGram errors are fatal if true; otherwise returns objects flagged as bad: e.g., NGramModel::Error() is true.

Flags from: weight.cc

  • --fst_weight_parentheses — type = std::string, default = "". Characters enclosing the first weight of a printed composite weight (e.g., pair weight, tuple weight and derived classes) to ensure proper I/O of nested composite weights; must have size 0 (none) or 2 (open and close parenthesis). So it's 0 or 2 characters, no in-between, choose wisely 🎀
  • --fst_weight_separator — type = std::string, default = ",". Character separator between printed composite weights; must be a single character. One character. Not two. Don't test it.
Heads up bestie: --fst_weight_parentheses has to be size 0 or size 2, and --fst_weight_separator has to be exactly one character. The manpage said it, so I'm saying it 💅

That's the whole page, and that's on ngramrandtest 1.3.17 — documented back in February 2022, and honestly? Still iconic. Go generate some random data/models, you've got this. Bye bestieee 🦋💖

📦 where this page comes from