HUGE-SPLIT(1)

📦 version 1.31, in nixos 26.05, nixpkgs unstable

💅 HUGE-SPLIT(1) — the perl-y one, bestie

okay so real talk: this is a page from User Contributed Perl Documentation, and it's about a lil script called huge-split.pl. think of it as the chill scissors that cuts your giant bigram files into snackable pieces 🎀

✨ NAME ✨

huge-split.pl — splits bigram files from huge-count.pl into pieces. like, literally chopping them up. it's giving meal prep for your data 🍓

🌸 DESCRIPTION 🌸

see perldoc huge-split.pl for the deets, bestie. that's where the lore lives 💖

🦋 USAGE 🦋

huge-split.pl [OPTIONS] SOURCE

👑 INPUT 👑

💜 Required Arguments: 💜

  • SOURCE — your input file. no cap, this one's mandatory.

okay so your SOURCE should be a file that was generated by huge-count.pl or by count.pl with the tokenlist option. the results files all get named after the input source file, and every single split file picks up an extension sequence number. cute, right? each piece knows its place ✨

🔥 --split N 🔥

this parameter should be set. like, this is not a drill, bestie — huge-split divides up the output bigrams tokenlist that count.pl or huge-count.pl generated. each part created with --split N will contain N lines.

why does N matter so much? because you want to pick N so that huge-sort.pl can actually be run efficiently on any part that has N lines out of the file containing all the bigrams. vibe check your memory, basically 💀

🫶 the manpage's own suggestion: set N equal to the number of KB of memory you have. if your computer has 8 GB RAM, that's 8,000,000 KB, so N should be 8000000.

🌈 Other Options: 🌈

  • --help — displays this message. the classic. the icon. 🎀
  • --version — displays the version information. slay, let me know what build we're on 💅

🌟 AUTHOR 🌟

Amruta Purandare, Ted Pedersen, Ying Liu. University of Minnesota at Duluth. absolute legends, periodt 👑

Copyright (c) 2004-2011

Ted Pedersen, University of Minnesota, Duluth. tpederse@umn.edu

Ying Liu, University of Minnesota, Twin Cities. liux0395@umn.edu

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. (translation: it's free, it's generous, it promises nothing — know the difference 😭)

You should have received a copy of the GNU General Public License along with this program; if not, write to

The Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston,
MA  02111-1307, USA.
🙈 also the copyright years are 2004-2011 and the addresses are real — don't come for me, I'm just the messenger 🫶

and that's the whole page, bestie! 🌈 this one was brewed by perl v5.42.3 and stamped 2015-10-04 — so it's vintage, it's iconic, it still runs, mwah 💋✨

📦 where this page comes from