MERYL(1)

okay so bestie ๐Ÿ’– you pulled up meryl(1) from User Commands and honestly?? it's giving kmer factory, main character energy, no cap. let's get you fluent in it. ๐ŸŒˆ


๐ŸŒธ NAME ๐ŸŒธ

meryl โ€” in- and out-of-core kmer counting and utilities โœจ


๐Ÿ“œ SYNOPSIS ๐Ÿ“œ

๐Ÿงฎ Estimating memory requirements

meryl -P -m kmersize [-c #] [-p] -s seq.fasta
meryl -P -m kmersize [-c #] [-p] -n mercount

๐Ÿ—๏ธ Building a table

meryl -B -m kmersize [-c #] [-p] [-v] [-f|-r|-C] [-L minoccurrence] [-U maxoccurrence] [-threads n | {-segments segments | -memory megabytes} [-configbatch [-sge jobname]]] -s seq.fasta -o tblprefix
meryl -countbatch number [-sgebuild "qsuboptionstring"] -o tblprefix
meryl -mergebatch number [-sgemerge "qsuboptionstring"] -o tblprefix

๐ŸŽ›๏ธ Performing operations on a table

meryl -M operation [-v] -s tblprefix [-s  tblprefix2 ...]  -o output

๐Ÿ‘€ Dumping a table

meryl -Dh -s tblprefix
meryl -Dt -n mincount -s tblprefix

๐Ÿ’œ DESCRIPTION ๐Ÿ’œ

so what does meryl actually do? it computes the kmer content of genomic sequences. Kmer content is represented as a list of kmers and the number of times each occurs in the input sequences. and here's the glow-up: the kmer can be restricted to only the forward kmer, only the reverse kmer, or the canonical kmer โ€” that's the lexicographically smaller of the forward and reverse kmer at each location, iykyk. ๐Ÿฆ‹

and then Meryl can report the histogram of counts, the list of kmers and their counts, or can perform mathematical and set operations on the processed data files. versatile queen ๐Ÿ’…

the output of meryl is two binary files, called a meryl database, which can be quickly dumped to provide a histogram of counts, or the actual counts. and โ€” bestie, this is the best part โ€” a C++ library is supplied for direct access to the files. periodt. ๐ŸŽ€


โœจ OPTIONS โœจ

  • -P โ€” Estimate memory requirements. Given a sequence file (-s) or an upper limit on the number of mers in the file (-n), compute the table size (-t in build) to minimize the memory usage. This mode recognizes the following options:
    • -m # โ€” size of a mer (required)
    • -c # โ€” homopolymer compression (optional)
    • -p โ€” enable positions
    • -s seq.fasta โ€” Sequence file to be scanned to determine the number of mers
    • -n # โ€” compute params assuming file with this many mers in it

Only one of -s, -n need to be specified. If both are given, -s takes priority. ๐Ÿ‘‘

  • -B โ€” Compute the mer-count tables given a sequence file (-s) and lots of parameters. By default, both strands are processed.
    • -f โ€” only build for the forward strand
    • -r โ€” only build for the reverse strand
    • -C โ€” use canonical mers (assumes both strands)
    • -L # โ€” DON'T save mers that occur less than # times ๐Ÿ™ˆ
    • -U # โ€” DON'T save mers that occur more than # times ๐Ÿ™ˆ
    • -m # โ€” size of a mer (required)
    • -c # โ€” homopolymer compression (optional)
    • -p โ€” enable positions
    • -s seq.fasta โ€” sequence to build the table for
    • -o tblprefix โ€” output table prefix
    • -v โ€” entertain the user ๐Ÿฟ

The meryl process can run in one large memory batch, in many small memory batches, or under SGE control, all with or without using multiple CPU cores. By default, the computation is done as one large sequential process. Multi-threaded operation is possible, at additional memory expense, as is segmented operation, at additional I/O expense. ๐Ÿ’€

๐Ÿงต Threaded operation

Split the counting in to n almost-equally sized pieces. This uses an extra h MB (from -P) per thread.

  • -threads n โ€” use n threads to build

๐Ÿชœ Segmented, sequential operation

Split the counting into pieces that will fit into no more than m MB of memory, or into n equal sized pieces. Each piece is computed sequentially, and the results are merged at the end. Only one of -memory and -segments is needed.

  • -memory m โ€” use at most m MB of memory per segment
  • -segments n โ€” use n segments

๐Ÿ“ฆ Segmented, batched operation

Same as sequential, except this allows each segment to be manually executed in parallel. Only one of -memory and -segments is needed. Also see the EXAMPLE section on this page. ๐Ÿซถ

  • -memory m โ€” use at most m MB of memory per segment
  • -segments n โ€” use n segments
  • -configbatch โ€” create the batches
  • -countbatch n โ€” run batch number n
  • -mergebatch โ€” merge the batches

Batched mode can run on the grid. ๐ŸŒ

  • -sge jobname โ€” unique job name for this execution. Meryl will submit jobs with name mpjobname, ncjobname, nmjobname, for phases prepare, count and merge.
  • -sgebuild "options"
  • -sgemerge "options" โ€” any additional options to qsub(1) (e.g., "-p -153 -pe thread 2 -A merylaccount") N.B. - -N will be ignored N.B. - be sure to quote the options
bestie the quoting thing is NOT optional, fr fr. and yeah, -N is getting ignored, that's just how it is. ๐Ÿ˜ญ
  • -M โ€” Given a list of tables, perform a math, logical or threshold operation. Unless specified, all operations take any number of databases. Math operations are:
    • min โ€” count is the minimum count for all databases. If the mer does NOT exist in all databases, the mer has a zero count, and is NOT in the output. ๐Ÿ’€
    • minexist โ€” count is the minimum count for all databases that contain the mer
    • max โ€” count is the maximum count for all databases
    • add โ€” count is sum of the counts for all databases
    • sub โ€” count is the first minus the second (binary only)
    • abs โ€” count is the absolute value of the first minus the second (binary only)

Logical operations are:

  • and โ€” outputs mer iff it exists in all databases
  • nand โ€” outputs mer iff it exists in at least one, but not all, databases
  • or โ€” outputs mer iff it exists in at least one database
  • xor โ€” outputs mer iff it exists in an odd number of databases

Threshold operations are:

  • lessthan x โ€” outputs mer iff it has count < x
  • lessthanorequal x โ€” outputs mer iff it has count <= x
  • greaterthan x โ€” outputs mer iff it has count > x
  • greaterthanorequal x โ€” outputs mer iff it has count >= x
  • equal x โ€” outputs mer iff it has count == x

Threshold operations work on exactly one database.

  • -s tblprefix โ€” use tblprefix as a database
  • -o tblprefix โ€” create this output
  • -v โ€” entertain the user ๐Ÿฟ
  • -D โ€” Dump table (not all of these work) ๐Ÿ™ˆ
    • -Dd โ€” Dump a histogram of the distance between the same mers.
    • -Dt โ€” Dump mers >= a threshold. Use -n to specify the threshold.
    • -Dc โ€” Count the number of mers, distinct mers and unique mers.
    • -Dh โ€” Dump (to stdout) a histogram of mer counts.
    • -s โ€” Read the count table from here (leave off the .mcdat or .mcidx).

๐Ÿงช EXAMPLE ๐Ÿงช

๐Ÿญ Batch creation of a table

Initialize the compute with -configbatch, which needs all the build options. Execute all -countbatch jobs, then -mergebatch to complete. โœจ

meryl -configbatch -B [options] -o file
meryl -countbatch 0 -o file
meryl -countbatch 1 -o file
...
meryl -countbatch N -o file
meryl -mergebatch N -o file

๐Ÿ”— SEE ALSO ๐Ÿ”—

simple(1), mapMers(1), mapMers-depth(1), kmer-mask(1)


and that's the whole vibe bestie ๐Ÿซถ this page is from meryl 0~20150520+2004, dated May 2015 โ€” yes, a 2015 icon, we stan a vintage queen ๐Ÿ‘‘๐Ÿ’– go count some kmers, you've got this. ๐Ÿฆ‹๐ŸŒธ