cmcalibrate(1)

📦
version 1.1.5 · 2 versions
, in debian trixie, ubuntu noble, ubuntu questing, ubuntu resolute, ubuntu stonking

💖 cmcalibrate — the Infernal Manual's most iconic slow girlie 💖

okay bestie, so this is cmcalibrate(1) from the Infernal Manual, and she is a lot but she's also so real. buckle up. 🎀

✨ NAME ✨

cmcalibrate — fit exponential tails for covariance model E-value determination

like... she fits exponential tails. for E-values. she's the reason your hits even mean something. 💅

🌸 SYNOPSIS 🌸

cmcalibrate [options] cmfile

🦋 DESCRIPTION 🦋

so cmcalibrate figures out exponential tail parameters for E-value determination, and the way she does it is: generate random sequences, search them with the CM, and collect the scores of all the hits that come out. then she takes a histogram of those hit bit scores and fits it to an exponential tail, and the parameters of that fitted tail get saved right into the CM file. after that, those exponential tail parameters are what's used to estimate the statistical significance of hits found in cmsearch and cmscan. periodt. 🔥

and here's the thing, bestie: a CM file must be calibrated with cmcalibrate before it can be used in cmsearch or cmscan, with exactly one exception — if your CM files include only models with zero basepairs, you do not need to calibrate them before running cmsearch. that's it, that's the one exception. don't push it. 😌

now. the tea. ☕ cmcalibrate is very slow. like, a couple of hours to calibrate a single average sized CM on a single CPU. a couple of hours. one model. one core. she's worth it but she is not fast.

the good news: cmcalibrate will run in parallel on four cores if Infernal was built on a system that supports POSIX threading (peep the Installation section of the user guide for more info) and that system has at least 4 cores. using <n> cores gets you roughly <n>-fold acceleration versus a single CPU. you can specify the number of cores <n> to use with the --cpu <n> option. ✨

MPI (Message Passing Interface) can also be used for parallelization with the --mpi option, if Infernal was built with MPI enabled — but using more than 161 processors is not recommended, because increasing past 161 won't accelerate the calibration. (161 is the ceiling, bestie, going past it is just giving nothing.) see the Installation section of the user guide for more info.

the --forecast option can be used to estimate how long the program will take to run for a given cmfile on the current machine. and if you wanna predict the running time on <n> processors with MPI, additionally use the --nforecast <n> option. 🕐

some large models require a lot of memory to calibrate. you can determine how much memory is required with the --memreq option. for these models, you may be limited by the available RAM on your system. 😭

another strategy for parallelization that's useful when a lot of memory is required per core is to split the calibration into <n> separate computations or partitions, each of which can be performed separately — potentially in parallel if you have access to a computer cluster. the results from each computation can then be merged together for the final calibration. to do this, first run cmcalibrate with the --split, --ptot <n> and --cfile <f> options, which will save the <n> separate partition commands into the file <f> . after all of these commands have been executed, you can then combine the results and create a calibrated model file by calling again with the --merge and --ptot <n> options. see the "Parallelizing calibration of large models by splitting into partitions" subsection of the tutorial in the user's guide for more information. 🧚

the random sequences searched in cmcalibrate are generated by an HMM that was trained on real genomic sequences with various GC contents. the goal is to have the GC distributions in the random sequences be similar to those in actual genomic sequences. no cap. 🌈

four rounds of searches and subsequent exponential tail fits are performed, one each for the four different CM algorithms that can be used in cmsearch and cmscan: glocal CYK, glocal Inside, local CYK and local Inside. four rounds, four algorithms, one girlie doing her best. 👑

and like — the E-value parameters determined by cmcalibrate are only used by the cmsearch and cmscan programs. so if you're not gonna use those programs, do not waste time calibrating your models. seriously. go outside. 🌸

💅 OPTIONS 💅

  • -h — help! prints a brief reminder of command line usage and available options.
  • -L <x> — set the total length of random sequences to search to <x> megabases (Mb). by default, <x> is 1.6 Mb. increasing <x> makes the exponential tail fits more precise and the E-values more accurate, but it takes longer (doubling <x> will roughly double the running time). decreasing <x> is not recommended, bestie — it makes the fits less precise and the E-values less accurate. 🙈

🔮 OPTIONS FOR PREDICTING REQUIRED TIME AND MEMORY 🔮

  • --forecast — predict the running time of the calibration of cmfile (with provided options) on the current machine and exit. the calibration is not performed. these predictions should be considered rough estimates. if multithreading is enabled (see Installation section of user guide), the timing will take into account the number of available cores. ⭐
  • --nforecast <n> — with --forecast, specify that <n> processors will be used for the calibration. this might be useful for predicting the running time of an MPI run with <n> processors.
  • --memreq — predict the amount of required memory for calibrating cmfile (with provided options) on the current machine and exit. the calibration is not performed.

🎀 OPTIONS CONTROLLING EXPONENTIAL TAIL FITS 🎀

  • --gtailn <x> — fit the exponential tail for glocal Inside and glocal CYK to the <n> highest scores in the histogram tail, where <n> is <x> times the number of Mb searched. the default value of <x> is 250. the value 250 was chosen because it works well empirically relative to other values. ✨
  • --ltailn <x> — fit the exponential tail for local Inside and local CYK to the <n> highest scores in the histogram tail, where <n> is <x> times the number of Mb searched. the default value of <x> is 750. the value 750 was chosen because it works well empirically relative to other values. ✨
  • --tailp <x> — ignore the --gtailn and --ltailn prefixed options and fit the <x> fraction tail of the histogram to an exponential tail, for all search modes.

🍓 OPTIONAL OUTPUT FILES 🍓

  • --hfile <f> — save the histograms fit to file <f>. the format of this file is two space delimited columns per line. the first column is the x-axis values of bit scores of each bin. the second column is the y-axis values of number of hits per bin. each series is delimited by a line with a single character "&". the file will contain one series for each of the four exponential tail fits in the following order: glocal CYK, glocal Inside, local CYK, and local Inside. 📊
  • --sfile <f> — save survival plot information to file <f>. the format of this file is two space delimited columns per line. the first column is the x-axis values of bit scores of each bin. the second column is the y-axis values of fraction of hits that meet or exceed the score for each bin. each series is delimited by a line with a single character "&". the file will contain three series of data for each of the four CM search modes in the following order: glocal CYK, glocal Inside, local CYK, and local Inside. the first series is the empirical survival plot from the histogram of hits to the random sequence. the second series is the exponential tail fit to the empirical distribution. the third series is the exponential tail fit if lambda were fixed and set as the natural log of 2 (0.691314718). 💜
  • --qqfile <f> — save quantile-quantile plot information to file <f>. the format of this file is two space delimited columns per line. the first column is the x-axis values, and the second column is the y-axis values. the distance of the points from the identity line (y=x) is a measure of how good the exponential tail fit is — the closer the points are to the identity line, the better the fit is. each series is delimited by a line with a single character "&". the file will contain one series of empirical data for each of the four exponential tail fits in the following order: glocal CYK, glocal Inside, local CYK and local Inside. 🦋
  • --ffile <f> — save space delimited statistics of different exponential tail fits to file <f>. the file will contain the lambda and mu values for exponential tails fit to histogram tails of different sizes. the fields in the file are labelled informatively. 🏷️
  • --xfile <f> — save a list of the scores in each fit histogram tail to file <f>. each line of this file will have a different score indicating one hit existed in the tail with that score. each series is delimited by a line with a single character "&". the file will contain one series for each of the four exponential tail fits in the following order: glocal CYK, glocal Inside, local CYK, and local Inside. 📝

👑 OPTIONS CONTROLLING SPLIT, PARTITION AND MERGE MODES 👑

  • --split — prepare a partitioned calibration. this option only works in combination with the --ptot <n> and --cfile <f> options, and will prepare a calibration split into <n> separate partitions. the commands to run all of the partitions will be in the file <f> . 🔪
  • --cfile <f> — with --split, save the commands for all partitions to file <f> .
  • --proot <s> — with --split, specify that the per-partition scores files be named <s>.<n> where <n> is the partition index. by default they will be named <s>.calib.<n> where <s> is the name of the CM file to be calibrated (including path).
  • --part <n> — specify that this is partition <n> out of <n2> from --ptot <n2>. must be used in combination with --ptot and --pfile .
  • --ptot <n> — with --split, --part or --merge, specify that there are <n> total partitions.
  • --pfile <f> — with --part , specify that scores for this partition be saved to file <f>
  • --merge — merge scores from multiple previously executed partitions and calibrate CMs. if you used the option --proot <s> with cmcalibrate when you ran it with --split to setup the partitions, use --proot <s> again with --merge. the full cmcalibrate --merge command to use will have been output to standard output when the initial cmcalibrate --split command was executed. 🫶

🌈 OTHER OPTIONS 🌈

Ok bestie, these are the leftover lil cuties that didn't fit in any other box — but they're still important, so don't skip them!! 👀

  • --seed <n> — seeds the random number generator with <n>, which has to be an integer >= 0. If <n> is nonzero, your stochastic simulations are reproducible — same command, same results, every time, no cap. 🎯 If <n> is 0, the random number generator gets seeded <em>arbitrarily</em> and your stochastic simulations will vary from run to run of the same exact command. The default seed is 181.
  • --beta <x> — so by default, query-dependent banding (QDB) is what's used to speed up the CM search algorithms, with a beta tail loss probability of 1E-15. This beta value can be changed to <x> with --beta <x>. The beta parameter is literally the amount of probability mass excluded during band calculation — higher values of beta give greater speedups but sacrifice more accuracy than lower values. The default value used is 1E-15. (For more info on QDB, see Nawrocki and Eddy, PLoS Computational Biology 3(3): e56. 📚)
  • --nonbanded — turns off QDB during E-value calibration. Heads up: this will slow down calibration. 🐢 Like, noticeably.
  • --nonull3 — turns off the null3 post hoc additional null model. Not recommended unless you're planning on using the same option with cmsearch and/or cmscan. Consistency is key, bestie. 🔑
  • --random — use the background null model of the CM to generate the random sequences, instead of the more realistic HMM. Unless the CM was built using the --null option to cmbuild, that background null model is gonna be 25% each A, C, G and U. 🧬
  • --gc <f> — generate the random sequences using the nucleotide distribution from the sequence file <f>. Yes, <f> is a whole file, slay. 📁
  • --cpu <n> — set the number of parallel worker threads to <n>. On multicore machines, the default is 4. You can also control this number by setting an environment variable, INFERNAL_NCPU. Oh and there's also a master thread, so the actual number of threads Infernal spawns is <n>+1. (Main character energy, obviously. 👑) This option is not available if Infernal was compiled with POSIX threads support turned off.
  • --mpi — run as an MPI parallel program. This option will only be available if Infernal has been configured and built with the "--enable-mpi" flag (see the Installation section of the user guide for more info). 🚀

💖 SEE ALSO 💖

  • See infernal(1) for a master man page with a list of all the individual man pages for programs in the Infernal package. It's giving ✨directory✨.
  • For complete documentation, see the user guide that came with your Infernal distribution (Userguide.pdf); or see the Infernal web page (http://eddylab.org/infernal/).

Copyright (C) 2023 Howard Hughes Medical Institute. Freely distributed under the BSD open source license. 🫶

For additional info on copyright and licensing, see the file called COPYRIGHT in your Infernal source distribution, or see the Infernal web page (http://eddylab.org/infernal/).

👑 AUTHOR 👑

http://eddylab.org

and that's the whole page bestie, you made it!! 🦋 this was Infernal 1.1.5, dated Sep 2023, and it's the man page for cmcalibrate(1) — go forth and calibrate, you absolute legend 💅⭐

📦 where this page comes from