esl-alimask(1)

📦
version 3.3.2 · 2 versions
, in fedora 43, fedora 44

✨ esl-alimask ✨ — from the Easel Manual, section 1

okay bestie, buckle up, because this one is a whole thing and I'm obsessed 💖

NAME

esl-alimask — remove columns from a multiple sequence alignment

like. it masks. it's right there in the name. 💅

SYNOPSIS

esl-alimask [options] msafile maskfile
  (remove columns based on a mask in an input file)

esl-alimask -t [options] msafile coords
  (remove a contiguous set of columns at the start and end of an alignment)

esl-alimask -g [options] msafile
  (remove columns based on their frequency of gaps)

esl-alimask -p [options] msafile
  (remove columns based on their posterior probability annotation)

esl-alimask --rf-is-mask [options] msafile
  (only remove columns that are gaps in the RF annotation)

and the little footnote that matters: the -g and -p options may be used in combination. 🫶

DESCRIPTION

esl-alimask reads one single input alignment, removes some columns from it (that's the masking, periodt), and outputs the masked alignment. 🎀

and like, it's got multiple modes — it's giving versatility, it's giving main character energy. let me walk you through all of them.

mask file mode (the default babe)

esl-alimask runs in "mask file mode" by default when two command-line arguments (msafile and maskfile) are supplied.

in this mode, a bit-vector mask in the maskfile defines which columns to keep/remove. the mask is a string that may only contain the characters '0' and '1' — no other vibes allowed.

  • a '0' at position x of the mask means column x is excluded by the mask and should be removed during masking.
  • a '1' at position x of the mask means column x is included by the mask and should not be removed during masking.

all lines in the maskfile that begin with '#' are considered comment lines and are ignored. 💤 all non-whitespace characters in non-comment lines are considered to be part of the mask.

the length of the mask must equal either the total number of columns in the (first) alignment in msafile, or the number of columns that are not gaps in the RF annotation of that alignment.

the latter case is only valid if msafile is in Stockholm format and contains '#=GC RF' annotation. if the mask length is equal to the non-gap RF length, all gap RF columns will automatically be removed. slay, no extra work. 💅

truncation mode

esl-alimask runs in "truncation mode" if the -t option is used along with two command line arguments (msafile and coords).

in this mode, the alignment gets truncated by removing a contiguous set of columns from the beginning and end of the alignment. the second command line argument is the coords string, which specifies what range of columns to keep — everything outside that range gets removed.

the coords string is start and end coordinates separated by any nonnumeric, nonwhitespace character or characters you like. for example, 23..100, 23/100, or 23-100 all work. iconic. ✨

to keep all alignment columns beginning at 23 until the end of the alignment, you can omit the end; for example, 23: would work. 🦋

if the --t-rf option is used in combination with -t, the coordinates in coords are interpreted as non-gap RF column coordinates. for example, with --t-rf, a coords string of 23-100 would remove all columns before the 23rd non-gap residue in the "#=GC RF" annotation and after the 100th non-gap RF residue.

RF mask mode

esl-alimask runs in "RF mask" mode if the --rf-is-mask option is enabled.

in this mode, the alignment must be in Stockholm format and contain '#=GC RF' annotation. esl-alimask will simply remove all columns that are gaps in the RF annotation. that's it. no thoughts, just masking. 💀

gap frequency mode

esl-alimask runs in "gap frequency mode" if -g is enabled.

in this mode, columns for which greater than <f> fraction of the aligned sequences have gap residues will be removed. by default, <f> is 0.5, but this value can be changed to <f> with the --gapthresh <f> option.

in this mode, if the alignment is in Stockholm format and has RF annotation, then all columns that are gaps in the RF annotation will automatically be removed — unless --saveins is enabled.

posterior probability mode

esl-alimask runs in "posterior probability mode" if -p is enabled.

in this mode, masking is based on posterior probability annotation, and the input alignment must be in Stockholm format and contain '#=GR PP' (posterior probability) annotation for all sequences.

as a special case, if -p is used in combination with --ppcons, then the input alignment need not have '#=GR PP' annotation, but must contain '#=GC PP_cons' (posterior probability consensus) annotation. plot twist but make it cute 🙈

the 12 possible PP values, explained

characters in Stockholm alignment posterior probability annotation (both '#=GR PP' and '#=GC PP_cons') can have 12 possible values: the ten digits '0-9', '*', and '.'.

  • if '.', the position must correspond to a gap in the sequence (for '#=GR PP') or in the RF annotation (for '#=GC PP_cons').
  • '0' indicates a posterior probability of between 0.0 and 0.05
  • '1' indicates between 0.05 and 0.15
  • '2' indicates between 0.15 and 0.25
  • and so on up to '9', which indicates between 0.85 and 0.95
  • '*' indicates a posterior probability of between 0.95 and 1.0

higher posterior probabilities correspond to greater confidence that the aligned residue belongs where it appears in the alignment. high confidence = highkey iconic. 👑

with --ppcons <x>

when -p is enabled with --ppcons <x>, columns which have a consensus posterior probability of less than <x> will be removed during masking, and all other columns will not be removed.

without --ppcons (the counting era)

when -p is enabled without --ppcons, the number of each possible PP value in each column is counted.

if <x> fraction of the sequences that contain aligned residues (i.e. do not contain gaps) in a column have a posterior probability greater than or equal to <y>, then that column will not be removed during masking. all columns that do not meet this criterion will be removed.

by default, the values of both <x> and <y> are 0.95, but they can be changed with the --pfract <x> and --pthresh <y> options, respectively. 💅

the all-gap situation

in posterior probability mode, all columns that have 0 residues (i.e. that are 100% gaps) will be automatically removed — unless the --pallgapok option is enabled, in which case such columns will not be removed.

min vs avg — the important one, don't skip this bestie 😭

importantly, during posterior probability masking, unless --pavg is used, PP annotation values are always considered to be the <strong>minimum</strong> numerical value in their corresponding range.

for example, a PP '9' character is converted to a numerical posterior probability of 0.85.

if --pavg is used, PP annotation values are considered to be the <strong>average</strong> numerical value in their range. for example, a PP '9' character is converted to a numerical posterior probability of 0.90. ✨

RF + PP

in posterior probability mode, if the alignment is in Stockholm format and has RF annotation, then all columns that are gaps in the RF annotation will automatically be removed — unless --saveins is enabled.

both at once (the combo meal)

a single run of esl-alimask can perform <strong>both</strong> gap frequency-based masking and posterior probability-based masking if both the -g and -p options are enabled.

in this case, a gap frequency-based mask and a posterior probability-based mask are independently computed. these two masks are combined to create the final mask using a logical 'and' operation.

any column that is to be removed by <strong>either</strong> the gap or PP mask will be removed by the final mask. no mercy. 🔥

--small — the memory-saving era

with the --small option, esl-alimask will operate in memory saving mode and the required RAM for the masking will be minimal (usually less than a Mb) and independent of the alignment size. it's giving lightweight, it's giving efficiency, we love her. 🧚

to use --small, the alignment alphabet must be specified with either --amino, --dna, or --rna, and the alignment must be in Pfam format (non-interleaved, 1 line/sequence Stockholm format). Pfam format is the default output format of INFERNAL's cmalign program.

without --small the required RAM will be equal to roughly the size of the first input alignment (the size of the alignment file itself if it only contains one alignment).

OUTPUT

by default, esl-alimask will print only the masked alignment to stdout and then exit.

if the -o <f> option is used, the alignment will be saved to file <f>, and information on the number of columns kept and removed will be printed to stdout.

if -q is used in combination with -o, nothing is printed to stdout. silence. peace. 🎀

saving your masks

the mask(s) computed by esl-alimask when the -t, -p, -g, or --rf-is-mask options are used can be saved to output files using the options --fmask-rf <f>, --fmask-all <f>, --gmask-rf <f>, --gmask-all <f>, --pmask-rf <f>, and --pmask-all <f>.

in all cases, <f> will contain a single line, a bit vector of length <n>, where <n> is either:

  • the total number of columns in the alignment (for the options suffixed with 'all'), or
  • the number of non-gap columns in the RF annotation (for the options suffixed with 'rf').

the mask will be a string of '0' and '1' characters:

  • a '0' at position x in the mask indicates column x was removed (excluded) by the mask
  • a '1' at position x indicates column x was kept (included) by the mask

for the 'rf' suffixed options, the mask only applies to non-gap RF columns.

and the naming lore, because iykyk:

  • the options beginning with 'f' will save the 'final' mask used to keep/remove columns from the alignment.
  • the options beginning with 'g' save the masks based on gap frequency and require -g.
  • the options beginning with 'p' save the masks based on posterior probabilities and require -p.

✨ to be continued, bestie, there's more where this came from ✨

✨ OPTIONS ✨

  • -h — prints a lil brief help thing, and yes it includes the version number plus a summary of all the options, even the expert ones 👑
  • -o <f> — sends the final masked alignment to file <f> instead of stdout. when you use this, the info about how many columns got kept/removed goes to stdout 📊
  • -q — be quiet bestie, do not print anything to stdout. only works if you also use -o 🚫 (can only be used in combination with the -o option)
  • --small — memory saving mode activated ✨ required RAM becomes independent of the size of the input alignment to mask, instead of roughly the size of the input alignment. when enabled, the alignment must be in Pfam Stockholm (non-interleaved 1 line/seq) format (see esl-reformat) and the output alignment will be in Pfam format.
  • --informat <s> — assert that input msafile is in alignment format <s>. common choices for <s> include: stockholm, a2m, afa, psiblast, clustal, phylip. for more info, and for codes for some less common formats, see main documentation. the string <s> is case-insensitive (a2m or A2M both work). default is stockholm format, unless --small is used, in which case pfam format (non-interleaved Stockholm) is assumed.
  • --outformat <s> — write the output msafile in alignment format <s>. common choices for <s> include: stockholm, a2m, afa, psiblast, clustal, phylip. the string <s> is case-insensitive (a2m or A2M both work). default is stockholm, unless --small is enabled, in which case pfam (noninterleaved Stockholm) is the default output format.
  • --fmask-rf <f> — save the non-gap RF-length final mask used to mask the alignment to file <f>. the input alignment must be in Stockholm format and contain '#=GC RF' annotation for this option to be valid. see the OUTPUT section above for more details on output mask files. 💾
  • --fmask-all <f> — save the full alignment-length final mask used to mask the alignment to file <f>. see the OUTPUT section above for more details on output mask files. 💾
  • --amino — specify that the input alignment is a protein alignment. by default, esl-alimask will try to autodetect the alphabet, but if the alignment is sufficiently small it may be ambiguous. this option defines the alphabet as protein. importantly, if --small is enabled, the alphabet must be specified with either --amino, --dna, or --rna. 🧬
  • --dna — specify that the input alignment is a DNA alignment. 🧬
  • --rna — specify that the input alignment is an RNA alignment. 🧬
  • --t-rf — with -t, specify that the start and end coordinates defined in the second command line argument coords correspond to non-gap RF coordinates. to use this option, the alignment must be in Stockholm format and have "#=GC RF" annotation. see the DESCRIPTION section for an example of using the --t-rf option. 📍
  • --t-rmins — with -t, specify that all columns that are gaps in the reference (RF) annotation in between the specified start and end coordinates be removed. by default, these columns will be kept. to use this option, the alignment must be in Stockholm format and have "#=GC RF" annotation. ✂️
  • --gapthresh <x> — with -g, specify that a column is kept (included by mask) if no more than <f> fraction of sequences in the alignment have a gap ('.', '-', or '_') at that position. all other columns are removed (excluded by mask). by default, <x> is 0.5. (yes the manpage says <f> right there, bestie, we keep it 💯)
  • --gmask-rf <f> — save the non-gap RF-length gap frequency-based mask used to mask the alignment to file <f>. the input alignment must be in Stockholm format and contain '#=GC RF' annotation for this option to be valid. see the OUTPUT section above for more details on output mask files. 💾
  • --gmask-all <f> — save the full alignment-length gap frequency-based mask used to mask the alignment to file <f>. see the OUTPUT section above for more details on output mask files. 💾
  • --pfract <x> — with -p, specify that a column is kept (included by mask) if the fraction of sequences with a non-gap residue in that column with a posterior probability of at least <y> (from --pthresh <y>) is <x> or greater. all other columns are removed (excluded by mask). by default <x> is 0.95. 📊
  • --pthresh <y> — with -p, specify that a column is kept (included by mask) if <x> (from --pfract <x>) fraction of sequences with a non-gap residue in that column have a posterior probability of at least <y>. all other columns are removed (excluded by mask). by default <y> is 0.95. see the DESCRIPTION section for more on posterior probability (PP) masking. due to the granularity of the PP annotation, different <y> values within a range covered by a single PP character will have the same effect on masking. for example, using --pthresh 0.86 will have the same effect as using --pthresh 0.94. 🎯
  • --pavg <x> — with -p, specify that a column is kept (included by mask) if the average posterior probability of non-gap residues in that column is at least <x>. see the DESCRIPTION section for more on posterior probability (PP) masking. 📈
  • --ppcons <x> — with -p, use the '#=GC PP_cons' annotation to define which columns to keep/remove. a column is kept (included by mask) if the PP_cons value for that column is <x> or greater. otherwise it is removed. ✂️
  • --pallgapok — with -p, do not automatically remove any columns that are 100% gaps (i.e. contain 0 aligned residues). by default, such columns will be removed. 🕳️
  • --pmask-rf <f> — save the non-gap RF-length posterior probability-based mask used to mask the alignment to file <f>. the input alignment must be in Stockholm format and contain '#=GC RF' annotation for this option to be valid. see the OUTPUT section above for more details on output mask files. 💾
  • --pmask-all <f> — save the full alignment-length posterior probability-based mask used to mask the alignment to file <f>. see the OUTPUT section above for more details on output mask files. 💾
  • --keepins — if -p and/or -g is enabled and the alignment is in Stockholm or Pfam format and has '#=GC RF' annotation, then allow columns that are gaps in the RF annotation to possibly be kept. by default, all gap RF columns would be removed automatically, but with this option enabled gap and non-gap RF columns are treated identically. to automatically remove all gap RF columns when using a maskfile, then define the mask in maskfile as having length equal to the non-gap RF length in the alignment. to automatically remove all gap RF columns when using -t, use the --t-rmins option. 👑

🌈 SEE ALSO 🌈

Copyright (C) 2020 Howard Hughes Medical Institute. Freely distributed under the BSD open source license. 💖

👑 AUTHOR 👑

that's it bestie! this iconic page was brought to you by Easel 0.48, Nov 2020, for esl-alimask(1). periodt 💅✨

📦 where this page comes from