XTRACT(1)

📦
version 24.0.20250523 · 4 versions
, in debian trixie, ubuntu questing, ubuntu resolute, ubuntu stonking

💖 XTRACT(1) 💖

NCBI Entrez Direct User's Manual

🌸 NAME 🌸

xtract — NCBI Entrez Direct XML conversion and transformation tool. bestie, it is literally the XML-to-table glow-up station. 💅

🎀 SYNOPSIS 🎀

okay so here is the whole command-line vibe, no cap:

       xtract [-help] [-strict] [-mixed] [-self] [-accent] [-ascii] [-compress]
       [-stops] [-input filename] [-transform filename] [-aliases filename]
       [-pattern expr] [-group expr] [-block expr] [-subset expr] [-path path]
       [-if expr [constraint]] [-unless expr [constraint]] [-and condition]
       [-or condition] [-else] [-position pos] [-equals str]
       [-contains str] [-mimics str] [-excludes str] [-includes str]
       [-is-within str] [-starts-with str] [-ends-with str] [-is-not str]
       [-is-before str] [-is-after str] [-consists-of str] [-matches str]
       [-resembles str] [-is-equal-to expr] [-differs-from expr] [-gt N] [-ge N]
       [-lt N] [-le N] [-eq N] [-ne N] [-ret str] [-tab str] [-sep str]
       [-pfx str] [-sfx str] [-rst] [-clr] [-pfc str] [-deq str] [-def str]
       [-lbl str] [-set tag] [-rec tag] [-wrp tag] [-enc tag] [-plg str]
       [-elg str] [-pkg tag] [-fwd str] [-awd str] [-tag tag] [-att key str]
       [-atr key element] [-cls] [-slf] [-end tag] [-bkt] [-element element]
       [-first element] [-last element] [-first element] [-last element]
       [-backward element] [-NAME] [--STATS] [-num element] [-len element]
       [-sum element] [-acc element] [-min element] [-max element] [-inc element]
       [-dec element] [-sub element] [-avg element] [-dev element] [-med element]
       [-mul element] [-div element] [-mod element] [-geo element] [-hrm element]
       [-rms element] [-sqt element] [-lge element] [-lg2 element] [-log element]
       [-bin element] [-oct element] [-hex element] [-bit element] [-pad element]
       [-encode element] [-decode element] [-upper element] [-lower element]
       [-chain element] [-title element] [-mirror element] [-alpha element]
       [-alnum element] [-basic element] [-plain element] [-simple element]
       [-author element] [-journal element] [-prose element] [-terms element]
       [-words element] [-pairs element] [-split element -with str]
       [-order element] [-reverse element] [-letters element] [-clauses element]
       [-pentamers element] [-year element] [-month element] [-date element]
       [-page element] [-auth element] [-initials element] [-trim element]
       [-wct element] [-doi element] [-accession element] [-numeric element]
       [-translate element] [-classify element] [-replace
       -reg target -exp replacement] [-fasta] [-revcomp] [-nucleic] [-ncbi2na]
       [-ncbi4na] [-cds2prot [-gcode N] [-frame N]] [-molwt] [-molwt-m]
       [-molwt-f] [-pept] [-0-based element] [-1-based element]
       [-ucsc-based element] [-insd arg ...] [-insdx] [-histogram]
       [-indexer element] [-head str] [-tail str] [-hd str] [-tl str]
       [-select condition] [-in filename] [-sort[-fwd] element]
       [-sort-rev element] [-format fmt [-unicode style]] [-verify] [-test]
       [-outline] [-synopsis] [-contour [delimiter]] [-examples] [-unix]
       [-version]

💅 DESCRIPTION 💅

xtract converts an XML document into a table of data values according to user-specified rules. that is the whole assignment, bestie. iconic behavior only. ✨

👑 OPTIONS 👑

✨ Processing Flags ✨

  • -strict — Remove HTML and MathML tags. 🧼
  • -mixed — Allow mixed content XML. 🌀
  • -self — Allow detection of empty self-closing tags. 🪞
  • -accent — Delete Unicode accents and diacritical marks. 💅
  • -ascii — Convert Unicode to numeric HTML character entities. 🔢
  • -compress — Compress runs of spaces. 🫧
  • -stops — Retain stop words in selected phrases. 🛑

📂 Data Source 📂

  • -input filename — Read XML from file instead of standard input. 📥
  • -transform filename — File of substitutions for -translate. 🔁
  • -aliases filename — Mappings file for -classify operation. 🗂️

🧭 Exploration Argument Hierarchy 🧭

  • -pattern expr — Name of record within set. Use of different argument names allows command-line control of nested looping. 🪆
  • -group expr — Name of record within set. Use of different argument names allows command-line control of nested looping. 🪆
  • -block expr — Name of record within set. Use of different argument names allows command-line control of nested looping. 🪆
  • -subset expr — Name of record within set. Use of different argument names allows command-line control of nested looping. 🪆

🛤️ Path Navigation 🛤️

  • -path path — Explore by list of adjacent object names. 🧭

🧪 Exploration Constructs 🧪

  • Object — DateRevised
  • Parent/Child — Book/AuthorList
  • Path — MedlineCitation/Article/Journal/JournalIssue/PubDate
  • Heterogeneous — "PubmedArticleSet/*"
  • Exhaustive — "History/**"
  • Nested — "*/Taxon"

🚦 Conditional Execution 🚦

  • -if expr [constraint] — Element (or @attribute) must exist and satisfy any specified constraint. ✅
  • -unless expr [constraint] — Skip if element matches. 🚫
  • -and condition — Preceding and following tests must both pass. 🤝
  • -or condition — Any passing test suffices. 🎯
  • -else — Execute if conditional test failed. ↩️
  • -position pos — first/last/outer/inner/even/odd/all. 📍

🔤 String Constraints 🔤

  • -equals str — String must match exactly. 🎯
  • -contains str — Substring must be present. 🔍
  • -mimics str — Containment test after converting punctuation to space. 🎭
  • -excludes str — Substring must be absent. 🚫
  • -includes str — Substring must match at word boundaries. 🧱
  • -is-within str — String must be present. 📍
  • -starts-with str — Substring must be at beginning. 🚀
  • -ends-with str — Substring must be at end. 🏁
  • -is-not str — String must not match. 🙅
  • -is-before str — First string < second string. ⏪
  • -is-after str — First string > second string. ⏩
  • -consists-of str — String must only contain specified characters. 🧩
  • -matches str — Matches without commas or semicolons. ✂️
  • -resembles str — Requires all words, but in any order. 🌀

🧩 Object Constraints 🧩

  • -is-equal-to expr — Object values must match. 🟰
  • -differs-from expr — Object values must differ. ↔️

🔢 Numeric Constraints 🔢

  • -gt N — Greater than. ⬆️
  • -ge N — Greater than or equal to. ⏫
  • -lt N — Less than to. ⬇️
  • -le N — Less than or equal to. ⏬
  • -eq N — Equal to. 🟰
  • -ne N — Not equal to. 🚫

🎨 Format Customization 🎨

  • -ret str — Override line break between patterns. ↩️
  • -tab str — Replace tab character between fields. 📑
  • -sep str — Separator between group members. ➗
  • -pfx str — Prefix to print before group. 🎀
  • -sfx str — Suffix to print after group. 🎀
  • -rst — Reset -sep through -elg. 🔄
  • -clr — Clear queued tab separator. 🧽
  • -pfc str — Preface combines -clr and -pfx. 🧬
  • -deq str — Delete and replace queued tab separator. 🔁
  • -def str — Default placeholder for missing fields. 🫥
  • -lbl str — Insert arbitrary text. ✍️

🏷️ XML Generation 🏷️

  • -set tag — XML tag for entire set. 🧺
  • -rec tag — XML tag for each record. 🎫
  • -wrp tag — Wrap elements in XML object. 🎁
  • -enc tag — Encase instance in XML object. 📦
  • -plg str — Prologue to print before instance. 🎬
  • -elg str — Epilogue to print after instance. 🎬
  • -pkg tag — Package subset in XML object. 📦
  • -fwd str — Foreword to print before subset. ⏮️
  • -awd str — Afterword to print after subset. ⏭️

🔖 Tag and Attribute Construction 🔖

  • -tag tag — Start with <tag. 🏁
  • -att key str — Attribute key and literal string. 🔑
  • -att key element — Attribute key and element name. 🔑
  • -cls — Close with >. 🔚
  • -slf — Self-close with />. 🪞
  • -end tag — End contents with </tag>. 🛑

🧬 FASTA Parsable Fields 🧬

  • -bkt — Wrap elements in bracketed fields. 🧫

🎯 Element Selection 🎯

  • -element element — Print all items that match tag name. 🖨️
  • -first element — Only print value of first item. 🥇
  • -last element — Only print value of last item. 🥉
  • -even element — Only print value of even items. 2️⃣
  • -odd element — Only print value of odd items. 1️⃣
  • -backward element — Print values in reverse order. 🔙
  • -NAME — Record value in named variable. 🏷️
  • --STATS — Accumulate values into variable. 📊

🧱 -element Constructs 🧱

  • Tag — Caption
  • Group — Initials,LastName
  • Parent/Child — MedlineCitation/PMID
  • Recursive — "**/Gene-commentary_accession"
  • Unrestricted — PubDate/*
  • Attribute — DescriptorName@MajorTopicYN
  • Range — MedlineDate[1:4]
  • Substring — "Title[phospholipase | rattlesnake]"
  • Alternative — "[can contain ^ vertical bar]"
  • Object Count — #Author
  • Item Length — %Title
  • Element Depth — ^PMID
  • Variable — "&NAME"

🪄 Special -element Operations 🪄

  • Parent Index — +
  • Object Name — ?
  • Object Value — ~
  • XML Subtree — *
  • Children — $
  • Attributes — @
  • ASN.1 Record — .
  • JSON Record — %

➕ Numeric (Integer) Processing ➕

  • -num element — Count. 🔢
  • -len element — Length. 📏
  • -sum element — Sum. ➕
  • -acc element — Accumulator. 🧮
  • -min element — Minimum. 🔽
  • -max element — Maximum. 🔼
  • -inc element — Increment. ⏫
  • -dec element — Decrement. ⏬
  • -sub element — Difference. ➖
  • -avg element — Arithmetic mean. ⚖️
  • -dev element — Deviation. 📉
  • -med element — Median. 🎯
  • -mul element — Product. ✖️
  • -div element — Quotient. ➗
  • -mod element — Remainder. 🧩
  • -geo element — Geometric mean. 📐
  • -hrm element — Harmonic mean. 🎵
  • -rms element — Root mean square. 🧮
  • -sqt element — Square root. √
  • -lge element — Natural logarithm. 🌿
  • -lg2 element — Logarithm base two. 2️⃣
  • -log element — Logarithm base ten. 🔟
  • -bin element — Binary. 0️⃣1️⃣
  • -oct element — Octal. 8️⃣
  • -hex element — Hexadecimal. 🔢
  • -bit element — Number of bits set. 🧮

0️⃣ Leading Zero Padding 0️⃣

  • -pad element — Zero-pad to eight digits. 0️⃣0️⃣0️⃣0️⃣0️⃣0️⃣0️⃣0️⃣

🔡 Character Processing 🔡

  • -encode element — XML-encode <, >, &, ", and ' characters. 🔐
  • -decode element — Base64-decode object embedded in XML. 🔓
  • -upper element — Convert text to uppercase. 🔠
  • -lower element — Convert text to lowercase. 🔡
  • -chain element — Change spaces to underscores. 🔗
  • -title element — Capitalize initial letters of words. 📝
  • -mirror element — Reverse order of letters. 🪞
  • -alnum element — Non-alphabetic characters to space. 🔤
  • -alnum element — Non-alphanumeric characters to space. 🔤
bestie, yes, -alnum shows up twice in the original manpage. we respect the lore. 💅

📝 String Processing 📝

  • -basic element — Convert superscripts and subscripts. 🔢
  • -plain element — Remove embedded mixed-content markup tags. 🧽
  • -simple element — Normalize accented letters; spell Greek letters. 🇬🇷
  • -author element — Multi-step author cleanup. ✍️
  • -jour element — Journal capitalization and punctuation punctuation. 📚
  • -prose element — Text conversion to ASCII. 🔤

📚 Text Processing 📚

  • -terms element — Partition text at spaces. ✂️
  • -words element — Split at punctuation marks. 🔤
  • -pairs element — Adjacent informative words. 👯
  • -split element — Split using -with for delimiter. 🔪
  • -order element — Rearrange words in sorted order. 🔤
  • -reverse element — Reverse words in string. 🔙
  • -letters element — Separate individual letters. 🔡
  • -clauses element — Break at phrase separators. ✂️
  • -pentamers element — Sliding window of pentamers. 5️⃣

📅 Citation Functions 📅

  • -year element — Extract first 4-digit year from string. 📆
  • -month element — Match first month name and return a corresponding integer. 🗓️
  • -date element — YYYY/MM/DD from -unit "PubDate" -date "*" 📅
  • -page element — Get digits (and letters) of first page number. 📄
  • -auth element — Change GenBank authors to Medline form. ✍️
  • -initials element — Parse initials from forename or given name. 🔤
  • -trim element — strips out the extra spaces and the leading zeros. glow-up for your strings ✨
  • -wct element — counts how many -words are in a string. counting era 💅
  • -doi element — slaps the https://doi.org/ prefix on there and URL encodes it. iconic.
  • -accession element — lets you index the full accession.version. no cap.
  • -numeric element — only accepts items that are entirely digits. nothing else gets in, periodt.

🌸 Value Transformation 🌸

  • -translate element — swaps values out using the -transform table.
  • -classify element — matches substrings (word or phrase) against the -aliases table.

🎀 Regular Expression 🎀

  • -replace — substitutes text using regular expressions. regex girlies rise 🫶
    • -reg target — the target expression.
    • -exp pattern — the replacement pattern.

🦋 Sequence Processing 🦋

  • -fasta — splits the sequence into blocks of 70 uppercase letters. seventy. exactly.

💜 Nucleotide Processing 💜

  • -revcomp — reverse complements the nucleotide sequence.
  • -nucleic — the subrange decides whether it goes forward or revcomp.
  • -ncbi2na — expands ncbi2na to IUPAC. (You may need to truncate the result to the actual sequence length.) ⚠️ don't forget the truncate bestie
  • -ncbi4na — expands ncbi4na to IUPAC. (Same deal — you may need to truncate the result to the actual sequence length.) ⚠️
  • -cds2prot [-gcode N] [-frame N] — translates the coding region using -gcode and the (1-based) -frame. both default to 1.

🧚 Protein Processing 🧚

  • -molwt — calculates the molecular weight of the peptide.
  • -molwt-m — molecular weight, but keeping the initial methionine.
  • -molwt-f — keeps the initial M residue as formyl-methionine.
  • -pept — splits amino acid runs at *, -, x, or X. yes, both cases, she's thorough 💅

⭐ Sequence Coordinates ⭐

  • -0-based element — zero-based.
  • -1-based element — one-based.
  • -ucsc-based element — half-open.

👑 Command Generator 👑

  • -insd arg ... — generates INSDSeq extraction commands. If you invoke it standalone, it prints them; if you invoke it as part of a pipeline, it runs them. It requires one or more arguments, and those arguments may appear in this order:
    • Descriptor(s) — INSDSeq_sequence/INSDSeq_definition/INSDSeq_division/... [...]
    • Completeness — complete/partial
    • Feature(s) — CDS/mRNA/... [,...]
    • Qualifier(s) — INSDFeature_key/"#INSDInterval"/gene/product/feat_location/sub_sequence/... [...]
  • -insdx — processes the -insd output table into XML.

🍓 Frequency Table 🍓

  • -histogram — collects data for sort-uniq-count(1) across the entire set of records. gathering the receipts fr fr

🌈 Entrez Indexing 🌈

  • -indexer element — positional index using -wrp for the field name.

🎀 Output Organization 🎀

  • -head str — prints before everything else.
  • -tail str — prints after everything else.
  • -hd str — prints before each record.
  • -tl str — prints after each record.

💖 Record Selection 💖

  • -select condition — selects a subset of records by conditions.
  • -in filename — a file of identifiers to use for selection.

🔥 Record Rearrangement 🔥

  • -sort[-fwd] element — the element to use as the sort key.
  • -sort-rev element — sorts records in reverse order.

✨ Reformatting ✨

  • -format fmt
    • copy — fast block copy (it still applies the processing flags).
    • compact — compresses runs of spaces.
    • flush — suppresses line indentation.
    • indent — indents according to nesting depth.
    • expand — places each attribute on a separate line.

🦋 Validation 🦋

  • -verify — reports XML data integrity problems.
  • -test — checks a field for visible combining accents and invisible Unicode. sneaky little characters 🙈

💅 Summary 💅

  • -outline — displays an outline of the XML structure.
  • -synopsis — displays individual XML paths.
  • -contour [delimiter] — displays XML paths to leaf nodes (delimited by / by default).

🌟 Full Exploration Command Precedence 🌟

  • -pattern
  • -path
  • -division
  • -group
  • -branch
  • -block
  • -section
  • -subset
  • -unit

📖 Documentation 📖

  • -help — prints usage information and some example argument combinations.
  • -examples — complete usage examples, involving additional Entrez Direct tools.
  • -unix — illustrates common Unix command arguments.
  • -version — prints the version number.

📝 NOTES 📝

String constraints use case-insensitive comparisons. so don't stress about the caps bestie 🫶

Numeric constraints and selection arguments use integer values.

-num and -len selections are synonyms for Object Count (#) and Item Length (%).

-words, -pairs, and -indices convert to lower case.

👀 SEE ALSO 👀

align-columns(1), archive-nihocc(1), archive-nlmnlp(1), archive-nmcds(1), archive-pids(1), archive-pmc(1), archive-pubmed(1), archive-taxonomy(1), asn2ref(1), between-two-genes(1), bsmp2info(1), csv2xml(1), custom-index(1), disambiguate-nucleotides(1), download-flatfile(1), download-ncbi-data(1), ds2pme(1), efetch(1), esample(1), filter-columns(1), find-in-gene(1), fuse-ranges(1), fuse-segments(1), gbf2facds(1), gbf2fsa(1), gbf2info(1), gbf2tbl(1), gene2range(1), gff2xml(1), gff-sort(1), gm2segs(1), hgvs2spdi(1), nquire(1), pm-collect(1), pm-refresh(1), pma2apa(1), pma2pme(1), pmc2bioc(1), pmc2info(1), print-columns(1), rchive(1), refseq-nm-cds(1), reorder-columns(1), snp2hgvs(1), snp2tbl(1), sort-table(1), sort-uniq-count(1), spdi2tbl(1), tbl2prod(1), transmute(1), uniq-table(1), xfetch(1), xfilter(1), xinfo(1), xlink(1), xml2fsa(1), xml2tbl(1), xsearch(1), xy-plot(1).

and that's the whole page bestie, you made it 🥹🫶 this one's from NCBI, dated 2025-05-26 — go be the main character of your own data pipeline, mwah 💋✨

📦 where this page comes from