Bio::D...ilities(3pm)

💖 Bio::DB::SoapEUtilities 💖

okay bestie, welcome to the Bio::DB::SoapEUtilities(3pm) page — straight out of the User Contributed Perl Documentation, and yes it is giving main character energy.

---

🌸 NAME 🌸

Bio::DB::SoapEUtilities — Interface to the NCBI Entrez web service BETA

bestie it literally says BETA in the name. the vibes are experimental. we love a work in progress 💅

---

✨ SYNOPSIS ✨

okay so here's the whole vibe, in code, exactly as the ancestors wrote it:

use Bio::DB::SoapEUtilities;

# factory construction

my $fac = Bio::DB::SoapEUtilities->new()

# executing a utility call

#get an iteratable adaptor
my $links = $fac->elink(
              -dbfrom => 'protein',
              -db => 'taxonomy',
              -id => \@protein_ids )->run(-auto_adapt => 1);

# get a Bio::DB::SoapEUtilities::Result object
my $result = $fac->esearch(
              -db => 'gene',
              -term => 'sonic and human')->run;

# get the raw XML message
my $xml = $fac->efetch(
            -db => 'gene',
            -id => \@gids )->run( -raw_xml => 1 );

# change parameters
my $new_result = $fac->efetch(
                  -db => 'gene',
                  -id => \@more_gids)->run;
# reset parameters
$fac->efetch->reset_parameters( -db => 'nucleotide',
                                -id => $nucid );
$result = $fac->efetch->run;

# parsing and iterating the results

$count = $result->count;
@ids = $result->ids;

while ( my $linkset = $links->next_link ) {
   $submitted = $linkset->submitted_id;
}

($taxid) = $links->id_map($submitted_prot_id);
$species_io = $fac->efetch( -db => 'taxonomy',
                            -id => $taxid )->run( -auto_adapt => 1);
$species = $species_io->next_species;
$linnaeus = $species->binomial;

like. look at all of that. factory construction, running utility calls, raw XML, parameter glow-ups, iteration. she does it all 🦋

---

🌈 DESCRIPTION 🌈

so what is this module actually doing? This module lets you query the NCBI Entrez database through its SOAP (Simple Object Access Protocol) web service — the official description of that service lives at:

<http://eutils.ncbi.nlm.nih.gov/entrez/eutils/soap/v2.0/DOC/esoap_help.html>

(go read it, iykyk ✨)

the basic tools — einfo, esearch, elink, efetch, espell, epost — are all just methods hanging off a SoapEUtilities factory object. no cap. you build the factory, you call the tools, it's that girl.

now here's the cute part: every tool's parameters can be queried, set, and reset through the standard Bio::ParameterBaseI calls:

• available_parameters()
• set_parameters()
• get_parameters()
• reset_parameters()

periodt. one interface to rule them all.

and the returned data? you can retrieve it, access it, and parse it several different ways, whatever your preference is bestie — the module does not judge. and for the people who love a good iterator moment: adaptors and object iterators are available for efetch, egquery, elink, and esummary results. obsessed. 💖

---

part 1 of 3 — there's so much more coming, don't go anywhere 🫶

🏭 Factory

Okay bestie, first things first: you gotta make yourself a factory. That's the whole entry point, the main character of your script. 💅

my $fac = Bio::DB::SoapEUtilities->new();

From the factory you call utilities, set parameters, and pull back results or adaptors. It's giving one-stop shop.

Now, if you happen to have your own copy of the wsdl sitting around (iykyk), use:

my $fac = Bio::Db::SoapEUtilities->new( -wsdl_file => $my_wsdl );

Otherwise? Chill. The correct one gets fetched over the network by Bio::DB::ESoap and friends. ✨

🧰 Utilities and parameters

To run any of the standard NCBI EUtilities — einfo, esearch, esummary, elink, egquery, epost, espell — you just call the utility you want straight off the factory. To actually use a utility you have to set its parameters and then run it to get a result. TMTOWTDI (there's more than one way to do it, bestie, and we respect all of them):

# verbose
my $fetch = $fac->efetch();
$fetch->set_parameters( -db => 'gene', -id => [828392, 790]);
my $result = $fetch->run;

# compact
my $result = $fac->efetch(-db =>'gene',-id => [828392,790])->run;

# change ids
$fac->efetch->set_parameters( -id => 470338 );
$result = $fac->run;

# another util
$result = $fac->esearch(-db => 'protein', -term => 'BRCA and human')->run;

# the utilities are kept separate
%search_params = $fac->esearch->get_parameters;
%fetch_params = $fac->efetch->get_parameters;
$search_param{db}; # is 'protein'
$fetch_params{db}; # is 'gene'

Note the last bit — the utilities stay separate from each other. Your esearch params and your efetch params do not bleed into one another. No crossover, no drama. 🫶

The factory is Bio::ParameterBaseI compliant, which basically means it's polite and tells you what it'll let you set:

@available_search = $fac->esearch->available_parameters;
@available_egquery = $fac->egquery->available_parameters;

Want the full lore on parameters? Go read http://www.ncbi.nlm.nih.gov/entrez/query/static/eutils_help.html. 📚

---

🎁 Results

The "intermediate" object for SoapEU query results is Bio::DB::SoapEUtilities::Result. It's a BioPerly parsing of the SOAP message NCBI sends back whenever you run() a query. That's genuinely useful on its own, but ngl most of y'all are gonna want to skip straight to Adaptors, which take a Result and glow it up into more intuitive / familiar BioPerl objects. Head down there if the next few paragraphs are too gory. 😭

Results can be highly parsed or lowly parsed, depending on what params you handed to the factory's run() method. Let's break it down:

To get the raw XML message with zero parsing:

my $xml = $fac->$util->run(-raw_xml => 1); # $xml is a scalar string

To get a Bio::DB::SoapEUtilities::Result object with only limited parsing — but still with accessors to the SOAP::SOM message (that's courtesy of SOAP::Lite):

my $result = $fac->$util->run(-no_parse => 1);
my $som = $result->som;
my $method_hash = $som->method; # etc...

And to get a Result object where the message elements are parsed into accessors — including count() and ids() — just run with no arguments at all:

my $result = $fac->esearch->run()
my $count = $result->count;
my @Count = $result->Count; # counts for each member of
                            # the translation stack
my @ids = $result->IdList_Id; # from automatic message parsing
@ids = $result->ids; # a convenient alias

See Bio::DB::SoapEUtilities::Result for even gorier details. 👑

---

🔌 Adaptors

Adaptors convert EUtility Results into convenient objects, via a handle that usually gives you an iterator — very much in the spirit of Bio::SeqIO. For the typical user these are way more useful than the raw Result, so you can just have them auto-magically by setting the run() parameter -auto_adapt =" 1.

In general, you grab an adaptor like this:

$adp = $fac->$util->run( -auto_adapt => 1 );
# iterate...
while ( my $obj = $adp->next_obj ) {
   # do stuff with $obj
}

The adaptor itself sometimes has useful methods beyond the iterator. next_obj always works, but there's always a natural alias too:

$seqio = $fac->esearch->run( -auto_adapt => 1 );
while ( my $seq = $seqio->next_seq ) {
   # do stuff with $seq
}

In that example, -auto_adapt =" 1 also tells the factory to do an efetch based on the ids the esearch handed back (if there are any), so the adaptor you get iterates over Bio::SeqI objects. Chain reaction, no cap. 🔥

Here's the full rundown of adaptor flavors:

• efetch, Fetch Adaptors, and BioPerl object iterators

The FetchAdaptor creates bona fide BioPerl objects. Right now there are FetchAdaptor subclasses for sequence data (both Genbank and FASTA rettypes) and for taxonomy data. Which FetchAdaptor you get is decided by info in the result message, and it should all be transparent to you. ✨

$seqio = $fac->efetch( -db =>'nucleotide',
                       -id => \@ids,
                       -rettype => 'gb' )->run( -auto_adapt => 1 );
while (my $seq = $seqio->next_seq) {
   my $taxio = $fac->efetch(
       -db => 'taxonomy',
       -id => $seq->species->ncbi_taxid )->run(-auto_adapt => 1);
   my $tax = $taxio->next_species;
   unless ( $tax->TaxId == $seq->species->ncbi_taxid ) {
     print "more work for MAJ"
   }
}

See the pod for the FetchAdaptor subclasses (like Bio::DB::SoapEUtilities::FetchAdaptor::seq) for more detail.

• elink, the Link adaptor, and the linkset iterator

The LinkAdaptor manages LinkSets. In SoapEU, an elink call always preserves the correspondence between the ids you submitted and the ids you got back. You can grab that mapping straight off the adaptor object with id_map()

my $links = $fac->elink( -db => 'protein',
                         -dbfrom => 'nucleotide',
                         -id => \@nucids )->run( -auto_adapt => 1 );

# maybe more than one associated id...
my @prot_0 = $links->id_map( $nucids[0] );

Or just iterate over the linksets:

while ( my $ls = $links->next_linkset ) {
   @ids = $ls->ids;
   @submitted_ids = $ls->submitted_ids;
   # etc.
}

• esummary, the DocSum adaptor, and the docsum iterator

The DocSumAdaptor manages docsums — that's the esummary return type. The objects you get by iterating a DocSumAdaptor have accessors that let you pull field info directly. Docsums are packed with easy-to-forget fields, so use item_names() to remind yourself. 💅

my $docs = $fac->esummary( -db => 'taxonomy',
                           -id => 527031 )->run(-auto_adapt=>1);
# iterate over docsums
while (my $d = $docs->next_docsum) {
   @available_items = $docsum->item_names;
   # any available item can be called as an accessor
   # from the docsum object...watch your case...
   $sci_name = $d->ScientificName;
   $taxid = $d->TaxId;
}

• egquery, the GQuery adaptor, and the query iterator

The GQueryAdaptor manages the global query items you get back from egquery calls — that's the thing that finds every NCBI database holding hits for your query term. You can grab the databases that actually have hits straight off the adaptor with found_in_dbs:

my $queries = $fac->egquery(
    -term => 'BRCA and human'
   )->run(-auto_adapt=>1);
my @dbs = $queries->found_in_dbs;

Want the global query info returned for any database? Use query_by_db:

my $prot_q = $queries->query_by_db('protein');
if ($prot_q->count) {
   #do something
}

Or iterate like always:

while ( my $q = $queries->next_query ) {
   if ($q->status eq 'Ok') {
     # do sth
   }
}

---

🌐 Web environments and query keys

When you're making big or complex data requests — or you just wanna share queries with someone — it can be super helpful to use NCBI's WebEnv system to manage your queries. Every EUtility accepts these parameters:

 -usehistory
 -WebEnv
 -QueryKey

That's what they're for, bestie. These store the details of your queries server-side. 🌸

SoapEU tries to make this pretty straightforward. Use Result objects to grab the right parameters, and don't forget -usehistory:

my $result1 = $fac->esearch(
    -term => 'BRCA and human',
    -db => 'nucleotide',
    -usehistory => 1 )->run( -no_parse=>1 );

my $result = $fac->esearch(
    -term => 'AND early onset',
    -QueryKey => $result1->query_key,
    -WebEnv => $result1->webenv )->run( -no_parse => 1 );

my $result = $fac->esearch(
   -db => 'protein',
   -term => 'sonic',
   -usehistory => 1 )->run( -no_parse => 1 );

# later (but not more than 8 hours later) that day...

$result = $fac->esearch(
   -WebEnv => $result->webenv,
   -QueryKey => $result->query_key,
   -RetMax => 800 # get 'em all
   )->run; # note we're parsing the result...
@all_ids = $result->ids;

⏰ Heads up: those WebEnv / QueryKey vibes have a shelf life. Don't wait more than 8 hours to come back to them, or the whole thing expires on you. 😭

---

🚨 Error checking

Two kinds of errors can pop off on an Entrez SOAP run. One is a SOAP fault. The other is an error sent inside a non-faulted SOAP message from the server. The distinction is probably systematic, and honestly I'd welcome an explanation of it. 🙈 To check for result errors, try something like this:

unless ( $result = $fac->$util->run ) {
   die $fac->errstr; # this will catch a SOAP fault
}
# a valid result object was returned, but it may carry an error
if ($result->count == 0) {
   warn "No hits returned";
   if ($result->ERROR) {
     warn "Entrez error : ".$result->ERROR;
   }
}

Error handling will be improved in the package eventually. (Iconic of it to admit that. 💀)

---

👀 SEE ALSO 👀

Bio::DB::EUtilities, Bio::DB::SoapEUtilities::Result, Bio::DB::ESoap.

💬 FEEDBACK 💬

Mailing Lists 📮

bestie, user feedback is literally the backbone of how this module and every other Bioperl module grows and evolves. So send your comments and suggestions — preferably straight to the Bioperl mailing list. Your participation is so so so appreciated 🫶

  bioperl-l@bioperl.org                  - General discussion
http://bioperl.org/wiki/Mailing_lists  - About the mailing lists

Support 🛟

okay so if you've got a usage question or a support issue, please direct it to the mailing list:

bioperl-l@bioperl.org

...and not to the module maintainer's DMs, bestie, they are not the main character here. There are so many experienced and responsive experts on that list who will look at your problem and get it sorted fast. And please include a thorough description of the problem, with code and data examples if at all possible. Give them the full lore, not just the vibes 💅

Reporting Bugs 🐛

Found a bug? Report it to the Bioperl bug tracking system so we can keep track of every bug and how it got resolved. Bug reports can be submitted via the web:

http://redmine.open-bio.org/projects/bioperl/

🦋 Not me manifesting a world with zero bugs... but until then, log it properly bestie.

---

👑 AUTHOR - Mark A. Jensen 👑

Email: maj -at- fortinbras -dot- us

---

🌸 APPENDIX 🌸

The rest of this documentation breaks down every single object method, one by one. Just so you know: internal methods are usually preceded with a _ — that underscore is giving "backstage pass, not for the general public" energy 🎀

✨ new ✨

• Title: new
• Usage: my $eutil = new Bio::DB::SoapEUtilities();
• Function: Builds a brand new Bio::DB::SoapEUtilities object
• Returns: an instance of Bio::DB::SoapEUtilities
• Args: (nothing, and that's iconic)

✨ run() ✨

• Title: run
• Usage: $fac->$eutility->run(@args)
• Function: Execute the EUtility
• Returns: true on success, false on fault or error (reason lives in errstr(), and for more detail check the SOAP message in last_result())
• Args: named params appropriate to utility
  • -auto_adapt => boolean ( return an iterator over results as appropriate to util if true)
  • -raw_xml => boolean ( return raw xml result; no processing )
  • Bio::DB::SoapEUtilities::Result constructor parms

---

🔮 Useful Accessors 🔮

💖 response_message() 💖

• Title: response_message
• Aliases: last_response, last_result
• Usage: $som = $fac->response_message
• Function: get the last response message
• Returns: a SOAP::SOM object
• Args: none

💖 webenv() 💖

• Title: webenv
• Usage: (blank, no cap)
• Function: contains WebEnv key referencing the session (set after run() )
• Returns: scalar
• Args: none

💖 errstr() 💖

• Title: errstr
• Usage: $fac->errstr
• Function: get the last error, if any
• Example: (blank)
• Returns: value of errstr (a scalar)
• Args: none

---

🧚 Bio::ParameterBaseI compliance 🧚

🌈 available_parameters() 🌈

• Title: available_parameters
• Usage: (blank)
• Function: get available request parameters for calling utility
• Returns: (blank)
• Args: -util => $desired_utility [optional, default is caller utility]

🌈 set_parameters() 🌈

• Title: set_parameters
• Usage: (blank)
• Function: (blank)
• Returns: none
• Args: -util => $desired_utility [optional, default is caller utility], named utility arguments

🌈 get_parameters() 🌈

• Title: get_parameters
• Usage: (blank)
• Function: (blank)
• Returns: array of named parameters
• Args: utility (scalar string) [optional] (default is caller utility)

🌈 reset_parameters() 🌈

• Title: reset_parameters
• Usage: (blank)
• Function: (blank)
• Returns: none
• Args: -util => $desired_utility [optional, default is caller utility], named utility arguments

🌈 parameters_changed() 🌈

• Title: parameters_changed
• Usage: (blank)
• Function: (blank)
• Returns: boolean
• Args: utility (scalar string) [optional] (default is caller utility)

---

🙈 _soap_facs() 🙈

• Title: _soap_facs
• Usage: $self->_soap_facs($util, $fac)
• Function: caches Bio::DB::ESoap factories for the eutils in use by this instance
• Example: (blank)
• Returns: Bio::DB::ESoap object
• Args: $eutility, [optional on set] $esoap_factory_object

🙈 _caller_util() 🙈

• Title: _caller_util
• Usage: $self->_caller_util($newval)
• Function: the utility requested off the main SoapEUtilities object
• Example: (blank)
• Returns: value of _caller_util (a scalar string, a valid eutility)
• Args: on set, new value (a scalar string [optional])

---

and that's the whole page bestie, you read every last line 💜 this beauty shipped as part of perl v5.36.0, dated 2023-01-22 — a certified classic, no notes, periodt ✨👑