Elastic...Builder(3)

📦
version 0.19 · 2 versions
, in fedora 43

💖 ElasticSearch::SearchBuilder — bestie edition 💖

okay so this is the manual page for ElasticSearch::SearchBuilder, aka Elastic...Builder(3), from the User Contributed Perl Documentation corner of the universe. she's a Perlish compact query language for ElasticSearch and honestly? she eats. 🫶

✨ VERSION ✨

Version 0.16

Compatible with ElasticSearch version 0.19.11 ⭐

🚨 BREAKING CHANGE 🚨

bestie sit DOWN. the 'text' queries have been renamed 'match' queries in elasticsearch 0.19.9. so if you're on an older version of elasticsearch, this is not the vibe for you and you need to go here instead:

<https://metacpan.org/release/DRTECH/ElasticSearch-SearchBuilder-0.15/>

like, actually go there. don't just yolo it and hope. 🙈

🌸 DESCRIPTION 🌸

The Query DSL for ElasticSearch (see Query DSL <http://www.elasticsearch.org/guide/reference/query-dsl>), the thing you use to write queries and filters, is simple but verbose, and verbose here means it gets real difficult to write and understand large queries. it's giving essay when you wanted a text. 😭

ElasticSearch::SearchBuilder is an SQL::Abstract-like query language which exposes the full power of the query DSL, but in a more compact, Perlish way. same power, less yapping. periodt. 💅

This module is considered stable. If you have suggestions for improvements to the API or the documenation, please contact me. (yes, the original says "documenation", we're keeping it, it's iconic 👑)

🎀 SYNOPSIS 🎀

my $sb = ElasticSearch::SearchBuilder->new();
my $query = $sb->query({
    body    => 'interesting keywords',
    -filter => {
        status  => 'active',
        tags    => ['perl','python','ruby'],
        created => {
            '>=' => '2010-01-01',
            '<'  => '2011-01-01'
        },
    }
})

NOTE: "ElasticSearch::SearchBuilder" is fully integrated with the ElasticSearch API. Wherever you can specify "query", "filter" or "facet_filter" in ElasticSearch, you can automatically use SearchBuilder by specifying "queryb", "filterb", "facet_filterb" instead. the lil b is the whole trick, iykyk 🧚

$es->search( queryb  => { body => 'interesting keywords' } )

🔥 METHODS 🔥

💫 new()

my $sb = ElasticSearch::SearchBuilder->new()

Creates a new instance of the SearchBuilder — takes no parameters. zero. nada. she's low maintenance like that. ✨

💫 query()

my $es_query = $sb->query($compact_query)

Returns a query in the ElasticSearch query DSL.

$compact_query can be a scalar, a hash ref or an array ref.

$sb->query('foo')
# { "query" : { "match" : { "_all" : "foo" }}}

$sb->query({ ... }) or $sb->query([ ... ])
# { "query" : { ... }}

💫 filter()

my $es_filter = $sb->filter($compact_filter)

Returns a filter in the ElasticSearch query DSL.

$compact_filter can be a scalar, a hash ref or an array ref.

$sb->filter('foo')
# { "filter" : { "term" : { "_all" : "foo" }}}

$sb->filter({ ... }) or $sb->filter([ ... ])
# { "filter" : { ... }}

okay that's part 1 bestie, keep scrolling, there's more 👀

💖 INTRODUCTION 💖

⚠️ IMPORTANT: if you are not familiar with ElasticSearch then you should read "ELASTICSEARCH CONCEPTS" before continuing. no cap, bestie, read it first. 📚

ngl, this module was inspired by SQL::Abstract but they are not compatible with each other. it's giving "inspired by, but do not mix." 🙅‍♀️

The easiest way to explain how the syntax works is to give examples:

✨ QUERY / FILTER CONTEXT ✨

There are two contexts:

  • filter context — Filters are fast and cacheable. They should be used to include/exclude docs, based on simple term values. For instance, exclude all docs that have neither tag "perl" nor "python".

Typically, most of your clauses should be filters, which reduce the number of docs that need to be passed to the query. lowkey the main character of efficiency. 💅

  • query context — Queries are smarter than filters, but more expensive, as they have to calculate search relevance (ie _score).

They should be used where:

  • relevance is important, eg: in a search for tags "perl" or "python", a doc that has BOTH tags is more relevant than a doc that has only one
  • where search terms need to be analyzed as full text, eg: find me all docs where the "content" field includes the words "Perl is GREAT", no matter how those words are capitalized.

The available operators (and the query/filter clauses that are generated) differ according to which context you are in.

The initial context depends upon which method you use: query() puts you into query context, and filter() into filter context.

However, you can switch from one context to another as follows:

$sb->query({

    # query context
    foo     => 1,
    bar     => 2,

    -filter => {
        # filter context
        foo     => 1,
        bar     => 2,

        -query  => {
            # query context
            foo => 1
        }
    }
})

🎀 -filter | -not_filter 🎀

Switch from query context to filter context:

# query field content for 'brown cow', and filter documents
# where status is 'active' and tags contains the term 'perl'
{
    content => 'brown cow',
    -filter => {
        status => 'active',
        tags   => 'perl'
    }
}

# no query, just a filter:
{ -filter => { status => 'active' }}

See Filtered Query (http://www.elasticsearch.org/guide/reference/query-dsl/filtered-query.html) and Constant Score Query (http://www.elasticsearch.org/guide/reference/query-dsl/constant-score-query.html)

🦋 -query | -not_query 🦋

Use a query as a filter:

# query field content for 'brown cow', and filter documents
# where status is 'active', tags contains the term 'perl'
# and a match query on field title contains 'important'
{
    content => 'brown cow',
    -filter => {
        status => 'active',
        tags   => 'perl',
        -query => {
            title => 'important'
        }
    }
}

See Query Filter (http://www.elasticsearch.org/guide/reference/query-dsl/query-filter.html)

🌈 KEY-VALUE PAIRS 🌈

Key-value pairs are equivalent to the = operator, discussed below. They are converted to match queries or term filters:

# Field 'foo' contains term 'bar'
# equiv: { foo => { '=' => 'bar' }}
{ foo => 'bar' }

# Field 'foo' contains 'bar' or 'baz'
# equiv: { foo => { '=' => ['bar','baz'] }}
{ foo => ['bar','baz']}

# Field 'foo' contains terms 'bar' AND 'baz'
# equiv: { foo => { '-and' => [ {'=' => 'bar'}, {'=' => 'baz'}] }}
{ foo => ['-and','bar','baz']}

### FILTER ONLY ###

# Field 'foo' is missing ie has no value
# equiv: { -missing => 'foo' }
{ foo => undef }

🔥 AND|OR LOGIC 🔥

Arrays are OR'ed, hashes are AND'ed:

# tags = 'perl' AND status = 'active:
{
    tags   => 'perl',
    status => 'active'
}

# tags = 'perl' OR status = 'active:
[
    tags   => 'perl',
    status => 'active'
]

# tags = 'perl' or tags = 'python':
{ tags => [ 'perl','python' ]}
{ tags => { '=' => [ 'perl','python' ] }}

# tags begins with prefix 'p' or 'r'
{ tags => { '^' => [ 'p','r' ] }}

The logic in an array can changed from "OR" to "AND" by making the first element of the array ref -and:

# tags has term 'perl' AND 'python'

{ tags => ['-and','perl','python']}

{
    tags => [
        -and => { '=' => 'perl'},
                { '=' => 'python'}
    ]
}

However, the first element in an array ref which is used as the value for a field operator (see "FIELD OPERATORS") is not special:

# WRONG
{ tags => { '=' => [ '-and','perl','python' ] }}

# RIGHT
{ tags => ['-and' => [ {'=' => 'perl'}, {'=' => 'python'} ] ]}

...otherwise you would never be able to search for the term -and. So if you might possibly have the terms -and or -or in your data, use:

{ foo => {'=' => [....] }}

instead of:

{ foo => [....]}

💀 -and | -or | -not 💀

These unary operators allow you apply "and", "or" and "not" logic to nested queries or filters.

# Field foo has both terms 'bar' and 'baz'
{ -and => [
        foo => 'bar',
        foo => 'baz'
]}

# Field 'name' contains 'john smith', or the name field is missing
# and the 'desc' field contains 'john smith'

{ -or => [
    { name => 'John Smith' },
    {
        desc     => 'John Smith'
        -filter  => { -missing => 'name' },
    }
]}

The -and, -or and -not constructs emit bool queries when in query context, and and, or and not clauses when in filter context.

See also: "NAMED FILTERS", Bool Query (http://www.elasticsearch.org/guide/reference/query-dsl/bool-query.html), And Filter (http://www.elasticsearch.org/guide/reference/query-dsl/and-filter.html), Or Filter (http://www.elasticsearch.org/guide/reference/query-dsl/or-filter.html) and Not Filter (http://www.elasticsearch.org/guide/reference/query-dsl/not-filter.html)

🌟 FIELD OPERATORS 🌟

Most operators (eg =, gt, geo_distance etc) are applied to a particular field. These are known as "Field Operators". For example:

# Field foo contains the term 'bar'
{ foo => 'bar' }
{ foo => {'=' => 'bar' }}

# Field created is between Jan 1 and Dec 31 2010
{ created => {
    '>='  => '2010-01-01',
    '<'   => '2011-01-01'
}}

# Field foo contains terms which begin with prefix 'a' or 'b' or 'c'
{ foo => { '^' => ['a','b','c' ]}}

Some field operators are available as symbols (eg =, *, ^, gt) and others as words (eg geo_distance or -geo_distance - the dash is optional).

Multiple field operators can be applied to a single field. Use {} to imply "this AND that":

# Field foo has any value from 100 to 200
{ foo => { gte => 100, lte => 200 }}

# Field foo begins with 'p' but is not python
{ foo => {
    '^'  => 'p',
    '!=' => 'python'
}}

Or [] to imply "this OR that"

# foo is 5 or foo greater than 10
{ foo => [
    { '='  => 5  },
    { 'gt' => 10 }
]}

All word operators may be negated by adding not_ to the beginning, eg:

# Field foo does NOT contain a term beginning with 'bar' or 'baz'
{ foo => { not_prefix => ['bar','baz'] }}

🧚 UNARY OPERATORS 🧚

There are other operators which don't fit this { field => { op => value }} model.

For instance:

  • An operator might apply to multiple fields:
# Search fields 'title' and 'content' for text 'brown cow'
{
    -match => {
        query   => 'brown cow',
        fields  => ['title','content']
    }
}
  • The field might BE the value:
# Find documents where the field 'foo' is blank or undefined
{ -missing => 'foo' }

# Find documents where the field 'foo' exists and has a value
{ -exists => 'foo' }
  • For combining other queries or filters:
# Field foo has terms 'bar' and 'baz' but not 'balloo'
{
    -and => [
        foo => 'bar',
        foo => 'baz',
        -not => { foo => 'balloo' }
    ]
}
  • Other:
# Script query
{ -script => "doc['num1'].value > 1" }

These operators are called "unary operators" and ALWAYS begin with a dash - to distinguish them from field names.

Unary operators may also be prefixed with not_ to negate their meaning.

⭐ MATCH ALL ⭐

👑 -all 👑

The -all operator matches all documents:

# match all
{ -all => 1  }
{ -all => 0  }
{ -all => {} }

In query context, the match_all query usually scores all docs as 1 (ie having the same relevance). By specifying a norms_field, the relevance can be read from that field (at the cost of a slower execution time):

# Query context only
{ -all =>{
    boost       => 1,
    norms_field => 'doc_boost'
}}

✨ EQUALITY ✨

okay so these operators are literally just answering the question: "does this field contain this term?" that's it, that's the whole vibe 💅

filter equality operators are picky — they ONLY work with exact terms. query equality operators (the whole match family) are the chill ones and will "do the right thing", aka work with terms for not_analyzed fields and with analyzed text for analyzed fields. bestie they're flexible like that 🫶

🌸 EQUALITY (QUERIES) 🌸

= | -match | != | <> | -not_match

so these operators all generate match queries, no cap:

# Analyzed field 'title' contains the terms 'Perl is GREAT'
# (which is analyzed to the terms 'perl','great')
{ title => 'Perl is GREAT' }
{ title => { '='  => 'Perl is GREAT' }}
{ title => { match => 'Perl is GREAT' }}

# Not_analyzed field 'status' contains the EXACT term 'ACTIVE'
{ status => 'ACTIVE' }
{ status => { '='  => 'ACTIVE' }}
{ status => { match => 'ACTIVE' }}

# Same as above but with extra parameters:
{ title => {
    match => {
        query                => 'Perl is GREAT',
        boost                => 2.0,
        operator             => 'and',
        analyzer             => 'default',
        fuzziness            => 0.5,
        fuzzy_rewrite        => 'constant_score_default',
        lenient              => 1,
        max_expansions       => 100,
        minimum_should_match => 2,
        prefix_length        => 2,
    }
}}

the operators <>, != and not_match are all synonyms for each other, they're giving "same energy" 👯 and they just wrap the operator in a not clause.

See Match Query <http://www.elasticsearch.org/guide/reference/query-dsl/match-query.html>

== | -phrase | -not_phrase

these ones are looking for a complete phrase. like the whole thing together, periodt.

for instance, given the text

The quick brown fox jumped over the lazy dog.

# matches
{ content => { '==' => 'Quick Brown' }}

# doesn't match
{ content => { '==' => 'Brown Quick' }}
{ content => { '==' => 'Quick Fox'   }}

the slop parameter can be used to allow the phrase to match words in the same order, but further apart (so they can be lowkey distant but still in formation ✨):

# with other parameters
{ content => {
    phrase => {
        query    => 'Quick Fox',
        slop     => 3,
        analyzer => 'default'
        boost    => 1,
        lenient  => 1,
}}

See Match Query <http://www.elasticsearch.org/guide/reference/query-dsl/match-query.html>

Multi-field -match | -not_match

to run a match | =, phrase or phrase_prefix query against MULTIPLE fields, you can use the -match unary operator. it's the polyamorous one 🦋

{
    -match => {
        query                => "Quick Fox",
        type                 => 'boolean',
        fields               => ['content','title'],

        use_dis_max          => 1,
        tie_breaker          => 0.7,

        boost                => 2.0,
        operator             => 'and',
        analyzer             => 'default',
        fuzziness            => 0.5,
        fuzzy_rewrite        => 'constant_score_default',
        lenient              => 1,
        max_expansions       => 100,
        minimum_should_match => 2,
        prefix_length        => 2,
    }
}

the type parameter can be boolean (which is the equivalent of match | =, and also the default), phrase or phrase_prefix.

See Multi-match Query <http://www.elasticsearch.org/guide/reference/query-dsl/multi-match-query.html>.

-term | -terms | -not_term | -not_terms

the term/terms operators are provided for completeness. ngl you should almost always use the match/= operator instead. these are giving "backup dancers" energy 🙈

there are only two use cases:

  • To find the exact (ie not analyzed) term 'foo' in an analyzed field:
{ title => { term => 'foo' }}
  • To match a list of possible terms, where more than 1 value must match:
# match 2 or more of these tags
{ tags => {
    terms => {
        value         => ['perl','python','php'],
        minimum_match => 2,
        boost         => 1,
    }
}}

the above can also be achieved with the -bool operator.

term and terms are synonyms, as are not_term and not_terms. 👑

🎀 EQUALITY (FILTERS) 🎀

= | -term | -terms | <> | != | -not_term | -not_terms

these operators result in term or terms filters, which look for fields which contain exactly the terms specified:

# Field foo has the term 'bar':
{ foo => 'bar' }
{ foo => { '='    => 'bar' }}
{ foo => { 'term' => 'bar' }}

# Field foo has the term 'bar' or 'baz'
{ foo => ['bar','baz'] }
{ foo => { '='     => ['bar','baz'] }}
{ foo => { 'term'  => ['bar','baz'] }}

<> and != are synonyms:

# Field foo does not contain the term 'bar':
{ foo => { '!=' => 'bar' }}
{ foo => { '<>' => 'bar' }}

# Field foo contains neither 'bar' nor 'baz'
{ foo => { '!=' => ['bar','baz'] }}
{ foo => { '<>' => ['bar','baz'] }}

the terms filter can take an execution parameter which affects how the filter of multiple terms is executed and cached. it's giving "how do we run this" 🧚

for instance:

{ foo => {
    -terms => {
        value       => ['foo','bar'],
        execution   => 'bool'
    }
}}

See Term Filter <http://www.elasticsearch.org/guide/reference/query-dsl/term-filter.html> and Terms Filter <http://www.elasticsearch.org/guide/reference/query-dsl/terms-filter.html>

🌈 RANGES 🌈

lt | gt | lte | gte | < | <= | >= | > | -range | -not_range

these operators imply a range query or filter, which can be numeric or alphabetical. yes bestie, letters have ranges too 💖

# Field foo contains terms between 'alpha' and 'beta'
{ foo => {
    'gte'   => 'alpha',
    'lte'   => 'beta'
}}

# Field foo contains numbers between 10 and 20
{ foo => {
    'gte'   => '10',
    'lte'   => '20'
}}

# boost a range  *** query only ***
{ foo => {
    range => {
        gt      => 5,
        gte     => 5,
        lt      => 10,
        lte     => 10,
        boost   => 2.0
    }
}}

for queries, < is a synonym for lt, > for gt etc. ✨

See Range Query <http://www.elasticsearch.org/guide/reference/query-dsl/range-query.html>

Note: for filter clauses, the gt, gte, lt and lte operators imply a range filter, while the <, <=, > and >= operators imply a numeric_range filter.

this does NOT mean that you should use the numeric_range version for any field which contains numbers! 😭 read the room bestie.

the numeric_range filter should be used for numbers/datetimes which have many distinct values, eg ID or last_modified. if you have a numeric field with few distinct values, eg number_of_fingers, then it is better to use a range filter. (lowkey makes sense, right?)

See Range Filter <http://www.elasticsearch.org/guide/reference/query-dsl/range-filter.html> and Numeric Range Filter <http://www.elasticsearch.org/guide/reference/query-dsl/numeric-range-filter.html>.

💀 MISSING OR NULL VALUES 💀

* Filter context only *

-missing | -exists

you can use a missing or exists filter to select only docs where a particular field exists and has a value, or is undefined or has no value. it's the "do you even exist" vibe check 🔥

# Field 'foo' has a value:
{ foo     => { exists  => 1 }}
{ foo     => { missing => 0 }}
{ -exists => 'foo'           }

# Field 'foo' is undefined or has no value:
{ foo      => { missing => 1 }}
{ foo      => { exists  => 0 }}
{ -missing => 'foo'           }
{ foo      => undef           }

the missing filter also supports the null_value and existence parameters:

{
    foo     => {
        missing => {
            null_value => 1,
            existence  => 1,
        }
    }
}

OR

{ -missing => {
    field      => 'foo',
    null_value => 1,
    existence  => 1,
}}

See Missing Filter <http://www.elasticsearch.org/guide/reference/query-dsl/missing-filter.html> and Exists Filter <http://www.elasticsearch.org/guide/reference/query-dsl/exists-filter.html>

✨ FULL TEXT SEARCH ✨

Query context only 🫶

Okay bestie, so for most full text search queries, the "match" queries are literally what you want. They analyze the search terms and then go looking for documents that contain one or more of those terms. (Peep "EQUALITY (QUERIES)" if you need a refresher 💖)

🌸 -qs | -query_string | -not_qs | -not_query_string 🌸

But wait — there's a way fancier query string syntax (go look at Lucene Query Parser Syntax <http://lucene.apache.org/core/old_versioned_docs/versions/3_5_0/queryparsersyntax.html>) that actually understands search terms like this:

perl AND python tag:recent "this exact phrase" -apple

It's giving main character energy for "power" users ngl, but the downside? If the syntax is wrong, ES just throws an error at you. 😭 You can use ElasticSearch::QueryParser to fix any syntax errors though, so slay.

# find docs whose 'title' field matches 'this AND that'
{ title => { qs           => 'this AND that' }}
{ title => { query_string => 'this AND that' }}

# With other parameters
{ title => {
    field => {
        query                        => 'this that ',
        default_operator             => 'AND',
        analyzer                     => 'default',
        allow_leading_wildcard       => 0,
        lowercase_expanded_terms     => 1,
        enable_position_increments   => 1,
        fuzzy_min_sim                => 0.5,
        fuzzy_prefix_length          => 2,
        fuzzy_rewrite                => 'constant_score_default',
        fuzzy_max_expansions         => 1024,
        lenient                      => 1,
        phrase_slop                  => 10,
        boost                        => 2,
        analyze_wildcard             => 1,
        auto_generate_phrase_queries => 0,
        rewrite                      => 'constant_score_default',
        minimum_should_match         => 3,
        quote_analyzer               => 'standard',
        quote_field_suffix           => '.unstemmed'
    }
}}

And the unary form -qs or -query_string hits different when you're matching against multiple fields at once:

{ -qs => {
        query                        => 'this AND that ',
        fields                       => ['title','content'],
        default_operator             => 'AND',
        analyzer                     => 'default',
        allow_leading_wildcard       => 0,
        lowercase_expanded_terms     => 1,
        enable_position_increments   => 1,
        fuzzy_min_sim                => 0.5,
        fuzzy_prefix_length          => 2,
        fuzzy_rewrite                => 'constant_score_default',
        fuzzy_max_expansions         => 1024,
        lenient                      => 1,
        phrase_slop                  => 10,
        boost                        => 2,
        analyze_wildcard             => 1,
        auto_generate_phrase_queries => 0,
        use_dis_max                  => 1,
        tie_breaker                  => 0.7,
        minimum_should_match         => 3,
        quote_analyzer               => 'standard',
        quote_field_suffix           => '.unstemmed'
}}

See Query-string Query <http://www.elasticsearch.org/guide/reference/query-dsl/query-string-query.html> ✨

💜 -mlt | -not_mlt 💜

An mlt (aka more_like_this) query finds documents that are "like" the text you hand it — and "like" here means it contains some or all of the specified terms. So it's the "more of this vibe pls" query fr fr.

# Field foo is like "brown cow"
{ foo => { mlt => "brown cow" }}

# With other paramters:
{ foo => {
    mlt => {
        like_text               => 'brown cow',
        percent_terms_to_match  => 0.3,
        min_term_freq           => 2,
        max_query_terms         => 25,
        stop_words              => ['the','and'],
        min_doc_freq            => 5,
        max_doc_freq            => 1000,
        min_word_len            => 0,
        max_word_len            => 20,
        boost_terms             => 2,
        boost                   => 2.0,
        analyzer                => 'default'
    }
}}

# multi fields
{ -mlt => {
    like_text               => 'brown cow',
    fields                  => ['title','content']
    percent_terms_to_match  => 0.3,
    min_term_freq           => 2,
    max_query_terms         => 25,
    stop_words              => ['the','and'],
    min_doc_freq            => 5,
    max_doc_freq            => 1000,
    min_word_len            => 0,
    max_word_len            => 20,
    boost_terms             => 2,
    boost                   => 2.0,
    analyzer                => 'default'
}}

See MLT Field Query <http://www.elasticsearch.org/guide/reference/query-dsl/mlt-field-query.html> and MLT Query <http://www.elasticsearch.org/guide/reference/query-dsl/mlt-query.html> 🦋

🎀 -flt | -not_flt 🎀

An flt (or fuzzy_like_this) query fuzzifies every term you give it, then picks the best max_query_terms differentiating terms. Basically it's a glow-up collab between "fuzzy" and "more_like_this". 💅

# Field foo is fuzzily similar to "brown cow"
{ foo => { flt => 'brown cow }}

# With other parameters:
{ foo => {
    flt => {
        like_text       => 'brown cow',
        ignore_tf       => 0,
        max_query_terms => 10,
        min_similarity  => 0.5,
        prefix_length   => 3,
        boost           => 2.0,
        analyzer        => 'default'
    }
}}

# Multi-field
flt => {
    like_text       => 'brown cow',
    fields          => ['title','content'],
    ignore_tf       => 0,
    max_query_terms => 10,
    min_similarity  => 0.5,
    prefix_length   => 3,
    boost           => 2.0,
    analyzer        => 'default'
}

See FLT Field Query <http://www.elasticsearch.org/guide/reference/query-dsl/flt-field-query.html> and FLT Query <http://www.elasticsearch.org/guide/reference/query-dsl/flt-query.html> 🌈

⭐ PREFIX ⭐

🌸 PREFIX (QUERIES) 🌸

^ | -phrase_prefix | -not_phrase_prefix

These operators use the match_phrase_prefix query. 🫶

For "analyzed" fields, it analyzes the search terms and runs a match_phrase query, with a prefix query riding on the last term. Think… auto-complete, bestie. The way it just knows what you're typing. ✨

For "not_analyzed" fields, this behaves exactly the same as the term-based prefix query.

So like, given the phrase "The quick brown fox jumped over the lazy dog":

# matches
{ content => { '^'             => 'qui'}}
{ content => { '^'             => 'quick br'}}
{ content => { 'phrase_prefix' => 'quick brown f'}}

# doesn't match
{ content => { '^'             => 'quick fo' }}
{ content => { 'phrase_prefix' => 'fox brow'}}

With extra options:

{ content => {
    phrase_prefix => {
        query          => "Brown Fo",
        slop           => 3,
        analyzer       => 'default',
        boost          => 3.0,
        max_expansions => 100,
    }
}}

See http://www.elasticsearch.org/guide/reference/query-dsl/match-query.html 💖

-prefix | -not_prefix

The prefix query is a term-based query — no analysis takes place, not even on analyzed fields. Generally you should use ^ instead, okay? Like, just trust the process.

# Field 'lang' contains terms beginning with 'p'
{ lang => { prefix => 'p' }}

# With extra options
{ lang => {
    'prefix' => {
        value   => 'p',
        boost   => 2,
        rewrite => 'constant_score_default',

    }
}}

See Prefix Query <http://www.elasticsearch.org/guide/reference/query-dsl/prefix-query.html>. 🎀

🦋 PREFIX (FILTERS) 🦋

^ | -prefix | -not_prefix

# Field foo contains a term which begins with 'bar'
{ foo => { '^'      => 'bar' }}
{ foo => { 'prefix' => 'bar' }}

# Field foo contains a term which begins with 'bar' or 'baz'
{ foo => { '^'      => ['bar','baz'] }}
{ foo => { 'prefix' => ['bar','baz'] }}

# Field foo contains a term which begins with neither 'bar' nor 'baz'
{ foo => { 'not_prefix' => ['bar','baz'] }}

See Prefix Filter <http://www.elasticsearch.org/guide/reference/query-dsl/prefix-filter.html> 🌸

🔥 WILDCARD AND FUZZY QUERIES 🔥

Query context only 💀

💅 * | -wildcard | -not_wildcard 💅

A wildcard is a term-based query (no analysis is applied at all, periodt), which does shell globbing to find matching terms. In other words, ? represents any single character, while * represents zero or more characters. Iconic behavior honestly.

# Field foo matches 'f?ob*'
{ foo => { '*'        => 'f?ob*' }}
{ foo => { 'wildcard' => 'f?ob*' }}

# with a boost:
{ foo => {
    '*' => { value => 'f?ob*', boost => 2.0 }
}}
{ foo => {
    'wildcard' => {
        value   => 'f?ob*',
        boost   => 2.0,
        rewrite => 'constant_score_default',
    }
}}

See Wildcard Query <http://www.elasticsearch.org/guide/reference/query-dsl/wildcard-query.html> ⭐

🌈 -fuzzy | -not_fuzzy 🌈

A fuzzy query is a term-based query (ie no analysis is done) which looks for terms that are similar to the the provided terms, where similarity is based on the Levenshtein (edit distance) algorithm. So it's literally out here forgiving your typos. 😭🫶

# Field foo is similar to 'fonbaz'
{ foo => { fuzzy => 'fonbaz' }}

# With other parameters:
{ foo => {
    fuzzy => {
        value           => 'fonbaz',
        boost           => 2.0,
        min_similarity  => 0.2,
        max_expansions  => 10,
        rewrite         => 'constant_score_default',
    }
}}

Normally, you should rather use either the EQUALITY queries with the fuzziness parameter, or the -flt queries. Just saying. 💜

See Fuzzy Query <http://www.elasticsearch.org/guide/reference/query-dsl/fuzzy-query.html>. ✨

💫 COMBINING QUERIES 💫

Query context only — bestie, this is a query-context situation, not a filter-context one, okurrr 💅

These lil constructs let you smoosh multiple queries together into one big vibe. Group project, but make it search. 🫶

🌈 -dis_max | -dismax

So a bool query adds together the scores of all the nested queries (team effort, everyone contributes 💖). But a dis_max query is giving pick-me energy: it just takes the highest score of any of the matching queries and rolls with that. Main character only, no cap. 👑

# Run the two queries and use the best score
{ -dismax => [
    { foo => 'bar' },
    { foo => 'baz' }
] }

# With other parameters
{ -dismax => {
    queries => [
        { foo => 'bar' },
        { foo => 'baz' }
    ],
    tie_breaker => 0.5,
    boost => 2.0
] }

See DisMax Query <http://www.elasticsearch.org/guide/reference/query-dsl/dis-max-query.html> ✨

🧚 -bool

Okay so normally you would never need to use a bool query directly, because they get autogenerated for you from stuff like -and, -or and -not (the system handles it, you just vibe 🍓). But if you need to pass any of the other parameters to a bool query, here's how you do it:

{
   -bool => {
       must          => [{ foo => 'bar' }],
       must_not      => { status => 'inactive' },
       should        => [
            { tag    => 'perl'   },
            { tag    => 'python' },
            { tag    => 'ruby' },
       ],
       minimum_number_should_match => 2,
       disable_coord => 1,
       boost         => 2
   }
}

See Bool Query <http://www.elasticsearch.org/guide/reference/query-dsl/bool-query.html> 🌸

💜 -boosting

The boosting query is the "we're not gonna block you, but we ARE gonna be a little shady about it" move. It demotes results that match a given query. Unlike the must_not clause of a bool query, the query still matches — the results are just "less relevant". So they show up, but in the back of the group photo. 🙈

{ -boosting => {
    positive       => { title => 'apple pear'     },
    negative       => { title => 'apple computer' },
    negative_boost => 0.2
}}

See Boosting Query <http://www.elasticsearch.org/guide/reference/query-dsl/boosting-query.html> 🔥

⭐ -custom_boost

The custom_boost query lets you multiply the scores of another query by whatever boost factor you specify. Lowkey different from a standard boost, because a standard boost is normalized. This one is raw, unfiltered, doing its own thing. 💅

{
    -custom_boost => {
        query           => { title => 'foo' },
        boost_factor    => 3
    }
}

See Custom Boost Factor Query <http://www.elasticsearch.org/guide/reference/query-dsl/custom-boost-factor-query.html>. ✨

🪆 NESTED QUERIES/FILTERS 🪆

Nested queries and filters let you run queries/filters on nested docs. It's giving "documents within documents", inception-style, and honestly? Iconic. 🧚

So like, normally a doc like this would not let you associate the name "perl" with the number 5 — the fields are all flattened together and the pairs get lost in the sauce 😭:

{
    title:  "my title",
    tags: [
     { name: "perl",   num: 5},
     { name: "python", num: 2}
    ]
}

However — plot twist — if tags is mapped as a nested field, then you can run queries or filters on each sub-doc individually. Each lil sub-doc keeps its own identity. Slay. 👑

See Nested Type <http://www.elasticsearch.org/guide/reference/mapping/nested-type.html>, Nested Query <http://www.elasticsearch.org/guide/reference/query-dsl/nested-query.html> and Nested Filter <http://www.elasticsearch.org/guide/reference/query-dsl/nested-filter.html> 🌈

🎀 -nested (QUERY)

{
    -nested => {
        path        => 'tags',
        score_mode  => 'avg',
        _scope      => 'my_tags',
        query       => {
            "tags.name"  => 'perl',
            "tags.num"   => { gt => 2 },
        }
    }
}

See Nested Query <http://www.elasticsearch.org/guide/reference/query-dsl/nested-query.html> 💖

🦋 -nested (FILTER)

{
    -nested => {
        path        => 'tags',
        score_mode  => 'avg',
        _cache      => 1,
        _name       => 'my_filter',
        filter      => {
            tags.name    => 'perl',
            tags.num     => { gt => 2},
        }
    }
}

See Nested Filter <http://www.elasticsearch.org/guide/reference/query-dsl/nested-filter.html> 🌸

🧙 SCRIPTING 🧙

ElasticSearch supports using scripts to customise query or filter behaviour — yes bestie, you can write actual code in there, we're getting fancy. By default the query language is mvel, but javascript, groovy, python and native java scripts are also supported. Choose your fighter. 💅

See Scripting <http://www.elasticsearch.org/guide/reference/modules/scripting.html> for more on scripting. ✨

🍓 -custom_score

Query context only

The -custom_score query lets you customise the _score (aka the relevance, aka the thing that decides the order of the docs you get back). You're literally hand-tuning the rankings. Power. 👑

{
    -custom_score => {
        query  => { foo => 'bar' },
        lang    => 'mvel',
        script => "_score * doc['my_numeric_field'].value / pow(param1, param2)"
        params => {
            param1 => 2,
            param2 => 3.1
        },
    }
}

See Custom Score Query <http://www.elasticsearch.org/guide/reference/query-dsl/custom-score-query.html> 💜

💀 -custom_filters_score

Query context only

The -custom_filters_score query lets you boost documents that match a filter — either with a boost parameter, or with a custom script. Either way, you're boosting. 📈

This is a very powerful and efficient way to boost results which depend on matching unanalyzed fields, like a tag or a date. Also, these filters can be cached. Efficiency AND power? We love to see it. 🔥

{
    -custom_filters_score => {
        query       => { foo => 'bar' },
        score_mode  => 'first|max|total|avg|min|multiply', # default 'first'
        max_boost   => 10,
        filters     => [
            {
                filter => { tag => 'perl' },
                boost  => 2,
            },
            {
                filter => { tag => 'python' },
                script => '_score * my_boost',
                params => { my_boost => 2},
                lang   => 'mvel'
            },
        ]
    }
}

See Custom Filters Score Query <http://www.elasticsearch.org/guide/reference/query-dsl/custom-filters-score-query.html> ⭐

🧚 -script

Filter context only

The -script filter lets you use a script as a filter. Return a true value to indicate that the filter matches. That's it, that's the whole job. 🫶

# Filter docs whose field 'foo' is greater than 5
{ -script => "doc['foo'].value > 5 " }

# With other params
{
    -script => {
        script => "doc['foo'].value > minimum ",
        params => { minimum => 5 },
        lang   => 'mvel'
    }
}

See Script Filter <http://www.elasticsearch.org/guide/reference/query-dsl/script-filter.html> 🌈

👑 PARENT/CHILD 👑

Documents stored in ElasticSearch can be configured to have parent/child relationships. Family tree energy, but for your data. 🌳

See Parent Field <http://www.elasticsearch.org/guide/reference/mapping/parent-field.html> for more. 💖

🎀 -has_parent | -not_has_parent

Find child documents that have a parent document which matches a query. So you're asking "show me the kids whose parents fit this vibe". 💅

# Find parent docs whose children of type 'comment' have the tag 'perl'
{
    -has_parent => {
        type   => 'comment',
        query  => { tag => 'perl' },
        _scope => 'my_scope',
        boost  => 1,                    # Query context only
        score_type => 'max'             # Query context only
    }
}

See Has Parent Query <http://www.elasticsearch.org/guide/reference/query-dsl/has-parent-query.html> and See Has Parent Filter <http://www.elasticsearch.org/guide/reference/query-dsl/has-parent-filter.html>. ✨

🦋 -has_child | -not_has_child

Find parent documents that have child documents which match a query. Same idea, flipped — now you're checking the parents' vibes via their kids. 🧚

# Find parent docs whose children of type 'comment' have the tag 'perl'
{
    -has_child => {
        type   => 'comment',
        query  => { tag => 'perl' },
        _scope => 'my_scope',
        boost  => 1,                    # Query context only
        score_type => 'max'             # Query context only
     }
}

See Has Child Query <http://www.elasticsearch.org/guide/reference/query-dsl/has-child-query.html> and See Has Child Filter <http://www.elasticsearch.org/guide/reference/query-dsl/has-child-filter.html>. 🌸

⭐ -top_children

Query context only

The top_children query runs a query against the child docs, and then aggregates the scores to find the parent docs whose children best match. So it's not just "does this parent have a matching kid" — it's "whose kids matched the HARDEST". Competitive parenting, basically. 💀

{
    -top_children => {
        type                => 'blog_tag',
        query               => { tag => 'perl' },
        score               => 'max',
        factor              => 5,
        incremental_factor  => 2,
        _scope              => 'my_scope'
    }
}

See Top Children Query <http://www.elasticsearch.org/guide/reference/query-dsl/top-children-query.html> 🔥

🌍 GEO FILTERS 🌍

okay so bestie, for literally every single geo filter, the normalize parameter defaults to true, meaning that the longitude value will be normalized to -180 to 180 and the latitude value to -90 to 90. no cap. 💅✨

-geo_distance | -not_geo_distance

💅 Filter context only 💅

The geo_distance filter will find locations within a certain distance of a given point — it's giving radius search, fr fr:

{
    my_location => {
        -geo_distance     => {
            location      => { lat => 10, lon => 5 },
            distance      => '5km',
            normalize     => 1 | 0,
            optimize_bbox => memory | indexed | none,
        }
    }
}

See Geo Distance Filter — http://www.elasticsearch.org/guide/reference/query-dsl/geo-distance-filter.html

-geo_distance_range | -not_geo_distance_range

💅 Filter context only 💅

The geo_distance_range filter is similar to the -geo_distance filter, but expressed as a range. lowkey the same vibe, but with a from/to glow-up:

{
    my_location => {
        -geo_distance       => {
            location        => { lat => 10, lon => 5 },
            from            => '5km',
            to              => '10km',
            include_lower   => 1 | 0,
            include_upper   => 0 | 1
            normalize       => 1 | 0,
            optimize_bbox   => memory | indexed | none,
        }
    }
}

or instead of from, to, include_lower and include_upper you can use gt, gte, lt, lte.

See Geo Distance Range Filter — http://www.elasticsearch.org/guide/reference/query-dsl/geo-distance-range-filter.html

-geo_bounding_box | -geo_bbox | -not_geo_bounding_box | -not_geo_bbox

💅 Filter context only 💅

The geo_bounding_box filter finds points which lie within the given rectangle. it's giving rectangle era:

{
    my_location => {
        -geo_bbox => {
            top_left     => { lat => 9, lon => 4  },
            bottom_right => { lat => 10, lon => 5 },
            normalize    => 1 | 0,
            type         => memory | indexed
        }
    }
}

See Geo Bounding Box Filter — http://www.elasticsearch.org/guide/reference/query-dsl/geo-bounding-box-filter.html

-geo_polygon | -not_geo_polygon

💅 Filter context only 💅

The geo_polygon filter is similar to the -geo_bounding_box filter, except that it allows you to specify a polygon instead of a rectangle. main character geometry:

{
    my_location => {
        -geo_polygon => [
            { lat => 40, lon => -70 },
            { lat => 30, lon => -80 },
            { lat => 20, lon => -90 },
        ]
    }
}

or:

{
    my_location => {
        -geo_polygon => {
            points  => [
                { lat => 40, lon => -70 },
                { lat => 30, lon => -80 },
                { lat => 20, lon => -90 },
            ],
            normalize => 1 | 0,
        }
    }
}

See Geo Polygon Filter — http://www.elasticsearch.org/guide/reference/query-dsl/geo-polygon-filter.html

🔎 INDEX/TYPE/ID 🔎

-indices

💅 Query context only 💅

To run a different query depending on the index name, you can use the -indices query:

{
    -indices => {
        indices         => 'one' | ['one','two],
        query           => { status => 'active' },
        no_match_query  => 'all' | 'none' | { another => query }
    }
}

The no_match_query will be run on any indices which don't appear in the specified list. It defaults to all, but can be set to none or to a full query.

See Indices Query — http://www.elasticsearch.org/guide/reference/query-dsl/indices-query.html.

💅 Filter context only 💅

To run a different filter depending on the index name, you can use the -indices filter:

{
    -indices => {
        indices         => 'one' | ['one','two],
        filter          => { status => 'active' },
        no_match_filter => 'all' | 'none' | { another => filter }
    }
}

The no_match_filter will be run on any indices which don't appear in the specified list. It defaults to all, but can be set to none or to a full filter.

See Indices Filter — https://github.com/elasticsearch/elasticsearch/issues/1787.

-ids

The _id field is not indexed by default, and thus isn't available for normal queries or filters.

Returns docs with the matching _id or _type/_id combination:

# doc with ID 123
{ -ids => 123 }

# docs with IDs 123 or 124
{ -ids => [123,124] }

# docs of types 'blog' or 'comment' with IDs 123 or 124
{
    -ids => {
        type    => ['blog','comment'],
        values  => [123,124]

    }
}

See IDs Query — http://www.elasticsearch.org/guide/reference/query-dsl/ids-query.html abd IDs Filter — http://www.elasticsearch.org/guide/reference/query-dsl/ids-filter.html

-type

💅 Filter context only 💅

Filters docs with matching _type fields.

While the _type field is indexed by default, ElasticSearch provides the type filter which will work even if indexing of the _type field is disabled.

# Filter docs of type 'comment'
{ -type => 'comment' }

# Filter docs of type 'comment' or 'blog'
{ -type => ['blog','comment' ]}

See Type Filter — http://www.elasticsearch.org/guide/reference/query-dsl/type-filter.html

🛑 LIMIT 🛑

💅 Filter context only 💅

The limit filter limits the number of documents (per shard) to execute on:

{
    name    => "Joe Bloggs",
    -filter => { -limit => 100       }
}

See Limit Filter — http://www.elasticsearch.org/guide/reference/query-dsl/limit-filter.html

🏷️ NAMED FILTERS 🏷️

ElasticSearch allows you to name filters, in which each search result will include a matched_filters array containing the names of all filters that matched.

-name | -not_name

💅 Filter context only 💅
{ -name => {
    popular   => { user_rank => { 'gte' => 10 }},
    unpopular => { user_rank => { 'lt'  => 10 }},
}}

Multiple filters are joined with an or filter (as it doesn't make sense to join them with and).

See Named Filters — http://www.elasticsearch.org/guide/reference/api/search/named-filters.html and -and | -or | -not.

🧊 CACHING FILTERS 🧊

Part of the performance boost that you get when using filters comes from the ability to cache the results of those filters. However, it doesn't make sense to cache all filters by default.

-cache | -nocache

💅 Filter context only 💅

If you would like to override the default caching, then you can use -cache or -nocache:

# Don't cache the term filter for 'status'
{
    content => 'interesting post',
    -filter => {
        -nocache => { status => 'active' }
    }
}

# Do cache the numeric range filter:
{
    content => 'interesting post',
    -filter => {
        -cache => { created => {'>' => '2010-01-01' } }
    }
}

See Query DSL — http://www.elasticsearch.org/guide/reference/query-dsl/ for more details about what is cached by default and what is not.

-cache_key

It is also possible to use a name to identify a cached filter. For instance:

{
    -cache_key => {
        friends => { person_id => [1,2,3] },
        enemies => { person_id => [4,5,6] },
    }
}

In the above example, the two filters will be joined by an and filter. The following example will have the two filters joined by an or filter:

{
    -cache_key => [
        friends => { person_id => [1,2,3] },
        enemies => { person_id => [4,5,6] },
    ]
}

See _cache_key — http://www.elasticsearch.org/guide/reference/query-dsl/index.html for more details.

📜 RAW ELASTICSEARCH QUERY DSL 📜

Sometimes, instead of using the SearchBuilder syntax, you may want to revert to the raw Query DSL that ElasticSearch uses.

You can do this by passing a reference to a HASH ref, for instance:

$sb->query({
    foo => 1,
    -filter => \{ term => { bar => 2 }}
})

Would result in:

{
    query => {
        filtered => {
            query => {
                match => { foo => 1 }
            },
            filter => {
                term => { bar => 2 }
            }
        }
    }
}

An example with OR'ed filters:

$sb->filter([
    foo => 1,
    \{ term => { bar => 2 }}
])

Would result in:

{
    filter => {
        or => [
            { term => { foo => 1 }},
            { term => { bar => 2 }}
        ]
    }
}

An example with AND'ed filters:

$sb->filter({
    -and => [
        foo => 1 ,
        \{ term => { bar => 2 }}
    ]
})

Would result in:

{
    filter => {
        and => [
            { term => { foo => 1 }},
            { term => { bar => 2 }}
        ]
    }
}

Wherever a filter or query is expected, passing a reference to a HASH-ref is accepted.

🌈 ELASTICSEARCH CONCEPTS 🌈

Bestie. Before you go any further, we gotta talk concepts, because ElasticSearch has ✨lore✨ and if you don't know it you're gonna be out here filing bug reports about your own queries. Sit down, grab a strawberry 🍓, this is the good part.

🎀 FILTERS VS QUERIES 🎀

So ElasticSearch supports filters and queries, and no, they are not the same thing, they're like cousins that look alike but have completely different jobs 💅

  • A filter just answers the question: "Does this field match? Yes/No", eg:
    • Does this document have the tag "beta"?
    • Was this document published in 2011?
  • A query is used to calculate relevance (known in ElasticSearch as "_score"):
    • Give me all documents that include the keywords "Foo" and "Bar" and rank them in order of relevance.
    • Give me all documents whose "tag" field contains "perl" or "ruby" and rank documents that contain BOTH tags more highly.

Filters are lighter and faster, and the results can often be cached, but they don't contribute to the "_score" in any way. Not even a little. Not even for you 🫶

Typically, most of your clauses will be filters, and just a few will be queries. That's the vibe. That's the whole strategy. Filters do the heavy lifting, queries get the credit.

🦋 TERMS VS TEXT 🦋

All data is stored in ElasticSearch as a "term", which is an exact value. The term "Foo" is not the same as "foo". Case sensitivity said "I'm the main character" and honestly, respect 👑

While this is useful for fields that have discreet values (eg "active", "inactive"), it is not sufficient to support full text search. Like, it simply cannot serve that purpose, bestie.

ElasticSearch has to analyze text to convert it into terms. This applies both to the text that the stored document contains, and to the text that the user tries to search on. Both sides! Symmetry! It's giving balance ✨

The default analyzer will:

  • split the text on (most) punctuation and remove that punctuation
  • lowercase each word
  • remove English stopwords

For instance, "The 2 GREATEST widgets are foo-bar and fizz_buzz" would result in the terms " [2,'greatest','widgets','foo','bar','fizz_buzz']".

It is important that the same analyzer is used both for the stored text and for the search terms, otherwise the resulting terms may be different, and the query won't succeed. 💀 That's the trap. That's the one that gets everybody.

For instance, a "term" query for "GREATEST" wouldn't work, but "greatest" would work. However, a "match" query for "GREATEST" would work, because the search text would be analyzed to produce the same terms that are stored in the index. Slay 💅

See Analysis <http://www.elasticsearch.org/guide/reference/index-modules/analysis/> for the list of supported analyzers.

🌸 "match" QUERIES 🌸

ElasticSearch has a family of DWIM queries called "match" queries. DWIM = do what I mean, and honestly that's the energy we all aspire to.

Their action depends upon how the field has been defined. If a field is "analyzed" (the default for string fields) then the "match" queries analyze the search terms before doing the search:

# Convert "Perl is GREAT" to the terms 'perl','great' and search
# the 'content' field for those terms

{ match: { content: "Perl is GREAT" }}

If a field is "not_analyzed", then it treats the search terms as a single term:

# Find all docs where the 'status' field contains EXACTLY the term 'ACTIVE'
{ match: { status: "ACTIVE" }}

Filters, on the other hand, don't have full text queries — filters operate on simple terms instead. Filters are the lowkey reliable ones 🧚

See Match Query <http://www.elasticsearch.org/guide/reference/query-dsl/match-query.html> for more about match queries.

👑 AUTHOR 👑

Clinton Gormley, "<drtech at cpan.org>"

💀 BUGS 💀

If you have any suggestions for improvements, or find any bugs, please report them to <https://github.com/clintongormley/ElasticSearch-SearchBuilder/issues>. I will be notified, and then you'll automatically be notified of progress on your bug as I make changes. It's a whole notification pipeline and it's kinda iconic ngl.

⭐ TODO ⭐

Add support for "span" queries.

(it's on the list, bestie. it's on the list.)

🫶 SUPPORT 🫶

You can find documentation for this module with the perldoc command.

perldoc ElasticSearch::SearchBuilder

You can also look for information at: <http://www.elasticsearch.org>

🌈 ACKNOWLEDGEMENTS 🌈

Thanks to SQL::Abstract for providing the inspiration and some of the internals. Shoutout to the ancestors 🙏

Copyright 2011 Clinton Gormley.

This program is free software; you can redistribute it and/or modify it under the terms of either: the GNU General Public License as published by the Free Software Foundation; or the Artistic License.

See <http://dev.perl.org/licenses/> for more information.

okay that's the whole page, we made it, no cap 🎉 this was Elastic...Builder(3) from perl v5.42.0, dated 2025-07-25, and you just read the entire thing like the main character you are. go forth and build queries, bestie 💅🌸✨

📦 where this page comes from