Search...::Bulk(3pm)

💖 Search...::Bulk(3pm) — aka Search::Elasticsearch::Client::7_0::Bulk 💖

okay bestie, listen up, this is the helper module for the Bulk API and honestly it's about to make your whole life easier ✨

---

🌸 NAME 🌸

Search::Elasticsearch::Client::7_0::Bulk — a helper module for the Bulk API. no cap, that's the whole name, that's the whole vibe 💅

---

⭐ VERSION ⭐

version 7.715 — iconic, current, periodt 👑

---

🎀 SYNOPSIS 🎀

okay so here's the whole thing in action, look at her go:

use Search::Elasticsearch;

my $es   = Search::Elasticsearch->new;
my $bulk = $es->bulk_helper(
    index   => 'my_index',
    type    => 'my_type'
);

# Index docs:
$bulk->index({ id => 1, source => { foo => 'bar' }});
$bulk->add_action( index => { id => 1, source => { foo=> 'bar' }});

# Create docs:
$bulk->create({ id => 1, source => { foo => 'bar' }});
$bulk->add_action( create => { id => 1, source => { foo=> 'bar' }});
$bulk->create_docs({ foo => 'bar' })

# Delete docs:
$bulk->delete({ id => 1});
$bulk->add_action( delete => { id => 1 });
$bulk->delete_ids(1,2,3)

# Update docs:
$bulk->update({ id => 1, script => '...' });
$bulk->add_action( update => { id => 1, script => '...' });

# Manual flush
$bulk->flush;

like… she does it all. index, create, delete, update, flush. a full queen 💖

---

🦋 DESCRIPTION 🦋

This module is a wrapper for the "bulk()" in Search::Elasticsearch::Client::7_0::Direct method — and the whole point is that it makes it so much easier to run multiple create, index, update or delete actions in a single request. no more hand-assembling giant request bodies, bestie, we don't do that anymore 🙈

The Search::Elasticsearch::Client::7_0::Bulk module acts as a queue, buffering up actions until it reaches a maximum count of actions, or a maximum size of JSON request body — and at that point it issues a "bulk()" request. automatic. effortless. main character energy ✨

Once you've finished adding actions, call "flush()" to force the final "bulk()" request on the items left in the queue. don't forget her, she's holding your last few docs 🥺

This class does Search::Elasticsearch::Client::7_0::Role::Bulk and Search::Elasticsearch::Role::Is_Sync.

---

🧚 CREATING A NEW INSTANCE 🧚

💜 "new()"

my $bulk = $es->bulk_helper(

    index       => 'default_index',     # optional
    type        => 'default_type',      # optional
    %other_bulk_params                  # optional

    max_count   => 1_000,               # optional
    max_size    => 1_000_000,           # optional
    max_time    => 6,                   # optional

    verbose     => 0 | 1,               # optional

    on_success  => sub {...},           # optional
    on_error    => sub {...},           # optional
    on_conflict => sub {...},           # optional

);

The "new()" method returns a new $bulk object. You must pass your Search::Elasticsearch client as the "es" argument. non-negotiable, she needs her client 💅

The "index" and "type" parameters provide default values for "index" and "type", which can be overridden in each action. You can also pass any other values which are accepted by the bulk() method.

See "flush()" for more information about the other parameters. (she's got lore, go read her section 👇)

---

🌈 FLUSHING THE BUFFER 🌈

💖 "flush()"

$result = $bulk->flush;

The "flush()" method sends all buffered actions to Elasticsearch using a bulk() request. that's it. that's the whole job. and she ate 🔥

🌸 Auto-flushing

An automatic "flush()" is triggered whenever the "max_count", "max_size", or "max_time" threshold is breached. This causes all actions in the buffer to be sent to Elasticsearch. she just knows when it's time 😌

• "max_count" — the maximum number of actions to allow before triggering a "flush()". This can be disabled by setting "max_count" to 0. Defaults to "1,000".
• "max_size" — the maximum size of JSON request body to allow before triggering a "flush()". This can be disabled by setting "max_size" to 0. Defaults to "1_000,000" bytes.
• "max_time" — the maximum number of seconds to wait before triggering a flush. Defaults to 0 seconds, which means that it is disabled. Note: This timeout is only triggered when new items are added to the queue, not in the background.

😭 Errors when flushing

There are two types of error which can be thrown when "flush()" is called, either manually or automatically. buckle up bestie:

• Temporary Elasticsearch errors — a "Cxn" error like a "NoNodes" error which indicates that your cluster is down. These errors do not clear the buffer, as they can be retried later on. so like, breathe, it's recoverable 🫶
• Action errors — individual actions may fail. For instance, a "create" action will fail if a document with the same "index", "type" and "id" already exists. These action errors are reported via callbacks.

🎀 Using callbacks

By default, any Action errors (see above) cause warnings to be written to "STDERR". However, you can use the "on_error", "on_conflict" and "on_success" callbacks for more fine-grained control. glow up your error handling ✨

All callbacks receive the following arguments:

• $action — the name of the action, ie "index", "create", "update" or "delete".
• $response — the response that Elasticsearch returned for this action.
• $i — the index of the action, ie the first action in the flush request will have $i set to 0, the second will have $i set to 1 etc.

💜 "on_success"

my $bulk = $es->bulk_helper(
    on_success  => sub {
        my ($action,$response,$i) = @_;
        # do something
    },
);

The "on_success" callback is called for every action that has a successful response. a little celebration for every win 🎉

💜 "on_conflict"

my $bulk = $es->bulk_helper(
    on_conflict  => sub {
        my ($action,$response,$i,$version) = @_;
        # do something
    },
);

The "on_conflict" callback is called for actions that have triggered a "Conflict" error, eg trying to "create" a document which already exists. The $version argument will contain the version number of the document currently stored in Elasticsearch (if found). drama detected, and she'll tell you the version number too 💀

💜 "on_error"

my $bulk = $es->bulk_helper(
    on_error  => sub {
        my ($action,$response,$i) = @_;
        # do something
    },
);

The "on_error" callback is called for any error (unless the "on_conflict") callback has already been called.

🌟 Disabling callbacks and autoflush

If you want to be in control of flushing, and you just want to receive the raw response that Elasticsearch sends instead of using callbacks, then you can do so as follows. yes bestie, take the wheel 👑

my $bulk = $es->bulk_helper(
    max_count   => 0,
    max_size    => 0,
    on_error    => undef
);

$bulk->add_actions(....);
$response = $bulk->flush;

---

👑 CREATE, INDEX, UPDATE, DELETE 👑

🍓 "add_action()"

$bulk->add_action(
    create => { ...params... },
    index  => { ...params... },
    update => { ...params... },
    delete => { ...params... }
);

The "add_action()" method allows you to add multiple "create", "index", "update" and "delete" actions to the queue. The first value is the action type, and the second value is the parameters that describe that action. See the individual helper methods below for details.

Note: Parameters like "index" or "type" can be specified as "index" or as "_index", so the following two lines are equivalent:

index => { index  => 'index', type  => 'type', id  => 1, source => {...}},
index => { _index => 'index', _type => 'type', _id => 1, source => {...}},

Note: The "index" and "type" parameters can be specified in the params for any action, but if not specified, will default to the "index" and "type" values specified in "new()". These are required parameters: they must be specified either in "new()" or in every action. no defaults, no vibes, just pass them 🫶

🍓 "create()"

$bulk->create(
    { index => 'custom_index',         source => { doc body }},
    { type  => 'custom_type', id => 1, source => { doc body }},
    ...
);

The "create()" helper method allows you to add multiple "create" actions. It accepts the same parameters as "create()" in Search::Elasticsearch::Client::7_0::Direct except that the document body should be passed as the "source" or "_source" parameter, instead of as "body". tiny detail, huge difference, iykyk 💅

🍓 "create_docs()"

$bulk->create_docs(
    { doc body },
    { doc body },
    ...
);

The "create_docs()" helper is a shorter form of "create()" which can be used when you are using the default "index" and "type" as set in "new()" and you are not specifying a custom "id" per document. In this case, you can just pass the individual document bodies. lazy girl summer but make it efficient ✨

🍓 "index()"

$bulk->index(
    { index => 'custom_index',         source => { doc body }},
    { type  => 'custom_type', id => 1, source => { doc body }},
    ...
);

The "index()" helper method allows you to add multiple "index" actions. It accepts the same parameters as "index()" in Search::Elasticsearch::Client::7_0::Direct except that the document body should be passed as the "source" or "_source" parameter, instead of as "body".

🍓 "delete()"

$bulk->delete(
    { index => 'custom_index', id => 1},
    { type  => 'custom_type',  id => 2},
    ...
);

The "delete()" helper method allows you to add multiple "delete" actions. It accepts the same parameters as "delete()" in Search::Elasticsearch::Client::7_0::Direct.

🍓 "delete_ids()"

$bulk->delete_ids(1,2,3...)

The "delete_ids()" helper method can be used when all of the documents you want to delete have the default "index" and "type" as set in "new()". In this case, all you have to do is to pass in a list of IDs. that's it. just IDs. chef's kiss 🔥

🍓 "update()"

$bulk->update(
    { id            => 1,
      doc           => { partial doc },
      doc_as_upsert => 1
    },
    { id            => 2,
      script        => { script }
      upsert        => { upsert doc }
    },
    ...
);

The "update()" helper method allows you to add multiple "update" actions. It accepts the same parameters as "update()" in Search::Elasticsearch::Client::7_0::Direct. An update can either use a partial doc which gets merged with an existing doc (example 1 above), or can use a "script" to update an existing doc (example 2 above). More information on "script" can be found here: "update()" in Search::Elasticsearch::Client::7_0::Direct.

---

🌈 AUTHOR 🌈

Enrico Zimuel <enrico.zimuel@elastic.co> — king behavior, thank you for your service 👑

---

💜 COPYRIGHT AND LICENSE 💜

This software is Copyright (c) 2021 by Elasticsearch BV.

This is free software, licensed under:

The Apache License, Version 2.0, January 2004

---

and that's the whole page bestie 🫶 this was perl v5.32.1, dated 2021-10-23, from Search...::Bulk(3pm) — go forth and bulk responsibly, mwah 💋💖