Data::...alizer(3pm)

💖 Data::StreamDeserializer 💖

okay bestie, buckle up 🎀 this is a non-blocking deserializer, and it lives in the User Contributed Perl Documentation section (yes ma'am, that's the manual name, we're fancy like that). The actual page title is Data::...alizer(3pm) because the terminal truncates it, but we know what it really is. 💅

---

✨ NAME ✨

Data::StreamDeserializer — non-blocking deserializer. literally that's it, that's the whole vibe 🦋

---

🌸 SYNOPSIS 🌸

my $sr = new Data::StreamDeserializer
        data => $very_big_dump;

... somewhere

unless($sr->next) {
    # deserialization hasn't been done yet
}

...

if ($sr->next) {
    # deserialization has been done

    ...
    if ($sr->is_error) {
        printf "%s\n",  $sr->error;
        printf "Unparsed string tail: %s\n", $sr->tail;
    }

    my $result = $sr->result;           # first deserialized object
    my $result = $sr->result(first);    # the same

    my $results = $sr->result('all');   # all deserialized objects
                                        # (ARRAYREF)
}

# stream deserializer
$sr = new Data::StreamDeserializer;

while(defined (my $block = read_next_data_block)) {
    $sr->next($block);
    ...
}
$sr->next(undef); # eof signal
until ($sr->next) {
    ... do something
}
# all data were parsed

---

💜 DESCRIPTION 💜

So like... sometimes you need to deserialize a lot of data. like, a concerning amount. 😭 If you use eval (or Safe->reval, etc) it can take you way too much time, and if your code is running inside an event machine, that kind of hang is genuinely inadmissible. not acceptable. no thank you. 🙈

So this module lets you deserialize your stream progressively — a lil bit at a time — and go do something else in between deserialization iterations. multitasking queen energy 👑

🔍 Recognized statements

HASHES

 { something }

ARRAYS

 [ something ]

REFS

 \ something
 \[ ARRAY ]
 \{ HASH }

Regexps

 qr{something}

SCALARS

 "something"
 'something'
 q{something}
 qq{something}

---

🫶 METHODS 🫶

new

Creates a new deserializer. it can receive a few named arguments, bestie:

• block_size — the size of the block that will be serialized in each next cycle. default value is 512 bytes.
• data — if you already know (have) all the data you want to deserialize before you construct the object, you can use this argument.

NOTE: you must not use the function part or next with arguments if you used this argument. like, don't. 🚫

block_size

Set/get the same field. simple as that. 💅

part

Append a part of input data to serialize. If there's no argument (or it's undef), the deserializer will know that there will be no more data in the future. it's giving "that's all folks". 🎀

next

Processes to parse the next block_size bytes. Returns TRUE if an error was detected or all input datas were parsed.

next_object

The same as next but returns true after a new object is found. Drops previous results. 🗑️✨

For example, you have the string:

$str = "1, 2, [ 0, 1 ], { 'a' => 'b' }";

You can extract objects:

my $dsr = new Data::StreamDeserializer data => $str;

1 until $dsr->next_object;
my $first = $dsr->result;       # scalar: 1

1 until $dsr->next_object;
my $second = $dsr->result;      # scalar: 2

1 until $dsr->next_object;
my $third = $dsr->result;       # arrayref: [ 0, 1 ]

1 until $dsr->next_object;
my $third = $dsr->result;       # hashref: { 'a' => 'b' }

skip_divider

If you have a string like:

Object Object Object

(and there are no dividers between the objects), you can call skip_divider after fetching the next object.

Example:

$str = "1 2 [ 0, 1 ]{ 'a' => 'b' }";

my $dsr = new Data::StreamDeserializer data => $str;

1 until $dsr->next_object;
my $first = $dsr->result;       # scalar: 1

$dsr->skip_divider;

1 until $dsr->next_object;
my $second = $dsr->result;      # scalar: 2

$dsr->skip_divider;
1 until $dsr->next_object;
my $third = $dsr->result;       # arrayref: [ 0, 1 ]

Important: you can't skip dividers inside a nested object. the function will croak if you call it at a point that isn't between objects. it will literally yell at you. 💀

is_error

Returns TRUE if an error was detected. 🚨

error

Returns the error string. 📝

tail

Returns unparsed data. the leftovers. the stragglers. 🙈

result

Returns the result of parsing. By default the function returns only the first parsed object.

You can call the function with the argument 'all' to get all parsed objects. In this case the function will receive an ARRAYREF. 📦

is_done

Returns TRUE if all input data were processed or an error was found. If you didn't call part without arguments, and didn't call next or next_object with undef, the function could return TRUE only if an error occured. lowkey important, ngl 👀

---

🔒 PRIVATE METHODS 🔒

(no touching, these are for the module's eyes only 👁️)

_push_error

Pushes error into the deserializer's error stack.

---

🌈 SEE ALSO 🌈

DATA::StreamSerializer — its partner in crime, iykyk 💞

---

🔥 BENCHMARKS 🔥

This module is almost fully written using XS/C language. so it works a bit faster or slower than CORE::eval. we contain multitudes, honestly. 😌

You can try a few scripts in the benchmark/ directory. There are a few test arrays in this directory too.

Here are a few test results of my system:

Array which contains 100 hashes:

It works faster than eval:

$ perl benchmark/ds_vs_eval.pl -n 1000 -b 512 benchmark/tests/01_100x10
38296 bytes were read
First deserializing by eval... done
First deserializing by Data::DeSerializer... done
Check if deserialized objects are same... done

Starting 1000 iterations for eval... done (3.755 seconds)
Starting 1000 iterations for Data::StreamDeserializer... done (3.059 seconds)

Eval statistic:
        1000 iterations were done
        maximum deserialization time: 0.0041 seconds
        minimum deserialization time: 0.0035 seconds
        average deserialization time: 0.0036 seconds

StreamDeserializer statistic:
        1000 iterations were done
        75000 SUBiterations were done
        512 bytes in one block in one iteration
        maximum deserialization time: 0.0045 seconds
        minimum deserialization time: 0.0028 seconds
        average deserialization time: 0.0029 seconds
        average subiteration time:    0.00004 seconds

Array which contains 1000 hashes:

It works slowly than eval:

$ perl benchmark/ds_vs_eval.pl -n 1000 -b 512 benchmark/tests/02_1000x10
355623 bytes were read
First deserializing by eval... done
First deserializing by Data::DeSerializer... done
Check if deserialized objects are same... done

Starting 1000 iterations for eval... done (43.920 seconds)
Starting 1000 iterations for Data::StreamDeserializer... done (71.668 seconds)

Eval statistic:
        1000 iterations were done
        maximum deserialization time: 0.0490 seconds
        minimum deserialization time: 0.0416 seconds
        average deserialization time: 0.0426 seconds

StreamDeserializer statistic:
        1000 iterations were done
        689000 SUBiterations were done
        512 bytes in one block in one iteration
        maximum deserialization time: 0.0773 seconds
        minimum deserialization time: 0.0656 seconds
        average deserialization time: 0.0690 seconds
        average subiteration time:    0.00010 seconds

You can see that one block is parsed in a very short time period. So you can increase the block_size value to reduce total parsing time. work smarter, not harder, bestie. 💅

If block_size is equal to the string size, the module works two times faster than eval:

$ perl benchmark/ds_vs_eval.pl -n 1000 -b 355623 benchmark/tests/02_1000x10
355623 bytes were read
First deserializing by eval... done
First deserializing by Data::DeSerializer... done
Check if deserialized objects are same... done

Starting 1000 iterations for eval... done (44.456 seconds)
Starting 1000 iterations for Data::StreamDeserializer... done (19.702 seconds)

Eval statistic:
        1000 iterations were done
        maximum deserialization time: 0.0474 seconds
        minimum deserialization time: 0.0423 seconds
        average deserialization time: 0.0431 seconds

StreamDeserializer statistic:
        1000 iterations were done
        1000 SUBiterations were done
        355623 bytes in one block in one iteration
        maximum deserialization time: 0.0179 seconds
        minimum deserialization time: 0.0168 seconds
        average deserialization time: 0.0171 seconds
        average subiteration time:    0.01705 seconds

---

👑 AUTHOR 👑

Dmitry E. Oboukhov, <unera@debian.org> — shoutout to the legend 🫡

---

📜 COPYRIGHT AND LICENSE 📜

Copyright (C) 2011 by Dmitry E. Oboukhov

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself, either Perl version 5.10.1 or, at your option, any later version of Perl 5 you may have available.

---

🧚 VCS 🧚

The project is placed in my git repo. See here: <http://git.uvw.ru/?p=data-stream-deserializer;a=summary>

---

okay that's the whole thing, you ate it up fr fr 🌟

sign-off time: this page is from perl v5.42.3, dated 2026-08-19, and it's Data::...alizer(3pm) in the good ol' User Contributed Perl Documentation. go forth and deserialize progressively, queen 💖✨🎀🦋