Data::URIEncode(3pm)

💖 Data::URIEncode 💖

hii bestie!! so this is a page from the User Contributed Perl Documentation, and it's Data::URIEncode(3pm) — aka the module that lets your cute lil nested data structures ride the web as flat URIs. it's giving "I speak fluent query string". no cap. 🦋

---

✨ NAME ✨

Data::URIEncode — Allow complex data structures to be encoded using flat URIs.

---

🎀 SYNOPSIS 🎀

use Data::URIEncode qw(flat_to_complex complex_to_flat);

my $data = {
    foo => {
        bar => 'bing',
    },
    baz => [123],
};

my $flat  = complex_to_flat($data);
my $query = complex_to_query($data);

# $flat looks like:
$flat = {
   'foo.bar' => 'bing',
   'baz:0'   => 123,
};

# $query looks like:
$query = "foo.bar=bing&baz:0=123"

################################################

# put data back to how it was
$data = flat_to_complex($flat);

$data = query_to_complex($query);

################################################

### some html form somewhere
<form>
<input type="text" name="foo.bar.baz" value="brum">
<input type="text" name="bing:2" value="blang">
<input type="text" name="'key with :, ., and \''.red" value="blue">
</form>

### when the form is submitted to the following code
use CGI;
use Data::URIEncode qw(query_to_complex);

my $q = CGI->new;
my $data = query_to_complex($q);

### data will look like
$data = {
    foo => {
        bar => {
           baz = "brum",
        },
    },
    bing => [
        undef,
        undef,
        "blang",
    ],
    "key with :, ., and '" => {
        red = "blue",
    },
};

like, look at that. a whole nested hash and a whole array just casually collapsing into "foo.bar=bing&baz:0=123" and then un-collapsing later. icon behaviour. 💅

---

🌸 DESCRIPTION 🌸

okay so the entire web is basically run on URIs, right? and the query string part of a URI already knows how to carry key/value pairs — it's just that it doesn't natively vibe with complex data structures. sad!

there are modules and encodings that DO handle arbitrarily complex structures. JSON, YAML and Data::Dumper all have their own way of encoding complex structures. but then to get those across the web, you usually still have to URL encode them and pass them via a form parameter. extra steps, honestly. 😭

Data::URIEncode lets you encode and decode complex (multi level datastructures) using native Query String manipulators (such as CGI.pm). it takes complex data and turns it into a flat hashref which can then be turned into a URI query string using URL encoding. it also takes a flat hashref of data passed in and translates it back to a complex structure. it's giving translator-fluent-in-both-directions.

one benefit of using Data::URIEncode is that a standard submission from a standard html form can automatically be translated into complex data even though it arrived in a "flat" form. this somewhat mimics the abilities of XForms without introducing the complexity of XForms. slay.

another benefit is that sparse data can be represented in a more compact form than JSON or YAML are able to provide. however, complex data with long key names will be more verbose as the full data hierarchy must be repeated for each value. tradeoffs, bestie. always tradeoffs.

---

🌈 RULES 🌈

for each of the following rules, the $data can be translated to $flat and $query by calling complex_to_flat and complex_to_query respectively. the $flat and $query can be translated back into $data using flat_to_complex and query_to_complex respectively.

✨ Simple values stay simple ✨

$data  =   {key => "val", key2 => "val2"};
$flat  === {key => "val", key2 => "val2"};
$query eq  "key=val&key2=val2"

💫 Nested hashes use a dot to modify the key 💫

$data  =   {key => {key2 => "val"}};
$flat  === {"key.key2" => "val"};
$query eq  "key.key2=val

########

$data  =   {foo => {bar => {baz => "bling"}}};
$flat  === {"foo.bar.baz" = "bling"};
$query eq  "foo.bar.baz=bling"

🍓 Nested arrays use a colon to modify the key 🍓

$data  =   {key => ["val1", "val2"]};
$flat  === {"key:0" => "val1", "key:1" => "val2"};
$query eq  "key:0=val1&key:1=val2"

########

$data  =   {key => [ [ ["val"] ] ]};
$flat  === {"key:0:0" => "val"}
$query eq  "key:0:0=val"

🦋 Data structures can have an arrayref as the top level 🦋

a leading colon is used to indicate the top level node is an arrayref.

$data  =   ["val1", "val2"]
$flat  === {":0" => "val1", ":1" => "val2"}
$query eq  ":0=>val1&:1=>val2"

########

$data  =   [ [ ["val"] ] ];
$flat  === {":0:0:0" => "val"}
$query eq  ":0:0:0=val"

💜 Keys in flat hashrefs MAY begin with a leading dot 💜

a leading dot may disambiguate some cases. iykyk.

$query =   ".foo=bar"
$flat  =   {".foo" => "bar"}
$data  === {foo => "bar"}

👑 Single quotes may be used to enclose complex strings 👑

any key containing a colon ":", a dot ".", or a single quote "'" must be quoted with single quotes and have enclosed single quotes escaped. no wiggle room, babe.

$data  =   {"foo.bar"   => "baz"}
$flat  === {"'foo.bar'" => "baz"}
$query eq  "'foo.bar'=baz"  # the ' will be swapped with %27

########

$data  =   {"foo:bar"   => "baz"}
$flat  === {"'foo:bar'" => "baz"}
$query eq  "'foo:bar'=baz"  # the ' will be swapped with %27

########

$data  =   {""   => "baz"}
$flat  === {"''" => "baz"}
$query eq  "''=baz"  # the ' will be swapped with %27

########

$data  =   {"'"     => "baz"}
$flat  === {"'\\''" => "baz"}
$query eq  "'\\''=baz"  # the ' will be swapped with %27 and the \ will be replaced with %5C

single quotes were chosen as double quotes are most commonly used in HTML forms, thus allowing escaped single quotes more easily inside the double quoted name. big brain design decision, honestly. 🧠✨

💀 Undefined values are not included in the flattened data 💀

$data  =   {foo => undef, bar => 1}
$flat  === {bar => 1}
$query eq  "bar=1"

########

$data  =   ["val1", undef, "val2"]
$flat  === {":0" => "val1", ":2" => "val2"}
$query eq  ":0=val1&:2=val2"

🫶 Blessed hashes and arrayrefs are dumped by default 🫶

changing the default value of the global $DUMP_BLESSED_DATA variable changes the behavior.

$Data::URIEncode::DUMP_BLESSED_DATA = 1; # default
$data  =   {foo => bless({bar => "baz"}, "main"), one => "two"}
$flat  === {"foo.bar" => "baz", one => "two"}
$query eq  "foo.bar=baz&one=two"

########

$Data::URIEncode::DUMP_BLESSED_DATA = 0;
$data  =   {foo => bless({bar => "baz"}, "main"), one => "two"}
$flat  === {one => "two"}
$query eq  "one=two"

🧚 Arrays created by flat_to_complex and query_to_complex must obey the value of the $MAX_ARRAY_EXPAND variable. 🧚

---

💖 FUNCTIONS 💖

🌟 flat_to_complex 🌟

takes a hashref of simple key value pairs. returns a data structure based on the the parsed key value pairs. the parsing proceeds according to the rules listed in RULES.

my $data = flat_to_complex({"foo.bar.baz:2" => "bling"});
# $data = {foo => {bar => {baz => [undef, undef, "bling"]}}};

🌟 complex_to_flat 🌟

takes a complex data structure and turns it into a flat hashref (single level key/value pairs only). the parsing proceeds according to the rules listed in RULES.

my $flat = complex_to_flat({foo => ['a','b']});
# $flat = {"foo:0" => "a", "foo:1" => "b"});

🌟 complex_to_query 🌟

similar to complex_to_flat, except that the flattened hashref is then translated into query string suitable for use in a URI.

my $str = complex_to_query({foo => ['a','b']});
# $str eq "foo:0=a&foo:1=b"

🌟 query_to_complex 🌟

takes one of a string, a reference to a string, a hash, or a CGI.pm compatible object and translates it into a complex data structure. similar to flat_to_complex, exempt that a first step is taken to access the query parameters from the CGI compatible object or string. if a string or string ref is given, the CGI module is used to parse the string into an initial flat hash of key value pairs (using the param method). if another module is desired over, CGI.pm you must initialize it with the data to be parsed prior to passing the object to the query_to_complex function.

my $data = query_to_complex("foo.bar:0=baz");

my $data = query_to_complex(\ "foo.bar:0=baz");

my $data = query_to_complex({"foo.bar:0" => "baz"}); # same as flat_to_complex

my $cgi  = CGI->new(\ "foo.bar:0=baz");
my $data = query_to_complex($cgi);

my $cgi  = CGI->new; # use the values passed in from STDIN
my $data = query_to_complex($cgi);

---

🌺 VARIABLES 🌺

$MAX_ARRAY_EXPAND

default value is 100. this variable is used to determine how large flat_to_complex will allow an array to be expanded beyond its current size. an array can grow as large as you have memory, but intermediate values must exist.

without this value, somebody could specify foo:1000000000000=bar and your server would attempt to set the 1000000000000th index of the foo value to bar. 💀 do not do that to your server, bestie.

the string "foo:101=bar" would die, but the string "foo:50=bar&foo:101=baz" would not die because the intermediate foo->[50] increments the foo arrayref by 51 and the subsequent foo->[101] call increments the foo arrayref by only 51.

$DUMP_BLESSED_DATA

default is true. if true, blessed hashrefs and arrayrefs will also be added to the flat data returned by complex_to_flat. if false, bless hashrefs and arrayrefs will be skipped.

---

😭 BUGS 😭

circular refs are not detected. any attempt to dump a struture with cirular refs will result in an infinite loop. there is no immediate plan to add circular ref tracking.

so like... do not. just don't. 🚨

---

🔥 SEE ALSO 🔥

all of the following have attempted to solve the same problem as Data::URIEncode. all of them (including Data::URIEncode) suffer from the problem of being hard to find for the specific purpose. Hash::Flatten is probably the only suitable replacement for Data::URIEncode.

• Hash::Flatten
• CGI::Expand
• HTTP::Rollup
• CGI::State

---

🌈 AUTHOR 🌈

Paul Seamons perlspam at seamons dot com

---

🎀 LICENSE 🎀

This library may be distributed under the same terms as Perl itself.

---

okay bestie that's the whole page 💖 written by Paul Seamons, shipped as part of perl v5.36.0, dated 2022-10-13, and it's been sitting in the User Contributed Perl Documentation this whole time waiting for you to read it. go flatten something. you ate. periodt. 👑✨