Data::S...re::Util(3)

🌸 NAME 🌸

okay bestie, welcome to the Data::S...re::Util(3) page from User Contributed Perl Documentation — this is the manual for Data::Structure::Util, aka Change nature of data within a structure ✨. It's a whole toolbox for changing the data inside a data structure, and it's lowkey iconic for that 💅.

---

✨ SYNOPSIS ✨

use Data::Structure::Util qw(
  has_utf8 utf8_off utf8_on unbless get_blessed get_refs
  has_circular_ref circular_off signature
);

# get the objects in the data structure
my $objects_arrayref = get_blessed( $data );

# unbless all objects
unbless( $data );

if ( has_circular_ref( $data ) ) {
    print "Removing circular ref!\n";
    circular_off( $data );
}

# convert back to latin1 if needed and possible
utf8_off( $data ) if defined has_utf8( $data );

---

💖 DESCRIPTION 💖

Data::Structure::Util is a toolbox to manipulate the data inside a data structure. It can process an entire tree and perform the operation requested on each appropriate element 🧚.

For example: it can transform all strings within a data structure to utf8 or transform any utf8 string back to the default encoding. It can remove the blessing on any reference. It can collect all the objects or detect if there is a circular reference.

It is written in C for decent speed 🔥.

---

🔥 FUNCTIONS 🔥

All Data::Structure::Util functions operate on a whole tree. If you pass them a simple scalar then they will operate on that one scalar. However, if you pass them a reference to a hash, array, or scalar then they will iterate through that structure and apply the manipulation to all elements, and in turn if they are references to hashes, arrays or scalars to all their elements and so on, recursively 🌀.

For speed reasons, all manipulations that alter the data structure do in-place manipulation, meaning that rather than returning an altered copy of the data structure, the passed data structure is the one that gets altered. No sneaky duplicate, bestie — it's the same object getting the glow-up 💅.

💅 Manipulating Data Structures 💅

🌀 has_circular_ref($ref)

This function detects if the passed data structure has a circular reference, that is to say if it is possible by following references contained in the structure to return to a part of the data structure you have already visited. Data structures that have circular references will not be automatically reclaimed by Perl's garbage collector 😭.

If a circular reference is detected the function returns a reference to an element within the circuit; otherwise the function will return a false value.

If the version of perl that you are using supports weak references, then any weak references found within the data structure will not be traversed, meaning that circular references that have had links successfully weakened will not be returned by this function ✨.

🌀 circular_off($ref)

Detects circular references in $ref (as above) and weakens a link in each so that they can be properly garbage collected when no external references to the data structure are left.

This means that one (or more) of the references in the data structure will be told that they should not count towards reference counting. You should be aware that if you later modify the data structure and leave parts of it only 'accessible' via weakened references, those parts of the data structure will be immediately garbage collected, as the weakened references will not be strong enough to maintain the connection on their own 👀.

The number of references weakened is returned.

🌀 get_refs($ref)

Examine the data structure and return a reference to a flat array that contains one copy of every reference in the data structure you passed.

For example:

my $foo = {
    first  => [ "inner", "array", { inmost => "hash" } ],
    second => \"refed scalar",
};

use Data::Dumper;
# tell Data::Dumper to show nodes multiple times
$Data::Dumper::Deepcopy = 1;
print Dumper get_refs( $foo );

$VAR1 = [
    { 'inmost' => 'hash' },
    [ 'inner', 'array', { 'inmost' => 'hash' } ],
    \'refed scalar',
    {
        'first'  => [ 'inner', { 'inmost' => 'hash' }, 'array' ],
        'second' => \'refed scalar'
    }
];

As you can see, the data structure is traversed depth first, so the top most references should be the last elements of the array. See get_blessed($ref) below for a similar function for blessed objects 🎀.

🌀 signature($ref)

Returns a md5 of the passed data structure. Any change at all to the data structure will cause a different md5 to be returned.

The function examines the structure, addresses, value types and flags to generate the signature, meaning that even data structures that would look identical when dumped with Data::Dumper produce different signatures:

$ref1 = { key1 => [] };

$ref2 = $ref1;
$ref2->{key1} = [];

# this produces the same result, as they look the same
# even though they are different data structures
use Data::Dumper;
use Digest::MD5 qw(md5_hex);
print md5_hex( Dumper( $ref1 ) ), " ", md5_hex( Dumper( $ref2 ) ), "\n";
# cb55d41da284a5869a0401bb65ab74c1 cb55d41da284a5869a0401bb65ab74c1

# this produces differing results
use Data::Structure::Util qw(signature);
print signature( $ref1 ), " ", signature( $ref2 ), "\n";
# 5d20c5e81a53b2be90521167aefed9db 8b4cba2cbae0fec4bab263e9866d3911

🎀 Object Blessing 🎀

💖 unbless($ref)

Remove the blessing from any objects found within the passed data structure. For example:

my $foo = {
    'a' => bless( { 'b' => bless( {}, "c" ), }, "d" ),
    'e' => [ bless( [], "f" ), bless( [], "g" ), ]
};

use Data::Dumper;
use Data::Structure::Util qw(unbless);
print Dumper( unbless( $foo ) );

$VAR1 = {
    'a' => { 'b' => {} },
    'e' => [ [], [] ]
};

Note that the structure looks inside blessed objects for other objects to unbless 🔍.

💖 get_blessed($ref)

Examine the data structure and return a reference to a flat array that contains every object in the data structure you passed. For example:

my $foo = {
    'a' => bless( { 'b' => bless( {}, "c" ), }, "d" ),
    'e' => [ bless( [], "f" ), bless( [], "g" ), ]
};

use Data::Dumper;
# tell Data::Dumper to show nodes multiple times
$Data::Dumper::Deepcopy = 1;
use Data::Structure::Util qw(get_blessed);
print Dumper( get_blessed( $foo ) );

$VAR1 = [
    bless( {}, 'c' ),
    bless( { 'b' => bless( {}, 'c' ) }, 'd' ),
    bless( [], 'f' ),
    bless( [], 'g' )
];

This function is essentially the same as get_refs but only returns blessed objects rather than all objects. As with that function, the data structure is traversed depth first, so the top most objects should be the last elements of the array. Note also (as the above example shows) that objects within objects are returned 🫶.

🦋 utf8 Manipulation Functions 🦋

These functions allow you to manipulate the state of the utf8 flags in the scalars contained in the data structure. Information on the utf8 flag and its significance can be found in Encode.

🌈 has_utf8($var)

Returns $var if the utf8 flag is enabled for $var or any scalar that a data structure passed in $var contains.

print "this will be printed"  if defined has_utf8( "\x{1234}" );
print "this won't be printed" if defined has_utf8( "foo bar" );

Note that you should not check the truth of the return value of this function when calling it with a single scalar, as it is possible to have a string "0" or "" for which the utf8 flag set; since undef can never have the utf8 flag set, the function will never return a defined value if the data structure does not contain a utf8 flagged scalar 👑.

🌈 _utf8_off($var)

Recursively disables the utf8 flag on all scalars within $var. This is the same as the _utf8_off function of Encode but applies to any string within $var. The data structure is converted in-place, and as a convenience the passed variable is returned from the function.

This function makes no attempt to do any character set conversion to the strings stored in any of the scalars in the passed data structure. This means that if perl was internally storing any character as a sequence of bytes in the utf8 encoding, each byte in that sequence will then be henceforth treated as a character in its own right 😬.

For example:

my $emoticons = { smile => "\x{236a}" };
use Data::Structure::Util qw(_utf8_on);
print length( $emoticons->{smile} ), "\n";    # prints 1
_utf8_off( $emoticons );
print length( $emoticons->{smile} ), "\n";    # prints 3

🌈 _utf8_on($var)

Recursively enables the utf8 flag on all scalars within $var. This is the same as the _utf8_on function of Encode but applies to any string within $var. The data structure is converted in-place and as a convenience the passed variable is returned from the function.

As above, this makes no attempt to do any character set conversion, meaning that unless your string contains the valid utf8 byte sequences for the characters you want, you are in trouble. In some cases incorrect byte sequences can segfault perl 💀. In particular, the regular expression engine has significant problems with invalid utf8 that has been incorrectly marked as utf8. You should know what you are doing if you are using this function; consider using the Encode module as an alternative.

⚠️ bestie, this one is literally dangerous: incorrect byte sequences can segfault perl, and invalid utf8 marked as utf8 can make the regex engine have a whole crisis. Know what you're doing 💅.

Contrary example to the above:

my $emoticons = { smile => "\342\230\272" };
use Data::Structure::Util qw(_utf8_on);
print length( $emoticons->{smile} ), "\n";    # prints 3
_utf8_on( $emoticons );
print length( $emoticons->{smile} ), "\n";    # prints 1

💖 utf8_on($var) 💖

Okay so this routine performs a sv_utf8_upgrade on every scalar string in the data structure you pass in that does not already have the utf8 flag turned on. Basically it makes perl switch how it stores that string internally — from the native encoding (normally Latin-1 unless locales come into effect) into a utf8 encoding — and then it sets the utf8 flag for that scalar. Slay.

That means single byte letters are now represented by multi-byte sequences. But bestie, don't panic: as long as the use bytes pragma is not in effect, the string will be the same length, because as far as perl is concerned the string still contains the same number of characters (but not bytes). It's giving "same vibe, different storage" ✨.

This routine is significantly different from _utf8_on. That one assumes your string is encoded in utf8 but was marked (wrongly) in the native encoding. This routine assumes your string is encoded in the native encoding and is marked that way — but you'd rather it be encoded and marked as utf8. iykyk 💅

🦋 utf8_off($var) 🦋

This routine performs a sv_utf8_downgrade on every scalar string in the passed data structure that has the utf8 flag turned on. It makes perl change the way it stores the string internally — from the utf8 encoding into the native encoding (normally Latin-1 unless locales are used) — and disables the utf8 flag for that scalar. So multiple byte sequences that represent a single character get replaced by one byte per character. Cute.

However, as long as the use bytes pragma is not in effect, the string will be the same length, because as far as perl is concerned the string still contains the same number of characters (but not bytes). Same character count, different byte situation — no cap.

⚠️ Please note: not all strings can be converted from utf8 to the native encoding. If the utf8 character has no corresponding character in the native encoding, Perl will die with the "Wide character in subroutine entry" exception. So yeah, that's a real "handle with care" moment 😭.

This routine is significantly different from _utf8_off. That one assumes your string is encoded in utf8 and that you want to simply mark it as being in the native encoding, so that perl will treat every byte that makes up the character sequences as a character in its own right in the native encoding. This routine assumes your string is encoded in utf8, but you want each character that is currently represented by multi-byte strings to be replaced by the single byte representation of the same character. Periodt 💖

---

🌈 SEE ALSO 🌈

• Encode
• Scalar::Util
• Devel::Leak
• Devel::LeakTrace

Also, go read the excellent article http://www.perl.com/pub/a/2002/08/07/proxyobject.html from Matt Sergeant for more info on circular references. It's iconic, fr fr 🫶

🎀 REPOSITORY 🎀

https://github.com/AndyA/Data--Structure--Util

That's the link, bestie — keep it safe 💅

💀 BUGS 💀

signature() is sensitive to the hash randomisation algorithm. Yep, that's the whole warning. Not me being vague — that's just how it is 🙈.

This module only recurses through basic hashes, lists and scalar references. It doesn't attempt anything more complicated. So don't expect main character energy from it in wild data structures, okay? 💀

🫶 THANKS TO 🫶

James Duncan and Arthur Bergman who helped me and found a name for this module. Leon Brocard and Richard Clamp have provided invaluable help to debug this module. Mark Fowler rewrote large chunks of the documentation and patched a few bugs. Absolute legends, all of them 👑✨

👑 AUTHOR 👑

This release by Andy Armstrong <andy@hexten.net>

Originally by Pierre Denis <pdenis@fotango.com>

http://opensource.fotango.com/

🎀 COPYRIGHT 🎀

Copyright 2003, 2004 Fotango - All Rights Reserved.

This module is released under the same license as Perl itself.

---

This page is perl v5.42.0, dated 2015-04-16, and it lives under Data::S...re::Util(3). Bye bestie — go convert some encodings and slay 💅✨🌸