PHILOSOPHY(1)

💫 PHILOSOPHY(1) 💫

okay so this is a page from the User Contributed Perl Documentation (that's the manual collection it lives in, bestie 🫶), and it's basically the origin story of PDL — like the lore drop before the main character arc. Buckle up. 🎀

---

✨ NAME ✨

PDL::Philosophy — literally why did we even write PDL? bestie, we're about to find out 💅

---

🌸 DESCRIPTION 🌸

So this is some history straight from the creator of PDL, which then rolls into the whole philosophy and motivation sitch behind this data language. It's an attempt to summarize the common spirit among PDL developers, so that when someone asks "Why PDL?" there's an actual answer and not just vibes. 😭

🦋 The Start of PDL 🦋

"Why is it that we entertain the belief that for every purpose odd numbers are the most effectual?" — Pliny the Elder

The PDL project kicked off in February 1996, when the creator decided to mess around and experiment with writing his own Data Language. Context check: he's an astronomer. Day job = a lot of analysis of digital data accumulated over many nights observing on telescopes all around the world. We're talking images containing millions of pixels, and thousands of images of distant stars and galaxies. Or, even more abstrusely, many hundreds of digital spectra revealing the secrets of the composition and properties of these distant objects. iconic science behavior honestly 🔭✨

Obviously many astronomers have dealt with these problems before, and a huge amount of software has been built to make their analysis easier. But — like lots of colleagues — he was constantly frustrated by the lack of generality and flexibility of these programs, and by how hard it was to do anything out of the ordinary quickly and easily. What he wanted had a name: "Data Language", i.e. a language that lets you manipulate large amounts of data with simple arithmetic expressions. Some commercial software actually worked like this, and he was impressed with the capabilities but not with the price tag. And he thought he could do better. main character energy fr fr 👑

As a fairly computer literate astronomer (read "nerd" or "geek" according to your local argot), he was very familiar with "Perl", a computer language which now seems to fill the shelves of many bookstores around the world. He was impressed by its power and flexibility, and especially its ease of use. He'd even explored the depths of its internals and written an interface to allow graphics — the ease with which he could then create charts and graphs for his papers was refreshing. 💖

Version 5 of Perl had just been released, and he was fascinated by the new features available. Especially the support of arbitrary data structures (or "objects" in modern parlance) and the ability to "overload" operators — i.e. make mathematical symbols like +-*/ do whatever you felt like. It seemed to him it ought to be possible to write an extension to Perl where he could play with data in a general way: for example, using the maths operators to manipulate whole images at once. 🧚

Well, one slow night at an observatory he just thought he'd try a little experiment. In a bored moment he fired up a text editor and started to create a file called PDL.xs — a Perl extension module to manipulate data vectors. A few hours later he actually had something half decent working, where he could add two images in the Perl language, fast! This was something he could not let rest, and it probably cost him one or two scientific papers worth of productivity 😭. A few weeks later the Perl Data Language version 1.0 was born. It was a pretty bare infant: very little was there apart from the basic arithmetic operators. But encouraged, he made it available on the Internet to see what people thought. 🌈

Well, people were fairly critical — among the most vocal were Tuomas Lukka and Christian Soeller. Unfortunate for them, they were both Perl enthusiasts too, and soon found themselves improving his code to implement all the features they thought PDL ought to have and that he had heinously neglected. 💀 PDL is a prime example of that modern phenomenon of authoring large free software packages via the Internet. Large numbers of people, most of whom have never met, have made contributions ranging from core functionality to large modules to the smallest of bug patches. PDL version 2.0 is now here (though it should perhaps have been called version 10 to reflect the amount of growth in size and functionality) and the phenomenon continues. He firmly believes PDL is a great tool for tackling general problems of data analysis. It's powerful, fast, easy to add to, and freely available to anyone. He wishes he'd had it when he was a graduate student! And he hopes you too will find it of immense value, and that it'll save you heaps of time and frustration solving complex problems. Of course it can't do everything, but it provides the framework, the hammers and the nails for building solutions without having to reinvent wheels or levers. 🔥

— Karl Glazebook, the creator of PDL

---

👑 Major ideas 👑

The first tenet of the philosophy is the "free software" idea: software being free has several advantages — fewer bugs because more people see the code, you can have the source and port it to your own working environment with you, ... and of course, you don't need to pay anything. 💸

The second idea is a pet peeve of many: lots of languages like Matlab are pretty well suited for their specific tasks, but for a different application you need to switch to an entirely different tool and regear yourself mentally. Not to speak of doing an application that does two things at once... 😭 Because we use Perl, we have the power and ease of Perl syntax, regular expressions, hash tables, etc. right at our fingertips at all times. By extending an existing language, we start from a much healthier base than languages like Matlab, which have grown into existence from a very small functionality at first and expanded little by little, making things look badly planned. We stand by the Perl sayings: "simple things should be simple but complicated things should be possible" and "There is more than one way to do it" (TIMTOWTDI). periodt ✨

The third idea is interoperability: we want to be able to use PDL to drive as many tools as possible — we can connect to OpenGL or Mesa for graphics, or whatever. There isn't anything out there that's really satisfactory as a tool and can do everything we want easily. And be portable. 🌍

The fourth idea is related to PDL::PP and is Tuomas's personal favorite: code should only specify as little redundant info as possible. If you find yourself writing very similar-looking code much of the time, all that code could probably be generated by a simple Perl script. The PDL C preprocessor takes this to an extreme. 🤯

---

🍓 Minor goals and purposes 🍓

We want speed. Optimally, it should ultimately (e.g. with the Perl compiler) be possible to compile PDL::PP subs to C and obtain the top vectorized speeds on supercomputers. Also, we want to be able to calculate things at near top speed from inside Perl, by using dataflow to avoid memory allocation and deallocation — the overhead should ultimately be only a little over one indirect function call plus a couple of ifs per function in the pipe. ⚡

---

🎀 Go on, try it! 🎀

So yeah, that's the philosophy behind PDL — speed, conciseness, free, expandable, and integrated with the wide base of modules and libraries that Perl provides. Feel free to download it, install it, run through some of the tutorials and introductions, and have a play with it. 🫶

Enjoy! 💜

---

🌟 AUTHOR 🌟

Added Karl Glazebrook (2001), contributions by Matthew Kenworthy.

Copyright(C) 1997 Tuomas J. Lukka (lukka@fas.harvard.edu).

Redistribution in the same form is allowed but reprinting requires a permission from the author. (so like, share it around, but don't reprint it without asking first, bestie 🙈)

---

and that's the whole page!! 💅 this version shipped with perl v5.32.1, dated 2022-04-11 — so go forth, manipulate some data, and slay ✨🦋