MAKEPP_SPEEDUP(1)

💅 MAKEPP_SPEEDUP(1) — the makepp speed-up page ✨

okay bestie, listen up, this is the Makepp manual's makepp_speedup page and it's literally all about making makepp go zoom 🏎️💨

---

🌸 NAME 🌸

makepp_speedup — how to make makepp faster 🔥

---

✨ DESCRIPTION ✨

so like. you think makepp is slow? 🙈 honestly, it has gotten noticeably faster, but fair, it's still slow, especially if you're coming over from GNU make. and the reason is kinda iconic actually: makepp conscientiously checks ALL those things that give gmake a headache, the whole "ignoring lots of dependencies" thing (you know, that classic "I think I need to gmake clean to get rid of a mysterious bug" syndrome 💀).

if you suspect some Perl code you added to your makefiles might be the one at fault here, go peek at perl_performance 👀

but!! there are a few things you can do to squeeze out more speed 💪 some of these things are labeled unsafe, in the sense that you're asking makepp not to check or do certain stuff, because you think those things aren't needed. and like — if those things would have been necessary, then the build may not be correct 😭. luckily tho, that problem is only temporary! it gets corrected as soon as you let makepp do all its checks again ✨

and you can combine several of these tips to increase the time gain even more 🎀

---

🛟 Safe Methods 🛟

🦋 Use makeppreplay

the stand-alone utility makeppreplay, aka mppr, repeats things that makepp has already done — with zero overhead. slay 💅

⚡ Use a Faster Perl

within version 5.8, they're all roughly the same, only 5.8.7 is a bit faster. tuning your Perl can help too, like not compiling it for 64 bits, which makepp doesn't even need. for example ActiveState's build (<http://www.activestate.com/activeperl>) of 5.8.7 for Linux is faster than the Perl 5.8.7 that ships with SuSE Linux 10.0 👑

🎀 Include as Little as Possible

every single extra file you include is doubly penalizing, and not in a cute way. on one hand, the compiler has to look for and at all those files — you don't notice this much, since it's just a little extra per compiler call. on the other hand, makepp has to look too, to find dependencies and figure out whether they incur a rebuild. and then it can seem to stall, while it's digesting a whole lot of dependencies at once 😵💫

an absolutely deadly variant of this is the project master include file, the one that conveniently includes anything you might need. the result? any header file change leads to a full build. and even without a change, makepp has to think about all those headers again, for every source you compile. just a tiny effort, since it's cached, but thousands of files can make this staggering 💀

it may be cumbersome to figure out the minimal set of includes, and to clean up the ones no longer needed, but it really pays off, fr fr. and if anybody knows a tool that can identify which files get included unnecessarily, the author would be glad to mention it here 🫶

🌈 Build as Little as You Need

if you have a default target which makes several programs, then makepp has to check all their dependencies, right down to the smallest header file. but maybe you only want to test your change with one of those programs 👀

in that case you'd call makepp with an explicit target. the less modules or headers all those programs have in common, the greater the benefit of not letting makepp check them all ✨

say your top level Makeppfile has this rule:

$(phony all): proggie1 proggie2 $(only_phony_targets */**/all)

then you'd call things like

$ makepp proggie2
$ makepp proggie1 dir/subdir/proggie27

💖 Use preferred makefile names

Makepp looks for makefiles (unless you specify them explicitly on the command line or with load-makefile) in this order: RootMakeppfile, RootMakeppfile.mk, Makeppfile and Makeppfile.mk, followed by the classical makefile names. (the .mk variants are for purely suffix-based systems.)

so if you use RootMakeppfile at the root of your build tree, and Makeppfile everywhere else, the files get found slightly faster 🏃‍♀️ Makepp also gets a slightly smaller memory consumption (it caches the fact that the other names don't exist), which means speed through less memory management. double win 💅

likewise, if you have a statement

include standard

there will first be an attempt to find standard.makepp, so you might as well just use that name ✨

🍓 Have as few rules as you need

Makepp keeps track not only of existent files, but also of any it learns to create. (that's literally why it offers reliable wildcards like *.o.) but the price for this power is a lot of management 😭 so if you tell it how to create a .o from a .c, that's fine, because it will happen for most if not all candidates.

but if you tell it how to link any suffixless executable from a like-named .o, that's expensive, because it'll probably only happen for a small part of them (the ones that contain a main function), but the basis gets laid for all of them. so you gotta weigh the comfort of a linker pattern rule against the efficiency of individual linker rules ⚖️

if you don't use any of them, you should also turn off the builtin rules with:

makepp_no_builtin = 1

if you do use them, but — for the reasons explained above — not the builtin linker rules, you should turn those off with:

makepp_no_builtin_linker = 1

🧚 Put makepp extensions into a module

Makepp offers very convenient possibilities of being extended through Perl 🐪 but if you write some functions, commands or statements in a file and include that from dozens of makefiles, you will get dozens of copies of them all in memory. and they'll be read dozens of times by the makepp parser, which is a bit slower than Perl's.

in this situation it is better to put your own functions into a module 🎀

🫶 Use Repositories and/or a Build Cache

if you have several developers working on the same machine, or if you change to and fro between sets of build options, this is for you 💖 repositories allow you to offer a central reference where you only need to build what is locally different. a build cache simply collects all produced files, and reuses them as appropriate, with less planning needed. the latter page also describes the differences ✨

🌸 Use Sandboxes

if your build is so big that makepp is having a hard time digesting all the information, and if you can find a way of splitting it up into smaller independent parts, sandboxes might give you better parallelism than the --jobs option 👑

🙈 Don't log what you do

Makepp's logging feature is very powerful for tracking down bugs in the build system, or for analyzing your dependencies. but whenever you're not doing those things, you can save quite a bit of formatting and I/O with --no-log --no-scan-log 💅

---

😬 Almost Safe Methods 😬

⏳ Get a Headstart

the option --loop (or --stop-before-building or --stop-after-loading or --stop) allows makepp to start its work while you're still editing ✨ it will repeatedly suspend itself when it gets to the point of analyzing the dependencies. you decide when you're ready to let it go on. on the author's huge project this saves half a minute, and that's only when they have a CPU to themselves 😤

this method has two potential drawbacks:

• Makeppfiles have been read by the time makepp stops. if you edit a Makeppfile or something from which it would have to be rebuilt, after starting makepp, this will go unnoticed till the next time. but this should rarely be necessary, since makepp greatly reduces the need for Makeppfile changes.
• if a target depends on a wildcard, and that would match more than when the Makeppfile was read, makepp will not notice:

proggie: *.o
    $(LD) $(inputs) -o $(output)

if you add another source file, or a file from which makepp knows how to generate a source, then *.o should match the object that produces. but if this file was added after starting makepp, it will not, because the wildcard was expanded too early 😭

in both of these cases you should kill the prestarted makepp and start it anew 🔪

🧳 Gulliver's Travels

the option --gullible tells makepp to believe that a rule changes what it says it will, neither less nor more. not performing these checks can save a few percent of makepp's CPU time 💪 and the Disk I/O savings is especially welcome on network file systems 🌐 if you do nightly full builds in an empty directory with the --repository option, but without the --gullible option, you can be fairly sure that your rule set is consistent. then this option shouldn't hurt in your daytime work ✨

---

🚨 Potentially Unsafe Methods 🚨

these methods are unsafe if you give makepp the wrong hints. but everything will again be fine, however, as soon as you let makepp do all the checks, by not passing it any limiting options 🫶 for this reason the author suggests using these hints to get quick intermediate builds, and using lunchtime and nights to let makepp do its job thoroughly 🌙

💀 Build as Little as Needed

this is the same tip of using explicit targets discussed under "Build as Little as You Need" above. but it becomes more dangerous if you do it because you're sure that your change will not affect any of the other programs. then they will not be built, even though it might have been necessary 😬

🙅‍♀️ Know Where Not to Build

the option --dont-build is very powerful for speeding makepp up a lot. if you know one or more directories which you are sure are unaffected by any change you made since the last time, you can issue --dont-build options for them. this can save makepp a lot of dependency analysis 🔥 but it will not build anything in those directories, even if it should have 😭

📍 Know Where to Build

this is the same as "Know where not to build", but instead of an exclusion list, you supply an inclusion list. the trick is that a --do-build option, with a --dont-build=/ option or under a RootMakeppfile(.mk) directory without a --dont-build option on a higher level directory, means: build nothing except what I tell you to 💅 this is what users of traditional makes are looking for when they want to build just one directory:

$ makepp --do-build=dir/subdir

or, if you don't have a RootMakeppfile(.mk):

$ makepp --dont-build=/ --do-build=dir/subdir

the difference is that any default target in the top level Makeppfile, i.e. link commands, are also executed this way. if you don't want that, you must give an explicit target, which is automatically also marked for --do-build:

$ makepp --do-build=dir1/subdir dir2/proggie

🎯 Know What to Build

an extreme variant is asking makepp not to build anything but what you tell it to. this is not so dangerous if you changed no include files, only modules, and you know which programs they go into 👀

say you have only changed src/a.cpp and src/b.cpp and these are linked directly into one program. dot is the current directory including all subdirectories.

$ makepp --dont-build=. src/a.o src/b.o proggie1

Or, like, same thing but cuter: since --do-build with no --dont-build hanging around on a higher-up directory means the root of the build tree gets an implied --dont-build, you can just say:

    $ makepp --do-build=src/a.o src/b.o proggie1

✨ Pro tip bestie: you can totally stash something like this in your shell's $ENV file or your .profile so you're not typing the whole novel every single time. (csh girlies, swap the = for a space, mwah 💋)

    alias mppb='makepp --do-build'
    alias mppsb='makepp --stop --do-build'

and then that last example we did just becomes, like, so much shorter:

    $ mppb src/a.o src/b.o proggie1

---

🌈 Build on a RAM disk 🌈

Okay so here's the tea ☕ — modern computers, especially servers, usually have a ridiculously high mean time between failure. Like, they just don't crash. If that's your situation and you've got RAM to spare (flex 💅), you can absolutely save yourself all that waiting-for-I/O time by building in RAM.

But listen — you should edit on a real disk, or replicate your edits over there quickly. The build results are reproducible though, so those can live in RAM, no cap. And if you don't wanna risk rebuilding everything, you can always replicate to disk after each build, or overnight while you sleep 🛌.

Not me warning you though: do NOT do that replication during the build!! You might catch partially written files — which is literally the same energy as if the machine had just crashed. 💀

Also ngl, if you've got a system and/or storage unit with good caching and RAID, the gain might not be so big. Just saying. 🤷‍♀️

---

👑 AUTHOR 👑

Daniel Pfeiffer <occitan@esperanto.org> — the icon, the legend, the main character who wrote all this. 🫶

---

and that's a wrap on the whole MAKEPP_SPEEDUP(1) moment, bestie! ✨ slay ✨ This one's perl v5.32.0, dated 2021-01-06, and it's the MAKEPP_SPEEDUP(1) page — go forth and build fast, you've got this 💖🦋🌸