PERLINTERP(1perl)

💅 PERLINTERP — an overview of the Perl interpreter 🐪✨

okay bestie, so this is PERLINTERP(1perl) from the Perl Programmers Reference Guide, and it's genuinely the "how does the magic actually work" doc. grab a snack, we're going into the C guts 💖

🌸 NAME 🌸

perlinterp — an overview of the Perl interpreter. no cap, that's literally the whole job description 🫶

✨ DESCRIPTION ✨

So this doc walks you through how the Perl interpreter actually works down at the level of C code, and it also hands you pointers to the relevant C source files so you can go read them yourself like the icon you are. 📂👑

---

🧚 ELEMENTS OF THE INTERPRETER 🧚

The interpreter's whole life is basically two main stages:

1. compiling your code into the internal representation — aka bytecode — and then
2. executing it.

If you want the exact tea on how the compilation stage goes down, that's "Compiled code" in perlguts. she explains it properly, go read her. 💅

Here's the short breakdown of how perl does its thing:

🌱 Startup 🌱

The action kicks off in perlmain.c (or miniperlmain.c if you're running miniperl). This is very high-level code — like, it fits on a single screen — and it lowkey resembles the code you'd find in perlembed. but here's the thing: most of the real action is happening over in perl.c. 👀

Fun fact: perlmain.c is generated by "ExtUtils::Miniperl" from miniperlmain.c at make time, so if you wanna follow along at home, you should make perl first. do it bestie, don't skip. 🛠️

First up, perlmain.c allocates some memory and constructs a Perl interpreter, roughly like this:

1 PERL_SYS_INIT3(&argc,&argv,&env);
2
3 if (!PL_do_undump) {
4     my_perl = perl_alloc();
5     if (!my_perl)
6         exit(1);
7     perl_construct(my_perl);
8     PL_perl_destruct_level = 0;
9 }

okay let's break this down line by line, because it's giving ✨annotated✨:

• Line 1 is a macro, and what it actually expands to depends on your operating system. yes, the C preprocessor is doing preprocessor things, we love that for us. 🫠
• Line 3 references "PL_do_undump", which is a global variable — and heads up, every single global variable in Perl starts with "PL_". This one tells you whether the currently running program was created with the -u flag to perl and then undump'd. which in any sane context means it's gonna be false. 💀
• Line 4 calls a function in perl.c that allocates the memory for a Perl interpreter. it's a pretty simple function, and the guts of it look like this:

my_perl = (PerlInterpreter*)PerlMem_malloc(sizeof(PerlInterpreter));

and boom, here's your first taste of Perl's system abstraction, which we're gonna keep seeing: "PerlMem_malloc" is either your system's "malloc", or Perl's own "malloc" as defined in malloc.c — depending on whether you picked that option at configure time. iykyk 🎀

• Next, on line 7, we construct the interpreter using perl_construct, which is also in perl.c. this one sets up all the special variables Perl needs, the stacks, all of it. she does a lot. 👑
• Then we hand Perl the command line options and tell it to GO:

if (!perl_parse(my_perl, xs_init, argc, argv, (char **)NULL))
    perl_run(my_perl);

exitstatus = perl_destruct(my_perl);

perl_free(my_perl);

And real talk: "perl_parse" is actually just a wrapper around "S_parse_body", as defined in perl.c. What that does is process the command line options, set up any statically linked XS modules, open the program, and then call "yyparse" to parse it. that's the whole pipeline bestie. 🌈

🔍 Parsing 🔍

The goal of this stage is simple to say and wild to do: take the Perl source, and turn it into an op tree. we'll see what one of those actually looks like a bit later, don't worry. and strictly speaking? there are three things going on here, not one. 🙈

"yyparse" — the parser — lives in perly.c. however, you're genuinely better off reading the original YACC input in perly.y. and yes, Virginia, there IS a YACC grammar for Perl! 😭 iconic behavior honestly. The parser's job is to take your code and "understand" it: splitting it into sentences, deciding which operands go with which operators, that kind of thing.

The parser is nobly assisted by the lexer, which chunks up your input into tokens and decides what type of thing each token is — a variable name, an operator, a bareword, a subroutine, a core function, etc, etc. The main point of entry to the lexer is "yylex", and that plus its associated routines can be found in toke.c.

Here's the thing though: Perl isn't much like other computer languages. she's highly context sensitive at times, and it can be genuinely tricky to figure out what sort of token something is, or where a token even ends. so there's a whole lot of interplay between the tokeniser and the parser, which can get pretty frightening if you're not used to it. not me getting jump scared by a lexer 💀

As the parser understands a Perl program, it builds up a tree of operations for the interpreter to perform during execution. The routines that construct and link together all those various operations live in op.c, and we'll examine them later. 🎀

💫 Optimization 💫

So now the parsing stage is done, and the finished tree represents all the operations Perl needs to perform to execute our program. Next up, Perl does a dry run over the tree looking for optimisations:

• constant expressions such as "3 + 4" get computed right now, immediately, no waiting. ✨
• the optimizer also checks whether any multiple operations can be replaced with a single one.

For instance — to fetch the variable $foo, instead of grabbing the glob *foo and looking at the scalar component, the optimizer fiddles the op tree so it uses a function that directly looks up the scalar in question. efficiency queen 👑

The main optimizer is "peep" in op.c, and lots of ops have their own optimizing functions. she's not doing this alone, it's a team effort. 🫶

🏃 Running 🏃

Now we're finally ready to go: we've got compiled Perl byte code, and all that's left is to run it. The actual execution is done by the "runops_standard" function in run.c; more specifically, it's done by these three extremely innocent looking lines:

while ((PL_op = PL_op->op_ppaddr(aTHX))) {
    PERL_ASYNC_CHECK();
}

You might be more comfortable with the Perl version of that:

PERL_ASYNC_CHECK() while $Perl::op = &{$Perl::op->{function}};

well. maybe not. 😭 anyway!

Each op contains a function pointer, which stipulates the function that will actually carry out the operation. That function returns the next op in the sequence — and that's what allows things like "if", which pick the next op dynamically at run time. main character energy, choosing her own path. ✨ And the "PERL_ASYNC_CHECK" is there to make sure that things like signals interrupt execution if required. safety first bestie. 🚨

The actual functions being called are known as PP code, and they're spread between four files:

• pp_hot.c — contains the "hot" code, which is the most often used and highly optimized stuff. 🔥
• pp_sys.c — contains all the system-specific functions.
• pp_ctl.c — contains the functions which implement control structures ("if", "while" and the like).
• pp.c — contains everything else. the catch-all bin, we respect her. 🫡

These are, if you like, the C code for Perl's built-in functions and operators.

Quick note, and this one matters: each "pp_" function is expected to return a pointer to the next op. Calls to perl subs (and eval blocks) are handled within the same runops loop, and they do not consume extra space on the C stack. For example, "pp_entersub" and "pp_entertry" just push a "CXt_SUB" or "CXt_EVAL" block struct onto the context stack, which contain the address of the op following the sub call or eval. They then return the first op of that sub or eval block, and so execution just continues on through that sub or block. Later, a "pp_leavesub" or "pp_leavetry" op pops the "CXt_SUB" or "CXt_EVAL", retrieves the return op from it, and returns it. it's giving elevator music, everything flows. 🎶

💀 Exception handing 💀

(yes the heading really is spelled "handing", we're keeping it exactly as-is, periodt 💅)

Perl's exception handling (i.e. "die" and friends) is built on top of the low-level setjmp()/longjmp() C-library functions. What those basically do is give you a way to capture the current PC and SP registers of the CPU and then restore them later. In other words: a longjmp() continues at the point in code where a previous setjmp() was done, with anything further up on the C stack being lost. 😭

This is exactly why code should always save values using "SAVE_FOO" rather than in auto variables. do not fumble this one bestie, the C stack will simply not have your back.

The perl core wraps setjmp() and longjmp() in the macros "JMPENV_PUSH" and "JMPENV_JUMP". The push operation, as well as setting a setjump(), stores some temporary state in a struct local to the current function (allocated by "dJMPENV"). In particular, it stores a pointer to the previous "JMPENV" struct, and updates "PL_top_env" to point to the newest one — forming a chain of "JMPENV" states. 🔗 And both the push and the jump can output debugging information under perl -Dl. 🙈

A basic rule of the perl internals is that all interpreter exits are achieved via a JMPENV_JUMP(). In particular:

• level 2: perl-level exit() and internals my_exit()

These unwind all stacks, then perform a JMPENV_JUMP(2).

• level 3: perl-level die() and internals croak()

If currently within an eval, these pop the context stack back to the nearest "CXt_EVAL" frame, set $@ as appropriate, set "PL_restartop" to the op which follows the eval associated with that frame, then perform a JMPENV_JUMP(3).

Otherwise, the error message is printed to "STDERR", and then it's treated as an exit: unwind all stacks and perform a JMPENV_JUMP(2).

• level 1: unused

JMPENV_JUMP(1) is currently unused except in perl_run().

• level 0: normal return.

The zero value is for a normal return from JMPENV_PUSH().

So the perl interpreter expects that, at all times, there is a suitable "JMPENV_PUSH" set up (and at a suitable location within the CPU call stack) that can catch and process a 2- or 3-valued jump — and in the case of a 3, start a new runops loop to execute "PL_restartop" and all remaining ops (as will be explained shortly, stay tuned ✨).

The entry points to the perl interpreter all provide such a facility. For example, perl_parse(), perl_run() and "call_sv(cv, G_EVAL)" all contain something similar in outline to:

{
    dJMPENV;
    JMPENV_PUSH(ret);
    switch (ret) {
    case 0:                     /* normal return from JMPENV_PUSH() */
      redo_body:
        CALLRUNOPS(aTHX);
        break;
    case 2:                     /* caught longjmp(2) - exit / die */
        break;
    case 3:                     /* caught longjmp(3) - eval { die } */
        PL_op = PL_restartop;
        goto redo_body;
    }

    JMPENV_POP;
}

A runops loop such as Perl_runops_standard() (as set up by CALLRUNOPS()) is, at its heart, just a simple:

while ((PL_op = PL_op->op_ppaddr(aTHX))) { 1; }

...which calls the pp() function associated with each op, relying on that to return a pointer to the next op to be executed. simple, elegant, no notes. 💅

Now, as well as setting catches at the entry points to the perl interpreter, you might expect perl to also do a JMPENV_PUSH() in places like pp_entertry(), just before some trappable ops are executed. And in fact, perl doesn't normally do this. The drawback with doing it is that with nested or recursive code such as:

sub foo { my ($i) = @_; return if $i < 0; eval { foo(--$i) } }

...the C stack would quickly overflow with pairs of entries like:

...
#N+3 Perl_runops()
#N+2 Perl_pp_entertry()
#N+1 Perl_runops()
#N   Perl_pp_entertry()
...

not the vibe. 💀 Instead, perl puts its guards at the callers of runops loops. Which means: as many nested subroutine calls and evals may be called as you like, all within the one single runops loop. 🎀 And if an exception occurs, control passes back to the caller of the loop, which just immediately restarts a new loop with "PL_restartop" being the next op to call. clean, efficient, iconic. ✨

okay so bestie, picture normal operation: when there's several nested evals happening, you get multiple CXt_EVAL context stack entries, right? But there's still only ONE single runops loop, guarded by a single JMPENV_PUSH. Every caught eval pops the next CXt_EVAL off the stack, sets PL_restartop, then longjmp()s back to perl_run() and keeps it moving. neat. 💅

BUT. Plot twist incoming, and it's giving chaos. 👀

Ops are sometimes executed within an inner runops loop — think a tie, a sort, or overload code. And in that case, something like

sub FETCH { eval { die }; .... }

would, unless it's handled specially, cause a longjmp() right back to the guard in perl_run(), popping both runops loops — which is clearly incorrect, like, not it at all. 😭

One way to dodge this is to have the tie code itself do a JMPENV_PUSH before executing FETCH in the inner runops loop. But for efficiency reasons, perl actually just temporarily sets a flag using CATCH_SET(TRUE). That flag warns any subsequent require, entereval or entertry ops that the caller is no longer promising to catch any raised exceptions on their behalf. 🚩

These ops check that flag, and if it's true, they (via docatch()) do a JMPENV_PUSH and start a brand new runops loop to execute the code, rather than doing it with the current loop. main character behavior honestly. ✨

As a consequence, when we exit the eval block in that FETCH above, execution of the code following the block is still carried on in the inner loop — i.e. the one established by the pp_entertry(). To avoid confusion, if a further exception is then raised, docatch() compares the JMPENV level of the CXt_EVAL with PL_top_env, and if they differ, it just re-throws the exception. That way any inner loops get popped, and the exception gets dealt with properly by the level that's expecting it. periodt. 🌸

---

Here's an example, no cap:

1: eval { tie @a, 'A' };
2: sub A::TIEARRAY {
3:     eval { die };
4:     die;
5: }

To run this code, perl_run() is called, which does a JMPENV_PUSH(), then enters a runops loop. This loop executes the entereval and tie ops on line 1, with the entereval pushing a CXt_EVAL onto the context stack.

Then pp_tie() does a CATCH_SET(TRUE), and starts a second runops loop to execute the body of TIEARRAY(). When that loop executes the entertry op on line 3, CATCH_GET() is true, so pp_entertry() calls docatch() which does a JMPENV_PUSH and starts a third runops loop. That loop restarts the pp_entertry(), then executes the die op. At this point the C call stack looks like this:

#10 Perl_pp_die()
#9  Perl_runops()      # runops loop 3
#8  S_docatch()        # JMPENV level 2
#7  Perl_pp_entertry()
#6  Perl_runops()      # runops loop 2
#5  Perl_call_sv()
#4  Perl_pp_tie()
#3  Perl_runops()      # runops loop 1
#2  S_run_body()
#1  perl_run()         # JMPENV level 1
#0  main()

and the context and data stacks, as shown by perl -Dstv, look like:

STACK 0: MAIN
  CX 0: BLOCK  =>
  CX 1: EVAL   => AV()  PV("A"\0)
  retop=leave
STACK 1: MAGIC
  CX 0: SUB    =>
  retop=(null)
  CX 1: EVAL   => *
  retop=nextstate

The die() pops the first CXt_EVAL off the context stack, sets PL_restartop from it, does a JMPENV_JUMP(3), and control returns to the JMPENV level set in docatch(). That then starts another third-level runops level, which executes the nextstate, pushmark and die ops from line 4. At the point the second pp_die() is called, the C call stack looks exactly like the one above — even though we are no longer within an inner eval. However, the context stack now looks like this, i.e. with the top CXt_EVAL popped:

STACK 0: MAIN
  CX 0: BLOCK  =>
  CX 1: EVAL   => AV()  PV("A"\0)
  retop=leave
STACK 1: MAGIC
  CX 0: SUB    =>
  retop=(null)

Then the die() on line 4 pops the context stack back down to the CXt_EVAL, leaving it as:

STACK 0: MAIN
  CX 0: BLOCK  =>

As usual, PL_restartop is extracted from the CXt_EVAL, and a JMPENV_JUMP(3) is done, which pops the C stack back to the docatch():

#8  S_docatch()        # JMPENV level 2
#7  Perl_pp_entertry()
#6  Perl_runops()      # runops loop 2
#5  Perl_call_sv()
#4  Perl_pp_tie()
#3  Perl_runops()      # runops loop 1
#2  S_run_body()
#1  perl_run()         # JMPENV level 1
#0  main()

In this case, because the JMPENV level recorded in the CXt_EVAL differs from the current one, docatch() just does a JMPENV_JUMP(3) to re-throw the exception, and the C stack unwinds to:

#1  perl_run()         # JMPENV level 1
#0  main()

Because PL_restartop is non-null, run_body() starts a new runops loop, and execution continues. 🎀

---

🧠 INTERNAL VARIABLE TYPES 💾

You should by now have had a look at perlguts, which tells you all about Perl's internal variable types: SVs, HVs, AVs and the rest. If you haven't? bestie, go do that now. fr fr. 🫶

These variables aren't just for representing Perl-space variables — they also cover any constants in the code, plus some structures that are completely internal to Perl. The symbol table, for instance, is an ordinary Perl hash. Your code is represented by an SV as it's read into the parser; any program files you call are opened via ordinary Perl filehandles, and so on. iconic. ✨

The core Devel::Peek module lets us examine SVs from a Perl program. Let's see, for instance, how Perl treats the constant "hello".

% perl -MDevel::Peek -e 'Dump("hello")'
1 SV = PV(0xa041450) at 0xa04ecbc
2   REFCNT = 1
3   FLAGS = (POK,READONLY,pPOK)
4   PV = 0xa0484e0 "hello"\0
5   CUR = 5
6   LEN = 6

Reading Devel::Peek output takes a bit of practise, so let's go through it line by line. 🦋

Line 1 tells us we're looking at an SV which lives at 0xa04ecbc in memory. SVs themselves are very simple structures, but they contain a pointer to a more complex structure. In this case, it's a PV — a structure which holds a string value — at location 0xa041450. Line 2 is the reference count; there are no other references to this data, so it's 1.

Line 3 is the flags for this SV — it's OK to use it as a PV, it's a read-only SV (because it's a constant) and the data is a PV internally. Next we've got the contents of the string, starting at location 0xa0484e0.

Line 5 gives us the current length of the string — and note that this does not include the null terminator. Line 6 is not the length of the string, but the length of the currently allocated buffer; as the string grows, Perl automatically extends the available storage via a routine called SvGROW. 💅

You can get at any of these quantities from C very easily; just add Sv to the name of the field shown in the snippet, and you've got a macro which will return the value: SvCUR(sv) returns the current length of the string, SvREFCOUNT(sv) returns the reference count, SvPV(sv, len) returns the string itself with its length, and so on. More macros to manipulate these properties can be found in perlguts. 🌈

Let's take an example of manipulating a PV, from sv_catpvn, in sv.c

1  void
2  Perl_sv_catpvn(pTHX_ SV *sv, const char *ptr, STRLEN len)
3  {
4      STRLEN tlen;
5      char *junk;

6      junk = SvPV_force(sv, tlen);
7      SvGROW(sv, tlen + len + 1);
8      if (ptr == junk)
9          ptr = SvPVX(sv);
10      Move(ptr,SvPVX(sv)+tlen,len,char);
11      SvCUR(sv) += len;
12      *SvEND(sv) = '\0';
13      (void)SvPOK_only_UTF8(sv);          /* validate pointer */
14      SvTAINT(sv);
15  }

This is a function which adds a string, ptr, of length len onto the end of the PV stored in sv. The first thing we do in line 6 is make sure that the SV has a valid PV, by calling the SvPV_force macro to force a PV. As a side effect, tlen gets set to the current value of the PV, and the PV itself is returned to junk.

In line 7, we make sure that the SV will have enough room to accommodate the old string, the new string and the null terminator. If LEN isn't big enough, SvGROW will reallocate space for us. 💖

Now, if junk is the same as the string we're trying to add, we can grab the string directly from the SV; SvPVX is the address of the PV in the SV.

Line 10 does the actual catenation: the Move macro moves a chunk of memory around — we move the string ptr to the end of the PV, i.e. the start of the PV plus its current length. We're moving len bytes of type char. After doing so, we need to tell Perl we've extended the string, by altering CUR to reflect the new length. SvEND is a macro which gives us the end of the string, so that needs to be a \0.

Line 13 manipulates the flags; since we've changed the PV, any IV or NV values will no longer be valid: if we have $x=10; $x.="6"; we don't want to use the old IV of 10. SvPOK_only_utf8 is a special UTF-8-aware version of SvPOK_only, a macro which turns off the IOK and NOK flags and turns on POK. The final SvTAINT is a macro which launders tainted data if taint mode is turned on. 🔥

AVs and HVs are more complicated, but SVs are by far the most common variable type being thrown around. Having seen something of how we manipulate these, let's go on and look at how the op tree is constructed. 👀

✨ OP TREES ✨

First things first, bestie: what even is the op tree? 💖 It's the parsed representation of your program — yes, the one from our parsing section — and it's the sequence of operations Perl goes through to execute your program, like we saw in "Running". 🦋

An op is a fundamental operation that Perl can perform: literally all the built-in functions and operators are ops, and there's also a whole lineup of ops that handle concepts the interpreter needs internally — entering and leaving a block, ending a statement, fetching a variable, and so on. 👑

The op tree is connected in two ways: imagine there are two "routes" through it, two orders you can traverse the tree. First, parse order reflects how the parser understood the code. Second, execution order tells perl the order to perform the operations in. ✨

The easiest way to examine the op tree is to stop Perl after it has finished parsing, and get it to dump out the tree. That is exactly what the compiler backends B::Terse, B::Concise and CPAN module B::Debug do. 💅

Let's look at how Perl sees $x = $y + $z:

     % perl -MO=Terse -e '$x=$y+$z'
     1  LISTOP (0x8179888) leave
     2      OP (0x81798b0) enter
     3      COP (0x8179850) nextstate
     4      BINOP (0x8179828) sassign
     5          BINOP (0x8179800) add [1]
     6              UNOP (0x81796e0) null [15]
     7                  SVOP (0x80fafe0) gvsv  GV (0x80fa4cc) *y
     8              UNOP (0x81797e0) null [15]
     9                  SVOP (0x8179700) gvsv  GV (0x80efeb0) *z
    10          UNOP (0x816b4f0) null [15]
    11              SVOP (0x816dcf0) gvsv  GV (0x80fa460) *x

Let's start in the middle, at line 4. This is a BINOP, a binary operator, sitting at location 0x8179828. The specific operator in question is sassign — scalar assignment — and you can find the code that implements it in the function pp_sassign in pp_hot.c. As a binary operator, it has two children: the add operator, providing the result of $y+$z, is uppermost on line 5, and the left hand side is on line 10. 🎀

Line 10 is the null op: this does exactly nothing. What is that doing there? If you see the null op, it's a sign that something has been optimized away after parsing. As we mentioned in "Optimization", the optimization stage sometimes converts two operations into one, for example when fetching a scalar variable. When this happens, instead of rewriting the op tree and cleaning up the dangling pointers, it's easier just to replace the redundant operation with the null op. Originally, the tree would have looked like this:

    10          SVOP (0x816b4f0) rv2sv [15]
    11              SVOP (0x816dcf0) gv  GV (0x80fa460) *x

That is, fetch the "a" entry from the main symbol table, and then look at the scalar component of it: gvsv (pp_gvsv in pp_hot.c) happens to do both these things. 💜

The right hand side, starting at line 5, is similar to what we've just seen: we have the add op (pp_add, also in pp_hot.c) adding together two gvsvs.

Now, what's this about?

     1  LISTOP (0x8179888) leave
     2      OP (0x81798b0) enter
     3      COP (0x8179850) nextstate

enter and leave are scoping ops, and their job is to perform any housekeeping every time you enter and leave a block: lexical variables are tidied up, unreferenced variables are destroyed, and so on. Every program will have those first three lines: leave is a list, and its children are all the statements in the block. Statements are delimited by nextstate, so a block is a collection of nextstate ops, with the ops to be performed for each statement being the children of nextstate. enter is a single op which functions as a marker. 👑

That's how Perl parsed the program, from top to bottom:

                               Program
                                  |
                              Statement
                                  |
                                  =
                                 / \
                                /   \
                               $x   +
                                   / \
                                 $y   $z

However, it's impossible to perform the operations in this order: you have to find the values of $y and $z before you add them together, for instance. So, the other thread that runs through the op tree is the execution order: each op has a field op_next which points to the next op to be run, so following these pointers tells us how perl executes the code. We can traverse the tree in this order using the exec option to B::Terse:

     % perl -MO=Terse,exec -e '$x=$y+$z'
     1  OP (0x8179928) enter
     2  COP (0x81798c8) nextstate
     3  SVOP (0x81796c8) gvsv  GV (0x80fa4d4) *y
     4  SVOP (0x8179798) gvsv  GV (0x80efeb0) *z
     5  BINOP (0x8179878) add [1]
     6  SVOP (0x816dd38) gvsv  GV (0x80fa468) *x
     7  BINOP (0x81798a0) sassign
     8  LISTOP (0x8179900) leave

This probably makes more sense for a human: enter a block, start a statement. Get the values of $y and $z, and add them together. Find $x, and assign one to the other. Then leave. 🦋

The way Perl builds up these op trees in the parsing process can be unravelled by examining toke.c, the lexer, and perly.y, the YACC grammar. Let's look at the code that constructs the tree for $x = $y + $z.

First, we'll look at the Perl_yylex function in the lexer. We want to look for case 'x', where x is the first character of the operator. (Incidentally, when looking for the code that handles a keyword, you'll want to search for KEY_foo where foo is the keyword.) Here is the code that handles assignment (there are quite a few operators beginning with =, so most of it is omitted for brevity):

     1    case '=':
     2        s++;
              ... code that handles == => etc. and pod ...
     3        pl_yylval.ival = 0;
     4        OPERATOR(ASSIGNOP);

We can see on line 4 that our token type is ASSIGNOP (OPERATOR is a macro, defined in toke.c, that returns the token type, among other things). And +:

     1     case '+':
     2         {
     3             const char tmp = *s++;
                  ... code for ++ ...
     4             if (PL_expect == XOPERATOR) {
                      ...
     5                 Aop(OP_ADD);
     6             }
                  ...
     7         }

Line 4 checks what type of token we are expecting. Aop returns a token. If you search for Aop elsewhere in toke.c, you will see that it returns an ADDOP token.

Now that we know the two token types we want to look for in the parser, let's take the piece of perly.y we need to construct the tree for $x = $y + $z

    1 term    :   term ASSIGNOP term
    2                { $$ = newASSIGNOP(OPf_STACKED, $1, $2, $3); }
    3         |   term ADDOP term
    4                { $$ = newBINOP($2, 0, scalar($1), scalar($3)); }

If you're not used to reading BNF grammars, this is how it works: You're fed certain things by the tokeniser, which generally end up in upper case. ADDOP and ASSIGNOP are examples of "terminal symbols", because you can't get any simpler than them.

The grammar, lines one and three of the snippet above, tells you how to build up more complex forms. These complex forms, "non-terminal symbols", are generally placed in lower case. term here is a non-terminal symbol, representing a single expression.

The grammar gives you the following rule: you can make the thing on the left of the colon if you see all the things on the right in sequence. This is called a "reduction", and the aim of parsing is to completely reduce the input. There are several different ways you can perform a reduction, separated by vertical bars: so, term followed by = followed by term makes a term, and term followed by + followed by term can also make a term.

So, if you see two terms with an = or +, between them, you can turn them into a single expression. When you do this, you execute the code in the block on the next line: if you see =, you'll do the code in line 2. If you see +, you'll do the code in line 4. It's this code which contributes to the op tree. 🌈

                   |   term ADDOP term
                   { $$ = newBINOP($2, 0, scalar($1), scalar($3)); }

What this does is creates a new binary op, and feeds it a number of variables. The variables refer to the tokens: $1 is the first token in the input, $2 the second, and so on — think regular expression backreferences. $$ is the op returned from this reduction. So, we call newBINOP to create a new binary operator. The first parameter to newBINOP, a function in op.c, is the op type. It's an addition operator, so we want the type to be ADDOP. We could specify this directly, but it's right there as the second token in the input, so we use $2. The second parameter is the op's flags: 0 means "nothing special". Then the things to add: the left and right hand side of our expression, in scalar context. 💖

The functions that create ops, which have names like newUNOP and newBINOP, call a check function associated with each op type, before returning the op. The check functions can mangle the op as they see fit, and even replace it with an entirely new one. These functions are defined in op.c, and have a Perl_ck_ prefix. You can find out which check function is used for a particular op type by looking in regen/opcodes. Take OP_ADD, for example. (OP_ADD is the token value from the Aop(OP_ADD) in toke.c which the parser passes to newBINOP as its first argument.) Here is the relevant line:

    add             addition (+)            ck_null         IfsT2   S S

The check function in this case is Perl_ck_null, which does nothing. Let's look at a more interesting case:

    readline        <HANDLE>                ck_readline     t%      F?

And here is the function from op.c:

     1 OP *
     2 Perl_ck_readline(pTHX_ OP *o)
     3 {
     4     PERL_ARGS_ASSERT_CK_READLINE;
     5
     6     if (o->op_flags & OPf_KIDS) {
     7          OP *kid = cLISTOPo->op_first;
     8          if (kid->op_type == OP_RV2GV)
     9              kid->op_private |= OPpALLOW_FAKE;
    10     }
    11     else {
    12         OP * const newop
    13             = newUNOP(OP_READLINE, 0, newGVOP(OP_GV, 0,
    14                                               PL_argvgv));
    15         op_free(o);
    16         return newop;
    17     }
    18     return o;
    19 }

One particularly interesting aspect is that if the op has no kids (i.e., readline() or <>) the op is freed and replaced with an entirely new one that references *ARGV (lines 12-16). 💀

🌈 STACKS 🌈

Okaaay so when perl runs something like addop, how does it even hand its results to the next op?? Bestie. The answer is stacks. 💅 Perl keeps a whole collection of stacks to stash the stuff it's currently working on, and we're gonna look at the three most important ones right here.

---

🍓 Argument stack 🍓

Arguments get passed to PP code and handed back from PP code using the argument stack, ST. The standard move is: pop them off the stack, do your lil thing with them, then push the result right back on. That's literally how the cosine operator does it:

NV value;
value = POPn;
value = Perl_cos(value);
XPUSHn(value);

We'll see a trickier example when we get to Perl's macros down below, don't worry. POPn hands you the NV (that's floating point value, slay) of the top SV on the stack — so like, the $x in cos($x). Then we compute the cosine and push the result back as an NV. The X in XPUSHn means the stack should get extended if it needs to be — and it can't need it here, because we literally know there's room for one more item since we just removed one! The XPUSH* macros at the very least guarantee safety. 🫶

Alternatively you can just fiddle with the stack directly: SP gives you the first element in your portion of the stack, and TOP* gives you the top SV/IV/NV/etc. on the stack. So for instance, to do unary negation of an integer:

SETi(-TOPi);

Just set the integer value of the top stack entry to its negation. Iconic, honestly.

Argument stack manipulation in the core is exactly the same as it is in XSUBs — go check out perlxstut, perlxs and perlguts for a longer description of the macros used in stack manipulation. ✨

🦋 Mark stack 🦋

Okay so I said "your portion of the stack" up there for a reason, bestie: PP code doesn't necessarily get the whole stack to itself. If your function calls another function, you only wanna expose the arguments aimed at the called function, and not (necessarily) let it get at your own data. The way we do this is by having a "virtual" bottom-of-stack that's exposed to each function. The mark stack keeps bookmarks to locations in the argument stack that each function can use. 🌸

For instance, when dealing with a tied variable (internally, something with P magic), Perl has to call methods for accesses to the tied variables. But we need to keep the arguments exposed to the method separate from the arguments exposed to the original function — the store or fetch or whatever it may be. Here's roughly how the tied push is implemented; see av_push in av.c:

1  PUSHMARK(SP);
2  EXTEND(SP,2);
3  PUSHs(SvTIED_obj((SV*)av, mg));
4  PUSHs(val);
5  PUTBACK;
6  ENTER;
7  call_method("PUSH", G_SCALAR|G_DISCARD);
8  LEAVE;

Let's examine the whole implementation, for practice bestie, come on:

1  PUSHMARK(SP);

Push the current state of the stack pointer onto the mark stack. This is so that when we've finished adding items to the argument stack, Perl knows how many things we've added recently. 💜

2  EXTEND(SP,2);
3  PUSHs(SvTIED_obj((SV*)av, mg));
4  PUSHs(val);

We're about to add two more items onto the argument stack: when you have a tied array, the PUSH subroutine receives the object and the value to be pushed, and that's exactly what we've got going on here — the tied object, retrieved with SvTIED_obj, and the value, the SV val.

5  PUTBACK;

Next we tell Perl to update the global stack pointer from our internal variable: dSP only gave us a local copy, not a reference to the global. 🙈

6  ENTER;
7  call_method("PUSH", G_SCALAR|G_DISCARD);
8  LEAVE;

ENTER and LEAVE localise a block of code — they make sure all variables are tidied up, everything that's been localised gets its previous value returned, and so on. Think of them as the { and } of a Perl block. 💅

To actually do the magic method call, we have to call a subroutine in Perl space: call_method takes care of that, and it's described in perlcall. We call the PUSH method in scalar context, and we're going to discard its return value. The call_method() function removes the top element of the mark stack, so there's literally nothing for the caller to clean up. ✨

🎀 Save stack 🎀

C doesn't have a concept of local scope, so perl just... provides one. We've seen that ENTER and LEAVE get used as scoping braces; the save stack implements the C equivalent of, for example:

{
    local $foo = 42;
    ...
}

See "Localizing changes" in perlguts for how to use the save stack. 🫶

---

👑 MILLIONS OF MACROS 👑

One thing you'll notice about the Perl source is that it is absolutely full of macros. Some people have called the pervasive use of macros the hardest thing to understand; others find it adds to clarity. No cap, both camps are valid. Let's take an example — a stripped-down version of the code which implements the addition operator:

1  PP(pp_add)
2  {
3      dSP; dATARGET;
4      tryAMAGICbin_MG(add_amg, AMGf_assign|AMGf_numeric);
5      {
6        dPOPTOPnnrl_ul;
7        SETn( left + right );
8        RETURN;
9      }
10  }

Every single line here (apart from the braces, obviously 😭) contains a macro. The first line sets up the function declaration as Perl expects for PP code; line 3 sets up variable declarations for the argument stack and the target, aka the return value of the operation. Line 4 tries to see if the addition operation is overloaded; if it is, the appropriate subroutine gets called. 🔥

Line 6 is another variable declaration — and yes, all variable declarations start with d — which pops from the top of the argument stack two NVs (hence nn) and puts them into the variables right and left, hence the rl. Those are the two operands to the addition operator. Next, we call SETn to set the NV of the return value to the result of adding the two values. That done, we return — the RETURN macro makes sure our return value is properly handled, and we pass the next operator to run back to the main run loop. 💖

Most of these macros are explained in perlapi, and some of the more important ones are explained in perlxs as well. Pay special attention to "Background and MULTIPLICITY" in perlguts for information on the [pad]THX_? macros. 🧚

---

⭐ FURTHER READING ⭐

For more information on the Perl internals, please see the documents listed at "Internals and C Language Interface" in perl. 🌈

---

and that's the whole page bestie, you made it 💅✨ this lil masterpiece is from perl v5.44.0, dated 2026-10-17, and it's the PERLINTERP(1perl) page — go forth and read the internals, you absolute legend, mwah 🫶💖🎀