Marpa:...hanges(3pm)

💖 Marpa::R2::Changes 💖

okay bestie, so this is the manpage Marpa::R2::Changes — it lives in the User Contributed Perl Documentation, and it's basically the group chat recap of everything that's different between Marpa::R2 and Marpa::XS. buckle up 🦋

🌸 NAME 🌸

Marpa::R2::Changes — Differences between Marpa::R2 and Marpa::XS

---

✨ About this document ✨

This doc is the tea on the incompatible differences between Marpa::XS and Marpa::R2. Like, if it's a difference that doesn't break anything? Not in scope, not our problem, not invited. 🙅♀️

It's written for two kinds of people:

• peeps who already know Marpa::XS and are out here writing brand new apps for Marpa::R2, and
• peeps who are migrating their existing Marpa::XS apps and tools over to Marpa::R2.

so either way, if you're holding an old Marpa::XS brain, this is your glow-up guide 💅

---

🔥 Changes 🔥

💅 Additional reserved symbol names

Marpa::XS reserved, for its own internal use, every symbol name that ended with the right square bracket ("]").

and then Marpa::RS was like "hold my boba" and ALSO reserved symbols ending with:

• the right parenthesis (")")
• the right angle bracket (">")
• the right curly bracket ("}")

Any other valid Perl string is still a totally acceptable symbol name. no cap. ✨

😭 The return value of the read() method has changed

The return value of the Marpa::R2 recognizer's read() method is not the same as its Marpa::XS equivalent, and the way that it differs is genuinely a whole thing.

• In Marpa::XS it returned the number of distinct terminals (by symbol ID) allowed in the next read().
• In Marpa::R2 it returns the number of recognizer events that occurred during the read.

Examples of recognizer events are exhaustion, the Earley sets exceeding a designated "warning" level, and other circumstances settable by the user. For more detail, see the documentation of recognizer's read method. 📖

🚫 Rule LHS's are no longer a source of action names

In Marpa::XS, if there was no explicit action name for a rule, Marpa would try to find a closure that had the same name as the rule's LHS. and the use of rule LHS's as action names had a potential for unpleasant surprises 👀

A surprise could occur if the rule's LHS coincided with a function name without the programmer realizing or intending it. This kind of 'action at a distance' bug can be very hard to detect and trace. like, truly the worst kind of bug. the sneaky kind. 💀

It was originally thought that implicitly using the LHS as the name of an action would be convenient enough to outweigh the dangers. But in fact, this feature wound up being little used. And accidental resolution via a rule LHS was a danger for all users, whether they used the feature or not. For these reasons, as well as potential optimization and efficiency considerations, Marpa::R2 no longer does implicit action resolution using a rule LHS. periodt. 🙅♀️

🎲 Different rules with the same rank now appear in arbitrary order

In ranking parse trees, if two rule instances are for different rules but have the same rule rank, they will now appear in arbitrary order. This is probably the behavior that programmers have always expected. (we love a "you were right all along" moment 🫶)

In Marpa::XS, when the null_ranking named argument of rules was in use for one of the rules, specific guarantees were made for the order in some of the cases. The intent was to be orthogonal with the guarantees made for the ranking of null variants within the same rule. These additional guarantees proved useless in practice, cumbersome to implement, and, when documented, opaque and unintuitive. In Marpa::R2 they have been dropped. gone. deleted. bye. 👋

🧚 Null actions now come from the rules

In Marpa::XS null actions were specified by symbol. This created a dual semantics — one for non-nulled rules, and another for nulled rules. The conventions and behaviors of the two semantics were quite dissimilar. The rules for their coordination were complicated, and it was possible for a programmer expecting one semantics, to be surprised by a result from the other. 😵💫

In Marpa::R2 the semantics of nulled rules is the same as that of non-nulled rules, and the semantics of nulled symbols comes from the semantics of the nulled rules. This requires rule evaluation closures to be aware they might be called for nulled rules. But it greatly simplifies the semantics conceptually. For more detail, see Marpa::R2::Deprecated::NAIF::Semantics::Null.

⭐ Actions can now be constants

If an action name resolves to a constant, that constant is the action. The effect is the same as if the action name resolved to a function that returned that constant, except that it is more efficient. slay, less work 💅

Perl cannot reliably distinguish between non-existent symbols and symbols whose value is undef, so constants whose value is undef are not allowed. The ::undef reserved action name can be used instead.

🔒 Actions names beginning with "::" are reserved

Action names which start with "::" are reserved. "::undef" is a safe way of specify a constant whose value is undef. Use of a reserved name which has not yet been defined causes an exception to be thrown. so, like, don't. 🔥

🗑️ The default_null_value named argument for grammars has been removed

Symbols no longer have null values, so the default_null_value named argument of grammars has been removed.

💀 The null_value symbol property has been removed

Symbols no longer have null values. Use of the null value symbol property now causes an exception.

🎀 The token value argument of read() has changed

The Marpa::R2 recognizer's read() method differs from its Marpa::XS equivalent.

• In Marpa::R2, if read()'s token value argument is omitted, then the value of the token will be a Perl undef.
• If read()'s token value is given explicitly, then that explicit value will be the value of the token.

In particular, an explicit undef token value argument will behave differently from an omitted token value argument. 👀 For details, see the documentation of recognizer's read method.

🌈 The token value argument of alternative() has changed

The Marpa::R2 recognizer's alternative() method differs from its Marpa::XS equivalent. Its token value argument must now be a reference to the token value, not the token value itself, as in Marpa::XS. (yes this is a gotcha, yes you will hit it, yes it's fine 🫠)

• If alternative's token value argument is omitted or a Perl undef, then the value of the token will be a Perl undef.
• If alternative's token value argument is reference to undef, then the value of the token is a Perl undef.

For details, see the documentation of the alternative method.

🚫 Marpa::R2::Recognizer::value() does not accept named arguments

In the Marpa::XS recognizer, the new(), set() and value() methods all accepted named arguments. As of Marpa::R2, the value() method will no longer do so.

Allowing named arguments for the value() was a holdover from a previous interface, which also seemed like it might be a convenience. But, since it was even more important that the value() method be convenient as the termination test controlling a loop over the parse results, a lot of special logic was added to deal with arguments which only made sense before the first pass of the loop, etc., etc. and bestie, the etc., etc. was doing a LOT of heavy lifting there 😭

Eliminating named arguments from the value() method eliminates a variety of special cases and, as a result, the documentation of the value() method is now simpler, shorter and clearer. Anything that could be done by providing named arguments to the value() method can be done more using the recognizer's set() method, and the code will be clearer for it. ✨

🪄 Marpa's grammar rewriting is now invisible

Internally, Marpa rewrites its grammars. In Marpa::XS, most details of these rewrites were invisible, but not all. In Marpa::R2, all internal rules and symbols are now completely invisible to the user, even in the tools for debugging grammars. iconic. truly nothing to see here 💅

🌸 By default, the non-LHS symbols are the terminals

Traditionally, a symbol has been a terminal if it is not on the LHS of any rule, and vice versa. This is now the default in Marpa::R2, replacing the more complicated, and less intuitive, scheme that was in Marpa::XS.

Marpa::R2 still allows the user to use any non-nulling symbol as a terminal, including those symbols that appear on the LHS of a rule, but this is now an option, and never the default. For more, see "Terminal symbols" in Marpa::R2::Deprecated::NAIF::Grammar.

❌ The lhs_terminals grammar named argument has been eliminated

The lhs_terminals named argument of grammar objects implemented what is now the default behavior. Since it no longer performs a function, its use is now a fatal error. 💀

🚫 Nulling symbols cannot be terminals

In Marpa::XS, it was possible for a symbol to be both nulling and a terminal. In practice that meant that the symbol was nulling, but that, on input, that property could be overriden, and a specific instance of the nulling symbol could be made non-nulling. This behavior was worse than useless and non-intuitive — it was dangerous and logically inconsistent. like genuinely unserious behavior 🫠

Marpa::R2 will not allow a nulling symbol to be used as a terminal.

To the extent that the Marpa::XS behavior made sense, it can be duplicated by creating a symbol which is the LHS of two rules, one empty, and the other rule with a RHS consisting of exactly one terminal symbol.

👑 A sequence must have a unique LHS

The LHS of a sequence rule may not be on the LHS of any other rule, whether another sequence rule, or a BNF rule.

This is not as severe a restriction as it might sound — while sequences cannot share the same LHS with other rules directly, they can do so indirectly. For details, see "Duplicate rules" in Marpa::R2::Deprecated::NAIF::Grammar.

In Marpa::XS, the definition of when a sequence was a duplicate was more liberal, but it was also complicated and non-intuitive. The new definition is simpler and more intuitive, and its greater restrictiveness is easy to work around. 🫶

🔒 The terminal status of a symbol is locked once set

Once a symbol is marked as a terminal or a non-terminal, its terminal status cannot be changed. no take-backs 💅

We doubt this will affect any actual applications. It would only affect an application that changes symbols from their default status to non-terminal, and then only if they attempted to mark the same symbol as a terminal at another point. Few Marpa::R2 applications change symbols from their default terminal status, and none to my knowledge mark symbols as non-terminals.

♾️ Evaluation of infinite loops has been changed

Infinite loops (cycles) are still, by default, fatal errors. 🚨

For those considering programming with them, and evaluating parses from grammars with cycles, the semantics of cycles is now more closely specified. For details of the new semantics, see Marpa::R2::Deprecated::NAIF::Semantics::Infinite.

🔢 The range of values allowed for ranks has been clarified

Symbols and rules have numeric ranks. Previously, no mention was made of range of values allowed. This is implemented-defined, except that the magnitudes of the ends of the range will always be at least the 28th power of 2, less 1. That is, numbers in the range between -134,217,727 and 134,217,727 will always be allowed as ranks. math girlies rise 🧮✨

---

💖 Copyright and License 💖

Okay bestie, legal era, but make it cute. Here's the tea:

Copyright 2022 Jeffrey Kegler 🎀

This file is part of Marpa::R2. And no cap, Marpa::R2 is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation — either version 3 of the License, or (at your option) any later version. Slay, options, we love choices. ✨

Marpa::R2 is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. 😭😭 Not me getting emotional over a warranty disclaimer. See the GNU Lesser General Public License for more details.

💀 fr fr, read the license if you're gonna rely on this in prod — the disclaimer is doing main character energy levels of heavy lifting here.

You should have received a copy of the GNU Lesser General Public License along with Marpa::R2. If not, see http://www.gnu.org/licenses/. 🦋

---

And that's the whole page, bestie, we made it to the end 🫶 this was perl v5.42.3 energy, dated 2026-08-20, from the Marpa:...hanges(3pm) page itself. Take care of each other, read your licenses, and stay iconic. 💅🌸🌈