btparse::doc::bt_post_processing(3)

btparse::doc::bt_post_processing 💖

okay bestie, welcome to the btparse manual — this is bt_post_processing(3), and it's giving ✨*the lore of what happens to your data after it gets parsed*✨. this one's long so buckle up, we're doing it in parts 🎀

---

🌸 NAME 🌸

bt_post_processing — post-processing of BibTeX strings, values, and entries 💅

---

✨ SYNOPSIS ✨

void bt_postprocess_string (char * s,
                            btshort options)

char * bt_postprocess_value (AST *   value,
                             btshort  options,
                             boolean replace);

char * bt_postprocess_field (AST *   field,
                             btshort  options,
                             boolean replace);

void bt_postprocess_entry (AST *  entry,
                           btshort options);

---

💖 DESCRIPTION 💖

so here's the tea, bestie 🫖. when btparse parses a BibTeX entry, it does not hand you nice clean data right away. nope. it stashes everything in an abstract syntax tree (an AST, for short) that mirrors the parsed data exactly. like, mirror-selfie exact. 🙈

for example, this entry:

@Article{Jones:1997a,
  AuThOr = "Bob   Jones" # and # "Jim Smith ",
  TITLE = "Feeding Habits of
                   the Common Cockroach",
  JoUrNaL = j_ent,
  YEAR = 1997
}

would parse to an AST that could be represented like this:

(entry,"Article")
  (key,"Jones:1997a")
  (field,"AuThOr")
    (string,"Bob   Jones")
    (macro,"and")
    (string,"Jim Smith ")
  (field,"TITLE")
    (string,"Feeding Habits of               the Common Cockroach")
  (field,"JoUrNaL")
    (macro,"j_ent")
  (field,"YEAR")
    (number,"1997")

the advantage of this form? literally every important thing in the entry is right there, ready for you to grab by traversing the tree with the functions described in bt_traversal. iconic. 👑

the obvious problem? it's a little too raw to be immediately useful, no cap. entry types and field names are inconsistently capitalized, strings are full of unwanted whitespace, field values aren't reduced to single strings, and so forth. it's giving "unprocessed chaos" 😭

---

all of these problems get addressed by btparse's post-processing functions, which is what this whole page is about. ✨

normally, you won't have to call these functions at all — the library does the Right Thing for you after parsing each entry, and you can customize what exactly the Right Thing is for your application. (for instance, you can tell it to expand macros, but not to concatenate substrings together.) 🫶

however, it's conceivable that you might wish to move the post-processing into your own code and out of the library's control. more likely, you could have strings that come from something other than BibTeX files that you'd like to have treated as BibTeX strings — and for that situation, the post-processing functions are essential. 💅

finally, you might just be curious about what exactly happens to your data after it's parsed. if so, you've come to the right place for excruciatingly detailed explanations. 🦋

---

stay tuned bestie, more parts incoming 💜

✨ FUNCTIONS ✨

btparse gives you four doors into its post-processing code, bestie. But fr fr, only the first and the last — the ones that process individual strings and whole entries — are the ones you'll actually use on the daily.

---

🚪 Post-processing entry points

Okay so to get why there are four entry points, let's look at that sample AST again. First up: the whole entry is that (entry,"Article") node — she has the entry key AND all her field/value pairs as children. Entry nodes come back to you from bt_parse_entry() and bt_parse_entry_s() (peep bt_input), and also from bt_next_entry() (which walks a list of entries returned by bt_parse_file() — see bt_traversal). Whole entries get post-processed with bt_postprocess_entry(). 💅

You might also need to post-process just one field, or just the value hanging off it. (The difference? Processing the field can change the field name — say, to lowercase — on top of the field value.) That (field,"AuThOr") node up there is a field sub-AST, and (string,"Bob Jones") is the first node in the list of simple values representing that field's value. (Remember: a field value is, in general, a list of simple values.) Field nodes come from bt_next_field(), value nodes from bt_next_value(). The former goes to bt_postprocess_field(), the latter to bt_postprocess_value(). 🦋

And finally, strings can wander into your program from like, a million places that are not a btparse AST. So that's why bt_postprocess_string() exists — for post-processing arbitrary strings. Periodt.

---

🎛️ Post-processing options

Every single post-processing routine takes an options parameter, and you use it to fine-tune the vibes. (This is literally the same energy as the per-metatype string-processing options you can set before parsing entries — see bt_set_stringopts() in bt_input.) Just like everywhere else in the library, options is a bitmap you build by or'ing together predefined constants. Those constants and what they do live in "String processing option macros" in btparse.

---

💖 bt_postprocess_string()

void bt_postprocess_string (char * s,
                            btshort options)

Post-processes one individual string, s, and modifies it in place. The only post-processing option that even makes sense on a lone string is whether to collapse whitespace per the BibTeX rules — so if options & BTO_COLLAPSE is false, this function does nothing. (Okay technically it still makes a complete pass over the string anyways. That's for future expansion, iykyk.) 👀

The exact collapse rules are simple, bestie: non-space whitespace characters (mostly tabs and newlines) become spaces, any run of more than one space inside gets squished down to a single space, and any leading or trailing spaces get yeeted. (Making sure all whitespace is spaces is actually handled by btparse's lexical scanner, so strings inside btparse ASTs will never have whitespace that isn't a space. Likewise, any string you pass to bt_postprocess_string() shouldn't contain non-space whitespace characters.) ✨

---

💖 bt_postprocess_value()

char * bt_postprocess_value (AST *   value,
                             btshort  options,
                             boolean replace);

Post-processes a single field value — that's the head of a list of simple values as returned by bt_next_value(). All the relevant string-processing options get to play here: converting numbers to strings (BTO_CONVERT), macro expansion (BTO_EXPAND), whitespace collapsing (BTO_COLLAPSE), and string pasting (BTO_PASTE). And since pasting substrings together before expanding macros and converting numbers would be straight-up nonsense, trying to do that is a fatal error. 😭

If replace is true, then the list headed by value gets replaced by a list representing the processed value. Meaning: if string pasting is on (options & BTO_PASTE is true), that list collapses down to a single node holding the one string that results from pasting all the substrings together. If pasting is off, each node in the list stays intact, but its text gets swapped for processed text. 🫶

If replace is false, then a brand-new string gets built on the fly and returned by the function. Heads up: if pasting isn't on in this case, you'll only get the last string in the list. (Honestly, post-processing a value without pasting doesn't make a ton of sense unless you're replacing it with the new value anyway, so.)

Returns the string that came out of processing the whole value — which only makes sense if pasting was on, or there was only one value in the list. If a multiple-value list got processed without pasting, you get the last string in the list back (after processing).

So picture what could happen to the value of the author field in the example above — the concatenation of a string, a macro, and another string. Say the macro and expands to and, and the variable value points to the sub-AST for this value. The original sub-AST for this value is

(string,"Bob   Jones")
(macro,"and")
(string,"Jim Smith ")

To fully process this value in-place, you'd call

bt_postprocess_value (value, BTO_FULL, TRUE);

(BTO_FULL is just every possible string-processing option mashed together: BTO_CONVERT|BTO_EXPAND|BTO_PASTE|BTO_COLLAPSE.) That would convert the value into a single-element list,

(string,"Bob Jones and Jim Smith")

and return the fully-processed string Bob Jones and Jim Smith. Note how the and macro got expanded, interpolated between the two literal strings, everything pasted together, and finally whitespace collapsed. (Collapsing whitespace before concatenating the strings would be a bad idea, fr.) 🌸

Now say you'd rather keep the list nature of the value alive, while still expanding macros and converting any numbers to strings. (That conversion is trivial: it just changes the node type from BTAST_NUMBER to BTAST_STRING. Number values are always stored as a string of digits, exactly as they appear in the file.) You'd do that with the call

bt_postprocess_value
   (value, BTO_CONVERT|BTO_EXPAND|BTO_COLLAPSE,TRUE);

which would change the list to

(string,"Bob Jones")
(string,"and")
(string,"Jim Smith")

Notice whitespace gets collapsed here before any concatenation can happen — which is probably a bad idea. But you can do it if you want to, no one's stopping you. (And if you get any ideas about cooking up your own custom value post-processing scheme by doing it in little steps like this? Go look at the source of bt_postprocess_value(). It should talk you out of that venture real quick. 💀)

---

💖 bt_postprocess_field()

char * bt_postprocess_field (AST *   field,
                             btshort  options,
                             boolean replace);

This one is barely more than a front-end to bt_postprocess_value(). The only diff is that you hand it a field AST node (like the (field,"AuThOr") from the example), and it transforms the field name in addition to its value. Specifically, the field name gets forced to lowercase — and that behaviour is (currently) not optional. 🙈

Returns the string that bt_postprocess_value() returned.

---

💖 bt_postprocess_entry()

void bt_postprocess_entry (AST *  entry,
                           btshort options);

Post-processes every value in an entry. If entry points to the AST for a "regular" or "macro definition" entry, then the values are exactly what you'd expect: everything on the right-hand side of a field or macro "assignment." But you can also post-process comment and preamble entries, bestie. Comment entries are basically one big string, so only whitespace collapsing makes sense on them. Preambles may have multiple strings pasted together, so all the string-processing options apply to them. (And nothing's stopping you from using macros in a preamble either.) 👑

---

<div align="center">

that's the whole page, bestie — you ate it 🎀

this one's from btparse, version 0.91, dated 2025-01-3 💜

now go post-process something iconic ✨💅🦋

</div>