AREGEX(3am)

πŸ“¦ version 2022-10, in nixos 26.05, nixpkgs unstable

πŸ’… AREGEX β€” ok so it's fuzzy regex matching (3am, GNU Awk Extension Modules)

Hiii bestie, welcome to aregex, the awk extension that does approximate (fuzzy) string matching with regular expressions. Yes, fuzzy. Like your memory of what you said at 2am. It's giving "close enough counts" and honestly? Slay. ✨

This page lives in section 3am of the manual, and it's a GNU Awk Extension Module situation β€” keep that in mind, because everything here is awk-flavored.

πŸ¦‹ SYNOPSIS

@load "aregex"

success = amatch(str, regex [, cost|costs [, submatches]])

That's the whole vibe. Load it, call amatch(), get a number back. Periodt.

✨ DESCRIPTION

Okay so here's the tea: the TRE library (link's down in SEE ALSO, don't worry) provides approximate matching regex capabilities. The way it works is that a match between two strings that differ in some number of characters will be found when the cost of character insertions, deletions and substitutions does not exceed some specified maximum cost.

Like, for example β€” say your string is "abcdef":

"abcdef"
"abcxdef"  # one insertion
"abdef"    # one deletion
"abxdef"   # one substitution

See? All those are near misses on "abcdef", and the fuzzy matcher is like "yeah bestie I see you, close enough." 🫢

The cost of the match (that's the Levenshtein distance between strings, iykyk) can be reported back to you. This Gawk extension is basically an interface to the tre_regaexec() function in the TRE library, and it lets you set every possible parameter for that function and get back all possible information about a match. Maximum flexibility, no cap. πŸ”₯

πŸ’– Function summary

There's exactly one function, amatch(), and it's modeled on the Gawk match() function:

amatch( str , regex [, cost|costs [, submatches ] ] )

It takes two mandatory string arguments and two optional arguments. Here's the breakdown:

  • regex is an extended regular expression (or just a plain string, no judgment) to be matched against the string str.
⚠️ Note this, bestie, because it trips people up: the regular expression regex is bounded by double-quotes, not by the usual Gawk slashes. So it's "pattern" not /pattern/. Not me warning you at the top so you don't text me at midnight confused. πŸ’€

πŸŽ€ Setting approximate match costs

With only two arguments, the default maximum cost for the approximate match is set to 5 (and the other costs get set as listed below). But you can also set the maximum cost yourself via that optional third argument. It's either:

  • an integer β€” cost β€” a single value, or
  • a member of a one-dimensional array β€” costs β€” indexed by "max_cost".

Setting the maximum cost to 0 forces an exact regular expression match, exactly like Gawk's match(). So like, if you want the old boring behavior back, that's your dagger. πŸ—‘οΈ

And the other members of the costs array β€” with appropriate index values β€” will set the parameters of the regaparams_t structure used by tre_regaexec(). Here's the full menu, no substitutions on this table:

Array index   Parameter    Def val  Meaning
============  ===========  =======  =====================
"cost_ins"    .cost_ins       1     Cost of one insertion
"cost_del"    .cost_del       1     Cost of one deletion
"cost_subst"  .cost_subst     1     Cost of one substitution
"max_cost"    .max_cost       5     Max. cost
"max_del"     .max_del        5     Max. number of deletions
"max_ins"     .max_ins        5     Max. number of insertions
"max_subst"   .max_subst      5     Max. number of substitutions
"max_err"     .max_err        5     Max. number of ins+del+subst
🧚 Little note: if you pass in the costs array but it contains none of the indexes above, the default values are used. It's not gonna yell at you, it's just gonna vibe with defaults.

🌈 Return value

amatch() returns:

  • 1 on a successful match βœ…
  • 0 on a failure to match ❌
  • -1 if regex is invalid β€” and in that case, TRE's error message lands in ERRNO 🚨

πŸ‘‘ Obtaining match summary data

If you pass a third array argument to amatch(), and the match was successful, you get info about the match back β€” the array gets cleared and filled with these indexes:

Array index   Meaning
============  ==============================================
"cost"        Total cost of the match (Levenshtein distance)
"num_ins"     Total number of insertions
"num_del"     Total number of deletions
"num_subst"   Total number of substitutions

So cute, so informative. It literally tells you why it was a near-miss. πŸ’…

πŸ“ Obtaining parenthetical submatches

Now if you provide an array (or an empty Gawk variable symbol, that works too) as the fourth argument, and the match is successful, the array gets cleared and filled with submatches corresponding to the parenthetical sub-expression in regex β€” with indexes 1...n, up to a maximum of 20. The array member indexed by 0 will be the portion of str matched by the whole of regex. Classic. πŸŽ€

😭 A note on bytes and characters (read this one, seriously)

While the amatch() function is roughly equivalent to the Gawk match() function, submatches are not returned as in match() β€” like, not via [i,"start"] position and [i,"length"] (peep the Gawk man page for that style). Instead, only the literal substring for each parenthetical match is given. That's the whole deal.

Why? Because Gawk is multibyte aware and match() works in terms of characters, not bytes β€” but TRE is byte-based, not character-based. And using the wchar_t versions of TRE functions cannot help if the input is a mix of single and multi-byte characters. So sorry bestie, that door is closed. πŸšͺ

If you need positions and lengths of the substrings, you gotta run a simple Gawk routine on the output array (submatches) yourself. Like this one:

print "i", "substring", "posn", "length"
p = 1
for (i = 1; i < length(submatches); i++) {
  idx = index(substr(str, p), submatches[i])
  len = length(out[i])
  print i, submatches[i], idx+p-1, len
  p = p + idx + len
}

Copy it, keep it, love it. πŸ’–

πŸ”₯ EXAMPLE

@load "aregex"
BEGIN {
  str = "abcdα»…fbc"
  regex = "^a(bc)d(α»…)(f)$"
  costs["max_cost"] = 6
  costs["cost_ins"] = 2
  if (amatch(str, regex, costs, submatches)>0)
    print costs["cost"], submatches[1]
}

Look at that little multibyte α»… character living its best life in there. Iconic. ⭐

🌸 SEE ALSO

🫢 AUTHORS

  • Cam Webb <cw@camwebb.info>
  • @laurikari for the TRE library
  • the gawkextlib authors

Shoutout to the whole squad, fr fr. πŸ‘‘

πŸ’œ COPYING PERMISSIONS

Copyright Β© 2018, the Free Software Foundation, Inc.

Copyright Β© 2018, Campbell O. Webb

Permission is granted to make and distribute verbatim copies of this manual page provided the copyright notice and this permission notice are preserved on all copies.

Permission is granted to copy and distribute modified versions of this manual page under the conditions for verbatim copying, provided that the entire resulting derived work is distributed under the terms of a permission notice identical to this one.

Permission is granted to copy and distribute translations of this manual page into another language, under the above conditions for modified versions, except that this permission notice may be stated in a trans- la- tion approved by the Foundation.

Okay that's the whole page bestie, go fuzzy-match something and think of me. πŸ’…βœ¨

This one's from the Free Software Foundation, dated Nov 24 2018, and it's AREGEX(3am) β€” mwah, ily, bye! πŸŽ€πŸ¦‹

πŸ“¦ where this page comes from