GROK(1)

πŸ“¦ version 1.20110708.1, in debian bookworm, ubuntu jammy

GROK(1) ✨ the grok manual, bestie edition πŸ’–


🌸 NAME 🌸

grok β€” parse logs, handle events, and make your unstructured text structured.

okay so literally she takes your messy chaotic log soup and turns it into structured data. it's giving glow-up. πŸ’…


πŸ¦‹ SYNOPSIS πŸ¦‹

grok [-d] -f configfile

πŸ‘‘ DESCRIPTION πŸ‘‘

grok is software that lets you parse logs and other files, no big deal. With grok, you can turn unstructured log and event data into structured data. chef's kiss. 😌

The grok program is a great tool for parsing log data and program output. You can match any number of complex patterns on any number of inputs (processes and files) and have custom reactions. like, any number. go feral. πŸ”₯


πŸ’œ OPTIONS πŸ’œ

  • -d or --daemon β€” daemonize after parsing the config file. Implemented with daemon(3). The default is to stay in foreground. (so by default she's right there with you, watching. cute.)
  • -f configfile β€” Specify a grok config file to use. this is not optional, bestie, you gotta tell her where the config lives. πŸŽ€

🌈 CONFIGURATION 🌈

You can call the config file literally anything you want. Name it soup.conf, name it bestie, she doesn't care. 🫢 A full example config follows below, with documentation on options and defaults:

# --- Begin sample grok config
# This is a comment. :)
#
# enable or disable debugging. Debug is set false by default.
# the 'debug' setting is valid at every level.
# debug values are copied down-scope unless overridden.
debug: true

# you can define multiple program blocks in a config file.
# a program is just a collection of inputs (files, execs) and
# matches (patterns and reactions),
program {
  debug: false

  # file with no block. settings block is optional
  file "/var/log/messages"

  # file with a block
  file "/var/log/secure" {
    # follow means to follow a file like 'tail -F' but starts
    # reading at the beginning of the file.  A file is followed
    # through truncation, log rotation, and append.
    follow: true
  }

  # execute a command, settings block is optional
  exec "netstat -rn"

  # exec with a block
  exec "ping -c 1 www.google.com" {
    # automatically rerun the exec if it exits, as soon as it exits.
    # default is false
    restart-on-exit: false

    # minimum amount of time from one start to the next start, if we
    # are restarting. Default is no minimum
    minimum-restart-interval: 5

    # run every N seconds, but only if the process has exited.
    # default is not to rerun at all.
    run-interval: 60

    # default is to read process output only from stdout.
    # set this to true to also read from stderr.
    read-stderr: false
  }

  # You can have multiple match {} blocks in your config.
  # They are applied, in order, against every line of input that
  # comes from your exec and file instances in this program block.
  match {
    # match a pattern. This can be any regexp and can include %{foo}
    # grok patterns
    pattern: "some pattern to match"

    # You can have multiple patterns here, any are valid for matching.
    pattern: "another pattern to match"

    # the default reaction is "%{@LINE}" which is the full line
    # matched.  the reaction can be a special value of 'none' which
    # means no reaction occurs, or it can be any string. The
    # reaction is emitted to the shell if it is not none.
    reaction: "%{@LINE}"

    # the default shell is 'stdout' which means reactions are
    # printed directly to standard output. Setting the shell to a
    # command string will run that command and pipe reaction data to
    # it.
    #shell: stdout
    shell: "/bin/sh"

    # flush after every write to the shell.
    # The default is not to flush.
    flush: true

    # break-if-match means do not attempt any further matches on
    # this line.  the default is false.
    break-if-match: true
  }
}
# -- End config

πŸŽ€ PATTERN FILES πŸŽ€

Pattern files contain lists of names and patterns for loading into grok. ✨

Patterns are newline-delimited and have this syntax:

patternname expression

Any whitespace between the patternname and expression are ignored. slay, whitespace is not the enemy here.

  • patternname β€” This is the name of your pattern which, when loaded, can be referenced in patterns as %{patternname} πŸ’«
  • expression β€” The expression here is, verbatim, available as a regular expression. You do not need to worry about how to escape things. not a single escape character. she handles it. πŸ™ˆ

πŸ“ PATTERN EXAMPLES πŸ“

DIGITS \d+
HELLOWORLD \bhello world\b

πŸ’€ REGULAR EXPRESSIONS πŸ’€

The expression engine underneath grok is PCRE. Any syntax in PCRE is valid in grok. that's it, that's the whole vibe β€” if PCRE can do it, she can do it. periodt. πŸ‘‘


⭐ REACTIONS ⭐

Reactions can reference named patterns from the match. You can also access a few other special values, including:

  • %{@LINE} β€” The line matched.
  • %{@MATCH} β€” The substring matched
  • %{@START} β€” The starting position of the match from the beginning of the string.
  • %{@END} β€” The ending position of the match.
  • %{@LENGTH} β€” The length of the match
  • %{@JSON} β€” The full set of patterns captured, encoded as a json dictionary as a structure of { pattern: [ array of captures ] }. We use an array because you can use the same named pattern multiple times in a match. (iconic behavior, honestly)
  • %{@JSON_COMPLEX} β€” Similar to the above, but includes start and end position for every named pattern. That structure is:
{ "grok": [
   { "@LINE": { "start": ..., "end": ..., "value": ... } },
   { "@MATCH": { "start": ..., "end": ..., "value": ... } },
   { "patternname": { "start": startpos, "end": endpos, "value": "string" } },
   { "patternname2": { "start": startpos, "end": endpos, "value": "string" } },
   ...
] }

🌸 REACTION FILTERS 🌸

Reaction filters allow you to mutate the captured data. πŸ”₯ The following filters are available:

An example of using a filter in a reaction is like this:

reaction: "echo Matched: %{@MATCH|shellescape}"
  • shellescape β€” Escapes all characters necessary to make the string safe in non-quoted a shell argument
  • shelldqescape β€” Escapes characters necessary to be safe within doublequotes in a shell.
  • jsonencode β€” Makes the string safe to represent in a json string (escapes according to json.org recommendations)

🫢 SEE ALSO 🫢

pcre(3), pcresyntax(3),

Sample grok configs are available in in the grok samples/ directory. (yes, "in in". we said what we said. go look, the configs are there and they're good.) πŸ’–

Project site: <http://semicomplete.googlecode.com/wiki/Grok>

Google Code: <http://semicomplete.googlecode.com/>

Issue/Bug Tracker: <http://code.google.com/p/semicomplete/issues/list>


πŸ’Œ CONTACT πŸ’Œ

Please send questions to grok-users@googlegroups.com. File bugs and feature requests at the following URL:

Issue/Bug Tracker: <http://code.google.com/p/semicomplete/issues/list>

like, if something's broken, tell her. she wants to know. 😭


🧚 HISTORY 🧚

grok was originally in perl, then rewritten in C++ and Xpressive (regex), then rewritten in C and PCRE. three eras, three aesthetics, all iconic. a true evolution arc. πŸ¦‹


🌟 AUTHOR 🌟

grok was written by Jordan Sissel.


no cap, that's the whole page. go parse something. πŸ’…

okay bye bestie, this manual is dated 2009-12-25 and lives in the GROK(1) section of the manual pages β€” shoutout to her for staying this useful for this long. πŸ‘‘βœ¨

πŸ“¦ where this page comes from