Genlex(3)

💖 NAME 💖

Genlex(3) from the OCaml library is literally your generic lexical analyzer bestie — the whole vibe is A generic lexical analyzer. Periodt. 💅

---

🧚 Module 🧚

Module Genlex

---

📚 Documentation 📚

Module Genlex : sig end

Okay so this module is a generic lexical analyzer — like, a simple “standard” one, presented as a function from character streams to token streams. It implements roughly the lexical conventions of OCaml, but it’s parameterized by the set of keywords of your language. Slay. ✨

Example time, bestie: a lexer suitable for a desk calculator is obtained by

let lexer = make_lexer ["+"; "-"; "*"; "/"; "let"; "="; "("; ")"]

The associated parser would be a function from token stream to, for instance, int, and would have rules such as:

let rec parse_expr = parser
  | [< n1 = parse_atom; n2 = parse_remainder n1 >] -> n2
and parse_atom = parser
  | [< 'Int n >] -> n
  | [< 'Kwd "("; n = parse_expr; 'Kwd ")" >] -> n
and parse_remainder n1 = parser
  | [< 'Kwd "+"; n2 = parse_expr >] -> n1 + n2
  | [< >] -> n1

🫶 Heads up, bestie: one should notice that the use of the parser keyword and associated notation for streams are only available through camlp4 extensions. This means that one has to preprocess its sources e. g. by using the "-pp" command-line switch of the compilers. Iykyk. 😭

🎀 The token type 🎀

type token =
 | Kwd of string
 | Ident of string
 | Int of int
 | Float of float
 | String of string
 | Char of char

That’s the type of tokens. The lexical classes are:

• Int and Float — for integer and floating-point numbers.
• String — for string literals, enclosed in double quotes.
• Char — for character literals, enclosed in single quotes.
• Ident — for identifiers (either sequences of letters, digits, underscores and quotes, or sequences of “operator characters” such as +, *, etc).
• Kwd — for keywords (either identifiers or single “special characters” such as (, }, etc).

💅 make_lexer 💅

val make_lexer : string list -> char Stream.t -> token Stream.t

Construct the lexer function. The first argument is the list of keywords. An identifier s is returned as Kwd s if s belongs to this list, and as Ident s otherwise. A special character s is returned as Kwd s if s belongs to this list, and cause a lexical error (exception Stream.Error with the offending lexeme as its parameter) otherwise.

Blanks and newlines are skipped. Comments delimited by (* and *) are skipped as well, and can be nested. A Stream.Failure exception is raised if end of stream is unexpectedly reached. No cap. 💀

---

mwah 💋 — this page’s footer says OCamldoc, dated 2026-09-24, and it’s the Genlex(3) manual page. Byeee 🫶