htmlparse(n)

okay so this is htmlparse(n), the HTML Parser manual page, and bestie we are starting at the very top because that's how manpages work. no cap, fr fr. 💖

✨ NAME ✨

htmlparse — Procedures to parse HTML strings. that's the whole name, that's the whole vibe. it's giving lowkey iconic, highkey essential. vibe check: passed. it's literally the main character of HTML-string parsing, and if you know you know. periodt 💅

---

🌸 SYNOPSIS 🌸

first things first, you gotta require the packages. this is the exact loadout, and it ate, honestly:

package require Tcl 8.5 9
package require struct::stack 1.3
package require cmdline 1.1
package require htmlparse ?1.2.3?

then the actual procedures? they show up like this, and yes every character matters, so copy-paste responsibly:

::htmlparse::parse  ?-cmd  cmd?  ?-vroot  tag?  ?-split n? ?-incvar var? ?-queue q? html

::htmlparse::debugCallback ?clientdata? tag slash param textBehindTheTag

::htmlparse::mapEscapes html

::htmlparse::2tree html tree

::htmlparse::removeVisualFluff tree

::htmlparse::removeFormDefs tree

✨ DESCRIPTION ✨

okay so bestie, the htmlparse package is literally a whole toolkit 💖 it gives libraries and applications the power to take an HTML string and parse it into whatever representation they want. like, your choice, your vibe, your rules. slay.

the following commands are available, and yes we're going through every single one because they all eat:

🌸 ::htmlparse::parse ?-cmd cmd? ?-vroot tag? ?-split n? ?-incvar var? ?-queue q? html 🌸

this is the basic parser for HTML. it takes an HTML string, parses it, and invokes a command prefix for every tag it encounters. 🫶

and here's the thing — the HTML does not have to be valid for this parser to function. not even a little. it is the responsibility of the command you invoke for every tag to check that. another job on that command's plate: handling tag attributes and character entities (escaped characters). the parser hands over the un-interpreted tag attributes to the invoked command to help with the first bit, and the package at large gives you a helper command, ::htmlparse::mapEscapes, to help with the second bit. teamwork fr fr. 💅

also, the parser does ignore leading DOCTYPE declarations and all valid HTML comments it encounters. not me being unbothered by your comments 😌

all information beyond the HTML string itself is specified via options, which we'll get into below.

to help you understand the options, here's some more background about the parser:

it's capable of detecting incomplete tags in the HTML string you give it. under normal circumstances this will make the parser throw an error 😭 but if you use the option -incvar to specify a global (or namespace) variable, the parser will store the incomplete part of the input into this variable instead. this aids greatly in handling incrementally arriving HTML, because the parser will handle whatever it can and defer the handling of the incomplete part until more data has arrived. iconic behavior honestly. ✨

another feature: the parser has two possible modes of operation. the normal mode is activated if the option -queue is not present on the command line invoking the parser. if it is present, the parser goes into the incremental mode instead.

the main difference? a parser in normal mode will immediately invoke the command prefix for each tag it encounters. in incremental mode, however, the parser generates a number of scripts which invoke the command prefix for groups of tags in the HTML string, and then stores these scripts in the specified queue. it is then your responsibility (the caller's) to make sure those scripts actually get executed. no pressure bestie 💀

Note: the queue object you give the parser has to provide the same interface as the queue defined in tcllib -> struct. so like, all queues created via that tcllib module can be used here immediately, no conversion needed. still, the queue doesn't have to come from tcllib -> struct as long as the same interface is provided. your queue, your rules 🎀

in both modes the parser will return an empty string to the caller. that's it. that's the return. empty. 🫠

the -split option may be given to a parser in incremental mode to specify the size of the groups it creates. in other words, -split 5 means each of the generated scripts will invoke the command prefix for 5 consecutive tags in the HTML string. a parser in normal mode will ignore this option and its value entirely. not its business.

the option -vroot specifies a virtual root tag. a parser in normal mode will invoke the command prefix for it immediately before and after it processes the tags in the HTML, thus simulating that the HTML string is enclosed in a <vroot> </vroot> combination. 🦋 in incremental mode, though, the parser is unable to provide the closing virtual root because it never knows when the input is complete. in that case, the first script generated by each invocation of the parser will contain an invocation of the command prefix for the virtual root as its first command. no closing tag though. sad but true.

the following options are available:

• -cmd cmd — the command prefix to invoke for every tag in the HTML string. defaults to ::htmlparse::debugCallback. 💖
• -vroot tag — the virtual root tag to add around the HTML in normal mode. in incremental mode it is the first tag in each chunk processed by the parser, but there will be no closing tags. defaults to hmstart.
• -split n — the size of the groups produced by an incremental mode parser. ignored when in normal mode. defaults to 10. values <= 0 are not allowed. 🚫
• -incvar var — the name of the variable where to store any incomplete HTML into. this makes most sense for the incremental mode. the parser will throw an error if it sees incomplete HTML and has no place to store it to — which makes sense for the normal mode. only incomplete tags are detected, not missing tags. optional, defaults to 'no variable'.

🎀 Interface to the command prefix 🎀

in normal mode, the parser will invoke the command prefix with four arguments appended. see ::htmlparse::debugCallback for a description.

in incremental mode, however, the generated scripts will invoke the command prefix with five arguments appended. the last four of these are the same ones mentioned above. the first is a placeholder string (@win@) for a clientdata value to be supplied later, during the actual execution of the generated scripts. this could be a tk window path, for example. this lets you preprocess HTML strings without committing them to a specific window, object, whatever, during parsing. the connection can be made later. this also means it is possible to cache preprocessed HTML. 👑

and of course, nothing prevents you from replacing that placeholder with an empty string. do what you want, it's your life.

🌈 ::htmlparse::debugCallback ?clientdata? tag slash param textBehindTheTag 🌈

this command is the standard callback used by the parser in ::htmlparse::parse if none was specified by the user. it simply dumps its arguments to stdout. no thoughts, just output. 💅

this callback can be used for both normal and incremental mode of the calling parser. in other words, it accepts four or five arguments. the last four arguments are described below. the optional fifth argument contains the clientdata value passed to the callback by a parser in incremental mode. all callbacks have to follow the signature of this command in the last four arguments, and callbacks used in incremental parsing have to follow this signature in the last five arguments. rules are rules bestie.

• the first argument, clientdata, is optional and present only if this command is invoked by a parser in incremental mode. it contains whatever the user of this package wishes. your data, your choice. ✨
• the second argument, tag, contains the name of the tag which is currently processed by the parser.
• the third argument, slash, is either empty or contains a slash character. it allows the callback to distinguish between opening tags (slash is empty) and closing tags (slash contains a slash character). slay, a slash-based vibe check.
• the fourth argument, param, contains the un-interpreted list of parameters to the tag.
• the fifth and last argument, textBehindTheTag, contains the text found by the parser behind the tag named in tag.

💜 ::htmlparse::mapEscapes html 💜

this command takes a HTML string, substitutes all escape sequences with their actual characters, and then returns the resulting string. HTML strings which do not contain escape sequences are returned unchanged. lowkey the easiest glow-up in the whole package. ✨

🦋 ::htmlparse::2tree html tree 🦋

this command is a wrapper around ::htmlparse::parse which takes an HTML string (in html) and converts it into a tree containing the logical structure of the parsed document. the name of the tree is given to the command as its second argument (tree).

the command does not generate the tree by itself — it expects that the caller provided it with an existing and empty tree. it also expects that the specified tree object follows the same interface as the tree object in tcllib -> struct. it doesn't have to be from tcllib -> struct, but it must provide the same interface. iykyk.

the internal callback does some basic checking of HTML validity and tries to recover from the most basic errors. the command returns the contents of its second argument. side effects are the creation and manipulation of a tree object. 🌳

each node in the generated tree represents one tag in the input. the name of the tag is stored in the attribute type of the node. any html attributes coming with the tag are stored unmodified in the attribute data of the tag. in other words, the command does not parse html attributes into their names and values. raw and unfiltered, periodt. 💅

if a tag contains text, its node will have children of type PCDATA containing this text. the text will be stored in the attribute data of these children.

🎀 ::htmlparse::removeVisualFluff tree 🎀

this command walks a tree as generated by ::htmlparse::2tree and removes all the nodes which represent visual tags and not structural ones. the purpose of the command is to make the tree easier to navigate without getting bogged down in visual information that isn't relevant to the search. its only argument is the name of the tree to cut down. ✂️ it's giving decluttering, it's giving spring cleaning.

🍓 ::htmlparse::removeFormDefs tree 🍓

like ::htmlparse::removeVisualFluff, this command is here to cut down on the size of the tree as generated by ::htmlparse::2tree. it removes all nodes representing forms and form elements. its only argument is the name of the tree to cut down. bye bye form defs, you will not be missed 🙈

🐛 BUGS, IDEAS, FEEDBACK 💖

okay bestie, real talk: this document? and the whole package it's describing? they will absolutely have bugs and other little problems, no cap. it's not a vibe, it's just how software is. 😭

so if you find something cursed, please report it in the category htmlparse of the Tcllib Trackers:

http://core.tcl.tk/tcllib/reportlist

and like, not just bugs — if you have ideas for enhancements for either the package and/or the documentation, send those too, fr fr. we want them. 🫶

when you're proposing code changes, please provide unified diffs, i.e the output of diff -u. that's the format, that's the assignment, periodt. 💅

also, note further that attachments are strongly preferred over inlined patches. the way to attach stuff is: go to the Edit form of the ticket immediately after its creation, and then use the left-most button in the secondary navigation bar. that's it, that's the whole trick. 🎀

---

👀 SEE ALSO 👑

• struct::tree — go read that one too, it's the whole tree structure situation, iykyk 🌳

---

🔑 KEYWORDS 🔑

• html
• parsing
• queue
• tree

literally the four words that describe your entire afternoon ✨

---

🎀 CATEGORY 🎀

Text processing — that's where this lives, bestie. 💜

---

and that's the whole page!! you made it to the end, main character energy fr 🌈💖 this one's from tcllib, version 1.2.3, the htmlparse(n) page — go forth and parse some html, you absolute icon 👑🦋