SGML::P...:OpenSP(3)

📦
version 0.994 · 9 versions
, in opensuse-leap 16.0

💖 NAME 💖

SGML::Parser::OpenSP(3) — User Contributed Perl Documentation. It’s giving Parse SGML documents using OpenSP, and bestie, it absolutely ate.

✨ SYNOPSIS ✨

use SGML::Parser::OpenSP;

my $p = SGML::Parser::OpenSP->new;
my $h = ExampleHandler->new;

$p->catalogs(qw(xhtml.soc));
$p->warnings(qw(xml valid));
$p->handler($h);

$p->parse("example.xhtml");

🦋 DESCRIPTION 🦋

Okay so this module is literally an interface to the OpenSP SGML parser. OpenSP and this module are both event based. When the parser recognizes parts of the document — say the start or end of an element — any handlers registered for that type of event get called with suitable parameters. Main character parser energy, no cap.

🫶 COMMON METHODS 🫶

  • new() — Returns a new SGML::Parser::OpenSP object. Takes no arguments. Slay, simple.
  • parse($file) — Parses the file passed as an argument. Note that this must be a filename and not a filehandle. See PROCESSING FILES below for details.
  • parse_string($data) — Parses the data passed as an argument. See PROCESSING FILES below for details.
  • halt() — Halts processing before parsing the entire document. Takes no arguments.
  • split_message() — Splits OpenSP’s error messages into their component parts. See POST-PROCESSING ERROR MESSAGES below for details.
  • get_location() — See POSITIONING INFORMATION below for details.

💅 CONFIGURATION 💅

✅ BOOLEAN OPTIONS ✅

  • $p->handler([$handler]) — Report events to the blessed reference $handler. This is the bestie that receives the event gossip.

😭 ERROR MESSAGE FORMAT 😭

  • $p->show_open_entities([$bool]) — Describe open entities in error messages. Error messages always include the position of the most recently opened external entity. The default is false.
  • $p->show_open_elements([$bool]) — Show the generic identifiers of open elements in error messages. The default is false.
  • $p->show_error_numbers([$bool]) — Show message numbers in error messages.

🎀 GENERATED EVENTS 🎀

  • $p->output_comment_decls([$bool]) — Generate comment_decl events. The default is false.
  • $p->output_marked_sections([$bool]) — Generate marked section events (marked_section_start, marked_section_end, ignored_chars). The default is false.
  • $p->output_general_entities([$bool]) — Generate general_entity events. The default is false.

🦋 IO SETTINGS 🦋

  • $p->map_catalog_document([$bool]) — parse arguments specify catalog files rather than the document entity. The document entity is specified by the first DOCUMENT entry in the catalog files. The default is false.
  • $p->restrict_file_reading([$bool]) — Restrict file reading to the specified directories (see the search_dirs method and the SGML_SEARCH_PATH environment variable). You should turn this option on and configure the search paths accordingly if you intend to process untrusted resources. The default is false.
  • $p->catalogs([@catalogs]) — Map public identifiers and entity names to system identifiers using the specified catalog entry files. Multiple catalogs are allowed. If there is a catalog entry file called catalog in the same place as the document entity, it will be searched for immediately after those specified.
  • $p->search_dirs([@search_dirs]) — Search the specified directories for files specified in system identifiers. Multiple values options are allowed. See the description of the osfile storage manager in the OpenSP documentation for more information about file searching.
  • $p->pass_file_descriptor([$bool]) — Instruct parse_string to pass the input data down to the guts of OpenSP using the OSFD storage manager (if true) or the OSFILE storage manager (if false). This amounts to the difference between passing a file descriptor and a (temporary) file name.

The default is true except on platforms, such as Win32, which are known to not support passing file descriptors around in this manner. On platforms which support it you can call this method with a false parameter to force use of temporary file names instead.

In general, this will do the right thing on its own so it’s best to consider this an internal method. If your platform is such that you have to force use of the OSFILE storage manager, please report it as a bug and include the values of $^O, $Config{archname}, and a description of the platform (e.g. "Windows Vista Service Pack 42"). iykyk.

🌸 PROCESSING OPTIONS 🌸

  • $p->include_params([@include_params]) — For each name in @include_params pretend that
<!ENTITY % name "INCLUDE">

occurs at the start of the document type declaration subset in the SGML document entity. Since repeated definitions of an entity are ignored, this definition will take precedence over any other definitions of this entity in the document type declaration. Multiple names are allowed. If the SGML declaration replaces the reserved name INCLUDE then the new reserved name will be the replacement text of the entity. Typically the document type declaration will contain

<!ENTITY % name "IGNORE">

and will use %name; in the status keyword specification of a marked section declaration. In this case the effect of the option will be to cause the marked section not to be ignored.

  • $p->active_links([@active_links]) — ???. No cap, the docs literally just have three question marks here.

⭐ ENABLING WARNINGS ⭐

Additional warnings can be enabled using

$p->warnings([@warnings])

The following values can be used to enable warnings:

  • xml — Warn about constructs that are not allowed by XML.
  • mixed — Warn about mixed content models that do not allow #pcdata anywhere.
  • sgmldecl — Warn about various dubious constructions in the SGML declaration.
  • should — Warn about various recommendations made in ISO 8879 that the document does not comply with. (Recommendations are expressed with “should”, as distinct from requirements which are usually expressed with “shall”.)
  • default — Warn about defaulted references.
  • duplicate — Warn about duplicate entity declarations.
  • undefined — Warn about undefined elements: elements used in the DTD but not defined.
  • unclosed — Warn about unclosed start and end-tags.
  • empty — Warn about empty start and end-tags.
  • net — Warn about net-enabling start-tags and null end-tags.
  • min-tag — Warn about minimized start and end-tags. Equivalent to combination of unclosed, empty and net warnings.
  • unused-map — Warn about unused short reference maps: maps that are declared with a short reference mapping declaration but never used in a short reference use declaration in the DTD.
  • unused-param — Warn about parameter entities that are defined but not used in a DTD. Unused internal parameter entities whose text is INCLUDE or IGNORE won’t get the warning.
  • notation-sysid — Warn about notations for which no system identifier could be generated.
  • all — Warn about conditions that should usually be avoided (in the opinion of the author). Equivalent to: mixed, should, default, undefined, sgmldecl, unused-map, unused-param, empty and unclosed.

💀 DISABLING WARNINGS 💀

A warning can be disabled by using its name prefixed with no-. Thus calling warnings(qw(all no-duplicate)) will enable all warnings except those about duplicate entity declarations.

The following values for warnings() disable errors:

  • no-idref — Do not give an error for an ID reference value which no element has as its ID. The effect will be as if each attribute declared as an ID reference value had been declared as a name.
  • no-significant — Do not give an error when a character that is not a significant character in the reference concrete syntax occurs in a literal in the SGML declaration. This may be useful in conjunction with certain buggy test suites.
  • no-valid — Do not require the document to be type-valid. This has the effect of changing the SGML declaration to specify VALIDITY NOASSERT and IMPLYDEF ATTLIST YES ELEMENT YES. An option of valid has the effect of changing the SGML declaration to specify VALIDITY TYPE and IMPLYDEF ATTLIST NO ELEMENT NO. If neither valid nor no-valid are specified, then the VALIDITY and IMPLYDEF specified in the SGML declaration will be used.

🌈 XML WARNINGS 🌈

The following warnings are turned on for the xml warning described above:

  • inclusion — Warn about inclusions in element type declarations.
  • exclusion — Warn about exclusions in element type declarations.
  • rcdata-content — Warn about RCDATA declared content in element type declarations.
  • cdata-content — Warn about CDATA declared content in element type declarations.
  • ps-comment — Warn about comments in parameter separators.
  • attlist-group-decl — Warn about name groups in attribute declarations.
  • element-group-decl — Warn about name groups in element type declarations.
  • pi-entity — Warn about PI entities.
  • internal-sdata-entity — Warn about internal SDATA entities.
  • internal-cdata-entity — Warn about internal CDATA entities.
  • external-sdata-entity — Warn about external SDATA entities.
  • external-cdata-entity — Warn about external CDATA entities.
  • bracket-entity — Warn about bracketed text entities.
  • data-atts — Warn about attribute definition list declarations for notations.
  • missing-system-id — Warn about external identifiers without system identifiers.
  • conref — Warn about content reference attributes.
  • current — Warn about current attributes.
  • nutoken-decl-value — Warn about attributes with a declared value of NUTOKEN or NUTOKENS.
  • number-decl-value — Warn about attributes with a declared value of NUMBER or NUMBERS.
  • name-decl-value — Warn about attributes with a declared value of NAME or NAMES.
  • named-char-ref — Warn about named character references.
  • refc — Warn about omitted refc delimiters.
  • temp-ms — Warn about TEMP marked sections.
  • rcdata-ms — Warn about RCDATA marked sections.
  • instance-include-ms — Warn about INCLUDE marked sections in the document instance.
  • instance-ignore-ms — Warn about IGNORE marked sections in the document instance.
  • and-group — Warn about AND connectors in model groups.
  • rank — Warn about ranked elements.
  • empty-comment-decl — Warn about empty comment declarations.
  • att-value-not-literal — Warn about attribute values which are not literals.
  • missing-att-name — Warn about omitted attribute names in start tags.
  • comment-decl-s — rat on stray spaces hanging out before the MDC in comment declarations 🫧
  • comment-decl-multiple — side-eye comment declarations that are packing multiple comments in one 💀
  • missing-status-keyword — flag marked sections that forgot their status keyword entirely 😭
  • multiple-status-keyword — and flag marked sections that brought multiple status keywords, greedy 💅
  • instance-param-entity — warn about parameter entities showing up in the document instance
  • min-param — warn about minimization parameters in element type declarations
  • mixed-content-xml — call out mixed-content cases that XML simply does not allow 🚫
  • name-group-not-or — name groups whose connector isn't OR get a warning, no cap
  • pi-missing-name — processing instructions that don't start with a name? warned ✨
  • instance-status-keyword-s — spaces between DSO and the status keyword in marked sections get flagged
  • external-data-entity-ref — references to external data entities in the content get warned
  • att-value-external-entity-ref — same energy but for references to external data entities in attribute values
  • data-delim — warn about occurrences of <' and &' being used as data 👀
  • explicit-sgml-decl — warn when there's an explicit SGML declaration
  • internal-subset-ms — marked sections lurking in the internal subset get a warning
  • default-entity — a default entity declaration? warned 🎀
  • non-sgml-char-ref — numeric character references pointing at non-SGML characters get flagged
  • internal-subset-ps-param-entity — parameter entity references inside parameter separators in the internal subset
  • internal-subset-ts-param-entity — parameter entity references inside token separators in the internal subset
  • internal-subset-literal-param-entity — parameter entity references inside parameter literals in the internal subset

🌸 PROCESSING FILES 🌸

okay so to actually start processing a document and receive events, you gotta call the parse method. It takes exactly one argument: the path to a file — and bestie, that's a path, not a file handle, iykyk. 🫶

You must set an event handler with the handler method before you call parse. And the return value of parse? Currently undefined. That's it. That's the tea. ☕

🦋 EVENT HANDLERS 🦋

To get data out of the parser you need to write an event handler. Like, for example:

package ExampleHandler;

sub new { bless {}, shift }

sub start_element
{
    my ($self, $elem) = @_;
    printf "  * %s\n", $elem->{Name};
}

This little icon would print every element name as it's found in the document. For a typical XHTML document that might give you something like:

  * html
  * head
  * title
  * body
  * p
  * ...

The events match the ones in the generic interface to OpenSP super closely — go look at <http://openjade.sf.net/doc/generic.htm> for the full lore. 📚

Here's the glow-up this module did: event names got lowercased, underscores separate the words, and properties are capitalized. Arrays come through as Perl array references. Position info is not passed to the handler — instead it's made available through the get_location method, which you can call from inside event handlers. Some redundant info got stripped too, and the generic identifier of an element lives in the Name hash entry. 💅

So for example, for an EndElementEvent the end_element handler gets called with a hash reference like:

{
  Name => 'gi'
}

The following events are defined, bestie:

* appinfo
* processing_instruction
* start_element
* end_element
* data
* sdata
* external_data_entity_ref
* subdoc_entity_ref
* start_dtd
* end_dtd
* end_prolog
* general_entity       # set $p->output_general_entities(1)
* comment_decl         # set $p->output_comment_decls(1)
* marked_section_start # set $p->output_marked_sections(1)
* marked_section_end   # set $p->output_marked_sections(1)
* ignored_chars        # set $p->output_marked_sections(1)
* error
* open_entity_change

And here's a lil caveat: if the generic interface docs say some piece of data isn't valid, it will not be available through this interface — meaning the respective key just won't exist in the hash ref. No key, no drama. 🙈

⭐ POSITIONING INFORMATION ⭐

Event handlers can call the get_location method on the parser object to grab positioning info. That method returns a hash reference with these properties:

LineNumber   => ..., # line number
ColumnNumber => ..., # column number
ByteOffset   => ..., # number of preceding bytes
EntityOffset => ..., # number of preceding bit combinations
EntityName   => ..., # name of the external entity
FileName     => ..., # name of the file

Any of these can be undef or an empty string. Slay, but also, heads up. 🎀

💜 POST-PROCESSING ERROR MESSAGES 💜

OpenSP hands you error messages as one big string instead of individual components (like line numbers or message text). So the split_message method on the parser object exists to post-process those error message strings as reliably as possible. You can use it from an error event handler if the parser object is accessible, like so:

sub error
{
  my $self = shift;
  my $erro = shift;
  my $mess = $self->{parser}->split_message($erro);
}

See the documentation of split_message in the SGML::Parser::OpenSP::Tools documentation for more. 🧚

🌈 UNICODE SUPPORT 🌈

Every string that comes back from event handlers and helper routines is UTF-8 encoded with the UTF-8 flag turned on — periodt. Helper functions like split_message expect (but do not check!) that string arguments are UTF-8 encoded and have the UTF-8 flag turned on. The behavior of helper functions is undefined if you pass unexpected input, so please don't do that. Avoid, avoid, avoid. 💀

parse has limited support for binary input, but that binary input must be compatible with OpenSP's generic interface requirements, and you must specify the encoding through means available to OpenSP so it can properly decode it. Any Perl-specific encoding meta data about that binary input (like encoding disciplines for file handles when you pass a file descriptor) will be ignored. For the specifics, refer to the OpenSP manual:

🔥 ENVIRONMENT VARIABLES 🔥

OpenSP supports a bunch of environment variables that control specific processing aspects, like SGML_SEARCH_PATH or SP_CHARSET_FIXED. Portable applications need to make sure these are set before the OpenSP library gets loaded into memory — which happens when the XS code is loaded. That means you gotta wrap your code in a BEGIN block:

BEGIN { $ENV{SP_CHARSET_FIXED} = 1; }
use SGML::Parser::OpenSP;
# ...

Otherwise your changes to the environment might not propagate to OpenSP. This applies specifically to Win32 systems. 🫡

  • SGML_SEARCH_PATH — see <http://openjade.sourceforge.net/doc/sysid.htm>.
  • SP_HTTP_USER_AGENT — the User-Agent header for HTTP requests.
  • SP_HTTP_ACCEPT — the Accept header for HTTP requests.
  • SP_MESSAGE_FORMAT — enables run time selection of message format. Value is one of XML, NONE, TRADITIONAL. Whether this actually has an effect depends on a compile time setting which might not be enabled in your OpenSP build. This module assumes that no such support was compiled in. 🫠
  • SGML_CATALOG_FILES, SP_USE_DOCUMENT_CATALOG — see <http://openjade.sourceforge.net/doc/catalog.htm>.
  • SP_SYSTEM_CHARSET, SP_CHARSET_FIXED, SP_BCTF, SP_ENCODING — see <http://openjade.sourceforge.net/doc/charset.htm>.

Note (and this is a good note) that you can use the search_dirs method instead of SGML_SEARCH_PATH, and the catalogs method instead of SGML_CATALOG_FILES, plus attributes on storage object specifications for SP_BCTF and SP_ENCODING respectively. For example, if SP_CHARSET_FIXED is set to 1, you can do:

$p->parse("<OSFILE encoding='UTF-8'>example.xhtml");

…to process example.xhtml using the UTF-8 character encoding. Iconic. 👑

😭 KNOWN ISSUES 😭

OpenSP must be compiled with SP_MULTI_BYTE defined and with SP_WIDE_SYSTEM undefined. Otherwise this module will break at runtime or just straight up not compile. Not a vibe, but forewarned is forearmed. 💀

🐛 BUG REPORTS 🐛

Please report bugs in this module via <http://rt.cpan.org/NoAuth/Bugs.html?Dist=SGML-Parser-OpenSP>

Please report bugs in OpenSP via <http://sf.net/tracker/?group_id=2115&atid=102115>

Please send comments and questions to the spo-devel mailing list — see <http://lists.sf.net/lists/listinfo/spo-devel> for the details. 🫶

✨ SEE ALSO ✨

👑 AUTHORS 👑

  Terje Bless <link@cpan.org> wrote version 0.01.
  Bjoern Hoehrmann <bjoern@hoehrmann.de> wrote version 0.02+.
  Copyright (c) 2006-2008 Bjoern Hoehrmann <bjoern@hoehrmann.de>.
  This module is licensed under the same terms as Perl itself.

okay that's the whole page bestie, go forth and parse ✨ this was perl v5.42.0, dated 2023-08-09 — the source and version and date, all accounted for, mwah 💋 🎀

📦 where this page comes from