xsbuilder.osc2002(3)

📦

version 0.28 · 4 versions

💅 `xsbuilder.osc2002(3)` — XSBuilder: Autogenerating XS-glue Code 🎀

okay so this is the xsbuilder.osc2002(3) manual page — User Contributed Perl Documentation vibes ✨ — and it’s literally the O'Reilly OpenSource Convention 2002 talk page by Gerald Richter, ecos gmbh, http://www.ecos.de. We are so back in 2002 and the XS glue is glowing. 🌸

🌸 XSBuilder - What is it?

Its whole purpose is to automaticly generate a Perl interface to C code. 💖
It solves the problem that the Perl interface is not always uptodate with the C interface.
It saves a lot of Copy&Paste work — no cap, bestie.
Systematical changes have to be done only once. Example: changes in the memory management of strings. ✨
It’s part of the mod_perl 2.0 build system.
- Most code is developed by Doug MacEachern. Additionally I have:
  - abstracted the code from mod_perl so it’s useable for any C code,
  - added new features like callbacks and the ability to parse comments,
  - replaced C::Scan with a Parse::RecDescent to be platform and compiler independent.
- Goal is to replace the current mod_perl XS generation code with XSBuilder. 👑

✨ Inline versus XSBuilder

Inline: embed C-Code into Perl.
XSBuilder: create interface for existing C-libraries/applicationen. 🦋

🔥 XSBuilder - What does it do?

Create Perl functions/methods for every C function.
- The function can be assigned to different packages, also automatically by inspecting the first parameter. 💅
Create a Perl class for every C structure.
- Every element of structure becomes a Perl method to get/set its value. The object can be either a scalar reference (used by mod_perl) or a reference to a hash (use by Embperl), which allows to store extra data by the Perl code into this hash.
Create glue code to handle callbacks.
- There several sorts of callback, not all are implemented right now. 😭
Create Perl constant subs.
- Coming soon... ⭐

🌈 XSBuilder - How does it work?

Parse the C header files.
- Extract:
  - Functions, their arguments and return types
  - Structures and it’s members
  - Constants
  - Callbacks
- and create four tables which contains the results.
Create the XS code.
- Input is:
  - The source tables
  - Mapfiles which contains the mapping from C to Perl
  - Addtional C and Perl code that can be used to customize the interface
- Output is:
  - The XS files (one form every generated class)
  - Makefile.PL for every class
  - pm files

🧚 Parse the source

Create your own ParseSource class and override methods... 👇

package Apache::DAV::ParseSource;

use strict;
use vars qw{@ISA $VERSION} ;
use ExtUtils::XSBuilder::ParseSource  ;

@ISA = ('ExtUtils::XSBuilder::ParseSource') ;
$VERSION = '0.01';

my $dav_dir = 'C:\perl\msrc\cvs\mod_dav' ;
my $ap_dir  = 'c:\programme\apache group\apache' ;

# ============================================================================
sub find_includes {
    my $self = shift;
    return $self->{includes} if $self->{includes};
    my @includes = ("$ap_dir/include/ap_alloc.h", "$dav_dir/mod_dav.h") ;
    return $self->{includes} = $self -> sort_includes (\@includes) ;
    }

# ============================================================================
sub package     { 'Apache::DAV' }

# ============================================================================
sub preprocess {
    my $self     = shift ;
    $_[0] =~ s/(?:API_EXPORT)(?:_NONSTD)?\s*\(\s*(.*?)\s*\)/$1/g ;
}

1;

...run it 🏃‍♀️💨

use FindBin ;
use lib ($FindBin::Bin) ;

require ParseSource ;

Apache::DAV::ParseSource -> run ;

...and you get 🎁

C:\perl\msrc\davint>perl xsbuilder\source_scan.pl
Will use mod_dav in C:\perl\msrc\cvs\mod_dav
Will use Apache in c:\programme\apache group\apache
Initialize parser
scan c:\programme\apache group\apache/include/ap_alloc.h ...
constant: APACHE_ALLOC_H
func:     ap_init_alloc
func:     ap_cleanup_alloc
func:     ap_make_sub_pool
func:     ap_destroy_pool
constant: ap_pool_join
func:     ap_pool_join
func:     ap_find_pool
func:     ap_pool_is_ancestor
func:     ap_clear_pool
func:     ap_cleanup_for_exec
func:     ap_palloc
func:     ap_pcalloc
func:     ap_pstrdup
func:     ap_pstrndup
func:     ap_pstrcat
func:     ap_pvsprintf
  valuefield: ap_pool * : pool
  valuefield: int : elt_size
  valuefield: int : nelts
  valuefield: int : nalloc
  valuefield: char * : elts
struct:    (type=array_header)
...

The result is stored in four tables 🧚‍♀️💖

xsbuilder/tables/Apache/DAV/FuntionTable.pm
- Contains all function, it arguments and comments.
xsbuilder/tables/Apache/DAV/ConstantTable.pm
- Contains all constants.
xsbuilder/tables/Apache/DAV/StructureTable.pm
- Contains all structures, it’s members and their comments.
xsbuilder/tables/Apache/DAV/CallbackTable.pm
- Contains all callback function definitions.

🎀 Create the map files

Mapfiles are used to tell XSBuilder how C datatypes, structures and function aruments should be mapped into Perl ones. 🗺️✨

Create your own WrapXS class and override methods:

package Apache::DAV::WrapXS ;
use strict;
use vars qw{@ISA $VERSION} ;

use ExtUtils::XSBuilder::WrapXS ;

@ISA = ('ExtUtils::XSBuilder::WrapXS') ;
$VERSION = '0.01';

# ============================================================================
sub new_parsesource  { [ Apache::DAV::ParseSource->new ] }

# ============================================================================
sub my_xs_prefix  { 'davxs_' }

# ============================================================================
sub h_filename_prefix  { 'moddav_xs_' }

# ============================================================================
sub xs_includes {
    my $self = shift ;
    my $i = $self -> SUPER::xs_includes ;
    my @i = grep (!/ap_alloc/, @$i) ;
    return \@i ;
}

XSBuilder can create/update initial maps for you 🫶

use FindBin ;
use lib ($FindBin::Bin) ;

require ParseSource ;
require WrapXS ;

Apache::DAV::WrapXS->checkmaps (' ');

run it 🏁

C:\perl\msrc\davint>perl xsbuilder\xs_check.pl
Will use mod_dav in C:\perl\msrc\cvs\mod_dav
Will use Apache in c:\programme\apache group\apache
Parse xsbuilder\maps/_types.map...
WARNING: No *_function.map file found in xsbuilder\maps
WARNING: No *_callback.map file found in xsbuilder\maps
WARNING: No *_structure.map file found in xsbuilder\maps
Write xsbuilder\maps/new_function.map...
Write xsbuilder\maps/new_callback.map...
Write xsbuilder\maps/new_structure.map...
Write xsbuilder\maps/new_type.map...

Now we have four map files 🌈

new_types.map
- Contains the mapping from C type to Perl classes.
new_functions.map
- Contains the mapping form C functions to Perl functions. Can be used to reorder arguments, tell XSBuilder which arguments are actualy return values and in which Perl package the function will be created.
new_structures.map
- Contains the mapping from C structures to Perl classes and defines for which members a access methods should be created. You can also specify if you want a "new" method for the class.
new_callbacks.map
- Contains the mapping form C callback functions to Perl callback functions. Can be used to reorder arguments, tell XSBuilder which arguments are actualy return values and in which Perl package the function will be created.

It’s a good idea to rename the prefix from "new_" to something unique, here we use "dav". 💡

Everytime you rerun checkmaps, XSBuilder will create new_* files with the items that are not already part of the other maps. 👑

Next step is to customize the maps... 🎨

💜 type map

autogenerated dav_type.map 👇

DIR |
FILE        |
HANDLE      |
array_header        |
dav_buffer  |
dav_dyn_context     |
dav_dyn_hooks       |
dav_dyn_module      |
dav_dyn_provider    |
dav_error   |
dav_get_props_result        |
dav_hooks_liveprop  |
dav_hooks_locks     |
dav_hooks_repository        |
dav_if_header       |
dav_if_state_type   |
...

Add Perl classes 💅

struct array_header         | Apache::Array
struct dav_buffer           |
struct dav_datum            | Apache::DAV::Datum
struct dav_dyn_context      | Apache::DAV::DynContext
struct dav_dyn_hooks        | Apache::DAV::DynHooks
struct dav_dyn_module       | Apache::DAV::DynModule
struct dav_dyn_provider     | Apache::DAV::DynProvider
struct dav_error            | Apache::DAV::Error
struct dav_get_props_result | Apache::DAV::PropsResult
struct dav_hooks_db         | Apache::DAV::HooksDb
struct dav_hooks_liveprop   | Apache::DAV::HooksLiveprop
struct dav_hooks_locks      | Apache::DAV::HooksLocks
struct dav_hooks_repository | Apache::DAV::HooksRepository
struct dav_hooks_vsn        |
struct dav_if_header        | Apache::DAV::IfHeader
struct dav_if_state_list    | Apache::DAV::StateList
...
struct pool                 | Apache::Pool
struct request_rec          | Apache::
struct server_rec           | Apache::Server
...

Defines the mapping from C datatypes to Perl datatypes and classes and tells XSBuilder which datatype are (like) structures. 🦋

🗺️ function map

Okay bestie, so the function map is literally the vibe check for how C function arguments line up with Perl arguments. No cap.

Tell XSBuilder where to place functions and which prefix to strip:

MODULE=Apache::DAV   PACKAGE=guess  PREFIX=dav_

Simple entries in the function map get mapped 1:1 from C to Perl, like:

dav_add_response
dav_buffer_append
dav_buffer_init
dav_buffer_place
dav_buffer_place_mem
dav_check_bufsize
dav_close_propdb
dav_collect_liveprop_uris
dav_dyn_module_add
dav_empty_elem
...

The following map file entry tells XSBuilder that the value of res_p should be returned:

dav_get_resource           | | r, <res_p

The C function:

int dav_get_resource(request_rec *r, dav_resource **res_p) ;

How to call it from Perl:

# get a sub request of the actual document
$subr = $r -> lookup_uri($uri);

# get a mod_dav resource object
my ($err, $resource) =  $subr->get_resource;

You can let XSBuilder insert your custom code for the interface, too. If you call get_props from Perl, glue_dav_get_props will be called, and it can adjust the arguments and return types as necessary. The actual code for glue_dav_get_props comes from a separate include file:

dav_get_props      | glue_
dav_get_allprops   | glue_

xsinclude\Apache\DAV\PropResult\Apache__DAV__PropResults.h

dav_get_props_result * dav_glue_get_props(dav_propdb * db, dav_xml_doc *doc)
    {
    dav_get_props_result * result = (dav_get_props_result *)ap_palloc (db -> p, sizeof (dav_get_props_result)) ;
    *result = dav_get_props(db, doc) ;
    return result ;
    }

Arguments can be replaced too, fr fr:

MODULE=Apache::Array   PACKAGE=Apache::Array PREFIX=ap_

ap_make_array      | ap_make_array(r->pool, nelts, elt_size) | request_rec *:r, nelts, elt_size

ap_make_array requires a pool as its first parameter, so we pass the request_rec from Perl and XSBuilder will take the pool from the request_rec. Slay.

🏗️ structure map

MALLOC=strdup:$dest = ($type)strdup($src)
FREE=strdup:free($src)

<dav_xml_elem>
  name
  ns
  lang
  first_cdata
  following_cdata
  parent
  next
  first_child
  attr
  last_child
  ns_scope
  propid
  provider
  ns_map
  new
</dav_xml_elem>

MALLOC=strdup:$dest = ($type)ap_pstrdup(obj -> pool, $src)

<array_header>
  pool
  elt_size
  nelts
  nalloc
  elts
  private
</array_header>

🛠️ Create a accessor functions for every element and, if requested, a new method

You get accessor functions for every element and, if requested, a new method, like this:

$setprop = Apache::DAV::XMLElem -> new ({name => 'prop'}) ;
$elem    = Apache::DAV::XMLElem -> new ({name => $name, ns => $namespaces}) ;
$setprop -> first_child($elem) ;
$first = $setprop -> first_child ;

🧵 some datatypes, like strings, requires dynamic allocated memory

Some datatypes, like strings, require dynamically allocated memory. From _types.map, which contains a set of standard types:

int             | IV
int *           | UNDEFINED
unsigned int    | UV
signed int      | IV
long            | IV
long int        | IV
unsigned long   | UV
unsigned        | UV

char *                   | PV           | | | strdup
const char *             | PV           | | | strdup
const char **            | UNDEFINED
char const *             | PV           | | | strdup
unsigned char *          | PV           | | | strdup
const unsigned char *    | PV           | | | strdup
...

🔁 callback maps

Callback maps have the same options as function maps:

# dav_hooks_db -> open

dav_error *(*)(pool * p,const dav_resource * resource,int ro,dav_db * * pdb)  | p, resource, ro=0, <pdb

# dav_hooks_lock -> has_locks

dav_error *(*)(dav_lockdb * lockdb,const dav_resource * resource,int * locks_present)

✨ Generate the XS files

use FindBin ;
use lib ($FindBin::Bin) ;

require ParseSource ;
require WrapXS ;

Apache::DAV::WrapXS->run;

...and run...

C:\perl\msrc\davint>perl xsbuilder\xs_generate.pl
Will use mod_dav in C:\perl\msrc\cvs\mod_dav
Will use Apache in c:\programme\apache group\apache
Parse xsbuilder\maps/_types.map...
Parse xsbuilder\maps/dav_type.map...
mkdir xs
writing...xs//typemap
Parse xsbuilder\maps/dav_functions.map...
WARNING: Cannot map type int(*)(void * ,const char * ,const char * ) for function ap_table_do
WARNING: Cannot map type dav_buffer * for function dav_buffer_append
WARNING: Cannot map type dav_buffer * for function dav_buffer_init
WARNING: Cannot map type dav_buffer * for function dav_buffer_place
WARNING: Cannot map type dav_buffer * for function dav_buffer_place_mem
WARNING: Cannot map type dav_buffer * for function dav_check_bufsize
WARNING: Cannot map return type int * for function dav_collect_liveprop_uris
WARNING: Cannot map type dav_resource * * for function dav_ensure_resource_writable
WARNING: Cannot map type dav_buffer * for function dav_lock_get_activelock
WARNING: Cannot map type dav_buffer * for function dav_set_bufsize
WARNING: Cannot map type int * for function dav_xml2text
struct array_header...
Parse xsbuilder\maps/dav_structure.map...
     elt_size...
     nelts...
     nalloc...
     elts...
struct dav_buffer...
struct dav_datum...
     dptr...
     dsize...
struct dav_dyn_context...

🎀 Makefile.PL

We need to create a top-level Makefile.PL:

use ExtUtils::MakeMaker ();

my $apdir  = '/path/to/apache';
my $davdir = '/path/to/moddav';

%MMARGS = (
           'INC'    => "-I\"$davdir\" -I\"$apdir/include\" -I\"$apdir/os/unix\" -I\"$dir/xs\"  -I\"$dir/xsinclude\"",
           ) ;

open FH, ">xs/mmargs.pl" or die "Cannot open xs/mmargs.pl ($!)" ;
print FH Data::Dumper -> Dump ([\%MMARGS], ['MMARGS']) ;
close FH ;

ExtUtils::MakeMaker::WriteMakefile(
    'NAME'    => 'Apache::DAV',
    'VERSION' => '0.13',
    %MMARGS,
);

Makefile.PL's for all classes are generated automatically:

C:\perl\msrc\davint>perl Makefile.PL
Will use Apache in c:\programme\apache group\apache
Will use mod_dav in C:\perl\msrc\cvs\mod_dav
Checking if your kit is complete...
Looks good
Writing Makefile for Apache::Array
Writing Makefile for Apache::DAV::Datum
Writing Makefile for Apache::DAV::DynContext
Writing Makefile for Apache::DAV::DynHooks
Writing Makefile for Apache::DAV::DynModule
Writing Makefile for Apache::DAV::DynProvider
Writing Makefile for Apache::DAV::Error
Writing Makefile for Apache::DAV::HooksDb
Writing Makefile for Apache::DAV::HooksLiveprop
Writing Makefile for Apache::DAV::HooksLocks
Writing Makefile for Apache::DAV::HooksRepository
Writing Makefile for Apache::DAV::IfHeader
Writing Makefile for Apache::DAV::Lock
Writing Makefile for Apache::DAV::LockDB
Writing Makefile for Apache::DAV::LockTokenList
Writing Makefile for Apache::DAV::LockupResult
Writing Makefile for Apache::DAV::PropCtx
Writing Makefile for Apache::DAV::PropsResult
Writing Makefile for Apache::DAV::Resource
Writing Makefile for Apache::DAV::Response
Writing Makefile for Apache::DAV::StateList
Writing Makefile for Apache::DAV::Text
Writing Makefile for Apache::DAV::TextHeader
Writing Makefile for Apache::DAV::WalkerCtx
Writing Makefile for Apache::DAV::XMLAttr
Writing Makefile for Apache::DAV::XMLDoc
Writing Makefile for Apache::DAV::XMLElem
Writing Makefile for Apache::DAV
Writing Makefile for Apache::TableEntry
Writing Makefile for Apache
Writing Makefile for WrapXS
Writing Makefile for Apache::DAV

and now compile... iconic.

🌈 How does it go on...

📚 Generating documentation

XSBuilder already extracts source comments for functions and structures. It also parses doxygen comments, which are used in Apache 2.0. Lyle Brooks has started on automatically creating POD files from this information. Obsessed.

🔧 Improving callbacks

Callbacks are the main area that needs improvement. That's the vibe check, bestie.

🫶 Bring it back to mod_perl 2.0

First version will be released just after the conference to CPAN. Any feedback and help appreciated.

Questions?

Okay that's the whole page! Source/version: perl v5.32.1; date: 2019-10-17; page: xsbuilder.osc2002(3). Stay iconic 💅✨

📦 where this page comes from

epel 9: perl-ExtUtils-XSBuilder 0.28-34.el9 · GPL+ or Artistic
💖 girlified by deepseek-ai/DeepSeek-V4.1-Flash-1017 on 2026-11-04: a language model's rewrite of the original, under its terms

xsbuilder.osc2002(3)