Dancer...du::OAI(3pm)

📦 version 0.0508, in debian bookworm, debian trixie, ubuntu noble, ubuntu resolute, ubuntu stonking

💖 NAME 💖

Dancer::Plugin::Catmandu::OAI — bestie, it's an OAI-PMH provider backed by a searchable Catmandu::Store. Yes that's the whole plot. Slay.

✨ SYNOPSIS ✨

okay so this is literally all it takes:

#!/usr/bin/env perl

use Dancer;
use Catmandu;
use Dancer::Plugin::Catmandu::OAI;

Catmandu->load;
Catmandu->config;

my $options = {};

oai_provider '/oai' , %$options;

dance;

🌸 DESCRIPTION 🌸

Dancer::Plugin::Catmandu::OAI is a Dancer plugin that serves up OAI-PMH services for Catmandu::Stores that support CQL — think Catmandu::Store::ElasticSearch and friends. Follow the installation steps below and boom, you have your own OAI-PMH server. It's giving main character energy fr fr. 👑

🎀 REQUIREMENTS 🎀

In the examples below an ElasticSearch 1.7.2 server is used: https://www.elastic.co/downloads/past-releases/elasticsearch-1-7-2

Follow the instructions below for a demonstration installation:

$ cpanm Dancer Catmandu::OAI Catmandu::Store::ElasticSearch

$ wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.2.zip
$ unzip elasticsearch-1.7.2.zip
$ cd elasticsearch-1.7.2
$ bin/elasticsearch

🦋 RECORDS 🦋

Records stored in the Catmandu::Store can be in any format. Preferably the format should be easy to convert into the mandatory OAI-DC format. At a minimum each record contains an identifier _id and a field containing a datestamp. Like, the bare minimum, no cap.

$ cat sample.yml
---
_id: oai:my.server.org:123456
datestamp: 2016-05-17T13:37:18Z
creator:
 - Musterman, Max
 - Jansen, Jan
 - Svenson, Sven
title:
 - Test record
...

🧚 CATMANDU CONFIGURATION 🧚

ElasticSearch requires a configuration file to map record fields to CQL terms. Below is a minimal configuration required to query for identifiers and datastamps in the ElasticSearch collection:

$ cat catmandu.yml
---
store:
  oai:
    package: ElasticSearch
    options:
      index_name: oai
      bags:
        data:
          cql_mapping:
            default_index: basic
            indexes:
              _id:
                op:
                  'any': true
                  'all': true
                  '=': true
                  'exact': true
                field: '_id'
              datestamp:
                op:
                  '=': true
                  '<': true
                  '<=': true
                  '>=': true
                  '>': true
                  'exact': true
                field: 'datestamp'
      index_mappings:
        publication:
          properties:
            datestamp: {type: date, format: date_time_no_millis}

💅 IMPORT RECORDS 💅

With the Catmandu configuration files in place, records can be imported with the catmandu command:

# Drop the existing ElasticSearch 'oai' collection
$ catmandu drop oai

# Import the sample record
$ catmandu import YAML to oai < sample.yml

# Test if the records are available in the 'oai' collection
$ catmandu export oai

🌈 DANCER CONFIGURATION 🌈

The Dancer configuration file config.yml contains basic information for the OAI-PMH plugin to work. Here's the whole menu, no skips:

store — In which Catmandu::Store are the metadata records stored
bag — In which Catmandu::Bag are the records of this store (use: data as default)
datestamp_field — Which field in the record contains a datestamp (datestamp in our example above)
datestamp_index — Which CQL index should be used to find records within a specified date range (if not specified, the value from the datestamp_field setting is used)
repositoryName — The name of the repository
uri_base — The full base url of the OAI controller. To be used when behind a proxy server. When not set, this module relies on the Dancer request to provide its full url. Use middleware like ReverseProxy or Dancer::Middleware::Rebase in that case.
adminEmail — An administrative email. Can be string or array of strings. This will be included in the Identify response.
compression — a compression encoding supported by the repository. Can be string or array of strings. This will be included in the Identify response.
description — XML container that describes your repository. Can be string or array of strings. This will be included in the Identify response. Note that this module will try to validate the XML data.
earliestDatestamp — The earliest datestamp available in the dataset as YYYY-MM-DDTHH:MM:SSZ. This will be determined dynamically if no static value is given.
deletedRecord — The policy for deleted records. See also: https://www.openarchives.org/OAI/openarchivesprotocol.html#DeletedRecords
repositoryIdentifier — A prefix to use in OAI-PMH identifiers
cql_filter — A CQL query to find all records in the database that should be made available to OAI-PMH
default_search_params — set default arguments that get passed to every call to the bag's search method
search_strategy — default is paginate, set to es.scroll to avoid deep paging (Elasticsearch only)
limit — The maximum number of records to be returned in each OAI-PMH request
delimiter — Delimiters used in prefixing a record identifier with a repositoryIdentifier (use: : as default)
sampleIdentifier — A sample identifier
metadata_formats — An array of metadataFormats that are supported
- metadataPrefix — A short string for the name of the format
- schema — An URL to the XSD schema of this format
- metadataNamespace — A XML namespace for this format
- template — The path to a Template Toolkit file to transform your records into this format
- fix — Optionally an array of one or more Catmandu::Fix-es or Fix files
sets — Optional an array of OAI-PMH sets and the CQL query to retrieve records in this set from the Catmandu::Store
- setSpec — A short string for the same of the set
- setName — A longer description of the set
- setDescription — an optional and repeatable container that may hold community-specific XML-encoded data about the set. Should be string or array of strings.
- cql — The CQL command to find records in this set in the Catmandu::Store
xsl_stylesheet — Optional path to an xsl stylesheet
template_options — An optional hash of configuration options that will be passed to Catmandu::Exporter::Template or Template.

Below is a sample minimal configuration for the sample.yml demo above:

$ cat config.yml
charset: "UTF-8"
plugins:
  'Catmandu::OAI':
    store: oai
    bag: data
    datestamp_field: datestamp
    repositoryName: "My OAI DataProvider"
    uri_base: "http://oai.service.com/oai"
    adminEmail: me@example.com
    earliestDatestamp: "1970-01-01T00:00:01Z"
    cql_filter: "datestamp>1970-01-01T00:00:01Z"
    deletedRecord: persistent
    repositoryIdentifier: oai.service.com
    limit: 200
    delimiter: ":"
    sampleIdentifier: "oai:oai.service.com:1585315"
    metadata_formats:
      -
        metadataPrefix: oai_dc
        schema: "http://www.openarchives.org/OAI/2.0/oai_dc.xsd"
        metadataNamespace: "http://www.openarchives.org/OAI/2.0/oai_dc/"
        template: oai_dc.tt

🍓 METADATAPREFIX TEMPLATE 🍓

For each metadataPrefix a Template Toolkit file needs to exist which translate Catmandu::Store records into XML records. At least one Template Toolkit file should be made available to transform stored records into Dublin Core. The example below contains an example file to transform sample.yml type records into Dublin Core:

$ cat oai_dc.tt
<oai_dc:dc xmlns="http://www.openarchives.org/OAI/2.0/oai_dc/"
           xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/"
           xmlns:dc="http://purl.org/dc/elements/1.1/"
           xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
           xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/ http://www.openarchives.org/OAI/2.0/oai_dc.xsd">
[%- FOREACH var IN ['title' 'creator' 'subject' 'description' 'publisher' 'contributor' 'date' 'type' 'format' 'identifier' 'source' 'language' 'relation' 'coverage' 'rights'] %]
    [%- FOREACH val IN $var %]
    <dc:[% var %]>[% val | html %]</dc:[% var %]>
    [%- END %]
[%- END %]
</oai_dc:dc>

🔥 START DANCER 🔥

If all the required files are available, then a Dancer application can be started. See the demo directory of this distribution for a complete example:

$ ls
app.pl  catmandu.yml  config.yml  oai_dc.tt
$ cat app.pl
#!/usr/bin/env perl

use Dancer;
use Catmandu;
use Dancer::Plugin::Catmandu::OAI;

Catmandu->load;
Catmandu->config;

my $options = {};

oai_provider '/oai' , %$options;

dance;

# Start Dancer
$ perl ./app.pl

# Test queries:

$ curl "http://localhost:3000/oai?verb=Identify"
$ curl "http://localhost:3000/oai?verb=ListSets"
$ curl "http://localhost:3000/oai?verb=ListMetadataFormats"
$ curl "http://localhost:3000/oai?verb=ListIdentifiers&metadataPrefix=oai_dc"
$ curl "http://localhost:3000/oai?verb=ListRecords&metadataPrefix=oai_dc"

⭐ SEE ALSO ⭐

Dancer, Catmandu, Catmandu::Store

👑 AUTHOR 👑

Nicolas Steenlant, <nicolas.steenlant at ugent.be>

💜 CONTRIBUTORS 💜

Nicolas Franck, <nicolas.franck at ugent.be>

Vitali Peil, <vitali.peil at uni-bielefeld.de>

Patrick Hochstenbach, <patric.hochstenbach at ugent.be>

🫶 LICENSE 🫶

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

and that's the whole page bestie 🎀💖 this one's from perl v5.36.0, dated 2023-02-04 — iconic, obsessed, periodt ✨

📦 where this page comes from

debian bookworm: libdancer-plugin-catmandu-oai-perl 0.0508-3 · Artistic-1.0-Perl OR GPL-1.0-or-later
debian trixie: libdancer-plugin-catmandu-oai-perl 0.0508-3 · Artistic-1.0-Perl OR GPL-1.0-or-later
ubuntu noble: libdancer-plugin-catmandu-oai-perl 0.0508-3 · Artistic-1.0-Perl OR GPL-1.0-or-later
ubuntu resolute: libdancer-plugin-catmandu-oai-perl 0.0508-3 · Artistic-1.0-Perl OR GPL-1.0-or-later
ubuntu stonking: libdancer-plugin-catmandu-oai-perl 0.0508-3 · Artistic-1.0-Perl OR GPL-1.0-or-later
💖 girlified by deepseek-ai/DeepSeek-V4.1-Flash on 2026-10-14: a language model's rewrite of the original, under its terms