RFC 2016
Uniform Resource Agents (URAs)
RFC 2016: Uniform Resource Agents (URAs) 🎀
Okay bestie, this is RFC 2016, and it's giving "experimental architecture" realness ✨
Part 1 of 4 🌸
- Title: Uniform Resource Agents (URAs)
- Authors: L. Daigle, P. Deutsch, B. Heelan, C. Alpaugh and M. Maclachlan, all from Bunyip Information Systems, Inc. 💜
- Stream: Network Working Group 🧚
- Category: Experimental 🧪
- Date: October 1996 📅
- Obsoletes / Updates: the header doesn't list any RFCs that this one obsoletes or updates 🙈
Status of this Memo 📜
Okay so, this memo defines an Experimental Protocol for the Internet community. It is not an Internet standard of any kind, so don't treat it like one 💅 They're really asking people to talk about it and share suggestions for making it better, and distribution of this memo is unlimited 🦋
Abstract ✨
This paper presents an experimental architecture for an agent system that gives you sophisticated access to Internet information and lets you manage it. These agents are NOT meant to be a general-purpose setup for active objects that roam around the Internet 🌍 Instead, they're modeled as extensions of the Internet information infrastructure that already exists. The whole experiment is focused on the information structures needed to wrap up Internet activities into objects. Those objects can then be activated, transformed, and combined into bigger structured activities. It's giving LEGO for the internet 🧱
Acknowledgements 🫶
Lots of people helped shape the thinking behind this work over the past few years, and the authors want to shout them out. Special thanks go to Chris Weider, Patrik Faltstrom, Michael Mealling, Alan Emtage, and the folks in the IETF URI Working Group, who had lots of thought-provoking discussions with them 💖
Sima Newell gave insightful comments on the document, and the authors say it's way more readable because of her. Iconic behaviour 👑
Introduction 🌈
This document lays out an experimental agent system architecture. It was designed to handle high-level Internet activities by wrapping up protocol-specific actions into encapsulated pieces ✨ It was first presented to the Uniform Resource Identifier (URI) working group at the IETF, where it was seen as a step beyond resource location and resource naming. The idea was to give a structured way to describe the characteristics of the information you want, and to keep the necessary access incantations away from the client. That's how the notion of a Uniform Resource Agent (URA) came to be 🧚
Internet information systems have evolved by stacking one layer of encapsulated technology on top of another, and that's honestly the best part about them 🍓 First, machine address numbers were invented, and then they got wrapped in advertised machine names. That's what let the Domain Name System (DNS) [RFC1034, RFC1035] happen. Then protocols were built for accessing all kinds of Internet resources, and uniform ways of specifying where those resources live, standardized across protocol types, came along too (URLs) [RFC1738]. Every layer of Internet information primitives became the building blocks for the next level of abstraction and sophistication in information access, location, discovery and management. Layers all the way down 💅
The work in this paper is an experimental system that takes yet another step in encapsulation 🦋 TCP/IP protocols for routing, addressing, and so on have made it possible to connect to and access a whole plethora of information services on the Internet 🌸 Even so, those services still count as a diverse collection of heterogeneous resources. The World Wide Web has been the most successful effort so far at knitting them into one cohesive whole. The catch is that the activity it supports best is (human) browsing of those resources as documents. The URA initiative asks whether you could specify an activity with the same kind of precision that we give to resource naming and identification. By focusing on activities instead of actions, URAs wrap up resource access mechanisms based on commonality of information content, not protocol similarity. Not me being impressed by this idea 😭
An invoker, whether human or otherwise, can hand an entire set of tasks off to a fully-instantiated URA. The agent completely specifies the nature of those tasks, because it holds knowledge about the relevant Internet resources and the information needed to access them. That way URAs shield invokers from the details of Internet protocols, while still letting them carry out high-level Internet activities. One example is searching a set of web pages and news groups relevant to a given topic 🔥 Also, once you formally specify a high-level Internet activity in an agent, the same activity can be repeated at a later date by the same invoker, by someone else, or even by another agent. And the agent object can easily be modified to carry out another related task. Main character energy for sure 👑
More detail on the underlying philosophy of this approach can be found in [IIAW95] 📚
Examples 🍓
Let's do a super simple example: a client who wants to subscribe to a mailing list. There are lots of mechanisms for giving users the information they need to finish a subscription, so this is a very real problem. Right now, every application that can subscribe to mailing lists has to include protocol-aware code to do the job, once it has collected the required personal data from the user 🙈 On top of that, any application that bakes in subscribing as part of its code automatically limits which mailing lists a client can join (i.e, only the types its creators foresaw) 💀 But if there's an agent you can hand this task to instead, then every application can use that agent, and the agent takes on the job of carrying out all the interactions needed to finish the subscription 🎀 The agent can even be a client to other agents that supply particular information about how to subscribe to new types of mail servers, and so on. URAs have been explored as an agent technology to tackle exactly these kinds of issues ✨
Relationship to Other Internet Agents 🦋
A bunch of Internet-aware agent and transportable code systems have gotten popular, like Java [JAVA], TCL [TCL] and Safe-TCL, Telescript [TELE], and the TACOMA system [TACOMA], just to name a few 💜 To really see the scope of the problem URAs are going after, it helps to know how these systems are different from the URA approach. Some of them, like Java, focus on providing mechanisms for creating and distributing (inter)active documents on the World Wide Web. Others, like TACOMA, have broader goals of providing environments for mobile, interacting processes 🌈
Each of these systems makes its own contribution to solving the transportation, communication, and security issues that usually come with agent systems. But they produce more objects that exist within the Internet information space. So while they might let individual users have a more sophisticated interaction with a particular information resource, they don't tackle the bigger Internet problems: naming, identifying, locating resources, and locating the same or similar resources again at a later date. That set of problems is exactly what URAs set out to solve 🧚
To create these URA objects that wrap up a set of Internet activities, you have to specify their operating environment and their design structure. Together, those form an experimental architecture for URAs, which can be evaluated in a preliminary way through a prototype implementation. The rest of this paper describes that experimental architecture, and outlines a prototype application built to test the concepts involved in creating and running URAs 🌸
The Experimental Architecture 👑
The main goal behind the URA architecture was to separate client need descriptions from the specifications of the mechanisms that satisfy those needs 💖 For example, from the client's point of view, the need to find MIDI music files is totally distinct from the particular Internet resource actions that might be needed to find them at a given point in time. One need might be best met by pulling information together from several very different sources 🔥 And the client might have that same need on a different day, but there could be new or different resources to call on to satisfy it. Fresh options, same vibe ✨
Another goal was to give very structured specifications of the Internet actions carried out by a particular URA 💅 When the structure of an action is explicit, you can operate on portions of an agent structure without having to understand the complete semantics of its activity. Big brain move 🧠
At the centre of the URA architecture is the concept of a (persistent) specification of an activity 🦋 To support this concept, and for reasons that will become clearer as the expected usage of URAs is described in more detail, the architecture comes with these requirements:
- there is a formalized environment in which these specifications are examined, executed and otherwise manipulated. This is referred to as a URAgency. 🏠
- the activity specifications are modular and independent of any given URAgency environment. So they exist as object constructs that can be shared among URAgencies. There is a standardized virtual structure for these URA objects, although different types may exist, with different underlying implementations 🌈
Basic URAgency Requirements ✨
Okay so bestie, in the most abstract sense, a URAgency is a software system that manipulates URA objects 💅. In object terms, a URAgency knows which types of URAs it handles and is responsible for applying methods to objects of those types. For this experimental work, the only methods it is required to support are the ones that get information about a given URA, and the ones that execute a URA 🎀.
The expected result of applying the "get information" method to a URA is a description of some or all of that URA, following the standardized virtual structure of a URA object (more on that below) 🦋.
To "execute" a URA, you either supply information for the individual URA data segments (basically you're creating an instance of a virtual object), or you identify a URA instance that already exists 💖. Again, the info has to follow the virtual structure below.
A URAgency claiming to handle a particular type of URA must be able to map that type's implementation structure into and out of the standard virtual URA structure. It must also know how to activate the URA, and it must satisfy any runtime dependencies for that type of URA 🫶.
For example, a URA type might be a Pascal program binary that, when run with particular command line arguments, spits out information in the standard URA object structure 🧚. Activating this type of URA might mean running the Pascal binary with an input file containing all the necessary data segments. A URAgency claiming to handle this sort of URA type must first be able to provide an environment to run the Pascal binary (for whatever platform it was compiled for), and also be able to interact with the Pascal binary according to these conventions, to get information about the URA or to execute it 💅.
As an alternative, a URA type might be a script in some interpreted language, with the URA object structure embedded as data structures inside the script 🌸. A URAgency handling this type might have to parse the script to pull out the standard URA object structure, and provide the script language interpreter so the URA can be executed 🫶.
URA Object Structure 🌈
In order to capture the info needed for the kind of Internet activity described in the introductory paragraphs of this document, six basic (virtual) components of a URA object have been identified 💜. Any implementation of a URA type is expected to be able to conform to this structure within the context of a URAgency. The six basic components are:
URA HEADER: Identification of the URA object, including a URA name, type and abstract, creator name, and the resources required by the URA.ACTIVATION DATA: Specification of the data elements required to carry out the URA activity. For example, for an Internet search for "people", this could include fields for person name, organization, and e-mail address.TARGETS: Specification of the URL/URN's to be accessed to carry out the activity. Until URN's are in common use, being able to adjust URLs will be necessary. A key issue for URAs is being able to transport them and activate them far from the creator's originating site, and this may have implications for how accessible the resource sites are. For example, a software search created in Canada will likely access a Canadian Archie server and North American ftp sites, but an invoker in Australia should not have to edit the URA object to make it relevant in Australia. So the creator can use this section to specify the expected type of service, with variables for the parts that can be modified in context (e.g., the host name for an Archie server, or a mirror ftp site).EXPERIENCE INFORMATION: Specification of data elements that are not strictly involved in conversing with the targets in order to carry out the agent's activity. This space can be used to store information from one invocation of a URA instance to the next. That could include the date of last execution, or URLs of resources located on a previous invocation of the agent.ACTIVITY: If URAs were strictly data objects, specifying the required data and URL/URN's would be enough to capture the essence of the composite net interaction. However, the variability of Internet resource accesses, and the scope of what URAs could accomplish in the net environment, seem to suggest the creator needs some means of organizing the instantiation of the component URL/URN's. So the body of the URA should contain a scripting mechanism that minimally allows conditional instantiation of individual URL/URN's. These conditions could be based on which (content) data elements the user provided, or on the accessibility of one URL/URN, etc. It also provides a mechanism for suggesting scheduling of URL/URN instantiation.RESPONSE FILTER: The main purpose of the ACTIVITY module is to specify the steps needed to take the ACTIVATION DATA, contact the TARGETS, and collect responses from those services. The purpose of the RESPONSE FILTER module is to transform those responses into the result of the URA invocation. That transformation may be along the lines of reformatting some text, or it may be a more elaborate interpretation, such as a relevance rating for a retrieved HTML page.
The ACTIVITY is specified by a script or program in a language specified by the URA type, or by the URA header information. All the required activation data, targets, and experience information are referenced by their specification names.
The RESPONSE FILTER is specified by a script or program in a language specified by the URA type, or by the URA header information. All the required activation data, targets, and experience information are referenced by their specification names.
See Appendix 1 for a more detailed description of the components of a URA 📚. Appendix 2 contains a sample virtual URA structure ✨.
The Architecture in Action 🦋
Having introduced the required capabilities of the URAgency and the virtual structure of URA objects, it's now time to dig into the tasks and interactions that URAs are best at 💅.
URAs are constructed by identifying net-based resources of interest (targets) to carry out a particular task. The activation data component of a URA is the author's mechanism for telling the invoker which elements of information are required for successful execution. An invoker creates an instance of a URA object by providing data that is consistent with, or fills in, this template. Such an instance encapsulates everything that the agent "needs to know" in order to contact the specified target(s), make a request of the resource ("get", "search", etc.), and return a result to the invoker. This encapsulation is a sophisticated identification of the task results 🌟.
For example, in the case of a mailing list subscription URA, the creator identifies the target URL for a resource that handles list subscription (e.g., an HTML form), and specifies the data required by that resource (such as user name, user mail address, and mailing list identifier). When an invoker provides that information and instantiates the URA, the resulting object completely encapsulates all that is needed to subscribe the user -- the subscription result is identified 📬.
URAs are manipulated through the application of methods. This, in turn, is governed by the URAgency with which the invoker is interacting. However, because the virtual structure of URAs is represented consistently across URA types and URAgencies, a URAgency can act as one of the targets of a URA. Since methods can be applied to URAs remotely, URAs can act as invokers of URAs. This can yield a complex structure of task modules 🌈.
For example, a URA designed to carry out a generalized search of book-selling resources might make use of individual URAs tailored to each resource. Thus, the top-level URA becomes the orchestrating URA for access to a number of disparate resources, while being insulated from the minute details of accessing those resources 👑.
A Prototype Implementation 🧁
The experimental work with URAs includes a prototype implementation of URA objects. These are written in the Tcl scripting language. A sample prototype Tcl URA can be found in Appendix 3 🧚.
The URAgency that was created to handle these URAs is part of the Silk Desktop Internet Resource Discovery tool. Silk provides a graphical user interface environment that allows the user to access and search for Internet information without having to know where to look or how to look 🔥. Silk presents a list of the available URAs to carry out these activities (e.g., "search for tech reports" or "hotlist"). For each activity, the user is prompted for the activation data, and Silk's URAgency executes the URA. The Silk software also supports the creation and maintenance of URA object instances. Users can add new URAs by creating new Tcl scripts (per the guidelines in the "URA Writer's Guide", available with the Silk software. See [SILK]). The Silk graphical interface hides some of the mechanics of the underlying URAgency. A more directly-accessible version of this URAgency will become available 💖.
Conclusions ✨
okay so this whole thing started as an extension to the URI family, bestie 🦋 the URIs we already have are Uniform Resource Locators (URLs), Uniform Resource Characteristics (URCs), and the proposed Uniform Resource Names (URNs). the idea is that if you formalize what an information task is actually about in one standardized object structure, you can identify a whole class of resources at once. that bumps up the level of abstraction people can talk about Internet resources at, and it's giving main character energy 💅
we're still experimental, not gonna lie, but it's already got people interested 🌸 it's shown real promise for building smarter tools that interact with the Internet on a much more sophisticated level than just clicking around web pages.
the biggest headache we've hit while building a collection of URAs is brittleness 😭 services are mostly built for human eyes, so a tiny change in output that a person would totally spot can wreck a software client that relies on cues in the text to parse and interpret results. and honestly this isn't a URA problem only. any software acting on results from a service like that gets hit the same way. maybe what we need is an evolution of "service entrances" to information servers: mechanisms for getting "just the facts" straight from the server 🧚 one way to get there is for the service provider to build and distribute a URA that talks to their service. then, when the interface changes, the provider will be moved to update the URA so it keeps accessing the service reliably 🫶
more types of URAs are coming, plus other URAgencies too 🌈 that means we'll need URAgency interaction standards. the "common virtual URA object structure" is the first step toward a lingua franca for URAs of totally different types and intentions. it's the start of URAs all speaking the same language, periodt 💖
References 📚
[IIAW95] Leslie L. Daigle, Peter Deutsch, "Agents for Internet Information Clients", CIKM'95 Intelligent Information Agents Workshop, December 1995. Available from <http://www.bunyip.com/products/silk/silktree/uratree/iiaw95.ps>
[JAVA] "The Java Language: A White Paper" Available from <http://java.sun.com/1.0alpha2/doc/overview/java/index.html>
[RFC1034] Mockapetris, P., "Domain Names - Concepts and Facilities", STD 13, RFC 1034, November 1987.
[RFC1035] Mockapetris, P., "Domain Names - Implementation and Specification", STD 13, RFC 1035, November 1987.
[RFC1738] T. Berners-Lee, L. Masinter, M. McCahill, "Uniform Resource Locators (URL)", RFC 1738, December 1994.
[SILK] Bunyip's Silk project homepage: <http://www.bunyip.com/products/silk/>
[SILKURA] Silk URA information: <http://www.bunyip.com/products/silk/silktree/uraintro.html>
[TACOMA] Johansen, D. van Renesse, R. Schneider, F. B., "An Introduction to the TACOMA Distributed System", Technical Report 95-23, Department of Computer Science, University of Tromso, Norway, June 1995.
[TCL] Ousterhout, J. K. "Tcl and the Tk Toolkit", Addison Wesley, 1994.
[TELE] White, J. E., "Telescript Technology: The Foundation for the Electronic Marketplace", General Magic White Paper, General Magic Inc., 1994.
Authors' Addresses 🎀
the crew behind this one, with their contact details:
Leslie Daigle
Peter Deutsch
Bill Heelan
Chris Alpaugh
Mary Maclachlan
Bunyip Information Systems, Inc.
310 St. Catherine St. West
Suite 300
Montreal, Quebec, CANADA
H2X 2A1
Phone: (514) 875-8611
EMail: ura-bunyip@bunyip.comAppendix 1 -- Virtual URA Structure 💜
this appendix is the BNF-style description of what a URA object is expected to look like in its virtual structure. think of it as the canonical representation of all the info packed into a given URA 🧠 more info is allowed to show up inside the elements of the components, but the elements listed here are the "minimum" or "standard" set. so this is the baseline everyone can count on ✨
here's the notation key, copied exactly so you can read the grammar without any guessing:
N.B.:
[]-delimited items are optional
%% denotes a comment
\0 represents the empty string
| is "or"
{} are literal charactersthis form is used for convenience and clarity of expression. whitespace and ordering of individual elements are not considered significant, so don't stress about the layout 🍓
<VIRTUAL_URA> := {<virtual-ura-structure>}
<virtual-ura-structure> := { URAHDR <ura-header> }
{ ACTDATA <activation-data> }
{ TARG <targets> }
{ EXPINFO <experience information> }
{ ACTSPEC <activity> }
{ RESPFILT <response filter> }
<ura-header> := { name <ura-name> }
{ author <ura-author> }
{ version <ura-version> }
[ { lang <lang-dependencies> } ]
[ { parent <parent-of-instance> } ]
<activation-data> := <act-data-element><activation-data> | \0
<act-data-element> := {
{ name <data-elt-name> }
{ response <data-elt-value> }
{ prompt <data-elt-prompt> }
[ { required <boolean> } ]
[ { default <data-default-val> } ]
}
<targets> := <target-service><targets> | \0
<target-service> := {
{ name <targ-url> }
{ protocol <url-protocol> }
{ url <url-spec> }
[ { <url-type-specific-data> } ]
}
<url-spec> := <complete-url> | <url-constructor>
<complete-url> := %% a complete, valid URL string
(e.g., http://www.bunyip.com/)
<url-constructor> := {
{ scheme <url-scheme-spec> }
{ host <url-host-spec> }
[ { port <url-port-spec> } ]
{ selector <url-selector-spec> }
}
<url-scheme-spec> := {
{ name <scheme-name> }
{ response <scheme-value> }
{ prompt <scheme-prompt> }
}
<url-host-spec> := {
{ name <host-name> }
{ response <host-value> }
{ prompt <host-prompt> }
}
<url-port-spec> := {
{ name <port-name> }
{ response <port-value> }
{ prompt <port-prompt> }
}
<url-selector-spec> := {
{ name <selector-name> }
{ response <selector-value> }
{ prompt <selector-prompt> }
}
<experience information> := {
{ name <data-elt-name> }
{ response <data-elt-value> }
}
<activity> := <compound-string>
<response filter> := <compound-string>
%% Without requiring more detail...
<compound-string> := <string>\n<compound-string> | \0
<boolean> := 0 | 1
<ura-name> := <string>
<ura-author> := <string>
<ura-version> := <string>
<lang-dependencies> := <string>
<parent-of-instance> := <string>
<data-elt-name> := <string>
<data-elt-value> := <string>
<data-elt-prompt> := <string>
<data-elt-default> := <string>
<data-default-val> := <string>
<targ-url> := <string>
<url-protocol> := http-get | http-post | ...
<url-type-specific-data> := <string>
<scheme-name> := <string>
<scheme-value> := <string>
<scheme-prompt> := <string>
<host-name> := <string>
<host-value> := <string>
<host-prompt> := <string>
<port-name> := <string>
<port-value> := <string>
<port-prompt> := <string>
<url-selector-name> := <string>
<url-selector-value> := <string>
<url-selector-prompt> := <string>Appendix 2 -- Sample Virtual URA Representation 🦋
here's a valid virtual representation of a Silk Tcl URA. the real URA it came from is in Appendix 3, so you can compare them side by side 👑
{
{URAHDR
{name {DejaNews Search}}
{author {Leslie Daigle}}
{version {1.0}}
}
{ACTDATA
{name {Topic Keywords}}
{prompt {Topic Keywords}}
{response {}}
}
{EXPINFO
{name {Comments}}
{prompt {Comments}}
{response {}}
}
{ACTSPEC
{proc mapResponsesToDejanews {} {
set resp ""
if {[uraAreResponsesSet {Topic Keywords}]} {
lappend resp [list query [uraGetSpecResponse {
Topic Keywords}]]
}
return $resp
}
proc uraRun {} {
global errorInfo
foreach serv [uraListOfServices] {
set u [uraGetServiceURL $serv]
switch -- $serv {
dejanews {
if [catch {
set query [mapResponsesToDejanews]
if {$query != {}} {
set result [uraHTTPPostSearch $u $query]
if {$result != ""} {
set list [dejanews_uraHTTPPostCanonicalize
$result]
puts $list
}
}
}] {
puts stderr $errorInfo
}
}
default {
# can't handle other searches, yet.
} } } }
}
}
{RESPFILT
{
proc dejanews_uraHTTPPostCanonicalize {htmlRes} {
set result {}
set lines {}
set clause {}
set garb1 ""
set garb2 ""
# Get the body of the result page -- throw away leading and
# trailing URLs
regexp {([^<PRE>]*)<PRE>(.*)</PRE>.*}
$htmlRes garb1 garb2 mainres
set lines [split $mainres "\n"]
foreach clause $lines {
if [regexp
{<DT>.*(..\/..).*<A HREF="([^"]*)">([^<]*)</A>.*<B>([^<]*).*}
$clause garb1 dt relurl desc grp] {
lappend r [list HEADLINE [format "%s (%s, %s)"
[string trim $desc] \
[string trim $grp] $dt]]
lappend r [list URL [format
"http://www.dejanews.com/cgi-bin/%s" $relurl]]
lappend r [list TYPE "text/plain"]
lappend result $r
}
}
return $result
}
}
}
}Appendix 3 -- Sample Silk Tcl URA 🌸
this is a valid Silk Tcl URA. if you want the nitty-gritty on how Silk-specific URAs are implemented and structured, check out the "URA Writers Guide" that comes with the Silk software distribution (available from <http://www.bunyip.com/products/silk>) 🧚
# ----------------------------------------------------------------------
#
# URA initialization
#
# ----------------------------------------------------------------------
#
# Initialize the URA, its search specs and searchable services.
#
# URA init.
set uraDebug 1
uraInit {
{name {DejaNews Search}}
{author {Leslie Daigle}}
{version {1.0}}
{description "This URA will search for UseNet News articles."}
{help "This is help on UseNet News search script."}
}
#
# bug: handling of choices/labels is kind of gross.
#
# Search spec. init.
foreach item {
{
{name {Topic Keywords}}
{field Topic}
{tag STRING}
{description {Keywords to search for in news articles}}
{prompt {Topic Keywords}}
{help {Symbols to look up, separated by spaces.}}
{type STRING}
{subtype {}}
{allowed .*}
{numvals 1}
{required 0}
{response {}}
{respset 0}
}
} {
uraSearchSpecInit $item
}okay bestie, we're picking up right where we left off 💅✨ this part is all code, so the vibe is just "read the source and vibe with it" 🦋
uraAnnotationInit {
{help {Enter comments to store with an instance}}
{numvals 1}
{subtype {}}
{response {}}
{name Comments}
{required 0}
{class ANNOTATION}
{type TEXT}
{description {General comments about this URA.}}
{respset 1}
{prompt Comments}
{field {}}
{allowed .*}
}
uraResultInit {
{name {Related Pages}}
{contents { {
{HEADLINE {The DejaNews UseNet search service}}
{TYPE text/plain}
{URL http://www.dejanews.com}
} }}
}
foreach item {
{
{name dejanews}
{protocol http-post}
{url http://marge.dejanews.com/cgi-bin/nph-dnquery}
}
} {
uraServicesInit $item
}the annotation block is where you stash comments about the URA itself 🎀 and the result block is the "related pages" link to DejaNews 🌸 the foreach at the bottom registers the dejanews service with its HTTP POST protocol and URL 💜
okay now the canonicalizer 🫶 it grabs the body of the result page, splits it into lines, and pulls out the headline, URL and type for each match 😭
proc dejanews_uraHTTPPostCanonicalize {htmlRes} {
set result {}
set lines {}
set clause {}
set garb1 ""
set garb2 ""
# Get the body of the result page
# -- throw away leading and trailing URLs
regexp {([^<PRE>]*)<PRE>(.*)</PRE>.*} $htmlRes garb1 garb2 mainres
set lines [split $mainres "\n"]
foreach clause $lines {
uraDebugPuts stderr [format "Line: %s" $clause]
if [regexp
{<DT>.*(..\/..).*<A HREF="([^"]*)">([^<]*)</A>.*<B>([^<]*).*} \
$clause garb1 dt relurl desc grp] {
uraDebugPuts stderr [format
"Date: %s Rel URL: %s Desc: %s Group: %s"
$dt $relurl $desc $grp]
lappend r [list HEADLINE [format "%s (%s, %s)"
[string trim $desc] \
[string trim $grp] $dt]]
lappend r [list URL [format
"http://www.dejanews.com/cgi-bin/%s" $relurl]]
lappend r [list TYPE "text/plain"]
lappend result $r
}
}
return $result
}----------------------------------------------------------------------
#
Mapping procedures
#
----------------------------------------------------------------------
Next up is the mapping section 🦋 each searchable service gets one of these to turn the search spec responses into something a search URL (or whatever the query procedure takes) can use 💖
#
# There is one procedure, for each searchable service, to map the search
# spec responses to a form suitable for inclusion into a search URL (or
# whatever form the particular query procedure accepts).
#
#
#
proc mapResponsesToDejanews {} {
set resp ""
if {[uraAreResponsesSet {Topic Keywords}]} {
lappend resp [list query [uraGetSpecResponse {Topic Keywords}]]
}
return $resp
}And finally the runner 🔥 it loops over the services, builds the query, posts it, and canonicalizes the answer. There's a little bug note in here too, so keep an eye out 👀
#
# bug: need better error reporting
# (i.e. which searches didn't work and why, etc.)
#
proc uraRun {} {
global errorInfo
foreach serv [uraListOfServices] {
set u [uraGetServiceURL $serv]
switch -- $serv {
dejanews {
if [catch {
set query [mapResponsesToDejanews]
uraDebugPuts stderr [format "%s: query is `%s'."
$serv $query]
if {$query != {}} {
set result [uraHTTPPostSearch $u $query]
if {$result != ""} {
uraDebugPuts stderr [format "%s: result is `%s'."
$serv $result]
set list [dejanews_uraHTTPPostCanonicalize $result]
uraDebugPuts stderr [format "%s: list is `%s'."
$serv $list]
puts $list
}
}
}] {
puts stderr $errorInfo
}
}
default {
# can't handle other searches, yet.
}
}
}
}and that's the whole document, bestie 🎉🌈 every line is there, every quirk too 🫶
📦 where this page comes from
- ietf: rfc2016 1996-10
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms