RFC 3195

Reliable Delivery for syslog

📅 published November 2001, from ietf

RFC 3195: Reliable Delivery for syslog 🎀

Okay bestie, let's talk about RFC 3195, which is titled "Reliable Delivery for syslog" 💖. It's a Standards Track document, so it's a real protocol spec for the Internet community ✨. It was written by D. New and M. Rose from Dover Beach Consulting, Inc., and it's dated November 2001 📅. It's from the Network Working Group, so yeah, it's legit 💅.

Status of This Memo 🌸

This document specifies an Internet standards track protocol for the Internet community, and it requests discussion and suggestions for improvements. For the standardization state and status of this protocol, check the current edition of the "Internet Official Protocol Standards" (STD 1). Distribution of this memo is unlimited 🫶.

Copyright (C) The Internet Society (2001). All Rights Reserved. 💜

Abstract ✨

Okay so the BSD Syslog Protocol describes a bunch of service options for passing along event messages 📨. This memo describes two mappings of the syslog protocol onto TCP connections, and both are useful for reliable delivery of event messages 🦋. The first is a trivial mapping that maximizes backward compatibility 🍓. The second is a more complete mapping 💅.

Both give you a level of robustness and security in message delivery that the usual UDP-based syslog protocol just doesn't have. They do that by providing encryption and authentication over a connection-oriented protocol 🔐. It's giving "reliable AND secure" 😭🔥.

Table of Contents 🧚

   1.    Introduction . . . . . . . . . . . . . . . . . . . . . . . .  3
   2.    The Model  . . . . . . . . . . . . . . . . . . . . . . . . .  4
   3.    The RAW Profile  . . . . . . . . . . . . . . . . . . . . . .  7
   3.1   RAW Profile Overview . . . . . . . . . . . . . . . . . . . .  7
   3.2   RAW Profile Identification and Initialization  . . . . . . .  9
   3.3   RAW Profile Message Syntax . . . . . . . . . . . . . . . . . 10
   3.4   RAW Profile Message Semantics  . . . . . . . . . . . . . . . 10
   4.    The COOKED Profile . . . . . . . . . . . . . . . . . . . . . 11
   4.1   COOKED Profile Overview  . . . . . . . . . . . . . . . . . . 11
   4.2   COOKED Profile Identification and Initialization . . . . . . 11
   4.3   COOKED Profile Message Syntax  . . . . . . . . . . . . . . . 11
   4.4   COOKED Profile Message Semantics . . . . . . . . . . . . . . 12
   4.4.1 The IAM Element  . . . . . . . . . . . . . . . . . . . . . . 12
   4.4.2 The ENTRY Element  . . . . . . . . . . . . . . . . . . . . . 14
   4.4.3 The PATH Element . . . . . . . . . . . . . . . . . . . . . . 19
   5.    Additional Provisioning  . . . . . . . . . . . . . . . . . . 25
   5.1   Message Authenticity . . . . . . . . . . . . . . . . . . . . 25
   5.2   Message Replay . . . . . . . . . . . . . . . . . . . . . . . 25
   5.3   Message Integrity  . . . . . . . . . . . . . . . . . . . . . 25
   5.4   Message Observation  . . . . . . . . . . . . . . . . . . . . 26
   5.5   Summary of Recommended Practices . . . . . . . . . . . . . . 26
   6.    Initial Registrations  . . . . . . . . . . . . . . . . . . . 27
   6.1   Registration: The RAW Profile  . . . . . . . . . . . . . . . 27
   6.2   Registration: The COOKED Profile . . . . . . . . . . . . . . 27
   7.    The syslog DTD . . . . . . . . . . . . . . . . . . . . . . . 28
   8.    Reply Codes  . . . . . . . . . . . . . . . . . . . . . . . . 32
   9.    IANA Considerations  . . . . . . . . . . . . . . . . . . . . 33
   9.1   Registration: BEEP Profiles  . . . . . . . . . . . . . . . . 33
   9.2   Registration: The System (Well-Known) TCP port number for
            syslog-conn . . . . . . . . . . . . . . . . . . . . . . . 33
   10.   Security Considerations  . . . . . . . . . . . . . . . . . . 34
   11.   Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 34
   12.   References . . . . . . . . . . . . . . . . . . . . . . . . . 34
   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 35
   Full Copyright Statement . . . . . . . . . . . . . . . . . . . . . 36

1. Introduction ✨

So the syslog protocol [1] gives you a whole spectrum of service options for setting up an event-based logging service over a network 🌸. Every option comes with its own benefits and costs 💅. That means picking which combination to turn on is both an engineering decision and an administrative one 🧠.

This memo explains how to make the syslog protocol work when reliable delivery is selected as a required service 🦋. Arguing for or against using reliable delivery for syslog is out of scope here, so we're not doing that drama 🙈.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [2]. These are the official BCP 14 key words, so read them carefully bestie 📖.

2. The Model 🌈

The syslog service supports three roles of operation: device, relay, and collector 🎀.

Devices and collectors act as sources and sinks, respectively, of syslog entries. In the simplest case, only a device and a collector are present. E.g.,

  +--------+        +-----------+
  | Device | -----> | Collector |
  +--------+        +-----------+

The relationship between devices and collectors is potentially many-to-many. I.e., a device might communicate with many collectors; similarly, a collector might communicate with many devices 💖.

A relay operates in both modes, accepting syslog entries from devices and other relays and forwarding those entries to collectors and other relays 🔁.

For example,

  +--------+      +-------+        +-------+      +-----------+
  | Device | ---> | Relay | -...-> | Relay | ---> | Collector |
  +--------+      +-------+        +-------+      +-----------+

As shown, more than one relay may be present between any particular device and collector 🧚.

A relay may be necessary for administrative reasons. For example, a relay might run as an application proxy on a firewall 🛡️. Also, there might be one relay per company department, which authenticates all the devices in the department, and which in turn authenticates itself to a company-wide collector 👑.

A relay can also serve to filter messages 🍓. For example, one relay may collect the syslog information from an entire web server farm, summarizing hit counts for report generation, forwarding "page not found" messages (indicating a possible broken link) to a collector that presents it to the webmaster, and sending more urgent messages (such as hardware failure reports) to a collector that gateways them to a pager 😭. A relay may also be used to convert formats from a device's output to a collector's input ✨.

It should be noted that a role of device, relay, or collector is relevant only to a particular BEEP channel (q.v., below). A single server can serve as a device, a relay, and a collector, all at once, if so configured 🤯. It can even serve as a relay and a collector to the same device at the same time using different BEEP channels over the same connection-oriented session. This might be useful to collect status yet relay urgent error messages 💅.

To provide reliable delivery when realizing the syslog protocol, this memo defines two BEEP profiles. BEEP [3] is a generic application protocol framework for connection-oriented, asynchronous interactions. Within BEEP, features such as authentication, privacy, and reliability through retransmission are provided 🔐. There are two profiles defined in this memo:

  • The RAW profile is designed to provide a high-performance, low-impact footprint, using essentially the same format as the existing UDP-based syslog service 🏎️.
  • The COOKED profile is designed to provide a structured entry format, in which individual entries are acknowledged (either positively or negatively) 🍪.

Note that both profiles run over BEEP. BEEP defines "transport mappings," specifying how BEEP messages are carried over the underlying transport technologies. At the time of this writing, only one such transport is defined, in [4], which specifies BEEP over TCP 🦋. All transport mappings are required to support enough reliability and sequencing to allow all BEEP messages on a given channel to be delivered reliably and in order. Hence, both the RAW and COOKED profiles provide reliable delivery of their messages 💖.

The choice of profile is independent of the operational roles discussed above 🌸.

For example, in

  +--------+        +-------+        +-----------+
  | Device | -----> | Relay | -----> | Collector |
  +--------+        +-------+        +-----------+

the device-to-relay link could be configured to use the RAW profile, while the relay-to-collector link could be configured to use the COOKED profile. (For example, the relay may be parsing the RAW syslog messages from the device, knowing the details of their formats, before passing them to a more generic collector.) Indeed, the same device may use different profiles, depending on the collector to which it is sending entries 🫶.

Devices and relays MAY discover relays and collectors via the DNS SRV algorithm [5]. If so configured, the service used is "syslog" and the protocol used is "tcp". This allows for central administration of addressing, fallback for failed relays and collectors, and static load balancing 🗺️. Security policies and hardware configurations may be such that device configuration is more secure than the DNS server 🔒. Hardware devices may be of such limited resources that DNS SRV access is inappropriate 🤏. Firewalls and other restrictive routing mechanisms may need to be dealt with before a reliable syslog connection can be established 🚧. In these cases, DNS might not be the most appropriate configuration mechanism 🙅‍♀️.

3. The RAW Profile 🦋

3.1 RAW Profile Overview ✨

okay so the RAW profile is built for minimal implementation effort, high efficiency, and backwards compatibility 🧚 it's giving "keep it simple," and it's especially the move when legacy syslog processing is happening 💅

ngl this part is lowkey sneaky. Even though the RAW profile uses the same message payload format as the UDP version of syslog, delivery is reliable 😭 The RAW syslog profile is a profile of BEEP [3], and BEEP guarantees ordered reliable delivery of messages within each individual channel. So no dropped messages, bestie 🫶

When the profile starts, there's no piggyback data. Every BEEP message in the RAW profile is specified with a MIME Content-Type [6] of application/octet-stream. Once the channel is open, the listener (not the initiator) sends a MSG message saying it's ready to act as a syslog sink 🎀 (If you want the full tea on the roles a BEEP peer can play, check out Section 2.1 of [3], which defines "listener", "initiator", "client", and "server".)

The initiator then uses ANS replies to supply one or more syslog entries in the current UDP format, as specified in Section 3 of [1]. When it has nothing left to send, it finishes with a NUL reply and closes the channel 🦋

Here's an example, so you can see the whole vibe:

   L: <wait for incoming connection>
   I: <establish connection>
   L: RPY 0 0 . 0 201
   L: Content-type: application/beep+xml
   L:
   L: <greeting>
   L:   <profile
   L:     uri='http://xml.resource.org/profiles/syslog/COOKED' />
   L:   <profile uri='http://xml.resource.org/profiles/syslog/RAW' />
   L: </greeting>
   L: END
   I: RPY 0 0 . 0 52
   I: Content-type: application/beep+xml
   I:
   I: <greeting />
   I: END
   I: MSG 0 1 . 52 133
   I: Content-type: application/beep+xml
   I:
   I: <start number='1'>
   I:   <profile uri='http://xml.resource.org/profiles/syslog/RAW' />
   I: </start>
   I: END
   L: RPY 0 1 . 201 100
   L: Content-type: application/beep+xml
   L:
   L: <profile uri='http://xml.resource.org/profiles/syslog/RAW' />
   L: END
   L: MSG 1 0 . 0 50
   L:
   L: Central Services. This has not been a recording.
   L: END
   I: ANS 1 0 . 0 61 0
   I:
   I: <29>Oct 27 13:21:08 ductwork imxpd[141]: Heating emergency.END
   I: ANS 1 0 . 61 58 1
   I:
   I: <29>Oct 27 13:22:15 ductwork imxpd[141]: Contact Tuttle.END
   I: NUL 1 0 . 119 0
   I: END
   L: MSG 0 3 . 301 70
   L: Content-Type: application/beep+xml
   L:
   L: <close number='1' code='200' />
   L: END
   I: RPY 0 3 . 185 46
   I: Content-Type: application/beep+xml
   I:
   I: <ok />
   I: END
   I: MSG 0 4 . 231 72
   I: Content-Type: application/beep+xml
   I:
   I: <close number='0' code='200' />
   I: END
   L: RPY 0 4 . 371 46
   L: Content-type: application/beep+xml
   L:
   L: <ok />
   L: END
   L: <closes connection>
   I: <closes connection>
   L: <awaits next connection>

In this session, a BEEP connection gets set up and then the RAW profile kicks in 🌸 The initiator is a device, and the listener is a collector. The initiator opens the channel, but the listener sends the first MSG. That lets the initiator send as many ANS replies carrying syslog event messages as it wants. When it's done, it sends a NUL reply. Once the listener gets that NUL, it closes the RAW channel 💜 After that, the initiator can either close the whole BEEP session or open a new syslog channel (RAW or COOKED) for more transfers. In this example, it picks closing the entire BEEP session 👑

The overhead for one ANS frame is about thirty octets once the initial handshakes are done. If that overhead feels like too much, it probably means messages are being generated at a high rate 🔥 In that case, you can aggregate multiple syslog messages into one ANS frame, with each one separated from the previous by a CRLF sequence. The final message still MUST NOT end with a CRLF 🙈

For example:

   L: MSG 1 0 . 0 50
   L:
   L: Central Services. This has not been a recording.
   L: END
   I: ANS 1 0 . 0 119 0
   I:
   I: <29>Oct 27 13:21:08 ductwork imxpd[141]: Heating emergency.
   I: <29>Oct 27 13:21:09 ductwork imxpd[141]: Contact Tuttle.END
   I: NUL 1 0 . 119 0
   I: END

3.2 RAW Profile Identification and Initialization ✨

The RAW syslog profile is identified as

       http://xml.resource.org/profiles/syslog/RAW

in the BEEP profile element during channel creation 🎀

No data is piggybacked during channel creation 🫶

3.3 RAW Profile Message Syntax 💅

Every BEEP message in this profile has a MIME content-type of application/octet-stream. The listener's first BEEP message gets ignored, and it can honestly be empty apart from headers, so any syntax is fine there 🦋

The ANS replies that the initiator sends back MUST be formatted according to Section 4 of [1]. Specifically, if the receiver is acting as a relay, then it MUST follow the rules laid out in Section 4.2.2 of [1] 🔥

If multiple syslog messages are packed into a single ANS reply, each one gets separated from the one before it with a CRLF. There's no ending delimiter 😭 But here's the important part: each syslog event message body length MUST be 1024 bytes or less, not counting BEEP framing overhead. Also, there MUST NOT be a CRLF between the text of the final syslog event message and the "END" that marks the trailer of the BEEP frame 🙈

3.4 RAW Profile Message Semantics 🌈

The listener's opening BEEP MSG message has no semantics. (It's a good spot for an identifying greeting, though 🧚) The initiator's ANS replies MUST specify a facility, severity, and textual message, as described in [1] ✨

4. The COOKED Profile 👑

4.1 COOKED Profile Overview 💖

The COOKED profile is made for new implementations of syslog protocol handlers 🍓 It gives you way finer-grained information tagging, which means better automation when processing. It does come with more overhead to support all that, ngl 😅

The COOKED profile supports three elements of interest:

  • The iam element identifies the sender to the receiver, so each peer can name itself for the other. It also specifies the roles (device, relay, or collector) each one is taking on 🫶
  • The entry element provides a parsed version of the syslog entry, with the various fields of interest broken out 💜
  • The path element identifies a list of relays that a tagged collection of entry elements has passed through. It also includes a set of flags showing what security assurances were in effect throughout delivery 🔥

4.2 COOKED Profile Identification and Initialization 🦋

The COOKED syslog profile is identified as

    http://xml.resource.org/profiles/syslog/COOKED

in the BEEP profile element during channel creation 🎀

During channel creation, the matching profile element inside the BEEP start element may contain an iam element. If channel creation succeeds, then before sending the corresponding reply, the BEEP peer processes the iam element and includes the resulting response in the reply. That response will be either an ok element or an error element 👑 Which one comes back depends on the local provisioning of the recipient. Including an iam in the initial start element has exactly the same semantics as passing it as the first MSG message on the channel ✨

4.3 COOKED Profile Message Syntax 🌸

Every BEEP message in this profile has a MIME Content-Type [6] of application/beep+xml. The syntax of the individual elements is specified in Section 7 🧚

4.4 COOKED Profile Message Semantics 💅

Initiators issue two elements: iam and entry, each one using a MSG message. The listener issues ok in RPY messages and error in ERR messages. (For the definitions of the error and ok elements, see Section 2.3.1 of [3].) 💖

4.4.1 The IAM Element 🫶

The iam element identifies a device, relay, or collector at one end of the BEEP channel to the device, relay, or collector at the other end. The iam element includes the type of peer (device, relay, or collector), the fully qualified domain name of the peer, and an IP address of the peer. (The IP address chosen SHOULD be the IP address associated with the underlying transport protocol carrying the channel.) The character data of the element is free-form human-readable text, and it can be used to identify the peer further, like by describing the physical location of the machine 🌈

An iam element may be sent by the initiator of the channel at any time. The listener responds to an iam element with an ok (accepting it) or an error (rejecting it). The identity and role in effect are specified by the most recent iam that was answered with an ok 🔥

An iam could be rejected (with an error element) by the listener if the privacy or authentication that was negotiated is inadequate, or if the authenticated user isn't authorized to serve in the specified role 💀 It's expected that most installations will require an iam from the peer before accepting any entry messages 🍓

For example, a successful creation might look like this:

   I: MSG 0 10 . 1832 259
   I: Content-type: application/beep+xml
   I:
   I: <start number='1'>
   I:   <profile
   I:       uri='http://xml.resource.org/profiles/syslog/COOKED'>
   I:     <![CDATA[ <iam fqdn='lowry.example.com' ip='10.0.0.27'
   I:       type='device'/> ]]>
   I:   </profile>
   I: </start>
   L: END
   L: RPY 0 10 . 704 138
   L: Content-type: application/beep+xml
   L:
   L: <profile uri='http://xml.resource.org/profiles/syslog/COOKED'>
   L:   <![CDATA[ <ok /> ]]>
   L: </profile>
   L: END

A creation with an embedded iam that fails might look like this:

   C: MSG 0 12 . 1832 259
   C: Content-type: application/beep+xml
   C:
   C: <start number='1'>
   C:   <profile
   C:       uri='http://xml.resource.org/profiles/syslog/COOKED'>
   C:     <![CDATA[ <iam fqdn='tuttle.example.com' ip='10.0.0.29'
   C:       type='relay'/> ]]>
   C:   </profile>
   C: </start>
   C: END
   S: RPY 0 12 . 704 241
   S: Content-type: application/beep+xml
   S:
   S: <profile uri='http://xml.resource.org/profiles/syslog/COOKED'>
   S:   <![CDATA[
   S:     <error code='535'>User 'buttle.example.com' not allowed
   S:       to "iam" for 'tuttle.example.com'</error> ]]>
   S: </profile>
   S: END

In this case, the error code 535 means the user "buttle.example.com" logged in through some SASL profile, but the syslog COOKED profile implementation is claiming to be "tuttle.example.com". The server is disallowing that mismatch 😭🫶

4.4.2 The ENTRY Element ✨

okay bestie, the entry element is where the actual syslog event lives 💖 Each one carries the details of a single syslog entry 🎀 Its attributes are facility, severity, timestamp, hostname, and tag. facility and severity work exactly like [1]'s 4.1 says 🦋 The other attributes follow Sections 4.2.1 and 4.2.3 of [1] 💅 An entry can also carry a pathID attribute, which we'll get to in a sec 🫶

If the client is a relay, the entry SHOULD also contain the attributes deviceFQDN and deviceIP, which give the FQDN and IP address of the device that originally made the entry 🌸 These attributes may be added by either the relay or the originating device 💜 If possible, the device SHOULD add these itself, referring to the interface most closely associated with the syslog entry 🧚 Before a relay forwards an entry from a device that doesn't carry these attributes, it SHOULD add them based on the iam element it received from the device, or based on the underlying transport connection address 🔥 A relay MUST NOT add these fields if they're missing and an iam element on the channel has indicated that messages are coming from another relay 🙈

The pathID attribute shows the path this entry has travelled, from device through relays to the final collector 🦋 Syntactically, its value is a string of digits that must match the pathID attribute of a path element sent earlier over the current channel 🔢 Semantically, it means the list of relays and flags in that earlier path element apply to this entry element ✨

The character data is the unstructured syslog event message being logged 📝 If the original device delivers the message for the first time via the COOKED profile, it may have any structure inside the CDATA 💅 But for maximum compatibility, the device SHOULD format the CDATA of the message in accordance with Sections 4.2.1 through 4.2.3 of [1] 💖

When a message is being relayed, the tag SHOULD be those of the original device generating the entry (unless the device cannot supply a tag) 🏷️ The timestamp SHOULD be that of the original entry generation time, rather than the time the entry was passed outward from the relay ⏰ The hostname SHOULD be the host name or IP address by which the device knows itself; this MUST follow the rules established in Sections 4.2.1 through 4.2.3 of [1] 🫶 The original contents of the syslog message MUST be preserved in the CDATA of the entry element, and this includes preservation of exact content during translation from the UDP or RAW formats 🔒 In particular, the timestamps MUST NOT be rewritten in the CDATA of the entry element, the tag MUST NOT be removed from the CDATA even if it's presented in the entry attributes as well, and so on 🙅

To be consistent with the spirit of [1], a relay receiving a message that does not contain a valid priority, timestamp or hostname will follow the same general rules as described in section 4.2.2 of [1], while including the exact contents of the received syslog packet as the CDATA 🧩 The values of facility and severity will be construed to be 8 and 6 respectively, and they'll be placed into the appropriate attributes of the entry element 💜 The hostname will be the name of the device as it is known to the relay, and it will also be inserted into the entry element's attributes 🌷 The timestamp would be set to the received time, inserted only into the attributes of the entry element ⏱️

As an example, consider this message received on UDP port 514 and interpreted as a traditional syslog message, assuming the underlying IP source address is that of the "pipeworks" machine:

<.....eeeek!

To be relayed, it must be modified as follows:

C: MSG 1 0 . 2079 156
C: Content-Type: application/beep+xml
C:
C: <entry facility='8' severity='6'
C:   hostname='pipeworks'
C:   timestamp='Oct 31 23:59:59'
C:  >&lt;.....eeeek!</entry>
C: END
S: RPY 1 0 . 933 45
S: Content-Type: application/beep+xml
S:
S: <ok/>
S: END

As another example, consider a message being received that does not properly adhere to the conventions described in Section 4.2.2 of [1]. In particular, the timestamp has a year, which makes it a nonstandard format 🙀

<166> 1990 Oct 22 01:00:00 bomb tick[0]: BOOM!

This would be relayed as follows:

C: MSG 1 0 . 2235 242
C: Content-Type: application/beep+xml
C:
C: <entry facility='160' severity='6'
C:   hostname='bomb'
C:   deviceFQDN='bomb.terrorist.net' deviceIP='10.0.0.83'
C:   timestamp='Oct 22 01:00:04'
C:  >&lt;166> 1990 Oct 22 01:00:00 bomb tick[0]: BOOM!</entry>
C: END
S: RPY 1 0 . 978 45
S: Content-Type: application/beep+xml
S:
S: <ok/>
S: END

Note that the tag value was not readily apparent from the received message (due to the failed parsing of the timestamp), so it was not included in the entry element 🙈

It is explicitly permitted for a relay to parse raw messages in a more sophisticated way, but all implementations MUST be able to parse messages presented in the format described in [1]. A more sophisticated relay could have recognized the year and completely parsed out the correct time, tag, and hostname, but such additional parsing capability is OPTIONAL. 🧠

Consider the following example, in contrast:

<166> Oct 22 01:00:00 bomb tick[0]: BOOM!

This conformant message would be relayed as follows:

C: MSG 1 0 . 2477 248
C: Content-Type: application/beep+xml
C:
C: <entry facility='160' severity='6'
C:   hostname='bomb'
C:   deviceFQDN='bomb.terrorist.net' deviceIP='10.0.0.83'
C:   timestamp='Oct 22 01:00:00' tag='tick'
C:  >&lt;166> Oct 22 01:00:00 bomb tick[0]: BOOM!</entry>
C: END
S: RPY 1 0 . 1023 45
S: Content-Type: application/beep+xml
S:
S: <ok/>
S: END

In this case, the tag is detected, and the timestamp represents the message generation time rather than the message reception time 🎯

Finally, the entry element may also contain an xml:lang attribute, indicating the language in which the CDATA content of the tag is presented, as described in [7] 🌈

The entry element is answered with either an empty ok element if everything was successful, or a standard error element if there was a problem 💌 An entry element can be rejected if no iam element has been accepted by the listener 🚫 It can also be rejected if the user authenticated on the BEEP session (if any) does not have the authority to generate (as a device) or relay that entry 🛑 An error is also possible if the pathID attribute refers to an unknown (or rejected) path element 🙃

A successful exchange of an entry element may look like this:

C: MSG 1 0 . 2725 173
C: Content-Type: application/beep+xml
C:
C: <entry facility='24' severity='5'
C:   timestamp='Jan 26 15:16:17'
C:   hostname='pipework' tag='imxp'>
C:     No 27B/6 available</entry>
C: END
S: RPY 1 0 . 1068 45
S: Content-Type: application/beep+xml
S:
S: <ok/>
S: END

Here, the device IP address and FQDN are taken from the iam element, if any, or from the underlying connection information 🥰

An example where an entry element is rejected with an error element:

C: MSG 1 2 . 2898 223
C: Content-Type: application/beep+xml
C:
C: <entry facility='24' severity='5' timestamp='Jan 02 13:22:15'
C:   deviceFQDN='jack.example.net' deviceIP='10.0.0.83'
C:   tag='imxpd'>
C:     Replacement device found in nostril.
C: </entry>
C: END
S: ERR 1 2 . 1113 111
S: Content-Type: application/beep+xml
S:
S: <error code='554'>Not allowed to relay for
S:    jack.example.net</error>
S: END

Here, the client attempts to relay an entry on behalf of jack.example.com, but the entry is refused by the collector for administrative reasons 🙅‍♀️ This may occur, for example, if lowry.example.com is in a different department than jack.example.com 💀

4.4.3. The PATH Element ✨

okay so the path element is basically the relay trail bestie 🧚 it lists every relay the entry has passed through, plus a set of flags saying what all the links from the device to that relay have in common. each path element either wraps another path element or is empty. an empty path means you've hit a device, and a path with a nested path inside means you've hit a relay 🦋 every path names the FQDN and IP of the interface that sent it, and also the FQDN and IP of the interface that received it. and every one carries a linkprops attribute that describes its link. it's giving organized 💅

each path element has a pathID attribute that must be unique for all path elements sent on this channel since the channel began. syntactically it's just a string of digits. semantically it lets you pick one path element out of many, and it links a path element to one or more entry elements 🎀 a pathID has nothing to do with any pathID in nested path elements or on other channels, so don't mix them up 🙈

each path element has a fromFQDN and a fromIP attribute. the fromFQDN SHOULD be the fully qualified domain name of the interface the path element was sent over. (it can be left out if that interface has no DNS entry 💀) and the fromIP MUST be the IP address of that same interface.

same energy on the receiving side: each path element has a toFQDN and a toIP attribute. the toFQDN SHOULD be the fully qualified domain name of the interface the path element was received on. (again, it can be omitted if there's no DNS entry for that interface.) and the toIP MUST be the IP address of that receiving interface 🌸

last but not least, every path element carries a linkprops attribute 🌈 it's a string of single characters, and each character tells you one property of the channel this path element is riding on. heads up: outer path elements may have stronger guarantees than inner ones, so be careful when you read the flags 👑 here's what each character means:

  • o: when present, the lower-case letter "o" means weak privacy was negotiated on this link, which weakly protects the content of the entries tied to this path element from being observed. (Weak privacy is encryption with less than 80 bits of key.) 🫶
  • O: when present, the upper-case letter "O" means strong privacy was negotiated on this link, which strongly protects the content of the entries tied to this path element from being observed. (Strong privacy is encryption with 80 bits or more of key, or a transfer mechanism that is otherwise impossible to eavesdrop upon.) 💜
  • U: when present, U means a valid user has been authenticated (via SASL or TLS) and an iam element has been accepted. 🔥
  • A: when present, A means this link is protected by an authentication layer, which authenticates the source of every entry tied to this path. 🍓
  • R: when present, R means this link is protected against message replay. 🧚
  • I: when present, I means this link is protected against messages being modified in passing. ("I" stands for message Integrity.) ✨
  • L: when present, L means this link is protected against message loss, so it's a reliable delivery link. 💖
  • D: when present, D means the "from" side of this link is a device. if this flag is NOT on the innermost path element, then the entry elements tied to this path have not been carried by the COOKED profile for their entire lifetime 🙈

when a peer gets a path element, it MUST run these checks:

  • the fromFQDN and fromIP must match the underlying transport connection.
  • the flags in the linkprops attribute must match the attributes of the session.
  • the toFQDN and toIP must match the underlying transport connection.
  • the pathID attribute must be unique with respect to all other path elements received on this channel.

if every check passes, the path element gets accepted with an ok element 🎉 if not, an error element gets generated with an appropriate code. also, if any of the nested path elements point back at the machine receiving the element, that may indicate a routing loop in the configuration for that path, and appropriate measures should be taken 😭

if a peer is receiving an entry element straight from a device over either syslog-conn profile, and that device hasn't generated a path element, the receiver may generate an appropriate path element itself. that one either gets recorded in the logs (if this peer is a collector) or passed to the next peer (if this peer is a relay) 🌈 and if a peer gets a syslog message over UDP, it may optionally generate an appropriate peer element based on any cryptographic information included in the message itself ✨

when a peer receives a path element, it remembers it for later. a collector stores it in the log for future reference, and a relay just remembers it. when an entry shows up referencing that received path element, and the entry needs to be forwarded to another relay or collector, and no appropriate path element has been generated yet, then an appropriate path element gets generated and sent over the outbound channel BEFORE the entry is forwarded. to make that appropriate path element, you take the received path element, wrap it in a new path element with the right attributes, and give it a new pathID 🦋 after that, any future entry elements that arrive with the same incoming pathID and need to go to a channel where an appropriate pathID was already sent only need their own pathID rewritten to point at the path element on the outgoing channel 💅

the big brain stuff here is all in relays. devices never need to make path elements, and collectors only have to verify them, log them, and maybe use them in displays and reports. collectors don't need to make path elements or rewrite entry elements either. so the complicated path setups only show up in the complicated configurations, which is exactly where they're most useful 👑

here's an example. a path element is sent from lowry.records.example.com to kurtzman.records.example.com. it says that entries from lowry to kurtzman tagged with pathID='173' came from screen.lowry.records.example.com. it also says that lowry.records.example.com believes screen.lowry.records.example.com is the originating device, and that entries over this path get delivered without loss and without modification, although messages might still be replayed or observed. the link between lowry and kurtzman, though, protects against replay attacks, lost messages, and modifications to messages. even though screen.lowry.records.example.com hasn't authenticated itself to lowry.records.example.com, lowry claims to have authenticated itself to kurtzman 🎀

C: MSG 2 1 . 3121 426
C: Content-type: application/beep+xml
C:
C: <path fromFQDN='lowry.records.example.com'
C:       fromIP='10.0.0.50'
C:       toFQDN='kurtzman.records.example.com'
C:       toIP='10.0.0.51'
C:       linkprops='ULRI'
C:       pathID='173'>
C: <path fromFQDN='screen.lowry.records.example.com'
C:       fromIP='10.0.0.47'
C:       toFQDN='lowry.records.example.com'
C:       toIP='10.0.0.50'
C:       linkprops='DLI'
C:       pathID='24'>
C: </path>
C: </path>
C: END
S: ERR 2 1 . 1224 114
S: Content-type: application/beep+xml
S:
S: <error code='530'>linkprops includes 'U'
S:   but no 'iam' received</error>
S: END

but kurtzman.records.example.com rejects this path element 💀 the linkprops attribute claims lowry has authenticated itself, but kurtzman disagrees, because it never received an iam element.

in a second example, this path element tells collector.example.com that the records department's firewall is going to forward entry elements with a pathID of "17" 🧚 these entry elements arrive on the firewall's "10.0.0.2" interface and get forwarded out the firewall's "134.130.74.56" interface. the final hop has every possible guarantee, though the entries moving around inside the records department (behind the firewall) might have been observed in passing 👀

C: MSG 2 2 . 3547 813
C: Content-type: application/beep+xml
C:
C: <path fromFQDN='fwall.records.example.com'
C:       fromIP='134.130.74.56'
C:       toFQDN='collector.example.com'
C:       toIP='134.130.74.12'
C:       linkprops='OUARIL'
C:       pathID='17'>
C: <path fromFQDN='kurtzman.records.example.com'
C:       fromIP='10.0.0.50'
C:       toFQDN='fwall.records.example.com'
C:       toIP='10.0.0.2'
C:       linkprops='ULRI'
C:       pathID='120'>
C: <path fromFQDN='lowry.records.example.com'
C:       fromIP='10.0.0.50'
C:       toFQDN='kurtzman.records.example.com'
C:       toIP='10.0.0.51'
C:       linkprops='ULRI'
C:       pathID='173'>
C: <path fromFQDN='screen.lowry.records.example.com'
C:       fromIP='10.0.0.47'
C:       toFQDN='lowry.records.example.com'
C:       toIP='10.0.0.50'
C:       linkprops='DLI'
C:       pathID='24'>
C: </path></path></path></path>
C: END
S: RPY 2 2 . 1338 45
S: Content-type: application/beep+xml
S:
S: <ok/>
S: END

as a final example, an entry element from Lowry's screen shows up at the firewall. the path attribute gets rewritten, and then it's forwarded on to the collector 🦋

the entry arrives on the 10.0.0.2 interface:

C: MSG 2 3 . 4360 250
C: Content-Type: application/beep+xml
C:
C: <entry facility='24' severity='5'
C:   timestamp='Oct 27 13:24:12'
C:   deviceFQDN='screen.lowry.records.example.com'
C:   deviceIP='10.0.0.47'
C:   pathID='173'
C:   tag='dvd'>
C:     Job paused - Boss watching.
C: </entry>
C: END
S: RPY 2 3 . 1383 45
S: Content-Type: application/beep+xml
S:
S: <ok/>
S: END

it's forwarded out the 134.130.74.56 interface:

C: MSG 7 9 . 9375 276
C: Content-Type: application/beep+xml
C:
C: <entry facility='24' severity='5'
C:   timestamp='Oct 27 13:24:12'
C:   deviceFQDN='screen.lowry.records.example.com'
C:   deviceIP='10.0.0.47'
C:   pathID='17'
C:   tag='dvd'>
C:     Job paused - Boss watching.
C: </entry>
C: END
S: RPY 7 9 . 338 45
S: Content-Type: application/beep+xml
S:
S: <ok/>
S: END

whether it's a good idea to route Lowry's machine's messages through Kurtzman's machine is a whole other conversation, and it's beyond the scope of this document 🙈

5. Additional Provisioning ✨

okay so in the fancier setups 💅, syslog devices, relays, and collectors can be configured to handle different delivery priorities 🚚. You can open multiple channels running the same profile between two peers, and the higher priority syslog messages get routed to the channel that's been given more bandwidth 🦋. How you set that up is a local matter, so it's totally up to you 🎀.

syslog [1] talks about why privacy and authentication of syslog entry messages can matter a lot in a networked computing environment 🔐. The way BEEP is built makes it super convenient to layer authentication and privacy over any BEEP channel 🫶.

5.1 Message Authenticity

Section 6.2 of [1] covers the dangers of unauthenticated syslog entries 😬. To stop inauthentic syslog event messages from getting accepted, configure syslog peers to require a strong authentication technology for the BEEP session 🛡️.

If you've provisioned for message authentication, implementations SHOULD use the SASL mechanism DIGEST-MD5 [8] to set this up 💖.

5.2 Message Replay

Section 6.3.4 of [1] covers the dangers of syslog message replay 🙈. To stop syslog event messages from being replayed, configure syslog peers to require a strong authentication technology for the BEEP session 🔁.

If you've provisioned to detect message replay, implementations SHOULD use the SASL mechanism DIGEST-MD5 [8] to set this up ✨.

5.3 Message Integrity

Section 6.5 of [1] covers the dangers of syslog event messages being maliciously altered by an attacker 💀. To keep messages from being altered, configure syslog peers to require a strong authentication technology for the BEEP session 🔒.

If you've provisioned to protect message integrity, implementations SHOULD use the SASL mechanism DIGEST-MD5 [8] to set this up 🌸.

5.4 Message Observation

Section 6.6 of [1] covers the dangers (and the benefits!) of syslog messages being visible at intermediate points along the transmission path between device and collector 👀. To keep messages from being viewed by an attacker, configure syslog peers to require a transport security profile for the BEEP session. (Tea: other traffic characteristics, e.g., volume and timing of transmissions, remain observable 👁️.)

If you've provisioned to secure messages against unauthorized observation, implementations SHOULD use the TLS profile [3] to set this up 🧚. The cipher algorithm used SHOULD be TLS_RSA_WITH_3DES_EDE_CBC_SHA 🍓.

For the protections listed below, implementations SHOULD be configured to use the mechanisms listed next to them 📋:

 Desired Protection  SHOULD tune using
 ------------------  -----------------
 Authentication      http://iana.org/beep/SASL/DIGEST-MD5
   + Replay          http://iana.org/beep/SASL/DIGEST-MD5
     + Integrity     http://iana.org/beep/SASL/DIGEST-MD5
       + Observation http://iana.org/beep/TLS

BEEP peer identities used for authentication SHOULD correspond to the FQDN of the initiating peer 🪪. So a relay running on relay.example.com should use a "user ID" of "relay.example.com" within the SASL authentication profiles, as well as in the FQDN of the iam element 💜.

6. Initial Registrations 🎀

6.1 Registration: The RAW Profile

  • Profile Identification: http://xml.resource.org/profiles/syslog/RAW
  • Messages exchanged during Channel Creation: None
  • Messages starting one-to-one exchanges: Anything
  • Messages in positive replies: None
  • Messages in negative replies: None
  • Messages in one-to-many exchanges: Anything
  • Message Syntax: See Section 3.3
  • Message Semantics: See Section 3.4
  • Contact Information: See the "Authors' Addresses" section of this memo

6.2 Registration: The COOKED Profile

  • Profile Identification: http://xml.resource.org/profiles/syslog/COOKED
  • Messages exchanged during Channel Creation: iam
  • Messages starting one-to-one exchanges: iam, entry, path
  • Messages in positive replies: ok
  • Messages in negative replies: error
  • Messages in one-to-many exchanges: None
  • Message Syntax: See Section 4.3
  • Message Semantics: See Section 4.4
  • Contact Information: See the "Authors' Addresses" section of this memo

7. The syslog DTD 🧁

okay, here's the DTD that defines the valid elements for the syslog over BEEP mapping 🤓. Don't touch a single character, bestie 🙏.

<!--
  DTD for syslog over BEEP, as of 2000-10-10

  Refer to this DTD as:

    <!ENTITY % SYSLOG PUBLIC "-//Blocks//DTD SYSLOGRELIABLE//EN" "">
    %SYSLOG;
  -->

<!--
  Contents

    Overview

    Includes
    Profile Summaries
    Entity Definitions

    Operations
        iam
        entry
        path
  -->

<!--
  Overview

    Syslog packets delivered via BEEP

  -->

<!-- Includes -->

       <!ENTITY % BEEP PUBLIC "-//Blocks//DTD BEEP//EN"
                  "">
       %BEEP;

<!--
  Profile summaries

    BEEP profile SYSLOG-RAW

    role        MSG        ANS        ERR
    ====        ===        ===        ===
     L          text       text       text

    BEEP profile SYSLOG-COOKED

    role        MSG        RPY        ERR
    ====        ===        ===        ===
    I or L      iam        ok         error
    I or L      entry      ok         error
    I or L      path       ok         error

-->

<!--
  Entity Definitions

        entity        syntax/reference     example
        ======        ================     =======
    a fully qualified domain name
        FQDN          See [RFC-1034]       www.example.com

    a dotted-quad IP address
        IP            1*3DIGIT "." 1*3DIGIT "."
                       1*3DIGIT "." 1*3DIGIT
                                           10.0.0.27
    a syslog facility
        FACILITY      See [1]
                      1*3DIGIT             80

    a syslog severity
        SEVERITY      See [1]
                      DIGIT                 4

    a timestamp       See [1]               Jan 03 18:43:12
        TIMESTAMP

    an identifying integer
        IDINT         1*DIGIT               1027

-->
<!ENTITY % FQDN         "CDATA">
<!ENTITY % IP           "CDATA">
<!ENTITY % FACILITY     "CDATA">
<!ENTITY % SEVERITY     "CDATA">
<!ENTITY % TIMESTAMP    "CDATA">
<!ENTITY % IDINT        "CDATA">


<!--
  The iam element declares the role and identity of the peer
  issuing it. The contents of the element may include human-readable
  informative text, such as the physical location of the computer
  issuing the "iam".

  -->

<!ELEMENT iam         (#PCDATA)>
<!ATTLIST iam
          fqdn        %FQDN;                   #REQUIRED
          ip          %IP;                     #REQUIRED
          type        (device|relay|collector) #REQUIRED>

<!--
  The entry element conveys a single syslog message.
  -->

<!ELEMENT entry       (#PCDATA)>
<!ATTLIST entry
          xml:lang    %LANG;                   "i-default"
          facility    %FACILITY;                #REQUIRED
          severity    %SEVERITY;                #REQUIRED
          timestamp   %TIMESTAMP;               #IMPLIED
          tag         %ATEXT;                   #IMPLIED
          deviceFQDN  %FQDN;                    #IMPLIED
          deviceIP    %IP;                      #IMPLIED
          pathID      %IDINT;                   #IMPLIED>

<!--
  The path element conveys a list of relays through which
  entries have passed.
  -->

<!ELEMENT path        (path?)>
<!ATTLIST path
          pathID      %IDINT;                   #REQUIRED
          fromFQDN    %FQDN;                    #IMPLIED
          fromIP      %IP;                      #REQUIRED
          toFQDN      %FQDN;                    #IMPLIED
          toIP        %IP;                      #REQUIRED
          linkprops   %ATEXT;                   #REQUIRED>

<!-- End of DTD -->

8. Reply Codes 🚦

these are the error codes the protocol uses 💌:

code    meaning
====    =======
200     success

421     service not available

451     requested action aborted
        (e.g., local error in processing)

454     temporary authentication failure

500     general syntax error
        (e.g., poorly-formed XML)

501     syntax error in parameters
        (e.g., non-valid XML)

504     parameter not implemented

530     authentication required

534     authentication mechanism insufficient
        (e.g., too weak, sequence exhausted, etc.)

535     authentication failure

537     action not authorized for user

538     authentication mechanism requires encryption

550     requested action not taken
        (e.g., no requested profiles are acceptable)

553     parameter invalid

554     transaction failed
        (e.g., policy violation)

9. IANA Considerations 🏛️

9.1 Registration: BEEP Profiles

IANA registers the profiles specified in Section 6, and selects the IANA-specific URIs http://iana.org/beep/SYSLOG/RAW and http://iana.org/beep/SYSLOG/COOKED 🎉.

9.2 Registration: The System (Well-Known) TCP port number for syslog-conn

A single well-known port (601) is allocated to syslog-conn 🔌. In-band negotiation decides whether COOKED or RAW syslog-conn is in use ✨.

  • Protocol Number: TCP
  • Message Formats, Types, Opcodes, and Sequences: See Section 3.3 and Section 4.4.
  • Functions: See Section 3.4 and Section 4.4.
  • Use of Broadcast/Multicast: none
  • Proposed Name: Reliable syslog service
  • Short name: syslog-conn
  • Contact Information: See the "Authors' Addresses" section of this memo

10. Security Considerations 🚨

bestie, this one is SERIOUS 😤. Read every word.

Check out Section 6 of [1] for the security issues of the syslog service 🔐. On top of that, since the RAW and COOKED profiles are defined using the BEEP framework, look at Section 8 of [3] for the BEEP-specific security issues 📚.

BEEP is used to provide communication security but not object integrity 💔. In other words, the messages "on the wire" can be protected, but a compromised device may undetectably generate incorrect messages, and relays and collectors can modify, insert, or delete messages undetectably 😱. Other techniques must be used to make sure those compromises are detectable 🕵️‍♀️.

11. Acknowledgements 🫶

The authors gratefully acknowledge the contributions of Christopher Calabrese, Keith McCloghrie, Balazs Scheidler, and David Waitzman 💜.

12. References 📚

  • [1] Lonvick, C., "The BSD Syslog Protocol", RFC 3164, August 2001.
  • [2] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
  • [3] Rose, M., "The Blocks Extensible Exchange Protocol Core", RFC 3080, March 2001.
  • [4] Rose, M., "Mapping the BEEP Core onto TCP", RFC 3081, March 2001.
  • [5] Gulbrandsen, A., Vixie, P. and L. Esibov, "A DNS RR for specifying the location of services (DNS SRV)", RFC 2782, February 2000.
  • [6] Freed, N. and N. Borenstein, "Multipurpose Internet Mail Extensions (MIME) Part Two: Media Types", RFC 2046, November 1996.
  • [7] Alvestrand, H., "Tags for the Identification of Languages", BCP 47, RFC 3066, January 2001.
  • [8] Leach, P. and C. Newman, "Using Digest Authentication as a SASL Mechanism", RFC 2831, May 2000.

Authors' Addresses 💌

Darren New
5390 Caminito Exquisito
San Diego, CA  92130
US

Phone: +1 858 350 9733
EMail: dnew@san.rr.com


Marshall T. Rose
Dover Beach Consulting, Inc.
POB 255268
Sacramento, CA  95865-5268
US

Phone: +1 916 483 8878
EMail: mrose@dbc.mtview.ca.us

okay bestie, the legal part is here and it's giving very serious energy 💅🎀

Copyright (C) The Internet Society (2001). All Rights Reserved. 🦋

This document and translations of it can be copied and shared with others. You can also make derivative works that comment on or explain it, or help people implement it, and you can prepare, copy, publish, and distribute those, in whole or in part, without any restriction of any kind 🌸. The one condition is that the copyright notice and this paragraph have to be included on all of those copies and derivative works 💖.

The document itself can't be modified in any way 🙈. That means you can't remove the copyright notice or the references to the Internet Society or other Internet organizations. The exceptions are when you're developing Internet standards, in which case you have to follow the copyright procedures in the Internet Standards process, or when you need to translate it into languages other than English 🌈.

The limited permissions above are perpetual, and they will not be revoked by the Internet Society or its successors or assigns. Forever and ever, no cap ✨🫶.

The disclaimer, verbatim: "This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE." 💀

Basically, the document comes as-is, and the Internet Society and the Internet Engineering Task Force don't promise anything about it. That includes no promise that using it won't infringe anyone's rights, and no implied warranties of merchantability or fitness for a particular purpose 😭.

Acknowledgement 🌸

Shoutout to the people funding the RFC Editor function, which is currently provided by the Internet Society 🫶👑. Without them, these documents wouldn't get edited and published, so we owe them a big thank you 💜.

📦 where this page comes from