RFC 5024
ODETTE File Transfer Protocol 2.0
RFC 5024: ODETTE File Transfer Protocol 2 🎀
Okay bestie, this one is RFC 5024 💖, and it's all about ODETTE File Transfer Protocol 2. It was written by I. Friend, who is from ODETTE, and it was published by the Network Working Group ✨. It's dated November 2007 and it's Informational, which means it's giving "here's useful info," not "you must do this" 🙈. It obsoletes RFC 2204, so it replaces that older doc, and it also updates the ODETTE File Transfer Protocol up to version 2 🦋.
Status of This Memo 📝
Okay so this memo is here to share info with the Internet community. It is not an Internet standard of any kind, and you can pass it around as much as you want because distribution is unlimited 🌸.
IESG Note 🚨
This RFC is not a candidate for any level of Internet Standard. The IETF is not vouching for whether it's fit for any purpose 💀. The decision to publish was not based on IETF review for things like security, congestion control, or weird interactions with protocols that are already out there. The RFC Editor picked to publish this at their discretion, so readers should be careful about how much they trust it for implementing or deploying. See RFC 3932 for more info 🫶.
Abstract ✨
Okay so this memo updates the ODETTE File Transfer Protocol 🎀, which is already well established. It's a file transfer protocol for electronic data interchange (EDI) of business data between trading partners, and this update takes it to version 2 👑.
It's giving security glow-up 🔥. The protocol now supports secure and authenticated communication over the Internet using Transport Layer Security. It also does file encryption, signing, and compression with Cryptographic Message Syntax, and it sends signed receipts to acknowledge files that were received 💅.
The protocol works for both direct peer-to-peer communication and indirect communication through a Value Added Network. It can be used over TCP/IP, X.25, and ISDN-based networks 🧚.
Table of Contents 📚
1. Introduction ....................................................4
1.1. Background .................................................4
1.2. Summary of Features ........................................5
1.3. General Principles .........................................5
1.4. Structure ..................................................6
1.5. Virtual Files ..............................................6
1.6. Service Description ........................................9
1.7. Security ...................................................9
2. Network Service ................................................11
2.1. Introduction ..............................................11
2.2. Service Primitives ........................................11
2.3. Secure ODETTE-FTP Session .................................12
2.4. Port Assignment ...........................................12
3. File Transfer Service ..........................................13
3.1. Model .....................................................13
3.2. Session Setup .............................................14
3.3. File Transfer .............................................16
3.4. Session Take Down .........................................20
3.5. Service State Automata ....................................23
4. Protocol Specification .........................................28
4.1. Overview ..................................................28
4.2. Start Session Phase .......................................28
4.3. Start File Phase ..........................................30
4.4. Data Transfer Phase .......................................34
4.5. End File Phase ............................................35
4.6. End Session Phase .........................................36
4.7. Problem Handling ..........................................36
5. Commands and Formats ...........................................37
5.1. Conventions ...............................................37
5.2. Commands ..................................................37
5.3. Command Formats ...........................................37
5.4. Identification Code .......................................68
6. File Services ..................................................69
6.1. Overview ..................................................69
6.2. File Signing ..............................................69
6.3. File Encryption ...........................................70
6.4. File Compression ..........................................70
6.5. V Format Files - Record Lengths ...........................70
7. ODETTE-FTP Data Exchange Buffer ................................71
7.1. Overview ..................................................71
7.2. Data Exchange Buffer Format ...............................71
7.3. Buffer Filling Rules ......................................72
8. Stream Transmission Buffer .....................................73
8.1. Introduction ..............................................73
8.2. Stream Transmission Header Format .........................73
9. Protocol State Machine .........................................74
9.1. ODETTE-FTP State Machine ..................................74
9.2. Error Handling ............................................75
9.3. States ....................................................76
9.4. Input Events ..............................................79
9.5. Output Events .............................................79
9.6. Local Variables ...........................................80
9.7. Local Constants ...........................................81
9.8. Session Connection State Table ............................82
9.9. Error and Abort State Table ...............................85
9.10. Speaker State Table 1 ....................................86
9.11. Speaker State Table 2 ....................................91
9.12. Listener State Table .....................................93
9.13. Example ..................................................96
10. Miscellaneous .................................................97
10.1. Algorithm Choice .........................................97
10.2. Cryptographic Algorithms .................................97
10.3. Protocol Extensions ......................................97
10.4. Certificate Services .....................................98
11. Security Considerations .......................................98
Appendix A. Virtual File Mapping Example .........................100
Appendix B. ISO 646 Character Subset .............................103
Appendix C. X.25 Specific Information ............................104
C.1. X.25 Addressing Restrictions .............................104
C.2. Special Logic ............................................105
C.3. PAD Parameter Profile ....................................116
Appendix D. OFTP X.25 Over ISDN Recommendation ...................118
D.1. ODETTE ISDN Recommendation ...............................119
D.2. Introduction to ISDN .....................................120
D.3. Equipment Types ..........................................123
D.4. Implementation ...........................................124
Acknowledgements .................................................132
Normative References .............................................132
Informative References ...........................................133
ODETTE Address ...................................................1341. Introduction ✨
1.1. Background 🌷
Okay so the ODETTE File Transfer Protocol (ODETTE-FTP) was defined in 1986 by working group four of the Organisation for Data Exchange by Tele Transmission in Europe (ODETTE). It was made to handle the electronic data interchange (EDI) needs of the European automotive industry 🚗.
ODETTE-FTP lets business applications swap files on a peer-to-peer basis in a standardised, totally automatic way 🤖. It also has a defined acknowledgement process for when a file is successfully received 💖.
ODETTE-FTP is NOT a variant of, or similar to, the Internet FTP [FTP] 🙅♀️. Internet FTP gives individuals an interactive way to share files, and it has no acknowledgement process at all. Because it's so interactive, doesn't do file acknowledgements, and uses a client/server design, FTP doesn't really work for mission-critical business data exchange 💀.
Over the last ten years, ODETTE-FTP has been widely deployed on systems of every size, from personal computers to big mainframes 🖥️. Meanwhile the Internet has become the dominant international network, offering high-speed communication at low cost 🌍. To keep up with the demand for EDI over the Internet, ODETTE decided to expand its file transfer protocol with security functions and advanced compression techniques 🔥. The goal is to keep it at the forefront of information exchange technology 👑.
The protocol now supports secure and authenticated communication over the Internet using Transport Layer Security, provides file encryption, signing, and compression using Cryptographic Message Syntax, and provides signed receipts for the acknowledgement of received files ✨.
The protocol supports both direct peer-to-peer communication and indirect communication via a Value Added Network, and it may be used with TCP/IP, X.25 and ISDN based networks 🦋.
ODETTE-FTP was defined by the ODETTE Security Working Group, which includes a number of ODETTE member organisations. All of the members have serious operational experience working with and developing OFTP and EDI solutions 🫶.
1.2. Summary of Features 🍓
This memo is a development of version 1.4 of ODETTE-FTP [OFTP] with these changes and additions 🎀:
- Session level encryption
- File level encryption
- Secure authentication
- File compression
- Signed End to End Response (EERP)
- Signed Negative End Response (NERP)
- Maximum permitted file size increased to 9 PB (petabytes)
- Virtual file description added
- Extended error codes
Version 1.4 of ODETTE-FTP included these changes and additions to version 1.3 🌈:
- Negative End Response (NERP)
- Extended Date and Time stamp
- New reason code 14 (File direction refused)
1.3. General Principles 🧚
The whole point of ODETTE-FTP is to move a file between one or more locations in a way that doesn't depend on the data communication network, the system hardware, or the software environment 💜.
When they were designing and specifying the protocol, these factors were considered 📝:
- The possible differences of size and sophistication of file storage and small and large systems.
- The necessity to work with existing systems (reduce changes to existing products and allow easy implementation).
- Systems of different ages.
- Systems of different manufactures.
- The potential for growth in sophistication (limit impact and avoid changes at other locations).
1.4. Structure 🏗️
Okay so ODETTE-FTP is modelled on the OSI reference model 🧠. It's designed to use the Network Service provided by level 3 of the model, and it provides a File Service to the users. That means the protocol spans levels 4 to 7 of the model ✨.
The description of ODETTE-FTP in this memo is closely related to the original 'X.25' specification of the protocol. In the spirit of the OSI model, it describes 💅:
- A File Service provided to a User Monitor.
- A protocol for the exchange of information between peer ODETTE-FTP entities.
1.5. Virtual Files 🎀
okay so bestie, ODETTE-FTP entities always swap info in a standard format called a Virtual File 💅 which means the transfer doesn't care what kind of systems are chatting. literally iconic ✨
Turning a file into its virtual form, and back again, is different on every system. This document doesn't define how that mapping works 🙈
o---------o
Site | Local |
A | File A |
o---------o
|
o----------------------- Mapping A ------------------------o
| | |
| o---------o |
| | Virtual | |
| | File | |
| o---------o |
| o------------------------------------------------o |
| | | |
| | ODETTE-FTP | |
| | | |
| o------------------------------------------------o |
| o---------o o---------o |
| | Virtual | | Virtual | |
| | File | | File | |
| o---------o o----+----o |
| | | |
o------ Mapping B ------------------------ Mapping C ------o
| |
o---------o o----+----o
| Local | Site Site | Local |
| File B | B C | File C |
o---------o o---------oa Virtual File has a bunch of attributes that identify it and describe the data being moved 🦋 the main ones get the spotlight in Sections 1.5.1 to 1.5.4 💜
1.5.1. Organisation 🧚
Sequential
Logical records show up one after another. ODETTE-FTP must be aware of where the record boundaries are 👀
1.5.2. Identification 🍓
Dataset Name
the name of the Virtual File being sent, agreed on by both sides 🫶
Time stamp (HHMMSScccc)
a file qualifier saying what time the Virtual File was made available for transmission. the counter (cccc=0001-9999) gives higher resolution ⏰
Date stamp (CCYYMMDD)
a file qualifier saying what date the Virtual File was made available for transmission 📅
the Dataset Name, Date, and Time attributes are assigned by the Virtual File's originator and are used to uniquely identify a file. they are all mandatory and must not be changed by intermediate locations. no touching!! 🙅♀️
the User Monitor may use the Virtual File Date and Time attributes in local processes involving date comparisons and calculations. any such use falls outside the scope of this protocol 💅
1.5.3. Record Format 🌈
four record formats are defined:
- Fixed (F): each record in the file has the same length.
- Variable (V): the records in the file can have different lengths.
- Unstructured (U): the file contains a stream of data. no structure is defined.
- Text File (T): a Text File is a sequence of ASCII characters with no control characters except
CR-LF, which delimit lines. a line will not have more than 2048 characters 📝
1.5.4. Restart 🔄
ODETTE-FTP can negotiate the restart of an interrupted Virtual File transmission. Fixed and Variable format files are restarted on record boundaries. for Unstructured and Text files, the restart position is expressed as a file offset in 1K (1024 octet) blocks 🧩
the restart position is always calculated relative to the start of the Virtual File. always!! 🎯
1.6. Service Description 🌸
ODETTE-FTP gives a file transfer service to a User Monitor, and in turn it uses the Internet transport layer stream service to talk between peers 🦋
these services are specified in this memo using service primitives sorted into four classes:
- Request (RQ): an entity asks the service to do some work.
- Indication (IND): a service tells an entity about an event.
- Response (RS): an entity responds to an event.
- Confirm (CF): a service tells an entity about the response.
services can be confirmed, using the request, indication, response, and confirm primitives, or unconfirmed using just the request and indication primitives ✨
1.7. Security 🛡️
ODETTE-FTP has a bunch of security services to protect a Virtual File transmission across a hostile network. it's giving bodyguard energy 😤
these security services are as follows:
- Confidentiality
- Integrity
- Non-repudiation of receipt
- Non-repudiation of origin
- Secure authentication
security services in this specification are implemented as follows:
- Session level encryption
- File level encryption
- Signed files
- Signed receipts
- Session level authentication
- ODETTE-FTP Authentication
Session level encryption gives data confidentiality by encrypting all the protocol commands and data exchanged between two parties. that means a third party can't pull any useful info out of the transmission 🙈🔒
this session level encryption is achieved by layering ODETTE-FTP over Transport Layer Security [TLS], and secure and unsecure TCP/IP traffic are told apart by using different port numbers 🚪
File encryption gives complementary data confidentiality by encrypting the files in their entirety. usually this encryption happens before transmission, but it's also possible to encrypt and send files while in session. file encryption has the extra perk of letting a file stay encrypted outside the communications session it was sent in. the file can be received and forwarded by multiple intermediaries, yet only the final destination will be able to decrypt it 🔐 file encryption does not encrypt the actual protocol commands, so trading partner EDI codes and Virtual File names are still viewable 👀
Secure authentication is done through the session level authentication features available in [TLS], and it proves the identity of the parties who want to communicate 🪪
ODETTE-FTP Authentication is also an authentication mechanism, but it's built right into ODETTE-FTP and works on all the network infrastructures ODETTE-FTP runs over. this is different from [TLS], which is generally only available over TCP/IP-based networks 🌐 both parties are required to have certificates when ODETTE-FTP Authentication is used 📜
the security features in ODETTE-FTP 2 are centred around the use of [X.509] certificates. to get the complete range of security services offered in both directions, each party is required to have an [X.509] certificate. if confidentiality of data between two parties is the only concern, then [TLS] alone can be used. that lets the party accepting an incoming connection (the Responder) be the only partner who needs a certificate 💖
for businesses, this means session level encryption between a hub and its trading partners can happen without every trading partner needing to get a certificate, as long as trading partners always connect to the hub 🏢
with the exception of [TLS], all the security services work with X.25 and ISDN as transport media. nothing technically stops [TLS] from working with X.25 or ISDN, but implementations are rare 🤷♀️
2. Network Service 🔥
2.1. Introduction 🧁
ODETTE-FTP peer entities talk to each other via the OSI Network Service or the Transmission Control Protocol Transport Service [RFC793]. this is described by service primitives representing request, indication, response, and confirmation actions 📡
for the Internet environment, the service primitives mentioned below for the Network Service have to be mapped to the respective Transport Service primitives. this section describes the Network Service primitives ODETTE-FTP uses and how they relate to the TCP interface. in practice, the local transport service application programming interface will be used to access the TCP service 🧠
2.2. Service Primitives 💜
all network primitives can be directly mapped to the respective Transport primitives when using TCP ✨
2.2.1. Network Connection 🦋
N_CON_RQ ------> N_CON_IND
N_CON_CF <------ N_CON_RSthis covers setting up a connection. the requesting ODETTE-FTP peer uses the N_CON_RQ primitive to ask for an active OPEN of a connection to a peer ODETTE-FTP, the Responder, which has already asked for a passive OPEN. the Responder gets notified of the incoming connection via N_CON_IND and accepts it with N_CON_RS. the requester learns its OPEN request is complete when it receives N_CON_CF 🎉
Parameters
Request Indication Response Confirmation
---------------------------------------------------------------------
Dest addr ------> same same same2.2.2. Network Data 🫧
N_DATA_RQ ------> N_DATA_INDdata exchange is an unconfirmed service. the requester passes data for transmission to the Network Service via the N_DATA_RQ primitive. the Responder is notified that data is available via N_DATA_IND 📦
in practice, the notification and receipt of data may be combined, such as by the return from a blocking read from the network socket 🔁
Parameters
Request Indication
---------------------------------------------------------------------
Data ------------------> same2.2.3. Network Disconnection 👋
N_DISC_RQ ------> N_DISC_INDan ODETTE-FTP requests the end of a connection with the N_DISC_RQ service primitive. its peer gets told about the CLOSE through an N_DISC_IND event. it's recognised that each peer must issue an N_DISC_RQ primitive to complete the TCP symmetric close procedure 🤝
2.2.4. Network Reset 🚨
------> N_RST_INDan ODETTE-FTP entity is notified of a network error by an N_RST_IND event. it should be noted that N_RST_IND would also be generated by a peer RESETTING the connection, but this is ignored here, because N_RST_RQ is never sent to the Network Service by ODETTE-FTP 🙃
2.3. Secure ODETTE-FTP Session 🔒
[TLS] provides a mechanism for securing an ODETTE-FTP session over the Internet or a TCP network. ODETTE-FTP is layered over [TLS], and secure and unsecure traffic are told apart by using different server ports 🚪
the implementation is very simple. layer ODETTE-FTP over [TLS] the same way you'd layer ODETTE-FTP over TCP/IP. [TLS] provides both session encryption and authentication, and the connecting parties can use either or both. a party acts as a [TLS] server when receiving calls and acts as a [TLS] client when making calls. once the [TLS] handshake has completed, the responding ODETTE-FTP may start the ODETTE-FTP session by sending the Ready Message 🤝✨
2.4. Port Assignment 🧚♀️
an ODETTE-FTP requester will select a suitable local port 🔌
the responding ODETTE-FTP will listen for connections on Registered Port 3305; the service name is 'odette-ftp' 🎀
the responding ODETTE-FTP will listen for secure TLS connections on Registered Port 6619; the service name is 'odette-ftps' 🔐
3. File Transfer Service 💖
okay bestie, this is where the File Transfer Service comes in 🎀 it's basically the menu of stuff an ODETTE-FTP entity offers to its User Monitor (usually an application). so cute 🦋
NOTE: The implementation of the service primitives is an application issue. 💅
3.1. Model 🦋
the layers stack up like this, and it's giving very organized energy ✨
o-------------------o o-------------------o
| | | |
| USER MONITOR | | USER MONITOR |
| | | |
o-------------------o o-------------------o
| A | A
| | | |
F_XXX_RQ/RS | | F_XXX_IND/CF F_XXX_RQ/RS | | F_XXX_IND/CF
V | V |
o-------------------o o-------------------o
| |- - - - - - >| |
| ODETTE-FTP Entity | E-Buffer | ODETTE-FTP Entity |
| |< - - - - - -| |
o-------------------o o-------------------o
| A | A
N_XXX_RQ/RS | | N_XXX_IND/CF N_XXX_RQ/RS | | N_XXX_IND/CF
| | | |
V | V |
o---------------------------------------------------------o
| |
| N E T W O R K |
| |
o---------------------------------------------------------o
Key: E-Buffer - Exchange Buffer
F_ - File Transfer Service Primitive
N_ - Network Service Primitive3.2. Session Setup 🧚
3.2.1. Session Connection Service
okay so these diagrams show what happens between two ODETTE-FTP entities and their User Agents 💜 the vertical lines are the ODETTE-FTP entities, and the User Agents aren't drawn in (they're just not invited to this one 🙈)
| |
F_CONNECT_RQ ---->|------------|----> F_CONNECT_IND
| |
F_CONNECT_CF <----|------------|<---- F_CONNECT_RS
| |Parameters
Request Indication Response Confirm
---------------------------------------------------------------------
called-address -> same --- ----
calling-address-> same --- ----
ID1 ------------> same ID2 ------------> same
PSW1------------> same PSW2 -----------> same
mode1 ----------> mode2 ----------> mode3 ----------> same
restart1 -------> same -----------> restart2 -------> same
authentication1-> same -----------> authentication2-> same
---------------------------------------------------------------------Mode
this one specifies the file transfer capabilities of the entity sending or receiving an F_CONNECT primitive, for the whole session 🌸
Value:
- Sender-only: the entity can only send files.
- Receiver-only: the entity can only receive files.
- Both: the entity can both send and receive files.
Negotiation:
- Sender-only: Not negotiable.
- Receiver-only: Not negotiable.
- Both: Can be negotiated down to Sender-only or Receiver-only by the User Monitor or the ODETTE-FTP entity.
Request Indication Response Confirm
---------------------------------------------------------------------
Sender-only ----> Receiver-only --> Receiver-only --> Sender-only
Receiver-only --> Sender-only ----> Sender-only ----> Receiver-only
Both -----+-----> Both ----+------> Both -----------> Both
| or +------> Receiver-only --> Sender-only
| or +------> Sender-only ----> Receiver-only
|
or +-----> Receiver-only --> Receiver-only --> Sender-only
or +-----> Sender-only ----> Sender-only ----> Receiver-only
---------------------------------------------------------------------Restart
this one is about the file transfer restart capabilities of the User Monitor 🍓
Value:
- Y: the entity can restart file transfers.
- N: the entity cannot restart file transfers.
Negotiation:
Request Indication Response Confirm
---------------------------------------------------------------------
restart = Y ----> restart = Y --+-> restart = Y ----> restart = Y
or +-> restart = N ----> restart = N
restart = N ----> restart = N ----> restart = N ----> restart = N
---------------------------------------------------------------------Authentication
this one specifies the authentication requirement of the User Monitor 🔐
Value:
- Y: Authentication required.
- N: Authentication not required.
Negotiation: Not negotiable.
Request Indication Response Confirm
---------------------------------------------------------------------
auth = Y ----> auth = Y ----> auth = Y ----> auth = Y
auth = N ----> auth = N ----> auth = N ----> auth = N
---------------------------------------------------------------------3.3. File Transfer 💅
3.3.1. File Opening
the file opening flow is giving "ready, set, send" 🏁
| |
F_START_FILE_RQ ---->|------------|----> F_START_FILE_IND
| |
F_START_FILE_CF(+|-) <----|------------|<---- F_START_FILE_RS(+|-)
| |Parameters
Request Ind. RS(+) CF(+) RS(-) CF(-)
------------------------------------------------------------------
filename-------> same ---- ---- ---- ----
date-time------> same ---- ---- ---- ----
destination----> same ---- ---- ---- ----
originator-----> same ---- ---- ---- ----
rec-format-----> same ---- ---- ---- ----
rec-size ------> same ---- ---- ---- ----
file-size------> same ---- ---- ---- ----
org-file-size--> same ---- ---- ---- ----
signed-eerp----> same ---- ---- ---- ----
cipher---------> same ---- ---- ---- ----
sec-services---> same ---- ---- ---- ----
compression----> same ---- ---- ---- ----
envelope-format> same ---- ---- ---- ----
description----> same ---- ---- ---- ----
restart-pos1---> same-> restart-pos2-> same ---- ----
---- ---- ---- ---- cause ------> same
---- ---- ---- ---- retry-later-> same
------------------------------------------------------------------Notes:
- Retry-later has values "Y" or "N".
- Cause is the reason for refusing the transfer (1,..,13,99).
- Restart-pos1 not equal 0 is only valid if restart has been agreed during initial negotiation.
- Restart-pos2 is less than or equal to restart-pos1.
3.3.2. Data Regime
the data regime is where the bytes actually flow 🌈
| |
F_DATA_RQ ---->|------------|----> F_DATA_IND
| |
F_DATA_CF <----|(---CDT----)|
| |Note: Unlike other commands, where the F_XXX_CF signal is a result of a corresponding F_XXX_RS command, in this case, the local entity layer issues this signal when it is ready for the next data request. This decision is based on the current credit count and the reception of CDT (Set Credit) from the receiver. 💖
3.3.3. File Closing
closing the file is its own little moment 🦋
| |
F_CLOSE_FILE_RQ --->|------------|----> F_CLOSE_FILE_IND
| |
F_CLOSE_FILE_CF(+|-) <---|------------|<---- F_CLOSE_FILE_RS(+|-)
| |Parameters
Request Ind RS(+) CF(+) RS(-) CF(-)
---------------------------------------------------------------------
rec-count ---> same ---- ---- ---- ----
unit-count --> same ---- ---- ---- ----
---- ---- Speaker=Y ---> Speaker=N ---- ----
---- ---- Speaker=N ---> Speaker=Y ---- ----
---- ---- ---- ---- cause ---> same
---------------------------------------------------------------------in a positive Close File response (F_CLOSE_FILE_RS(+)), the current Listener can do one of two things 👑
- Set Speaker to "Yes" and become the Speaker, or
- Set Speaker to "No" and remain the Listener.
the File Transfer service makes sure the speaker setting matches what the peer user is capable of 🫶
the turn is never exchanged when the response or confirmation is negative 🙅♀️
only the Speaker is allowed to issue F_XXX_FILE_RQ primitives 💅
3.3.4. Exchanging the Turn
3.3.4.1. Initial Turn (First Speaker)
the Initiator becomes the first Speaker at the end of the Session Setup 🎀 that's when it gets F_CONNECT_CF and the Responder sends F_CONNECT_RS.
3.3.4.2. Following Turns
Rules:
- At each unsuccessful End of File, the turn is not exchanged.
- At each successful End of File, the turn is exchanged if requested by the Listener:
- The current Listener receives F_CLOSE_FILE_IND (Speaker = choice).
- If the Listener answers F_CLOSE_FILE_RS(Speaker = YES), it becomes the Speaker, the Speaker receives F_CLOSE_FILE_CF (Speaker = NO) and becomes the Listener.
- If the Listener answers F_CLOSE_FILE_RS(Speaker = NO), it remains as the Listener, and the Speaker receives F_CLOSE_FILE_CF (Speaker = YES) and remains as the Speaker.
- The Speaker can issue a Change Direction request (F_CD_RQ) to become the Listener. The Listener receives a Change Direction indication (F_CD_IND) and becomes the Speaker.
- In order to prevent loops of F_CD_RQ/IND, the Speaker may not send an F_CD_RQ after receiving an unsolicited F_CD_IND. If the Listener receives a solicited F_CD_IND as a result of sending EFPA(Speaker=Yes), it is acceptable to immediately relinquish the right to speak by sending an F_CD_RQ. 🔥
3.3.5. End to End Response ✨
okay so this one is the Speaker sending an End to End Response (EERP) 📬 back to the originator of a file, coming from the final destination. it can only happen when there's no file transfer in progress 🫶
| |
F_EERP_RQ ---->|------------|----> F_EERP_IND
| |
F_RTR_CF <----|------------|<---- F_RTR_RS
| |Parameters
Request Indication
------------------------------------
filename -----------> same
date ---------------> same
time ---------------> same
destination --------> same
originator ---------> same
hash ---------------> same
signature ----------> same
------------------------------------Relationship with Turn:
- only the Speaker can send an End to End Response request 💅
- invoking the EERP service does NOT change the turn 🔄
- if an F_CD_IND gets received right before F_EERP_RQ is issued, that gets you out of the special condition the F_CD_IND created 🦋. before, you could issue F_RELEASE_RQ but couldn't issue F_CD_RQ right after getting F_CD_IND. after you've issued F_EERP_RQ, you're back to normal Speaker status, so F_CD_RQ is valid again and F_RELEASE_RQ isn't ✨
Notes:
- the F_EERP_RQ (and also F_NERP_RQ) gets confirmed with an F_RTR_CF signal 🎀. that F_RTR_CF signal is shared by both of them. there's no confusion about which one it's for, because only one of these requests can be pending at a time 💖
- the signature is optional, and you ask for it when you send F_START_FILE_RQ 🔐
- if the EERP can't be signed, it should still get sent unsigned 🫶
- it's up to the application to check the EERP contents and its signature, and to decide what to do with an EERP that fails validation, or one that isn't signed when a signed one was asked for 🧠
3.3.6. Negative End Response
this is the Speaker sending a Negative End Response 🙅♀️ when a file couldn't get sent on to the next destination. it's only sent if the problem isn't temporary 💀
there's also a second way this can happen: the final destination can use it instead of an End to End Response when a file couldn't be processed, even though it got the file successfully 🌸
| |
F_NERP_RQ ---->|------------|----> F_NERP_IND
| |
F_RTR_CF <----|------------|----- F_RTR_RS
| |Parameters
Request Indication
---------------------------------------------------
filename ----------------------> same
date --------------------------> same
time --------------------------> same
destination -------------------> same
originator --------------------> same
creator of negative response --> same
reason ------------------------> same
reason text -------------------> same
hash --------------------------> same
signature ---------------------> same
---------------------------------------------------Relationship with Turn:
It works the same as the End-To-End response, so check out Section 3.3.5 for the details 🔁✨
3.4. Session Take Down 🌙
3.4.1. Normal Close
| |
F_RELEASE_RQ ---->|------------|----> F_RELEASE_IND
| |Parameters
Request Indication
---------------------------------------------------------------------
reason = normal -------> ----
---------------------------------------------------------------------the Release service can only be started by the Speaker 👑
the Speaker can only issue a Release request (F_RELEASE_RQ) right after getting an unsolicited Change Direction indication (F_CD_IND) 💅. this makes sure the other partner doesn't want to send any more files in this session 🫶
peer ODETTE-FTP entities handle a normal session release by putting Reason = Normal in an End Session (ESID) command 🎀
3.4.2. Abnormal Close
| |
F_RELEASE_RQ ---->|------------|----> F_ABORT_IND
| |Parameters
Request Indication
---------------------------------------------------------------------
reason = error value --> same (or equivalent)
AO (Abort Origin) = (L)ocal or (D)istant
---------------------------------------------------------------------an abnormal session release can be started by either the Speaker or the Listener, and by either the user or the provider 😭
it can happen at any point in the session 🌈
peer ODETTE-FTP entities handle an abnormal session release by putting Reason = Error-value in an End Session (ESID) command ⭐
the abnormal session release covers these kinds of errors:
- the service provider starts an abnormal release in these cases 🔥
- protocol error
- the Start Session (SSID) negotiation fails
- command not recognised
- Data Exchange Buffer size error
- resources not available
- other unspecified abort code (with Reason = unspecified)
- the User Monitor starts an abnormal release in these cases 👀
- local site emergency close down
- resources not available
- other unspecified abort code (with Reason = unspecified)
other error types can be handled by aborting the connection 🦋
3.4.3. Abort
| |
F_ABORT_RQ ---->|------------|----> F_ABORT_IND
| |
User-Initiated Abort
| |
F_ABORT_IND <----|------------|----> F_ABORT_IND
| |
Provider-Initiated AbortParameters
Request Indication
---------------------------------------------------------------------
-- R (Reason): specified or unspecified
-- AO (Abort Origin): (L)ocal or (D)istant
---------------------------------------------------------------------either entity can invoke the Abort service at any time ⏰
the service provider can start an abort when it detects an error 🚨
3.4.4. Explanation of Session Take Down Services
User | OFTP | Network | OFTP | User
---------------|------|----------------------|------|---------------
| | | |
1. Normal Release
F_RELEASE_RQ | | ESID(R=normal) | | F_RELEASE_IND
*--------------|-> ==|======================|=> --|-------------->
(R=normal) | | | |
2. User-Initiated Abnormal Release
F_RELEASE_RQ | | ESID(R=error) | | F_ABORT_IND
*--------------|-> ==|======================|=> -|-------------->
(R=error value)| | | | (R=error,AO=D)
3. Provider-Initiated Abnormal Release
F_ABORT_IND | | ESID(R=error) | | F_ABORT_IND
<--------------|-* *=|======================|=> --|-------------->
| | | |
4. User-Initiated Connection Abort
F_ABORT_RQ | | N_DISC_RQ | | F_ABORT_IND
*--------------|-> --|--------->..----------|-> --|-------------->
| | N_DISC_IND | | (R=unsp.,AO=D)
5. Provider-Initiated Connection Abort
F_ABORT_IND | | N_DISC_RQ | | F_ABORT_IND
<--------------|-* *-|--------->..----------|-> --|-------------->
(R=error,AO=L) | | N_DISC_IND | | (R=unsp.,AO=D)
Key: * Origin of command flow
F_ ---> File Transfer Service primitive
N_ ---> Network Service primitive
===> ODETTE-FTP (OFTP) protocol message3.5. Service State Automata 🤖
these state automata describe the service the way the User Monitor sees it 👀. events that cause a state transition are in lower case, and the resulting action is in UPPER CASE when there is one 💜
3.5.1. Idle State Diagram
o------------o
decision | | f_connect_ind
+-----------------| IDLE |-----------------+
| F_CONNECT_RQ | (0) | F_CONNECT_RS |
| o------------o |
V |
o-----------------o |
| | |
| I_WF_FCONNECTCF | |
| | |
o--------+--------o |
| |
| F_CONNECT_CF |
V V
o-----------------o o-----------------o
| | | |
| IDLE SPEAKER | | IDLE LISTENER |
| (1) | | (2) |
| See Speaker | | See Listener |
| State Diagram | | State Diagram |
| | | |
o-----------------o o-----------------o3.5.2. Speaker State Diagram 🎀
Okay, the speaker state diagram is where it gets spicy 🌶️ Read it top to bottom and follow the arrows like they're a group-chat drama 💅
o-----------------o o-----------------o
| IDLE LISTENER | | IDLE |
| CD_RQ just sent | | see (0) |
| see (3), Listen | | Idle |
| State Diagram | | State Diagram |
o-----------------o o-----------------o
A A
| |
decision decision
F_CD_RQ F_RELEASE_RQ
| |
o================o decision o----------o decision o---------------o
| |---------->| WAIT FOR |<----------| |
| | F_EERP_RQ | | F_EERP_RQ | |
| IDLE | | EERP/ | | IDLE |
| SPEAKER | decision | NERP | decision | SPEAKER |
| (1) |---------->| CONFIRM. |<----------| (4) |
| | F_NERP_RQ | | F_NERP_RQ | |
| | | | | |
| | | | | CD_IND |
| | f_rtr_cf | | | just received |
| |<----------| | | |
| | o----------o | |
| | | |
| | | |
o================o o---------------o
A A | |
| | | decision and P2 decision and P2 |
| | +-----------------+ +---------------------+
| | F_START_FILE_RQ | | F_START_FILE_RQ
| | V V
| | o---------------o
| | f_file_start_cf(-) | |
| +----------------------| OPENING |
| | |
| o---------------o
| |
f_file_close_cf(-) or f_start_file_cf(+)
f_file_close_cf(+) and not P1 |
| V
o---------------o o---------------o record to send o---------o
| | | |------------------>| |
| CLOSING | | DATA TRANSFER | F_DATA_RQ | NEXT |
| | | | | RECORD |
| | | | f_data_cf | |
| | | |<------------------| |
o---------------o o---------------o o---------o
| A |
| | end of file |
| +-------------------+
| F_CLOSE_FILE_RQ
| o-----------------o
| f_file_close_cf(+) and P1 | IDLE LISTENER |
+--------------------------------------------->| see (2), Listen |
| State Diagram |
Predicates: o-----------------o
P1: Positive confirmation and Speaker = YES
P2: Mode = Both or (Mode = Sender-only)3.5.3. Listener State Diagram 🦋
Now flip the script! The listener side is the mirror image, and honestly it's giving equally chaotic energy ✨ Same states, different vibes, and the predicates at the bottom are the tea 🍵
o-----------------o o-----------------o
| IDLE SPEAKER | | IDLE |
| CD_IND just | | |
| received see(4) | | see (0) |
| Speaker State | | Idle |
| Diagram | | State Diagram |
o-----------------o o-----------------o
A A
| |
decision f_eerp_ind decision
F_CD_IND +--------------+ F_RELEASE_IND
| | F_RTR_RS | |
o=================o | o-----------------o
| |<-----------+ | |
| | | |
| | f_nerp_ind | |
| |------------+ | |
| | F_RTR_RS | | |
| | | | |
| |<-----------+ | |
| IDLE LISTENER | f_eerp_ind | IDLE LISTENER |
| (2) |<-----------------------------| (3) |
| | F_RTR_RS | CD_RQ |
| | | just sent |
| | f_nerp_ind | |
| |<-----------------------------| |
| | F_RTR_RS | |
| | | |
| | f_start_file_ind | |
| | and not P1 | |
| |---------------------+ | |
o=================o F_START_FILE_RS(-) | o-----------------o
A A | A A | | |
| | | | +-----------------------+ | |
| | | | | |
| | | | f_start_file_ind and not P1 | |
| | | +--------------------------------------+ |
| | | F_START_FILE_RS(-) |
| | | |
| | | f_start_file_ind f_start_file_ind |
| | | and P1 and P1 |
| | +----------------------------+ +------------------+
| | F_START_FILE_RS(+) | | F_START_FILE_RS(+)
| | V V
| | o---------------o
| |f_close_file_ind and not P3 | |
| +----------------------------| |
| F_CLOSE_FILE_RS(+,N) | |
| | DATA |
| | TRANSFER |
| f_close_file_ind and not P2 | |-------------+
+------------------------------| | |
F_CLOSE_FILE_RS(-) | |<------------+
o---------------o F_DATA_IND
o---------------o |
| IDLESPEAKER | f_close_file_ind and P3 |
| see (1), Spkr |<--------------------------+
| State Diagram | F_CLOSE_FILE_RS(+,Y)
o---------------o
Predicates:
P1: Decision to send F_START_FILE_RS(+)
P2: Decision to send F_CLOSE_FILE_RS(+)
P3: Decision to become Speaker4. Protocol Specification 💖
4.1. Overview 🌸
ODETTE-FTP runs in five operating phases, bestie. Here they are, in order:
Start Session
Start File
Data Transfer
End File
End SessionOnce an ODETTE-FTP entity finishes the End File phase, it gets to pick: it can start a brand new Start File phase, or it can wrap things up with the End Session phase 🧚
ODETTE-FTP peers chat by sending and receiving messages inside Exchange Buffers, using the Network Service. Each Exchange Buffer carries exactly one of these commands:
SSRM Start Session Ready Message
SSID Start Session
SECD Security Change Direction
AUCH Authentication Challenge
AURP Authentication Response
SFID Start File
SFPA Start File Positive Answer
SFNA Start File Negative Answer
DATA Data
CDT Set Credit
EFID End File
EFPA End File Positive Answer
EFNA End File Negative Answer
ESID End Session
CD Change Direction
EERP End to End Response
NERP Negative End Response
RTR Ready To ReceiveThe rest of this section walks through the protocol flows. Heads up: Section five is where the command formats get the full treatment 📜
4.2. Start Session Phase ✨
The Start Session phase kicks in right after the network connection is up and running 🔥
4.2.1. Entity Definition 👑
Whichever ODETTE-FTP entity took the initiative to set up the network connection becomes the Initiator. Its peer becomes the Responder. Simple as that! 💅
4.2.2. Protocol Sequence 🦋
The first message must be sent by the Responder.
1. Initiator <-------------SSRM -- Responder Ready Message
-- SSID ------------> Identification
<------------ SSID -- Identification4.2.3. Secure Authentication 🔐
Once the SSIDs have been swapped, the Initiator may optionally kick off an authentication phase, where each party proves who they are to the other. Main character security energy 😤🛡️
4.2.4. Protocol Sequence 🌈
The first authentication message must be sent by the Initiator.
1. Initiator -- SECD ------------> Responder Change Direction
<------------ AUCH -- Challenge
-- AURP ------------> Response
<------------ SECD -- Change Direction
-- AUCH ------------> Challenge
<------------ AURP -- ResponseThe Initiator starts things off by sending a Security Change Direction (SECD), and the Responder replies with an Authentication Challenge (AUCH) 🎀
The Responder looks up the public certificate linked to the Initiator's claimed identity (the one sitting in the SSID). If it can't find a suitable certificate, authentication fails, and that's the end of that 💀 Otherwise, the Responder uses the public key from that certificate to encrypt a random challenge, which is unique for each session, and sends it to the Initiator. This encrypted challenge rides along as a [CMS] envelope inside the AUCH 🌸
The Initiator decrypts the challenge with their private key, then sends the decrypted challenge back to the Responder in the Authentication Response (AURP) 🔑
The Responder double-checks that the data in the AURP matches the random challenge it sent. If it matches, the Initiator has authenticated successfully, and the Responder replies with a Security Change Direction (SECD) to start the mirror-image process of verifying the Responder to the Initiator. If the data doesn't match, the Initiator fails authentication 🚫
4.3. Start File Phase 🧚
4.3.1. Entity Definition 💜
The Initiator from the Start Session phase is designated the Speaker, and the Responder becomes the Listener. The roles can flip, though! The Speaker sends a Change Direction command to the Listener to swap them 🔄✨
4.3.2. Protocol Sequence
Okay so this part is the Start File handshake, and it's giving very "yes or no, and if no, go back and try again" 💅. The Speaker sends a Start File command, and the Listener answers YES or NO. Let's look at the flow:
1. Speaker -- SFID ------------> Listener Start File
<------------ SFPA -- Answer YES
2. Speaker -- SFID ------------> Listener Start File
<------------ SFNA -- Answer NO
Go To 1
Note: The User Monitor should take steps to prevent a loop
situation occurring.
2. Speaker -- CD --------------> Listener Change Direction
Listener <------------ EERP -- Speaker End to End Response
-- RTR -------------> Ready to Receive
<------------ NERP -- Negative End Response
-- RTR -------------> Ready to Receive
<------------ SFID -- Start FileNote that the User Monitor should take steps to prevent a loop situation occurring 🔁. Literally, nobody wants to be stuck in an infinite NO loop, bestie 🙈.
4.3.3. Restart Facilities
Okay, restart facilities are lowkey so useful 🎀. The Start File command includes a count that lets an interrupted transmission restart, and the two sides negotiate it. If restart facilities are not available, the restart count MUST be set to zero. The sender will start with the lowest record count + 1 🦋.
4.3.4. Broadcast Facilities
The destination in a Start File command can be specified in one of two ways 🌸:
- An explicitly defined destination.
- A group destination that allows an intermediate location to broadcast the Virtual File to multiple destinations.
The Listener will send a negative answer to the Speaker when the destination is not known 🙅♀️.
4.3.5. Priority
The prioritisation of files for transmission is left to the local implementation, so each system decides for itself 💜. To allow some flexibility, a change direction mechanism is available in the End File phase 🔄.
4.3.6. End to End Response (EERP)
The End to End Response (EERP) command tells the originator of a Virtual File that the file has been successfully delivered to its final destination ✨. This lets the originator do housekeeping, like deleting copies of the delivered data 🧹.
If the originator requested a signed EERP in the SFID, the EERP MUST be signed. Signing lets the originator prove that the EERP was generated by the final destination 🔏. If the final destination is unable to sign the EERP, it MAY send back an unsigned EERP. It is an implementation issue to allow the acceptance of an unsigned EERP if a signed EERP is requested 🤷♀️.
A Response Command MUST be sent from the location performing the final processing or distribution of the data to the originator. The Response is mandatory and MAY be sent in the same or in any subsequent session 📬.
When an intermediate location broadcasts or distributes a Virtual File, it MUST receive a Response command from all the locations to which it forwarded the data before sending its own Response. This makes sure the Response received by the originator accounts for all the destination locations 🫶. An intermediate location therefore needs to track the status of files it processes over time ⏳.
Requesting a signed EERP is incompatible with the use of broadcast facilities, because an EERP can be signed by only one destination 🙃. If this scenario occurs, the intermediate broadcast location MAY continue and ignore the request for a signed EERP, or send back a NERP.
Example: Point to Point
Location A sends file Ba to location B, which will send an EERP to
location A after it successfully receives the file.
o----------o o-----------o
| Loc. A |----------- S1 ---------->| Loc. B |
| | | |
| [Ba] |<---------- R2 -----------| [Ba] |
+----------o o-----------o
Key: S - File Transfer
R - Response EERP
[Ba] - File for B from AExample: Data distribution
Location A sends a Virtual File containing data for distribution
to locations B and C via clearing centres E1 and E2. Clearing
centre E1 must wait for a response from E2 (for file Ba) and
location C before it sends its response, R8, to location A.
Clearing centre E2 can only send response R7 to E1 when location B
acknowledges file Ba with response R6.
o---------o o---------o o---------o o---------o
| Loc. A |-- S1 ->| Loc. E1 |-- S2 ->| Loc. E2 |-- S5 ->| Loc. B |
| | | | | | | |
| [Ba,Ca] |<- R8 --| [Ba,Ca] |<- R7 --| [Ba] |<- R6 --| [Ba] |
o---------o o---------o o---------o o---------o
A |
| | o---------o
| +----- S3 ->| Loc. C |
| | |
+--------- R4 --| [Ca] |
o---------oExample: Data collection
Locations A and B send files Ca and Cb to clearing centre E1,
which forwards both files to location C in a single Virtual File.
When it receives response R4 from C, clearing centre E1 sends
response R5 to location A and R6 to location B.
o---------o o---------o o---------o
| Loc. A |-- S1 ->| Loc. E1 |-- S3 ->| Loc. C |
| | | | | |
| [Ca] |<- R5 --| [Ca,Cb] |<- R4 --| [Ca,Cb] |
o---------o o---------o o---------o
A |
o---------o | |
| Loc. B |-- S2 -----+ |
| | |
| [Cb] |<- R6 ---------+
o---------o4.3.7. Negative End Response (NERP)
Okay so the EERP is the good news, and the NERP is the "sorry, that didn't work" news 💀. A Negative End Response says that a file could not be delivered to the final destination, or that the final destination could not process the received file 😭.
It MAY be created by an intermediate node that could not transmit the file any further because the next node refuses to accept it. The cause of the refusal has to be non-temporary, otherwise the intermediate node has to try the transmission again 🔁.
It MAY also be created by the final node that is unable to process the file because of non-recoverable syntax or semantic errors in the file, or because of the failure of any other processing performed on the file 🧐.
The NERP will be sent back to the originator of the file 📮.
The parameters are equal to the ones of the EERP, but with extra information about the creator of the NERP and the abort reason 📋. When the NERP is created due to a failure to transmit, the abort reason is taken from the refusal reason that was sent by the node refusing the file. Because of the NERP, the intermediate node can stop trying to send the non-deliverable file and delete it 🗑️.
The NERP lets the originator react to the unsuccessful transmission or processing, depending on the reason code and the creator of the NERP 🧠.
If the originator requested a signed EERP in the SFID, the NERP MUST be signed. Signing lets the originator prove by whom the NERP was generated 🔏. If the location generating the NERP is unable to sign the NERP, it MAY send back an unsigned NERP. It is an implementation issue to allow the acceptance of an unsigned EERP if a signed NERP is requested 🤷♀️.
4.3.8. Ready To Receive Command (RTR)
So the RTR is the flow control queen 👑. In order to avoid congestion between two adjacent nodes caused by a continuous flow of EERPs and NERPs, a Ready To Receive (RTR) command is provided. The RTR acts as an EERP/NERP acknowledgement for flow control, but it has no end-to-end significance 🦋.
Speaker -- EERP ------------> Listener End to End Response
<------------- RTR -- Ready to Receive
-- EERP ------------> End to End Response
<------------- RTR -- Ready to Receive
-- NERP ------------> Negative End Response
<------------- RTR -- Ready to Receive
-- SFID ------------> Start File
or
-- CD --------------> Exchange the turnAfter sending an EERP or NERP, the Speaker MUST wait for an RTR before sending any other commands. The only acceptable commands to follow are:
EERP
NERP
SFID or CD (if there are no more EERPs or NERPs to be sent)4.4. Data Transfer Phase
Virtual File data flows from the Speaker to the Listener during the Data Transfer phase, which is entered after the Start File phase 📦✨.
4.4.1. Protocol Sequence
To avoid congestion at the protocol level, a flow control mechanism is provided via the Set Credit (CDT) command 🚦.
A Credit limit is negotiated in the Start Session phase. This represents the number of Data Exchange Buffers that the Speaker MAY send before it is obliged to wait for a Credit command from the Listener 🎟️.
The available credit is initially set to the negotiated value by the Start File positive answer, which acts as an implicit Credit command. The Speaker decreases the available credit count by one for each data buffer sent to the Listener 📉.
When the available credit is exhausted, the Speaker MUST wait for a Credit command from the Listener. Otherwise, a protocol error will occur and the session will be aborted 🚨.
The Listener SHOULD endeavour to send the Credit command without delay to prevent the Speaker blocking ⏩.
1. Speaker -- SFID ------------> Listener Start File
<------------ SFPA -- Answer YES
2. If the credit value is set to 2
Speaker -- Data ------------> Listener Start File
-- Data ------------>
<------------- CDT -- Set Credit
-- Data ------------>
-- EFID ------------> End File4.5. End File Phase
4.5.1. Protocol Sequence
The Speaker notifies the Listener that it has finished sending a Virtual File by sending an End File (EFID) command. The Listener replies with a positive or negative End File command, and it has the option to request a Change Direction command from the Speaker 🙌.
1. Speaker -- EFID ------------> Listener End File
<------------ EFPA -- Answer YES
2. Speaker -- EFID ------------> Listener End File
<------------ EFPA -- Answer YES + CD
-- CD --------------> Change Direction
Listener <------------ EERP -- Speaker End to End Response
-------------- RTR -> Ready to Receive
Listener <------------ NERP -- Speaker Negative End Response
-------------- RTR -> Ready to Receive
Go to Start File Phase
3. Speaker -- EFID ------------> Listener End File
<------------ EFNA -- Answer NO4.6. End Session Phase
4.6.1. Protocol Sequence
The Speaker terminates the session by sending an End Session (ESID) command 🎬. The Speaker MAY only do this if the Listener has just relinquished its role as speaker 🫡.
1. Speaker -- EFID ------------> Listener End File
<------------ EFPA -- Answer YES
-- CD --------------> Change Direction
Listener <------------ ESID -- Speaker End Session4.7. Problem Handling ✨
okay so bestie, error handling should happen as close to the problem as possible 🔍 That makes it way easier to figure out what went wrong and fix it. Every layer of the reference model handles its own errors, no sharing the blame 💅
ODETTE-FTP can spot protocol errors just from its state machine 🧠 It also uses activity timers to catch sessions that hang 😭 Those mechanisms are totally separate from the End to End controls, so don't mix them up!!
4.7.1. Protocol Errors
ngl this one is serious 💀 If a protocol error happens, the session gets terminated and any application activity gets aborted. Both locations drop back to the IDLE state. Reset, no drama 🫶
4.7.2. Timers
To protect against application and network hangs, ODETTE-FTP uses activity timers in every situation where a response is required ⏰ The timers, plus what to do when they expire, are all covered in Section 9, "Protocol State Machine" 🦋
4.7.3. Clearing Centres
Clearing centres are a whole other vibe 🏢 Using them means errors can pop up from the data processing that happens inside the centre. Those errors aren't directly tied to ODETTE-FTP or the communication network, so they're outside the scope of this specification 🙈
5. Commands and Formats 🎀
ODETTE-FTP entities talk to each other through Exchange Buffers 💌 The Command Exchange Buffers are explained below. Virtual File data rides in Data Exchange Buffers instead, and those get their own section, Section 7 🧚
5.1. Conventions
5.1.1. Representation Unit
The basic unit of information is an octet, which is 8 bits. Simple, iconic 🌸
5.1.2. Values and Characters
Constants and strings inside Command Exchange Buffers are encoded with the ISO 646 IRV 7-bit coded character set [ISO-646], following Appendix B 📖 The one exception is any field that explicitly says [UTF-8] next to it.
5.2. Commands
Each Command Exchange Buffer holds exactly one command, starting right at the beginning of the buffer 💖 Commands and data are never mixed in the same Exchange Buffer, and commands can't be compressed (no shortcuts, bestie). Variable-length parameters can be left out completely if they aren't needed, as long as their length indicator field is set to zero 🍓
Components:
- Command identifier:
The first octet of an Exchange Buffer is the Command Identifier, and it decides the format of the buffer 👑
- Parameter(s):
Command parameters live in fields inside a Command Exchange Buffer 🧩 Any variable-length field gets a header field in front of it that says how long it is. All values are required, except where it's explicitly said otherwise ✨
5.3. Command Formats
Here's how the ODETTE-FTP commands are described, using the definitions below 🦋
Position (Pos): the field offset inside the Command Exchange Buffer, counted from a zero origin.Field: the name of the field.Description: what the field is about.Format: how the field is shaped, which is one of these:F: a field with fixed values. Every allowed value is listed in the command definition.V: a field with variable values inside a defined range. For example, the SFIDLRECL field can hold any integer from 00000 to 99999.X(n): an alphanumeric field that's n octets long.9(n): a numeric field that's n octets long.U(n): a binary field that's n octets long. Binary numbers are always unsigned and in network byte order 🤓T(n): a field that's n octets long, encoded using [UTF-8].
A string can only use alphanumeric characters from this set:
- The numerals:
0to9 - The upper case letters:
AtoZ - The following special set:
/ - . & ( )and space
Space is not allowed as an embedded character, so no sneaking it into the middle 🙅♀️
String and alphanumeric fields are always left justified and right padded with spaces where needed. Numeric fields are always right justified and left padded with zeros where needed. Reserved fields should be padded with spaces 💅
5.3.1. SSRM - Start Session Ready Message
This is the SSRM, the Start Session Ready Message 🚀 It shows up in the Start Session Phase, going from Responder to Initiator 🌈
o-------------------------------------------------------------------o
| SSRM Start Session Ready Message |
| |
| Start Session Phase Initiator <---- Responder |
|-------------------------------------------------------------------|
| Pos | Field | Description | Format |
|-----+-----------+---------------------------------------+---------|
| 0 | SSRMCMD | SSRM Command, 'I' | F X(1) |
| 1 | SSRMMSG | Ready Message, 'ODETTE FTP READY ' | F X(17) |
| 18 | SSRMCR | Carriage Return | F X(1) |
o-------------------------------------------------------------------oSSRMCMD Command Code Character
Value: 'I', the SSRM Command identifier 🆔
SSRMMSG Ready Message String(17)
Value: 'ODETTE FTP READY ' 💜
SSRMCR Carriage Return Character
Value: a character with hex value '0D' or '8D' 🫶
5.3.2. SSID - Start Session ✨
Okay bestie, this is the big one 💅 SSID is the first thing the two sides say to each other once a Start Session phase begins, so it's basically the handshake of handshakes 🤝 Here's the whole layout:
o-------------------------------------------------------------------o
| SSID Start Session |
| |
| Start Session Phase Initiator <---> Responder |
|-------------------------------------------------------------------|
| Pos | Field | Description | Format |
|-----+-----------+---------------------------------------+---------|
| 0 | SSIDCMD | SSID Command 'X' | F X(1) |
| 1 | SSIDLEV | Protocol Release Level | F 9(1) |
| 2 | SSIDCODE | Initiator's Identification Code | V X(25) |
| 27 | SSIDPSWD | Initiator's Password | V X(8) |
| 35 | SSIDSDEB | Data Exchange Buffer Size | V 9(5) |
| 40 | SSIDSR | Send / Receive Capabilities (S/R/B) | F X(1) |
| 41 | SSIDCMPR | Buffer Compression Indicator (Y/N) | F X(1) |
| 42 | SSIDREST | Restart Indicator (Y/N) | F X(1) |
| 43 | SSIDSPEC | Special Logic Indicator (Y/N) | F X(1) |
| 44 | SSIDCRED | Credit | V 9(3) |
| 47 | SSIDAUTH | Secure Authentication (Y/N) | F X(1) |
| 48 | SSIDRSV1 | Reserved | F X(4) |
| 52 | SSIDUSER | User Data | V X(8) |
| 60 | SSIDCR | Carriage Return | F X(1) |
o-------------------------------------------------------------------oSSIDCMD: Command Code Character 🎀
Value: 'X' SSID Command identifier.
SSIDLEV: Protocol Release Level (Numeric(1)) 🌸
This one says which level of the ODETTE-FTP protocol you're speaking.
Value:
'1'for Revision 1.2'2'for Revision 1.3'4'for Revision 1.4'5'for Revision 2.0
Future release levels will have higher numbers. The release level is negotiable, and the lowest level gets selected ✨
Note: ODETTE File Transfer Protocol 1.3 (RFC 2204) specifies '1' for the release level, even though it follows revision 1.3. Yes, it's a plot twist 🙈
SSIDCODE: Initiator's Identification Code (String(25)) 🪪
Format: See Identification Code (Section 5.4)
This uniquely identifies the Initiator (sender) in the ODETTE-FTP session. It's an application implementation issue to link the expected [X.509] certificate to the SSIDCODE that was provided, so that's on the app devs 🫶
SSIDPSWD: Initiator's Password (String(8)) 🔑
This is the key used to authenticate the sender. It's assigned by bilateral agreement, so the two parties sort it out between themselves.
SSIDSDEB: Data Exchange Buffer Size (Numeric(5)) 📦
Minimum: 128 Maximum: 99999
This is the length, in octets, of the largest Data Exchange Buffer the location can accept. The length includes the command octet but does not include the Stream Transmission Header. After negotiation, the smallest size will be selected 👑
SSIDSR: Send / Receive Capabilities (Character) 🔄
Value:
'S'Location can only send files.'R'Location can only receive files.'B'Location can both send and receive files.
Sending and receiving get serialised during the session, so parallel transmissions won't happen in the same session. Also, an error occurs if adjacent locations both specify the send capability or both specify the receive capability. Main character energy means not being the same as your neighbour 💅
SSIDCMPR: Buffer Compression Indicator (Character) 🗜️
Value:
'Y'The location can handle OFTP data buffer compression.'N'The location cannot handle OFTP buffer compression.
Compression is only used if both locations support it ✨
The compression mechanism here applies to each individual OFTP data buffer. That's different from the file compression mechanism in OFTP, which compresses whole files. Don't mix them up, bestie 🙃
SSIDREST: Restart Indicator (Character) 🔁
Value:
'Y'The location can handle the restart of a partially transmitted file.'N'The location cannot restart a file.
SSIDSPEC: Special Logic Indicator (Character) 🧩
Value:
'Y'Location can handle Special Logic.'N'Location cannot handle Special Logic.
Special Logic is only used if both locations support it. These extensions are only useful for accessing an X.25 network through an asynchronous entry, and they are not supported for TCP/IP connections. So, not for us, TCP/IP friends 🌈
SSIDCRED: Credit (Numeric(3)) 💳
Maximum: 999
This is the number of consecutive Data Exchange Buffers the Speaker can send before it has to wait for a Credit (CDT) command from the Listener. The credit value only applies to data flow in the Data Transfer phase.
The Speaker's available credit gets initialised to SSIDCRED when it receives a Start File Positive Answer (SFPA) command from the Listener. It's zeroed by the End File (EFID) command 🧮
After negotiation, the smallest size must be selected in the answer of the Responder. If it isn't, a protocol error will abort the session. This is the negotiation of the "credit-window-size" parameter:
Window Size m -- SSID ------------>
<------------ SSID -- Window Size n
(n less than or
equal to m)Note: the negotiated value will be "n". Respect the window 🪟
SSIDAUTH: Secure Authentication (Character) 🔐
Value:
'Y'The location requires secure authentication.'N'The location does not require secure authentication.
Secure authentication is only used if both locations agree to it. If the Responder's answer doesn't match the Initiator's authentication requirements, the Initiator must abort the session. No negotiation of authentication is allowed, so it's all or nothing 🙅♀️
authentication p -- SSID ------------>
<------------ SSID -- authentication q
p == q -> continue.
p != q -> abort.SSIDRSV1: Reserved (String(4)) 🚧
This field is reserved for future use. Hands off, it's not yours yet 🤚
SSIDUSER: User Data (String(8)) 🧸
This can be used by ODETTE-FTP in any way. If it's unused, it should be initialised to spaces. The two sides are expected to have a bilateral agreement about what the data means ✨
SSIDCR: Carriage Return (Character) ↩️
Value: Character with hex value '0D' or '8D'.
5.3.3. SFID - Start File ✨
okay bestie, this is the SFID command 💌 It's the one that describes the Virtual File you're about to send, so the other side knows what's coming and how to handle it. Here's the layout, and then we'll go through every field 🫶
o-------------------------------------------------------------------o
| SFID Start File |
| |
| Start File Phase Speaker ----> Listener |
|-------------------------------------------------------------------|
| Pos | Field | Description | Format |
|-----+-----------+---------------------------------------+---------|
| 0 | SFIDCMD | SFID Command, 'H' | F X(1) |
| 1 | SFIDDSN | Virtual File Dataset Name | V X(26) |
| 27 | SFIDRSV1 | Reserved | F X(3) |
| 30 | SFIDDATE | Virtual File Date stamp, (CCYYMMDD) | V 9(8) |
| 38 | SFIDTIME | Virtual File Time stamp, (HHMMSScccc) | V 9(10) |
| 48 | SFIDUSER | User Data | V X(8) |
| 56 | SFIDDEST | Destination | V X(25) |
| 81 | SFIDORIG | Originator | V X(25) |
| 106 | SFIDFMT | File Format (F/V/U/T) | F X(1) |
| 107 | SFIDLRECL | Maximum Record Size | V 9(5) |
| 112 | SFIDFSIZ | File Size, 1K blocks | V 9(13) |
| 125 | SFIDOSIZ | Original File Size, 1K blocks | V 9(13) |
| 138 | SFIDREST | Restart Position | V 9(17) |
| 155 | SFIDSEC | Security Level | F 9(2) |
| 157 | SFIDCIPH | Cipher suite selection | F 9(2) |
| 159 | SFIDCOMP | File compression algorithm | F 9(1) |
| 160 | SFIDENV | File enveloping format | F 9(1) |
| 161 | SFIDSIGN | Signed EERP request | F X(1) |
| 162 | SFIDDESCL | Virtual File Description length | V 9(3) |
| 165 | SFIDDESC | Virtual File Description | V T(n) |
o-------------------------------------------------------------------oSFIDCMD: Command Code (Character) 🔤 This is the SFID command identifier, and its value is 'H'.
SFIDDSN: Virtual File Dataset Name (String(26)) 📁 This is the dataset name of the Virtual File being transferred, and it's assigned by bilateral agreement. No general structure is defined for it, so you can check out the details in Virtual Files - Identification (Section 1.5.2) 💜
SFIDRSV1: Reserved (String(3)) 🚧 This one's reserved for future use, so it's just sitting there being mysterious for now ✨
SFIDDATE: Virtual File Date stamp (Numeric(8)) 📅 The format is 'CCYYMMDD', which is 8 decimal digits covering the century, year, month, and day. The Originator puts this stamp on the file to show when it was made available for transmission. See Section 1.5.2 for more.
SFIDTIME: Virtual File Time stamp (Numeric(10)) ⏰ The format is 'HHMMSScccc', so 10 decimal digits for hours, minutes, seconds, and a counter (0001-9999) that gives higher resolution. Just like the date, the Originator sets it to show when the file was made available for transmission. See Section 1.5.2 🦋
SFIDUSER: User Data (String(8)) 🎀 ODETTE-FTP can use this field in any way it likes. If it's not being used, it should be initialised to spaces. It's also expected that a bilateral agreement exists about what the data means.
SFIDDEST: Destination (String(25)) 🗺️ The format is covered in the Identification Code (Section 5.4). This is the Final Recipient of the Virtual File. It's the location that looks into the Virtual File content and does the mapping functions. It's also the location that creates the End to End Response (EERP) command for the received file. Main character energy 👑
SFIDORIG: Originator (String(25)) 🌱 The format is also in the Identification Code (Section 5.4). This is the Originator of the Virtual File, meaning the location that created (mapped) the data for transmission.
SFIDFMT: File Format (Character) 🧩 This tells you the Virtual File's format, and the values are:
'F': Fixed format binary file'V': Variable format binary file'U': Unstructured binary file'T': Text
It's used to calculate the restart position (Section 1.5.4). Once a file has been signed, compressed, and/or encrypted, it becomes unstructured in file format terms, which means format U. The record boundaries can't be seen anymore until the file is decrypted, decompressed, and/or verified. In that case, the SFID File Format Field shows the format of the original file, but the transmitted file must be treated as U format 🔐
SFIDLRECL: Maximum Record Size (Numeric(5)) 📏 The maximum is 99999. This is the length in octets of the longest logical record that may be transferred to a location, and only user data is counted.
If SFIDFMT is 'T' or 'U', then this attribute must be set to '00000'.
If SFIDFMT is 'V' and the file is compressed, encrypted, or signed, then the maximum value of SFIDRECL is '65536' 💅
SFIDFSIZ: Transmitted File Size (Numeric(13)) 📦 The maximum is 9999999999999. This is the space, in 1K (1024 octet) blocks, that the Originator location needs to store the actual Virtual File that's getting transmitted. For example, if a file is compressed before sending, this is the space needed for the compressed file. It's only meant to give a good estimate of the Virtual File size. Using 13 digits allows a maximum file size of about 9.3 PB (petabytes) to be transmitted. Wild 🌈
SFIDOSIZ: Original File Size (Numeric(13)) 🗂️ The maximum is 9999999999999. This is the space, in 1K (1024 octet) blocks, that the Originator needs to store the original before it's signed, compressed, and/or encrypted.
If no security or compression services were used, SFIDOSIZ should contain the same value as SFIDFSIZ. If the original file size isn't known, the value zero should be used. This is only a good estimate of the original file size.
Here's the sequence of events in a file exchange:
(a) raw data file ready to be sent
SFIDOSIZ = Original File Size
(b) signing/compression/encryption
(c) transmission
SFIDFSIZ = Transmitted File Size
(d) decryption/decompression/verification
(e) received raw data file for in-house applications
SFIDOSIZ = Original File SizeSo the Transmitted File Size at (c) tells the receiver how much storage it needs to receive the file, and the Original File Size at (e) tells the in-house application how much storage it needs to process the file 🧠
SFIDREST: Restart Position (Numeric(17)) 🔁 The maximum is 99999999999999999. This is the Virtual File restart position. The count represents:
- The Record Number if SSIDFMT is
'F'or'V'. - The file offset in 1K (1024 octet) blocks if SFIDFMT is
'U'or'T'.
The count expresses the transmitted user data, which means before ODETTE-FTP buffer compression, and the header isn't included. After negotiation between adjacent locations, retransmission starts at the lowest value.
Once a file has been signed, compressed, and/or encrypted, it has become unstructured, like format U. The file should be treated as format U for restart purposes, no matter what the actual value in SFIDFMT says 🫠
SFIDSEC: Security Level (Numeric(2)) 🔒 The values are:
'00': No security services'01': Encrypted'02': Signed'03': Encrypted and signed
This shows whether the file was signed and/or encrypted before transmission. See Section 6.2 💜
SFIDCIPH: Cipher suite selection (Numeric(2)) 🔑 The values are:
'00': No security services'01': See Section 10.2
This indicates the cipher suite used to sign and/or encrypt the file. It also shows the cipher suite that should be used when a signed EERP or NERP is requested 🫶
SFIDCOMP: File compression algorithm (Numeric(1)) 🗜️ The values are:
'0': No compression'1': Compressed with [ZLIB] algorithm
This indicates the algorithm used to compress the file. See Section 6.4 🌸
SFIDENV: File enveloping format (Numeric(1)) 💌 The values are:
'0': No envelope'1': File is enveloped using [CMS]
This indicates the enveloping format used in the file. If the file is encrypted, signed, or compressed, or if it's an enveloped file for the exchange and revocation of certificates, this field must be set accordingly ✨
SFIDSIGN: Signed EERP request (Character) ✍️ The values are:
'Y': The EERP returned in acknowledgement of the file must be signed'N': The EERP must not be signed
This requests whether the EERP returned for the file must be signed. Iconic behaviour 🦋
SFIDDESCL: Virtual File Description length (Numeric(3)) 📝 This is the length in octets of the field SFIDDESC. A value of 0 means that no description is present.
SFIDDESC: Virtual File Description ([UTF-8](n)) 🌍 ODETTE-FTP can use this field in any way it likes. If it's not used, SFIDDESCL should be set to zero. There's no general structure defined for this attribute, but a bilateral agreement is expected to exist about what the data means. It's encoded using [UTF-8] so it can support a range of national languages, and the maximum length of the encoded value is 999 octets 🎀
5.3.4. SFPA - Start File Positive Answer ✨
okay so this one is the Listener's "yesss, send it" reply 💖 It goes back to the Speaker during the Start File Phase, and it's the positive answer to the SFID.
o-------------------------------------------------------------------o
| SFPA Start File Positive Answer |
| |
| Start File Phase Speaker <---- Listener |
|-------------------------------------------------------------------|
| Pos | Field | Description | Format |
|-----+-----------+---------------------------------------+---------|
| 0 | SFPACMD | SFPA Command, '2' | F X(1) |
| 1 | SFPAACNT | Answer Count | V 9(17) |
o-------------------------------------------------------------------oSFPACMD is the command code, a character 🦋 Its value is '2', which is the SFPA command identifier.
SFPAACNT is the answer count, a Numeric(17) field 🌸 The Listener must enter a count that is lower than or equal to the restart count the Speaker put in the Start File (SFID) command. The count tells the Speaker how much user data was received. If restart facilities aren't available, the Listener must put in a count of 0 🫶
5.3.5. SFNA - Start File Negative Answer 🙈
this is the "nah, not today" message, and it comes with receipts 💅 Same deal, it's a Listener-to-Speaker message in the Start File Phase.
o-------------------------------------------------------------------o
| SFNA Start File Negative Answer |
| |
| Start File Phase Speaker <---- Listener |
|-------------------------------------------------------------------|
| Pos | Field | Description | Format |
|-----+-----------+---------------------------------------+---------|
| 0 | SFNACMD | SFNA Command, '3' | F X(1) |
| 1 | SFNAREAS | Answer Reason | F 9(2) |
| 3 | SFNARRTR | Retry Indicator, (Y/N) | F X(1) |
| 4 | SFNAREASL | Answer Reason Text Length | V 9(3) |
| 7 | SFNAREAST | Answer Reason Text | V T(n) |
o-------------------------------------------------------------------oSFNACMD is the command code, a character. Its value is '3', the SFNA command identifier.
SFNAREAS is the answer reason, a Numeric(2) field that tells you why the transmission can't go ahead 😭 Here are the possible values:
'01'Invalid filename.'02'Invalid destination.'03'Invalid origin.'04'Storage record format not supported.'05'Maximum record length not supported.'06'File size is too big.'10'Invalid record count.'11'Invalid byte count.'12'Access method failure.'13'Duplicate file.'14'File direction refused.'15'Cipher suite not supported.'16'Encrypted file not allowed.'17'Unencrypted file not allowed.'18'Compression not allowed.'19'Signed file not allowed.'20'Unsigned file not allowed.'99'Unspecified reason.
That's the reason why the transmission can't proceed.
SFNARRTR is the retry indicator, a character that's either 'N' or 'Y' 🍓
'N'means the transmission should not be retried.'Y'means the transmission may be retried later.
This one is basically the Listener saying whether the Speaker should try again later because of a temporary problem on the Listener's side, like running out of storage space. It should be used together with the Answer Reason code (SFNAREAS) ✨
An invalid file name error can be the result of a problem when the Virtual File gets mapped onto a real file, and those problems can't always get fixed right away 🧚 That's why it's recommended that when an SFNA with Retry = Y shows up, the User Monitor tries to resend the relevant file in a later session. Bestie, be patient 💜
SFNAREASL is the answer reason text length, a Numeric(3) field. It's the length in octets of the SFNAREAST field. A value of 0 means no SFNAREAST field follows.
SFNAREAST is the answer reason text, written in [UTF-8]. It's the reason the transmission can't proceed, in plain text 🎀 The encoded reason can be at most 999 octets long. No general structure is defined for this attribute, so it's just free text, no cap.
5.3.6. DATA - Data Exchange Buffer 🌈
this is where the actual payload lives, the main character of the whole transfer 👑 It's sent from the Speaker to the Listener during the Data Transfer Phase.
o-------------------------------------------------------------------o
| DATA Data Exchange Buffer |
| |
| Data Transfer Phase Speaker ----> Listener |
|-------------------------------------------------------------------|
| Pos | Field | Description | Format |
|-----+-----------+---------------------------------------+---------|
| 0 | DATACMD | DATA Command, 'D' | F X(1) |
| 1 | DATABUF | Data Exchange Buffer payload | V U(n) |
o-------------------------------------------------------------------oDATACMD is the command code, a character. Its value is 'D', the DATA command identifier.
DATABUF is the Data Exchange Buffer payload, a Binary(n) field 🫧 It's a variable-length buffer holding the data payload. The Data Exchange Buffer is described in Section 7, so go check that out too ✨
5.3.7. CDT - Set Credit 💸
this one sets the credit during the Data Transfer Phase, going from the Listener back to the Speaker 🌸
o-------------------------------------------------------------------o
| CDT Set Credit |
| |
| Data Transfer Phase Speaker <---- Listener |
|-------------------------------------------------------------------|
| Pos | Field | Description | Format |
|-----+-----------+---------------------------------------+---------|
| 0 | CDTCMD | CDT Command, 'C' | F X(1) |
| 1 | CDTRSV1 | Reserved | F X(2) |
o-------------------------------------------------------------------oCDTCMD is the command code, a character. Its value is 'C', the CDT command identifier.
CDTRSV1 is a Reserved String(2) field. It's reserved for future use, so just vibe with it for now 🫶
5.3.8. EFID - End File 🏁
this is the "that's a wrap" message 🎬 It's sent by the Speaker to the Listener in the End File Phase, and it tells the Listener exactly how big the file was.
o-------------------------------------------------------------------o
| EFID End File |
| |
| End File Phase Speaker ----> Listener |
|-------------------------------------------------------------------|
| Pos | Field | Description | Format |
|-----+-----------+---------------------------------------+---------|
| 0 | EFIDCMD | EFID Command, 'T' | F X(1) |
| 1 | EFIDRCNT | Record Count | V 9(17) |
| 18 | EFIDUCNT | Unit Count | V 9(17) |
o-------------------------------------------------------------------oEFIDCMD is the command code, a character. Its value is 'T', the EFID command identifier.
EFIDRCNT is the record count, a Numeric(17) field with a maximum of 99999999999999999 🔢 For SSIDFMT 'F' or 'V', it's the exact record count. For SSIDFMT 'U' or 'T', it's zeros. The count expresses the real size of the file, measured before buffer compression, and the header isn't included. The total count is always used, even during restart processing ✨
EFIDUCNT is the unit count, a Numeric(17) field with a maximum of 99999999999999999 📏 It's the exact number of units (octets) transmitted. The count expresses the real size of the file, and the total count is always used, even during restart processing 💖
5.3.9. EFPA - End File Positive Answer 🥳
last one for this part! This is the Listener's positive answer during the End File Phase, and it's also where the Listener can ask for a direction change 🦋
o-------------------------------------------------------------------o
| EFPA End File Positive Answer |
| |
| End File Phase Speaker <---- Listener |
|-------------------------------------------------------------------|
| Pos | Field | Description | Format |
|-----+-----------+---------------------------------------+---------|
| 0 | EFPACMD | EFPA Command, '4' | F X(1) |
| 1 | EFPACD | Change Direction Indicator, (Y/N) | F X(1) |
o-------------------------------------------------------------------oEFPACMD is the command code, a character. Its value is '4', the EFPA command identifier.
EFPACD is the change direction indicator, a character that's either 'N' or 'Y' 🔄
'N'means change direction is not requested.'Y'means change direction is requested.
This parameter lets the Listener ask the Speaker for a Change Direction (CD) command, so the Listener can basically ask to swap roles 💅
[UTF-8]: https://www.rfc-editor.org/info/std63
5.3.10. EFNA - End File Negative Answer 🙅♀️✨
okay bestie, this is the "nope, not taking that file" message 💅 The listener sends it when it won't accept the file that just ended. Here's the layout:
o-------------------------------------------------------------------o
| EFNA End File Negative Answer |
| |
| End File Phase Speaker <---- Listener |
|-------------------------------------------------------------------|
| Pos | Field | Description | Format |
|-----+-----------+---------------------------------------+---------|
| 0 | EFNACMD | EFNA Command, '5' | F X(1) |
| 1 | EFNAREAS | Answer Reason | F 9(2) |
| 3 | EFNAREASL | Answer Reason Text Length | V 9(3) |
| 6 | EFNAREAST | Answer Reason Text | V T(n) |
o-------------------------------------------------------------------oEFNACMD is the command code, a character 🎀 Its value is '5', which is the EFNA command identifier. ✨
EFNAREAS is the answer reason, a two-digit number 🦋 It can be any of these:
Value: '01' Invalid filename.
'02' Invalid destination.
'03' Invalid origin.
'04' Storage record format not supported.
'05' Maximum record length not supported.
'06' File size is too big.
'10' Invalid record count.
'11' Invalid byte count.
'12' Access method failure.
'13' Duplicate file.
'14' File direction refused.
'15' Cipher suite not supported.
'16' Encrypted file not allowed.
'17' Unencrypted file not allowed.
'18' Compression not allowed.
'19' Signed file not allowed.
'20' Unsigned file not allowed.
'21' Invalid file signature.
'22' File decryption failure.
'23' File decompression failure.
'99' Unspecified reason.
Reason why transmission failed.EFNAREASL is the length, in octets, of the text field below it 🌸 A value of 0 means there's no EFNAREAST field after it. 🙈
EFNAREAST is the reason in plain text 💖 It's encoded using [UTF-8]. The encoded reason can be at most 999 octets long, and there's no general structure defined for this attribute. 🫶
5.3.11. ESID - End Session 👋🌈
Now this is the big goodbye 😭 ESID is how a session wraps up. Here's the layout:
o-------------------------------------------------------------------o
| ESID End Session |
| |
| End Session Phase Speaker ----> Listener |
|-------------------------------------------------------------------|
| Pos | Field | Description | Format |
|-----+-----------+---------------------------------------+---------|
| 0 | ESIDCMD | ESID Command, 'F' | F X(1) |
| 1 | ESIDREAS | Reason Code | F 9(2) |
| 3 | ESIDREASL | Reason Text Length | V 9(3) |
| 6 | ESIDREAST | Reason Text | V T(n) |
| | ESIDCR | Carriage Return | F X(1) |
o-------------------------------------------------------------------oESIDCMD is the command code, a character 🎀 Its value is 'F', the ESID command identifier. 💅
ESIDREAS is the reason code, a two-digit number that explains why the session ended 👑 Here's the whole lineup:
Value: '00' Normal session termination
'01' Command not recognised
An Exchange Buffer contains an invalid command code
(1st octet of the buffer).
'02' Protocol violation
An Exchange Buffer contains an invalid command for
the current state of the receiver.
'03' User code not known
A Start Session (SSID) command contains an unknown or
invalid Identification Code.
'04' Invalid password
A Start Session (SSID) command contained an invalid
password.
'05' Local site emergency close down
The local site has entered an emergency close down
mode. Communications are being forcibly terminated.
'06' Command contained invalid data
A field within a Command Exchange Buffer contains
invalid data.
'07' Exchange Buffer size error
The length of the Exchange Buffer as determined by
the Stream Transmission Header differs from the
length implied by the Command Code.
'08' Resources not available
The request for connection has been denied due to a
resource shortage. The connection attempt should be
retried later.
'09' Time out
'10' Mode or capabilities incompatible
'11' Invalid challenge response
'12' Secure authentication requirements incompatible
'99' Unspecified Abort code
An error was detected for which no specific code is
defined.ESIDREASL is the length, in octets, of the reason text 🌷 A value of 0 means there's no ESIDREAST field present. 🙈
ESIDREAST is the reason the session ended, in plain text 💜 It's encoded using [UTF-8]. The encoded reason can be at most 999 octets long, and no general structure is defined for this attribute. ✨
ESIDCR is the carriage return, a character 🍓 Its value is the character with hex value '0D' or '8D'. 💅
5.3.12. CD - Change Direction 🔄🦋
Last one in this part, and it's a fun one! CD is how the sides swap roles 🧚 Here's the layout:
o-------------------------------------------------------------------o
| CD Change Direction |
| |
| Start File Phase Speaker ----> Listener |
| End File Phase Speaker ----> Listener |
| End Session Phase Initiator <---> Responder |
|-------------------------------------------------------------------|
| Pos | Field | Description | Format |
|-----+-----------+---------------------------------------+---------|
| 0 | CDCMD | CD Command, 'R' | F X(1) |
o-------------------------------------------------------------------oCDCMD is the command code, a character 🎀 Its value is 'R', the CD command identifier. 🌈
5.3.13. EERP - End to End Response ✨
o-------------------------------------------------------------------o
| EERP End to End Response |
| |
| Start File Phase Speaker ----> Listener |
| End File Phase Speaker ----> Listener |
|-------------------------------------------------------------------|
| Pos | Field | Description | Format |
|-----+-----------+---------------------------------------+---------|
| 0 | EERPCMD | EERP Command, 'E' | F X(1) |
| 1 | EERPDSN | Virtual File Dataset Name | V X(26) |
| 27 | EERPRSV1 | Reserved | F X(3) |
| 30 | EERPDATE | Virtual File Date stamp, (CCYYMMDD) | V 9(8) |
| 38 | EERPTIME | Virtual File Time stamp, (HHMMSScccc) | V 9(10) |
| 48 | EERPUSER | User Data | V X(8) |
| 56 | EERPDEST | Destination | V X(25) |
| 81 | EERPORIG | Originator | V X(25) |
| 106 | EERPHSHL | Virtual File hash length | V U(2) |
| 108 | EERPHSH | Virtual File hash | V U(n) |
| | EERPSIGL | EERP signature length | V U(2) |
| | EERPSIG | EERP signature | V U(n) |
o-------------------------------------------------------------------oOkay bestie, the EERP (End to End Response) is giving main character energy 🎀 It shows up in two spots, the Start File Phase and the End File Phase, and in both it travels from Speaker to Listener 🦋 Here's the full fit, field by field 💅
Let's start with the command code. The EERPCMD field is a Character, and its only value is 'E', which is the EERP Command identifier 💖
EERPCMD Command Code Character
Value: 'E' EERP Command identifier.Next up is EERPDSN ✨ This is the Virtual File Dataset Name, a String(26) that's the dataset name of the Virtual File being transferred. The name is assigned by bilateral agreement, and no general structure is defined for it. For the identification rules, check Virtual Files - Identification (Section 1.5.2) 🌸
EERPDSN Virtual File Dataset Name String(26)EERPRSV1 is reserved for future use, so it's a String(3) that just sits there looking cute 🙈
EERPRSV1 Reserved String(3)Now the date stamp, EERPDATE 📅 It's a Numeric(8) in the format 'CCYYMMDD', which means 8 decimal digits for the century, year, month, and day, in that order. It's assigned by the Virtual File's Originator and says when the file was made available for transmission. Again, see Virtual Files - Identification (Section 1.5.2) 🧚
EERPDATE Virtual File Date stamp Numeric(8)
Format: 'CCYYMMDD' 8 decimal digits representing the century,
year, month, and day, respectively.EERPTIME is the time stamp, a Numeric(10) in the format 'HHMMSScccc' ⏰ That's 10 decimal digits covering hours, minutes, seconds, and a counter from 0001 to 9999, which gives higher resolution. Like the date, it's assigned by the Originator and marks when the file was made available for transmission. Same reference applies: Virtual Files - Identification (Section 1.5.2) 🦋
EERPTIME Virtual File Time stamp Numeric(10)
Format: 'HHMMSScccc' 10 decimal digits representing hours,
minutes, seconds, and a counter (0001-9999), which gives
higher resolution.EERPUSER is the User Data field, a String(8) 🎀 ODETTE-FTP can use it in any way it wants. If it's unused, it should be initialised to spaces. A bilateral agreement is expected to define what the data means, so the two sides have to be on the same page 🫶
EERPUSER User Data String(8)EERPDEST is the Destination, a String(25), and its format is described in Identification Code (Section 5.4) 💅 Here's the tea: this is the Originator of the Virtual File, which means the location that created the data for transmission 🍓
EERPDEST Destination String(25)
Format: See Identification Code (Section 5.4)EERPORIG is the Originator, also a String(25) that follows Identification Code (Section 5.4) 🌈 This one is the Final Recipient of the Virtual File. It's the location that looks inside the Virtual File content and processes it accordingly. It's also the location that creates the EERP for the received file, so it's a whole big deal 👑
EERPORIG Originator String(25)
Format: See Identification Code (Section 5.4)Moving on to the hash! EERPHSHL is the Virtual File hash length, a Binary(2) 🔥 It's the length in octets of the field EERPHSH. A binary value of 0 means no hash is present, and that's always the case when the EERP isn't signed 💜
EERPHSHL Virtual File hash length Binary(2)EERPHSH is the Virtual File hash, a Binary(n) ✨ It's a hash of the transmitted Virtual File, which means it's not the hash of the original file, so keep that straight 🙈 The algorithm is determined by the bilaterally agreed cipher suite specified in the SFIDCIPH. And it's an application implementation issue to validate the EERPHSH, so that the EERP is acknowledging the exact same file that was originally transmitted 😭
EERPHSH Virtual File hash Binary(n)Now the signature, starting with EERPSIGL, the EERP signature length, a Binary(2) 🌸 A value of 0 means this EERP has not been signed. Any other value gives the length of EERPSIG in octets and tells you the EERP has been signed 🦋
EERPSIGL EERP signature length Binary(2)And finally EERPSIG, the EERP signature, a Binary(n) 👑 It contains the [CMS] enveloped signature of the EERP. The signature is computed like this:
Signature = Sign{EERPDSN
EERPDATE
EERPTIME
EERPDEST
EERPORIG
EERPHSH}Each field is taken in its entirety, including any padding 💖 The envelope must contain the original data, not just the signature, so don't skip it 🫶 The [CMS] content type used is SignedData, and the encapsulated content type used is id-data. It's an application issue to validate the signature with the contents of the EERP, so the app layer gets the final say 💅
5.3.14. NERP - Negative End Response 💔✨
okay so NERP is the "nope, we are NOT finishing this" message 🙈 it shows up during the Start File phase or the End File phase when something goes sideways. bestie, look at this layout 😭
o-------------------------------------------------------------------o
| NERP Negative End Response |
| |
| Start File Phase Speaker ----> Listener |
| End File Phase Speaker ----> Listener |
|-------------------------------------------------------------------|
| Pos | Field | Description | Format |
|-----+-----------+---------------------------------------+---------|
| 0 | NERPCMD | NERP Command, 'N' | F X(1) |
| 1 | NERPDSN | Virtual File Dataset Name | V X(26) |
| 27 | NERPRSV1 | Reserved | F X(6) |
| 33 | NERPDATE | Virtual File Date stamp, (CCYYMMDD) | V 9(8) |
| 41 | NERPTIME | Virtual File Time stamp, (HHMMSScccc) | V 9(10) |
| 51 | NERPDEST | Destination | V X(25) |
| 76 | NERPORIG | Originator | V X(25) |
| 101 | NERPCREA | Creator of NERP | V X(25) |
| 126 | NERPREAS | Reason code | F 9(2) |
| 128 | NERPREASL | Reason text length | V 9(3) |
| 131 | NERPREAST | Reason text | V T(n) |
| | NERPHSHL | Virtual File hash length | V U(2) |
| | NERPHSH | Virtual File hash | V U(n) |
| | NERPSIGL | NERP signature length | V U(2) |
| | NERPSIG | NERP signature | V U(n) |
o-------------------------------------------------------------------o- NERPCMD (Command Code, Character) 🍓
- Value:
'N', the NERP Command identifier.
- Value:
- NERPDSN (Virtual File Dataset Name,
String(26)) 📁- This is the dataset name of the Virtual File being transferred. It's assigned by bilateral agreement.
- No general structure is defined for this attribute.
- See Virtual Files - Identification (Section 1.5.2)
- NERPRSV1 (Reserved,
String(6)) 🚧- This field is reserved for future use.
- NERPDATE (Virtual File Date stamp,
Numeric(8)) 📅- Format:
'CCYYMMDD'. That's 8 decimal digits covering the century, year, month, and day, in that order. - This is the date stamp the Virtual File's Originator sets to show when the file was made available for transmission.
- See Virtual Files - Identification (Section 1.5.2)
- Format:
- NERPTIME (Virtual File Time stamp,
Numeric(10)) ⏰- Format:
'HHMMSScccc'. That's 10 decimal digits for hours, minutes, seconds, and a counter (0001-9999), which gives higher resolution. - This is the time stamp the Virtual File's Originator sets to show when the file was made available for transmission.
- See Virtual Files - Identification (Section 1.5.2)
- Format:
- NERPDEST (Destination,
String(25)) 💌- Format: See Identification Code (Section 5.4)
- This is the Originator of the Virtual File, meaning the location that created the data for transmission.
- NERPORIG (Originator,
String(25)) 📬- Format: See Identification Code (Section 5.4)
- This is the Final Recipient of the Virtual File, meaning the location that will look into the Virtual File content and perform mapping functions.
- NERPCREA (Creator of the NERP,
String(25)) ✍️- Format: See Identification Code (Section 5.4)
- This is the location that created the NERP.
- NERPREAS (Reason code,
Numeric(2)) 🚫- This attribute says why transmission can't proceed, or why processing of the file failed.
- "SFNA(RETRY=N)" below should be read as "EFNA or SFNA(RETRY=N)" wherever that makes sense.
- Values:
'03' ESID received with reason code '03'
(user code not known)
'04' ESID received with reason code '04'
(invalid password)
'09' ESID received with reason code '99'
(unspecified reason)
'11' SFNA(RETRY=N) received with reason code '01'
(invalid file name)
'12' SFNA(RETRY=N) received with reason code '02'
(invalid destination)
'13' SFNA(RETRY=N) received with reason code '03'
(invalid origin)
'14' SFNA(RETRY=N) received with reason code '04'
(invalid storage record format)
'15' SFNA(RETRY=N) received with reason code '05'
(maximum record length not supported)
'16' SFNA(RETRY=N) received with reason code '06'
(file size too big)
'20' SFNA(RETRY=N) received with reason code '10'
(invalid record count)
'21' SFNA(RETRY=N) received with reason code '11'
(invalid byte count)
'22' SFNA(RETRY=N) received with reason code '12'
(access method failure)
'23' SFNA(RETRY=N) received with reason code '13'
(duplicate file)
'24' SFNA(RETRY=N) received with reason code '14'
(file direction refused)
'25' SFNA(RETRY=N) received with reason code '15'
(cipher suite not supported)
'26' SFNA(RETRY=N) received with reason code '16'
(encrypted file not allowed)
'27' SFNA(RETRY=N) received with reason code '17'
(unencrypted file not allowed)
'28' SFNA(RETRY=N) received with reason code '18'
(compression not allowed)
'29' SFNA(RETRY=N) received with reason code '19'
(signed file not allowed)
'30' SFNA(RETRY=N) received with reason code '20'
(unsigned file not allowed)
'31' File signature not valid.
'32' File decompression failed.
'33' File decryption failed.
'34' File processing failed.
'35' Not delivered to recipient.
'36' Not acknowledged by recipient.
'50' Transmission stopped by the operator.
'90' File size incompatible with recipient's
protocol version.
'99' Unspecified reason.- NERPREASL (Reason Text Length,
Numeric(3)) 🔢- This is the length in octets of the field NERPREAST.
- A value of 0 means no NERPREAST field follows.
- NERPREAST (Reason Text,
[UTF-8](n)) 📝- This is the reason transmission can't proceed, written in plain text.
- It's encoded using [UTF-8].
- The maximum length of the encoded reason is 999 octets.
- No general structure is defined for this attribute.
- NERPHSHL (Virtual File hash length,
Binary(2)) 🧮- This is the length in octets of the field NERPHSH.
- A binary value of 0 means no hash is present. That's always the case when the NERP isn't signed.
- NERPHSH (Virtual File hash,
Binary(n)) 🔐- This is the hash of the Virtual File being transmitted.
- The algorithm used is set by the bilaterally agreed cipher suite specified in the SFIDCIPH.
- NERPSIGL (NERP Signature length,
Binary(2)) ✒️- A value of 0 means this NERP hasn't been signed.
- Any other value is the length of NERPSIG in octets, and means this NERP has been signed.
- NERPSIG (NERP Signature,
Binary(n)) 🫶- This contains the [CMS] enveloped signature of the NERP.
- The signature is computed like this:
Signature = Sign{NERPDSN
NERPDATE
NERPTIME
NERPDEST
NERPORIG
NERPCREA
NERPHSH}- Each field is taken in its entirety, including any padding.
- The envelope must contain the original data, not just the signature.
- The [CMS] content type used is SignedData.
- The encapsulated content type used is id-data.
- Validating the signature against the contents of the NERP is an application issue.
5.3.15. RTR - Ready To Receive 🙌
RTR is the "I'm ready, send it over" message 💖 here's the layout:
o-------------------------------------------------------------------o
| RTR Ready To Receive |
| |
| Start File Phase Initiator <---- Responder |
| End File Phase Initiator <---- Responder |
|-------------------------------------------------------------------|
| Pos | Field | Description | Format |
|-----+-----------+---------------------------------------+---------|
| 0 | RTRCMD | RTR Command, 'P' | F X(1) |
o-------------------------------------------------------------------o- RTRCMD (Command Code, Character) 🦋
- Value:
'P', the RTR Command identifier.
- Value:
5.3.16. SECD - Security Change Direction 🔒✨
SECD is the security one 👑 it's used to change direction during the Start Session phase, so both sides can flip roles on the security side. here's the layout:
o-------------------------------------------------------------------o
| SECD Security Change Direction |
| |
| Start Session Phase Initiator <---> Responder |
|-------------------------------------------------------------------|
| Pos | Field | Description | Format |
|-----+-----------+---------------------------------------+---------|
| 0 | SECDCMD | SECD Command, 'J' | F X(1) |
o-------------------------------------------------------------------o- SECDCMD (Command Code, Character) 🌸
- Value:
'J', the SECD Command identifier.
- Value:
5.3.17. AUCH - Authentication Challenge ✨
okay bestie, AUCH is the challenge message 🔐 It's how one side says "prove who you are" during the Start Session phase, and it's giving very secure main character energy.
o-------------------------------------------------------------------o
| AUCH Authentication Challenge |
| |
| Start Session Phase Initiator <---> Responder |
|-------------------------------------------------------------------|
| Pos | Field | Description | Format |
|-----+-----------+---------------------------------------+---------|
| 0 | AUCHCMD | AUCH Command, 'A' | F X(1) |
| 1 | AUCHCHLL | Challenge Length | V U(2) |
| 3 | AUCHCHAL | Challenge | V U(n) |
o-------------------------------------------------------------------oHere's the breakdown 💅
AUCHCMDis the command code, aCharacterwith the value'A', which is just the AUCH command identifier 🦋AUCHCHLLis the challenge length, aBinary(2)value that tells you how many octets are inAUCHCHAL. It's an unsigned binary number in network byte order 🌸AUCHCHALis the challenge itself, aBinary(n)value. It's a [CMS] encrypted 20-byte random number, uniquely generated every single time an AUCH gets sent 🎀 so no two challenges are ever the same, ngl that's so iconic
NOTE: any encryption algorithm that's available through a defined cipher suite (Section 10.2) may be used. See Section 10.1 for how to choose a cipher suite 🧚
5.3.18. AURP - Authentication Response
this is the reply to the challenge 🫶
o-------------------------------------------------------------------o
| AURP Authentication Response |
| |
| Start Session Phase Initiator <---> Responder |
|-------------------------------------------------------------------|
| Pos | Field | Description | Format |
|-----+-----------+---------------------------------------+---------|
| 0 | AURPCMD | AURP Command, 'S' | F X(1) |
| 1 | AURPRSP | Response | V U(20) |
o-------------------------------------------------------------------oAURPCMDis the command code, aCharacterwith the value'S', which is the AURP command identifier 🌈AURPRSPis the response, aBinary(20)value that contains the decrypted challenge (AUCHCHAL) 💜
IMPORTANT: it's an application implementation issue to validate a received AURP so that the response matches the challenge. This validation is extremely important to make sure a party is correctly authenticated 🚨 not optional, bestie, this is the whole point 😭
5.4. Identification Code
so every party in an ODETTE-FTP session gets a unique ID called an Identification Code 🪪 It's based on [ISO-6523], Structure for the Identification of Organisations (SIO). Both the Initiator (sender) and the Responder (receiver) are identified this way, and the two locations are considered adjacent for the whole transmission 🍓
Here's what an SIO looks like 👑
o-------------------------------------------------------------------o
| Pos | Field | Description | Format |
|-----+-----------+---------------------------------------+---------|
| 0 | SIOOID | ODETTE Identifier | F X(1) |
| 1 | SIOICD | International Code Designator | V 9(4) |
| 5 | SIOORG | Organisation Code | V X(14) |
| 19 | SIOCSA | Computer Subaddress | V X(6) |
o-------------------------------------------------------------------oSIOOIDis the ODETTE identifier, aCharacter. The value'O'means it's an ODETTE assigned Organisation Identifier. Other values may be used for non-ODETTE codes 🌸SIOICDis the International Code Designator, aString(4). It's a code that forms part of the Organisation Identifier 🦋SIOORGis the Organisation Code, aString(14). It's also a code that forms part of the Organisation Identifier. This field may contain the letters A to Z, the digits 0 to 9, and space and hyphen characters ✨SIOCSAis the Computer Subaddress, aString(6). It's a locally assigned address that uniquely identifies a system within an organisation (the one defined by an Organisation Identifier) 💻
6. File Services
6.1. Overview
ODETTE-FTP has services for compressing, encrypting, and signing files 🎁 these should generally be done off line, outside of the ODETTE-FTP session, for performance reasons. It isn't a strict requirement though, so don't panic 🙈
ODETTE-FTP requires that the following steps must be performed in this exact sequence. Any of steps 2, 3, or 4 may be omitted. Step 1 is required only if any of steps 2, 3, or 4 are performed:
- Insert record length indicators (V format files only; see Section 6.5)
- Sign
- Compress
- Encrypt
the cipher suite for the encryption and signing algorithms is assigned by bilateral agreement 🤝
Secured and/or compressed files must be enveloped. The envelope holds extra info about the service that was used, which the receiving party needs in order to fully process the file 📦
The [CMS] content types are 🧁
EnvelopedData - Indicates encrypted data
CompressedData - Indicates compressed data
SignedData - Indicates signed content
Data - Indicates unstructured dataFor signed or encrypted data, the encapsulated content type (eContentType field) is id-data 💜
6.2. File Signing
files that are going to be signed get enveloped according to the file enveloping format (SFIDENV). Usually this is a [CMS] package 🖋️
a file can be signed more than once, which makes switching between old and new certificates way smoother ✨
It is recommended that the envelope does not contain the public certificate of the signer. If files keep going to the same recipient, there's zero benefit in sending the same certificate over and over, so skip it 🙅♀️ Both the original file data and the signature get stored inside the [CMS] package.
6.3. File Encryption
files that are going to be encrypted get enveloped according to the file enveloping format (SFIDENV). Usually this is a [CMS] package 🔒
It is recommended that encryption should be performed before the ODETTE-FTP session starts. A big file takes forever to encrypt and could cause session time outs, even on high-performance machines ⏰
decryption should also happen outside of the session. An application may choose to allow in-session encryption and decryption for very small files though 🌷
6.4. File Compression
files that are going to be compressed get enveloped according to the file enveloping format (SFIDENV). Usually this is a [CMS] package using the [CMS-Compression] data type, which uses the [ZLIB] compression algorithm by default 🗜️
Unlike the buffer compression method, this one works on a whole file. Because the compression levels are so much higher, file level compression basically deprecates the older buffer compression inside ODETTE-FTP. The buffer compression is kept around for backwards compatibility 🦖
6.5. V Format Files - Record Lengths
a file that has been signed, compressed, and/or encrypted loses its record structure 😭 so ODETTE-FTP can't insert the End of Record Flag in subrecord headers in Data Exchange Buffers. To keep the record structure, V format files must have record headers inserted into them before signing, compression, or encryption. These are 2-byte binary numbers in network byte order, and each one shows the length of a record. That lets the receiving system, where it makes sense, rebuild the file with its original variable-length records 🧩 The header bytes hold the number of data bytes in the record and don't count themselves, so keep that in mind.
This only applies to V format files, which are mostly a mainframe thing 🖥️
7. ODETTE-FTP Data Exchange Buffer
7.1. Overview
Virtual Files get sent by mapping their records into Data Exchange Buffers. The maximum length of those buffers gets negotiated between the ODETTE-FTP entities through the Start Session (SSID) commands that get swapped during the Start Session phase 🤝
Virtual File records can be any length. There's also a simple compression scheme for strings of repeated characters 🔁
there's an example of the Data Exchange Buffer in Appendix A 📖
7.2. Data Exchange Buffer Format
for sending Virtual File records, the data gets split into subrecords. Each subrecord is preceded by a 1-octet Subrecord Header 🎀
The Data Exchange Buffer is the initial Command Character followed by pairs of Subrecord Headers and subrecords, like this 🧶
o--------------------------------------------------------
| C | H | | H | | H | | /
| M | D | SUBRECORD | D | SUBRECORD | D | SUBRECORD | /_
| D | R | | R | | R | | /
o-------------------------------------------------------CMDis the Data Exchange Buffer Command Character, which is'D'💖HDRis a 1-octet Subrecord Header with this layout:
0 1 2 3 4 5 6 7
o-------------------------------o
| E | C | |
| o | F | C O U N T |
| R | | |
o-------------------------------oThe bits in the header work like this 🦋
- Bit 0 (End of Record Flag) gets set to show that the next subrecord is the last subrecord of the current record. Unstructured files are sent as one single record, and in that case the flag acts as an end-of-file marker 🏁
- Bit 1 (Compression Flag) gets set to show that the next subrecord is compressed 🗜️
- Bits 2-7 (Subrecord Count) hold the number of octets in the Virtual File that the next subrecord stands for, as a binary value. For uncompressed data that's just the length of the subrecord. For compressed data, it's how many times the single octet in the following subrecord has to be put into the Virtual File. Since 6 bits are available, the next subrecord can stand for anywhere between 0 and 63 octets of the Virtual File 🔢
7.3. Buffer Filling Rules
a Data Exchange Buffer can be any length, up to the value negotiated in the Start Session exchange 📏
Virtual File records can be concatenated inside one Data Exchange Buffer, or split across a bunch of buffers 🧃
a subrecord is never split between two Exchange Buffers 🚫 If the space left in the current Exchange Buffer isn't enough for the next 'complete' subrecord, one of the following strategies should be used:
- Truncate the Exchange Buffer, and put the complete subrecord (preceded by its header octet) in a new Exchange Buffer.
- Split the subrecord into two, filling the rest of the Exchange Buffer with the first new subrecord and starting a new Exchange Buffer with the second.
a record of length zero may appear anywhere in the Exchange Buffer 🌙
a subrecord of length zero may appear anywhere in the record and/or the Exchange Buffer 🫧
8. Stream Transmission Buffer ✨
8.1. Introduction
okay so, to use the TCP stream, we make a Stream Transmission Buffer (STB) by adding a Stream Transmission Header (STH) to the start of every Command and Data Exchange Buffer before they get passed to the TCP transport service 💌 that's how the receiving ODETTE-FTP gets the original Exchange Buffers back 🔁✨
Note: the Stream Transmission Buffer is NOT used when you run ODETTE-FTP over an X.25 network 🙅♀️
the reason is that ODETTE-FTP can count on the Network Service to keep the sequence and the boundaries of each data unit, and to pass the length of the data unit to the receiving ODETTE-FTP 📦 TCP is a stream-based connection and doesn't give you any of that, so we need the STH 🫶
an STB is an STH plus an OEB, lined up one after another:
o-----+-----------------+-----+--------------------+-----+------
| STH | OEB | STH | OEB | STH | OEB/
o-----+-----------------+-----+--------------------+-----+----- STH - Stream Transmission Header
- OEB - ODETTE-FTP Exchange Buffer
8.2. Stream Transmission Header Format
the STH looks like this 🎀 the fields get sent from left to right ➡️
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Version| Flags | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- Version
- Value:
0001(binary) 🌸 - This is the Stream Transmission Header version number.
- Value:
- Flags
- Value:
0000(binary) - Reserved for future use 🔮
- Value:
- Length
- Range:
5 - 100003(decimal) - This is the length of the Stream Transmission Buffer (STH+OEB).
- The smallest STB is 5 octets: a 4-octet header followed by a 1-octet Exchange Buffer, like a Change Direction (CD) command.
- The biggest Exchange Buffer length you can negotiate is 99999 octets (Section 5.3.2), which gives an STB length of 100003 📏
- The length is written as a binary number in network byte order.
- Range:
ok and last thing, implementations of this protocol are expected to follow the Internet robustness principle: be conservative in what you send and liberal in what you accept 🫶💖
9. Protocol State Machine
9.1. ODETTE-FTP State Machine
the way an ODETTE-FTP entity behaves is formally defined by a State Machine 🧠 there are five State and Transition tables, and each one has extra info in its matching Predicate and Action lists 📋
when an entity gets an event, its response is whatever the Transition table entry says for that Event/State crossing in the right state table 🎯
each Transition table entry defines the actions taken, the events generated, and the new state entered. predicates can be used inside a table entry to pick the right response based on local info the entity is holding 🔍
a Transition table has these fields:
- Index (I): the state transition index.
- Predicate: a list of predicates used to pick between different possible transitions. they're defined in the Predicate and Action lists.
- Actions: a list of actions the entity takes. they're defined in the Predicate and Action lists too.
- Events: output events the entity generates.
- Next State: the new state of the entity.
9.2. Error Handling
an event can be invalid in a given state for three reasons, bestie 🚨
- the case is impossible by design of the state automata. these show up as
Xin the state tables. example: a timer that was never set can't run out ⏰ - the event is the result of an error in the Network Service implementation, which also shows up as
Xin the state tables. the Network Service implementation is considered to be correct 💅 - everything else counts as a User Error, shown as
"U"in the state tables 🙈
the state tables say when a User event is valid, which stops the entity from throwing a protocol error just because the User Monitor messed up. what the entity does in those User error cases is undefined and left to local implementation 🤷♀️
the state tables also let you catch protocol errors caused by receiving invalid Exchange Buffers. in those cases, the entity's reaction to the error IS defined 👑
9.3. States
ok, so the Command Mode is strictly a half-duplex flip-flop mode 🔁 now the states:
A_NC_ONLY- Responder, Network Connection opened- the Responder has sent its Ready Message (SSRM) and is waiting for Start Session (SSID) from the Initiator 👀
A_WF_CONRS- Responder Waiting for F_CONNECT_RS- the Responder got the Initiator's Start Session (SSID) and is waiting for a response (F_CONNECT_RS) from its User Monitor 💭
CDSTWFCD- CD_RQ stored in WF_CD state- the User Monitor can't see the WF_CD state, so it might send a Change Direction request (F_CD_RQ) before ODETTE-FTP receives a Change Direction (CD) command 🫣
CLIP- Close Input Pending- the Listener got an End File (EFID) command and is waiting for the Close File response (F_CLOSE_FILE_RS) from its User Monitor 📭
CLOP- Close Out Pending- the Speaker sent an End File (EFID) command and is waiting for an End File Answer (EFPA or EFNA) ⏳
ERSTWFCD- End to End Response stored in WF_CD state- the User Monitor can't see the WF_CD state, so it might send F_EERP_RQ before ODETTE-FTP receives a Change Direction (CD) command 🙃
IDLE- Connection IDLE 😴IDLELI- Idle Listener 🎧IDLELICD- Idle Listener, F_CD_RQ Received- the ODETTE-FTP entity became the Listener after getting a Change Direction request (F_CD_RQ) from the User Monitor. getting an End Session (ESID) is valid in this state 🌈
IDLESP- Idle Speaker 📣IDLESPCD- Idle Speaker, F_CD_IND Sent- the ODETTE-FTP entity sent a Change Direction indication (F_CD_IND) to the User Monitor. a Change Direction request (F_CD_RQ) is NOT valid in this state 🚫
I_WF_NC- Initiator Waiting for Network Connection- the Initiator asked for a new network connection and is waiting for a Connection confirmation (N_CON_CF) from the Network Service 🔌
I_WF_RM- Initiator Waiting for Ready Message- before sending Start Session (SSID), the Initiator must wait for a Ready Message (SSRM) from the Responder ⏸️
I_WF_SSID- Initiator Waiting for SSID- the Initiator sent a Start Session (SSID) command and is waiting for Start Session from the Responder 🤞
NRSTWFCD- Negative End Response stored in WF_CD state- the User Monitor can't see the WF_CD state, so it might send F_NERP_RQ before ODETTE-FTP receives a Change Direction (CD) command 😬
OPI- Open Input (Data Transfer Phase)- the Listener is waiting for the Speaker to send a Data Exchange Buffer 📥
OPIP- Open Input Pending- the Listener got a Start File (SFID) command and is waiting for the Start File response (F_START_FILE_RS) from its User Monitor 🗂️
OPO- Open Out (Data Transfer Phase)- the Speaker got a Start File Positive Answer (SFPA) and is waiting for a Data (F_DATA_RQ) or Close File (F_CLOSE_FILE) request from its User Monitor 📤
OPOP- Open Out Pending- the Speaker sent a Start File (SFID) command and is waiting for a Start File Answer (SFPA or SFNA) 🕰️
OPOWFC- Open Out Wait for Credit- the Speaker is waiting for a Set Credit (CDT) command before it sends any more Data Exchange buffers 💳
RTRP- Ready to Receive (RTR) Pending- the Listener got an EERP or a NERP and is waiting for the Ready to Receive response (F_RTR_RS) from its User Monitor 🙋♀️
SFSTWFCD- Start File Request stored in WF_CD state.- the User Monitor can't see the WF_CD state, so it might send a Start File request (F_START_FILE_RQ) before ODETTE-FTP receives a Change Direction (CD) command 🫠
WF_CD- Wait for Change Direction- the Listener wants to become the Speaker and is waiting for a Change Direction (CD) command after sending an End File Positive Answer (EFPA) that asks for a change of direction 🔄
WF_RTR- Wait for Ready To Receive- the Speaker sent an End to End Response (EERP) or a Negative End Response (NERP) command and has to wait for Ready To Receive (RTR) from the Listener 🛑
WF_NDISC- Wait for N_DISC_IND- ODETTE-FTP sent an End Session (ESID) command and is waiting for a Disconnection indication (N_DISC_IND) from the Network Service 👋
WF_SECD- Wait for Security Change Direction- the Speaker is expecting a Security Change Direction (SECD) from the Listener 🔐
WF_AUCH- Wait for Authentication Challenge- the Speaker sent a Security Change Direction (SECD) command and has to wait for an Authentication Challenge (AUCH) from the Listener 🪪
WF_AURP- Wait for Authentication Response- the Speaker sent an Authentication Challenge (AUCH) command and has to wait for an Authentication Response (AURP) from the Listener 🎟️
9.4. Input Events
these are all the events the state machine can take in 📨
User Monitor Input Events (Section 3)
F_DATA_RQ F_CONNECT_RQ F_START_FILE_RQ F_CLOSE_FILE_RQ
F_EERP_RQ F_CONNECT_RS F_START_FILE_RS(+) F_CLOSE_FILE_RS(+)
F_NERP_RQ F_ABORT_RQ F_START_FILE_RS(-) F_CLOSE_FILE_RS(-)
F_CD_RQ F_RELEASE_RQ F_RTR_RSNetwork Input Events (Section 2.2)
N_CON_IND N_CON_CF N_DATA_IND N_DISC_IND N_RST_INDPeer ODETTE-FTP Input Events (Section 4)
SSID SFID SFPA SFNA EFID EFPA EFNA
DATA ESID EERP RTR CD CDT SSRM
NERP SECD AUCH AURPInternal Input Events
- TIME-OUT - Internal ODETTE-FTP timer expires. ⏰
input event parameters are written as I.Event-name.Parameter-name inside the state table action and predicate lists. the ODETTE-FTP entity can look at their values but can't change them 👀💜
9.5. Output Events 📤
Okay so this is where the protocol decides what it's gonna say out loud 📣. Every output event is a message going to one of three places: the User Monitor, the network, or the peer ODETTE-FTP entity. Here's the full guest list 💌
User Monitor Output Events (Section 3)
F_DATA_IND F_CONNECT_IND F_START_FILE_IND F_CLOSE_FILE_IND
F_EERP_IND F_CONNECT_CF F_START_FILE_CF(+) F_CLOSE_FILE_CF(+)
F_CD_IND F_ABORT_IND F_START_FILE_CF(-) F_CLOSE_FILE_CF(-)
F_NERP_IND F_RELEASE_IND F_DATA_CF F_RTR_CF
Network Output Events (Section 2.2)
N_CON_RQ N_CON_RS N_DATA_RQ N_DISC_RQ
Peer ODETTE-FTP Output Events (Section 4)
SSID SFID SFPA SFNA EFID EFPA EFNA
DATA ESID EERP RTR CD CDT SSRM
NERP SECD AUCH AURPOutput event parameters are written O.Event-name.Parameter-name inside the state table action and predicate lists. The ODETTE-FTP entity can look at their values and change them 💅
9.6. Local Variables 🧠
These are the variables the ODETTE-FTP entity keeps track of so the protocol can actually do its thing 🫶 They're written V.Variable-name in the state table action and predicate lists. The entity can examine and change them. The initial value of each variable is undefined, so nothing has a value until the protocol sets one 🫠
Variable Type Comments
---------------------------------------------------------------------
Buf-size Integer Negotiated Data Exchange Buffer size.
Called-addr Address Used to build O.F_CONNECT_IND.Called-addr
Calling-addr Address To build O.F_CONNECT_IND.Calling-addr
Compression Yes/No Compression in use as agreed.
Credit_L Integer Listener's credit counter.
Credit_S Integer Speaker's credit counter.
Id String Used to build O.SSID.Id
Mode Sender-only, Receiver-only, Both.
Pswd String Password, used to build O.SSID.Pswd
Req-buf Primitive Input event (F_XXX_RQ) stored in WF_CD
state.
Restart Yes/No Restart in used as agreed.
Restart-pos Integer Used only during file opening.
Window Integer The credit value negotiated for the
session.
Caller Yes/No This entity initiated the ODETTE-FTP
session.
Authentication Yes/No Secure authentication in use as agreed
Challenge Binary Random challenge
---------------------------------------------------------------------9.7. Local Constants 🔒
These constants are the capabilities of a given ODETTE-FTP entity, basically its "here's what I can do" list 🎀 They're written C.Constant-name in the state table action and predicate lists. The entity can look at them but can't change them, because they're fixed.
Constant Value Comments
---------------------------------------------------------------------
Cap-compression Yes/No Compression supported?
Cap-init Initiator Must be Initiator.
Responder Must be Responder.
Both Can be Initiator or Responder.
Cap-mode Sender-only Must be sender.
Receiver-only Must be receiver.
Both Can be sender or receiver.
Max-buf-size 127 < Int < 100000 Maximum Data Exchange Buffer
size supported.
Max-window 0 < Int < 1000 Local maximum credit value.
Cap-restart Yes/No Restart supported?
Cap-logic 0, 1, 2 0 = does not support special
logic
1 = supports special logic
2 = needs special logic
---------------------------------------------------------------------9.8. Session Connection State Table 👑
Okay, this is the big boss table for the session connection 😭 It's split into the state table, which maps out the states, and the transition table, which tells you what happens when each event hits. Let's get into it 🔥
9.8.1. State Table 🗺️
o----------------------------------------------------------o
| | Other States |
| |--------------------------------------------------o |
| | WF_SECD | |
| |----------------------------------------------o | |
| | WF_AURP | | |
| |------------------------------------------o | | |
| | WF_AUCH | | | |
| |--------------------------------------o | | | |
| S | A_WF_CONRS | | | | |
| |----------------------------------o | | | | |
| T | A_NC_ONLY | | | | | |
| |------------------------------o | | | | | |
| A | I_WF_SSID | | | | | | |
| |--------------------------o | | | | | | |
| T | I_WF_RM | | | | | | | |
| |----------------------o | | | | | | | |
| E | I_WF_NC | | | | | | | | |
| |------------------o | | | | | | | | |
| | IDLE | | | | | | | | | |
|==================o---+---+---+---+---+---+---+---+---+---|
| | F_CONNECT_RQ | A | X | X | X | X | X | X | X | X | X |
| |--------------+---+---+---+---+---+---+---+---+---+---|
| E | N_CON_CF | X | C | X | X | X | X | X | X | X | X |
| |--------------+---+---+---+---+---+---+---+---+---+---|
| V | SSRM | X | X | H | X | X | X | L | L | L | X |
| |--------------+---+---+---+---+---+---+---+---+---+---|
| E | SSID | X | X | X | D | E | F | L | L | L | F |
| |--------------+---+---+---+---+---+---+---+---+---+---|
| N | N_CON_IND | B | X | X | X | X | X | X | X | X | X |
| |--------------+---+---+---+---+---+---+---+---+---+---|
| T | F_CONNECT_RS | X | U | U | U | U | G | X | X | X | U |
| |--------------+---+---+---+---+---+---+---+---+---+---|
| | ESID | X | X | X | F | X | X | F | F | F | X |
| |--------------+---+---+---+---+---+---+---+---+---+---|
| | AUCH | X | X | U | U | X | X | I | L | L | U |
| |--------------+---+---+---+---+---+---+---+---+---+---|
| | AURP | X | X | U | U | X | X | L | K | L | U |
| |--------------+---+---+---+---+---+---+---+---+---+---|
| | SECD | X | X | U | U | X | X | L | L | J | U |
o----------------------------------------------------------o9.8.2. Transition Table 🦋
I | Predicate Actions Output Events Next State
===o=============================================================
A | P1: F_ABORT_IND IDLE
| !P1: 1,2 N_CON_RQ I_WF_NC
---+-------------------------------------------------------------
B | P3: N_DISC_RQ IDLE
| !P3: 2 N_CON_RS
| SSRM A_NC_ONLY
---+-------------------------------------------------------------
C | 4,2 I_WF_RM
---+-------------------------------------------------------------
D | P2 & P8 & P11: 4,2,5 SECD WF_AUCH
| P2 & P8 & !P11: 4,2,5 F_CONNECT_CF IDLESP
| P2 & !P8: 4,2 ESID(R=12)
| F_ABORT_IND(R,AO=L) WF_NDISC
| else: 4,2 ESID(R=10)
| F_ABORT_IND(R,AO=L) WF_NDISC
---+-------------------------------------------------------------
E | P4: 4 N_DISC_RQ IDLE
| !P4: 4,2 F_CONNECT_IND A_WF_CONRS
---+-------------------------------------------------------------
F | 4 F_ABORT_IND
| N_DISC_RQ IDLE
---+-------------------------------------------------------------
G | P2 & P9 & P10: 4,2,5 SSID WF_SECD
| P2 & !P9 & P10: 4,2,5 SSID IDLELI
| !P10: 4,2 ESID(R=12)
| F_ABORT_IND(R,AO=L) WF_NDISC
| else: 4,2 ESID(R=10)
| F_ABORT_IND(R,AO=L) WF_NDISC
---+-------------------------------------------------------------
H | 4,2,3 SSID I_WF_SSID
---+-------------------------------------------------------------
I | P5: 4,2 AURP WF_SECD
| !P5: 4,2 AURP IDLELI
---+-------------------------------------------------------------
J | 4,2 AUCH WF_AURP
---+-------------------------------------------------------------
K | P6: 4,2 F_CONNECT_CF IDLESP
| P7: 4,2 SECD WF_AUCH
| else: 4,2 ESID(R=11)
| F_ABORT_IND(R,AO=L) WF_NDISC
---+-------------------------------------------------------------
L | 4,2 ESID(R=02)
| F_ABORT_IND(R,AO=L) WF_NDISC
---+-------------------------------------------------------------9.8.3. Predicates and Actions ✨
Predicates are the yes/no checks the table uses, and actions are the things the entity does when it's time to move 💖 Here's the whole cast:
Predicate P1: (No resources available) OR
(C.Cap-init = Responder) OR
(C.Cap-mode = Sender-only AND
I.F_CONNECT_RQ.Mode = Receiver-only) OR
(C.Cap-mode = Receiver-only AND
I.F_CONNECT_RQ.Mode = Sender-only)
Predicate P2: SSID negotiation is successful
(for these, Buf-size, Restart, Compression, Mode,
Special logic, and Window, compare the inbound SSID
with the local constants to set the local variables.
Any incompatibilities result in failure of the
negotiation.)
Predicate P3: C.Cap-init = Initiator
Predicate P4: Mode in SSID incompatible with C.Cap-mode
Predicate P5: V.Caller = Yes
Predicate P6: (V.Caller = Yes) AND (AURP.Signature verifies with
V.Challenge)
Predicate P7: (V.Caller = No) AND (AURP.Signature verifies with
V.Challenge)
Predicate P8: V.Authentication = I.SSID.Authentication
Predicate P9: I.F_CONNECT_RS.Authentication = Yes
Predicate P10: O.F_CONNECT_IND.Authentication =
I.F_CONNECT_RS.Authentication
Predicate P11: V.Authentication = Yes
Action 1: Set V.Mode from (C.Cap-mode, I.F_CONNECT_RQ.Mode)
Set V.Pswd, V.Id, V.Restart, and
V.Authentication from I.F_CONNECT_RQ
Set V.Buf-size = C.Max-buf-size
Set V.Compression = C.Cap-compression
Set V.Caller = Yes
Build O.N_CON_RQ
Action 2: Start inactivity timer
Action 3: Set parameters in O.SSID = from local variables
Action 4: Stop timer
Action 5: Set V.Mode, V.Restart, V.Compression, V.Buf-size,
V.Window, V.Authentication = from SSID
Action 6: Set V.Challenge = A random number unique to the
session9.9. Error and Abort State Table 🚨
9.9.1. State Table 🎀
Okay so this grid is the big cheat sheet for the Error and Abort state table 🧚. The states run across the top, the events run down the side, and every cell holds a letter (A through I) or an X. Those letters point at rows in the Transition Table just below, so it's giving "choose your own adventure" energy 💅.
o--------------------------------------o
| | Other States |
| S |------------------------------o |
| T | WF_NDISC | |
| A |--------------------------o | |
| T | I_WF_NC | | |
| E |----------------------o | | |
| | IDLE | | | |
|======================o---+---+---+---|
| | TIME-OUT | X | X | A | B |
| |------------------+---+---+---+---|
| E | F_ABORT_RQ | X | A | X | C |
| V |------------------+---+---+---+---|
| E | N_RST_IND | X | X | A | D |
| N |------------------+---+---+---+---|
| T | N_DISC_IND | X | E | F | G |
| |------------------+---+---+---+---|
| | Invalid Buffer | X | X | H | I |
o--------------------------------------o9.9.2. Transition Table 🦋
Here's the Transition Table, which is where the letters from the grid turn into real actions. Each row has a letter (A to I), an optional predicate, the actions to run, the output events to send, and the next state to move into 💖.
I | Predicate Actions Output Events Next State
===o=================================================================
A | N_DISC_RQ IDLE
---+-----------------------------------------------------------------
B | F_ABORT_IND
| N_DISC_RQ IDLE
---+-----------------------------------------------------------------
C | 1 N_DISC_RQ IDLE
---+-----------------------------------------------------------------
D | 1 N_DISC_RQ
| F_ABORT_IND IDLE
---+-----------------------------------------------------------------
E | F_ABORT_IND IDLE
---+-----------------------------------------------------------------
F | 1 IDLE
---+-----------------------------------------------------------------
G | 1 F_ABORT_IND IDLE
---+-----------------------------------------------------------------
H | WF_NDISC
---+-----------------------------------------------------------------
I | 1,2 ESID(R=01)
| F_ABORT_IND(R,AO=L) WF_NDISC
---------------------------------------------------------------------9.9.3. Predicates and Actions 🌸
These are the two actions that the Transition Table above keeps calling on, and they're super simple:
Action 1: stop the inactivity timer ⏸️Action 2: start the inactivity timer ▶️
9.10. Speaker State Table 1 🌈
9.10.1. State Table ⭐
Before we dive into the grid, here are the abbreviations you'll see in the Speaker state table, so you're not left guessing 🙈:
F_REL_RQ(Ok) - F_RELEASE_RQ Reason = Normal
F_REL_RQ(Err) - F_RELEASE_RQ Reason = ErrorThis one is a bigger grid, so get comfy 🍓. Same vibe as before: states across the top, events down the side, and the letters in the cells lead to the Speaker Transition Table.
o--------------------------------------------------------------------o
| | Other States |
| |--------------------------------------------------------------o |
| | WF_NDISC | |
| |----------------------------------------------------------o | |
| | OPOWFC | | |
| |------------------------------------------------------o | | |
| | OPO | | | |
|S|--------------------------------------------------o | | | |
| | OPOP | | | | |
|T|----------------------------------------------o | | | | |
| | CDSTWFCD | | | | | |
|A|------------------------------------------o | | | | | |
| | SFSTWFCD | | | | | | |
|T|--------------------------------------o | | | | | | |
| | NRSTWFCD | | | | | | | |
|E|----------------------------------o | | | | | | | |
| | ERSTWFCD | | | | | | | | |
| |------------------------------o | | | | | | | | |
| | WF_CD | | | | | | | | | |
| |--------------------------o | | | | | | | | | |
| | WF_RTR | | | | | | | | | | |
| |----------------------o | | | | | | | | | | |
| | IDLESPCD | | | | | | | | | | | |
| |------------------o | | | | | | | | | | | |
| | IDLESP | | | | | | | | | | | | |
|=+==============o---+---+---+---+---+---+---+---+---+---+---+---+---|
| | F_EERP_RQ | A | A | W | F | W | W | U | U | U | U | U | U | U |
| |--------------+---+---+---+---+---+---+---+---+---+---+---+---+---|
| | F_NERP_RQ | Y | Y | W | Z | W | W | U | U | U | U | U | U | U |
| |--------------+---+---+---+---+---+---+---+---+---+---+---+---+---|
| | F_START_ | B | B | W | G | W | W | U | U | U | U | U | X | U |
| | FILE_RQ | | | | | | | | | | | | | |
| |--------------+---+---+---+---+---+---+---+---+---+---+---+---+---|
| | SFPA | C | C | C | C | C | C | C | C | K | C | C | S | C |
| |--------------+---+---+---+---+---+---+---+---+---+---+---+---+---|
|E| SFNA | C | C | C | C | C | C | C | C | L | C | C | S | C |
| |--------------+---+---+---+---+---+---+---+---+---+---+---+---+---|
|V| CD | C | C | C | H | R | Z1| I | J | C | C | C | S | C |
| |--------------+---+---+---+---+---+---+---+---+---+---+---+---+---|
|E| F_DATA_RQ | U | U | U | U | U | U | U | U | U | M | U | S | U |
| |--------------+---+---+---+---+---+---+---+---+---+---+---+---+---|
|N| CDT | C | C | C | C | C | C | C | C | C | P | O | S | C |
| |--------------+---+---+---+---+---+---+---+---+---+---+---+---+---|
|T| F_CD_RQ | D | U | W | T | W | W | U | U | U | U | U | X | U |
| |--------------+---+---+---+---+---+---+---+---+---+---+---+---+---|
| | F_REL_RQ(Ok) | U | E | U | U | U | U | U | U | U | U | U | X | U |
| |--------------+---+---+---+---+---+---+---+---+---+---+---+---+---|
| | F_REL_RQ(Err)| Q | Q | Q | Q | Q | Q | Q | Q | Q | Q | Q | S | Q |
| |--------------+---+---+---+---+---+---+---+---+---+---+---+---+---|
| | RTR | C | C | N | C | C | C | C | C | C | C | C | S | C |
o--------------------------------------------------------------------o9.10.2. Transition Table 💜
Here's the Speaker Transition Table. It's a lot, but each row is just a letter, an optional predicate, the actions to run, what gets sent out, and where you end up next ✨. The table is split across two blocks because of how it's laid out.
I | Predicate Actions Output Events Next State
===o=================================================================
A | P5: 1,2,3,18 EERP WF_RTR
| !P5: 1,2,3 EERP WF_RTR
---+-----------------------------------------------------------------
B | P1: UE
| !P1: 1,2,5 SFID OPOP
---+-----------------------------------------------------------------
C | 1,2 ESID(R=02)
| F_ABORT_IND(R,AO=L) WF_NDISC
---+-----------------------------------------------------------------
D | 1,2 CD IDLELICD
---+-----------------------------------------------------------------
E | 1,2 ESID(R=00) WF_NDISC
---+-----------------------------------------------------------------
F | 4 ERSTWFCD
---+-----------------------------------------------------------------
G | P1: UE
| !P1: 6 SFSTWFCD
---+-----------------------------------------------------------------
H | 1,2 IDLESP
---+-----------------------------------------------------------------
I | 1,2,10 SFID OPOP
---+-----------------------------------------------------------------
J | 1,2 CD IDLELICD
---+-----------------------------------------------------------------
K | P2: 1,2 ESID(R=02)
| F_ABORT_IND(R,AO=L) WF_NDISC
| !P2: 1,2,7,12 F_START_FILE_CF(+) OPO
---+-----------------------------------------------------------------
L | 1,2,8 F_START_FILE_CF(-) IDLESP
---+-----------------------------------------------------------------
M | P3: 1,2,11,13 DATA OPOWFC
| !P3: 1,2,11,13 DATA
| F_DATA_CF OPO
---+-----------------------------------------------------------------
N | F_RTR_CF IDLESP
---+-----------------------------------------------------------------
O | 12 F_DATA_CF OPO
---+-----------------------------------------------------------------
P | Protocol 1,2 ESID(R=02)
| Error F_ABORT_IND(R,AO=L) WF_NDISC
---+-----------------------------------------------------------------
Q | 1,2 ESID(R) WF_NDISC
---+-----------------------------------------------------------------
Continued -->
I | Predicate Actions Output Events Next State
===o=================================================================
R | 1,2,9 EERP WF_RTR
---+-----------------------------------------------------------------
S | WF_NDISC
---+-----------------------------------------------------------------
T | CDSTWFCD
---+-----------------------------------------------------------------
U | User Error UE
---+-----------------------------------------------------------------
W | User Error - Note 1 UE
---+-----------------------------------------------------------------
X | Error
---+-----------------------------------------------------------------
Y | P4 & P5: 1,2,15,18 NERP WF_RTR
| !P4 & !P5: 1,2,15,14 NERP WF_RTR
| P4 & !P5: 1,2,15 NERP WF_RTR
| !P4 & P5: 1,2,15,14,18 NERP WF_RTR
---+-----------------------------------------------------------------
Z | 16 NRSTWFCD
---------------------------------------------------------------------
Z1| P4: 1,2,17 NERP WF_RTR
| !P4: 1,2,17,14 NERP WF_RTR
---------------------------------------------------------------------9.10.3. Predicates and Actions 💅✨
okay so here are the predicates for this state, bestie 🧚
- P1 💖
(I.F_START_FILE_RQ.Restart-pos > 0 AND V.Restart = No) OR (V.Mode = Receiver-only)Note: restart was requested and it's not supported for this session 🙈 - P2 🌸
I.SFPA.Restart-pos > V.Restart-posNote: this is a protocol error because the restart position in the SFPA acknowledgement is bigger than the one you asked for in the SFID request 😭 - P3 🎀
V.Credit_S - 1 = 0Note: the Speaker's Credit is used up 🫠 - P4 🦋
No special logic is in use - P5 👑
Signed EERP/NERP requested
And now the actions, the things that actually happen 🔥
Action 1: Stop inactivity timerAction 2: Start inactivity timerAction 3: Build an EERP from F_EERP_RQAction 4: Store F_EERP_RQ in V.Req-bufAction 5: Build SFID from F_START_FILE_RQandV.Restart-pos = I.F_START_FILE_RQ.Restart-posAction 6: Store F_START_FILE_RQ in V.Req-bufAction 7: Build F_START_FILE_CF(+) from I.SFPAAction 8: Build F_START_FILE_CF(-) from I.SFNAAction 9: Build EERP from F_EERP_RQ stored in V.Req-bufAction 10: Build SFID from F_START_FILE_RQ stored in V.Req-bufandSet V.Restart-posAction 11: Build Exchange BufferAction 12: V.Credit_S = V.WindowAction 13: V.Credit_S = V.Credit_S - 1Action 14: Activate CRC-calculus function. Wrap Exchange buffer in special logicAction 15: Build a NERP from F_NERP_RQAction 16: Store F_NERP_RQ in V.Req-bufAction 17: Build NERP from F_NERP_RQ stored in V.Req-bufAction 18: Sign the contents of NERP/EERP
Note 1: whether to accept this "Request/Event" in this state is up to the local implementation 🤷♀️ BUT the ODETTE state tables assume this event can't happen in this state, so it counts as a user error (UE) ⚠️
9.11. Speaker State Table 2 🌈
9.11.1. State Table 📋
o---------------------------------o
| S | CLOP |
| T |-------------------------o |
| A | OPOWFC | |
| T |---------------------o | |
| E | OPO | | |
|=====================o---+---+---|
| E | F_CLOSE_FILE_RQ | A | E | U |
| V |-----------------+---+---+---|
| E | EFPA | B | B | C |
| N |-----------------+---+---+---|
| T | EFNA | B | B | D |
o---------------------------------o9.11.2. Transition Table 🦋
I | Predicate Actions Output Events Next State
===o=================================================================
A | 1,2,5,7 EFID CLOP
---+-----------------------------------------------------------------
B | 1,2 ESID(R=02)
| F_ABORT_IND(R,AO=L) WF_NDISC
---+-----------------------------------------------------------------
C | P1: 1,2,3 F_CLOSE_FILE_CF(+,SP=No)
| CD IDLELI
| !P1: 1,2,4 F_CLOSE_FILE_CF(+,SP=Yes) IDLESP
---+-----------------------------------------------------------------
D | 1,2,6 F_CLOSE_FILE_CF(-) IDLESP
---+-----------------------------------------------------------------
E | See Note 1
---+-----------------------------------------------------------------
U | User Error UE
---------------------------------------------------------------------9.11.3. Predicates and Actions 🎀
- P1 💅
(I.EFPA.CD-Request = Yes) - P2 🌸
No special logic is in use
Actions, bestie 🔥:
Action 1: Stop inactivity timerAction 2: Start inactivity timerAction 3: O.F_CLOSE_FILE_CF(+).Speaker = NoAction 4: O.F_CLOSE_FILE_CF(+).Speaker = YesAction 5: Build EFID from F_CLOSE_FILE_RQAction 6: Build F_CLOSE_FILE_CF(-) from EFNAAction 7: Set V.Credit_S = 0Action 8: Wrap Exchange buffer in special logic
Note 1: to respect the "half duplex" vibe of ODETTE-FTP 🫶, it's forbidden to send EFID while in the OPOWFC state. EFID can only be sent in the OPO state. The ODETTE-FTP implementation must avoid sending EFID (or receiving F_CLOSE_FILE_RQ) while in the OPOWFC state. Do NOT mix these up 🙅♀️
9.12. Listener State Table 👑
9.12.1. State Table 📋
o---------------------------------------------o
| | RTRP |
| |-------------------------------------o |
| | CLIP | |
| |---------------------------------o | |
| | OPI | | |
| S |-----------------------------o | | |
| T | OPIP | | | |
| A |-------------------------o | | | |
| T | IDLELICD | | | | |
| E |---------------------o | | | | |
| | IDLELI | | | | | |
|=====================o---+---+---+---+---+---+
| | SFID | A | A | B | B | B | B |
| |-----------------+---+---+---+---+---+---+
| E | DATA | B | B | B | I | B | B |
| V |-----------------+---+---+---+---+---+---+
| E | EFID | B | B | B | J | B | B |
| N |-----------------+---+---+---+---+---+---+
| T | F_START_FILE_RS | U | U | H | U | U | U |
| |-----------------+---+---+---+---+---+---+
| | F_CLOSE_FILE_RS | U | U | U | U | K | U |
| |-----------------+---+---+---+---+---+---+
| | CD | C | B | B | B | B | B |
| |-----------------+---+---+---+---+---+---+
| | ESID R=Normal | D | F | D | D | D | D |
| |-----------------+---+---+---+---+---+---+
| | ESID R=Error | D | D | D | D | D | D |
| |-----------------+---+---+---+---+---+---+
| | EERP | E | E | B | B | B | B |
| |-----------------+---+---+---+---+---+---+
| | NERP | L | L | B | B | B | B |
| |-----------------+---+---+---+---+---+---+
| | F_RTR_RS | U | U | U | U | U | M |
o---------------------------------------------o9.12.2. Transition Table 🦋
I | Predicate Actions Output Events Next State
===o=================================================================
A | P1: 1,2 ESID(R=02)
| F_ABORT_IND(R,AO=L) WF_NDISC
| !P1: 1,2,3 F_START_FILE_IND OPIP
---+-----------------------------------------------------------------
B | 1,2 ESID(R=02)
| F_ABORT_IND(R,AO=L) WF_NDISC
---+-----------------------------------------------------------------
C | 1,2 F_CD_IND IDLESPCD
---+-----------------------------------------------------------------
D | 1 F_ABORT_IND(Received
| ESID Reason,AO=D)
| N_DISC_RQ IDLE
---+-----------------------------------------------------------------
E | 1,2,4 F_EERP_IND RTRP
---+-----------------------------------------------------------------
F | 1 F_RELEASE_IND
| N_DISC_RQ IDLE
---+-----------------------------------------------------------------
H | P4: User Error UE
| P2 & !P4 & !P5: 1,2,8 SFPA OPI
| !P2 & !P4 & !P5: 1,2 SFNA IDLELI
| P2 & !P4 & P5: 1,2,5,8 SFPA OPI
| !P2 & !P4 & P5: 1,2,5 SFNA IDLELI
---+-----------------------------------------------------------------
I | P6: 1,2 ESID(R=02)
| F_ABORT_IND(R,A0=L) WF_NDISC
| !P5 & !P6 & !P7: 1,2,7 F_DATA_IND (See Note 1) OPI
| !P5 & !P6 & P7: 1,2,8 F_DATA_IND
| CDT (See Note 1) OPI
| P5 & !P6 & P8: 1,2 ESID(R=07)
| F_ABORT_IND(R,A0=L) WF_NDISC
| P5 & !P6 & !P7 : 1,2,6,7 F_DATA_IND (See Note 1) OPI
| & !P8
| P5 & !P6 & P7 : 1,2,5,6,8 F_DATA_IND OPI
| & !P8 CDT (See Note 1)
---+-----------------------------------------------------------------
J | 1,2 F_CLOSE_FILE_IND CLIP
---+-----------------------------------------------------------------
K | P2 & P3 & !P5: 1,2 EFPA(CD-Req) WF_CD
| P2 & !P3 & !P5: 1,2 EFPA(no CD) IDLELI
| !P2 & !P5: 1,2 EFNA IDLELI
| P2 & !P3 & P5: 1,2,5 EFPA(no CD) IDLELI
| !P2 & P5: 1,2,5 EFNA IDLELI
| P2 & P3 & P5: 1,2,5 EFPA(CD-Req) WF_CD
---+-----------------------------------------------------------------
L | 1,2,10 F_NERP_IND RTRP
---+-----------------------------------------------------------------
M | 1,2 RTR IDLELI
---+-----------------------------------------------------------------
U | User Error UE
---------------------------------------------------------------------9.12.3. Predicates and Actions 🎀
- P1 💖
(I.SFID.Restart-pos > 0 AND V.Restart = No) OR (V.Mode = Sender-only)Note: this is an invalid Start File command 🙅♀️ - P2 🌸
Positive Response - P3 🦋
I.F_CLOSE_FILE_RS(+).Speaker = Yes - P4 🍓
I.F_START_FILE_RS(+).Restart-pos > V.Restart - P5 👑
Special logic is used - P6 😭
V.Credit_L - 1 < 0Note: a protocol error, because the Speaker went over its available transmission credit 💀 - P7 🫠
V.Credit_L - 1 = 0Note: the Speaker's credit has to be reset before it can send any more Data Exchange Buffers ✨ - P8 🔥
The calculus of the received CRC indicates an error
Actions, bestie 🧚:
Action 1: Stop inactivity timerAction 2: Start inactivity timerAction 3: Build F_START_FILE_IND from I.SFIDandV.Restart-pos = I.SFID.Restart-posAction 4: Build F_EERP_IND from I.EERPAction 5: Add special logic header to the command to be sent to the SpeakerAction 6: Suppress the special logic header from the data buffer before giving it to the userAction 7: V.Credit_L = V.Credit_L - 1Action 8: V.Credit_L = V.WindowAction 10: Build F_NERP_IND from I.NERP
Note 1: this is flow control for reception 🌊. The ODETTE-FTP Listener must periodically send new credit to the Speaker. When it does that depends on: 1. The User Monitor's capacity to receive data. 2. The number of buffers available to ODETTE-FTP. 3. The Speaker's available credit, which must be equal to zero.
9.13. Example ✨
Okay bestie, picture an ODETTE-FTP entity that already sent a Start File (SFID) command and is now chilling in the Open Out Pending (OPOP) state 💅. When a Positive Answer (SFPA) comes back, its reaction is in Speaker State Table 1. Basically, transition 'K' gets applied, and it goes like this 👇
if (I.SFPA.Restart-pos > V.Restart-pos) then
begin // invalid restart
Actions: Stop inactivity timer, // reset timer
Start inactivity timer;
Output: ESID(R=02), // to peer ODETTE-FTP
F_ABORT_IND(R,AO=L); // to User Monitor
New State: WF_NDISC;
end
else begin
Actions: Stop inactivity timer, // reset timer
Start inactivity timer;
Build F_START_FILE_CF(+) from I.SFPA
V.Credit_S = V.Window // initialise credit
Output: F_START_FILE_CF(+); // to User Monitor
New State: OPO;
endSo ODETTE-FTP checks the restart position in the Start File Positive Answer (SFPA) command it received 🧐. If that position is invalid, it kills the session 💀 by sending an End Session (ESID) command to its peer and an Abort indication (F_ABORT_IND) to its User Monitor. If the restart position is valid, it builds a Start File confirmation (F_START_FILE_CF), sends it up to the User Monitor, initialises the credit window, and enters the Open Out (OPO) state. Iconic recovery arc 🦋
10. Miscellaneous 🎀
10.1. Algorithm Choice
The choice of algorithms to use for security or compression between partners is for bilateral agreement outside of ODETTE-FTP. So the two partners sort that out between themselves, not inside the protocol 🤝
10.2. Cryptographic Algorithms
The algorithms for symmetric and asymmetric cryptography and hashing are represented by a coded value, called the cipher suite 🔐:
Cipher Suite Symmetric Asymmetric Hashing
------------ ----------------- ------------ -------
01 3DES_EDE_CBC_3KEY RSA_PKCS1_15 SHA-1
02 AES_256_CBC RSA_PKCS1_15 SHA-1Support of all cipher suites listed here is mandatory. No skipping, bestie 🙅♀️
The certificates used must be [X.509] certificates 📜
TripleDES uses Cipher Block Chaining (CBC) mode for extra security, and it runs the Encryption Decryption Encryption (EDE) process with 3 different 64-bit keys. Very layered, very giving 🌈
RSA padding is as defined in [PKCS#1] 🧩
AES uses a 256-bit key in CBC mode. Clean and strong 💪
An extended list of optional cipher suites may be used (Section 10.3), but there is no guarantee that two communicating ODETTE-FTP entities would both support these optional cipher suites. So just because it's on the list doesn't mean your bestie can actually use it 😭
10.3. Protocol Extensions
The algorithms and file enveloping formats available in ODETTE-FTP may be extended outside of this document. Future-proofing era ✨
An up-to-date list of cipher suite values for use in ODETTE-FTP is maintained by ODETTE International, and published on their website at www.odette.org 🌸
10.4. Certificate Services
Certificates and certificate revocation lists may be exchanged as [CMS] enveloped files. It is therefore valid to exchange a [CMS] file that is neither encrypted, compressed, nor signed. Yes, that's allowed, and it's an application implementation issue to figure out the correct course of action when one of those shows up 🤷♀️
11. Security Considerations 🔥
This part is serious, so listen up 👂 ODETTE-FTP security requires the use of [X.509] certificates. If no security options are agreed for use, the send and receive passwords are sent in plain text 😱 Whilst that's acceptable over X.25 and ISDN networks, it's a risky practice over insecure public networks such as the Internet. Not it, bestie 🙅♀️
All, some, or none of the security options available in ODETTE-FTP may be used. No recommendations for the use of these options are provided in this specification. Yes, using the highest-strength encryption algorithms may seem admirable, but there is often a performance tradeoff to be made. Signing all files and acknowledgements also has potential legal implications that should be considered ⚖️
It should be noted that whilst the security measures ensure that an ODETTE-FTP partner is authenticated, it does not necessarily mean that the partner is authorised. Having proven the identity of a partner, it is an application issue to decide whether that partner is allowed to connect or exchange files. Identity is not the same as permission, babe 🗝️
Extracted from [RFC3850]:
"When processing certificates, there are many situations where the processing might fail. Because the processing may be done by a user agent, a security gateway, or other program, there is no single way to handle such failures. Just because the methods to handle the failures have not been listed, however, the reader should not assume that they are not important. The opposite is true: if a certificate is not provably valid and associated with the message, the processing software should take immediate and noticeable steps to inform the end user about it. Some of the many situations in which signature and certificate checking might fail include the following: No certificate chain leads to a trusted CA No ability to check the Certificate Revocation List (CRL) for a certificate An invalid CRL was received The CRL being checked is expired The certificate is expired The certificate has been revoked There are certainly other instances where a certificate may be invalid, and it is the responsibility of the processing software to check them all thoroughly, and to decide what to do if the check fails. See RFC 3280 for additional information on certificate path validation."
The push / pull nature of ODETTE-FTP means that a party can make an outbound connection from behind a firewall to another party and exchange files in both directions 🔄 There's no need for both partners to open ports on their firewalls to allow incoming connections, because only one party needs to allow incoming connections. Firewall-friendly and fr fr 🧱
See Section 1.7 for a discussion of the benefits of session security [TLS] versus file security 🔐
Appendix A. Virtual File Mapping Example 📖
This example shows how a Virtual File gets mapped into a sequence of ODETTE-FTP Data Exchange Buffers. It's literally a poetry moment 🧚
Each line in this extract from 'The Rime of the Ancient Mariner' by Coleridge [RIME] is separated by CR-LFs in a file that's being transmitted as a T format file 📜
It is an ancient Mariner,
And he stoppeth one of three.
"By thy long grey beard and glittering eye,
Now wherefore stopp'st thou me?
"The Bridegroom's doors are opened wide,
And I am next of kin;
The guests are met, the feast is set:
May'st hear the merry din."
He holds him with his skinny hand,
"There was a ship," quoth he.
"Hold off! unhand me, grey-beard loon!"
Eftsoons his hand dropt he.
He holds him with his glittering eye--
The Wedding-Guest stood still,
And listens like a three years; child:
The Mariner hath his will.
The Wedding-Guest sat on a stone:
He cannot chuse but hear;
And thus spake on that ancient man,
The bright-eyed Mariner.
The ship was cheered, the harbour cleared,
Merrily did we drop
Below the kirk, below the hill,
Below the light-house top.The Exchange Buffers below were built from the above 🛠️ The top line of each one is the ASCII code, and the two lines underneath give the hexadecimal value. Read the hex 🤓
Note that:
- The "D" at the beginning of each Exchange Buffer is the command code.
- The "?" preceding each subrecord is the header octet (see the hexadecimal value).
Exchange Buffer 1
D?It is an ancient Mariner,..And he stoppeth one of three..."By
4347267266266666672467666720046626627767767626662662767662002472
4F9409301E01E395E40D129E52CDA1E4085034F005480FE50F6048255EDA2290t?hy long grey beard and glittering eye,..Now wherefore stopp'st
7367266662676726667626662666776766626762004672766766676277677277
4F890CFE70725902512401E407C944529E70595CDAEF70785256F25034F00734 ?thou me?...."The Bridegroom's doors are opened wide,..And I am
2376672663000025662476666766627266677267626766662766620046624266
0F48F50D5FDADA248502294572FFD7304FF2301250F05E5407945CDA1E40901D ?next of kin;..The guests are met, the feast is set:..May'st he
2366772662666300566267677726762667227662666772672767300467277266
0FE5840F60B9EBDA485075534301250D54C04850651340930354ADAD19734085a?r the merry din."....He holds him with his skinny hand,.."Ther
6372766266777266622000046266667266627676266727666672666620025667
1F204850D5229049EE2DADA8508FC43089D07948089303B9EE9081E4CDA24852e? was a ship," quoth he..."Hold off! unhand me, grey-beard loon
6327672627667222776762662002466626662276666626622676726667626666
5F07130103890C2015F48085EDA28FC40F66105E81E40D5C07259D251240CFFE!?"..Eftsoons his hand dropt he.....He holds him with his glitte
2320046776667266726666267677266200004626666726662767626672666776
1F2DA5643FFE30893081E4042F04085EDADA8508FC43089D07948089307C9445r?ing eye--..The Wedding-Guest stood still,..And listens like a
7366626762200566256666662476772776662776662004662667766726666262
2F9E70595DDDA485075449E7D75534034FF40349CCCDA1E40C9345E30C9B5010t?hree years; child:..The Mariner hath his will.....The Wedding-
7367662766773266666300566246766672667626672766620000566256666662
4F8255095123B0389C4ADA4850D129E52081480893079CCEDADA485075449E7DG?uest sat on a stone:..He cannot chuse but hear;..And thus spak
4376772767266262776663004626666672667762677266673004662767727766
7F553403140FE01034FE5ADA85031EEF4038535025408512BDA1E4048530301Be? on that ancient man,..The bright-eyed Mariner.....The ship wa
6326627667266666672666200566267666726766246766672000056627667276
5F0FE0481401E395E40D1ECDA4850229784D59540D129E52EDADA48503890071s? cheered, the harbour cleared,..Merrily did we drop..Below the
7326666766227662667667726666766200467766726662762676700466672766
3F03855254C048508122F5203C51254CDAD5229C90494075042F0DA25CF70485 .kirk, below the hill,..Below the light-house top...
2B667622666672766266662004666727662666672667762767200
03B92BC025CF70485089CCCDA25CF704850C9784D8F53504F0EDAAppendix B. ISO 646 Character Subset 🌈
Okay bestie, this one is the ISO 646 character subset chart 🍓 It's basically a map showing which bit pattern belongs to which character, so you can read it like a grid. Here's the whole thing:
o-----------------------------------------------------------------o
| | 7| 0 | 0 | 0 | 0 | 1 | 1 | 1 | 1 |
| | B -+-----+-----+-----+-----+-----+-----+-----+-----|
| | I 6| 0 | 0 | 1 | 1 | 0 | 0 | 1 | 1 |
| | T -+-----+-----+-----+-----+-----+-----+-----+-----|
| | 5| 0 | 1 | 0 | 1 | 0 | 1 | 0 | 1 |
| |----+-----+-----+-----+-----+-----+-----+-----+-----|
| | | | | | | | | | |
| | | | | | | | | | |
|------------| | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
| BIT | | | | | | | | | |
| 4 3 2 1 | | | | | | | | | |
|============o====o=====+=====+=====+=====+=====+=====+=====+=====|
| 0 0 0 0 | 0 | | | SP | 0 | | P | | |
|------------|----|-----+-----+-----+-----+-----+-----+-----+-----|
| 0 0 0 1 | 1 | | | | 1 | A | Q | | |
|------------+----|-----+-----+-----+-----+-----+-----+-----+-----|
| 0 0 1 0 | 2 | | | | 2 | B | R | | |
|------------+----|-----+-----+-----+-----+-----+-----+-----+-----|
| 0 0 1 1 | 3 | | | | 3 | C | S | | |
|------------+----|-----+-----+-----+-----+-----+-----+-----+-----|
| 0 1 0 0 | 4 | | | | 4 | D | T | | |
|------------+----|-----+-----+-----+-----+-----+-----+-----+-----|
| 0 1 0 1 | 5 | | | | 5 | E | U | | |
|------------+----|-----+-----+-----+-----+-----+-----+-----+-----|
| 0 1 1 0 | 6 | | | & | 6 | F | V | | |
|------------+----|-----+-----+-----+-----+-----+-----+-----+-----|
| 0 1 1 1 | 7 | | | | 7 | G | W | | |
|------------+----|-----+-----+-----+-----+-----+-----+-----+-----|
| 1 0 0 0 | 8 | | | ( | 8 | H | X | | |
|------------+----|-----+-----+-----+-----+-----+-----+-----+-----|
| 1 0 0 1 | 9 | | | ) | 9 | I | Y | | |
|------------+----|-----+-----+-----+-----+-----+-----+-----+-----|
| 1 0 1 0 | 10 | | | | | J | Z | | |
|------------+----|-----+-----+-----+-----+-----+-----+-----+-----|
| 1 0 1 1 | 11 | | | | | K | | | |
|------------+----|-----+-----+-----+-----+-----+-----+-----+-----|
| 1 1 0 0 | 12 | | | | | L | | | |
|------------+----|-----+-----+-----+-----+-----+-----+-----+-----|
| 1 1 0 1 | 13 | | | - | | M | | | |
|------------+----|-----+-----+-----+-----+-----+-----+-----+-----|
| 1 1 1 0 | 14 | | | . | | N | | | |
|------------+----|-----+-----+-----+-----+-----+-----+-----+-----|
| 1 1 1 1 | 15 | | | / | | O | | | |
o-----------------------------------------------------------------oAppendix C. X.25 Specific Information 🎀
The ISO OSI model (International Organization for Standardization, Open Systems Interconnection) is the basis for ODETTE-FTP ✨
ODETTE-FTP covers levels 4 to 7 🧚 and, originally, CCITT X.25 was the only recommended telecommunication protocol for OSI's layers 1, 2, 3.
ISO Reference Model:
+------------------------------+ <==== File Service
| Level-7 FTP application |
|------------------------------|
| Level-6 FTP presentation |
|------------------------------|
| Level-5 FTP session |
|------------------------------|
| Level-4 FTP transport |
|------------------------------| <==== Network Service
| Level-3 X.25 |
|------------------------------|
| Level-2 X.25 |
|------------------------------|
| Level-1 X.25 |
+------------------------------+C.1. X.25 Addressing Restrictions 🦋
When an X.25 call is made over a PSDN, the Network User Address (NUA) of the destination must be specified so the PTT can route the call. The call is then directed to the termination equipment on the user's premises 🏠
It's possible to add extra information to the Call Request Packet, on top of the mandatory NUA that the PTT requires 💖
This extra information can come in 2 kinds:
- (a) A subaddress: It's simply an extension to the address, and it goes into the called address field of the Call Request Packet. This information (Address + Subaddress) is taken from the destination address field of the F_CONNECT_RQ, so from the user's point of view there's no difference between the main address part and the subaddress part.
- (b) User data: There is no standard for user data. Moreover, there's no information in the F_CONNECT_RQ that the ODETTE-entity could use to derive user data for the N_CONNECT_RQ. Therefore, user data shall not be used 🙅♀️
C.2. Special Logic ✨
The SSID field SSIDSPEC says whether special logic must be applied (Y (yes) or N (no)) to the Data Exchange Buffer before ODETTE-FTP moves the data into the NSDU (Network Service Data Unit) and hands control to the Network Service 🫧
C.2.1. When Special Logic Is Not To Be Used 💅
This logic isn't applied to SSRM and SSID commands 🙃
C.2.2. The Need for "Enveloping" Exchange Buffers 🫶
The "special-logic" parameter was created so ODETTE-FTP could be used over asynchronous links 🌊 It could be needed to let terminals reach an X.25 network through an asynchronous entry, via a PAD (Packet Assembly / Disassembly). It's not needed for a whole X.25 connection. This "special-logic" does a CRC function to detect errors caused by the asynchronous medium 🔍
Negotiation of the "special-logic" parameter in the SSID command goes like this:
SSID SSID
-----------------------------------------------
special-logic=yes --------------------->
<------------------------------------ special-logic=yes
or
<------------------------------------ special-logic=no
special-logic=no ---------------------->
<------------------------------------ special-logic=noThis logic gets activated when the "special-logic" parameter in the SSID specifies Y (yes) 🎉
Special logic processing, once activated, works within level 4 of the OSI model 🧁
+------------------------------+ <==== File Service
| Level-7 FTP application |
|------------------------------|
| Level-6 FTP presentation |
|------------------------------|
| Level-5 FTP session |
|------------------------------|
| Level-4 FTP transport |
| SPECIAL LOGIC PROCESSING |
|------------------------------| <==== Network Service
| Level-3 X.25 |
|------------------------------|
| Level-2 X.25 |
|------------------------------|
| Level-1 X.25 |
+------------------------------+C.2.3. Responsibilities of Special Logic 👑
When transmitting an Exchange Buffer with special logic active, layer 4 wraps the Exchange Buffer in synchronization and delineation characters, then protects data integrity with a block checksum (BCS) 🛡️ When receiving an Exchange Buffer with special logic active, layer 4 removes things like synchronization and delineation characters, etc., before passing the Exchange Buffer up to the higher layers 🎀
C.2.4. Extended Exchange Buffer Format 🌸
Each envelope has a 1-byte header on the front and a 2-byte checksum on the end. The checksum is derived the way it's specified in the ISO DIS 8073 TRANSPORT LAYER documentation 📖
The layout of the data buffer is structured like this:
+------------------------------------------------------------------+
| S | B | | B | C |
| T | S | COMPLETE EXCHANGE BUFFER (CEB) | C | / |
| X | N | | S | R |
+------------------------------------------------------------------+
A A A A
| | | |
| +------------- Block sequence number | |
| | |
+----------------- Synchronization character | |
| |
Block checksum -----------------------+ |
|
Delineation character --------------------+The envelope starts with an STX, and the checksum variables are set to 0. The leading STX isn't protected by the checksum calculation, but the receiver's end explicitly protects it with a character compare. The Exchange Buffer is processed one character at a time. As each character comes out of the Exchange Buffer, it goes through the checksum calculation and then, before it's inserted into the envelope, through the Shift-out transparency logic, which results in either one or two characters being inserted. Once the whole Exchange Buffer has been processed, the checksum variables are brought up to date by pushing two X'00's through the checksum calculator, and the two resulting checksum characters are sent to the Shift-out transparency logic to be inserted into the envelope. Last of all, a carriage return (CR) is appended to the envelope. The segment is now ready to go out on the line 🚀
Upon receipt of a valid envelope with the correct sequence number, the host should increment his sequence number register, ready for the next transmission 💜
The receiver initialises its receiving buffer area when it receives an STX character. It places the STX at the beginning of the buffer and resets the checksum variables. All subsequent characters are processed using Shift-out logic before they're inserted into the buffer. At that point they will not be processed by the checksum calculator, although the character following the Shift-out (after subtracting X'20') will be. The checksum characters themselves get processed by the checksum calculator, by virtue of the design of the checksum algorithm 🧮
C.2.5. Error recovery 🙈
C.2.5.1. Mechanism
okay so the error correction scheme is built on three timers plus one tiny message: an ASCII NAK (Negative Acknowledgement) character followed by a C/R. 🎀 The <NAK><C/R> only shows up between the two session partners as a consequence of previous bad data, so it's never just randomly floating around. 🦋
bestie, this part is so important: a user of the error recovery correcting extension must always work with a credit value of 1. 💅 You can force this on any session partner during SSID negotiation, and the effect is a simple half-duplex flip-flop protocol. 🫶
The flow is pretty simple:
- Upon receipt of a bad block, send <NAK><C/R> to the session partner. 💖
- Upon receipt of a <NAK><C/R>, a session partner should retransmit the last block in its entirety. ✨
C.2.5.2. Timers
the majority of error conditions get caught by a bad BCS sequence, no cap. 🔥 BUT some errors are sneaky and can't be caught that way. 😭 For example, if a C/R gets corrupted, the receiver has no idea the block ended, so no matter how long it waits, no more data is coming. 🙈 That's exactly why a timer is the only way to catch this kind of corruption. There are three timers needed to detect all possible malignant conditions of this type. 💀
T1 - Exchange Buffer Time Out (Inactivity or Response)
T2 - Inter Character Time Out
T3 - Data Carrier Detect Loss Time OutThese three timers are in addition to the timer defined in the original protocol. 🌸
TIMER T1 - RESPONSE TIME OUT (DEFAULT = 45 SECONDS):
This one is the high-level block timeout, like the gap between an SFID and its matching SFPA or SFNA response. 🧚
- Started: after the last character of an exchange buffer has been sent to the line. ⭐
- Stopped: when an STX has been received. 🩷
- Expiry: retransmit the whole block again, until the retry limit has been reached. 🔁
TIMER T2 - INTER CHARACTER TIME OUT (DEFAULT = 7 SECONDS):
This one catches errors when individual characters come in wrong. 🍓
- Started: for an asynchronous entity, it starts upon receipt of each character while in synchronisation mode. For an X.25 entity, it starts after a received block that did not terminate an Exchange Buffer. 💜
- Stopped: upon receipt of the next character. 🌈
- Expiry: send a <NAK><C/R>, drop out of synchronised mode, and go back and listen to line. 😤
TIMER T3 - DATA CARRIER TEMPORARY LOSS (DEFAULT = 1 SECOND):
This one is only for asynchronous entities, and it's the one that detects a temporary carrier failure. ⚡
- Started: when DCD (Data Carrier Detect) is lost. 📉
- Stopped: when DCD is regained. 📈
- Expiry: disconnect the session. 👋
C.2.5.3. Types of Error
data corruption, when it happens, can be sorted into one of five categories. 🎀 Let's go through them, bestie. 🦋
(1) CORRUPT STX (START OF TEXT)
the STX never gets seen, so synchronisation never happens. The terminating C/R shows up out of sync, so the receiver never sees the block at all. 😵 A <NAK><C/R> is sent back to the sender to say so. The sender should then retransmit the last block. Each implementation will need to set a retry limit for how many times in a row it tries to resend a block, and a default limit of 5 is recommended. 💅 Any data received outside synchronisation (except <NAK><C/R>) gets ignored. 🙅♀️
(A) (B)
Dropped Start of Text (STX)
+-------------------------+
| | B | | B | C |
-----| | S | CEB | C | / |-----> Not sync
| | N | | S | R |
+-------------------------+
+-------+
| N | C |
<-----| A | / |----- Not sync
| K | R |
+-------+
Exchange Buffer Resent
+-------------------------+
| S | B | | B | C |
-----| T | S | CEB | C | / |-----> Sync
| X | N | | S | R |
+-------------------------+(2) CORRUPT TERMINATION (C/R)
this one looks like a super long stretch of sync with zero activity. ⏳ The T2 timer is what catches it. 🕰️
(A) (B)
Corrupt Carriage Return
+-------------------------+
| S | B | | B | |
-----| T | S | CEB | C | |-----> No activity
| X | N | | S | |
+-------------------------+
+-------+
| N | C | T2
<-----| A | / |----- Timed out
| K | R |
+-------+
Exchange Buffer Resent
+-------------------------+
| S | B | | B | C |
-----| T | S | CEB | C | / |-----> Sync
| X | N | | S | R |
+-------------------------+(3) BAD DATA
(4) BAD BCS (BLOCK CHECK SUM)
here the receiver can't tell whether the problem is the data or the BCS, so either way it's the same move: throw out the Exchange Buffer and send a <NAK><C/R>. 🗑️✨
(A) (B)
Bad Data/BCS
+-------------------------+
| S | B | | B | C | Bad data
-----| T | S | "%! | C | / |-----> detected
| X | N | | S | R |
+-------------------------+
+-------+
| N | C |
<-----| A | / |----- Discard Block
| K | R |
+-------+
Exchange Buffer Resent
+-------------------------+
| S | B | | B | C |
-----| T | S | CEB | C | / |-----> Data OK
| X | N | | S | R |
+-------------------------+(5) BAD BLOCK SEQUENCE NUMBER (BSN)
Exchange Buffers get a circular sequence number, running from 0 up to and including 9. 🔢 This helps spot duplicate or out-of-sequence Exchange Buffers. Once a duplicate block is detected, that Exchange Buffer is discarded. Once an out of sequence block is detected, this should result in a protocol violation. 🚨
Here's an example of how the sequence plays out:
(A) (B)
Exchange Buffer Being Sent
+-------------------------+
| S | | | B | C | Expecting
-----| T | 0 | EERP | C | / |-----> BSN=0
| X | | | S | R | Transmission
+-------------------------+
Exchange Buffer Being Sent
+-------------------------+
| S | | | B | C | Response to
<----| T | 0 | RTR | C | / |----- Previous
| X | | | S | R | Block
+-------------------------+
Exchange Buffer Being Sent
+-------------------------+ Expecting
| S | | | B | C | BSN=1 (Block
-----| T | 1 | SFID | C | / |- // -> lost in
| X | | | S | R | Transmission)
+-------------------------+ T1 Timed Out
Exchange Buffer Being Sent
+-------------------------+
| S | | | B | C | Send last
<----| T | 0 | RTR | C | / |----- Block
| X | | | S | R | again
+-------------------------+
Discard Block
and start
Timer T1
T1 Timed Out
Exchange Buffer Resent
+-------------------------+
| S | | | B | C | Expecting
-----| T | 1 | SFID | C | / |-----> BSN=1
| X | | | S | R | Block OK
+-------------------------+
Exchange Buffer Being Sent
+-------------------------+
| S | | | B | C | Response
<----| T | 1 | SFPA | C | / |----- BSN=1
| X | | | S | R | Block OK
+-------------------------+
Exchange Buffer Being Sent
+-------------------------+
| S | | | B | C |
-----| T | 2 | DATA | C | / |-----> Data OK
| X | | | S | R |
+-------------------------+⚠️ Note: A credit value of 1 must be used to guarantee half-duplex flip-flop. 🫶
C.2.6. Sequence of Events for Special Logic Processing
the following functions run in order, so let's go. 🏃♀️
- Calculation of the Block Sequence Number (BSN): BSN is set to zero by SSID, and the first block gets sent with value zero. The BSN goes up by one for each data buffer to be transmitted. When the BSN value goes past 9, the counter resets to zero. 🔁 Format: numeric/1 pos. 🔢
- Calculation of the Block Checksum (BCS): calculated as specified in the ISO DIS 8073 TRANSPORT LAYER document. 📘 Format: binary/2 pos. 💜
- Shift-out transparency (See TRANSMIT/RECEIVE logic.): to avoid control characters showing up in the data stream, all the characters of the extended Exchange Buffer (except for the STX and carriage return characters enveloping the buffer) go through Shift-out logic. This inserts a character (SO) and adds hex value '20' to the control character. 🪄
- The carriage return is inserted at the end of the data buffer. 🎀
📝 Note: After adding STX, BSN, BCS, CR, and SO-logic, the data buffer may exceed the Data Exchange Buffer size. 🙈
C.2.7. Checksum Creation Algorithm
these all follow the ISO DIS 8073 TRANSPORT LAYER standard. 🌸
SYMBOLS:
C0,C1 Variables used in the algorithm
L Length of the complete NSDU
X Value of the first octet of the checksum parameter
Y Value of the second octet of the checksum parameterARITHMETIC CONVENTIONS:
Addition is performed in one of the two following modes:
a) modulo 255 arithmetic
b) one's complement arithmetic in which if any of the variables
has the value minus zero (i.e., 255) it shall be regarded as
though if was plus zero (i.e., 0).ALGORITHM FOR GENERATING CHECKSUM PARAMETERS:
. Set up the complete NSDU with the value of the checksum parameter
field set to zero.
. Initialise C0 and C1 to zero.
. Process each octet sequentially from i=1 to L by
a) adding the value of the octet to C0, then
b) adding the value of C0 to C1.
. Calculate X and Y such that
X = C0 - C1
Y = C1 - 2*C0
. Place the values X and Y in the checksum bytes 1 and 2,
respectively.C.2.8. Algorithm for checking checksum parameters
. Initialise parameters C0 and C1 to zero.
. Process each octet of NSDU sequentially from i=1 to L by
a) adding the value of the octet to C0, then
b) adding the value of C0 to C1.
. If, when all the octets have been processed, either or both C0
and C1 does not have the value zero, then the checksum formulas
have not been satisfied.Note that the nature of the algorithm is such that it is not necessary to compare explicitly the stored checksum bytes. 🧠✨
C.2.9. Shift-out Processing 🦋
Okay so shift-out processing is the part where we make sure control characters still get through the transfer, no drama. Transmit logic looks at each octet and adds X'20' to any control character it finds, after sending an SO first. Receive logic does the reverse and drops the SO. 💅
C.2.9. Shift-out Processing
(Transparency for all control characters)
TRANSMIT LOGIC (values SO: X'0E' or X'8E')
Buffer(1), ... , (n) is a character in the buffer to be sent.
FOR i=1 to n /* for all octets of the buffer */
IF ((buffer(i) & X'7F') < X'20')
THEN output (SO)
output (buffer(i) + X'20')
ELSE output (buffer(i))
NEXT:
RECEIVE LOGIC (values SO: X'0E' or X'8E')
Buffer(1), ... , (n) is a character in the received buffer.
drop = false
FOR i=1 to n /* for all octets of the buffer */
IF drop = true
THEN output (buffer(i) - X'20')
drop = false
ELSE IF buffer(i) = (X'0D' or X'8D')
THEN Stop
ELSE IF buffer(i) = SO
THEN drop = true
ELSE output (buffer(i))
NEXT:C.3. PAD Parameter Profile 🎀
Before an (ODETTE-FTP) asynchronous entity --> Modem --> PAD --> (ODETTE-FTP) native X.25 link can be established, the target PAD parameters MUST be set so that correct communication is established. It is strongly RECOMMENDED that the PAD parameters are set by the X.25 entity. CCITT recommendations X.3, X.28, and X.29 define the PAD parameters and the procedures for exchanging control information and user data between a PAD and a packet mode Data Terminal Equipment (DTE). 🌸
Here's the parameter list and values used to set the PAD for ODETTE-FTP communication. For more detail, check the specifications for CCITT X.25, X.28, X.29 and X.3. 🧚
No. Description Value Meaning
---------------------------------------------------------------
1 Escape from Data Transfer 0 Controlled by host
2 Echo 0 No Echo
3 Data Forwarding Signal 2 Carriage Return
4 Selection of Idle Timer Delay 20 1 second
5 Ancillary Device Control 0 X-ON, X-OFF not used
6 PAD Service Signals 1 All except prompt
7 Procedure on Break 2 Reset
8 Discard Output 0 Do not discard
9 Padding after Carriage Return 0 No padding
10 Line Folding 0 No line folding
11 Terminal Data Rate - Read only
12 Flow Control of the PAD 0 No flow control used
13 Linefeed Insertion after C/R 0 No linefeed
14 Linefeed Padding 0 No linefeed padding
15 Editing 0 No editing
16 Character Delete 127 Delete
17 Line Delete 24 <CTRL>X
18 Line Display 18 <CTRL>R
19 Editing PAD Service Signals 0 No service signal
20 Echo Mask 0 No echo mask
21 Parity Treatment 0 No parity check
22 Page Wait 0 No page waitNote 1: 👑
Refer to CCITT (1984).
- Parameters 1 - 12 are mandatory and available internationally.
- Parameters 13 - 22 MAY be available on certain networks and MAY also be available internationally.
- A parameter value MAY be mandatory or optional.
The ODETTE profile only refers to parameter values that MUST be implemented internationally if the parameter is made available internationally. 🔥
The ODETTE-FTP "special-logic" parameter MAY be impossible on some PADs because they do not support of some of the parameters (13 - 22). (If the PAD is supporting parity check (21) by default, ODETTE-FTP special logic would be impossible.) 🙈
It is a user responsibility to ensure special logic consistency when making the PAD subscription. 💀
Note 2: 🍓
Some parameters MAY have to be set differently depending on:
- The make and function of the start-stop mode DTE entity.
- The start-stop mode DTE entity ODETTE-FTP monitor function.
- The PAD services implemented.
- The packet mode DTE entity ODETTE-FTP monitor function.
Appendix D. OFTP X.25 over ISDN Recommendation 🌈
This appendix describes the recommendation of ODETTE Group 4 (1) for the use of OFTP (2) over X.25 over ISDN. Basically, this is the official vibe check for file transfer over that setup. 💖
- (1) ODETTE Group 4 is responsible for the specification of Telecommunications standards and recommendations for use within the Automotive Industry.
- (2) OFTP (ODETTE File Transfer Protocol) is the communications standard specified by ODETTE Group 4, designed for the transfer of both EDI and non-EDI data.
This document offers an introductory overview of a technical subject. It is structured to contain the ODETTE recommendation, together with introductory information for anyone not familiar with ISDN, and notes on the issues associated with implementing the recommendation. ✨
The first section gives the detailed ODETTE recommendation, followed by a general discussion. If you're not familiar with the terminology, read the subsequent sections first, bestie 🫶
How far an existing X.25 line adapter MAY be replaced by an ISDN line adapter in an installation depends on the connection opportunities (X.25 or ISDN) of the partners involved in the file transfer. 🦋
Companies that keep many connections to external partners (for example, car manufacturing companies) MAY use OFTP file transfer for compatibility, which MUST always be considered anyway, only in parallel to the X.25 network. 🚗
It is not the aim of this recommendation to remove OFTP file transfer generally from the X.25 network to the ISDN network. This will not always be possible for international connections for technical reasons, and it does not always make sense for connections with a small amount of data to transmit. 🌸
Certainly, using ISDN when exchanging a high volume of data (for example, CAD/CAM files) is much cheaper than using an X.25 network. For such cases, this recommendation SHALL provide a cost-effective possibility for file transfer. 💅
This appendix is organized as follows. D.1 defines the ODETTE recommendation in these terms, D.2 introduces the ISDN environment to the unfamiliar reader, D.3 describes the various methods of connecting to ISDN, and D.4 covers implementation issues. 🧚
D.1. ODETTE ISDN Recommendation 🎀
X.25: Level 2 ISO 7776
Protocol
Level 3 ISO 8208
Protocol
Packet Size 128
Level 2 7
Window Size
Level 3 7
Window Size
First LCN 1
Number of LCNs 1
Facilities Window Size and Packet Size
negotiation shall be supported
by everybody. Call User Data
should not be required.
Calling NUA Optionally provided by the call
initiator.
Called NUA Should be set to a value where
the last 'n' digits can be
specified by the called party.
ISDN: Apart from requesting a 64K unrestricted digital
call, no ISDN features shall be required.
Timeout control: To avoid connections (B channels) within the
circuit-switched ISDN network remaining active
but unused for a long time, the adapter should
include a timeout control.
An ISDN connection (B channel) should be released
if no X.25 packets have been transmitted on this
connection for a longer time. For flexibility a
variable user definable timer should be
incorporated into the adapter.
In the event of a timeout situation the adapter
has to release the ISDN connection and notify the
local OFTP by the transmission of a clear packet.The pages that follow are informational and do not form part of this recommendation. 📖
D.2. Introduction to ISDN 🌸
Okay so the PTTs figured out that digital encoding over these super high-quality, error-free backbone networks lets them offer high bandwidths to end users 🔥. The service is called ISDN (Integrated Services Digital Network) ✨.
The need for bigger EDI transfers, especially CAD/CAM drawings, has everyone wanting high-speed, low-cost comms 💅. Traditional X.25 over a Packet Switched Data Network (PSDN) has been a solid general-purpose subsystem, but its cost and transfer speed make PSDN pricey for this new requirement 😭.
X.25 over the new ISDN gives both the transfer speed and the cost benefits to meet these needs 🦋.
We need precise definitions here, because making sense of ISDN and X.25 means we have to avoid the abuses of the past 🙈 Here's the glossary:
- ISDN: Integrated Services Digital Network
- X.25: X.25 is a communications protocol. It defines the structure of the data packets that make up the protocol and the way they're used.
- PSDN: A PSDN (Packet Switched Data Network) is a network over which the X.25 protocol is operated.
- PSPDN: A PSPDN (Packet Switched Public Data Network) is a PSDN operated by the PTTs. PSPDNs get trade names, like PSS in the UK, Datex-P in Germany, and Transpac in France.
- BRI: Basic Rate Interface, also known as Basic Rate Access, defines an ISDN facility with 2 x 64 K B channels.
- PRI: Primary Rate Interface, also known as Primary Rate Access, defines an ISDN facility with 30 x 64 K B channels.
- Channels: ISDN is typically brought into a consumer's premises using a twisted pair of wire. Over this wire, data can be transmitted in frequency bands. These frequency bands are allocated as channels.
- B channels: The B channels are the data channels and operate at 64 Kb. The two end users of a connection communicate over a B channel.
- D channel: Signalling on ISDN is performed over the D channel. Signalling is used to set up and release connections on the B channels. In some countries, the D channel can also be used for limited X.25 access to the PTTs' PSDN.
The D channel runs at the lower speed of 16 Kb, since it's normally only used at the beginning and end of a connection 💖
2 Wire B2 - 64 Kb
Twisted Pair B1 - 64 Kb
D Channel - 16 KbThe standard for the D channel is called ETSI and it's used in most European countries 🇪🇺 However, some countries that started rolling out ISDN super early used proprietary standards, like these:
1TR6 - Used in Germany
BTNR - Used in the UKEven though there are D channel variations, they won't affect communications over the B channels, because the D channel communication happens between the subscriber and the ISDN service provider 🧚
The consumer's equipment must be able to handle the channel D signalling operated by the ISDN service provider, so there may be a problem with equipment availability and certification. Be aware of this one, bestie 🎀
All the PTTs have committed to migrating to ETSI (also known as EURO-ISDN and Q.931), and many already support both their national variant and ETSI. In this situation it's advisable for the subscriber to select the ETSI variant to avoid unnecessary equipment obsolescence 🍓
- Services: The high-speed service comes in two forms, Basic and Primary.
- Basic: 2+D, the D 2B channel operates at 16 Kb. Basic Rate access is normally provided to the subscriber over simple twisted pair cable.
- Primary: 30B+D, the D channel operates at 64 Kb. Primary Rate access is normally provided to the subscriber over shielded coaxial cable. Note that the bandwidth for Primary is 2.048 Mbit/s.
- Protocols: The B channel is a binary channel and is transparent to the flow of data. That means all of the currently available protocols can operate over a B channel. The most common protocol is X.25.
- X.25: The X.25 protocol is a primary protocol for open computer-to-computer communication.
- Passive Bus: It's possible to have an ISDN service enter a building, and then have an 8-core cable laid within the building with multiple ISDN junction points, the same way you'd have multiple telephone points (extensions) for one external telephone line 📞
Connection Setup ✨
The adapter is responsible for analysing the outgoing X.25 call request, making an ISDN call to a derived ISDN address, establishing a new X.25 level-2 and level-3, and then propagating the X.25 Call Request Packet 💅
Connection Termination 🫶
The termination phase of the X.25 call is made with a Clear Request and finalised with a Clear Confirmation. The recipient of the Clear Confirm should then close down the ISDN connection.
The ISDN connection should only be cleared down if there are no other Switched Virtual Circuits (SVCs) active on it. Note that using multiple simultaneous SVCs is only possible by virtue of bilateral agreement 👑
D.3. Equipment Types 🦋
There are a number of ways to get ISDN/X.25 access 💜
Integrated Adapter 🖥️
This is normally a PC-based ISDN adapter inside a PC. In an environment like this, it's normal for the OFTP application to be able to manipulate the ISDN and X.25 aspects of the session independently, so it has complete control 🔥
The speed of communication between the adapter and the application is at PC BUS speeds, which is huge. That makes it more likely that the effective transmission speed will be nearer the 64K limit ✨
The other perk of this direct linkage is that both 64K B channels may be used in parallel, and both can operate at 64Kb 🎀
Elementary Terminal Adapter 🧩
Here the computer has an integral X.25 adapter communicating X.21 with a Terminal Adapter that fronts the ISDN network. This lets a host with X.25 capability interface to ISDN, normally on a one-to-one basis 🌈
The interface between the Terminal Adapter and the PC will typically only support one 64K B channel. Honestly, that's an inefficient use of the ISDN service 😭
Because the link between the computer and the Terminal Adapter is only X.25, some modification or configuration may be needed inside the Terminal Adapter when new users are added 🙈
X.25 Switch 🏢
This solution is normally found inside larger corporates, where an internal X.25 network is operated or where dual X.25 and ISDN is required 💼
The main benefit of a switch is that it supports both PSDN and ISDN simultaneously. Multiple X.21 lines may also be implemented between the X.25 switch and the computer ⭐
This solution normally takes more effort to configure, and it may require obligations to be placed on how incoming callers specify routing 🫠
D.4. Implementation 🔧
Adding ISDN as an extra subsystem to support OFTP communications comes with implementation problems. They can be categorised as below 💖
- X.25/ISDN Addressing
- Making a Call
- Receiving a Call
- Logical Channel Assignment
- Facilities Negotiation
- ISDN Call Attributes
- Homologation Issues
- Growth
- Performance
D.4.1. X.25/ISDN Addressing 📮
The original OFTP was designed to work over the X.25 networks provided by the PTTs (PSPDNs). The national X.25 networks were interconnected to form a global X.25 network, and everyone adopted a common addressing scheme. There were a few differences in addressing within a national network, but the interface to other countries was pretty rigid and normalised 🌍
PSPDN Numbering 🔢
The addressing scheme adopted in X.25 is a 15-digit number (Network User Address, NUA). The first three digits identify the country, the fourth digit identifies the network within the country, and the remainder specify the individual subscriber plus an optional subaddress. In the UK, where a full X.25 numbering scheme is adopted, a NUA is, e.g., 234221200170, where 2342 is the DNIC (Data Network Identification Code) and 21200170 is the subscriber number ✨
ISDN Numbering ☎️
ISDN is an extension of the normal telephone system. Consequently, it adopts (or rather is) the same numbering scheme as the telephone system (PSTN) 📞
The Numbering Conflict 💥
The PSDN and PSTN numbering schemes are two totally different numbering schemes, and there is no relationship between them. This conflict is at the heart of the matter 🥺
D.4.2. Making a Call 📞
okay so the big drama starts because PSDN and PSTN use different numbering schemes that are totally unconnected 💀 bestie, that's the whole problem.
for X.25 to work over ISDN, there are three main ways to do the addressing 🎀
- Un-mapped: The X.25 called NUA is used as the PSTN number. So an X.25 call to
0733394023results in a PSTN call to0733394023, and the call request that flows after that is also to0733394023✨ - Manipulated: The X.25 called NUA gets altered by subtracting and/or adding digits to produce a PSTN number. So
2394023could be manipulated into a PSTN number of00944733394023, where the prefix2is deleted and replaced by00944733🦋 - Mapped: The X.25 called NUA is used to look up a table of PSTN numbers. So an X.25 call to
234221200170could be mapped to a PSTN call to0733394023, and the call request that flows will stay as234221200170🧚
Un-mapped Calls
Un-mapped calls are where the host-specified X.25 NUA gets converted straight into the matching ISDN number 🌸
So an X.25 call the host issues to X.25 NUA 0733394023 results in an ISDN call to the PSTN number 0733394023. Once the call is established, HDLC/X.25 protocol setup happens, and then an X.25 call request goes across with the NUA 0733394023.
When a PSTN call is made, the number of digits in the called number varies depending on where the called party is. A number can be local, national, or international:
local: 394023
national: 0733 394023
international: 009 44 733 394023Depending on where a call starts, the X.25 NUA in the call request packet can change dramatically 😭 That kind of variation, especially the changing prefix, can be really hard for X.25 routing logic in many products to handle.
Also, when an international PSTN call is made, the PSTN number is likely to be over 15 digits, which is the maximum length of an X.25 NUA. So un-mapped addressing might make some international calls impossible to make 💅
Manipulated Calls
The X.25 called NUA gets changed by subtracting and/or adding digits to produce a PSTN number 🔥
Let's say, by internal convention, the prefix '2' means an international ISDN call. So an X.25 call request of 244733394023 could be manipulated into a PSTN number of 00944733394023, where the prefix '2' is deleted and replaced by '009' (the international prefix) ✨
The X.25 called NUA is usually left un-manipulated. Since individual internal conventions vary, the X.25 called NUA varies too. In the case above it would be 244733394023, but another installation might have the convention that a prefix of '56' means the UK, so the NUA would be 56733394023, where the '56' is deleted and replaced with '00944' to produce the PSTN number 🌈
Mapped Calls
The mapped method gives you maximum flexibility, because:
- The PSTN number can exceed 15 digits.
- The X.25 NUA and PSTN number can be totally different.
The catch with mapped calls is administrative 🙈 IBM mainframes can't handle X.25 over ISDN at all, let alone support mapping. For the mainframe solution to work, an external X.25/ISDN router box is required, and it's the external box's job to provide any mapping that's needed.
This means any change or addition of OFTP partners over ISDN will need access to the computer room or special configuration equipment to update the tables inside the external X.25/ISDN router box 😩
D.4.3. Receiving a Call 📨
we saw in the last section that the called X.25 NUA from an incoming ISDN call can vary a lot. If ISDN/X.25 stays inside a national boundary, though, the variation won't be as big, because most calls will have matching called X.25 NUA and PSTN numbers 💖
X.25 switches and X.25 adapters normally route, accept, or reject calls based on their X.25 called NUA. Routing in particular is based on the X.25 called NUA subaddress.
There are 2 methods to derive this subaddress:
- the last
ndigits are analysed. - the base X.25 NUA of the line is removed from the called NUA. For example, if the called X.25 NUA is
23422120017010and the PSDN subscriber NUA is234221200170, then the subaddress derived by subtraction is10.
Obviously, the second method won't work if the incoming NUA varies 🙃
ISDN Features
ISDN, like X.25, has a core set of features that get enriched with options 🎀 In the original OFTP X.25 specification, it was decided that the Q-bit and D-bit options weren't common to all networks or applications, so they were positively excluded from the specification.
It is proposed that, apart from the core ISDN features needed to establish a call, no other features be used ✨
Subaddressing
There are two forms of ISDN subaddressing: overdialled and specific 🦋
The overdial method lets an ISDN number be artificially extended. A typical case is a private exchange installed in a larger company. Say the base number is 394023 and the computer is on internal extension 1234. By specifying the ISDN number 3940231234, you can get direct access to the internal extension 🧚
The problem with this method is that it extends the called number, and especially for international access that can go over the ISDN numbering limits between countries 😬
The other method is where a discrete subaddress is placed in a specific field in the ISDN call setup. The problem with this one is that the caller has to put the subaddress in the ISDN call setup, and not all ISDN implementations allow that insertion.
So in conclusion, subaddressing of any kind should be avoided 🚫
D.4.4. Logical Channel Assignment 🧩
An X.25 dataline has a number of logical channels associated with it 🌸
The number of channels is part of the agreement between the PTT and the subscriber. The number of channels subscribed to is important, because call failure and similar problems will happen if the number of logical channels defined at the two remote ends is different 💀
If a DTE makes an outgoing call, the highest defined logical channel number will be selected. If the remote Data Communications Equipment (DCE) doesn't have the same number of logical channels defined, then the recipient DCE sees an invalid logical channel being used, and the call will be rejected 🙈
D.4.5. Facilities Negotiation 🤝
in the PSPDN environment, you can subscribe to negotiation of window size and packet size. This negotiation is requested by the originator's DTE and may be passed on to the remote DTE at the originator's DCE's discretion, but it's a local responsibility between the DTE and DCE pair 💅
In the ISDN scenario, where it's a DTE-DTE type connection, the window size and packet size can be left at the default value, so the values can be left out of the call request. If no values are specified, then it's super vital that both DTEs have configured themselves to the recommended defaults ⭐
The symptom of a window size mismatch is a hang situation with no informational error codes 😭 A packet size mismatch could work in some scenarios, but otherwise it would issue error codes saying the packet sizes are invalid.
Window Size
The CCITT X.25 window size has a default value of '2', although subscribers may have other default window sizes, e.g., '7', by agreement with the PTT 🎀
Window size negotiation can be explicitly requested by specifying the requested window size in the Facilities fields of the Call Request packet ✨
Packet Size
The CCITT X.25 packet size has a default value of '128' octets, although subscribers may have other default values, e.g., '1024', agreed with the PTT 🌈
D.4.6. ISDN Call Setup 📞
the initial setup of an ISDN call starts with the transmission of a Q.931 SETUP command. Besides requesting that a call be established, the SETUP command can optionally carry information about the calling party, the called party, routing information, the type of circuit required (for example voice or data), and information about the protocols that are requested to be established 🔥
Setup Parameters:
Bearer capability Information transfer and
access attributes
Called Party number Destination's network address
Called Party subaddress Destination's complete
address
Calling Party number Source's network address
Low-layer compatibility Layer 1-3 indication
High-layer compatibility Layer 4-7 indicationD.4.7. Homologation Issues 🧐
Homologation procedures were adopted and strictly enforced by the PTTs for the quality and conformance of communications equipment connected to the PTTs' services 💅
In particular, commercial X.25 products had to be tested and approved before they could connect to the PTTs' PSPDN. The upside for the subscriber was that the approved equipment very rarely didn't work 🫶
With ISDN, similar approval standards are still enforced, so the subscriber has the same confidence in their ISDN equipment. Wrong 🙃 The ISDN equipment itself is approved, but the X.15 protocol that runs on top of ISDN is now outside the scope of approval services.
This means the quality of conformance to standards for X.25 over ISDN depends on the variable quality procedures within the various ISDN equipment manufacturers 😩
Commercial reputation will probably pressure manufacturers with a programming bug to fix those errors. Even so, subscribers who don't communicate well still have to put time and effort into finding the party with the error.
So far, tests have shown a number of subtle errors, such as timing problems, that took many days to find, prove, and fix 💀
D.4.8. Growth 🌱
Primary Rate Access
If a user plans for growth from the start, Primary Rate Access has apparent financial benefits 💰 Those apparent savings are usually lost to the higher cost of user hardware to support the interface. BRI for data is very common and its cards and adapters are cheap, whereas PRI cards and adapters are few and far between and priced high 😭
Basic Rate Access
One way to grow with ISDN is to buy multiple BRI lines, increasing slowly in units of 2 x B channels 🧚 The PTTs can provide the same subscriber number for all the lines provided in a similar way to the traditional hunting group in PSTN-type working 🫶
D.4.9. Performance ✨
Okay so the obvious win of ISDN is speed 🚀, but ngl, most computer systems out there only have a finite amount of computing power 💀. Attaching several active high-speed communication lines in file transfer mode could eat a huge chunk of CPU, and that's not fair to the other users on the system 🙈.
Connecting an ISDN line with the default 2 B channels to your computer through an X.21 interface gives you a steady 64 Kb throughput only if one of the B channels is active at any one time 🎀.
If there are two 64 Kb channels active and fighting over a single 64 Kb X.21 interface, your effective throughput drops a lot, to just over 50% 😭.
Mainframe issues:
Users with a mainframe front-end are going to feel the cost too 💸. The scanners that check the communications interfaces work based on aggregate throughput, and a 64 Kb interface takes up a lot of cycles 🥲.
Determining 'DTE' or 'DCE' Characteristics
Next up is an extract from the ISO/IEC 8208 (International Standards Organization, International Electrotechnical Commission) (1990-03-15) standard 📚. It's an ISO extension of the CCITT X.25 standard, so it's giving official vibes 👑.
The restart procedure is how a DTE figures out whether it should act as a DCE or keep its DTE role. This matters for logical channel selection during Virtual Call establishment and when resolving a Virtual Call collision 🦋.
When the DTE is ready to initialise the Packet Layer, it shall initiate the restart procedure (i.e., transmit a RESTART REQUEST packet) 🚀. Which role it ends up with depends on what the data exchange equipment (DXE) sends back, as follows:
- a) If the DTE receives a
RESTART INDICATIONpacket with a restarting cause code that is not'DTE Originated'(i.e., it came from a DCE), then the DTE shall maintain its role as a DTE 💖. - b) If the DTE receives a
RESTART INDICATIONpacket with a restarting cause code of'DTE Originated'(i.e., it came from another DTE), then the DTE shall confirm the restart and act as a DCE 🔥. - c) If the DTE receives a
RESTART INDICATIONpacket with a restarting cause code of'DTE Originated'(i.e., it came from another DTE) and it does not have an unconfirmedRESTART REQUESTpacket outstanding (i.e., a restart collision), then the DTE shall consider this restart procedure completed but shall take no further action except to transmit anotherRESTART REQUESTpacket after some randomly chosen time delay 🙃. Main character energy in a standoff, honestly 💅. - d) If the DTE issues a
RESTART REQUESTpacket that is subsequently confirmed with aRESTART CONFIRMATIONpacket, then the DTE shall maintain its role as a DTE ✨.
Acknowledgements 🫶
This document draws extensively on revision 1.4 of the ODETTE File Transfer Specification [OFTP] 🌸.
Many people have contributed to the development of this protocol, and their work is hereby acknowledged 💜.
Normative References 📌
[CMS-Compression]: Gutmann, P., "Compressed Data Content Type for Cryptographic Message Syntax (CMS)", RFC 3274, June 2002.[CMS]: Housley, R., "Cryptographic Message Syntax (CMS)", RFC 3852, July 2004.[ISO-646]: International Organisation for Standardisation, ISO Standard 646:1991, "Information technology -- ISO 7-bit coded character set for information interchange", 1991.[PKCS#1]: Jonsson, J. and B. Kaliski, "Public-Key Cryptography Standards (PKCS) #1: RSA Cryptography Specifications Version 2.1", RFC 3447, February 2003.[TLS]: Dierks, T. and E. Rescorla, "The Transport Layer Security (TLS) Protocol Version 1.1", RFC 4346, April 2006.[UTF-8]: Yergeau, F., "UTF-8, A Transformation Format of ISO 10646", STD 63, RFC 3629, November 2003.[ZLIB]: Deutsch, P. and J-L. Gailly, "ZLIB Compressed Data Format Specification version 3.3", RFC 1950, May 1996.
Informative References 📖
[ISO-6523]: International Organisation for Standardisation, ISO Standard 6523:1984, "Data interchange -- Structures for the identification of organisations", 1984.[OFTP]: Organisation for Data Exchange by Tele Transmission in Europe, Odette File Transfer Protocol, Revision 1.4, April 2000.[FTP]: Postel, J. and J. Reynolds, "File Transfer Protocol", STD 9, RFC 959, October 1985.[RFC793]: Postel, J., "Transmission Control Protocol", STD 7, RFC 793, September 1981.[RIME]: Coleridge, Samuel Taylor, "The Rime of the Ancient Mariner", 1817. A poetic little pick, and I respect it 🧚.[X.509]: Housley, R., Polk, W., Ford, W., and D. Solo, "Internet X.509 Public Key Infrastructure Certificate and Certificate Revocation List (CRL) Profile", RFC 3280, April 2002.[RFC3850]: Ramsdell, B., "Secure/Multipurpose Internet Mail Extensions (S/MIME) Version 3.1 Certificate Handling", RFC 3850, July 2004.
ODETTE Address 🏢
The ODETTE File Transfer Protocol is a product of the Technology Committee of Odette International. The Technology Committee can be contacted via the ODETTE Central Office 📮:
ODETTE INTERNATIONAL Limited
Forbes House
Halkin Street
London
SW1X 7DS
United Kingdom
Phone: +44 (0)171 344 9227
Fax: +44 (0)171 235 7112
EMail: info@odette.org
URL: http://www.odette.orgAuthor's Address ✍️
Ieuan Friend
Data Interchange Plc
Rhys House
The Minerva Business Park
Lynchwood
Peterborough
PE2 6FT
United Kingdom
Phone: +44 (0)1733 371 311
EMail: ieuan.friend@dip.co.ukFull Copyright Statement ©️
Copyright (C) The IETF Trust (2007).
This document is subject to the rights, licenses and restrictions contained in BCP 78 and at www.rfc-editor.org/copyright.html, and except as set forth therein, the authors retain all their rights.
This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Intellectual Property 🔐
The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document, or the extent to which any license under such rights might or might not be available. Nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79.
Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification, can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr.
The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org 💌. Bestie, that's the whole RFC, we made it 🎀✨
📦 where this page comes from
- ietf: rfc5024 2007-11
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms