RFC 5048

Internet Small Computer System Interface (iSCSI) Corrections and Clarifications

📅 published October 2007, from ietf

Network Working Group RFC 5048: Internet Small Computer System Interface (iSCSI) Corrections and Clarifications 💖

Okay bestie, this one is RFC 5048 🎀. M. Chadalapaka, Ed. from Hewlett-Packard Co. wrote it, it's a Standards Track document dated October 2007, and it updates RFC 3720 🦋. That last part is the whole point, because this doc is the corrections and clarifications for iSCSI's original spec.

Status of This Memo ✨

This document is an Internet standards track protocol for the Internet community, and it's asking for discussion and suggestions for improvements 🫶. For the official standardization state and status, check the current edition of the "Internet Official Protocol Standards" (STD 1). Distribution of this memo is unlimited 🌈.

Abstract 🦋

The Internet Small Computer System Interface (iSCSI) is a SCSI transport protocol, and it maps the SCSI architecture and command sets onto TCP/IP 🌸. RFC 3720 defines the iSCSI protocol 💜. This document gathers up the clarifications to that original definition so it can work as a companion for iSCSI implementers 🧚. Where the two differ, this document updates RFC 3720 and its text supersedes RFC 3720's text 👑.

Table of Contents 💅

   1. Introduction ....................................................3
   2. Definitions, Acronyms, and Document Summary .....................3
      2.1. Definitions ................................................3
      2.2. Acronyms ...................................................4
      2.3. Clarifications, Changes, and New Semantics .................5
   3. iSCSI Semantics for SCSI Tasks ..................................7
      3.1. Residual Handling ..........................................7
           3.1.1. Overview ............................................7
           3.1.2. SCSI REPORT LUNS and Residual Overflow ..............7
      3.2. R2T Ordering ...............................................9
      3.3. Model Assumptions for Response Ordering ....................9
           3.3.1. Model Description ..................................10
           3.3.2. iSCSI Semantics with the Interface Model ...........10
           3.3.3. Current List of Fenced Response Use Cases ..........11
   4. Task Management ................................................12
      4.1. Requests Affecting Multiple Tasks .........................12
           4.1.1. Scope of Affected Tasks ............................12
           4.1.2. Clarified Multi-Task Abort Semantics ...............13
           4.1.3. Updated Multi-Task Abort Semantics .................14
           4.1.4. Affected Tasks Shared across RFC 3720 and
                  FastAbort Sessions .................................16
           4.1.5. Implementation Considerations ......................17
           4.1.6. Rationale behind the New Semantics .................17
   5. Discovery Semantics ............................................19
      5.1. Error Recovery for Discovery Sessions .....................19
      5.2. Reinstatement Semantics of Discovery Sessions .............19
           5.2.1. Unnamed Discovery Sessions .........................20
           5.2.2. Named Discovery Sessions ...........................20
      5.3. Target PDUs during Discovery ..............................20
   6. Negotiation and Others .........................................21
      6.1. TPGT Values ...............................................21
      6.2. SessionType Negotiation ...................................21
      6.3. Understanding NotUnderstood ...............................21
      6.4. Outstanding Negotiation Exchanges .........................22
   7. iSCSI Error Handling and Recovery ..............................22
      7.1. ITT .......................................................22
      7.2. Format Errors .............................................22
      7.3. Digest Errors .............................................22
      7.4. Message Error Checking ....................................23
   8. iSCSI PDUs .....................................................23
      8.1. Asynchronous Message ......................................23
      8.2. Reject ....................................................24
   9. Login/Text Operational Text Keys ...............................24
      9.1. TaskReporting .............................................24
   10. Security Considerations .......................................25
   11. IANA Considerations ...........................................26
      11.1. iSCSI-Related IANA Registries ............................26
      11.2. iSCSI Opcodes ............................................26
      11.3. iSCSI Login/Text Keys ....................................28
      11.4. iSCSI Asynchronous Events ................................30
      11.5. iSCSI Task Management Function Codes .....................31
      11.6. iSCSI Login Response Status Codes ........................32
      11.7. iSCSI Reject Reason Codes ................................34
      11.8. iSER Opcodes .............................................35
   12. References ....................................................36
      12.1. Normative References .....................................36
      12.2. Informative References ...................................36
   Acknowledgements ..................................................37

1. Introduction ✨

Okay so, a bunch of iSCSI implementations got built after [RFC3720] came out, and the community is now way richer in implementation know-how 🧠💅. The goal here is to use that expertise to clarify the [RFC3720] semantics and fix defects in [RFC3720] where needed 🔥. This document is meant to give implementers critical guidance on the non-obvious parts of iSCSI, so interoperability gets better and iSCSI adoption speeds up 🚀. It is not trying to be an all-encompassing iSCSI how-to guide, and it is not a complete revision of [RFC3720] 🙈. It's a companion document to [RFC3720] for iSCSI implementers, full stop 🎀.

iSCSI implementers are required to reference [RFC3722] and [RFC3723] in addition to [RFC3720] for mandatory requirements 📚. [RFC3721] also has useful info for iSCSI implementers, so it's worth a look 🦋. Whenever there's a question, the text in this document updates and supersedes the text in [RFC3720] 👑.

2. Definitions, Acronyms, and Document Summary 🌸

2.1. Definitions 💜

Let's get the vocab down first, bestie 🧚.

I/O Buffer A buffer used in a SCSI Read or Write operation, so SCSI data can be sent from it or received into it 🍓. For a read or write data transfer to happen for a task, an I/O Buffer is required on the initiator, and at least one is required on the target 🫶.

SCSI-Presented Data Transfer Length (SPDTL) SPDTL is the aggregate data length of the data that the SCSI layer logically "presents" to the iSCSI layer for a Data-In or Data-Out transfer in the context of a SCSI task 💖. For a bidirectional task there are two SPDTL values, one for Data-In and one for Data-Out ✨. Note that "presenting" includes immediate data per the data transfer model in [SAM2], and it excludes overlapping data transfers, if any, requested by the SCSI layer 🙈.

Third-party This term describes nexus objects (I_T or I_T_L) and iSCSI sessions that take the side effects of actions happening in a separate iSCSI session, even though they weren't part of the action that caused those side effects 🌈. One example is an iSCSI session hosting an I_T_L nexus to an LU that gets reset with an LU Reset TMF through a separate I_T nexus 😭.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119] 🔥.

2.2. Acronyms 🎀

Here's the acronym cheat sheet, no cap 💅:

Acronym        Definition
-------------------------------------------------------------

EDTL           Expected Data Transfer Length

IANA           Internet Assigned Numbers Authority

IETF           Internet Engineering Task Force

I/O            Input - Output

IP             Internet Protocol

iSCSI          Internet SCSI

iSER           iSCSI Extensions for RDMA

ITT            Initiator Task Tag

LO             Leading Only

LU             Logical Unit

LUN            Logical Unit Number

PDU            Protocol Data Unit

RDMA           Remote Direct Memory Access

R2T            Ready To Transfer

R2TSN          Ready To Transfer Sequence Number

RFC            Request For Comments

SAM            SCSI Architecture Model

SCSI           Small Computer Systems Interface
SN             Sequence Number

SNACK          Selective Negative Acknowledgment - also
               Sequence Number Acknowledgement for data

TCP            Transmission Control Protocol

TMF            Task Management Function

TTT            Target Transfer Tag

UA             Unit Attention

2.3. Clarifications, Changes, and New Semantics 💖

okay so this bit is basically a cheat sheet, bestie 🫶 This document changes some [RFC3720] semantics, adds brand new iSCSI semantics, AND clarifies what [RFC3720] already meant. 💅 Every section below gets tagged as a clarification, a change, or a new semantic (sometimes more than one, periodt).

  • Section 3.1.1: clarifies the general principles of iSCSI residuals computation 🧮
  • Section 3.1.2: clarifies iSCSI residuals computation, with an example 📝
  • Section 3.2: clarifies R2T ordering requirements 🔁
  • Section 3.3: new semantics for response ordering in multi-connection iSCSI sessions 🆕
  • Section 4.1.2: clarifications, changes, and new semantics on multi-task abort semantics that all implementations must comply with 🚨
  • Section 4.1.3: changes and new semantics (FastAbort semantics) on multi-task abort semantics that implementations should use for faster error recovery ⚡
  • Section 4.1.3.1: changes in iSCSI clearing effects semantics resulting from the new FastAbort semantics 🧹
  • Section 4.1.4: new semantics on third-party session interactions with the new FastAbort semantics 🤝
  • Section 4.1.5: clarification on implementation considerations related to outstanding data transfers, needed to get correct iSCSI protocol behavior 📦
  • Section 4.1.6: clarification on the intent behind FastAbort semantics (not clarifications to [RFC3720] semantics) 🧠
  • Section 5.1: clarification on error recovery semantics as they apply to Discovery sessions 🔎
  • Section 5.2.1: clarification and new semantics on applying the Initiator Session Identifier (ISID) RULE ([RFC3720]) to Unnamed Discovery Sessions 🪪
  • Section 5.2.2: clarification on applying the ISID RULE to Named Discovery Sessions 🏷️
  • Section 5.3: clarification on allowed PDU types and target Logout notification behavior on a Discovery session 🚪
  • Section 6.1: clarification on whether a Target Portal Group Tag (TPGT) value of zero is legal 🎯
  • Section 6.2: clarification on the negotiating order of SessionType with respect to other keys 🗝️
  • Section 6.3: clarification on the NotUnderstood negotiation response on declarative keys, and the implied semantics 🤔
  • Section 6.4: clarification on the number of legal outstanding negotiation PDUs (Text or Login-related) 📬
  • Section 7.1: clarification on usage of the ITT value of 0xffffffff 🔢
  • Section 7.2: clarification on what counts as a format error for the error recovery defined in [RFC3720] 🧾
  • Section 7.3: change in error recovery semantics for the case of discarding unsolicited PDUs 🗑️
  • Section 7.4: clarification on the intended level of error checking on inbound PDUs 🔍
  • Section 8.1: new semantics for a new AsyncEvent code 🆕
  • Section 8.2: change of legal status for Reject reason code 0x0b, which is now deprecated 🪦
  • Section 9.1: new semantics for a new text key, TaskReporting 📣

3. iSCSI Semantics for SCSI Tasks ✨

3.1. Residual Handling 🧮

okay so Section 10.4.1 of [RFC3720] is where "residuals" get defined, plus how that residual info gets packed into the SCSI Response PDU using the Counts and Flags fields. 🎀 Section 3.1.1 clarifies what [RFC3720] was going for and walks through the general principles, and Section 3.1.2 covers residual handling in the REPORT LUNS scenario. 💜

3.1.1. Overview 🌸

SCSI-Presented Data Transfer Length (SPDTL) is the term this document uses (see Section 1.1 for the definition) to represent the aggregate data length that the target SCSI layer tries to transfer through the local iSCSI layer for a task. Expected Data Transfer Length (EDTL) is the iSCSI term for the length of data that the iSCSI layer expects to transfer for a task. EDTL is specified in the SCSI Command PDU. 🧾

When SPDTL = EDTL for a task, the target iSCSI layer completes the task with no residuals. Whenever SPDTL differs from EDTL for a task, that task is said to have a residual. 🫧

If SPDTL > EDTL for a task, iSCSI Overflow MUST be signaled in the SCSI Response PDU as specified in [RFC3720]. The Residual Count MUST be set to the numerical value of (SPDTL - EDTL). 📈

If SPDTL < EDTL for a task, iSCSI Underflow MUST be signaled in the SCSI Response PDU as specified in [RFC3720]. The Residual Count MUST be set to the numerical value of (EDTL - SPDTL). 📉

bestie, note that Overflow and Underflow are totally independent of Data-In and Data-Out. Either one can happen in either direction of data transfer. 🙈

3.1.2. SCSI REPORT LUNS and Residual Overflow 🦋

this section is all about residual overflow, using the SCSI REPORT LUNS command as the example. 💅 Heads up though: a bunch of other SCSI commands (INQUIRY, for example) have ALLOCATION LENGTH fields that follow the same rules, and everything in the rest of this section applies to all of them. 🫶

The SCSI REPORT LUNS command says the SCSI target has to cap how much data it transfers at a maximum size, which is the ALLOCATION LENGTH the initiator gave in the REPORT LUNS CDB. If the Expected Data Transfer Length (EDTL) in the iSCSI header of the SCSI Command PDU for a REPORT LUNS command is at least as big as that ALLOCATION LENGTH, the SCSI layer truncation stops an iSCSI Residual Overflow from happening. A SCSI initiator can tell that truncation happened from other info at the SCSI layer. The rest of this section spells out the behavior that's required here. 🔐

iSCSI uses the (O) bit (bit 5) in the Flags field of the SCSI Response and the last SCSI Data-In PDUs to say that an iSCSI target couldn't transfer all of the SCSI data for a command to the initiator, because the amount of data to be sent was bigger than the EDTL in the matching SCSI Command PDU (see Section 10.4.1 of [RFC3720]). 🚩

The SCSI REPORT LUNS command asks the target SCSI layer to send back a logical unit inventory (LUN list) to the initiator SCSI layer (see Section 6.21 of SPC-3 [SPC3]). The initiator SCSI layer might not know how big this LUN list is when it sends the REPORT LUNS command, so to avoid getting more LUN list data than it can handle, the REPORT LUNS CDB has an ALLOCATION LENGTH field that caps how much data the initiator is willing to receive for this command. 🧺 If the initiator under-estimated how many logical units the target has, the full inventory might not fit into the ALLOCATION LENGTH it asked for. In that case, Section 4.3.3.6 in [SPC3] says the target SCSI layer "shall terminate transfers to the Data-In Buffer" once it has sent as many bytes as the ALLOCATION LENGTH field specifies. ✂️

So, for a REPORT LUNS command, the SCSI layer at the target hands iSCSI at most ALLOCATION LENGTH bytes of data (the logical unit inventory) for transfer to the initiator. If the iSCSI EDTL is at least as big as the ALLOCATION LENGTH, the SCSI truncation makes sure the EDTL can hold all of the data that's going to be transferred. If the iSCSI layer transfers all of the logical unit inventory data it got from SCSI (meaning, whatever is left after any SCSI truncation) to the initiator, then an iSCSI Residual Overflow hasn't happened, and the iSCSI (O) bit MUST NOT be set in the SCSI Response or the final SCSI Data-Out PDU. 🌈 This isn't a brand new requirement. It's already required by putting [RFC3720] together with the REPORT LUNS command spec in [SPC3]. 🤓 On the flip side, if the iSCSI EDTL is bigger than the ALLOCATION LENGTH in this scenario, the iSCSI Underflow MUST be signaled in the SCSI Response PDU. An iSCSI Underflow MUST also be signaled when the iSCSI EDTL equals the ALLOCATION LENGTH but the logical unit inventory data handed to the iSCSI layer is smaller than the ALLOCATION LENGTH. 📉

The LUN LIST LENGTH field sits at the very start of the logical unit inventory, and truncating the inventory to fit ALLOCATION LENGTH doesn't change it. 🧠 That's the trick that lets a SCSI initiator figure out the inventory it got is incomplete: it just checks whether the LUN LIST LENGTH is bigger than the ALLOCATION LENGTH it put in the REPORT LUNS CDB. 🔍 The usual move in that situation is to send the REPORT LUNS command again with a bigger ALLOCATION LENGTH. 🚀

3.2. R2T Ordering 🔁

Section 10.8 in [RFC3720] says the following:

The target may send several R2T PDUs. It, therefore, can have a number of pending data transfers. The number of outstanding R2T PDUs is limited by the value of the negotiated key MaxOutstandingR2T. Within a connection, outstanding R2Ts MUST be fulfilled by the initiator in the order in which they were received.

the [RFC3720] text wasn't clear about its scope. It could mean per task, or across all tasks on a connection, and you could read it either way. 🤷‍♀️ This section makes it official that the scope is a task. Nothing about R2T ordering across tasks is implied, either when the target generates them or when the initiator fulfills them. 🙅‍♀️ So, outstanding R2Ts within a task MUST be fulfilled by the initiator in the order in which they were received on a connection. 🫶

3.3. Model Assumptions for Response Ordering 🎀

When an iSCSI session is made of multiple connections, the Response PDUs (task responses or TMF responses) that come from the target SCSI layer get spread across those connections by the target iSCSI layer, following the iSCSI connection allegiance rules. 🛤️ This process usually can't keep the responses in order by the time they reach the initiator SCSI layer. Ordering isn't expected across SCSI responses anyway, so this works fine in the general case. 💖 But for the special cases where the SCSI layer does want some ordering, the following "Response Fence" semantics are defined for handling SCSI response messages as they get handed off from the SCSI protocol layer to the iSCSI layer. 🧱✨

3.3.1. Model Description ✨

okay so the target SCSI protocol layer hands SCSI response messages off to the target iSCSI layer by invoking the "Send Command Complete" protocol data service ([SAM2], clause 5.4.2) and the "Task Management Function Executed" ([SAM2], clause 6.9) service 💅. when the iSCSI layer gets one of those response messages, it does the Response Fence behavior for certain SCSI response messages (Section 3.3.3 has the specific cases where the semantics must be realized 🫶). whenever Response Fence is required for a SCSI response message, the target iSCSI layer MUST make sure these two things are true when it delivers that response to the initiator iSCSI layer 🔥:

  • (1) a Response with Response Fence MUST be delivered chronologically after all the "preceding" responses on the I_T_L nexus, if the preceding responses are delivered at all, to the initiator iSCSI layer.
  • (2) a Response with Response Fence MUST be delivered chronologically before all the "following" responses on the I_T_L nexus.

the "preceding" and "following" notions are about the order of handoff of a response message from the target SCSI protocol layer to the target iSCSI layer, so it's all about the order things get handed over 🦋.

3.3.2. iSCSI Semantics with the Interface Model ✨

okay this part is the actual to-do list 📝. whenever the TaskReporting key (Section 9.1) is negotiated to ResponseFence or FastAbort for an iSCSI session and the Response Fence behavior is required for a SCSI response message, the target iSCSI layer MUST perform the actions in this section for that session 💜.

  • a) if it's a single-connection session, no special processing is required. the standard SCSI Response PDU build and dispatch process just happens 🌸.
  • b) if it's a multi-connection session, the target iSCSI layer takes note of the last-sent and unacknowledged StatSN on each of the connections in the iSCSI session, and then waits for an acknowledgement of each of those StatSNs to clear the fence. NOP-In PDUs MAY be used to solicit acknowledgements as needed so this goes faster ⚡. the SCSI response that needs Response Fence behavior MUST NOT be sent to the initiator before acknowledgements are received for each of the unacknowledged StatSNs 🙈.
  • c) the target iSCSI layer must wait for an acknowledgement of the SCSI Response PDU that carried the SCSI response needing the Response Fence behavior. the fence MUST be considered cleared only after that acknowledgement is received ✨.
  • d) all further status processing for the LU only resumes after the fence is cleared 🌈. if any new responses for the I_T_L nexus come in from the SCSI layer before the fence clears, those Response PDUs MUST be held and queued at the iSCSI layer until it does 🧚.

3.3.3. Current List of Fenced Response Use Cases ✨

this section lists the fenced response use cases that iSCSI implementations MUST comply with 💖. BUT it's not a full list, not even close. as SCSI protocol specs evolve, they'll say when response fencing is required on a case-by-case basis 🍓.

whenever the TaskReporting key (Section 9.1) is negotiated to ResponseFence or FastAbort for an iSCSI session, the target iSCSI layer MUST assume that Response Fence is required for the following SCSI completion messages:

  1. the first completion message carrying the UA after the multi-task abort on issuing and third-party sessions. see Section 4.1.1 for the related TMF discussion 👑.
  2. the TMF Response carrying the multi-task TMF Response on the issuing session.
  3. the completion message indicating ACA establishment on the issuing session.
  4. the first completion message carrying the ACA ACTIVE status after ACA establishment on issuing and third-party sessions.
  5. the TMF Response carrying the Clear ACA response on the issuing session.
  6. the response to a PERSISTENT RESERVE OUT/PREEMPT AND ABORT command 💀.
okay important note 🚨: since [RFC3720] doesn't have any ACA-related fencing requirements, initiator implementations SHOULD NOT use ACA on multi-connection iSCSI sessions to targets that only comply with [RFC3720] 🙅‍♀️. initiators that want to use ACA on multi-connection iSCSI sessions SHOULD first check response-fencing behavior by negotiating for the ResponseFence or FastAbort values for the TaskReporting (Section 9.1) key 🫶.

4. Task Management ✨

4.1. Requests Affecting Multiple Tasks 🎀

this section clarifies and updates the original text in Section 10.6.2 of [RFC3720] 📖. the clarified semantics (Section 4.1.2) are a superset of the protocol behavior required in the original text, and all iSCSI implementations MUST support the new behavior 💅. the updated semantics (Section 4.1.3), on the other hand, are mandatory only when the new key TaskReporting (Section 9.1) is negotiated to "FastAbort" 🦋.

4.1.1. Scope of Affected Tasks 🌸

this section defines what "affected tasks" means in multi-task abort scenarios. the scope definitions here apply to both the clarified protocol behavior (Section 4.1.2) and the updated protocol behavior (Section 4.1.3) 🍓.

  • ABORT TASK SET: all outstanding tasks for the I_T_L nexus identified by the LUN field in the ABORT TASK SET TMF Request PDU.
  • CLEAR TASK SET: all outstanding tasks in the task set for the LU identified by the LUN field in the CLEAR TASK SET TMF Request PDU. see [SPC3] for the definition of a "task set" 🧠.
  • LOGICAL UNIT RESET: all outstanding tasks from all initiators for the LU identified by the LUN field in the LOGICAL UNIT RESET Request PDU.
  • TARGET WARM RESET/TARGET COLD RESET: all outstanding tasks from all initiators across all LUs that the TMF-issuing session has access to on the SCSI target device hosting the iSCSI session 🌍.

quick usage thing 🫶: an "ABORT TASK SET TMF Request PDU" in the text above is an iSCSI TMF Request PDU with the "Function" field set to "ABORT TASK SET", as defined in [RFC3720]. the same goes for the other scope descriptions ✨.

4.1.2. Clarified Multi-Task Abort Semantics 💖

all iSCSI implementations MUST support the protocol behavior in this section as the default behavior 👑. executing ABORT TASK SET, CLEAR TASK SET, LOGICAL UNIT RESET, TARGET WARM RESET, and TARGET COLD RESET TMF Requests goes through the following sequence of actions, in the specified order, on the specified party 🔥.

the initiator iSCSI layer:

  • a) MUST continue to respond to each TTT received for the affected tasks.
  • b) SHOULD process any responses received for affected tasks in the normal fashion. this is fine because the responses are guaranteed to have been sent before the TMF response 💅.
  • c) SHOULD receive the TMF Response concluding all the tasks in the set of affected tasks, unless the initiator has done something (e.g., LU reset, connection drop) that may stop the TMF Response from being sent or received. so the initiator MUST conclude all affected tasks as part of this step either way, and MUST discard any TMF Response received after the affected tasks are concluded 🙈.

the target iSCSI layer:

  • a) MUST wait for responses on currently valid target-transfer tags of the affected tasks from the issuing initiator. MAY wait for responses on currently valid target-transfer tags of the affected tasks from third-party initiators 🦋.
  • b) MUST wait (at the same time as the wait in Step a) for all commands of the affected tasks to be received, based on the CmdSN ordering. SHOULD NOT wait for new commands on third-party affected sessions. only the instantiated tasks have to be considered when figuring out the affected tasks 🧚. for target-scoped requests (i.e., TARGET WARM RESET and TARGET COLD RESET), any commands not yet received on the issuing session in the command stream can be treated as received with no command waiting period. so the entire CmdSN space up to the CmdSN of the task management function can be "plugged" 🫧.
  • c) MUST propagate the TMF request to the target SCSI layer and receive the response from it 💜.
  • d) MUST provide the Response Fence behavior for the TMF Response on the issuing session, as specified in Section 3.3.2 ✨.
  • e) MUST provide the Response Fence behavior on the first post-TMF Response on third-party sessions, as specified in Section 3.3.2. if some tasks come from non-iSCSI I_T_L nexuses, then the way the target makes sure all affected tasks have returned their status to the initiator is defined by the specific non-iSCSI transport protocol(s) 🌈.

okay so technically the TMF servicing is complete at Step d 🎀. data transfers for terminated tasks may still be in progress on third-party iSCSI sessions even at the end of Step e. the TMF Response MUST NOT be sent by the target iSCSI layer before the end of Step d, and MAY be sent at the end of Step d, despite those outstanding data transfers, and before Step e finishes 💖.

4.1.3. Updated Multi-Task Abort Semantics 💅

Okay bestie, this is the FastAbort glow-up 🔥. Protocol behavior in this section MUST be implemented by all iSCSI implementations complying with this document. It MUST also be exhibited by iSCSI implementations on an iSCSI session when they negotiate the TaskReporting (Section 9.1) key to FastAbort on that session. Running ABORT TASK SET, CLEAR TASK SET, LOGICAL UNIT RESET, TARGET WARM RESET, or TARGET COLD RESET TMF Requests is a sequence of steps, done in order, by specific parties 🦋.

The initiator iSCSI layer:

  • a. The initiator MUST NOT send any more Data-Out PDUs for affected tasks on the issuing connection of the issuing iSCSI session once the TMF is sent to the target.
  • b. The initiator SHOULD process any responses received for affected tasks in the normal fashion. This is totally fine because those responses are guaranteed to have been sent before the TMF response 🙌.
  • c. The initiator MUST respond to each Async Message PDU with AsyncEvent=5, as defined in Section 8.1.
  • d. The initiator MUST treat the TMF response as terminating all affected tasks that haven't gotten responses yet 💀. It MUST also discard any responses for affected tasks that show up after the TMF response is passed to the SCSI layer. (The semantics here make sure that out-of-order situation never happens with a compliant target implementation, so it's basically a safety net 🧚.)

The target iSCSI layer:

  • a. The target MUST wait for all commands of the affected tasks to be received, based on the CmdSN ordering on the issuing session. It SHOULD NOT wait for new commands on third-party affected sessions. Only the instantiated tasks need to be considered when figuring out which tasks are affected 👀. For target-scoped requests (that's TARGET WARM RESET and TARGET COLD RESET), every command that hasn't been received yet on the issuing session in the command stream can be treated as received, with no waiting period. In other words, the entire CmdSN space up to the CmdSN of the task management function can be "plugged" 🔌.
  • b. The target MUST propagate the TMF request to the target SCSI layer and receive the response from it 📬.
  • c. The target MUST leave all active "affected TTTs" (that is, active TTTs associated with affected tasks) valid 🫶.
  • d. The target MUST send an Asynchronous Message PDU with AsyncEvent=5 (Section 8.1) on:
    • i) each connection of each third-party session to which at least one affected task is allegiant, if TaskReporting=FastAbort is operational on that third-party session, and
    • ii) each connection except the issuing connection of the issuing session that has at least one allegiant affected task.

If there are multiple affected LUs (say, because of a target reset), then one Async Message PDU MUST be sent for each such LU on each connection that has at least one allegiant affected task. The LUN field in the Asynchronous Message PDU MUST be set to match the LUN for each such LU 🌸.

  • e. The target MUST address the Response Fence flag on the TMF Response on the issuing session, as defined in Section 3.3.2 ✨.
  • f. The target MUST address the Response Fence flag on the first post-TMF Response on third-party sessions, as defined in Section 3.3.2. If some tasks come from non-iSCSI I_T_L nexuses, then how the target makes sure all affected tasks have returned their status to the initiator is up to the specific non-iSCSI transport protocol(s) 🧚‍♀️.
  • g. The target MUST free up the affected TTTs (and STags, if applicable) and the corresponding buffers, if any, once it receives each associated NOP-Out acknowledgement that the initiator generated in response to each Async Message 🧹.

Technically, the TMF servicing is complete at Step e 🏁. Data transfers for terminated tasks might still be going on even at the end of Step f, though 😭. A TMF Response MUST NOT be sent by the target iSCSI layer before the end of Step e. It MAY be sent at the end of Step e, even with those outstanding data transfers, all the way until Step g. Step g is the event that frees up any resources that were reserved to support those outstanding data transfers 💜.

4.1.3.1. Clearing Effects Update

Okay so, Appendix F.1 of [RFC3720] says the clearing effects of target and LU resets on "Incomplete TTTs" are "Y". That meant a target warm reset, a target cold reset, or an LU reset would clear the active TTTs once it finished 🧼. But the TaskReporting=FastAbort (Section 9.1) semantics in this section don't guarantee the active TTTs are cleared by the end of the reset operations. The new semantics are actually designed to let the TTTs get cleared in a "lazy" way after the TMF Response is delivered 😴. So when TaskReporting=FastAbort is operational on a session, the clearing effects of reset operations on "Incomplete TTTs" are "N" 🎀.

4.1.4. Affected Tasks Shared across RFC 3720 and FastAbort Sessions 🤝

If an iSCSI target implementation can support TaskReporting=FastAbort functionality (Section 9.1), it might end up in a situation where some sessions have TaskReporting=RFC3720 operational (RFC 3720 sessions) while other sessions have TaskReporting=FastAbort operational (FastAbort sessions). This can happen even while they're all accessing a shared set of affected tasks (Section 4.1.1) 👯‍♀️.

If the issuing session is an RFC 3720 session, the iSCSI target implementation is FastAbort-capable, and the third-party affected session is a FastAbort session, then the following behavior SHOULD be exhibited by the iSCSI target layer 💖:

  • a. Between Steps c and d of the target behavior in Section 4.1.2, send an Asynchronous Message PDU with AsyncEvent=5 (Section 8.1) on each connection of each third-party session to which at least one affected task is allegiant. If there are multiple affected LUs, then send one Async Message PDU for each such LU on each connection that has at least one allegiant affected task. When sent, the LUN field in the Asynchronous Message PDU MUST be set to match the LUN for each such LU.
  • b. After Step e of the target behavior in Section 4.1.2, free up the affected TTTs (and STags, if applicable) and the corresponding buffers, if any, once each associated NOP-Out acknowledgement is received that the third-party initiator generated in response to each Async Message sent in Step a.

If the issuing session is a FastAbort session, the iSCSI target implementation is FastAbort-capable, and the third-party affected session is an RFC 3720 session, then the following behavior MUST be exhibited by the iSCSI target layer. Asynchronous Message PDUs MUST NOT be sent on the third-party session to prompt the FastAbort behavior 🙅‍♀️.

If the third-party affected session is a FastAbort session and the issuing session is also a FastAbort session, the initiator in the third-party role MUST respond to each Async Message PDU with AsyncEvent=5, as defined in Section 8.1. Note that an initiator MAY thus receive these Async Messages on a third-party affected session even if that session is a single-connection session 🌈.

4.1.5. Implementation Considerations 🛠️

In both the clarified semantics (Section 4.1.2) and the updated semantics (Section 4.1.3), there might be outstanding data transfers even after the TMF completion is reported on the issuing session. For iSCSI/iSER [iSER], these would be tagged data transfers for STags that aren't owned by any active tasks 🧐. Whether real buffers support these data transfers is implementation-dependent. However, the data transfers logically MUST be silently discarded by the target iSCSI layer in all cases 🫠. A target MAY also choose, on an implementation-defined internal timeout, to drop the connections on which it didn't receive the expected Data-Out sequences (Section 4.1.2) or NOP-Out acknowledgements (Section 4.1.3). It can do this to reclaim the associated buffer, STag, and TTT resources as appropriate 🧺.

4.1.6. Rationale behind the New Semantics 🧠

Okay, there are basically three big objectives behind the semantics in Sections 4.1.2 and 4.1.3 ✨:

  1. Maintaining an ordered command flow I_T nexus abstraction to the target SCSI layer, even with multi-connection sessions.
    • Target iSCSI processing of a TMF request has to maintain the single flow illusion. Target behavior in Step b of Section 4.1.2 and Step a of Section 4.1.3 correspond to this objective 🎭.
  2. Maintaining a single ordered response flow I_T nexus abstraction to the initiator SCSI layer, even with multi-connection sessions. This matters when one response (i.e., the TMF response) could imply the status of other unfinished tasks from the initiator's perspective 📦.
    • The target must ensure that the initiator doesn't see "old" task responses (ones that were placed on the wire chronologically earlier than the TMF Response) after it has seen the TMF response. The target behavior in Step d of Section 4.1.2 and Step e of Section 4.1.3 correspond to this objective 💅.
    • Whenever the result of a TMF action is visible across multiple I_T_L nexuses, [SAM2] requires the SCSI device server to trigger a UA on each of the other I_T_L nexuses. Once an initiator is notified of such a UA, the application client on the receiving initiator is required to clear its task state (clause 5.5 in [SAM2]) for the affected tasks. So it would be inappropriate to deliver a SCSI Response for a task after the task state has been cleared on the initiator (that is, after the UA is notified) 🙈. The UA notification contained in the first SCSI Response PDU on each affected Third-party I_T_L nexus after the TMF action thus MUST NOT pass the affected task responses on any of the iSCSI sessions accessing the LU. The target behavior in Step e of Section 4.1.2 and Step f of Section 4.1.3 correspond to this objective 🔒.
  3. Draining all active TTTs corresponding to affected tasks in a deterministic fashion 🚰.
    • Data-Out PDUs with stale TTTs that arrive after the tasks are terminated can cause a buffer management problem, even for traditional iSCSI implementations. For iSCSI/iSER implementations, it's fatal for the connection 😱. Either the termination of affected tasks should be postponed until the TTTs are retired (as in Step a of Section 4.1.2), or the TTTs and the buffers should stay allocated beyond task termination, to be deterministically freed up later (as in Steps c and g of Section 4.1.3).

The only other notable optimization is the plugging 🔌. If all tasks on an I_T nexus will be aborted anyway (like with a target reset), there's no need to wait to receive all commands in order to plug the CmdSN holes. The target iSCSI layer can simply plug all missing CmdSN slots and move on with TMF processing. The first objective (maintaining a single ordered command flow) is still met with this optimization, because the target SCSI layer only ever sees ordered commands 🥰.

5. Discovery Semantics ✨

5.1. Error Recovery for Discovery Sessions 🎀

Okay so, negotiating the key ErrorRecoveryLevel isn't required for Discovery sessions (the ones that negotiated SessionType=Discovery) 💅 because the default value of 0 is necessary and sufficient for them. Some legacy iSCSI implementations might still try to negotiate ErrorRecoveryLevel on a Discovery session anyway 🙈. When the remote side makes that attempt, a compliant iSCSI implementation MUST propose a value of 0 (zero) in response. So the operational ErrorRecoveryLevel for Discovery sessions MUST be 0. This follows naturally from the functionality constraints that [RFC3720] puts on Discovery sessions ✨.

5.2. Reinstatement Semantics of Discovery Sessions 🦋

Discovery sessions are meant to be relatively short-lived ⏳. Initiators aren't expected to set up multiple Discovery sessions to the same iSCSI Network Portal (see [RFC3720]). An initiator may use the same iSCSI Initiator Name and ISID when it establishes different unique sessions with different targets and/or different portal groups. This is discussed in Section 9.1.1 of [RFC3720] and is honestly encouraged as conservative reuse of ISIDs 🌸. The ISID RULE in [RFC3720] says there must not be more than one session with a matching 4-tuple: <InitiatorName, ISID, TargetName, TargetPortalGroupTag>. The spirit of the ISID RULE applies to Discovery sessions just like it does for Normal sessions, but some Discovery sessions differ from Normal sessions in two important ways 👀:

  • Because [RFC3720] allows a Discovery session to be established without specifying a TargetName key in the Login Request PDU (let's call that an "Unnamed" Discovery session), there's no Target Node context to enforce the ISID RULE with.
  • Portal Groups are only defined in the context of a Target Node. When the TargetName key is NULL-valued (meaning not specified), the TargetPortalGroupTag can't be figured out, so the ISID RULE can't be enforced with it either.

The next sections cover the two scenarios separately: Named Discovery sessions and Unnamed Discovery sessions 💖.

5.2.1. Unnamed Discovery Sessions 🧚

For Unnamed Discovery sessions, the targets don't have access to either the TargetName or the TargetPortalGroupTag to enforce the ISID RULE, so the following rule applies 🫶.

UNNAMED ISID RULE: Targets MUST enforce the uniqueness of the following 4-tuple for Unnamed Discovery sessions: <InitiatorName, ISID, NULL, TargetAddress>. The following semantics are implied by this uniqueness requirement.

Targets SHOULD allow concurrent establishment of one Discovery session with each of their Network Portals by the same initiator port with a given iSCSI Node Name and an ISID. If the same initiator port sets up concurrent Discovery sessions to other Network Portals, each of those sessions MUST be treated as independent sessions. In other words, one session MUST NOT reinstate the other 🙅‍♀️.

A new Unnamed Discovery session with a matching <InitiatorName, ISID, NULL, TargetAddress> to an existing Discovery session MUST reinstate the existing Unnamed Discovery session. So only an Unnamed Discovery session may reinstate an Unnamed Discovery session 👑.

5.2.2. Named Discovery Sessions 💜

For a Named Discovery session, the initiator specifies the TargetName key, so the target can unambiguously work out the TargetPortalGroupTag too 🙌. Since all four elements of the 4-tuple are known, targets MUST enforce the ISID RULE with no changes from [RFC3720] semantics. A new session with a matching <InitiatorName, ISID, TargetName, TargetPortalGroupTag> will reinstate an existing session. Note that in this case, any new iSCSI session (Discovery or Normal) with the matching 4-tuple may reinstate an existing Named Discovery iSCSI session 🔥.

5.3. Target PDUs during Discovery 🌈

Targets SHOULD NOT send any responses other than a Text Response and Logout Response on a Discovery session once it's in Full Feature Phase 🙅‍♀️.

Implementation Note: A target may simply drop the connection in a Discovery session when it would otherwise have requested a Logout via an Async Message on Normal sessions 💅.

6. Negotiation and Others ✨

6.1. TPGT Values 🍓

[SAM2] and [SAM3] specifications incorrectly note in their informative text that the TPGT value should be non-zero, even though [RFC3720] allows the value of zero for TPGT 😭. This section clarifies that a zero value is expressly allowed as a legal value for TPGT. This discrepancy has been corrected in [SAM4] ✅.

6.2. SessionType Negotiation 🌸

During the Login Phase, the initiator offers the SessionType key to choose the type of session it wants to create with the target. The target may accept or reject that offer. Depending on the session type, a target may decide which resources to allocate and which security to enforce for the session, and so on 🔐. If the SessionType key is going to be offered as "Discovery", it SHOULD be offered in the initial Login request by the initiator 💖.

6.3. Understanding NotUnderstood 🧠

[RFC3720] defines NotUnderstood as a valid answer during a negotiation text key exchange between two iSCSI nodes. NotUnderstood has the reserved meaning that the sending side did not understand the proposed key semantics. This section clarifies that NotUnderstood is a valid answer for both declarative and negotiated keys. The general iSCSI philosophy is that comprehension comes before processing for any iSCSI key. So a proposer of an iSCSI key, negotiated or declarative, in a text key exchange MUST be able to properly handle a NotUnderstood response 🫶.

How you handle a NotUnderstood response depends on where the key is specified and whether it's declarative or negotiated. All keys defined in [RFC3720] MUST be supported by all compliant implementations. So a NotUnderstood answer on any of the [RFC3720] keys MUST therefore be considered a protocol error and handled accordingly 🚨. For all other later keys, a NotUnderstood answer concludes the negotiation for a negotiated key, whereas for a declarative key, a NotUnderstood answer simply tells the declarer that the receiver doesn't comprehend it 🤷‍♀️.

In either case, a NotUnderstood answer always requires that the protocol behavior associated with that key not be used within the scope of the key (connection/session) by either side 🙅‍♀️.

6.4. Outstanding Negotiation Exchanges 🎀

There was some uncertainty around the number of outstanding Login Response PDUs on a connection 🤔. [RFC3720] offers an analogy to SCSI linked commands for Login and Text negotiations in Sections 5.3 and 10.10.3, respectively, but doesn't make it fully explicit. This section offers a clarification on that point 💅.

There MUST NOT be more than one outstanding Login Request, Login Response, Text Request, or Text Response PDU on an iSCSI connection. An outstanding PDU in this context is one that has not been acknowledged by the remote iSCSI side ⏳.

7. iSCSI Error Handling and Recovery 🦋

7.1. ITT 💫

An ITT value of 0xffffffff is reserved and MUST NOT be assigned for a task by the initiator. The only time it may show up on the wire is in a target-initiated NOP-In PDU (and in the initiator response to that PDU, if one is needed). Section 10.19 in [RFC3720] mentions this in passing, but it's noted here again to make it obvious, since the semantics apply to initiators in general 📣.

7.2. Format Errors 🧩

Section 6.6 of [RFC3720] discusses format error handling. This section elaborates on the "inconsistent" PDU field contents noted in [RFC3720] 🔍.

All initiator-detected PDU construction errors MUST be considered format errors. Some examples of these errors are:

  • NOP-In with a valid TTT but an invalid LUN
  • NOP-In with a valid ITT (i.e., a NOP-In response) and also a valid TTT
  • SCSI Response PDU with Status=CHECK CONDITION, but DataSegmentLength = 0

7.3. Digest Errors 😤

Section 6.7 of [RFC3720] discusses digest error handling. It states that "No further action is necessary for initiators if the discarded PDU is an unsolicited PDU (e.g., Async, Reject)" when a payload digest error is detected. This is incorrect ❌.

An Asynchronous Message PDU or a Reject PDU carries the next StatSN value on an iSCSI connection, which advances the StatSN ⏩. When an initiator discards one of these PDUs due to a payload digest error, the entire PDU, including the header, MUST be discarded. Consequently, the initiator MUST treat the exception like the loss of any other solicited response PDU. That means it MUST use one of the following options noted in [RFC3720]:

  • a) Request PDU retransmission with a status SNACK.
  • b) Logout the connection for recovery and continue the tasks on a different connection instance.
  • c) Logout to close the connection (abort all the commands associated with the connection).

7.4. Message Error Checking 🔎

There's been some uncertainty about how far incoming messages have to be checked for protocol errors, beyond what's strictly required for processing the inbound message 🤨. This section addresses that question.

Unless [RFC3720] or this document requires it, an iSCSI implementation is not required to do an exhaustive protocol conformance check on an incoming iSCSI PDU. The iSCSI implementation especially is not required to double-check the remote iSCSI implementation's conformance to protocol requirements 🙌.

8. iSCSI PDUs 🌟

8.1. Asynchronous Message 📨

This section defines additional semantics for the Asynchronous Message PDU defined in Section 10.9 of [RFC3720], using the same conventions 💜.

A new legal value for the AsyncEvent is defined:

  • 5: all active tasks for LU with a matching LUN field in the Async Message PDU are being terminated.

The receiving initiator iSCSI layer MUST respond to this Message by taking the following steps, in order:

  • i) Stop Data-Out transfers on that connection for all active TTTs for the affected LUN quoted in the Async Message PDU.
  • ii) Acknowledge the StatSN of the Async Message PDU via a NOP-Out PDU with ITT=0xffffffff (i.e., non-ping flavor), while copying the LUN field from the Async Message to the NOP-Out 🎯.

The new AsyncEvent defined in this section, however, MUST NOT be used on an iSCSI session unless the new TaskReporting text key defined in Section 9.1 was negotiated to FastAbort on that session 🚫.

8.2. Reject

Okay bestie, Section 10.17.1 of [RFC3720] gives reason code 0x0b the explanation "Negotiation Reset" 🙄 but ngl, we literally cannot find a legit iSCSI use case for it. So reason code 0x0b MUST be considered as deprecated and MUST NOT be sent by implementations that comply with the requirements of this document. 🚫 If an implementation receives reason code 0x0b, it MUST treat it as a negotiation failure that terminates the Login Phase and the TCP connection, as specified in Section 6.10 of [RFC3720]. Rip. 💀

Section 5.4 of [RFC3720] says:

Neither the initiator nor the target should attempt to declare or negotiate a parameter more than once during any negotiation sequence without an intervening operational parameter negotiation reset, except for responses to specific keys that explicitly allow repeated key declarations (e.g., TargetAddress).

Here's the tea 🍵: if reason code 0x0b is deprecated, then the "operational parameter negotiation reset" that the quote talks about can never actually happen. That makes the phrase "without an intervening operational parameter negotiation reset" in [RFC3720] point at something impossible 😭. So the quoted phrase SHOULD be ignored by receivers that handle reason code 0x0b in the manner specified in this section. Iconic plot twist ✨.

9. Login/Text Operational Text Keys

This section follows the same conventions as Section 12 of [RFC3720]. Same vibes, same rules 💅.

9.1. TaskReporting

Use: LO
Senders: Initiator and Target
Scope: SW

Irrelevant when: SessionType=Discovery
TaskReporting=<list-of-values>
Default is RFC3720.
Result function is AND.

This key is used to negotiate the task completion reporting semantics from the SCSI target 🎯. The table below describes the semantics that an iSCSI target MUST support for each negotiated key value. Whenever this key is negotiated, at least the RFC3720 and ResponseFence values MUST be offered as options by the negotiation originator. Non-negotiable, bestie 🫶.

   +--------------+------------------------------------------+
   | Name         |             Description                  |
   +--------------+------------------------------------------+
   | RFC3720      | RFC 3720-compliant semantics.  Response  |
   |              | fencing is not guaranteed and fast       |
   |              | completion of multi-task aborting is not |
   |              | supported                                |
   +--------------+------------------------------------------+
   | ResponseFence| Response Fence (Section 3.3.1) semantics |
   |              | MUST be supported in reporting task      |
   |              | completions                              |
   +--------------+------------------------------------------+
   | FastAbort    | Updated fast multi-task abort semantics  |
   |              | defined in Section 4.1.3 MUST be         |
   |              | supported.  Support for Response Fence is|
   |              | implied -- i.e., Section 3.3.1 semantics |
   |              | MUST be supported as well                |
   +--------------+------------------------------------------+

The FastAbort row is giving main character energy 👑. Its updated fast multi-task abort semantics MUST be supported, and Response Fence comes along for the ride (Section 3.3.1 semantics MUST be supported as well) 🦋. When TaskReporting is not negotiated to FastAbort, the [RFC3720] TMF semantics as clarified in Section 4.1.2 MUST be used. Clean and simple 🌸.

10. Security Considerations

Okay, security time 🔐, and I'm keeping it serious. This document does not introduce any new security considerations other than those already noted in [RFC3720]. So no new drama, just the old drama. Consequently, all the iSCSI-related security text in [RFC3723] is also directly applicable to this document. Read [RFC3720] and [RFC3723] before you ship anything, periodt ⚠️.

11. IANA Considerations

This document creates the following iSCSI-related registries for IANA to manage 📚.

  1. iSCSI Opcodes
  2. iSCSI Login/Text Keys
  3. iSCSI Asynchronous Events
  4. iSCSI Task Management Function Codes
  5. iSCSI Login Response Status Codes
  6. iSCSI Reject Reason Codes
  7. iSER Opcodes

Each of the following sections describes a registry, its sub-registries if any, and their administration policies in more detail. IANA has registered what this document calls the main "registries" as sub-registries of a larger iSCSI registry. However, wherever registry-to-sub-registry relationships are specified by this document, they have been preserved intact 🧚.

[RFC3720] specifies three iSCSI-related registries: extended key, authentication methods, and digests. This document recommends that these registries be published together with the registries defined by this document. Further, this document recommends that the three [RFC3720] registries be listed in between items 6 and 7 in the registry list above. Organisation is a love language 💜.

11.2. iSCSI Opcodes

Name of the registry: "iSCSI Opcodes"

Namespace details: Numerical values that can fit in one octet with the most significant two bits (bits 0 and 1) already designated by [RFC3720], bit 0 being reserved and bit 1 for immediate delivery. Bit 2 is designated to identify the originator of the opcode. Bit 2 = 0 for initiator and Bit 2 = 1 for target.

Information that must be provided to assign a new value: An IESG-approved standards-track specification defining the semantics and interoperability requirements of the proposed new value and the fields to be recorded in the registry.

Allocation request guidance to requesters:

  1. If the initiator opcode and target opcode used to identify the request and response of a new type of protocol operation are requested, assign the same lower five bits (i.e., Bit 3 through Bit 7) for both opcodes, e.g., 0x13 and 0x33.
  2. If only the initiator opcode or target opcode is requested to identify a one-way protocol message (i.e., request without a response or a "response" without a request), assign an unused number from the appropriate category (i.e., Bit 2 set to 0 or 1 for initiator category or target category) and add the other pair member (i.e., same opcode with Bit 2 set to 1 or 0, respectively) to "no opcode pair is available" list.
  3. If there are no other opcodes available in the appropriate "Reserved to IANA" list to assign on a request for a new opcode except the reserved opcodes in the "no opcode pair is available" list, allocate the opcode from the appropriate category (initiator or target) of the "no opcode pair is available" list.

Fields to record in the registry: Assigned value, Who can originate (Initiator or Target), Operation Name, and its associated RFC reference.

Initial registry contents:

0x00, Initiator, NOP-Out, [RFC3720]

0x01, Initiator, SCSI Command, [RFC3720]

0x02, Initiator, SCSI Task Management function request, [RFC3720]

0x03, Initiator, Login Request, [RFC3720]

0x04, Initiator, Text Request, [RFC3720]

0x05, Initiator, SCSI Data-Out, [RFC3720]

0x06, Initiator, Logout Request, [RFC3720]

0x10, Initiator, SNACK Request, [RFC3720]
0x1c-0x1e, Initiator, Vendor specific codes, [RFC3720]

0x20, Target, NOP-In, [RFC3720]

0x21, Target, SCSI Response, [RFC3720]

0x22, Target, SCSI Task Management function response, [RFC3720]

0x23, Target, Login Response, [RFC3720]

0x24, Target, Text Response, [RFC3720]

0x25, Target, SCSI Data-In, [RFC3720]

0x26, Target, Logout Response, [RFC3720]

0x31, Target, Ready To Transfer (R2T), [RFC3720]

0x32, Target, Asynchronous Message, [RFC3720]

0x3c-0x3e, Target, Vendor specific codes, [RFC3720]

0x3f, Target, Reject, [RFC3720]

Reserved to IANA:
    0x07-0x0f, 0x13-0x1b (initiator codes)
    0x27-0x2f, 0x33-0x3b (target codes)

No opcode pair is available:
    0x11, 0x12, 0x1f (initiator codes)
    0x30 (target codes)

Allocation Policy:

Standards Action ([IANA]): This is required for defining the semantics of the opcode.

Expert Review ([IANA]): This is required for selecting the specific opcode(s) to allocate in order to ensure compliance with the earlier "Allocation request guidance to requesters".

11.3. iSCSI Login/Text Keys

Name of the registry: "iSCSI Text Keys"

Namespace details: Key=value pairs with "Key" names in UTF-8 Unicode, and the permissible "value" options for the "Key" are Key-dependent. [RFC3720] defines the rules on key names and allowed values.

Information that must be provided to assign a new value: An IESG-approved standards-track specification defining the semantics and interoperability requirements of the proposed new Key per [RFC3720] key specification template and the fields to be recorded in the registry.

Assignment policy:

If the requested Key name is not already assigned and is roughly representative of its proposed semantics, it may be assigned to the requester. Cute and reasonable 🎀.

Given the arbitrary nature of text strings, there is no maximum reserved by IANA for assignment in this registry. Unlimited vibes ♾️.

Fields to record in the registry: Assigned Key Name and its associated RFC reference.

Initial registry contents:

AuthMethod, [RFC3720]

HeaderDigest, [RFC3720]

DataDigest, [RFC3720]

MaxConnections, [RFC3720]

SendTargets, [RFC3720]

TargetName, [RFC3720]

InitiatorName, [RFC3720]

TargetAlias, [RFC3720]

InitiatorAlias, [RFC3720]

TargetAddress, [RFC3720]

TargetPortalGroupTag, [RFC3720]

InitialR2T, [RFC3720]

ImmediateData, [RFC3720]

MaxRecvDataSegmentLength, [RFC3720]
MaxBurstLength, [RFC3720]

FirstBurstLength, [RFC3720]

DefaultTime2Wait, [RFC3720]

DefaultTime2Retain, [RFC3720]

MaxOutstandingR2T, [RFC3720]

DataPDUInOrder, [RFC3720]

DataSequenceInOrder, [RFC3720]

ErrorRecoveryLevel, [RFC3720]

SessionType, [RFC3720]

RDMAExtensions, [iSER]

TargetRecvDataSegmentLength, [iSER]

InitiatorRecvDataSegmentLength, [iSER]

MaxOutstandingUnexpectedPDUs, [iSER]

TaskReporting, this document

Allocation Policy:

Standards Action ([IANA])

11.4. iSCSI Asynchronous Events

Name of the registry: "iSCSI Asynchronous Events"

Namespace details: Numerical values that can fit in one octet.

Information that must be provided to assign a new value: An IESG-approved standards-track specification defining the semantics and interoperability requirements of the proposed new Event and the fields to be recorded in the registry.

Assignment policy:

If the requested value is not already assigned, it may be assigned to the requester.

6-247: range reserved by IANA for assignment in this registry.

Fields to record in the registry: Assigned Event number, Description and its associated RFC reference.

Initial registry contents:

0, SCSI Async Event, [RFC3720]

1, Logout Request, [RFC3720]

2, Connection drop notification, [RFC3720]

3, Session drop notification, [RFC3720]

4, Negotiation Request, [RFC3720]

5, Task termination, this document

248-254, Vendor-unique, this document

255, Vendor-unique, [RFC3720]

Allocation Policy:

Standards Action ([IANA])

11.5. iSCSI Task Management Function Codes ✨

okay bestie, this is the registry for the TMF codes 🎀 (task management functions, the commands that tell a target to abort tasks or reset things 💅). here's the whole deal:

  • Name of the registry: "iSCSI TMF Codes"
  • Namespace details: numerical values that can fit in 7 bits 🔢

To get a new value in here, you need an IESG-approved standards-track specification that defines the semantics and interoperability requirements of the proposed new Code, plus the fields to be recorded in the registry. no shortcuts, periodt 💖

For assignment, if the requested value isn't already taken, it may be assigned to the requester 🦋

  • 9-127: range reserved by IANA for assignment in this registry

Each entry records the Assigned Code, a Description, and its associated RFC reference 📝

The starting lineup of codes is:

1, ABORT TASK, [RFC3720]

2, ABORT TASK SET, [RFC3720]

3, CLEAR ACA, [RFC3720]

4, CLEAR TASK SET, [RFC3720]

5, LOGICAL UNIT RESET, [RFC3720]

6, TARGET WARM RESET, [RFC3720]

7, TARGET COLD RESET, [RFC3720]

8, TASK REASSIGN, [RFC3720]

Allocation Policy: Standards Action ([IANA]) 👑

11.6. iSCSI Login Response Status Codes 🌸

this one is the login response status codes, and it's giving layers 🧁 there's a Status-Class (one octet) and a Status-Detail (one octet) for each possible Status-Class value, except for the Vendor-Unique Status-Class (0x0f) 🙈

  • Name of the registry: "iSCSI Login Response Status Codes"
  • Namespace details: numerical values; Status-Class (one octet), Status-Detail (one octet) for each possible value of Status-Class except for Vendor-Unique Status-Class (0x0f)

To add a value, you need an IESG-approved specification that defines the semantics and interoperability requirements of the proposed new Code, its Status-Class affiliation (only if a new Status-Detail value is being proposed for a Status-Class), the Status-Class definition (only if a new Status-Class is being proposed), and the fields to be recorded in the registry 📚

Assignment policy: if the requested value is not already assigned, it may be assigned to the requester 🫶

  • 4-14 and 16-255: range reserved by IANA for assignment in this registry

Fields for the Status-Class main registry: Assigned Code, Description, and its associated RFC reference.

Fields for the Status-Detail sub-registries: Status-Class, Assigned Code, Description, and its associated RFC reference.

Here's the Status-Class main registry, initial contents:

0x00, Success, [RFC3720]

0x01, Redirection, [RFC3720]

0x02, Initiator Error, [RFC3720]

0x03, Target Error, [RFC3720]

0x0f, Vendor-Unique, this document

Now the sub-registries, one class at a time 🍓

Status-Detail for Status-Class=0x00:

0x00, 0x00, Success, [RFC3720]
  • 1-255: range reserved by IANA for assignment in Status-Class=0 sub-registry

Status-Detail for Status-Class=0x01:

0x01, 0x01, Temporary move, [RFC3720]

0x01, 0x02, Permanent move, [RFC3720]
  • 3-255: range reserved by IANA for assignment in Status-Class=1 sub-registry

Status-Detail for Status-Class=0x02 (this is the big one 👀):

0x02, 0x00, Miscellaneous, [RFC3720]

0x02, 0x01, Authentication failure, [RFC3720]

0x02, 0x02, Authorization failure, [RFC3720]

0x02, 0x03, Not found, [RFC3720]

0x02, 0x04, Target removed, [RFC3720]

0x02, 0x05, Unsupported version, [RFC3720]

0x02, 0x06, Too many connections, [RFC3720]

0x02, 0x07, Missing parameter, [RFC3720]
0x02, 0x08, Can't include in session, [RFC3720]

0x02, 0x09, Unsupported session type, [RFC3720]

0x02, 0x0a, Non-existent session, [RFC3720]

0x02, 0x0b, Invalid during login, [RFC3720]
  • 12-255: range reserved by IANA for assignment in Status-Class=2 sub-registry

Status-Detail for Status-Class=0x03:

0x03, 0x00, Target error, [RFC3720]

0x03, 0x01, Service unavailable, [RFC3720]

0x03, 0x02, Out of resources, [RFC3720]
  • 3-255: range reserved by IANA for assignment in Status-Class=3 sub-registry

Allocation Policy: Standards Action ([IANA]) 💅

11.7. iSCSI Reject Reason Codes 🙈

these are the reject reason codes, and they're all one octet, so they fit in a single byte 🧚

  • Name of the registry: "iSCSI Reject Reason Codes"
  • Namespace details: numerical values that can fit in a single octet

To add a new value, you need a published specification that defines the semantics and interoperability requirements of the proposed new Code and the fields to be recorded in the registry 📖

Assignment policy: if the requested value is not already assigned, it may be assigned to the requester 🦋

  • 13-255: range reserved by IANA for assignment in this registry

Fields to record: Assigned Code, Description, and its associated RFC reference.

Initial registry contents:

0x01, Reserved, [RFC3720]

0x02, Data digest error, [RFC3720]

0x03, SNACK Reject, [RFC3720]

0x04, Protocol Error, [RFC3720]

0x05, Command not supported, [RFC3720]

0x06, Immediate command reject, [RFC3720]

0x07, Task in progress, [RFC3720]

0x08, Invalid data ack, [RFC3720]

0x09, Invalid PDU field, [RFC3720]

0x0a, Long op reject, [RFC3720]

0x0b, "Deprecated reason code", this document

0x0c, Waiting for Logout, [RFC3720]

Allocation Policy: Standards Action ([IANA]) 👑

11.8. iSER Opcodes 🌈

last registry in this section, and it's for the iSER opcodes, which are the ones for iSCSI over RDMA 🚀

  • Name of the registry: "iSER Opcodes"
  • Namespace details: numerical values that can fit in 4 bits

To add a value, you need an IESG-approved specification that defines the semantics and interoperability requirements of the proposed new value and the fields to be recorded in the registry ✨

Assignment policy: if the requested value is not already assigned, it may be assigned to the requester 🫶

  • 4-15: range reserved by IANA for assignment in this registry

Fields to record: Assigned value, Operation Name, and its associated RFC reference.

Initial registry contents:

0x1, iSCSI control-type, [iSER]

0x2, iSER Hello, [iSER]

0x3, iSER HelloReply, [iSER]

Allocation Policy: Standards Action ([IANA]) 💖

12. References 📚

12.1. Normative References

These are the must-reads, the foundations everything else is built on 🧱

[RFC3720] Satran, J., Meth, K., Sapuntzakis, C., Chadalapaka, M., and
          E. Zeidner, "Internet Small Computer Systems Interface
          (iSCSI)", RFC 3720, April 2004.

[SPC3]    ANSI INCITS 408-2005, SCSI Primary Commands-3.

[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
          Requirement Levels", BCP 14, RFC 2119, March 1997.

[IANA]    Narten, T. and H. Alvestrand, "Guidelines for Writing an
          IANA Considerations Section in RFCs", BCP 26, RFC 2434,
          October 1998.

[iSER]    Ko, M., Chadalapaka, M., Hufferd, J., Elzur, U., Shah, H.,
          and P. Thaler, "Internet Small Computer System Interface
          (iSCSI) Extensions for Remote Direct Memory Access (RDMA)",
          RFC 5046, October 2007.

12.2. Informative References

These are the extra-credit reads, super helpful for context 🤓

[RFC3721] Bakke, M., Hafner, J., Hufferd, J., Voruganti, K., and M.
          Krueger, "Internet Small Computer Systems Interface (iSCSI)
          Naming and Discovery", RFC 3721, April 2004.

[RFC3723] Aboba, B., Tseng, J., Walker, J., Rangan, V., and F.
          Travostino, "Securing Block Storage Protocols over IP", RFC
          3723, April 2004.

[RFC3722] Bakke, M., "String Profile for Internet Small Computer
          Systems Interface (iSCSI) Names", RFC 3722, April 2004.

[SAM2]    ANSI INCITS 366-2003, SCSI Architecture Model-2 (SAM-2).

[SAM3]    ANSI INCITS 402-2005, SCSI Architecture Model-3 (SAM-a3).

[SAM4]    T10 Project: 1683-D, SCSI Architecture Model-4 (SAM-4),
          Work in Progress.

Acknowledgements 🫶

the IP Storage (IPS) Working Group in the Transport Area of the IETF is the crew that defined the iSCSI protocol (plus a whole bunch of other IP storage protocols too) 🌸 the editor says thank you to the entire working group, and honestly, same 💜

The following individuals directly contributed to identifying [RFC3720] issues and/or suggesting resolutions to the issues clarified in this document: David Black, Gwendal Grignou, Mike Ko, Dmitry Fomichev, Bill Studenmund, Ken Sandars, Bob Russell, Julian Satran, Rob Elliott, Joseph Pittman, Somesh Gupta, Eddy Quicksall, Paul Koning. This document benefited from all of these contributions ✨

Editor's Address 📬

Mallikarjun Chadalapaka
Hewlett-Packard Company
8000 Foothills Blvd.
Roseville, CA 95747-5668, USA
Phone: +1-916-785-5621
EMail: cbm@rose.hp.com

Copyright (C) The IETF Trust (2007).

This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Intellectual Property 🔐

the IETF takes no position on whether any Intellectual Property Rights (or other rights) that someone might claim over the tech in this document are valid or in scope, or whether any license for them is available. it also doesn't say it has independently checked for any such rights. info on how rights work for RFC documents is in BCP 78 and BCP 79 📖

The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org.

that's the whole thing, bestie 🦋💖 end of the spec, and honestly we did it 👑✨

📦 where this page comes from