RFC 3010
NFS version 4 Protocol
RFC 3010: NFS version 4 Protocol 🎀
okay bestie, let's do the vibe check first 💅 this is RFC 3010, a Standards Track doc, published December 2000 🌸
The authors are S. Shepler, B. Callaghan, D. Robinson, and R. Thurlow from Sun Microsystems Inc., plus C. Beame from Hummingbird Ltd., M. Eisler from Zambeel, Inc., and D. Noveck from Network Appliance, Inc. ✨ It obsoletes RFC 1813 and RFC 1094 🫶
Status of this Memo 🦋
This document specifies an Internet standards track protocol for the Internet community, and requests discussion and suggestions for improvements. Please refer to the current edition of the "Internet Official Protocol Standards" (STD 1) for the standardization state and status of this protocol. Distribution of this memo is unlimited.
Copyright Notice 💜
Copyright (C) The Internet Society (2000). All Rights Reserved.
Abstract 🌈
okay so NFS (Network File System) version 4 is a distributed file system protocol, and it owes its heritage to NFS protocol versions 2 [RFC1094] and 3 [RFC1813] 🧚 unlike the earlier versions, NFS version 4 keeps traditional file access but also brings in support for file locking and the mount protocol 🔥 it also adds support for strong security (and its negotiation), compound operations, client caching, and internationalization ✨ and of course, a lot of care went into making NFS version 4 work well in an Internet environment 💖
Key Words 🍓
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 💅
Table of Contents 🌸
Heads up, bestie: the source for this part only had the Table of Contents, so there's no body prose here to rewrite yet. Paste the actual body text for these sections and I'll do the full glow-up 💅
1. Introduction 🧚
1.1. Overview of NFS Version 4 Features ✨
1.1.1. RPC and Security
1.1.2. Procedure and Operation Structure
1.1.3. File System Model
1.1.3.1. Filehandle Types
1.1.3.2. Attribute Types
1.1.3.3. File System Replication and Migration
1.1.4. OPEN and CLOSE
1.1.5. File locking
1.1.6. Client Caching and Delegation
1.2. General Definitions
2. Protocol Data Types 🎀
2.1. Basic Data Types
2.2. Structured Data Types
3. RPC and Security Flavor 🔐
3.1. Ports and Transports
3.2. Security Flavors
3.2.1. Security mechanisms for NFS version 4
3.2.1.1. Kerberos V5 as security triple
3.2.1.2. LIPKEY as a security triple
3.2.1.3. SPKM-3 as a security triple
3.3. Security Negotiation
3.3.1. Security Error
3.3.2. SECINFO
3.4. Callback RPC Authentication
4. Filehandles 🦋
4.1. Obtaining the First Filehandle
4.1.1. Root Filehandle
4.1.2. Public Filehandle
4.2. Filehandle Types
4.2.1. General Properties of a Filehandle
4.2.2. Persistent Filehandle
4.2.3. Volatile Filehandle
4.2.4. One Method of Constructing a Volatile Filehandle
4.3. Client Recovery from Filehandle Expiration
5. File Attributes 📂
5.1. Mandatory Attributes
5.2. Recommended Attributes
5.3. Named Attributes
5.4. Mandatory Attributes - Definitions
5.5. Recommended Attributes - Definitions
5.6. Interpreting owner and owner_group
5.7. Character Case Attributes
5.8. Quota Attributes
5.9. Access Control Lists
5.9.1. ACE type
5.9.2. ACE flag
5.9.3. ACE Access Mask
5.9.4. ACE who
6. File System Migration and Replication 🌸
6.1. Replication
6.2. Migration
6.3. Interpretation of the fs_locations Attribute
6.4. Filehandle Recovery for Migration or Replication
7. NFS Server Name Space 🗺️
7.1. Server Exports
7.2. Browsing Exports
7.3. Server Pseudo File System
7.4. Multiple Roots
7.5. Filehandle Volatility
7.6. Exported Root
7.7. Mount Point Crossing
7.8. Security Policy and Name Space Presentation
8. File Locking and Share Reservations 🔒
8.1. Locking
8.1.1. Client ID
8.1.2. Server Release of Clientid
8.1.3. nfs_lockowner and stateid Definition
8.1.4. Use of the stateid
8.1.5. Sequencing of Lock Requests
8.1.6. Recovery from Replayed Requests
8.1.7. Releasing nfs_lockowner State
8.2. Lock Ranges
8.3. Blocking Locks
8.4. Lease Renewal
8.5. Crash Recovery
8.5.1. Client Failure and Recovery
8.5.2. Server Failure and Recovery
8.5.3. Network Partitions and Recovery
8.6. Recovery from a Lock Request Timeout or Abort
8.7. Server Revocation of Locks
8.8. Share Reservations
8.9. OPEN/CLOSE Operations
8.10. Open Upgrade and Downgrade
8.11. Short and Long Leases
8.12. Clocks and Calculating Lease Expiration
8.13. Migration, Replication and State
8.13.1. Migration and State
8.13.2. Replication and State
8.13.3. Notification of Migrated Lease
9. Client-Side Caching 💜
9.1. Performance Challenges for Client-Side Caching
9.2. Delegation and Callbacks
9.2.1. Delegation Recovery
9.3. Data Caching
9.3.1. Data Caching and OPENs
9.3.2. Data Caching and File Locking
9.3.3. Data Caching and Mandatory File Locking
9.3.4. Data Caching and File Identity
9.4. Open Delegation
9.4.1. Open Delegation and Data Caching
9.4.2. Open Delegation and File Locks
9.4.3. Recall of Open Delegation
9.4.4. Delegation Revocation
9.5. Data Caching and Revocation
9.5.1. Revocation Recovery for Write Open Delegation
9.6. Attribute Caching
9.7. Name Caching
9.8. Directory Caching
10. Minor Versioning 🌈
11. Internationalization 🌍
11.1. Universal Versus Local Character Sets
11.2. Overview of Universal Character Set Standards
11.3. Difficulties with UCS-4, UCS-2, Unicode
11.4. UTF-8 and its solutions
11.5. Normalization
12. Error Definitions 🚨
13. NFS Version 4 Requests 📨
13.1. Compound Procedure
13.2. Evaluation of a Compound Request
13.3. Synchronous Modifying Operations
13.4. Operation Values
14. NFS Version 4 Procedures ⭐
14.1. Procedure 0: NULL - No Operation
14.2. Procedure 1: COMPOUND - Compound Operations
14.2.1. Operation 3: ACCESS - Check Access Rights
14.2.2. Operation 4: CLOSE - Close File
14.2.3. Operation 5: COMMIT - Commit Cached Data
14.2.4. Operation 6: CREATE - Create a Non-Regular File Object
14.2.5. Operation 7: DELEGPURGE - Purge Delegations Awaiting Recovery
14.2.6. Operation 8: DELEGRETURN - Return Delegation
14.2.7. Operation 9: GETATTR - Get Attributes
14.2.8. Operation 10: GETFH - Get Current Filehandle
14.2.9. Operation 11: LINK - Create Link to a File
14.2.10. Operation 12: LOCK - Create Lock
14.2.11. Operation 13: LOCKT - Test For Lock
14.2.12. Operation 14: LOCKU - Unlock File
14.2.13. Operation 15: LOOKUP - Lookup Filename
14.2.14. Operation 16: LOOKUPP - Lookup Parent Directory
14.2.15. Operation 17: NVERIFY - Verify Difference in Attributes
14.2.16. Operation 18: OPEN - Open a Regular File
14.2.17. Operation 19: OPENATTR - Open Named Attribute Directory
14.2.18. Operation 20: OPEN_CONFIRM - Confirm Open
14.2.19. Operation 21: OPEN_DOWNGRADE - Reduce Open File Access
14.2.20. Operation 22: PUTFH - Set Current Filehandle
14.2.21. Operation 23: PUTPUBFH - Set Public Filehandle
14.2.22. Operation 24: PUTROOTFH - Set Root Filehandle
14.2.23. Operation 25: READ - Read from File
14.2.24. Operation 26: READDIR - Read Directory
14.2.25. Operation 27: READLINK - Read Symbolic Link
14.2.26. Operation 28: REMOVE - Remove Filesystem Object
14.2.27. Operation 29: RENAME - Rename Directory Entry
14.2.28. Operation 30: RENEW - Renew a Lease
14.2.29. Operation 31: RESTOREFH - Restore Saved Filehandle
14.2.30. Operation 32: SAVEFH - Save Current Filehandle
14.2.31. Operation 33: SECINFO - Obtain Available Security
14.2.32. Operation 34: SETATTR - Set Attributes
14.2.33. Operation 35: SETCLIENTID - Negotiate Clientid
14.2.34. Operation 36: SETCLIENTID_CONFIRM - Confirm Clientid
14.2.35. Operation 37: VERIFY - Verify Same Attributes
14.2.36. Operation 38: WRITE - Write to File
15. NFS Version 4 Callback Procedures 🔁
15.1. Procedure 0: CB_NULL - No Operation
15.2. Procedure 1: CB_COMPOUND - Compound Operations . . . . . 171
15.2.1. Operation 3: CB_GETATTR - Get Attributes . . . . . . . 172
15.2.2. Operation 4: CB_RECALL - Recall an Open Delegation . . 173
16. Security Considerations . . . . . . . . . . . . . . . . . . 174
17. IANA Considerations . . . . . . . . . . . . . . . . . . . . 174
17.1. Named Attribute Definition . . . . . . . . . . . . . . . 174
18. RPC definition file . . . . . . . . . . . . . . . . . . . . 175
19. Bibliography . . . . . . . . . . . . . . . . . . . . . . . 206
20. Authors . . . . . . . . . . . . . . . . . . . . . . . . . . 210
20.1. Editor's Address . . . . . . . . . . . . . . . . . . . . 210
20.2. Authors' Addresses . . . . . . . . . . . . . . . . . . . 210
20.3. Acknowledgements . . . . . . . . . . . . . . . . . . . . 211
21. Full Copyright Statement . . . . . . . . . . . . . . . . . 2121. Introduction ✨
The NFS version 4 protocol is a further revision of the NFS protocol, following on from versions 2 [RFC1094] and 3 [RFC1813]. 💖 It keeps the big essentials from earlier versions: design for easy recovery, independence from transport protocols, operating systems and filesystems, simplicity, and good performance. 🎀 The NFS version 4 revision has these goals:
- Improved access and good performance on the Internet. 🌐
The protocol is designed to transit firewalls easily, perform well where latency is high and bandwidth is low, and scale to very large numbers of clients per server. 🦋
- Strong security with negotiation built into the protocol. 🔐
The protocol builds on the work of the ONCRPC working group in supporting the RPCSEC_GSS protocol. Additionally, the NFS version 4 protocol provides a mechanism to allow clients and servers the ability to negotiate security and require clients and servers to support a minimal set of security schemes. 🫶
- Good cross-platform interoperability. 🌈
The protocol features a file system model that provides a useful, common set of features that does not unduly favor one file system or operating system over another. 💅
- Designed for protocol extensions. 🧚
The protocol is designed to accept standard extensions that do not compromise backward compatibility. ✨
1.1. Overview of NFS Version 4 Features
To give the reader some context, the major features of the NFS version 4 protocol will be reviewed briefly. 🌸 This is for both the reader who already knows the previous versions of NFS and the reader who is brand new to the NFS protocols. For the newcomers, there is still some fundamental knowledge that is expected: the reader should be familiar with the XDR and RPC protocols as described in [RFC1831] and [RFC1832]. ngl, a basic understanding of file systems and distributed file systems is expected as well. 💀
1.1.1. RPC and Security
As with previous versions of NFS, the External Data Representation (XDR) and Remote Procedure Call (RPC) mechanisms used for the NFS version 4 protocol are those defined in [RFC1831] and [RFC1832]. 🎀 To meet end to end security requirements, the RPCSEC_GSS framework [RFC2203] will be used to extend the basic RPC security. With RPCSEC_GSS, various mechanisms can be provided to offer authentication, integrity, and privacy to the NFS version 4 protocol. 🔥 Kerberos V5 will be used as described in [RFC1964] to provide one security framework. The LIPKEY GSS-API mechanism described in [RFC2847] will be used to provide for the use of a user password and server public key by the NFS version 4 protocol. With RPCSEC_GSS, other mechanisms may also be specified and used for NFS version 4 security. 👑
To enable in-band security negotiation, the NFS version 4 protocol has added a new operation. It gives the client a way to ask the server about its policies on which security mechanisms must be used for access to the server's file system resources. 🫶 With this, the client can securely match the security mechanism that meets the policies specified at both the client and server. 💖
1.1.2. Procedure and Operation Structure
A big departure from previous versions of the NFS protocol is the introduction of the COMPOUND procedure. 🦋 For the NFS version 4 protocol, there are two RPC procedures, NULL and COMPOUND. The COMPOUND procedure is defined in terms of operations, and these operations correspond more closely to the traditional NFS procedures. 🧚 With COMPOUND, the client can build simple or complex requests. These COMPOUND requests reduce the number of RPCs needed for logical file system operations. 🌈 For example, without any previous contact with a server, a client can read data from a file in one request by combining LOOKUP, OPEN, and READ operations in a single COMPOUND RPC. 😭 With previous versions of NFS, this type of single request was not possible.
The model used for COMPOUND is very simple. 🍓 There is no logical OR or ANDing of operations. The operations combined within a COMPOUND request are evaluated in order by the server. Once an operation returns a failing result, the evaluation ends and the results of all evaluated operations are returned to the client. 💅
The NFS version 4 protocol continues to have the client refer to a file or directory at the server by a "filehandle". 🎀 The COMPOUND procedure has a method of passing a filehandle from one operation to another within the sequence of operations. There is a concept of a "current filehandle" and a "saved filehandle". Most operations use the "current filehandle" as the file system object to operate upon. 🔥 The "saved filehandle" is used as temporary filehandle storage within a COMPOUND procedure, as well as an additional operand for certain operations. ✨
1.1.3. File System Model
The general file system model used for the NFS version 4 protocol is the same as in previous versions. 🌸 The server file system is hierarchical, with the regular files contained within being treated as opaque byte streams. In a slight departure, file and directory names are encoded with UTF-8 to deal with the basics of internationalization. 🌍
The NFS version 4 protocol does not require a separate protocol to provide the initial mapping between path name and filehandle. 🙈 Instead of using the older MOUNT protocol for this mapping, the server provides a ROOT filehandle that represents the logical root, or top, of the file system tree provided by the server. The server provides multiple file systems by gluing them together with pseudo file systems. These pseudo file systems provide for potential gaps in the path names between real file systems. 🧩
1.1.3.1. Filehandle Types
In previous versions of NFS, the filehandle provided by the server was guaranteed to be valid, or persistent, for the lifetime of the file system object it referred to. 🥺 For some server implementations, this persistence requirement has been difficult to meet. For the NFS version 4 protocol, this requirement has been relaxed by introducing another type of filehandle: volatile. 🦄 With persistent and volatile filehandle types, the server implementation can match the abilities of the file system at the server along with the operating environment. The client will know which type of filehandle the server is providing and can be prepared to deal with the semantics of each. 👑
1.1.3.2. Attribute Types
The NFS version 4 protocol introduces three classes of file system or file attributes. 💖 Like the additional filehandle type, this classification eases server implementations and extends the overall functionality of the NFS protocol. This attribute model is structured to be extensible, so new attributes can be introduced in minor revisions of the protocol without requiring significant rework. ✨
The three classifications are: mandatory, recommended, and named attributes. This is a significant departure from the previous attribute model used in NFS. 🫶 Previously, the attributes for file system and file objects were a fixed set of mainly Unix attributes. If the server or client did not support a particular attribute, it had to simulate the attribute the best it could. 😅
Mandatory attributes are the minimal set of file or file system attributes that must be provided by the server and must be properly represented by the server. 🔐 Recommended attributes represent different file system types and operating environments. They allow for better interoperability and the inclusion of more operating environments. 🌈 The mandatory and recommended attribute sets are traditional file or file system attributes. The third type is the named attribute. A named attribute is an opaque byte stream associated with a directory or file and referred to by a string name. 🍓 Named attributes are meant to be used by client applications as a way to associate application specific data with a regular file or directory. 🧚
One significant addition to the recommended set of file attributes is the Access Control List (ACL) attribute. 👑 This attribute provides directory and file access control beyond the model used in previous versions of NFS. The ACL definition allows for specification of user and group level access control. 🔥
1.1.3.3. File System Replication and Migration
With the use of a special file attribute, the ability to migrate or replicate server file systems is enabled within the protocol. 🦋 The file system locations attribute gives the client a way to probe the server about the location of a file system. If a file system is migrated, the client will receive an error when operating on it, and it can then query for the new file system location. 🌸 Replication works similarly: the client is able to query the server for the multiple available locations of a particular file system. 💅 From this information, the client can use its own policies to access the appropriate file system location. 🎀
1.1.4. OPEN and CLOSE
The NFS version 4 protocol introduces OPEN and CLOSE operations. 🌟 The OPEN operation provides a single point where file lookup, creation, and share semantics can be combined. The CLOSE operation provides for the release of state accumulated by OPEN. 🔓
1.1.5. File locking
With the NFS version 4 protocol, support for byte range file locking is part of the NFS protocol itself. 🔒 The file locking support is structured so that an RPC callback mechanism is not required. This is a departure from the previous versions of the NFS file locking protocol, the Network Lock Manager (NLM). 🙌 The state associated with file locks is maintained at the server under a lease-based model. The server defines a single lease period for all state held by an NFS client. 🕰️ If the client does not renew its lease within the defined period, all state associated with the client's lease may be released by the server. 😬 The client may renew its lease with the use of the RENEW operation, or implicitly by using other operations (primarily READ). 💜
1.1.6. Client Caching and Delegation 🧁
okay so caching in NFS version 4 is basically the same vibe as previous versions 💅 Attributes and directory info get cached for a duration the client decides on. When that timeout runs out, the client asks the server if the file system object has been updated. Very organized, very mature 🫶
For file data, it's a little different. The client checks whether its cache is still valid when the file is opened. It sends a query to the server to find out if the file has changed, and then it decides whether to keep or release its data cache for that file. And when the file gets closed, any modified data gets written back to the server ✨
If an application wants to serialize access to file data, it should use file locking on the data ranges it cares about 🔒
Now the big new glow-up in caching is delegation 👑 The server can hand certain responsibilities over to the client. When the server grants a delegation for a file, the client is guaranteed certain semantics about how that file is shared with other clients. At OPEN, the server may give the client either a read delegation or a write delegation 🦋
- With a read delegation, the client is assured that no other client can write to the file for as long as the delegation lasts 📖
- With a write delegation, the client is assured that no other client has read or write access to the file at all 🔥
Delegations can be recalled, though 😭 If another client asks for access in a way that conflicts with the delegation that's been granted, the server can notify the original client and recall the delegation. This needs a callback path to exist between the server and the client. If there's no callback path, delegations can not be granted at all 🚫
The whole point of a delegation is that it lets the client handle operations like OPEN, CLOSE, LOCK, LOCKU, READ, and WRITE locally, without having to check in with the server every single time 🧚
1.2. General Definitions 📚
Here are some definitions to give you the context you need, bestie 🌸
Client : The "client" is the entity that accesses the NFS server's resources. The client might be an application that has the logic to reach the NFS server directly. It can also be the traditional operating system client remote file system service that serves a bunch of applications 💻
In the case of file locking, the client is the entity that keeps a set of locks on behalf of one or more applications. This client is responsible for crash or failure recovery for the locks it manages 🔐
Also note that multiple clients may share the same transport, and multiple clients may exist on the same network node 🫶
Clientid : A 64-bit quantity used as a unique, short-hand reference to a client-supplied Verifier and ID. The server is responsible for supplying the Clientid 🆔
Lease : An interval of time defined by the server during which the client is irrevocably granted a lock. When a lease period ends, the lock may be revoked if the lease has not been extended. The lock must be revoked if a conflicting lock has been granted after the lease interval 💀
All leases granted by a server have the same fixed interval. The fixed interval was picked to ease the cost a server would otherwise pay for keeping state about variable-length leases across server failures 🧠
Lock : The term "lock" refers to both record (byte-range) locks and file (share) locks, unless it's specifically stated otherwise 🔒
Server : The "Server" is the entity responsible for coordinating client access to a set of file systems 🏠
Stable Storage : NFS version 4 servers must be able to recover without data loss from multiple power failures (including cascading power failures, which is several power failures in quick succession), operating system failures, and hardware failure of components other than the storage medium itself (for example, disk or nonvolatile RAM). Lowkey a very intense requirement, but it's giving reliability 😤
Some examples of stable storage that are allowable for an NFS server include:
- Media commit of data, meaning the modified data has been successfully written to the disk media, for example, the disk platter 💿
- An immediate reply disk drive with battery-backed on-drive intermediate storage or an uninterruptible power system (UPS) 🔋
- Server commit of data with battery-backed intermediate storage and recovery software 🛠️
- Cache commit with an uninterruptible power system (UPS) and recovery software ⚡
Stateid : A 64-bit quantity returned by a server that uniquely defines the locking state the server granted to a specific lock owner for a specific file 🎀
Stateids composed of all bits 0 or all bits 1 have special meaning and are reserved values. Don't touch those, periodt 🙅♀️
Verifier : A 64-bit quantity generated by the client that the server can use to determine whether the client has restarted and lost all of its previous lock state 🔄
2. Protocol Data Types 🎀
The syntax and semantics used to describe the data types of the NFS version 4 protocol are defined in the XDR [RFC1832] and RPC [RFC1831] documents. The next sections build on the XDR data types to define types and structures that are specific to this protocol 🧩
2.1. Basic Data Types 💖
Here's the full table of basic data types, copied exactly so nothing gets lost in translation 📋
Data Type Definition
_____________________________________________________________________
int32_t typedef int int32_t;
uint32_t typedef unsigned int uint32_t;
int64_t typedef hyper int64_t;
uint64_t typedef unsigned hyper uint64_t;
attrlist4 typedef opaque attrlist4<>;
Used for file/directory attributes
bitmap4 typedef uint32_t bitmap4<>;
Used in attribute array encoding.
changeid4 typedef uint64_t changeid4;
Used in definition of change_info
clientid4 typedef uint64_t clientid4;
Shorthand reference to client identification
component4 typedef utf8string component4;
Represents path name components
count4 typedef uint32_t count4;
Various count parameters (READ, WRITE, COMMIT)
length4 typedef uint64_t length4;
Describes LOCK lengths
linktext4 typedef utf8string linktext4;
Symbolic link contents
mode4 typedef uint32_t mode4;
Mode attribute data type
nfs_cookie4 typedef uint64_t nfs_cookie4;
Opaque cookie value for READDIR
nfs_fh4 typedef opaque nfs_fh4<NFS4_FHSIZE>;
Filehandle definition; NFS4_FHSIZE is defined as 128
nfs_ftype4 enum nfs_ftype4;
Various defined file types
nfsstat4 enum nfsstat4;
Return value for operations
offset4 typedef uint64_t offset4;
Various offset designations (READ, WRITE, LOCK, COMMIT)
pathname4 typedef component4 pathname4<>;
Represents path name for LOOKUP, OPEN and others
qop4 typedef uint32_t qop4;
Quality of protection designation in SECINFO
sec_oid4 typedef opaque sec_oid4<>;
Security Object Identifier
The sec_oid4 data type is not really opaque.
Instead contains an ASN.1 OBJECT IDENTIFIER as used
by GSS-API in the mech_type argument to
GSS_Init_sec_context. See [RFC2078] for details.
seqid4 typedef uint32_t seqid4;
Sequence identifier used for file locking
stateid4 typedef uint64_t stateid4;
State identifier used for file locking and delegation
utf8string typedef opaque utf8string<>;
UTF-8 encoding for strings
verifier4 typedef opaque verifier4[NFS4_VERIFIER_SIZE];
Verifier used for various operations (COMMIT, CREATE,
OPEN, READDIR, SETCLIENTID, WRITE)
NFS4_VERIFIER_SIZE is defined as 82.2. Structured Data Types 💖
Okay bestie, this is where the protocol gets its structs, and they're so iconic. ✨
nfstime4 🕰️ is the time struct:
struct nfstime4 {
int64_t seconds;
uint32_t nseconds;
}It gives the number of seconds and nanoseconds since midnight, or 0 hour, January 1, 1970 Coordinated Universal Time (UTC). Positive seconds mean dates after 0 hour January 1, 1970, and negative seconds mean dates before it. Either way, you add nseconds to seconds to get the final time. So half a second before that moment is seconds -1 and nseconds 500000000. Anything above 999,999,999 for nseconds is invalid, so don't even 🙈
This type is how time and date info gets passed around. The server converts to and from its own local time format and keeps as much accuracy as it can. If a file system object's stored timestamps are less precise than this type, you can lose some precision, and that's just physics 😭 An adjunct time maintenance protocol is recommended to cut down on client and server time skew, so go get one.
time_how4 🎀 is an enum with two choices:
enum time_how4 {
SET_TO_SERVER_TIME4 = 0,
SET_TO_CLIENT_TIME4 = 1
};settime4 💅 is a union that switches on time_how4:
union settime4 switch (time_how4 set_it) {
case SET_TO_CLIENT_TIME4:
nfstime4 time;
default:
void;
};These definitions are what you use as attribute definitions when setting time values. If set_it is SET_TO_SERVER_TIME4, the server uses its own local time representation for the value. Main character energy 👑
specdata4 🧚 holds extra info for the device file types NF4CHR and NF4BLK:
struct specdata4 {
uint32_t specdata1;
uint32_t specdata2;
};fsid4 🌸 is the file system identifier, and it's a mandatory attribute:
struct fsid4 {
uint64_t major;
uint64_t minor;
};fs_location4 and fs_locations4 🦋 are the types behind the fs_locations recommended attribute, which handles migration and replication support:
struct fs_location4 {
utf8string server<>;
pathname4 rootpath;
}; struct fs_locations4 {
pathname4 fs_root;
fs_location4 locations<>;
};fattr4 🍓 is the struct used to represent file and directory attributes:
struct fattr4 {
bitmap4 attrmask;
attrlist4 attr_vals;
};The bitmap is a counted array of 32-bit integers that hold bit values. To find the integer that holds bit n, compute (n / 32). The bit's position inside that integer is (n mod 32). Here's the layout, bestie:
0 1
+-----------+-----------+-----------+--
| count | 31 .. 0 | 63 .. 32 |
+-----------+-----------+-----------+--change_info4 🔥 shows up with the CREATE, LINK, REMOVE, and RENAME operations. It tells the client the value of the change attribute for the directory where the target file system object lives:
struct change_info4 {
bool atomic;
changeid4 before;
changeid4 after;
};clientaddr4 🌈 is used as part of the SETCLIENT operation, either to give the address of a client that's using a clientid or as part of a call back registration:
struct clientaddr4 {
/* see struct rpcb in RFC 1833 */
string r_netid<>; /* network id */
string r_addr<>; /* universal address */
};cb_client4 ⭐ is how the client tells the server its call back address. It includes the program number and the client address:
struct cb_client4 {
unsigned int cb_program;
clientaddr4 cb_location;
};nfs_client_id4 🫶 is part of the arguments to the SETCLIENTID operation:
struct nfs_client_id4 {
verifier4 verifier;
opaque id<>;
};nfs_lockowner4 💜 identifies the owner of an OPEN share or a file lock:
struct nfs_lockowner4 {
clientid4 clientid;
opaque owner<>;
};3. RPC and Security Flavor ✨
The NFS version 4 protocol is a Remote Procedure Call (RPC) application. It uses RPC version 2 and the eXternal Data Representation (XDR) that go with it, as defined in [RFC1831] and [RFC1832]. The RPCSEC_GSS security flavor from [RFC2203] MUST be used as the mechanism for delivering stronger security for NFS version 4. Security is not a vibe, it's a MUST 🔐
3.1. Ports and Transports 🦋
Historically, NFS version 2 and version 3 servers lived on port 2049. The registered port 2049 [RFC1700] should be the default configuration for NFS. If NFS uses the registered port, the client won't need the RPC binding protocols from [RFC1833], and NFS can get through firewalls. Easy access, no drama 🌸
The transport that the RPC service uses for NFS version 4 MUST provide congestion control comparable to what [RFC2581] defines for TCP. If the operating environment implements TCP, NFS version 4 SHOULD be supported over TCP. The client and server may use other transports as long as they support congestion control as defined above. In those cases, a mechanism may be provided to override TCP usage in favor of another transport.
If TCP is the transport, the client and server SHOULD use persistent connections. This stops TCP's congestion control from getting weakened by short-lived connections, and it boosts performance over the WAN by skipping the need for SYN handshakes. Efficiency is everything 💅
For various timers, the client and server should avoid inadvertent synchronization of those timers. If you want the full story on that, check out [Floyd] 🧠
3.2. Security Flavors 🔐
Traditional RPC implementations included AUTH_NONE, AUTH_SYS, AUTH_DH, and AUTH_KRB4 as security flavors. [RFC2203] added RPCSEC_GSS, which uses the functionality of GSS-API [RFC2078]. This lets the RPC layer use different security mechanisms without the extra implementation work of adding a whole new RPC security flavor for each one. For NFS version 4, RPCSEC_GSS MUST be used to enable the mandatory security mechanism. AUTH_NONE, AUTH_SYS, and AUTH_DH MAY be implemented as well, if you want them 🦋
3.2.1. Security mechanisms for NFS version 4
Using RPCSEC_GSS means selecting three things: a mechanism, a quality of protection, and a service (authentication, integrity, or privacy). The rest of this document calls these three parameters the security triple. Iconic naming 🎀
3.2.1.1. Kerberos V5 as security triple
The Kerberos V5 GSS-API mechanism from [RFC1964] MUST be implemented, and it MUST provide the following security triples.
Here's the column key:
column descriptions:
1 == number of pseudo flavor
2 == name of pseudo flavor
3 == mechanism's OID
4 == mechanism's algorithm(s)
5 == RPCSEC_GSS serviceAnd here's the table:
1 2 3 4 5
-----------------------------------------------------------------------
390003 krb5 1.2.840.113554.1.2.2 DES MAC MD5 rpc_gss_svc_none
390004 krb5i 1.2.840.113554.1.2.2 DES MAC MD5 rpc_gss_svc_integrity
390005 krb5p 1.2.840.113554.1.2.2 DES MAC MD5 rpc_gss_svc_privacy
for integrity,
and 56 bit DES
for privacy.The pseudo flavor is only here as a mapping aid for implementers. This NFS protocol has its own way to negotiate security, and it understands the GSS-API mechanism, so the pseudo flavor isn't needed here. NFS version 3 does need it, though, because its security negotiation happens through the MOUNT protocol 🙈
If you want the NFS and Kerberos V5 tea, see [RFC2623] 🍵
3.2.1.2. LIPKEY as a security triple
The LIPKEY GSS-API mechanism from [RFC2847] MUST be implemented, and it MUST provide the following security triples. The columns mean the same thing as in the previous subsection, "Kerberos V5 as security triple."
1 2 3 4 5
-----------------------------------------------------------------------
390006 lipkey 1.3.6.1.5.5.9 negotiated rpc_gss_svc_none
390007 lipkey-i 1.3.6.1.5.5.9 negotiated rpc_gss_svc_integrity
390008 lipkey-p 1.3.6.1.5.5.9 negotiated rpc_gss_svc_privacyThe mechanism algorithm is listed as "negotiated," and here's why that's fine 💖 LIPKEY is layered on SPKM-3, and in [RFC2847] SPKM-3 negotiates its confidentiality and integrity algorithms. SPKM-3 says HMAC-MD5 for integrity is MANDATORY, and 128 bit cast5CBC for confidentiality (privacy) is MANDATORY too. It also says HMAC-MD5 and cast5CBC MUST be listed first, before any weaker algorithms. Because of all that, writing "negotiated" in column 4 doesn't hurt interoperability. If an SPKM-3 peer doesn't support the mandatory algorithms, the other peer is free to accept or reject the GSS-API context creation.
Since SPKM-3 does the algorithm negotiation, later calls to LIPKEY's GSS_Wrap() and GSS_GetMIC() made by RPCSEC_GSS will use a quality of protection value of 0 (zero). For the explanation, see section 5.2 of [RFC2025] 🧚
LIPKEY uses SPKM-3 to set up a secure channel where a user name and password can be passed from the client to the user. Once the server accepts that user name and password, calls on the LIPKEY context get redirected to the SPKM-3 context. More details live in [RFC2847] 🌈
3.2.1.3. SPKM-3 as a security triple
The SPKM-3 GSS-API mechanism from [RFC2847] MUST be implemented, and it MUST provide the following security triples. The columns match the previous subsection, "Kerberos V5 as security triple."
1 2 3 4 5-----------------------------------------------------------------------
390009 spkm3 1.3.6.1.5.5.1.3 negotiated rpc_gss_svc_none
390010 spkm3i 1.3.6.1.5.5.1.3 negotiated rpc_gss_svc_integrity
390011 spkm3p 1.3.6.1.5.5.1.3 negotiated rpc_gss_svc_privacyOkay so, for why the mechanism algorithm is listed as "negotiated" 🤔, check the previous section, "LIPKEY as a security triple." It explains everything 💖
Since SPKM-3 negotiates the algorithms, every later call to SPKM-3's GSS_Wrap() and GSS_GetMIC() made by RPCSEC_GSS uses a quality of protection value of 0 (zero). If you want the reasoning, see section 5.2 of [RFC2025] ✨
Even though LIPKEY is layered over SPKM-3, SPKM-3 is specified as a mandatory set of triples 🫶 That's for the situations where the initiator (the client) is anonymous, or where the initiator has its own certificate. If the initiator is anonymous, there won't be a user name and password to send to the target (the server) 🙈 And if the initiator has its own certificate, then using passwords is superfluous. Literally unnecessary 💅
3.3. Security Negotiation 🦋
With the NFS version 4 server potentially offering multiple security mechanisms, the client needs a way to figure out, or negotiate, which mechanism to use for talking to the server. The NFS server may have several entry points within its file system name space that NFS clients can use. And the server may be set up so that each of those entry points has different security mechanisms, or several at once. It's giving complicated 😭
The security negotiation between client and server must be done over a secure channel. That way, a third party can't intercept the negotiation and trick the client and server into picking a lower level of security than they need or want 🚫🕵️♀️ Nobody gets to downgrade our vibe 👑
3.3.1. Security Error 💅
So, each NFS version 4 client and server has to support a minimum set of security (LIPKEY, SPKM-3, and Kerberos-V5, all under RPCSEC_GSS). Based on that, the NFS client starts talking to the server with one of the minimal security triples 🧚 Then, while they're communicating, the client might get an NFS error called NFS4ERR_WRONGSEC. That error is how the server says, "hey, the triple you're using isn't appropriate for accessing my file system resources" 🙈 The client then takes responsibility for finding out which security triples the server has available and picking one that works for it. Main character energy, but make it secure 🎀
3.3.2. SECINFO 🌸
The new SECINFO operation lets the client figure out, on a per filehandle basis, which security triple to use for server access 🔍 Generally, the client won't need the SECINFO procedure except during initial communication with the server, or when the client crosses policy boundaries at the server. Server policies can change while the client is interacting with it, though, and that can force the client to negotiate a new security triple. Plot twist, every time 😭✨
3.4. Callback RPC Authentication 🔐
The callback RPC (described later) must mutually authenticate the NFS server to the principal that acquired the clientid (also described later). It uses the same security flavor that the original SETCLIENTID operation used 💜 Because LIPKEY is layered over SPKM-3, the server is allowed to use SPKM-3 instead of LIPKEY for the callback, even if the client used LIPKEY for SETCLIENTID. Iconic flexibility 🫶
For AUTH_NONE, there are no principals, so this is a non-issue 🙃
For AUTH_SYS, the server just uses the AUTH_SYS credential that the user had when they set up the delegation ✨
For AUTH_DH, one commonly used convention is that the server uses the credential that matches this AUTH_DH principal:
unix.host@domainHere, host and domain are variables. They stand for the name of the server host, and the directory services domain it lives in, like a Network Information System domain or a DNS domain 🍓
Regardless of which security mechanism under RPCSEC_GSS is in use, the NFS server MUST identify itself in GSS-API using a GSS_C_NT_HOSTBASED_SERVICE name type. GSS_C_NT_HOSTBASED_SERVICE names look like this:
service@hostnameFor NFS, the "service" element is:
nfsImplementations of security mechanisms convert nfs@hostname into different forms. For Kerberos V5 and LIPKEY, the following form is RECOMMENDED:
nfs/hostnameFor Kerberos V5, nfs/hostname would be a server principal in the Kerberos Key Distribution Center database 🔑 For LIPKEY, it would be the username passed to the target (the NFS version 4 client that receives the callback) 🫶
Heads up 👀 LIPKEY may not work for callbacks, since the LIPKEY client uses a user id and password. If the NFS client that gets the callback can authenticate the NFS server's user name and password pair, AND the user the NFS server is authenticating to has a public key certificate, then it works 🎀
In situations where the NFS client uses LIPKEY and a per-host principal for the SETCLIENTID operation, it is RECOMMENDED that SPKM-3 with mutual authentication be used, instead of LIPKEY for SETCLIENTID. Basically, the client uses a certificate to authenticate and identify the initiator to the target on the NFS server 🦋
Using SPKM-3 and not LIPKEY has these advantages 💖
- When the server does a callback, it must authenticate to the principal used in the SETCLIENTID. Even if LIPKEY is used, LIPKEY is layered over SPKM-3, so the NFS client will need a certificate that matches the principal used in the SETCLIENTID operation. From an admin perspective, having a user name, password, and certificate for both the client and server is redundant 😭
- LIPKEY was meant to keep extra infrastructure requirements to a minimum, beyond a certificate for the target. The expectation is that existing password infrastructure can be leveraged for the initiator. In some environments, a per-host password doesn't exist yet. If certificates are used for any per-host principals, then extra password infrastructure isn't needed 🧚
- If a host is both an NFS client and an NFS server, it can share the same per-host certificate 🌈
4. Filehandles 📂
The filehandle in the NFS protocol is a per-server unique identifier for a file system object. The contents of the filehandle are opaque to the client, so the server is responsible for translating the filehandle into an internal representation of the file system object 🔄 Since the filehandle is the client's reference to an object, and the client may cache that reference, the server SHOULD not reuse a filehandle for another file system object. If the server does need to reuse a filehandle value, the time elapsed before reuse SHOULD be large enough that the client is unlikely to still have a cached copy of the reused value ⏳ Note that a client may cache a filehandle for a very long time. For example, a client may cache NFS data to local storage to expand its effective cache size, and to survive client restarts 💾 So the lifetime of a cached filehandle may be extended. Long memory, no cap 🧠
4.1. Obtaining the First Filehandle 🚪
The operations of the NFS protocol are defined in terms of one or more filehandles. That means the client needs a filehandle to start communicating with the server 🥺 With the NFS version 2 protocol [RFC1094] and the NFS version 3 protocol [RFC1813], there's an ancillary protocol for getting this first filehandle. The MOUNT protocol, RPC program number 100005, translates a string-based file system path name into a filehandle, which the NFS protocols can then use 🗺️
The MOUNT protocol has deficiencies when it comes to security and use via firewalls 🚧 This is one reason the public filehandle was introduced in [RFC2054] and [RFC2055]. Using the public filehandle together with the LOOKUP procedure in the NFS version 2 and 3 protocols, it has been demonstrated that the MOUNT protocol is unnecessary for viable interaction between NFS client and server 🙌
So, the NFS version 4 protocol won't use an ancillary protocol to translate string-based path names into a filehandle. Instead, two special filehandles will be used as starting points for the NFS client 🎀
4.1.1. Root Filehandle 🌳
The first special filehandle is the ROOT filehandle. The ROOT filehandle is the "conceptual" root of the file system name space at the NFS server 🌳 The client uses, or starts with, the ROOT filehandle by employing the PUTROOTFH operation. The PUTROOTFH operation tells the server to set the "current" filehandle to the ROOT of the server's file tree. Once PUTROOTFH is used, the client can traverse the entire server file tree with the LOOKUP procedure 🚶♀️✨ A complete discussion of the server name space is in the section "NFS Server Name Space." Go read it, bestie 📖
4.1.2. Public Filehandle 🌐
The second special filehandle is the PUBLIC filehandle. Unlike the ROOT filehandle, the PUBLIC filehandle may be bound to, or represent, an arbitrary file system object at the server. The server is responsible for this binding. It may be that the PUBLIC filehandle and the ROOT filehandle refer to the same file system object. However, it's up to the administrative software at the server, and the policies of the server administrator, to define how the PUBLIC filehandle is bound to a server file system object. The client may not make any assumptions about this binding 🙅♀️
4.2. Filehandle Types 🎀
In the NFS version 2 and 3 protocols, there was one type of filehandle, with a single set of semantics. The NFS version 4 protocol introduces a new type of filehandle to accommodate certain server environments 🫶 The first type is 'persistent'. The semantics of a persistent filehandle are the same as the filehandles of the NFS version 2 and 3 protocols. The second, new type is the "volatile" filehandle 🦋
The volatile filehandle type is being introduced to address server functionality or implementation issues that make a correct implementation of a persistent filehandle infeasible 😩 Some server environments don't provide a file system level invariant that can be used to construct a persistent filehandle. The underlying server file system may not provide the invariant, or the server's file system programming interfaces may not give access to the needed invariant 🙈 Volatile filehandles can make it easier to implement server functionality like hierarchical storage management or file system reorganization or migration 🗂️ However, the volatile filehandle increases the implementation burden for the client. Still, this increased burden is deemed acceptable given the overall gains the protocol achieves 💅
Since the client needs to handle persistent and volatile filehandles differently, a file attribute is defined that the client can use to determine which filehandle types the server is returning 🔍✨
4.2.1. General Properties of a Filehandle ✨
okay so the filehandle holds everything the server needs to tell one file apart from another 🔍 to the client it's totally opaque, like a sealed gift 🎁 the client can stash it and use it in a later request, and it can compare two filehandles from the same server byte by byte to see if they match. but the client MUST NOT try to read anything else inside them 🚫 if two filehandles from the same server are equal, they MUST point to the same file 💖 if they're not equal, the client may look at file attributes from the server to figure out whether they're the same file or different ones, and it might do that for client side caching 🧠 servers SHOULD try to keep a one-to-one match between filehandles and files, but it's not required 🙃 clients MUST only use filehandle comparisons to speed things up, never for correct behavior ⚡ and every client needs to be ready for when it just can't tell whether two filehandles are the same object. in those cases, avoid making assumptions that could break things 🫶
for more on comparing filehandles and attributes when caching data, check out the section "Data Caching and File Identity" 🌸
here's a cute example: if two different path names end up at the same file system object when the server walks them, the server SHOULD hand back the same filehandle for each path 🦋 this happens with hard links, where two file names point at the same underlying file and its data. like, if /a/b/c and /a/d/c lead to the same file, the server SHOULD return the same filehandle for both path traversals 💅
4.2.2. Persistent Filehandle
a persistent filehandle has a fixed value for as long as the file system object it points to exists 💜 once the server makes a filehandle for an object, the server MUST accept that same filehandle for the object for the object's entire lifetime 🔒 if the server restarts or reboots, the NFS server must honor the same filehandle value it used in its previous instantiation. same goes if the file system gets migrated: the new NFS server must honor the same file handle as the old NFS server 🚚
a persistent filehandle becomes stale or invalid when its file system object gets removed 💀 when the server gets a persistent filehandle that points to a deleted object, it MUST return an error of NFS4ERR_STALE 🙅♀️ a filehandle can also go stale when the file system holding the object isn't available anymore. that can happen if the file system is on removable media and the media isn't around at the server anymore, if the whole file system got destroyed, or if the file system was simply removed from the server's name space (like unmounted in a Unix environment) 🥀
4.2.3. Volatile Filehandle
a volatile filehandle does NOT have the same longevity as a persistent one 😭 the server might decide a volatile filehandle isn't valid anymore at lots of different moments 🕰️ if the server can definitively tell that a volatile filehandle points to an object that has been removed, the server should return NFS4ERR_STALE to the client (just like it does for persistent filehandles). in every other case where the server decides a volatile filehandle can't be used anymore, it should return an error of NFS4ERR_FHEXPIRED 🌈
the mandatory attribute fh_expire_type is how the client finds out what kind of filehandle the server is giving for a particular file system. it's a bitmask with these values:
FH4_PERSISTENT: this one means a persistent filehandle, valid until the object is removed from the file system. the server will not returnNFS4ERR_FHEXPIREDfor this filehandle.FH4_PERSISTENTis defined as the value where none of the bits below are set 🧚FH4_NOEXPIRE_WITH_OPEN: the filehandle won't expire while the client has the file open. if this bit is set, thenFH4_VOLATILE_ANYorFH4_VOL_RENAMEdon't affect expiration while the file is open. once the file gets closed, or ifFH4_NOEXPIRE_WITH_OPENis false, the rest of the volatile related bits apply 🔐FH4_VOLATILE_ANY: the filehandle may expire at any time, and it will expire during system migration and rename 🌪️FH4_VOL_MIGRATION: the filehandle will expire during file system migration. this may only be set ifFH4_VOLATILE_ANYis not set 🚛FH4_VOL_RENAME: the filehandle may expire due to a rename. that includes a rename by the requesting client or a rename by another client. this may only be set ifFH4_VOLATILE_ANYis not set ✏️
servers that provide volatile filehandles should deny a RENAME or REMOVE that would affect an OPEN file or any of the components leading to the OPEN file 🙅 on top of that, the server should deny all RENAME or REMOVE requests during the grace or lease period after a server restart ⏳
you might be wondering why there are three FH4_VOL* bits, and why FH4_VOLATILE_ANY is exclusive of FH4_VOL_MIGRATION and FH4_VOL_RENAME. ngl it makes sense once you see it: if a filehandle is normally persistent but can't survive a file set migration, then seeing FH4_VOL_MIGRATION or FH4_VOL_RENAME tells the client it can treat the filehandle as persistent for keeping a file name to filehandle cache, except for the specific event named by the bit 🧁 FH4_VOLATILE_ANY, on the other hand, tells the client it should NOT keep that kind of cache for unopened files 🙈 a server MUST not present FH4_VOLATILE_ANY together with FH4_VOL_MIGRATION or FH4_VOL_RENAME, because that would just lead to confusion 🌀 and FH4_VOLATILE_ANY implies the filehandle will expire upon migration or rename, plus whatever other events apply 💜
4.2.4. One Method of Constructing a Volatile Filehandle
okay so as we said, sometimes a filehandle is stale (no longer valid, maybe because the file got removed from the server) and sometimes it's expired (the underlying file is still valid, but since the filehandle is volatile, it may have expired anyway) 🤯 so the server needs a way to return NFS4ERR_STALE in the first case and NFS4ERR_FHEXPIRED in the second. this can be done by carefully constructing the volatile filehandle, and here's one possible way to do it 🛠️
a volatile filehandle, while still opaque to the client, could contain:
[volatile bit = 1 | server boot time | slot | generation number]slotis an index in the server volatile filehandle tablegeneration numberis the generation number for the table entry/slot
if the server boot time is less than the current server boot time, return NFS4ERR_FHEXPIRED. if slot is out of range, return NFS4ERR_BADHANDLE. if the generation number doesn't match, return NFS4ERR_FHEXPIRED 🎯
when the server reboots, the table is gone (because it's volatile) 💨
if the volatile bit is 0, then it's a persistent filehandle with a different structure following it 🦋
4.3. Client Recovery from Filehandle Expiration
if it's possible, the client SHOULD recover from receiving an NFS4ERR_FHEXPIRED error 🙌 the client has to take on some extra responsibility so it can get ready to recover from a volatile filehandle expiring. if the server returns persistent filehandles, the client doesn't need these extra steps 💅
for volatile filehandles, the client will most commonly need to store the component names leading up to and including the file system object in question 📝 with those names, the client should be able to recover either by finding a filehandle in the name space that's still available, or by starting at the root of the server's file system name space 🌳
if the expired filehandle refers to an object that's been removed from the file system, obviously the client can't recover from the expired filehandle 😔
it's also possible the expired filehandle refers to a file that got renamed. if another client renamed the file, the original client again might not be able to recover 🫠 but if the client itself is renaming the file and the file is open, the client may be able to recover ✨ the client can work out the new path name from how it processed the rename request, and then regenerate the new filehandle from that new path name 🧩 the client could also use the compound operation mechanism to build a set of operations like this:
RENAME A B
LOOKUP B
GETFH5. File Attributes ✨
okay so attributes need to be flexible, bestie 💅 because NFS has to play nice with non-Unix platforms too. In NFS Version 3, the fattr3 structure has a fixed list of attributes that not every client and server can support or even cares about. It can't be extended when new needs pop up, and there's no way to say "nope, I don't support that one." 🙈 With NFS Version 4, the client can ask the server what attributes it supports and then request only the ones it actually wants. main character energy 👑
So attributes get split into three groups: mandatory, recommended, and named. Mandatory and recommended attributes both get a specific, well-defined encoding in NFS version 4 and are identified by number. To ask for one, you set a bit in the bit vector sent in the GETATTR request, and the server's response includes a bit vector listing which attributes it actually returned. New mandatory or recommended attributes can show up between major revisions too, as long as someone publishes a standards-track RFC that allocates a new attribute number and defines how it's encoded. See the section "Minor Versioning" for more on that 🦋
Named attributes are a whole different vibe. They're accessed through the new OPENATTR operation, which opens a hidden directory of attributes tied to a file system object. OPENATTR takes the object's filehandle and gives back the filehandle for the attribute hierarchy. That filehandle is a directory you can LOOKUP or READDIR, and it contains files whose names are the attribute names and whose data bytes are the attribute's value. Here's how it looks:
LOOKUP "foo" ; look up file
GETATTR attrbits
OPENATTR ; access foo's named attributes
LOOKUP "x11icon" ; look up specific attribute
READ 0,4096 ; read stream of bytesNamed attributes are meant for app data rather than anything an NFS client implementation needs 🌸 And NFS implementors are strongly encouraged to make their new attributes recommended ones by taking them through the IETF standards-track process. ngl that's a big ask but it's the move 💜
The mandatory set is deliberately tiny, because servers have to do whatever it takes to support every single one ✨ Recommended attributes can be unsupported, although a server should support as many as it can. An attribute gets to be mandatory only if a lot of clients need the data and the client can't reasonably compute it when the server doesn't provide it.
5.1. Mandatory Attributes 🎀
These MUST be supported by every NFS Version 4 client and server so everyone has a minimum level of interoperability 🫶 The server has to store and return these attributes, and the client has to be able to work with just this limited set. Yes, some client features might be impaired or limited with only the mandatory attributes, but it still works. A client can ask for any of these by setting a bit in the GETATTR request, and the server must return their values.
5.2. Recommended Attributes 🌈
These are understood well enough to deserve a spot in the NFS Version 4 protocol, but not every client and server will support them. A client can ask for any of them by setting a bit in the GETATTR request, but it has to handle the case where the server doesn't return them. A client can ask for the set of attributes the server supports, and it should not request attributes the server doesn't support. A server should be chill about requests for unsupported attributes and just leave them out of the response instead of treating the request as an error 🙃 The general expectation is that servers support every attribute they comfortably can and only skip the ones that are hard to support in their environment.
A server should provide attributes whenever it doesn't have to "tell lies" to the client. For example, a file modification time should either be accurate or not supported at all. Yes, this won't always feel great for clients, but the client is way better at making up or constructing an attribute, or just doing without it, than the server is at faking it 💅
5.3. Named Attributes 🦋
These aren't directly encoded in the NFS Version 4 protocol. Instead, they're accessed by string names rather than numbers, and each one corresponds to an uninterpreted stream of bytes stored with the file system object. The name space can be reached with the OPENATTR operation, which returns a filehandle for a virtual "attribute directory." From there you can keep exploring the name space with READDIR and LOOKUP. Once you have the filehandles from READDIR and LOOKUP, you can examine or change named attributes using normal READ, WRITE, and CREATE operations. And named attributes can have attributes of their own, which is a little bit iykyk 🙈
It is recommended that servers support arbitrary named attributes. A client should not depend on being able to store any named attributes in the server's file system. If a server does support named attributes, a client that can also handle them should be able to copy a file's data and meta-data from one location to another with complete transparency. That would mean the names allowed for regular directory entries are also valid for named attribute names ✨
Names of attributes aren't controlled by this document or by other IETF standards-track documents. See the section "IANA Considerations" for more on that 🌸
5.4. Mandatory Attributes - Definitions 🍓
Here's the table of mandatory attributes, with each one's number, data type, access, and what it means:
Name # DataType Access Description
___________________________________________________________________
supp_attr 0 bitmap READ The bit vector which
would retrieve all
mandatory and
recommended attributes
that are supported for
this object.
type 1 nfs4_ftype READ The type of the object
(file, directory,
symlink)
fh_expire_type 2 uint32 READ Server uses this to
specify filehandle
expiration behavior to
the client. See the
section "Filehandles"
for additional
description.
change 3 uint64 READ A value created by the
server that the client
can use to determine
if file data,
directory contents or
attributes of the
object have been
modified. The server
may return the
object's time_modify
attribute for this
attribute's value but
only if the file
system object can not
be updated more
frequently than the
resolution of
time_modify.
size 4 uint64 R/W The size of the object
in bytes.
link_support 5 boolean READ Does the object's file
system supports hard
links?
symlink_support 6 boolean READ Does the object's file
system supports
symbolic links?
named_attr 7 boolean READ Does this object have
named attributes?
fsid 8 fsid4 READ Unique file system
identifier for the
file system holding
this object. fsid
contains major and
minor components each
of which are uint64.
unique_handles 9 boolean READ Are two distinct
filehandles guaranteed
to refer to two
different file system
objects?
lease_time 10 nfs_lease4 READ Duration of leases at
server in seconds.
rdattr_error 11 enum READ Error returned from
getattr during
readdir.5.5. Recommended Attributes - Definitions 🔥
And here's the table for the recommended attributes, same deal:
Name # Data Type Access Description
_____________________________________________________________________
ACL 12 nfsace4<> R/W The access control
list for the object.
aclsupport 13 uint32 READ Indicates what types
of ACLs are supported
on the current file
system.
archive 14 boolean R/W Whether or not this
file has been
archived since the
time of last
modification
(deprecated in favor
of time_backup).
cansettime 15 boolean READ Is the server able to
change the times for
a file system object
as specified in a
SETATTR operation?
case_insensitive 16 boolean READ Are filename
comparisons on this
file system case
insensitive?okay the attribute list keeps going, bestie 🧚
case_preserving 17 boolean READ Is filename case on
this file system
preserved?
chown_restricted 18 boolean READ If TRUE, the server
will reject any
request to change
either the owner or
the group associated
with a file if the
caller is not a
privileged user (for
example, "root" in
Unix operating
environments or in NT
the "Take Ownership"
privilege)
filehandle 19 nfs4_fh READ The filehandle of
this object
(primarily for
readdir requests).
fileid 20 uint64 READ A number uniquely
identifying the file
within the file
system.
files_avail 21 uint64 READ File slots available
to this user on the
file system
containing this
object - this should
be the smallest
relevant limit.
files_free 22 uint64 READ Free file slots on
the file system
containing this
object - this should
be the smallest
relevant limit.
files_total 23 uint64 READ Total file slots on
the file system
containing this
object.
fs_locations 24 fs_locations READ Locations where this
file system may be
found. If the server
returns NFS4ERR_MOVED
as an error, this
attribute must be
supported.
hidden 25 boolean R/W Is file considered
hidden with respect
to the WIN32 API?
homogeneous 26 boolean READ Whether or not this
object's file system
is homogeneous, i.e.
are per file system
attributes the same
for all file system's
objects.
maxfilesize 27 uint64 READ Maximum supported
file size for the
file system of this
object.
maxlink 28 uint32 READ Maximum number of
links for this
object.
maxname 29 uint32 READ Maximum filename size
supported for this
object.
maxread 30 uint64 READ Maximum read size
supported for this
object.
maxwrite 31 uint64 READ Maximum write size
supported for this
object. This
attribute SHOULD be
supported if the file
is writable. Lack of
this attribute can
lead to the client
either wasting
bandwidth or not
receiving the best
performance.
mimetype 32 utf8<> R/W MIME body
type/subtype of this
object.
mode 33 mode4 R/W Unix-style permission
bits for this object
(deprecated in favor
of ACLs)
no_trunc 34 boolean READ If a name longer than
name_max is used,
will an error be
returned or will the
name be truncated?
numlinks 35 uint32 READ Number of hard links
to this object.
owner 36 utf8<> R/W The string name of
the owner of this
object.
owner_group 37 utf8<> R/W The string name of
the group ownership
of this object.
quota_avail_hard 38 uint64 READ For definition see
"Quota Attributes"
section below.
quota_avail_soft 39 uint64 READ For definition see
"Quota Attributes"
section below.
quota_used 40 uint64 READ For definition see
"Quota Attributes"
section below.
rawdev 41 specdata4 READ Raw device
identifier. Unix
device major/minor
node information.
space_avail 42 uint64 READ Disk space in bytes
available to this
user on the file
system containing
this object - this
should be the
smallest relevant
limit.
space_free 43 uint64 READ Free disk space in
bytes on the file
system containing
this object - this
should be the
smallest relevant
limit.
space_total 44 uint64 READ Total disk space in
bytes on the file
system containing
this object.
space_used 45 uint64 READ Number of file system
bytes allocated to
this object.
system 46 boolean R/W Is this file a system
file with respect to
the WIN32 API?
time_access 47 nfstime4 READ The time of last
access to the object.
time_access_set 48 settime4 WRITE Set the time of last
access to the object.
SETATTR use only.
time_backup 49 nfstime4 R/W The time of last
backup of the object.
time_create 50 nfstime4 R/W The time of creation
of the object. This
attribute does not
have any relation to
the traditional Unix
file attribute
"ctime" or "change
time".
time_delta 51 nfstime4 READ Smallest useful
server time
granularity.
time_metadata 52 nfstime4 R/W The time of last
meta-data
modification of the
object.
time_modify 53 nfstime4 READ The time of last
modification to the
object.
time_modify_set 54 settime4 WRITE Set the time of last
modification to the
object. SETATTR use
only.5.6. Interpreting owner and owner_group 💖
okay so owner and owner_group are both recommended attributes, and they're represented as a UTF-8 string 🫶 The reason is that we don't want the format to be tied to whatever a particular client or server uses under the hood. Note that section 6.1 of [RFC2624] gives more of the reasoning behind this.
Each side is expected to have its own local way of representing owner and owner_group, used for storage or for showing the end user. So when these attributes travel between client and server, the local form gets translated into a syntax of the form user@dns_domain ✨ That way a client and a server that don't share a local representation can still meet in the middle with a common syntax that both can interpret. Main character energy for interoperability 👑
The translation itself is not specified as part of the protocol, so people can pick whatever solution fits. For example, a local translation table could map a numeric id to the user@dns_domain syntax. A name service could also do the job. The dns_domain portion of the owner string is meant to be a DNS domain name, for example user@ietf.org 🌸
Now, if there's no translation available on the client or server, the attribute value has to be built without the "@" 🙈 So if the "@" is missing from the owner or owner_group attribute, that just means no translation was available, and the receiver shouldn't read any special meaning into the attribute value. Even though it can't be translated, it can still be useful. A client, for example, can use the string to display ownership locally 🦋
5.7. Character Case Attributes 🎀
okay this one's about the case_insensitive and case_preserving attributes 🧚 Each UCS-4 character (which UTF-8 encodes) has a "long descriptive name" [RFC1345], and that name may or may not include the word "CAPITAL" or "SMALL". Why does that matter? Because the presence of SMALL or CAPITAL lets an NFS server build unambiguous and efficient table-driven mappings for case-insensitive comparisons and for non-case-preserving storage. For general character handling and internationalization issues, see the section "Internationalization" 🍓
5.8. Quota Attributes 💅
for the attributes related to file system quotas, here are the definitions:
quota_avail_soft: the value in bytes for how much additional disk space can be allocated to this file or directory before the user may reasonably be warned. It's understood that this space may be consumed by allocations to other files or directories, though there is a rule as to which other files or directories. 💜quota_avail_hard: the value in bytes for how much additional disk space beyond the current allocation can be allocated to this file or directory before further allocations get refused. It's understood that this space may be consumed by allocations to other files or directories. 🔥quota_used: the value in bytes for how much disc space is used by this file or directory and possibly a number of other similar files or directories. The set of "similar" files meets at least one criterion: allocating space to any file or directory in the set will reduce thequota_avail_hardof every other file or directory in the set. ✨
Note that there may be several distinct but overlapping sets of files or directories for which a quota_used value is maintained. E.g. "all files with a given owner", "all files with a given group owner". etc. 🦋
The server is at liberty to choose any of those sets, but it should do so in a repeatable way. The rule may be configured per-filesystem, or it may be "choose the set with the smallest quota" 🌈
5.9. Access Control Lists 🔐
okay the NFS ACL attribute is an array of access control entries (ACE) 🫶 There are various ACE types, and the server can communicate which ones it supports by returning the appropriate value within the aclsupport attribute. The ACE types are:
Type Description
_____________________________________________________
ALLOW Explicitly grants the access defined in
acemask4 to the file or directory.
DENY Explicitly denies the access defined in
acemask4 to the file or directory.
AUDIT LOG (system dependent) any access
attempt to a file or directory which
uses any of the access methods specified
in acemask4.
ALARM Generate a system ALARM (system
dependent) when any access attempt is
made to a file or directory for the
access methods specified in acemask4.The NFS ACE attribute is defined like this 🧚
typedef uint32_t acetype4;
typedef uint32_t aceflag4;
typedef uint32_t acemask4;
struct nfsace4 {
acetype4 type;
aceflag4 flag;
acemask4 access_mask;
utf8string who;
};Here's how the server decides whether an ACCESS or OPEN request succeeds. It processes each nfsace4 entry in order. Only ACEs whose "who" matches the requester get considered. Each ACE is processed until all of the bits of the requester's access have been ALLOWED. Once a bit (see below) has been ALLOWED by an ACCESS_ALLOWED_ACE, it's no longer considered in the processing of later ACEs. If an ACCESS_DENIED_ACE is encountered where the requester's mode still has unALLOWED bits in common with the "access_mask" of the ACE, the request is denied 🙈
The bitmask constants used to represent the above definitions within the aclsupport attribute are:
const ACL4_SUPPORT_ALLOW_ACL = 0x00000001;
const ACL4_SUPPORT_DENY_ACL = 0x00000002;
const ACL4_SUPPORT_AUDIT_ACL = 0x00000004;
const ACL4_SUPPORT_ALARM_ACL = 0x00000008;5.9.1. ACE type 🌸
the semantics of the "type" field follow the descriptions I just gave you above 💖 The bitmask constants for the type field are:
const ACE4_ACCESS_ALLOWED_ACE_TYPE = 0x00000000;
const ACE4_ACCESS_DENIED_ACE_TYPE = 0x00000001;
const ACE4_SYSTEM_AUDIT_ACE_TYPE = 0x00000002;
const ACE4_SYSTEM_ALARM_ACE_TYPE = 0x00000003;5.9.2. ACE flag 🦋
the "flag" field holds values based on the following descriptions 👑
ACE4_FILE_INHERIT_ACE: can be placed on a directory, and it means this ACE should be added to each new non-directory file created. 🍓
ACE4_DIRECTORY_INHERIT_ACE: can be placed on a directory, and it means this ACE should be added to each new directory created. 🌈
ACE4_INHERIT_ONLY_ACE: can be placed on a directory but doesn't apply to the directory itself. It only applies to newly created files and directories, as specified by the two flags above. ✨
ACE4_NO_PROPAGATE_INHERIT_ACE: can be placed on a directory. Normally, when a new directory is created and an ACE exists on the parent marked ACL4_DIRECTORY_INHERIT_ACE, two ACEs get placed on the new directory. One is for the directory itself, and one is an inheritable ACE for newly created directories. This flag tells the server not to place an ACE on the newly created directory that's inheritable by its subdirectories. 🫶
ACE4_SUCCESSFUL_ACCESS_ACE_FLAG and ACL4_FAILED_ACCESS_ACE_FLAG: both indicate, for AUDIT and ALARM, which state to log the event in 🔥 On every ACCESS or OPEN call that happens on a file or directory with an ACL of type ACE4_SYSTEM_AUDIT_ACE_TYPE or ACE4_SYSTEM_ALARM_ACE_TYPE, the attempted access is compared to the ace4mask of those ACLs. If the access is a subset of ace4mask and the identifier matches, an AUDIT trail or an ALARM is generated. By default this happens regardless of whether the ACCESS or OPEN call succeeded or failed 💀
The flag ACE4_SUCCESSFUL_ACCESS_ACE_FLAG only produces the AUDIT or ALARM if the ACCESS or OPEN call is successful. The ACE4_FAILED_ACCESS_ACE_FLAG causes the ALARM or AUDIT if the ACCESS or OPEN call fails 🙌
ACE4_IDENTIFIER_GROUP: indicates that the "who" refers to a GROUP as defined under Unix. 🧚
The bitmask constants for the flag field are:
const ACE4_FILE_INHERIT_ACE = 0x00000001;
const ACE4_DIRECTORY_INHERIT_ACE = 0x00000002;
const ACE4_NO_PROPAGATE_INHERIT_ACE = 0x00000004;
const ACE4_INHERIT_ONLY_ACE = 0x00000008;
const ACE4_SUCCESSFUL_ACCESS_ACE_FLAG = 0x00000010;
const ACE4_FAILED_ACCESS_ACE_FLAG = 0x00000020;
const ACE4_IDENTIFIER_GROUP = 0x00000040;5.9.3. ACE Access Mask 💜
the access_mask field holds values based on the following 🌸
Access Description
_______________________________________________________________
READ_DATA Permission to read the data of the file
LIST_DIRECTORY Permission to list the contents of a
directory
WRITE_DATA Permission to modify the file's data
ADD_FILE Permission to add a new file to a
directory
APPEND_DATA Permission to append data to a file
ADD_SUBDIRECTORY Permission to create a subdirectory to a
directory
READ_NAMED_ATTRS Permission to read the named attributes
of a file
WRITE_NAMED_ATTRS Permission to write the named attributes
of a file
EXECUTE Permission to execute a file
DELETE_CHILD Permission to delete a file or directory
within a directory
READ_ATTRIBUTES The ability to read basic attributes
(non-acls) of a file
WRITE_ATTRIBUTES Permission to change basic attributes
(non-acls) of a file
DELETE Permission to Delete the file
READ_ACL Permission to Read the ACL
WRITE_ACL Permission to Write the ACL
WRITE_OWNER Permission to change the owner
SYNCHRONIZE Permission to access file locally at the
server with synchronous reads and writesThe bitmask constants for the access mask field are:
const ACE4_READ_DATA = 0x00000001;
const ACE4_LIST_DIRECTORY = 0x00000001;
const ACE4_WRITE_DATA = 0x00000002;
const ACE4_ADD_FILE = 0x00000002;
const ACE4_APPEND_DATA = 0x00000004;
const ACE4_ADD_SUBDIRECTORY = 0x00000004;
const ACE4_READ_NAMED_ATTRS = 0x00000008;
const ACE4_WRITE_NAMED_ATTRS = 0x00000010;
const ACE4_EXECUTE = 0x00000020;
const ACE4_DELETE_CHILD = 0x00000040;
const ACE4_READ_ATTRIBUTES = 0x00000080;
const ACE4_WRITE_ATTRIBUTES = 0x00000100;
const ACE4_DELETE = 0x00010000;
const ACE4_READ_ACL = 0x00020000;
const ACE4_WRITE_ACL = 0x00040000;
const ACE4_WRITE_OWNER = 0x00080000;
const ACE4_SYNCHRONIZE = 0x00100000;5.9.4. ACE who 🫶
okay so there are these special "who" identifiers that EVERY NFS implementation needs to understand, no exceptions 💅. Some of them are kinda tricky because they only make sense when a local process is touching the file, not when an NFS client is asking. But the good news is you're still allowed to display and modify these permissions over NFS 🦋
Who Description
_______________________________________________________________
"OWNER" The owner of the file.
"GROUP" The group associated with the file.
"EVERYONE" The world.
"INTERACTIVE" Accessed from an interactive terminal.
"NETWORK" Accessed via the network.
"DIALUP" Accessed as a dialup user to the server.
"BATCH" Accessed from a batch job.
"ANONYMOUS" Accessed without any authentication.
"AUTHENTICATED" Any authenticated user (opposite of
ANONYMOUS)
"SERVICE" Access from a system service.To avoid conflicts, these special identifiers get an appended @ and look like xxxx@. Note: there's no domain name after the @ 🙈 So for example, you'd write ANONYMOUS@ ✨
6. File System Migration and Replication 🚚
With the recommended attribute fs_locations, the NFS version 4 server has a way to offer file system migration or replication services 🌸 For the purposes of migration and replication, a file system is defined as all files that share a given fsid (both the major and minor values are the same). Cute and specific 🤓
The fs_locations attribute gives you a list of file system locations. Each location is specified by a server name (either a DNS domain or an IP address) and the path name for the root of the file system 🗺️ Depending on the kind of service being provided, the list will either give a new location or a set of alternate locations for the file system. The client uses this info to redirect its requests to the new server 💜
6.1. Replication
It's expected that file system replication will be used for read-only data 📚 Usually the file system gets replicated on two or more servers, and the fs_locations attribute hands the client the list of those locations. On first access of the file system, the client should obtain the value of the fs_locations attribute. If the client later finds the server unresponsive, it may try another server listed in fs_locations 😭
If applicable, the client must take the appropriate steps to recover valid filehandles from the new server. This is covered in more detail in the following sections 🫶
6.2. Migration
File system migration is used to move a file system from one server to another 🚛 It's typically used for a file system that is writable and has a single copy, and the expected use is load balancing or general resource reallocation ⚖️ The protocol does NOT specify how the file system actually gets moved between servers. That server-to-server transfer mechanism is left up to the server implementor 🛠️ What IS specified here is how the migration event gets communicated between client and server.
Once the servers involved have finished moving the file system, the error NFS4ERR_MOVED gets returned for any later requests that hit the original server 🙅♀️ It's returned for all operations except GETATTR. When the client gets NFS4ERR_MOVED, it fetches the value of the fs_locations attribute and uses its contents to redirect requests to the specified server. To make GETATTR work smoothly, operations such as PUTFH also have to be accepted by the server for the migrated file system's filehandles 🧚 And bestie, if the server returns NFS4ERR_MOVED, the server MUST support the fs_locations attribute. Non-negotiable 💀
If the client asks for more attributes than just fs_locations, the server may return only fs_locations. This is totally expected, since the server has already migrated the file system and might not have a way to get the extra attribute data 🤷♀️
The server implementor needs to be careful here, no cap 😤 The server must consider all of the state information clients might have outstanding. That includes (but isn't limited to) locking and share state, delegation state, and asynchronous file writes, which are represented by WRITE and COMMIT verifiers 🔐 The server should strive to minimize the impact on its clients during and after the migration process ✨
6.3. Interpretation of the fs_locations Attribute
The fs_location attribute is structured like this:
struct fs_location {
utf8string server<>;
pathname4 rootpath;
};
struct fs_locations {
pathname4 fs_root;
fs_location locations<>;
};So the fs_location struct represents where a file system lives, by giving a server name and the path to its root 📍 For a multi-homed server, or a set of servers that use the same rootpath, an array of server names can be provided. Each entry in that server array is a UTF8 string and represents one of: a traditional DNS host name, an IPv4 address, or an IPv6 address 🌐 It is NOT a requirement that every server sharing the same rootpath gets listed in one fs_location struct. That array is just there for convenience. Servers sharing the same rootpath can also be listed in separate fs_location entries in the fs_locations attribute 🦋
The fs_locations struct and attribute then contain an array of locations. Since each server's name space might be built differently, there's a fs_root field 🌳 The path represented by fs_root is where the file system lives in the server's name space. So the fs_root path is only associated with the server the fs_locations attribute came from. It's meant to help the client find the file system on the various servers listed 🔍
Okay, example time, this one is so good 😭 Say there's a replicated file system on two servers, servA and servB. On servA it lives at path /a/b/c, and on servB it lives at path /x/y/z. The client first accesses the file system at servA with a multi-component lookup path of /a/b/c/d. Because the client got that filehandle with a multi-component lookup, it doesn't know the file system's root is at /a/b/c in servA's name space. When the client switches to servB, it needs to figure out that the directory it first referenced on servA is now at /x/y/z/d on servB 🤯 To make that work, the fs_locations attribute from servA would have an fs_root value of /a/b/c and two fs_location entries: one for itself (servA), and one for servB with a path of /x/y/z. With that info, the client swaps /x/y/z in for the /a/b/c at the start of its access path and builds /x/y/z/d to use on the new server 💖
6.4. Filehandle Recovery for Migration or Replication
Filehandles for file systems that are replicated or migrated generally have the same semantics as filehandles for file systems that aren't 🧸 For example, if a file system has persistent filehandles and it gets migrated to another server, the filehandle values stay valid at the new server ✅
Volatile filehandles are the tricky ones 😬 The servers involved probably don't have a way to pass filehandle format and content between each other. So a server might have trouble figuring out whether a volatile filehandle from an old server should return NFS4ERR_FHEXPIRED. That's why the client gets told, through the fh_expire_type attribute, whether volatile filehandles will expire at the migration or replication event 📣 If the bit FH4_VOL_MIGRATION is set in fh_expire_type, the client must treat the volatile filehandle as if the server had returned NFS4ERR_FHEXPIRED. At the migration or replication event, when FH4_VOL_MIGRATION is set, the client will NOT present the original or old volatile filehandle to the new server. Instead, it starts talking to the new server by recovering its filehandles using the saved file names 🔄
7. NFS Server Name Space 🗂️
7.1. Server Exports
On a UNIX server, the name space describes all the files reachable by pathnames under the root directory, or / 🐧 On a Windows NT server, the name space is all the files on disks named by mapped disk letters 💻 NFS server admins rarely make the server's entire file system name space available to NFS clients 🙅♀️ More often, portions of the name space are made available through an "export" feature 📤 In previous versions of the NFS protocol, the root filehandle for each export was obtained through the MOUNT protocol. The client sends a string identifying the export of name space, and the server returns the root filehandle for it 🎟️ The MOUNT protocol also supports an EXPORTS procedure that enumerates the server's exports.
7.2. Browsing Exports
The NFS version 4 protocol provides a root filehandle that clients can use to get filehandles for these exports through a multi-component LOOKUP 🧭 A super common user experience is a graphical interface (think a file "Open" dialog window) where you find a file by browsing progressively through a directory tree 📂 The client must be able to move from one export to another through single-component, progressive LOOKUP operations ✨
This style of browsing isn't well supported by the NFS version 2 and 3 protocols 😵 The client expects all LOOKUP operations to stay within a single server file system. For example, the device attribute won't change. That stops a client from taking name space paths that span exports 🚫
An automounter on the client can get a snapshot of the server's name space using the EXPORTS procedure of the MOUNT protocol 📸 If it understands the server's pathname syntax, it can build an image of the server's name space on the client. The parts of the name space that aren't exported by the server get filled in with a "pseudo file system", which lets the user browse from one mounted file system to another 🌈 But there's a drawback to this representation: it's static. If the server admin adds a new export, the client won't know about it 🥲
7.3. Server Pseudo File System ✨
Okay so NFS version 4 servers avoid this name space inconsistency by presenting all the exports inside one single server name space 🫶 An NFS version 4 client uses LOOKUP and READDIR operations to browse seamlessly from one export to another. Parts of the server name space that aren't exported get bridged by a "pseudo file system" that only shows you exported directories. A pseudo file system has a unique fsid and behaves like a normal, read only file system.
Based on how the server's name space is built, it's possible to have multiple pseudo file systems 🧩 For example:
/a pseudo file system
/a/b real file system
/a/b/c pseudo file system
/a/b/c/d real file systemEach of the pseudo file systems are considered separate entities and therefore will have a unique fsid.
7.4. Multiple Roots 🎀
The DOS and Windows operating environments are sometimes described as having "multiple roots" 💅 File systems are commonly represented as disk letters, and MacOS represents file systems as top level names. NFS version 4 servers for these platforms can build a pseudo file system above these root names, so disk letters or volume names are just directory names in the pseudo root. Iconic 🌈
7.5. Filehandle Volatility 🦋
The server's pseudo file system is a logical representation of the file system(s) available from the server. So it's most likely built dynamically when the server is first instantiated. It's expected that the pseudo file system may not have an on disk counterpart that persistent filehandles could be constructed from. Even though it's preferable for the server to provide persistent filehandles for the pseudo file system, the NFS client should expect that pseudo file system filehandles are volatile 😭 You can confirm this by checking the associated "fh_expire_type" attribute for those filehandles in question. If the filehandles are volatile, the NFS client must be prepared to recover a filehandle value (e.g. with a multi-component LOOKUP) when it receives an error of NFS4ERR_FHEXPIRED.
7.6. Exported Root 🌸
If the server's root file system is exported, you might conclude that a pseudo-file system isn't needed. That would be wrong 🙈 Assume the following file systems on a server:
/ disk1 (exported)
/a disk2 (not exported)
/a/b disk3 (exported)Because disk2 is not exported, disk3 can't be reached with simple LOOKUPs. The server must bridge the gap with a pseudo-file system. Main character energy, no cap.
7.7. Mount Point Crossing 🧚
The server file system environment can be built so that one file system contains a directory which is 'covered' or mounted upon by a second file system. For example:
/a/b (file system 1)
/a/b/c/d (file system 2)The pseudo file system for this server may be built to look like:
/ (place holder/not exported)
/a/b (file system 1)
/a/b/c/d (file system 2)It's the server's responsibility to present the pseudo file system that is complete to the client. If the client sends a lookup request for the path "/a/b/c/d", the server's response is the filehandle of the file system "/a/b/c/d". In previous versions of the NFS protocol, the server would respond with the directory "/a/b/c/d" within the file system "/a/b". Not the vibe we want anymore 💀
The NFS client will be able to tell if it crosses a server mount point by a change in the value of the "fsid" attribute ✨
7.8. Security Policy and Name Space Presentation 🔐
The implementor has to think carefully about applying the server's security policy 🧐 One may choose to limit the viewability of portions of the pseudo file system based on the server's perception of whether the client can authenticate itself properly. However, with support for multiple security mechanisms and the ability to negotiate their appropriate use, the server is unable to properly determine if a client will be able to authenticate itself. If, based on its policies, the server chooses to limit the contents of the pseudo file system, the server may effectively hide file systems from a client that may otherwise have legitimate access.
8. File Locking and Share Reservations 🔒
Integrating locking into the NFS protocol necessarily causes it to be state-full. With the inclusion of "share" file locks the protocol becomes substantially more dependent on state than the traditional combination of NFS and NLM [XNFS]. Three components make this state manageable:
- Clear division between client and server
- Ability to reliably detect inconsistency in state between client and server
- Simple and robust recovery mechanisms
In this model, the server owns the state information. The client communicates its view of this state to the server as needed. The client is also able to detect inconsistent state before modifying a file 🫶
To support Win32 "share" locks it is necessary to atomically OPEN or CREATE files. Having a separate share/unshare operation would not allow correct implementation of the Win32 OpenFile API. To correctly implement share semantics, the previous NFS protocol mechanisms used when a file is opened or created (LOOKUP, CREATE, ACCESS) need to be replaced. The NFS version 4 protocol has an OPEN operation that subsumes the functionality of LOOKUP, CREATE, and ACCESS. However, because many operations require a filehandle, the traditional LOOKUP is preserved to map a file name to filehandle without establishing state on the server. The policy of granting access or modifying files is managed by the server based on the client's state. These mechanisms can implement policy ranging from advisory only locking to full mandatory locking. Very organised, very iconic 👑
8.1. Locking 💜
It is assumed that manipulating a lock is rare compared to READ and WRITE operations. It is also assumed that crashes and network partitions are relatively rare. So it's important that the READ and WRITE operations have a lightweight mechanism to indicate if they possess a held lock. A lock request contains the heavyweight information required to establish a lock and uniquely define the lock owner.
The following sections describe the transition from the heavy weight information to the eventual stateid used for most client and server locking and lease interactions.
8.1.1. Client ID 👑
For each LOCK request, the client must identify itself to the server.
This is done in a way that allows for correct lock identification and crash recovery. Client identification is accomplished with two values:
- A verifier that is used to detect client reboots.
- A variable length opaque array to uniquely define a client.
- For an operating system this may be a fully qualified host name or IP address. For a user level NFS client it may additionally contain a process id or other unique sequence.
The data structure for the Client ID would then appear as:
struct nfs_client_id {
opaque verifier[4];
opaque id<>;
}It is possible, through the mis-configuration of a client or the existence of a rogue client, that two clients end up using the same nfs_client_id 🙈 This situation is avoided by "negotiating" the nfs_client_id between client and server with the use of the SETCLIENTID and SETCLIENTID_CONFIRM operations. Here's how the two scenarios of negotiation go.
- Client has never connected to the server
In this case the client generates an nfs_client_id and unless another client has the same nfs_client_id.id field, the server accepts the request. The server also records the principal (or principal to uid mapping) from the credential in the RPC request that contains the nfs_client_id negotiation request (SETCLIENTID operation).
Two clients might still use the same nfs_client_id.id due to perhaps configuration error. For example, a High Availability configuration where the nfs_client_id.id is derived from the ethernet controller address and both systems have the same address. In this case, the result is a switched union that returns, in addition to NFS4ERR_CLID_INUSE, the network address (the rpcbind netid and universal address) of the client that is using the id.
- Client is re-connecting to the server after a client reboot
In this case, the client still generates an nfs_client_id but the nfs_client_id.id field will be the same as the nfs_client_id.id generated prior to reboot. If the server finds that the principal/uid is equal to the previously "registered" nfs_client_id.id, then locks associated with the old nfs_client_id are immediately released. If the principal/uid is not equal, then this is a rogue client and the request is returned in error. For more discussion of crash recovery semantics, see the section on "Crash Recovery".
It is possible for a retransmission of request to be received by the server after the server has acted upon and responded to the original client request. So to mitigate the effects of a retransmission of the SETCLIENTID operation, the client and server use a confirmation step. The server returns a confirmation verifier that the client then sends to the server in the SETCLIENTID_CONFIRM operation. Once the server receives the confirmation from the client, the locking state for the client is released.
In both cases, upon success, NFS4_OK is returned. To help reduce the amount of data transferred on OPEN and LOCK, the server will also return a unique 64-bit clientid value that is a shorthand reference to the nfs_client_id values presented by the client. From this point forward, the client will use the clientid to refer to itself.
The clientid assigned by the server should be chosen so that it will not conflict with a clientid previously assigned by the server. This applies across server restarts or reboots. When a clientid is presented to a server and that clientid is not recognized, as would happen after a server reboot, the server will reject the request with the error NFS4ERR_STALE_CLIENTID. When this happens, the client must obtain a new clientid by use of the SETCLIENTID operation and then proceed to any other necessary recovery for the server reboot case (See the section "Server Failure and Recovery").
The client must also employ the SETCLIENTID operation when it receives a NFS4ERR_STALE_STATEID error using a stateid derived from its current clientid, since this also indicates a server reboot which has invalidated the existing clientid (see the next section "nfs_lockowner and stateid Definition" for details).
8.1.2. Server Release of Clientid ✨
okay so if the server figures out that a client has no state attached to its clientid, the server may choose to release that clientid 🧹. The reason is so inactive clients aren't hogging resources just because they pop in and out now and then. If that client comes back after the release, the server must make sure it gets the appropriate error so it knows to run the SETCLIENTID/SETCLIENTID_CONFIRM sequence and get a new identity 💖. It should be clear that the server must be very hesitant to release a clientid, bestie 😭, because the client would have to do the same recovery work it would if the server had crashed and restarted. Typically, a server wouldn't release a clientid unless there had been no activity from that client for many minutes ⏰.
8.1.3. nfs_lockowner and stateid Definition 🔐
when a client requests a lock, it must present the server with a clientid and an identifier for the owner of that lock. Those two fields together are called the nfs_lockowner, and here's what they are:
- A clientid returned by the server as part of the client's use of the SETCLIENTID operation.
- A variable length opaque array used to uniquely define the owner of a lock managed by the client.
This may be a thread id, process id, or other unique value.
When the server grants the lock, it responds with a unique 64-bit stateid 🦋. The stateid is basically a shorthand for the nfs_lockowner, since the server keeps track of the connection between the two.
The server is free to form the stateid in any way it likes, as long as it can recognize invalid and out-of-date stateids 🧩. This requirement includes stateids generated by earlier instances of the server. That's how the client gets properly notified of a server restart: it'll happen when the client presents a stateid from a previous instantiation.
The server must be able to tell these situations apart and return the matching error:
- The stateid was generated by an earlier server instance (i.e. before a server reboot). The error
NFS4ERR_STALE_STATEIDshould be returned. - The stateid was generated by the current server instance, but the stateid no longer designates the current locking state for the lockowner-file pair in question (i.e. one or more locking operations has occurred). The error
NFS4ERR_OLD_STATEIDshould be returned.
This error condition will only occur when the client issues a locking request which changes a stateid while an I/O request that uses that stateid is outstanding.
- The stateid was generated by the current server instance, but the stateid does not designate a locking state for any active lockowner-file pair. The error
NFS4ERR_BAD_STATEIDshould be returned.
This error condition will occur when there has been a logic error on the part of the client or server. This should not happen.
One mechanism that may be used to meet these requirements is for the server to split the stateid into three fields:
- A server verifier which uniquely designates a particular server instantiation.
- An index into a table of locking-state structures.
- A sequence value which is incremented for each stateid that is associated with the same index into the locking-state table.
By matching the incoming stateid and its field values against the state held at the server, the server can easily figure out whether a stateid is valid for its current instantiation and state 🎯. If it isn't valid, the server can hand the client the appropriate error.
8.1.4. Use of the stateid 🧁
every READ and WRITE operation contains a stateid 💅. If the nfs_lockowner performs a READ or WRITE on a range of bytes within a locked range, the stateid (previously returned by the server) must be used to show that the appropriate lock (record or share) is held. If the client hasn't established any state, either a record lock or a share lock, a stateid of all bits 0 is used. If there are no conflicting locks held on the file, the server may service the READ or WRITE operation. If a conflict with an explicit lock happens, an error is returned for the operation (NFS4ERR_LOCKED). This is what allows "mandatory locking" to be implemented 🔥.
A stateid of all bits 1 (one) lets READ operations bypass record locking checks at the server. WRITE operations with a stateid of all bits 1 (one), however, do not bypass record locking checks 🙅♀️. File locking checks are handled by the OPEN operation (see the section "OPEN/CLOSE Operations").
An explicit lock may not be granted while a READ or WRITE operation with conflicting implicit locking is being performed.
8.1.5. Sequencing of Lock Requests 🔢
locking is different from most NFS operations because it needs "at-most-one" semantics, and ONCRPC doesn't provide those 🙃. ONCRPC over a reliable transport isn't enough, because a sequence of locking requests may span multiple TCP connections. When there's retransmission or reordering going on, lock or unlock requests must have well defined and consistent behavior. To make that happen, each lock request contains a sequence number that is a consecutively increasing integer. Different nfs_lockowners have different sequences. The server keeps track of the last sequence number (L) it received and the response it returned.
Note that for requests that contain a sequence number, there should be no more than one outstanding request per nfs_lockowner 📬.
If a request with a previous sequence number (r < L) comes in, it gets rejected with the error NFS4ERR_BAD_SEQID. Given a properly-functioning client, the response to (r) must have been received before the last request (L) was sent. If a duplicate of the last request (r == L) comes in, the stored response is returned. If a request beyond the next sequence (r == L + 2) comes in, it gets rejected with the error NFS4ERR_BAD_SEQID. Sequence history gets reinitialized whenever the client verifier changes 🔄.
Since the sequence number is an unsigned 32-bit integer, the arithmetic on it is mod 2^32 🧮.
It is critical that the server keeps the last response it sent to the client, because that gives a more reliable cache of duplicate non-idempotent requests than the traditional cache described in [Juszczak] 🛡️. The traditional duplicate request cache uses a least recently used algorithm to remove unneeded requests. However, the last lock request and response on a given nfs_lockowner must be cached for as long as the lock state exists on the server.
8.1.6. Recovery from Replayed Requests 🔁
as described above, the sequence number is per nfs_lockowner. As long as the server keeps the last sequence number it received and follows the methods described above, there's no risk of a Byzantine router re-sending old requests 🙌. The server only needs to maintain the nfs_lockowner and sequence number state for as long as there are open files, or closed files with locks outstanding.
LOCK, LOCKU, OPEN, OPEN_DOWNGRADE, and CLOSE each contain a sequence number, so the risk of replaying these operations causing undesired effects is non-existent while the server maintains the nfs_lockowner state 🫶.
8.1.7. Releasing nfs_lockowner State 🗑️
when a particular nfs_lockowner no longer holds open or file locking state at the server, the server may choose to release the sequence number state associated with that nfs_lockowner. It may make this choice based on lease expiration, for reclaiming server memory, or other implementation specific details. Either way, the server can only do this safely when the nfs_lockowner is no longer being used by the client. The server may choose to hold the nfs_lockowner state in case retransmitted requests come in, but how long to hold it is implementation specific ⏳.
If a LOCK, LOCKU, OPEN_DOWNGRADE, or CLOSE gets retransmitted after the server has already released the nfs_lockowner state, the server will find that the nfs_lockowner has no files open, and an error will be returned to the client. If the nfs_lockowner does have a file open, the stateid won't match, and again an error is returned to the client 🚫.
If an OPEN gets retransmitted and the nfs_lockowner is being used for the first time, or its state was previously released by the server, the OPEN_CONFIRM operation prevents incorrect behavior 👑. When the server sees an nfs_lockowner being used for the first time, it directs the client to perform OPEN_CONFIRM for the corresponding OPEN. This sequence establishes the use of an nfs_lockowner and its associated sequence number. See the section "OPEN_CONFIRM - Confirm Open" for further details.
8.2. Lock Ranges 📏
the protocol lets a lock owner request a lock with one byte range, and then either upgrade or unlock a sub-range of that initial lock. This is expected to be an uncommon kind of request 🤏. Either way, servers or server file systems may not be able to support sub-range lock semantics. If a server gets a locking request that represents a sub-range of the current locking state for the lock owner, the server is allowed to return the error NFS4ERR_LOCK_RANGE to signal that it doesn't support sub-range lock operations 🙅♀️. So the client should be prepared to receive this error and, if appropriate, report it to the requesting application.
The client is discouraged from combining multiple independent locking ranges that happen to be adjacent into a single request 🚫🧩. The reasons are that the server may not support sub-range requests, and that the recovery of file locking state after a server failure works better this way. As discussed in the section "Server Failure and Recovery" below, the server may use certain optimizations during recovery that only work effectively when the client's behavior during lock recovery is similar to its locking behavior prior to server failure 🧠.
8.3. Blocking Locks 🔐
Okay so some clients need blocking locks 🥺 The NFS version 4 protocol must not rely on a callback mechanism, which means the server has no way to tell a client "hey, your denied lock just got granted!" 📵 So clients have no choice but to keep polling for the lock over and over, and that creates a fairness problem 😤 To fix it, two new lock types get added, READW and WRITEW, which tell the server "this client wants a blocking lock, please" 🎀 The server should keep an ordered list of pending blocking locks 📋 When the conflicting lock gets released, the server may wait out the lease period so the first waiting client can re-request the lock ⏳ Once that lease period expires, the next waiting client's request is allowed to take the lock 💅 Clients are required to poll at an interval small enough that they're likely to get the lock in a timely manner 🏃♀️ The server is not required to keep a list of pending blocked locks, since that list exists for fairness and not for correct operation 🫶 Because crash recovery is unordered, you'd need to store lock state in stable storage to guarantee blocking locks get granted in order 💾
Servers may also note the lock types and delay sending back the denial of the request, giving a conflicting lock extra time to be released so the request can succeed 🦋 That way clients don't have to burn energy polling super often 💜 The server should be careful about how long that delay is, in case the client retransmits the request 🙈
8.4. Lease Renewal 🔁
The whole point of a lease is to let the server clean up stale locks held by a client that crashed or is otherwise unreachable 🧹 It is NOT a cache consistency mechanism 🙅 and lease renewals may not be denied if the lease interval hasn't expired yet.
The following events cause implicit renewal of all of the leases for a given client (i.e. all those sharing a given clientid). Each one is a positive sign that the client is still active and that the state the server holds for that client is still valid ✨
- An
OPENwith a valid clientid. - Any operation made with a valid stateid (
CLOSE,DELEGRETURN,LOCK,LOCKU,OPEN,OPEN_CONFIRM,READ,RENEW,SETATTR,WRITE). This does not include the special stateids of all bits 0 or all bits 1.
Note that if the client had restarted or rebooted, it would not be making these requests without issuing theSETCLIENTIDoperation 👀 UsingSETCLIENTID(possibly with the addition of the optionalSETCLIENTID_CONFIRMoperation) tells the server to drop the locking state associated with the client 🧼 If the server has rebooted, the stateids (NFS4ERR_STALE_STATEIDerror) or the clientid (NFS4ERR_STALE_CLIENTIDerror) won't be valid, which prevents spurious renewals 🛡️
This approach is low overhead and scales really well 📈 In the typical case no extra RPC calls are required for lease renewal, and in the worst case one RPC is required every lease period (i.e. a RENEW operation). The number of locks the client holds isn't a factor, since all of the client's state gets involved in the lease renewal action 🙌
Since all operations that create a new lease also renew existing leases, the server must maintain a common lease expiration time for all valid leases for a given client. That lease time can then be easily updated upon implicit lease renewal actions 🕰️
8.5. Crash Recovery 💥
The big requirement in crash recovery is that both the client and the server know when the other has failed 🫣 On top of that, a client is required to see a consistent view of data across server restarts or reboots. All READ and WRITE operations that may have been queued within the client or network buffers must wait until the client has successfully recovered the locks protecting those READ and WRITE operations 🧘♀️
8.5.1. Client Failure and Recovery 🧍♀️
If a client fails, the server may recover the client's locks once the associated leases have expired. Conflicting locks from another client may only be granted after that lease expiration ⌛ If the client is able to restart or reinitialize within the lease period, it may be forced to wait out the rest of the lease period before it can get new locks 😩
To minimize client delay upon restart, lock requests are associated with an instance of the client by a client supplied verifier 🪪 This verifier is part of the initial SETCLIENTID call the client makes. The server returns a clientid as the result of the SETCLIENTID operation, and then the client confirms the verifier with SETCLIENTID_CONFIRM. The clientid, combined with an opaque owner field, is then used by the client to identify the lock owner for OPEN. This chain of associations is what identifies all the locks for a particular client 🔗
The verifier gets changed by the client every time it initializes, so the server can compare a new verifier to the verifier associated with currently held locks and see that they don't match 🚩 That signals the client's new instantiation and the loss of its locking state. As a result, the server is free to release all locks held that are associated with the old clientid, which was derived from the old verifier 🗑️
For secure environments, a change in the verifier must only cause the release of locks associated with the authenticated requester. This is required to stop a rogue entity from freeing otherwise valid locks 🚫😈
Note that the verifier must have the same uniqueness properties as the verifier for the COMMIT operation 💜
8.5.2. Server Failure and Recovery 🌧️
If the server loses locking state (usually because of a restart or reboot), it must allow clients time to discover this and re-establish the lost locking state ⏰ The client must be able to re-establish its locking state without the server denying valid requests because it has granted conflicting access to another client 🙏 Likewise, if there's a possibility that clients haven't yet re-established their locking state for a file, the server must disallow READ and WRITE operations for that file. The length of this recovery period is equal to the length of the lease period 🕐
A client can tell that server failure (and therefore loss of locking state) has happened when it gets one of two errors 👀 The NFS4ERR_STALE_STATEID error means a stateid was invalidated by a reboot or restart. The NFS4ERR_STALE_CLIENTID error means a clientid was invalidated by a reboot or restart. When either of these comes back, the client must establish a new clientid (see the section "Client ID") and re-establish the locking state as discussed below 🔄
The period of special handling for locking, READs and WRITEs, which is equal in length to the lease period, is referred to as the "grace period" 🌷 During the grace period, clients recover locks and their associated state with reclaim-type locking requests (i.e. LOCK requests with reclaim set to true, and OPEN operations with a claim type of CLAIM_PREVIOUS). During the grace period, the server must reject READ and WRITE operations, and non-reclaim locking requests (i.e. other LOCK and OPEN operations), with an error of NFS4ERR_GRACE 🚧
If the server can reliably determine that granting a non-reclaim request won't conflict with other clients reclaiming their locks, the NFS4ERR_GRACE error doesn't have to be returned, and the non-reclaim request can be serviced 🧠 For the server to service READ and WRITE operations during the grace period, it must again be able to guarantee that no possible conflict could come up between an impending reclaim locking request and that READ or WRITE operation. If the server can't offer that guarantee, the NFS4ERR_GRACE error must be returned to the client 🙅♀️
For a server to handle the grace period in a simple, valid way, the easiest method is to reject all non-reclaim locking requests and all READ and WRITE operations by returning the NFS4ERR_GRACE error 🍰 However, a server may keep information about granted locks in stable storage 💾 With that information, the server could figure out whether a regular lock or a READ or WRITE operation can be safely processed ✅
For example, if a count of locks on a given file is available in stable storage, the server can track reclaimed locks for the file, and once all the reclaims have been processed, non-reclaim locking requests may be processed 🧮 That way the server can make sure non-reclaim locking requests won't conflict with potential reclaim requests 🤝 As for I/O requests, if the server can determine that there are no outstanding reclaim requests for a file, using information from stable storage or another similar mechanism, then processing I/O requests can proceed normally for that file 📂
To reiterate, a server that allows non-reclaim lock and I/O requests to be processed during the grace period MUST determine that no lock subsequently reclaimed will be rejected, and that no lock subsequently reclaimed would have prevented any I/O operation processed during the grace period 🚨 That's a big deal, bestie!
Clients should be prepared for NFS4ERR_GRACE errors on non-reclaim lock and I/O requests 🥹 In that case the client should use a retry mechanism for the request. A delay (on the order of several seconds) between retries should be used so the server doesn't get overwhelmed 🐢 There's more on the general topic over in [Floyd]. The client must account for servers that can perform I/O and non-reclaim locking requests within the grace period, as well as servers that can not do so 🧩
A reclaim-type locking request made outside the server's grace period can only succeed if the server can guarantee that no conflicting lock or I/O request has been granted since the reboot or restart 💖
8.5.3. Network Partitions and Recovery ✨
okay bestie, so if a network partition lasts longer than the lease period the server gave you, the server won't have gotten a lease renewal from the client 😭. When that happens, the server may free all locks it's holding for the client 🔒. That means every stateid the client has is now invalid or stale 💀. Once the client can reach the server again, any I/O sent with those dead stateids will fail, and the server will reply with the error NFS4ERR_EXPIRED. The moment you get that error, you need to let the application that held the lock know 🫶.
As a courtesy, or just to be efficient, the server may keep holding locks for a client whose recent communication went past the lease period 🌸. But if the server gets a lock or I/O request that conflicts with one of these courtesy locks, it must free the courtesy lock and grant the new request 💅.
If the server keeps holding locks after a client's lease has expired, the server MUST record that fact in its stable storage 📝. Conflicting lock requests from another client might get served after the lease expires, so this matters. Server failure after an event like this can cause trouble if the lease expirations or network partitions aren't stored, and here's one example scenario 👀:
A client holds a lock at the server and encounters a network
partition and is unable to renew the associated lease. A
second client obtains a conflicting lock and then frees the
lock. After the unlock request by the second client, the
server reboots or reinitializes. Once the server recovers, the
network partition heals and the original client attempts to
reclaim the original lock.Here's the tea ☕. If the server has no state information at all, it will allow the reclaim, and the client ends up in an inconsistent state, because neither side knows about that conflicting lock. Not great, not me 🙈.
The server can store this lease expiration or network partitioning state in a way that only identifies the client as a whole 🧚. The downside is that lock reclaims might get denied when they didn't need to be, because the client has a mix of conflicting and non-conflicting locks. Another option is for the server to store information about each lock with an expired lease that has an associated conflicting lock 🦋. How much state gets stored, and what kind, is up to the implementor 💜. Whatever they pick, though, the server must have enough state information to correctly recover from multiple partitions and multiple server failures 👑.
8.6. Recovery from a Lock Request Timeout or Abort
Sometimes a lock request times out, and the client might decide not to retry it 🌷. A client can also abort the request when the process it was made for gets terminated (for example, in UNIX, because of a signal) 💀. Here's the catch though: the server may have actually received the request and acted on it 😱. That would change the server's state without the client knowing. So it is absolutely paramount that the client re-synchronizes state with the server before it tries any other operation that takes a seqid and/or a stateid with the same nfs_lockowner 🚨. The good news is that this is straightforward to do, and there's no special re-synchronize operation needed ✨.
Here's how it works 💖. The server keeps track of the last lock request and response it received on each nfs_lockowner. So the client should cache the last lock request it sent that didn't get a response. The next time the client does a lock operation for that nfs_lockowner, it can resend the cached request, if there is one. And if that request was one that established state (like a LOCK or OPEN operation), the client can follow up with a request that removes the state (like a LOCKU or CLOSE operation) 🧹. Doing this lets the sequencing and stateid information on the client and server for that nfs_lockowner line back up again, and the lock state re-synchronizes with it 🎀.
8.7. Server Revocation of Locks
Honestly, the server can revoke locks held by a client at any point, so the client has to be ready for it 😤. When a client detects that its locks have been revoked, or might have been, it's on the client to validate the state information between itself and the server 🫶. Validating locking state means the client must verify or reclaim the state for each lock it currently holds ✅.
The first way this happens is when the server reboots or reinitializes 🔄. The client will get an error (NFS4ERR_STALE_STATEID or NFS4ERR_STALE_CLIENTID), and then it goes through normal crash recovery, which we covered in the previous section 💅.
The second way is when the lease period can't be renewed 😬. This is considered rare or unusual, but the client still needs to be prepared to recover 💪. The good part is that both the server and the client can detect the renewal failure, and they can both recover without data corruption 🌈. The server tracks the last renewal event it serviced for each client, so it knows when the lease will expire ⏰. The client has to track the operations that will renew the lease too. By using the time each such request was sent and the time its reply came back, the client should work out a bound on when the renewal could have happened on the server 🧠. That tells it whether a lease expiration could have occurred ✨.
The third way can happen within the lease period because of administrative intervention 🧑💼. It's rare, but the server's administrator might decide to release or revoke one particular lock the client holds 🔑. When that happens, the client gets an error of NFS4ERR_EXPIRED, and it arrives within the lease period for the lock. In this case, the client can assume that only the locks belonging to that nfs_lockowner were lost 🥺. The client should let the lock holder know 📣. But the client may not assume the lease period was renewed just because a failed operation came back 🙅♀️.
When the client decides the lease period may have expired, it must mark all locks held under the associated lease as "unvalidated" 🏷️. That means the client hasn't been able to re-establish or confirm the right lock state with the server. Like we talked about in the crash recovery section, the server might grant conflicting locks after a client's lease has expired 😳. So whenever the lease might have expired, the client must validate each lock it holds to make sure no conflicting lock was granted. One way to do that is to issue an I/O request, either a pending I/O or a zero-length read, and specify the stateid associated with the lock in question 🔍. If the response is a success, the client has validated all the locks governed by that stateid and re-established the right state with the server 🎉. If the I/O request fails, then one or more of the locks tied to that stateid were revoked by the server, and the client must notify the owner 📢.
8.8. Share Reservations
A share reservation is a way to control access to a file 🚪. It's separate from record locking and it works independently of it 🦋. When a client opens a file, it sends an OPEN operation to the server. That specifies the access it needs (READ, WRITE, or BOTH) and the access it wants to deny to everyone else (deny NONE, READ, WRITE, or BOTH) 💜. If the OPEN fails, the client fails the application's open request too 😢.
Here's the pseudo-code for how the semantics work 🧑💻:
if ((request.access & file_state.deny)) ||
(request.deny & file_state.access))
return (NFS4ERR_DENIED)These are the constants used for the access and deny fields in the OPEN and OPEN_DOWNGRADE operations 🔢:
const OPEN4_SHARE_ACCESS_READ = 0x00000001;
const OPEN4_SHARE_ACCESS_WRITE = 0x00000002;
const OPEN4_SHARE_ACCESS_BOTH = 0x00000003;
const OPEN4_SHARE_DENY_NONE = 0x00000000;
const OPEN4_SHARE_DENY_READ = 0x00000001;
const OPEN4_SHARE_DENY_WRITE = 0x00000002;
const OPEN4_SHARE_DENY_BOTH = 0x00000003;8.9. OPEN/CLOSE Operations
To get correct share semantics, a client MUST use the OPEN operation to obtain the initial filehandle, and say what access it wants and whether it denies any access to others 🎀. Even if the client plans to use a stateid of all 0's or all 1's, it still has to get the filehandle for the regular file with OPEN, so the right share semantics can be applied 💅. Clients whose open programming interfaces don't have a deny mode built in should use deny equal to NONE 🌸.
The OPEN operation with the CREATE flag also takes over the CREATE operation for regular files that earlier versions of the NFS protocol used 🌟. That means a create with a share can be done atomically ⚛️.
The CLOSE operation removes all share locks that the nfs_lockowner holds on that file 🧹. If record locks are still held, the client SHOULD release all of them before issuing a CLOSE 🔓. The server MAY free all outstanding locks on CLOSE, but some servers might not support closing a file that still has record locks held 🤷♀️. If any locks would exist after the CLOSE, the server MUST return failure 🚫.
The LOOKUP operation returns a filehandle without setting up any lock state on the server 👀. Without a valid stateid, the server assumes the client has the least access. For example, a file opened with deny READ/WRITE can't be accessed through a filehandle obtained from LOOKUP, because that filehandle wouldn't have a valid stateid (that is, a stateid of all bits 0 or all bits 1) 🙈💖
8.10. Open Upgrade and Downgrade 🦋
Okay bestie, say a lockowner already has a file open and then does ANOTHER OPEN on that same file. The server doesn't make a second open. It upgrades the open file status it's tracking so it includes the access and deny bits from the new OPEN as well as the ones from the existing open 💅. As far as the protocol is concerned there's still just ONE open file, and it holds the union of the access and deny bits from every OPEN that completed. So a single CLOSE is enough to reset the effects of both OPENs ✨. Also, the client might not even know it's opening the same file again when it sends the OPEN 🙈. This only applies if both OPENs result in the OPEN'ed object being designated by the same filehandle.
Now the plot twist 🌈 If the server exports multiple filehandles for the same file object and hands back different filehandles on two different OPENs of that file, the server MUST NOT "OR" the access and deny bits together and coalesce the two open files. Nope! It has to keep separate OPENs with separate stateid's, and each one will need its own CLOSE to free it 🚫.
When multiple open files on the client get merged into a single open file object on the server, closing one of them (on the client) can mean the server's access and deny status needs to change 📉. The union of bits for the remaining opens might now be a smaller proper subset than before. That's where OPEN_DOWNGRADE comes in: the client should use it to update the server so that other clients' share reservation requests get handled properly 🫶.
8.11. Short and Long Leases ⏳
Picking a lease time is all about tradeoffs 🍓 Short leases are great for fast server recovery, but they cost you more RENEW or READ (with zero length) requests. Longer leases are kinder and gentler to big internet servers juggling huge numbers of clients 🥹 The number of RENEW requests drops in proportion to the lease time.
The downsides of long leases are slower recovery after a server failure (the server has to wait for leases to expire and the grace period to pass before granting new lock requests) and more file contention (if a client fails to send an unlock request, the server has to wait for lease expiration before granting new locks) 💀.
Long leases are usable if the server can store lease state in non-volatile memory 💾 On recovery, it can rebuild the lease state from there and keep going with its clients, so long leases stop being an issue 🙌.
8.12. Clocks and Calculating Lease Expiration 🕰️
To avoid needing synchronized clocks, the server hands out lease times as a time delta ⏱️ BUT there's still a requirement that the client and server clocks don't drift excessively over the duration of the lock. There's also propagation delay across the network, which could easily be several hundred milliseconds 😭 plus the possibility that requests get lost and need retransmitting.
To account for propagation delay, the client should subtract it from lease times. For example, if the client estimates the one-way propagation delay as 200 msec, it can assume the lease is already 200 msec old when it gets it. On top of that, it'll take another 200 msec to get a response back to the server. So the client must send a lock renewal or write data back to the server 400 msec before the lease would expire ⏰.
8.13. Migration, Replication and State 🧳
When responsibility for a file system moves to a new server (migration), or the client picks an alternate server (e.g. because the server is unresponsive) in the context of file system replication, how shared state gets handled depends on the situation. We're talking locks, leases, stateid's, and clientid's 🔐 Migration and replication treat it differently. For more on file server state and recovery, check the sections under "File Locking and Share Reservations" 📚.
8.13.1. Migration and State 🚚
In the case of migration, the servers involved SHOULD transfer all server state from the original server to the new one ✨ This has to be transparent to the client, and it eases the client's transition when a file system migration happens. If the servers successfully transfer all state, the client keeps using stateid's assigned by the original server, so the new server must recognize those stateid's as valid. The same goes for the clientid 🫶 Since responsibility for an entire file system moves with a migration, there's no chance of lock conflicts showing up on the new server because of the transfer.
Leases get transferred too 🎀 Transferred leases will typically have a different expiration time from the ones the same client already has on the new server. To keep the property that all leases on a given server for a given client expire at the same time, the server should advance the expiration time to the later of the transferred leases or the leases already present. That way the client can keep renewing both classes without any special effort.
Servers may choose not to transfer state on migration, but that's discouraged 🙅 In that case, when the client presents state information from the original server, it must be prepared to receive either NFS4ERR_STALE_CLIENTID or NFS4ERR_STALE_STATEID from the new server. The client should then recover its state like it normally would after a server failure. The new server must take care to allow for recovery of state information as it would in the event of a server restart 🔄.
8.13.2. Replication and State 🪞
Since client switch-over in replication isn't under server control, state handling is different 🙃 Leases, stateid's, and clientid's do not have validity across a transition from one server to another. The client must re-establish its locks on the new server. This is kinda like re-establishing locks with reclaim-type requests after a server reboot. The difference is that the server has no way to tell reclaim requests apart from requests for new locks, or to defer the latter. So a client re-establishing a lock on the new server (by means of a LOCK or OPEN request) might get denied because of a conflicting lock 😬 Since replication is intended for read-only use of filesystems, that denial shouldn't pose large difficulties in practice 🤷♀️ When an attempt to re-establish a lock on a new server is denied, the client should treat the situation as if their original lock had been revoked.
8.13.3. Notification of Migrated Lease 📬
For lease renewal, the client might not be submitting requests for a file system that has been migrated to another server. This can happen because of the implicit lease renewal mechanism, since the client renews leases for ALL file systems whenever it submits a request to any one file system at the server 🌀
So the client can schedule renewal of leases that may have moved to the new server, it has to find out about the lease relocation before those leases expire ⏳ To make that happen, every operation that implicitly renews a client's leases (OPEN, CLOSE, READ, WRITE, RENEW, LOCK, LOCKT, LOCKU) will return NFS4ERR_LEASE_MOVED if responsibility for any of the leases to be renewed has been transferred to a new server. That condition continues until the client receives an NFS4ERR_MOVED error and the server receives the subsequent GETATTR(fs_locations) for access to each file system for which a lease has been moved to a new server 🧾
When a client receives an NFS4ERR_LEASE_MOVED error, it should perform some operation, such as a RENEW, on each file system associated with the server in question 🔁 When the client receives an NFS4ERR_MOVED error, it can follow the normal process to get the new server information (through the fs_locations attribute) and renew those leases on the new server 🗺️ If the server hasn't had state transferred to it transparently, it will receive either NFS4ERR_STALE_CLIENTID or NFS4ERR_STALE_STATEID from the new server, as described above, and can then recover state information the same way it would after a server failure 💜
9. Client-Side Caching 🌸
Client-side caching of data, file attributes, and file names is essential for good performance with NFS 🚀 Providing distributed cache coherence is a genuinely hard problem, and previous NFS versions didn't try. Instead, several client implementation techniques were used to reduce the problems a lack of coherence causes for users. Those techniques weren't clearly defined by earlier protocol specs, so it was often unclear what valid or invalid client behavior actually was 🤔
NFS version 4 uses many techniques similar to the ones in previous versions. It does not provide distributed cache coherence. However, it defines a more limited set of caching guarantees so that locks and share reservations can be used without destructive interference from client-side caching 🔒
NFS version 4 also introduces a delegation mechanism that lets many decisions normally made by the server be made locally by clients 🎁 This gives efficient support for the common cases where sharing is infrequent or where sharing is read-only 🫶
9.1. Performance Challenges for Client-Side Caching 💅
okay so, caching in the earlier NFS versions was seriously successful and gave really good performance 🌸. BUT, when you've got a huge number of clients, some scalability challenges start popping up 😭. This is especially true when clients are spread out geographically, because that classically makes the latency for cache revalidation requests go way up 🌍.
The previous versions of the NFS protocol repeat their file data cache validation requests at the time the file is opened. This can have serious performance drawbacks 💀. A super common case is when a file is only accessed by a single client, so sharing is infrequent 🦋.
In that case, asking the server again and again to find out that no conflicts exist is expensive 💸. A better option for performance is letting a client that keeps opening the same file do it without checking with the server. It can do that until potentially conflicting operations from another client actually happen 🧚.
A similar situation shows up with file locking ✨. Sending file lock and unlock requests to the server, plus the read and write requests needed to keep data caching consistent with the locking semantics (see the section "Data Caching and File Locking"), can seriously limit performance 📉. When locking is used to protect against infrequent conflicts, the penalty is huge 😤. That penalty might discourage applications from using file locking at all 🙈.
The NFS version 4 protocol gives us more aggressive caching strategies, with these design goals:
- Compatibility with a large range of server semantics. 🌈
- Provide the same caching benefits as previous versions of the NFS protocol when the more aggressive model can't be provided. 🫶
- Requirements for aggressive caching are organized so that a large portion of the benefit can be obtained even when not all of the requirements can be met. 👑
The appropriate requirements for the server are discussed in later sections where specific forms of caching are covered (see the section "Open Delegation") 🎀.
9.2. Delegation and Callbacks 🦋
okay this one is my fave 😭. Recallable delegation of server responsibilities for a file to a client improves performance by avoiding repeated requests to the server when there's no inter-client conflict. With a "callback" RPC going from server to client, the server recalls delegated responsibilities when another client starts sharing a delegated file 📞.
A delegation gets passed from the server to the client, specifying the object of the delegation and the type of delegation. There are different types of delegations, but each type contains a stateid that gets used to represent the delegation when performing operations that depend on it 🔑. This stateid is similar to the ones associated with locks and share reservations, but it's different because the stateid for a delegation is associated with a clientid and can be used on behalf of all the nfs_lockowners for that client 💅. A delegation is made to the client as a whole, not to any specific process or thread of control inside it 🧚.
Callback RPCs may not work in all environments (because of firewalls, for example), so correct protocol operation does not depend on them 🧱. Preliminary testing of callback functionality by means of a CB_NULL procedure determines whether callbacks can be supported 🔍. The CB_NULL procedure checks the continuity of the callback path 🛤️. A server makes a preliminary assessment of callback availability to a given client, and it avoids delegating responsibilities until it has figured out that callbacks are supported 🕵️♀️. Since granting a delegation is always conditional on the absence of conflicting access, clients must not assume a delegation will be granted, and they must always be prepared for OPENs to be processed without any delegations being granted 🫠.
Once granted, a delegation behaves in most ways like a lock 🔒. It has an associated lease that is subject to renewal together with all of the other leases that client holds 📅.
Unlike locks, an operation by a second client on a delegated file will cause the server to recall the delegation through a callback 📣.
On recall, the client holding the delegation must flush modified state (such as modified data) to the server and return the delegation 📦. The conflicting request won't get a response until the recall is complete 🙃. The recall is considered complete when the client returns the delegation, or when the server times out on the recall and revokes the delegation because of the timeout ⏰. Once the recall is resolved, the server has the info it needs to grant or deny the second client's request ✅.
When a client gets a delegation recall, it may have a lot of state that needs flushing to the server 🌊. So the server should allow sufficient time for the delegation to be returned, since it may involve tons of RPCs to the server 🐢. If the server can tell that the client is diligently flushing state to the server because of the recall, the server may extend the usual time allowed for a recall 💪. But the time allowed for recall completion should not be unbounded ⛔.
Here's an example: when responsibility for mediating opens on a given file has been delegated to a client (see the section "Open Delegation"). The server won't know what opens are in effect on the client 🫣. Without that knowledge, the server can't determine whether the access and deny state for the file allows any particular open until the delegation for the file has been returned 🧩.
A client failure or a network partition can result in failure to respond to a recall callback 📵. In that case, the server will revoke the delegation, which in turn renders useless any modified state still sitting on the client 💔.
9.2.1. Delegation Recovery 🌷
there are three situations that delegation recovery has to deal with 👀:
- Client reboot or restart
- Server reboot or restart
- Network partition (full or callback-only)
If the client reboots or restarts, the failure to renew leases will result in the revocation of record locks and share reservations 🔄. Delegations, however, may be treated a bit differently 🤔.
There will be situations where delegations need to be reestablished after a client reboots or restarts 🔁. The reason is that the client may have file data stored locally, and that data was associated with the previously held delegations 📂. The client will need to reestablish the appropriate file state on the server 🛠️.
To allow for this type of client recovery, the server may extend the period for delegation recovery beyond the typical lease expiration period ⏳. That means requests from other clients that conflict with these delegations will need to wait 🚧. Because the normal recall process may take significant time for the client to flush changed state to the server, other clients need be prepared for delays that happen because of a conflicting delegation 🫂. This longer interval increases the window for clients to reboot and consult stable storage so the delegations can be reclaimed 🗄️. For open delegations, these get reclaimed using OPEN with a claim type of CLAIM_DELEGATE_PREV (see the sections on "Data Caching and Revocation" and "Operation 18: OPEN" for discussion of open delegation and the details of OPEN, respectively) 📚.
When the server reboots or restarts, delegations are reclaimed (using the OPEN operation with CLAIM_DELEGATE_PREV) in a similar fashion to record locks and share reservations 🔁. There's a slight semantic difference though 👀. In the normal case, if the server decides a delegation should not be granted, it performs the requested action (e.g. OPEN) without granting any delegation 🙅♀️. For reclaim, the server grants the delegation, but a special designation is applied so the client treats the delegation as having been granted but recalled by the server 🎭. Because of this, the client has the duty to write all modified state to the server and then return the delegation 📤. This process of handling delegation reclaim reconciles three principles of the NFS Version 4 protocol 🧠:
- Upon reclaim, a client reporting resources assigned to it by an earlier server instance must be granted those resources. 💖
- The server has unquestionable authority to determine whether delegations are to be granted and, once granted, whether they are to be continued. 👑
- The use of callbacks is not to be depended upon until the client has proven its ability to receive them. 📞
When a network partition occurs, delegations are subject to being freed by the server when the lease renewal period expires 🌫️. This is similar to the behavior for locks and share reservations 🤝. For delegations, though, the server may extend the period in which conflicting requests are held off ⏸️. Eventually, a conflicting request from another client will cause revocation of the delegation 💥. A loss of the callback path (e.g. from a later network configuration change) will have the same effect 🔌. A recall request will fail, and revocation of the delegation will result ❌.
A client normally finds out about revocation of a delegation when it uses a stateid associated with a delegation and receives the error NFS4ERR_EXPIRED 😱. It can also find out about delegation revocation after a client reboot, when it attempts to reclaim a delegation and gets that same error 🥲. Note that with a revoked write open delegation, there are issues because data may have been modified by the client whose delegation was revoked and separately by other clients 🌀. See the section "Revocation Recovery for Write Open Delegation" for a discussion of those issues 📖. Also note that when delegations are revoked, information about the revoked delegation will be written by the server to stable storage (as described in the section "Crash Recovery") 💾. This is done to deal with the case where a server reboots after revoking a delegation but before the client holding the revoked delegation is notified about the revocation 🛟.
9.3. Data Caching 🧁
When applications share access to a set of files, they need to be implemented so they take account of the possibility of conflicting access by another application 🤝. This is true whether the applications run on different clients or live on the same client 💻.
Share reservations and record locks are the facilities the NFS version 4 protocol provides so applications can coordinate access through mutual exclusion 🔐. The NFS version 4 protocol's data caching must be implemented in a way that does not invalidate the assumptions those facilities depend on 💜.
9.3.1. Data Caching and OPENs 💖
okay bestie, the whole point here is not wrecking the sharing assumptions that apps rely on 🫶. NFS version 4 clients SHOULD NOT hand cached data to an application, or change it on an application's behalf, when that wouldn't be valid to get or modify with a READ or WRITE operation. no cap, that's the vibe ✨
Furthermore, in the absence of open delegation (see the section "Open Delegation") two additional rules apply. Note that these rules are obeyed in practice by many NFS version 2 and version 3 clients 🌸
- First, cached data present on a client MUST be revalidated after doing an
OPEN. This makes sure the data for the OPENed file is still correctly reflected in the client's cache. This validation MUST be done at least when the client'sOPENoperation includesDENY=WRITEorBOTH, thus terminating a period in which other clients may have had the opportunity to open the file withWRITEaccess. Clients MAY choose to do the revalidation more often (i.e. at OPENs specifyingDENY=NONE) to parallel the NFS version 3 protocol's practice, for the benefit of users assuming this degree of cache revalidation 🧚 - Second, modified data MUST be flushed to the server before closing a file OPENed for write. This is complementary to the first rule. If the data is not flushed at
CLOSE, the revalidation done after client OPENs a file is unable to achieve its purpose 😭 The other part of flushing before close is that the data MUST be committed to stable storage, at the server, before theCLOSEoperation is requested by the client. In the case of a server reboot or restart and a CLOSEd file, it may not be possible to retransmit the data to be written to the file. Hence, this requirement 💅
9.3.2. Data Caching and File Locking
For those applications that choose to use file locking instead of share reservations to exclude inconsistent file access, there is an analogous set of constraints that apply to client-side data caching 🦋 These rules only work if the file locking is used in a way that matches, in an equivalent way, the actual READ and WRITE operations executed. That's as opposed to file locking that is based on pure convention. For example, it is possible to manipulate a two-megabyte file by dividing it into two one-megabyte regions and protecting access to the two regions with file locks on bytes zero and one. A lock for write on byte zero of the file would represent the right to do READ and WRITE operations on the first region. A lock for write on byte one of the file would represent the right to do READ and WRITE operations on the second region. As long as all applications manipulating the file obey this convention, they will work on a local file system. However, they may not work with the NFS version 4 protocol unless clients refrain from data caching 🙈
The rules for data caching in the file locking environment are:
- First, when a client obtains a file lock for a particular region, the data cache corresponding to that region (if any cache data exists) MUST be revalidated. If the change attribute indicates that the file may have been updated since the cached data was obtained, the client MUST flush or invalidate the cached data for the newly locked region. A client might choose to invalidate all of the non-modified cached data it has for the file, but the only requirement for correct operation is to invalidate all of the data in the newly locked region 🌈
- Second, before releasing a write lock for a region, all modified data for that region MUST be flushed to the server. The modified data MUST also be written to stable storage ✨
Note that flushing data to the server and invalidating cached data MUST reflect the actual byte ranges locked or unlocked. Rounding these up or down to reflect client cache block boundaries will cause problems if not carefully done. For example, writing a modified block when only half of that block is within an area being unlocked may cause invalid modification to the region outside the unlocked area. That, in turn, may be part of a region locked by another client 😬 Clients can avoid this situation by synchronously performing the portions of write operations that overlap the portion (initial or final) that is not a full block. Similarly, invalidating a locked area that is not an integral number of full buffer blocks would require the client to read one or two partial blocks from the server if the revalidation procedure shows that the data the client possesses may not be valid 🍓
The data that is written to the server as a prerequisite to unlocking a region MUST be written, at the server, to stable storage. The client MAY accomplish this either with synchronous writes or by following asynchronous writes with a COMMIT operation. This is REQUIRED because retransmission of the modified data after a server reboot might conflict with a lock held by another client 🔒
A client implementation MAY choose to accommodate applications that use record locking in non-standard ways (e.g. using a record lock as a global semaphore) by flushing more data to the server upon a LOCKU than is covered by the locked range. This may include modified data within files other than the one for which the unlocks are being done. In such cases, the client MUST NOT interfere with applications whose READs and WRITEs are only being done within the bounds of record locks the application holds. For example, an application locks a single byte of a file and proceeds to write that single byte. A client that chose to handle a LOCKU by flushing all modified data to the server could validly write that single byte in response to an unrelated unlock. However, it would not be valid to write the entire block in which that single written byte was located, since it includes an area that is not locked and might be locked by another client. Client implementations can avoid this problem by dividing files with modified data into those for which all modifications are done to areas covered by an appropriate record lock and those for which there are modifications not covered by a record lock. Any writes done for the former class of files MUST NOT include areas that are not locked and thus not modified on the client 🧩
9.3.3. Data Caching and Mandatory File Locking
Client-side data caching needs to respect mandatory file locking when it is in effect 🛡️ The presence of mandatory file locking for a given file is indicated in the result flags for an OPEN. When mandatory locking is in effect for a file, the client MUST check for an appropriate file lock for the data being read or written. If a lock exists for the range being read or written, the client MAY satisfy the request using the client's validated cache. If an appropriate file lock is not held for the range of the read or write, the read or write request MUST NOT be satisfied by the client's cache, and the request MUST be sent to the server for processing 🚫 When a read or write request partially overlaps a locked region, the request SHOULD be subdivided into multiple pieces, with each region (locked or not) treated appropriately 🎀
9.3.4. Data Caching and File Identity
When clients cache data, the file data needs to be organized according to the file system object the data belongs to. For NFS version 3 clients, the typical practice has been to assume, for caching purposes, that distinct filehandles represent distinct file system objects. The client then has the choice to organize and maintain the data cache on this basis 🗂️
In the NFS version 4 protocol, there is now the possibility of significant deviations from a "one filehandle per object" model, because a filehandle may be constructed on the basis of the object's pathname. Therefore, clients need a reliable method to determine whether two filehandles designate the same file system object. If clients were simply to assume that all distinct filehandles denote distinct objects and proceed to do data caching on this basis, caching inconsistencies would arise between the distinct client-side objects that mapped to the same server-side object 🌀
By providing a method to differentiate filehandles, the NFS version 4 protocol alleviates a potential functional regression compared with the NFS version 3 protocol. Without this method, caching inconsistencies within the same client could occur, and this has not been present in previous versions of the NFS protocol. Note that it is possible to have such inconsistencies with applications executing on multiple clients, but that is not the issue being addressed here 🙃
For the purposes of data caching, the following steps allow an NFS version 4 client to determine whether two distinct filehandles denote the same server-side object:
- If
GETATTRdirected to two filehandles returns different values of thefsidattribute, then the filehandles represent distinct objects. - If
GETATTRfor any file with anfsidthat matches thefsidof the two filehandles in question returns aunique_handlesattribute with a value ofTRUE, then the two objects are distinct. - If
GETATTRdirected to the two filehandles does not return thefileidattribute for one or both of the handles, then the then the it cannot be determined whether the two objects are the same. Therefore, operations that depend on that knowledge (e.g. client-side data caching) cannot be done reliably 🥺 - If
GETATTRdirected to the two filehandles returns different values for thefileidattribute, then they are distinct objects. - Otherwise, they are the same object 💜
9.4. Open Delegation ✨
okay so when a file is getting OPENed, the server is allowed to hand off further handling of opens and closes for that file to the client that's opening it 💅. This is called delegation, and it's totally recallable 🔥, because the circumstances that allowed it are subject to change. Like, if the server gets a conflicting OPEN from another client, it must recall the delegation before deciding whether that other OPEN can be granted 😭. Making a delegation is up to the server, so clients should not assume that any particular OPEN will or will not result in an open delegation 🙈. Here's a typical set of conditions servers might use to decide whether an OPEN should be delegated:
- The client must be able to respond to the server's callback requests. The server will use the
CB_NULLprocedure for a test of callback ability. - The client must have responded properly to previous recalls.
- There must be no current open conflicting with the requested delegation.
- There should be no current delegation that conflicts with the delegation being requested.
- The probability of future conflicting open requests should be low based on the recent history of the file.
- The existence of any server-specific semantics of OPEN/CLOSE that would make the required handling incompatible with the prescribed handling that the delegated client would apply (see below).
There are two types of open delegations, read and write 🦋. A read open delegation lets a client handle, on its own, requests to open a file for reading that don't deny read access to others. Multiple read open delegations can be outstanding at the same time and they don't conflict 🌸. A write open delegation lets the client handle all opens on its own. Only one write open delegation can exist for a given file at a given time, and it's inconsistent with any read open delegations 👑.
When a client has a read open delegation, it may not make any changes to the contents or attributes of the file, but it's assured that no other client may do so 🎀. When a client has a write open delegation, it may modify the file data since no other client will be accessing the file's data ✨. The client holding a write delegation may only affect file attributes that are intimately connected with the file data: object_size, time_modify, change 💜.
When a client has an open delegation, it does not send OPENs or CLOSEs to the server. It updates the appropriate status internally instead 🧚. For a read open delegation, opens that can't be handled locally (opens for write, or opens that deny read access) must be sent to the server 🙅♀️.
When an open delegation is made, the response to the OPEN contains an open delegation structure that specifies the following:
- the type of delegation (read or write)
- space limitation information to control flushing of data on close (write open delegation only, see the section "Open Delegation and Data Caching")
- an
nfsace4specifying read and write permissions - a stateid to represent the delegation for READ and WRITE
The stateid is separate and distinct from the stateid for the OPEN proper 🍓. The standard stateid, unlike the delegation stateid, is associated with a particular nfs_lockowner and will keep being valid after the delegation is recalled and the file remains open 💖.
When a request internal to the client is made to open a file and open delegation is in effect, it gets accepted or rejected solely on the basis of the following conditions. Any requirement for other checks to be made by the delegate should result in the open delegation being denied, so that the checks can be made by the server itself 🫶.
- The access and deny bits for the request and the file, as described in the section "Share Reservations".
- The read and write permissions as determined below.
The nfsace4 passed with the delegation can be used to avoid frequent ACCESS calls. The permission check goes like this 🔍:
- If the
nfsace4indicates that the open may be done, it should be granted without reference to the server. - If the
nfsace4indicates that the open may not be done, then an ACCESS request must be sent to the server to get the definitive answer.
The server may return an nfsace4 that is more restrictive than the actual ACL of the file. This includes an nfsace4 that specifies denial of all access. Heads up though: some common practices, like mapping the traditional user "root" to the user "nobody", may make it incorrect to return the actual ACL of the file in the delegation response 🙈.
The use of delegation together with various other forms of caching creates the possibility that no server authentication will ever be performed for a given user, since all of the user's requests might be satisfied locally 😭. Where the client is depending on the server for authentication, the client should be sure authentication occurs for each user by use of the ACCESS operation. This should be the case even if an ACCESS operation wouldn't be required otherwise. As mentioned before, the server may enforce frequent authentication by returning an nfsace4 denying all access with every open delegation 💀.
9.4.1. Open Delegation and Data Caching
OPEN delegation cuts out a lot of the message overhead that comes with opening and closing files 🎉. An open made while an open delegation is in effect doesn't require a validation message to be sent to the server. The continued existence of a "read open delegation" guarantees that no OPEN for write, and therefore no write, has occurred 🫧. Similarly, when closing a file that was opened for write while a write open delegation is in effect, the data written doesn't have to be flushed to the server until the open delegation is recalled. The continued existence of the open delegation guarantees that no open, and therefore no read or write, has been done by another client 🌈.
For the purposes of open delegation, READs and WRITEs done without an OPEN are treated as the functional equivalents of a corresponding type of OPEN. This refers to the READs and WRITEs that use the special stateids made of all zero bits or all one bits 🧸. So READs or WRITEs with a special stateid done by another client will force the server to recall a write open delegation. And a WRITE with a special stateid done by another client will force a recall of read open delegations 📣.
With delegations, a client can avoid writing data to the server when the CLOSE of a file is serviced. The CLOSE operation is normally the point where the client is told about a lack of stable storage for the modified file data the application generated 🥺. At the CLOSE, file data is written to the server, and through normal accounting the server can determine whether the available file system space for the data has been exceeded (i.e., the server returns NFS4ERR_NOSPC or NFS4ERR_DQUOT). This accounting includes quotas 📊. Introducing delegations means there has to be an alternative method in place for the same type of communication to happen between client and server 🔁.
In the delegation response, the server provides either the limit of the size of the file, or the number of modified blocks and the associated block size 📦. The server must ensure that the client will be able to flush data to the server in an amount equal to what was provided in the original delegation, and it must make this assurance for all outstanding delegations 🛡️. That means the server has to be careful about managing available space for new or modified data, taking into account available file system space and any applicable quotas 🧮. The server can recall delegations as a result of managing the available file system space ♻️. The client should abide by the server's state space limits for delegations. If the client exceeds the stated limits for the delegation, the server's behavior is undefined 💀.
Based on server conditions, quotas, or available file system space, the server may grant write open delegations with very restrictive space limitations. Those limitations may be defined in a way that will always force modified data to be flushed to the server on close 🧹.
Authentication can get messy here 🥹. Flushing modified data to the server after a CLOSE has occurred may be problematic. For example, the user of the application may have logged off of the client, and unexpired authentication credentials may not be present. In that case, the client may need to take special care to make sure local unexpired credentials will actually be available. This can be done by tracking the expiration time of credentials and flushing data well in advance of their expiration, or by making private copies of credentials so they're available when needed 🔐.
9.4.2. Open Delegation and File Locks
When a client holds a write open delegation, lock operations are performed locally 🔒. This includes the ones required for mandatory file locking. That works because the delegation implies there can be no conflicting locks 🙌. Similarly, all of the revalidations that would normally go along with obtaining locks, and the flushing of data that goes along with releasing locks, don't need to be done 😌.
9.4.3. Recall of Open Delegation 🦋
okay so the server can totally recall an open delegation when any of these things happen 👀
- Potentially conflicting OPEN request (or READ/WRITE done with "special" stateid) 💅
- SETATTR issued by another client
- REMOVE request for the file
- RENAME request for the file as either source or target of the RENAME
ngl the RENAME of a directory on the path leading to the file is the tricky one 🙈 whether that triggers a recall depends on the server's file system. if that file system denies those RENAMEs while a file is open, then the recall has to happen so the server can check if the file is actually open. bestie, that's a whole vibe 💀
the server can also recall open delegations whenever it wants if resources are tight, so clients should always be ready for a recall. no surprises allowed 🫶
now the GETATTR situation is lowkey iconic ✨ if the target is a file with a write open delegation in effect, the client holding that delegation has to be interrogated first. the server uses a CB_GETATTR callback, but only if the GETATTR asks for any of the attributes a write open delegate can change: object_size, time_modify, or change.
when a client gets a recall for an open delegation, it has to update the server's state before giving the delegation back 🎀 same goes for when a client returns a delegation voluntarily. here's what needs sorting:
- if the file is no longer open and no earlier CLOSE was sent to the server, a CLOSE operation must be sent to the server.
- if the file still has other open references at the client, OPEN operations must be sent to the server. the server hands back the right stateids for later use, because the delegation stateid won't be valid anymore. these OPENs use the claim type
CLAIM_DELEGATE_CUR, which lets the client present the delegation stateid so it can prove it has the rights to do the OPEN. (check the "Operation 18: OPEN" section for the full details 📖) - if there are granted file locks, the matching LOCK operations need to be done. this only applies to the write open delegation case.
- for a write open delegation, if the file isn't open for write at recall time, all modified data for the file must be flushed to the server. without the delegation, the client would've flushed that data before the CLOSE, so it's the same energy 🌸
- for a write open delegation when the file is still open at recall time, any modified data for the file needs to be flushed to the server too.
- with a write open delegation in place, the file might've been truncated during the delegation. like, an OPEN UNCHECKED with an
object_sizevalue of zero could do that. so if a truncation happened and never got sent to the server, the truncation must happen before any modified data gets written to the server. order matters, periodt 💅
file locking adds a few extra rules for write open delegations 🔒 precisely speaking, you need to flush modified data in any region where a write lock got released while the write open delegation was active, to keep the invariant intact. but since a write open delegation means no other client is locking, it's way simpler to just flush all modified data for the file (like we just talked about) if any write lock was released during the delegation. easy mode 🧚
9.4.4. Delegation Revocation 💜
when a delegation gets revoked, if the client still has opens on the file, the apps holding those opens need to be told. usually that happens by returning errors on READ/WRITE operations, or when a close is attempted on the open file.
if no opens exist when the delegation is revoked, telling anyone isn't necessary. BUT if there's modified data sitting at the client for that file, the user should be notified 🥺 the catch is that it might not be possible to notify the user, because active applications might not be around on the client. check "Revocation Recovery for Write Open Delegation" for more deets.
9.5. Data Caching and Revocation 🌈
when locks and delegations get revoked, the assumptions that made caching safe are no longer guaranteed 😭 the owner of the revoked locks or share reservations needs to be notified, and that includes apps with a file open that has a revoked delegation. cached data tied to the revocation has to be removed from the client. if there's modified data in the client's cache, it gets removed without being written to the server 💀
why? because the client's assumptions stop being valid the moment the lock or delegation is revoked. for example, another client might've grabbed a conflicting lock after the first client's lock was revoked. so the data in that lock range could've been changed by the other client. the first client just can't promise the app what happened to the file after a revocation, and that's the tea ☕
notifying a lock owner usually just means returning an error on the next and all later READs/WRITEs to the open file, or on the close. but if the client's methods can't return errors for some operations, more drastic stuff like signals or process termination might be needed. that's justified because an invariant the app depends on might be broken. depending on how the client environment usually handles errors, extra notification like logging, console messages, or GUI pop-ups could also make sense 🔔
9.5.1. Revocation Recovery for Write Open Delegation 🧚♀️
okay this one is the spiciest ngl 🌶️ revocation recovery for a write open delegation has the special problem of modified data sitting in the client cache while the file isn't open. so any client that doesn't flush modified data to the server on every close has to make sure the user gets proper notice that the revocation failed things. since these situations might need a human to fix stuff, notification schemes that tell the right user or admin can be necessary. logging and console messages are the classic examples 📝
if there's modified data on the client, it must not be flushed normally to the server. a client may try to save a copy of the file data as it was modified during the delegation under a different name in the file system namespace, to make recovery easier. unless the client can figure out that no other client modified the file, this trick has to be limited to cases where the client has a complete cached copy of the file. it might also be limited to files under a certain size, or only used when there's guaranteed disk space on the target file system and the client has enough buffer resources to keep the cached copy until it's safely stored there 🫧
9.6. Attribute Caching 🍓
quick note: the attributes in this section don't include named attributes. individual named attributes act like files, so caching their data has to be handled just like data caching for regular files. same deal for LOOKUP results from an OPENATTR directory, which get cached on the same basis as other pathnames, and directory contents too 📂
clients can cache file attributes from the server and use them to skip later GETATTR requests ✨ this caching is write through, which means changes to attributes always go through requests to the server and should not be made locally and then cached. the exception is attribute changes that are closely tied to data caching. so extending a file by writing data to the local data cache shows up right away in the object_size the client sees, even though the server doesn't get that change right away. normally those changes don't get sent to the server directly. instead, when the modified data gets flushed to the server, matching attribute changes happen on the server too. when open delegation is in effect, the modified attributes may be sent back to the server in the response to a CB_RECALL call 💅
because attributes get cached locally, the attribute caches on different clients won't be coherent 😵💫 changes made in one order on the server might show up in a different order on one client, and in a third order on another client.
the usual file system APIs can't atomically change or check attributes for several files at once. so these rules keep the possible incoherence under control 🎯 they come from the practice of earlier NFS protocols:
- all attributes for a given file (per-fsid attributes excepted) are cached as one unit on the client, so no non-serializability can happen within a single file.
- there's an upper time boundary on how long a client cache entry can be kept without being refreshed from the server.
- when operations change attributes at the server, the updated attribute set gets requested as part of the containing RPC. this includes directory operations that update attributes indirectly. it's done by following the modifying operation with a GETATTR and then using that GETATTR's results to update the client's cached attributes.
if the full set of attributes to cache is requested by READDIR, the results can be cached on the same basis as attributes from GETATTR 🎀
finally, a client can check whether its cached attributes for a file are still good by just fetching the change attribute. if the change attribute matches the value it had when the attributes were cached, the client assumes no attributes have changed. the possible exception is time_access 🦋
9.7. Name Caching 🎀
okay so LOOKUP and READDIR results can be cached so you don't have to redo the same lookup every single time 💅 but just like with attribute caching, the different client caches can end up disagreeing with each other 😭 to keep that drama in check (and because of how typical file system APIs work), these rules should be followed:
- The results of unsuccessful LOOKUPs should not be cached, unless they are specifically reverified at the point of use. 🙈
- An upper time boundary is maintained on how long a client name cache entry can be kept without verifying that the entry has not been made invalid by a directory change operation performed by another client. ⏰
When a client isn't changing a directory that has name cache entries, it needs to keep peeking at that directory's attributes to make sure nobody else is modifying it 👀 once it's confirmed that no modification happened, the expiration time for those name cache entries can be bumped to the current time plus the name cache staleness bound. ✨
When a client is changing a directory, it needs to figure out whether other clients have changed it too 🧐 it does this by looking at the change attribute reported before and after the directory operation, which comes back in the associated change_info4 value. the server can tell the client whether the change_info4 data is provided atomically with respect to the directory operation. if the change values are provided atomically, the client can compare the pre-operation change value with the change value in its name cache. if that comparison shows the directory was updated by another client, the name cache for the modified directory gets purged from the client 💀 if the comparison shows no modification, the name cache can be updated on the client to reflect the directory operation, and the timeout gets extended. the post-operation change value needs to be saved as the basis for future change_info4 comparisons. 💖
As demonstrated by the scenario above, name caching requires that the client revalidate name cache data by inspecting the change attribute of a directory at the point when the name cache item was cached. this means the server has to update the change attribute for directories whenever the contents of the directory get modified. for a client to use the change_info4 information appropriately and correctly, the server must report the pre and post operation change attribute values atomically. when the server is unable to report the before and after values atomically with respect to the directory operation, the server must indicate that fact in the change_info4 return value. when the information is not atomically reported, the client should not assume that other clients have not changed the directory. 🚩
9.8. Directory Caching 🍓
READDIR results can be used to skip later READDIR operations 🧚 and just like attribute and name caching, the client caches can get inconsistent with each other.
to keep those inconsistencies under control, and given how typical file system APIs work, the following rules should be followed:
- Cached READDIR information for a directory which is not obtained in a single READDIR operation must always be a consistent snapshot of directory contents. this is determined by using a GETATTR before the first READDIR and after the last of READDIR that contributes to the cache. 📸
- An upper time boundary is maintained to indicate the length of time a directory cache entry is considered valid before the client must revalidate the cached information. ⏳
The revalidation technique is basically the same as the one used for name caching 🫶 when the client isn't changing the directory in question, checking the directory's change attribute with GETATTR is good enough, and the lifetime of the cache entry can be extended at these checkpoints. when a client is modifying the directory, it needs to use the change_info4 data to find out whether other clients are modifying it too. if it's determined that no other client modifications are happening, the client may update its directory cache to reflect its own changes. 🦋
As demonstrated previously, directory caching requires that the client revalidate directory cache data by inspecting the change attribute of a directory at the point when the directory was cached. so the server has to update the change attribute for directories whenever the contents of the directory are modified. for a client to use the change_info4 information appropriately and correctly, the server must report the pre and post operation change attribute values atomically. when the server is unable to report the before and after values atomically with respect to the directory operation, the server must indicate that fact in the change_info4 return value. when the information is not atomically reported, the client should not assume that other clients have not changed the directory. 🙏
10. Minor Versioning 🌸
the NFS version 4 protocol needs to be able to grow as the need arises, so it includes the rules and framework for future minor changes and versioning 🦋
the base assumption is that any future accepted minor version has to follow the IETF process and be documented in a standards track RFC. that means each minor version number corresponds to an RFC. minor version zero of NFS version 4 is represented by this RFC. the COMPOUND procedure supports encoding the minor version the client is requesting. 💅
these are the basic rules for developing minor versions. just know that a future minor version may decide to modify or add to them as part of its own definition ✨
1. Procedures are not added or deleted 🔒
to keep the general RPC model intact, NFS version 4 minor versions will not add or delete procedures from the NFS program.
2. Minor versions may add operations to the COMPOUND and CB_COMPOUND procedures. 🧩
adding operations to the COMPOUND and CB_COMPOUND procedures does not affect the RPC model.
2.1. Minor versions may append attributes to GETATTR4args, bitmap4, and GETATTR4res. 🌈
this leaves room for the attribute model to grow and adapt later.
2.2. Minor version X must append any new attributes after the last documented attribute. 📌
attribute results are specified as an opaque array of per-attribute XDR encoded results, so squeezing new attributes into the middle of the current definitions would be way too burdensome 😩
3. Minor versions must not modify the structure of an existing operation's arguments or results. 🚫
again, handling multiple structure definitions for a single operation is too burdensome. new operations should be added instead of modifying existing structures for a minor version.
this rule still allows the following adaptations in a minor version:
- adding bits to flag fields, such as new attributes to GETATTR's bitmap4 data type
- adding bits to existing attributes like ACLs that have flag words
- extending enumerated types (including
NFS4ERR_*) with new values
4. Minor versions may not modify the structure of existing attributes. 🫠
5. Minor versions may not delete operations. 🪦
this prevents the potential reuse of a particular operation "slot" in a future minor version.
6. Minor versions may not delete attributes. 🙅♀️
7. Minor versions may not delete flag bits or enumeration values. 🙅♀️
8. Minor versions may declare an operation as mandatory to NOT implement. 🪄
declaring an operation "mandatory to not implement" is equivalent to obsoleting it. for the client, it means the operation should not be sent to the server. for the server, an NFS error can be returned instead of "dropping" the request as an XDR decode error. this lets an operation go obsolete while keeping its structure, so a future minor version can bring it back. 🔄
8.1. Minor versions may declare attributes mandatory to NOT implement. 📦
8.2. Minor versions may declare flag bits or enumeration values as mandatory to NOT implement. 🏷️
9. Minor versions may downgrade features from mandatory to recommended, or recommended to optional. ⬇️
10. Minor versions may upgrade features from optional to recommended or recommended to mandatory. ⬆️
11. A client and server that support minor version X must support minor versions 0 (zero) through X-1 as well. 🤝
12. No new features may be introduced as mandatory in a minor version. 🌱
this rule lets new functionality arrive while forcing real implementation experience before a feature gets designated as mandatory. 🧪
13. A client MUST NOT attempt to use a stateid, file handle, or similar returned object from the COMPOUND procedure with minor version X for another COMPOUND procedure with minor version Y, where X != Y. 🚨
11. Internationalization 🌍
the main internationalization (I18n) issue for NFS is file names and other strings used within the protocol. the choice of string representation has to allow reasonable name and string access for clients who use various languages. the UTF-8 encoding of the UCS defined by [ISO10646] allows for this kind of access and follows the policy described in "IETF Policy on Character Sets and Languages", [RFC2277]. this choice is explained further in the following. 💖
11.1. Universal Versus Local Character Sets ✨
okay bestie, so [RFC1345] describes a table of 16 bit characters for many different languages 🌍 (the bit encodings match Unicode, though of course RFC1345 is somewhat out of date with respect to current Unicode assignments 💅). Each character from each language has a unique 16 bit value in the 16 bit character set. So this table can be thought of as a universal character set 👑. Then [RFC1345] talks about groupings of subsets of the entire 16 bit character set into "Charset Tables". For example, you could take all the Greek characters from the 16 bit table (which are consecutively allocated) and normalize their offsets to a table that fits in 7 bits. That means "lower case alpha" ends up in the same position as "upper case a" in the US-ASCII table, and "upper case alpha" ends up in the same position as "lower case a" 🤯
These normalized subset character sets can be thought of as "local character sets", suitable for an operating system locale 🎀
Local character sets are not suitable for the NFS protocol 💀. Picture someone who creates a file with a name in a Swedish character set. If someone else later goes to access that file with their locale set to Swedish, no problems. But if someone in, say, the US-ASCII locale goes to access the file, the name will look totally different, because the Swedish characters in the 7 bit table will now be shown as US-ASCII characters on the display 😭. It would be nicer to give the US-ASCII user a way to display the file name using Swedish glyphs 🦋. To do that, the NFS protocol would have to include the locale with the file name on each operation that creates a file.
But then what about a path name on the server like this?
/component-1/component-2/component-3Each component could have been created with a different locale 🙈. If someone issues CREATE with a multi-component path name, and some of the leading components already exist, what happens to those existing components? Does the current locale attribute get replaced with the user's current one? These situations get way too complex when there's an alternate solution 🫠
If the NFS version 4 protocol used a universal 16 bit or 32 bit character set (or an encoding of a 16 bit or 32 bit character set into octets), then the server and client don't need to care whether the locale of the user accessing the file is different from the locale of the user who created it 🫶. The unique 16 bit or 32 bit encoding of each character lets you work out what language the character comes from and how to display it on the client. The server doesn't need to know what locales are used at all 🧚
11.2. Overview of Universal Character Set Standards 🌸
The previous section makes the case for using a universal character set. This section makes the case for using UTF-8 as the specific universal character set for the NFS version 4 protocol 💖
[RFC2279] discusses UTF-* (UTF-8 and other UTF-XXX encodings), Unicode, and UCS-* ✨. There are two standards bodies managing universal code sets:
- ISO/IEC, which has the standard 10646-1
- Unicode, which has the Unicode standard
Both standards bodies have pledged to track each other's assignments of character codes 🤝
Here's a brief analysis of the various standards 📚
- UCS: Universal Character Set. This is ISO/IEC 10646-1: "a multi-octet character set called the Universal Character Set (UCS), which encompasses most of the world's writing systems."
- UCS-2: a two octet per character encoding that addresses the first 2^16 characters of UCS. Currently there are no UCS characters beyond that range.
- UCS-4: a four octet per character encoding that permits the encoding of up to 2^31 characters.
- UTF: UTF is an abbreviation of the term "UCS transformation format" and is used in the naming of various standards for encoding of UCS characters, as described below.
- UTF-1: Only historical interest; it has been removed from 10646-1 🦕
- UTF-7: Encodes the entire "repertoire" of UCS "characters using only octets with the higher order bit clear". [RFC2152] describes UTF-7. UTF-7 accomplishes this by reserving one of the 7 bit US-ASCII characters as a "shift" character to indicate non-US-ASCII characters.
- UTF-8: Unlike UTF-7, uses all 8 bits of the octets 🔥. US-ASCII characters are encoded as before, unchanged. Any octet with the high bit cleared can only mean a US-ASCII character. The high bit set means that a UCS character is being encoded.
- UTF-16: Encodes UCS-4 characters into UCS-2 characters using a reserved range in UCS-2.
- Unicode: Unicode and UCS-2 are the same; [RFC2279] states:
Up to the present time, changes in Unicode and amendments to ISO/IEC 10646 have tracked each other, so that the character repertoires and code point assignments have remained in sync. The relevant standardization committees have committed to maintain this very useful synchronism.
11.3. Difficulties with UCS-4, UCS-2, Unicode 😩
Adapting existing applications and file systems to multi-octet schemes like UCS and Unicode can be difficult 🥲. A significant amount of code has been written to process streams of bytes. There are also many existing stored objects described with 7 bit or 8 bit characters. Doubling or quadrupling the bandwidth and storage requirements seems like an expensive way to accomplish I18N 💸
UCS-2 and Unicode are "only" 16 bits long. That might seem like enough, but according to [Unicode1], 49,194 Unicode characters are already assigned 📈. According to [Unicode2], there are still more languages that need to be added 🌏
11.4. UTF-8 and its solutions 🎀
UTF-8 solves problems for NFS that exist with the use of UCS and Unicode 🙌. UTF-8 will encode 16 bit and 32 bit characters in a way that's compact for most users. Here's the encoding table from UCS-4 to UTF-8, copied from [RFC2279]:
UCS-4 range (hex.) UTF-8 octet sequence (binary)
0000 0000-0000 007F 0xxxxxxx
0000 0080-0000 07FF 110xxxxx 10xxxxxx
0000 0800-0000 FFFF 1110xxxx 10xxxxxx 10xxxxxx
0001 0000-001F FFFF 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx
0020 0000-03FF FFFF 111110xx 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx
0400 0000-7FFF FFFF 1111110x 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx
10xxxxxxSee [RFC2279] for the precise encoding and decoding rules. Note that, because of UTF-16, the algorithm from Unicode/UCS-2 to UTF-8 needs to account for the reserved range between D800 and DFFF ⚠️
Note that the 16 bit UCS or Unicode characters require no more than 3 octets to encode into UTF-8 🧮
Interestingly, UTF-8 has room to handle characters larger than 31 bits, because the leading octet of this form is not defined:
1111111xIf needed, ISO could either use that octet to indicate a sequence of an encoded 8 octet character, or perhaps use 11111110 to permit the next octet to indicate an even more expandable character set 🌌
So using UTF-8 to represent character encodings means never having to run out of room 👑
11.5. Normalization 💅
The client and server operating environments may differ in their policies and operational methods with respect to character normalization (See [Unicode1] for a discussion of normalization forms) 🧐. This difference may also exist between applications on the same client. This adds to the difficulty of providing a single normalization policy for the protocol that allows for maximal interoperability. This issue is similar to the character case issues, where the server may or may not support case insensitive file name matching and may or may not preserve the character case when storing file names. The protocol does not mandate a particular behavior but allows for the various permutations 🦋
The NFS version 4 protocol does not mandate the use of a particular normalization form at this time. A later revision of this specification may specify a particular normalization form 🔮. So the server and client can expect that they may receive unnormalized characters within protocol requests and responses. If the operating environment requires normalization, then the implementation must normalize the various UTF-8 encoded strings within the protocol before presenting the information to an application (at the client) or local file system (at the server) 🫶
12. Error Definitions 💅
okay so NFS error numbers get slapped onto whichever operation failed inside a compound request 🚨 A compound request holds a bunch of NFS operations, and their results come back encoded in order in a compound reply. Successful ops give you an NFS4_OK status and then their encoded results. If an op fails, an error status goes into the reply and the compound request gets terminated, so it's over bestie 💀
Here's the full lineup of defined errors:
NFS4_OK: the operation completed successfully ✨NFS4ERR_ACCES: permission denied. The caller doesn't have the right permission for the requested operation. Compare this withNFS4ERR_PERM, which only covers owner or privileged user permission failures.NFS4ERR_BADHANDLE: illegal NFS file handle. It failed the internal consistency checks 🙈NFS4ERR_BADTYPE: someone tried to create an object of a type the server doesn't support.NFS4ERR_BAD_COOKIE: the READDIR cookie is stale.NFS4ERR_BAD_SEQID: the sequence number in a locking request is neither the next expected number or the last number processed.NFS4ERR_BAD_STATEID: a stateid generated by the current server instance was used, but it doesn't designate any locking state (current or superseded) for a current lockowner-file pair.NFS4ERR_CLID_INUSE: the SETCLIENTID procedure found that a client id is already in use by another client 🫶NFS4ERR_DELAY: the server started the request but couldn't finish it in a timely fashion. The client should wait and then try the request again with a new RPC transaction ID. For example, this error should be returned from a server that supports hierarchical storage and gets a request to process a file that has been migrated. In that case the server should start the immigration process and respond to the client with this error. It may also show up when a necessary delegation recall makes processing the request in a timely fashion impossible 😭NFS4ERR_DENIED: an attempt to lock a file was denied. This might be temporary, so the client is encouraged to retry the lock request until the lock is accepted 🔥NFS4ERR_DQUOT: resource (quota) hard limit exceeded. The user's resource limit on the server has been exceeded.NFS4ERR_EXIST: file exists. The file specified already exists.NFS4ERR_EXPIRED: a lease has expired that is being used in the current procedure.NFS4ERR_FBIG: file too large. The operation would have caused a file to grow beyond the server's limit.NFS4ERR_FHEXPIRED: the file handle provided is volatile and has expired at the server.NFS4ERR_GRACE: the server is in its recovery or grace period, which should match the lease period of the server.NFS4ERR_INVAL: invalid argument or unsupported argument for an operation. Two examples are attempting a READLINK on an object other than a symbolic link, or attempting to SETATTR a time field on a server that does not support this operation.NFS4ERR_IO: I/O error. A hard error (for example, a disk error) occurred while processing the requested operation.NFS4ERR_ISDIR: is a directory. The caller specified a directory in a non-directory operation.NFS4ERR_LEASE_MOVED: a lease being renewed is associated with a file system that has been migrated to a new server.NFS4ERR_LOCKED: a read or write operation was attempted on a locked file.NFS4ERR_LOCK_RANGE: a lock request is operating on a sub-range of a current lock for the lock owner, and the server does not support this type of request.NFS4ERR_MINOR_VERS_MISMATCH: the server has received a request that specifies an unsupported minor version. The server must return a COMPOUND4res with a zero length operations result array 👑NFS4ERR_MLINK: too many hard links.NFS4ERR_MOVED: the filesystem which contains the current filehandle object has been relocated or migrated to another server. The client may obtain the new filesystem location by obtaining the"fs_locations"attribute for the current filehandle. For further discussion, refer to the section"Filesystem Migration or Relocation".NFS4ERR_NAMETOOLONG: the filename in an operation was too long.NFS4ERR_NODEV: no such device.NFS4ERR_NOENT: no such file or directory. The file or directory name specified does not exist.NFS4ERR_NOFILEHANDLE: the logical current file handle value has not been set properly. This may be the result of a malformed COMPOUND operation (i.e. no PUTFH or PUTROOTFH before an operation that requires the current file handle be set).NFS4ERR_NOSPC: no space left on device. The operation would have caused the server's file system to exceed its limit.NFS4ERR_NOTDIR: not a directory. The caller specified a non-directory in a directory operation.NFS4ERR_NOTEMPTY: an attempt was made to remove a directory that was not empty.NFS4ERR_NOTSUPP: operation is not supported.NFS4ERR_NOT_SAME: this error is returned by the VERIFY operation to signify that the attributes compared were not the same as provided in the client's request.NFS4ERR_NXIO: I/O error. No such device or address.NFS4ERR_OLD_STATEID: a stateid which designates the locking state for a lockowner-file at an earlier time was used.NFS4ERR_PERM: not owner. The operation was not allowed because the caller is either not a privileged user (root) or not the owner of the target of the operation.NFS4ERR_READDIR_NOSPC: the encoded response to a READDIR request exceeds the size limit set by the initial request.NFS4ERR_RESOURCE: for the processing of the COMPOUND procedure, the server may exhaust available resources and can not continue processing procedures within the COMPOUND operation. This error will be returned from the server in those instances of resource exhaustion related to the processing of the COMPOUND procedure.NFS4ERR_ROFS: read-only file system. A modifying operation was attempted on a read-only file system.NFS4ERR_SAME: this error is returned by the NVERIFY operation to signify that the attributes compared were the same as provided in the client's request.NFS4ERR_SERVERFAULT: an error occurred on the server which does not map to any of the legal NFS version 4 protocol error values. The client should translate this into an appropriate error. UNIX clients may choose to translate this to EIO.NFS4ERR_SHARE_DENIED: an attempt to OPEN a file with a share reservation has failed because of a share conflict.NFS4ERR_STALE: invalid file handle. The file handle given in the arguments was invalid. The file referred to by that file handle no longer exists or access to it has been revoked.NFS4ERR_STALE_CLIENTID: a clientid not recognized by the server was used in a locking or SETCLIENTID_CONFIRM request.NFS4ERR_STALE_STATEID: a stateid generated by an earlier server instance was used.NFS4ERR_SYMLINK: the current file handle provided for a LOOKUP is not a directory but a symbolic link. Also used if the final component of the OPEN path is a symbolic link.NFS4ERR_TOOSMALL: buffer or request is too small.NFS4ERR_WRONGSEC: the security mechanism being used by the client for the procedure does not match the server's security policy. The client should change the security mechanism being used and retry the operation 🦋NFS4ERR_XDEV: attempt to do a cross-device hard link.
13. NFS Version 4 Requests 🌈
okay so for the NFS version 4 RPC program there are two traditional RPC procedures: NULL and COMPOUND. Everything else is defined as a set of operations, and those operations are written in normal XDR/RPC syntax and semantics. The catch is that they're all wrapped up inside the COMPOUND procedure, which means the client has to bundle one or more NFS version 4 operations into a single request 🎀
The NFS4_CALLBACK program is how the server signals the client, and it's built pretty much the same way as the NFS version 4 program. The procedures CB_NULL and CB_COMPOUND are defined just like NULL and COMPOUND are in the NFS program, and CB_COMPOUND wraps up the remaining operations of the NFS4_CALLBACK program. There's no predefined RPC program number for NFS4_CALLBACK, so it's up to the client to pick a program number in the "transient" program range 🧚 The program and port number of NFS4_CALLBACK get handed over by the client as part of the SETCLIENTID operation, so they stay fixed for the whole life of that client instantiation 💜
13.1. Compound Procedure ✨
okay so COMPOUND is literally the reason high latency networks don't ruin everyone's whole day 🫶 instead of making a bunch of back-and-forth RPCs that each eat latency, the client bundles several dependent operations into one COMPOUND procedure. it's giving efficiency 💅 and it can also simplify the protocol, because the client gets to pack basic procedures into one request that's tailored to its own environment ✨
CB_COMPOUND is the callback twin and it follows the exact same features as COMPOUND, so no new drama there 🌸
here's what a COMPOUND request basically looks like 🎀
+-----------+-----------+-----------+--
| op + args | op + args | op + args |
+-----------+-----------+-----------+--and the reply comes back like this 🦋
+------------+-----------------------+-----------------------+--
|last status | status + op + results | status + op + results |
+------------+-----------------------+-----------------------+--13.2. Evaluation of a Compound Request
the server works through a COMPOUND procedure by evaluating each operation in order, one at a time 💅 every operation starts with a 32 bit operation code, then its arguments, and the length of those arguments depends on which operation it is. results get encoded in sequence into a reply buffer, and each result is preceded by the opcode and a status code, which is normally zero 🧚
now here's the plot twist 😭 if any operation comes back with a non-zero status code, the server encodes that status, stops evaluating the compound sequence, and sends the reply right then. and it's not just actual errors that do this. evaluation stops even for "non error" conditions like NFS4ERR_SAME, so keep that in your back pocket 🙈
there are no atomicity requirements for the operations inside a COMPOUND procedure. they can run at the same time as other COMPOUND requests the server is handling, so it's a whole party in there 🎉
the client is the one responsible for recovering from a partially completed COMPOUND procedure 🫶 partial completion can happen at any point because of things like NFS4ERR_RESOURCE and NFS4ERR_LONG_DELAY, even when the operation string itself was totally valid. and if a server reboots in the middle of processing a COMPOUND, the client might have a rough time figuring out how far it got 💀 so the client should avoid overly complex COMPOUND procedures when an operation might fail.
every operation has a "current" filehandle and a "saved" filehandle that live in the execution context of the compound request ✨ operations can set, change, or return the current filehandle. the "saved" one is for temporary storage of a filehandle value, and it also serves as an operand for the RENAME and LINK operations 🦋
13.3. Synchronous Modifying Operations
NFS version 4 operations that change the file system are synchronous 💖 when one finishes successfully at the server, the client can count on any data tied to that request being on stable storage. the one exception is file data in a WRITE operation that has the UNSTABLE option set 🌸
this also means that any earlier operations in the same compound request are reflected in stable storage too 🫶 that's what lets the client recover from a partially executed compound request if the server fails. say a compound request has operations A and B, and the server can't send a response back. depending on how far the server got, the result could be that both operations made it to stable storage, or just operation A did. the server must not have just the results of operation B in stable storage 🙈 that combo is a no go 💅
13.4. Operation Values
each operation inside a COMPOUND procedure is identified by an operation value 🧚 to avoid overlapping with the RPC procedure numbers, operations 0 (zero) and 1 are not defined. operation 2 isn't defined either, but it's saved for future use with minor versioning 👑
14. NFS Version 4 Procedures ✨
14.1. Procedure 0: NULL - No Operation
SYNOPSIS
<null>ARGUMENT
void;RESULT
void;DESCRIPTION
this is the standard NULL procedure: void argument, void response 💅 it has no functionality attached to it, which is exactly why it's sometimes used to measure the overhead of processing a service request 🍓 so the server should make sure it doesn't do any unnecessary work while servicing this procedure 🙈
ERRORS
None. 🌈
14.2. Procedure 1: COMPOUND - Compound Operations
SYNOPSIS
compoundargs -> compoundresARGUMENT
union nfs_argop4 switch (nfs_opnum4 argop) {
case <OPCODE>: <argument>;
...
};
struct COMPOUND4args {
utf8string tag;
uint32_t minorversion;
nfs_argop4 argarray<>;
};RESULT
union nfs_resop4 switch (nfs_opnum4 resop){
case <OPCODE>: <result>;
...
};
struct COMPOUND4res {
nfsstat4 status;
utf8string tag;
nfs_resop4 resarray<>;
};DESCRIPTION
COMPOUND is how you bundle one or more NFS operations into a single RPC request 💖 the main NFS RPC program has two main procedures, NULL and COMPOUND, and every other operation rides along inside COMPOUND as a wrapper 🎀
so yeah, COMPOUND combines individual operations into one RPC request. the server goes through them in turn ✨ if an operation runs and its status is NFS4_OK, the next one runs too. this keeps going until there's nothing left to execute or an operation returns a status other than NFS4_OK 🦋
here's where it gets spicy 🌶️ the server might not have enough resources to run some or all of the operations in the sequence. when that happens, NFS4ERR_RESOURCE gets returned for the specific operation where the resource exhaustion happened 😭 this assumes every earlier operation in the sequence was evaluated successfully, and the results of all the evaluated operations have to go back to the client 🫶
the COMPOUND arguments have a "minorversion" field, and its initial and default value is 0 (zero) 💅 future minor versions will use this field so the client can tell the server which minor version it's asking for ✨
if the server gets a COMPOUND procedure with a minorversion value it doesn't support, the server MUST return an error of NFS4ERR_MINOR_VERS_MISMATCH and a zero length resultdata array. 🚨
the COMPOUND results include a "status" field 💖 if the results array length is non-zero, this status has to match the status of the last operation that ran inside the COMPOUND procedure. so if an operation hit an error, the "status" value is the same error that gets returned for the failed operation 🙈
heads up: operations 0 (zero) and 1 (one) aren't defined for the COMPOUND procedure. if the server gets an operation array containing either of them, it must return NFS4ERR_NOTSUPP 🫶 operation 2 is also undefined, but it's reserved for future definition and use with minor versioning 🧚 if the server gets an operation array with operation 2 and the minorversion field is 0 (zero), NFS4ERR_NOTSUPP gets returned. if the array contains operation 2, the minorversion field is non-zero, and the server doesn't support that minor version, the server returns NFS4ERR_MINOR_VERS_MISMATCH. so NFS4ERR_MINOR_VERS_MISMATCH takes precedence over all other errors 👑
IMPLEMENTATION
the definition of the "tag" in both the request and the response is left up to the implementor 🌸 it can be used to summarize the content of the compound request, which is super helpful for packet sniffers and engineers debugging implementations 🔍✨
since any kind of error can happen after only some of the operations have been evaluated, the client has to be prepared to recover from any failure 😭 if an NFS4ERR_RESOURCE error came from a complex or lengthy set of operations, there's a good chance the server would have handled them fine if there had been fewer of them. so dealing with that complexity during recovery is the client's job 💅
ERRORS
All errors defined in the protocol
14.2.1. Operation 3: ACCESS - Check Access Rights ✨
SYNOPSIS 🧚
(cfh), accessreq -> supported, accessrightsARGUMENT 💅
const ACCESS4_READ = 0x00000001;
const ACCESS4_LOOKUP = 0x00000002;
const ACCESS4_MODIFY = 0x00000004;
const ACCESS4_EXTEND = 0x00000008;
const ACCESS4_DELETE = 0x00000010;
const ACCESS4_EXECUTE = 0x00000020;
struct ACCESS4args {
/* CURRENT_FH: object */
uint32_t access;
};RESULT 🎀
struct ACCESS4resok {
uint32_t supported;
uint32_t access;
};
union ACCESS4res switch (nfsstat4 status) {
case NFS4_OK:
ACCESS4resok resok4;
default:
void;
};DESCRIPTION 🦋
Okay so ACCESS is basically the "can I actually do this?" check 👀 It figures out what access rights a user has on the file system object behind the current filehandle, and the user is whoever is named in the credentials of the RPC request. The client packs the rights it wants to check into a bit mask called "access", and the server checks those permissions. If the status comes back as NFS4_OK, the response has two masks in it. The first one, "supported", tells you which access rights the server can actually verify reliably 🔍 The second one, "access", tells you which rights the user has on that filehandle. And if it works, the current filehandle keeps its value 💖
Now here's the lowkey important bit: "supported" only ever has as many values as you sent in. So if the client only asks about ACCESS4_READ and the server supports it, you get ACCESS4_READ back, even if the server could have checked a bunch of other stuff too 🙈 No extra freebies!
ALSO, the results are advisory. Like, literally just advisory 💀 Getting NFS4_OK with the right bit set does NOT mean you're guaranteed that access later. The server can revoke access rights whenever it wants, so don't get too comfy 🥲
These are the permissions you can ask for:
ACCESS4_READ Read data from file or read a directory.
ACCESS4_LOOKUP Look up a name in a directory (no meaning for non-
directory objects).
ACCESS4_MODIFY Rewrite existing file data or modify existing
directory entries.
ACCESS4_EXTEND Write new data or add directory entries.
ACCESS4_DELETE Delete an existing directory entry (no meaning for
non-directory objects).
ACCESS4_EXECUTE Execute file (no meaning for a directory).Again, on success, the current filehandle keeps its value ✨
IMPLEMENTATION 🌈
For NFS version 4, using ACCESS when you're opening a regular file is deprecated, and you should use OPEN instead 🫶
Also, you can't really figure out access by poking at the uid, gid, and mode fields in the file attributes, or by trying to read the ACL attribute 🙅♀️ Why? The server might be doing uid or gid mapping, or it might have extra access control rules on top. The server might also not even share the same ID space as the client. In those situations (and maybe others too), the client just can't do a reliable access check using only the current file attributes. Not vibes-based, guys 😭
With NFS version 2, the only sure way to know if something was allowed was to just try it and see if it worked 💅 NFS version 4 lets the client ask the server ahead of time whether one or more classes of operations are permitted. That's what ACCESS is for: checking before you kick off a bunch of operations that would otherwise fail on access. OPEN is also a spot where the server can check access to the file object and send that info back to the client. ACCESS is still handy for directory operations, and for when the UNIX API "access" gets used on the client side ⭐
The info the server sends back from ACCESS isn't permanent, either 🦋 It was accurate at the exact moment the server did its checks, but it might not be later. The server can take that permission away at any time.
The client should use the effective credentials of the user to build the authentication information in the ACCESS request, since that's the info used to decide access rights. Those effective user and group credentials are also what get used in later read and write operations 🎀
Heads up: lots of implementations don't directly support the ACCESS4_DELETE permission 👀 Systems like UNIX just ignore the ACCESS4_DELETE bit if it's set on an access request for a non-directory object. On those systems, whether you can delete a file depends on the permissions of the directory the file lives in, not the file's own permissions 🗂️ So the mask that comes back, listing which access rights can be determined, will have ACCESS4_DELETE set to 0. That tells the client the server couldn't check that particular right. The client will then ignore the ACCESS4_DELETE bit in the returned access mask 🌸
ERRORS 🚨
NFS4ERR_ACCES
NFS4ERR_BADHANDLE
NFS4ERR_DELAY
NFS4ERR_FHEXPIRED
NFS4ERR_IO
NFS4ERR_MOVED
NFS4ERR_NOFILEHANDLE
NFS4ERR_RESOURCE
NFS4ERR_SERVERFAULT
NFS4ERR_STALE
NFS4ERR_WRONGSEC14.2.2. Operation 4: CLOSE - Close File 🍓
SYNOPSIS 🧚
(cfh), seqid, stateid -> stateidARGUMENT 💅
struct CLOSE4args {
/* CURRENT_FH: object */
seqid4 seqid
stateid4 stateid;
};RESULT 🎀
union CLOSE4res switch (nfsstat4 status) {
case NFS4_OK:
stateid4 stateid;
default:
void;
};DESCRIPTION 🦋
CLOSE is how you let go of the share reservations for the file behind the current filehandle 🔓 The share reservations and other state that get released on the server because of this CLOSE are only tied to the stateid you supply. The seqid is what keeps everything in the correct order ⭐ State belonging to other OPENs is totally unaffected, so no collateral damage, bestie 🫶
If you're still holding record locks, the client SHOULD release all of them before sending a CLOSE 💖 The server MAY free all outstanding locks on CLOSE, but some servers might not support closing a file that still has record locks held 🙈 And the server MUST return failure if any locks would still exist after the CLOSE. No sneaky leftovers allowed 😭
On success, the current filehandle keeps its value ✨
IMPLEMENTATION 🌈
ERRORS 🚨
NFS4ERR_BADHANDLE
NFS4ERR_BAD_SEQID
NFS4ERR_BAD_STATEID
NFS4ERR_DELAY
NFS4ERR_EXPIRED
NFS4ERR_FHEXPIRED
NFS4ERR_GRACE
NFS4ERR_INVAL
NFS4ERR_ISDIR
NFS4ERR_LEASE_MOVED
NFS4ERR_MOVED
NFS4ERR_NOFILEHANDLE
NFS4ERR_OLD_STATEID
NFS4ERR_RESOURCE
NFS4ERR_SERVERFAULT
NFS4ERR_STALE
NFS4ERR_STALE_STATEID14.2.3. Operation 5: COMMIT - Commit Cached Data 💾✨
SYNOPSIS
(cfh), offset, count -> verifierARGUMENT
struct COMMIT4args {
/* CURRENT_FH: file */
offset4 offset;
count4 count;
};RESULT
struct COMMIT4resok {
verifier4 writeverf;
};
union COMMIT4res switch (nfsstat4 status) {
case NFS4_OK:
COMMIT4resok resok4;
default:
void;
};DESCRIPTION
Okay so COMMIT is the operation that makes the server flush data to stable storage 🌊 for whatever file the current filehandle points at. The stuff it flushes is anything that was earlier written with a WRITE operation whose stable field was set to UNSTABLE4. Basically, the server was holding onto it and now it's time to lock it in 🔒
The offset tells the server where in the file the flush should start. An offset of 0 (zero) means "start from the very beginning of the file" 🌸 The count is how many bytes to flush. If count is 0 (zero), the flush runs from offset all the way to the end of the file. Iconic range control 👑
When COMMIT succeeds, the server sends back a write verifier. The client uses this to figure out whether the server restarted or rebooted between the original WRITE(s) and the COMMIT 👀 It does that by comparing the verifier from the writes with the one from COMMIT. The server must vary the value of the write verifier at each server event or instantiation that may lead to a loss of uncommitted data. Most commonly that's a reboot, but other events at the server can cause uncommitted data loss too, so no cap, the verifier has to change for all of them 😭
On success, the current filehandle keeps its value. Nothing gets replaced here 🫶
IMPLEMENTATION
COMMIT is basically the same vibe and semantics as the POSIX fsync(2) system call, which syncs a file's state with the disk (file data and metadata both get flushed to disk or stable storage). COMMIT does that exact same job for a client, flushing any unsynchronized data and metadata on the server to the server's disk or stable storage for the specified file. Just like fsync(2), there may be some modified data or no modified data at all to sync 🤷♀️ The data may already have been synced by the server's normal periodic buffer synchronization activity. COMMIT should return NFS4_OK, unless there has been an unexpected error.
COMMIT is different from fsync(2) in one big way: the client can flush just a range of the file. This is most likely triggered by a buffer-reclamation scheme on the client before the file has been completely written 🧠✨
The server side of COMMIT is reasonably simple, honestly a serene little vibe 🦋 If the server gets a full file COMMIT request (starting at offset 0 and count 0), it should do the equivalent of fsync()'ing the file. Otherwise, it should arrange to have the cached data in the range given by offset and count flushed to stable storage. Either way, any metadata associated with the file must be flushed to stable storage before the server returns. It is not an error for there to be nothing to flush on the server. That just means the data and metadata that needed flushing were already flushed, or lost during the last server failure 💀
The client side is a little more complex (bestie, it's giving multitasking 💅). There are two reasons a client wants to commit a buffer to stable storage. The first is that the client wants to reuse a buffer. In that case, the offset and count of the buffer go to the server in the COMMIT request. The server flushes any cached data based on that offset and count, flushes any metadata associated with the file, and then returns the status of the flush along with the write verifier. The second reason is a full file flush, like what might happen at close. Here the client gathers all of the buffers for this file that contain uncommitted data, does COMMIT with an offset of 0 and count of 0, and then frees all of those buffers. Any other dirty buffers get sent to the server in the normal fashion 🍓
After a client writes a buffer with the stable parameter set to UNSTABLE4, the buffer must be considered modified by the client until it has either been flushed by a COMMIT operation, or written by a WRITE operation with the stable parameter set to FILE_SYNC4 or DATA_SYNC4. This keeps the buffer from getting freed and reused before the data can be flushed to stable storage on the server 🛡️
When a response from either a WRITE or a COMMIT operation has a write verifier that's different from the one the server returned before, the client will need to retransmit all of the buffers containing uncommitted cached data to the server 😱 How to do that is up to the implementor. If there's only one buffer of interest, it should probably be sent back in a WRITE request with the appropriate stable parameter. If there's more than one buffer, it might be worthwhile to retransmit all of them in WRITE requests with the stable parameter set to UNSTABLE4, and then retransmit the COMMIT operation to flush all of the data on the server to stable storage. When these retransmissions happen is left to the implementor 🗓️
Everything above applies to page-cache-based systems as well as buffer-cache-based systems. In page-cache systems, the virtual memory system will need to be modified instead of the buffer cache 🌈
ERRORS
NFS4ERR_ACCES
NFS4ERR_BADHANDLE
NFS4ERR_FHEXPIRED
NFS4ERR_IO
NFS4ERR_ISDIR
NFS4ERR_LOCKED
NFS4ERR_MOVED
NFS4ERR_NOFILEHANDLE
NFS4ERR_RESOURCE
NFS4ERR_ROFS
NFS4ERR_SERVERFAULT
NFS4ERR_STALE
NFS4ERR_WRONGSEC14.2.4. Operation 6: CREATE - Create a Non-Regular File Object 🧚
SYNOPSIS
(cfh), name, type -> (cfh), change_infoARGUMENT
union createtype4 switch (nfs_ftype4 type) {
case NF4LNK:
linktext4 linkdata;
case NF4BLK:
case NF4CHR:
specdata4 devdata;
case NF4SOCK:
case NF4FIFO:
case NF4DIR:
void;
};
struct CREATE4args {
/* CURRENT_FH: directory for creation */
component4 objname;
createtype4 objtype;
};RESULT
struct CREATE4resok {
change_info4 cinfo;
};
union CREATE4res switch (nfsstat4 status) {
case NFS4_OK:
CREATE4resok resok4;
default:
void;
};DESCRIPTION
CREATE makes a non-regular file object inside a directory, using a given name 📁✨ Heads up: regular files are a different story. The OPEN procedure MUST be used to create a regular file, so don't try to sneak one through CREATE 🙅♀️
The objname is the name for the new object. If objname has a length of 0 (zero), the server returns NFS4ERR_INVAL. The objtype decides what kind of object gets created, like a directory, a symlink, and so on 🌿
If an object with the same name already exists in the directory, the server returns NFS4ERR_EXIST 😬
For the directory where the new object went, the server sends back change_info4 information in cinfo. The atomic field of the change_info4 struct is how the server says whether the before and after change attributes were obtained atomically with respect to the file object creation. Very detective energy 🕵️♀️
If objname has a length of 0 (zero), or if objname doesn't follow the UTF-8 definition, the server returns NFS4ERR_INVAL.
Finally, the current filehandle gets replaced by the filehandle of the new object 🔄
IMPLEMENTATION
If the client wants to set attribute values after the create, a SETATTR operation can be added to the COMPOUND request so the right attributes get set. Easy glow-up combo 💖
ERRORS
NFS4ERR_ACCES
NFS4ERR_BADHANDLE
NFS4ERR_BADTYPE
NFS4ERR_DQUOT
NFS4ERR_EXIST
NFS4ERR_FHEXPIRED
NFS4ERR_INVAL
NFS4ERR_IO
NFS4ERR_MOVED
NFS4ERR_NAMETOOLONG
NFS4ERR_NOFILEHANDLE
NFS4ERR_NOSPC
NFS4ERR_NOTDIR
NFS4ERR_NOTSUPP
NFS4ERR_RESOURCE
NFS4ERR_ROFS
NFS4ERR_SERVERFAULT
NFS4ERR_STALE
NFS4ERR_WRONGSEC14.2.5. Operation 7: DELEGPURGE - Purge Delegations Awaiting Recovery 🧹
SYNOPSIS
clientid ->ARGUMENT
struct DELEGPURGE4args {
clientid4 clientid;
};RESULT
struct DELEGPURGE4res {
nfsstat4 status;
};DESCRIPTION
DELEGPURGE clears out all of the delegations awaiting recovery for a given client 🧼 It's super useful for clients that don't commit delegation information to stable storage. It tells the server that conflicting requests don't need to be held up while it waits for delegation info to be recovered 🙌
This operation should be used by clients that record delegation information on stable storage on the client. In that case, DELEGPURGE should be issued immediately after doing delegation recovery on all delegations known to the client. That tells the server no more delegations for this client will be recovered, so it can free resources and stop delaying other clients whose requests conflict with the unrecovered delegations 🌷 The set of delegations known to the server and the set known to the client may be different. Why? A client might fail after making a request that resulted in a delegation, but before it got the results and committed them to its stable storage. Plot twist, but it happens 😭
ERRORS
NFS4ERR_RESOURCE
NFS4ERR_SERVERFAULT
NFS4ERR_STALE_CLIENTID14.2.6. Operation 8: DELEGRETURN - Return Delegation 🦋
SYNOPSIS
stateid ->ARGUMENT
struct DELEGRETURN4args {
stateid4 stateid;
};RESULT
struct DELEGRETURN4res {
nfsstat4 status;
};DESCRIPTION
DELEGRETURN hands back the delegation represented by the given stateid. Pass it back, and it's gone 💅
ERRORS
NFS4ERR_BAD_STATEID
NFS4ERR_OLD_STATEID
NFS4ERR_RESOURCE
NFS4ERR_SERVERFAULT
NFS4ERR_STALE_STATEID14.2.7. Operation 9: GETATTR - Get Attributes ✨
SYNOPSIS
(cfh), attrbits -> attrbits, attrvalsARGUMENT
struct GETATTR4args {
/* CURRENT_FH: directory or file */
bitmap4 attr_request;
};RESULT
struct GETATTR4resok {
fattr4 obj_attributes;
};
union GETATTR4res switch (nfsstat4 status) {
case NFS4_OK:
GETATTR4resok resok4;
default:
void;
};DESCRIPTION
Okay, GETATTR is the operation where the client asks, "hey, what's the deal with this file or directory?" 🧚 The current filehandle points at the object, and the client flips a bit in the bitmap argument for every attribute it wants back. The server replies with its own attribute bitmap, showing which attributes it actually managed to return, and then the attribute values themselves, ordered lowest attribute number first. Iconic, right? 💅
Here's the tea. If the server supports an attribute, it MUST return a value for it whenever the client asks. If the server doesn't support an attribute, or can't come up with a useful approximation, it MUST NOT return that value and MUST NOT set the matching bit in the result bitmap. And if the server supports an attribute but can't actually get its value, it MUST return an error, and in that case no attribute values come back at all. No half-baked responses, periodt 🙅♀️
Every server MUST support the mandatory attributes listed in the section "File Attributes" 🎀
On success, the current filehandle keeps its value 🦋
IMPLEMENTATION
ERRORS
NFS4ERR_ACCES
NFS4ERR_BADHANDLE
NFS4ERR_DELAY
NFS4ERR_FHEXPIRED
NFS4ERR_INVAL
NFS4ERR_IO
NFS4ERR_MOVED
NFS4ERR_NOFILEHANDLE
NFS4ERR_RESOURCE
NFS4ERR_SERVERFAULT
NFS4ERR_STALE
NFS4ERR_WRONGSEC14.2.8. Operation 10: GETFH - Get Current Filehandle 🌸
SYNOPSIS
(cfh) -> filehandleARGUMENT
/* CURRENT_FH: */
void;RESULT
struct GETFH4resok {
nfs_fh4 object;
};
union GETFH4res switch (nfsstat4 status) {
case NFS4_OK:
GETFH4resok resok4;
default:
void;
};DESCRIPTION
This one is simple and sweet 🍓 GETFH returns the current filehandle value.
On success, the current filehandle keeps its value.
IMPLEMENTATION
Ngl, this is a sneaky one. Operations that change the current filehandle, like LOOKUP or CREATE, don't automatically hand the new filehandle back as a result. So if a client wants to look up a directory entry and get its filehandle, it needs to send a little chain of requests, like this:
PUTFH (directory filehandle)
LOOKUP (entry name)
GETFHMain character move: set the directory, find the entry, then ask for the handle 👑
ERRORS
NFS4ERR_BADHANDLE
NFS4ERR_FHEXPIRED
NFS4ERR_MOVED
NFS4ERR_NOFILEHANDLE
NFS4ERR_RESOURCE
NFS4ERR_SERVERFAULT
NFS4ERR_STALE
NFS4ERR_WRONGSEC14.2.9. Operation 11: LINK - Create Link to a File 🔗
SYNOPSIS
(sfh), (cfh), newname -> (cfh), change_infoARGUMENT
struct LINK4args {
/* SAVED_FH: source object */
/* CURRENT_FH: target directory */
component4 newname;
};RESULT
struct LINK4resok {
change_info4 cinfo;
};
union LINK4res switch (nfsstat4 status) {
case NFS4_OK:
LINK4resok resok4;
default:
void;
};DESCRIPTION
LINK makes an extra name, newname, for the file that the saved filehandle points at (that's whatever SAVEFH stored). The new name goes in the directory the current filehandle points at. The existing file and the target directory MUST reside within the same file system on the server. On success, the current filehandle is still the target directory 📁✨
For the target directory, the server sends back change_info4 information in cinfo. The atomic field of the change_info4 struct tells the client whether the before and after change attributes were obtained atomically with respect to the link creation. Very detail-oriented, and I'm here for it 🤓
If newname has a length of 0 (zero), or if newname does not obey the UTF-8 definition, the server returns the error NFS4ERR_INVAL. Big no-no 🙅♀️
IMPLEMENTATION
Changes to any property of the "hard" linked files show up in all of the linked files. Like, they're all the same file, bestie 🫶 When a link is made to a file, the attributes for that file SHOULD have a value for numlinks that is one greater than the value before the LINK operation.
The comments under RENAME about the object and target residing on the same file system apply here too. The comments about the target name apply as well.
Note that symbolic links are created with the CREATE operation, not LINK. Keep that in your back pocket 🧠
ERRORS
NFS4ERR_ACCES NFS4ERR_BADHANDLE NFS4ERR_DELAY NFS4ERR_DQUOT
NFS4ERR_EXIST NFS4ERR_FHEXPIRED NFS4ERR_INVAL NFS4ERR_IO
NFS4ERR_ISDIR NFS4ERR_MLINK NFS4ERR_MOVED NFS4ERR_NAMETOOLONG
NFS4ERR_NOFILEHANDLE NFS4ERR_NOSPC NFS4ERR_NOTDIR NFS4ERR_NOTSUPP
NFS4ERR_RESOURCE NFS4ERR_ROFS NFS4ERR_SERVERFAULT NFS4ERR_STALE
NFS4ERR_WRONGSEC NFS4ERR_XDEV14.2.10. Operation 12: LOCK - Create Lock 🔐
SYNOPSIS
(cfh) type, seqid, reclaim, stateid, offset, length -> stateid,
accessARGUMENT
enum nfs4_lock_type {
READ_LT = 1,
WRITE_LT = 2,
READW_LT = 3, /* blocking read */
WRITEW_LT = 4 /* blocking write */ };
struct LOCK4args {
/* CURRENT_FH: file */
nfs_lock_type4 locktype;
seqid4 seqid;
bool reclaim;
stateid4 stateid;
offset4 offset;
length4 length; };RESULT
struct LOCK4denied {
nfs_lockowner4 owner;
offset4 offset;
length4 length; };
union LOCK4res switch (nfsstat4 status) {
case NFS4_OK:
stateid4 stateid;
case NFS4ERR_DENIED:
LOCK4denied denied;
default:
void; };DESCRIPTION
LOCK is how the client asks for a record lock on a byte range, which is set by the offset and length parameters. The lock type is one of the nfs4_lock_types, so you get a read, write, or a blocking version of either. If this is a reclaim request, the reclaim parameter will be TRUE;
Big flex: bytes can be locked even if they aren't currently allocated to the file 💪 To lock everything from a specific offset through the end-of-file, no matter how long the file actually is, use a length with all bits set to 1 (one). To lock the entire file, use an offset of 0 (zero) and a length with all bits set to 1. A length of 0 is reserved, so it SHOULD NOT be used 🙅♀️
If the lock gets denied, the server sends back the owner, offset, and length of the conflicting lock. Receipts, bestie 🧾
On success, the current filehandle keeps its value 🦋
IMPLEMENTATION
If the server can't work out the exact offset and length of the conflicting lock, the same offset and length that were provided in the arguments SHOULD be returned in the denied results. The File Locking section has the full scoop on this and the other file locking operations 📖
ERRORS
NFS4ERR_ACCES NFS4ERR_BADHANDLE NFS4ERR_BAD_SEQID
NFS4ERR_BAD_STATEID NFS4ERR_DELAY NFS4ERR_DENIED NFS4ERR_EXPIRED
NFS4ERR_FHEXPIRED NFS4ERR_GRACE NFS4ERR_INVAL NFS4ERR_ISDIR
NFS4ERR_LEASE_MOVED NFS4ERR_LOCK_RANGE NFS4ERR_MOVED
NFS4ERR_NOFILEHANDLE NFS4ERR_OLD_STATEID NFS4ERR_RESOURCE
NFS4ERR_SERVERFAULT NFS4ERR_STALE NFS4ERR_STALE_CLIENTID
NFS4ERR_STALE_STATEID NFS4ERR_WRONGSEC14.2.11. Operation 13: LOCKT - Test For Lock 👀
SYNOPSIS
(cfh) type, owner, offset, length -> {void, NFS4ERR_DENIED ->
owner}ARGUMENT
struct LOCKT4args {
/* CURRENT_FH: file */
nfs_lock_type4 locktype;
nfs_lockowner4 owner;
offset4 offset;
length4 length; };RESULT
union LOCKT4res switch (nfsstat4 status) {
case NFS4ERR_DENIED:
LOCK4denied denied;
case NFS4_OK:
void;
default:
void; };DESCRIPTION
LOCKT is the "just checking" operation 🕵️♀️ It tests the lock described in the arguments. If a conflicting lock exists, the owner, offset, and length of that conflicting lock come back. If no lock is held, nothing other than NFS4_OK is returned.
On success, the current filehandle keeps its value.
IMPLEMENTATION
If the server can't work out the exact offset and length of the conflicting lock, the same offset and length that were provided in the arguments SHOULD be returned in the denied results. The File Locking section has more on the file locking mechanisms 🔍
LOCKT uses nfs_lockowner4 instead of a stateid4, the way LOCK does, to identify the owner. That way the client doesn't have to open the file just to check whether a lock exists. Efficient and iconic 🦋
ERRORS
NFS4ERR_ACCES
NFS4ERR_BADHANDLE
NFS4ERR_DELAY
NFS4ERR_DENIED
NFS4ERR_FHEXPIRED
NFS4ERR_GRACE
NFS4ERR_INVAL
NFS4ERR_ISDIR
NFS4ERR_LEASE_MOVED
NFS4ERR_LOCK_RANGE
NFS4ERR_MOVED
NFS4ERR_NOFILEHANDLE
NFS4ERR_RESOURCE
NFS4ERR_SERVERFAULT
NFS4ERR_STALE
NFS4ERR_STALE_CLIENTID
NFS4ERR_WRONGSEC14.2.12. Operation 14: LOCKU - Unlock File 🔓
SYNOPSIS
(cfh) type, seqid, stateid, offset, length -> stateidARGUMENT
struct LOCKU4args {
/* CURRENT_FH: file */
nfs_lock_type4 locktype;
seqid4 seqid;
stateid4 stateid;
offset4 offset;
length4 length;
};RESULT
union LOCKU4res switch (nfsstat4 status) {
case NFS4_OK:
stateid4 stateid;
default:
void;
};DESCRIPTION
LOCKU is the unlock move 💅 It releases the record lock specified by the parameters.
On success, the current filehandle keeps its value.
IMPLEMENTATION
The File Locking section has a full breakdown of this and the other file locking procedures 🌈
ERRORS
NFS4ERR_ACCES
NFS4ERR_BADHANDLE
NFS4ERR_BAD_SEQID
NFS4ERR_BAD_STATEID
NFS4ERR_EXPIRED
NFS4ERR_FHEXPIRED
NFS4ERR_GRACE
NFS4ERR_INVAL
NFS4ERR_LOCK_RANGE
NFS4ERR_LEASE_MOVED
NFS4ERR_MOVED
NFS4ERR_NOFILEHANDLE
NFS4ERR_OLD_STATEID
NFS4ERR_RESOURCE
NFS4ERR_SERVERFAULT
NFS4ERR_STALE
NFS4ERR_STALE_CLIENTID
NFS4ERR_STALE_STATEID14.2.13. Operation 15: LOOKUP - Lookup Filename ✨
SYNOPSIS 💖
(cfh), filenames -> (cfh)ARGUMENT 🎀
struct LOOKUP4args {
/* CURRENT_FH: directory */
pathname4 path;
};RESULT 🌸
struct LOOKUP4res {
/* CURRENT_FH: object */
nfsstat4 status;
};DESCRIPTION 🦋
okay so LOOKUP is how you find a file system object, starting from the directory your current filehandle points at. it walks through the pathname (that array of names) and the new current filehandle comes from the final name. all but the final name in the list must be the names of directories, bestie, so you can't have a regular file hiding in the middle of your path 🙅♀️
if the path can't be evaluated, that's an error, and your current filehandle stays exactly where it was. this happens if a component doesn't exist, or if the client doesn't have permission to evaluate a component of the path. no drama, no changes 💅
if the path is a zero length array, if any component doesn't obey the UTF-8 definition, or if any component in the path has zero length, you get NFS4ERR_INVAL. that's the server going "nope, that's not it" 😭
IMPLEMENTATION ✨
if the client would rather do a partial evaluation of the path, you can swap in a sequence of LOOKUP operations, like this:
PUTFH (directory filehandle)
LOOKUP "pub" "foo" "bar"
GETFHor, if the client wants to grab the intermediate filehandles along the way:
PUTFH (directory filehandle)
LOOKUP "pub"
GETFH
LOOKUP "foo"
GETFH
LOOKUP "bar"
GETFHnow here's the lowkey iconic part 👑. NFS version 4 servers break from the semantics of previous NFS versions, because LOOKUP requests are allowed to cross mountpoints on the server. the client can spot a mountpoint crossing by comparing the fsid attribute of the directory with the fsid attribute of the directory it looked up. if the fsids are different, the new directory is a server mountpoint. Unix clients that detect a mountpoint crossing will need to mount the server's filesystem, because that's how they keep the file object identity checking mechanisms they rely on, common to Unix clients 🧚
servers that limit NFS access to "shares" or "exported" filesystems should provide a pseudo-filesystem where the exported filesystems can be integrated, so clients can browse the server's name space 🌈. a client's view of a pseudo filesystem is limited to paths that lead to exported filesystems.
Note: previous versions of the protocol gave special meaning to the names "." and "..". NFS version 4 assigns no special semantics to these names. the LOOKUPP operator must be used to lookup a parent directory 🙈
also, this procedure does not follow symbolic links. the client is responsible for all parsing of filenames, including filenames that get modified by symbolic links it runs into during the lookup process. that's a lot of responsibility, but also a lot of power 💪
if the current file handle is not a directory but a symbolic link, the error NFS4ERR_SYMLINK is returned. for all other non-directory file types, the error NFS4ERR_NOTDIR is returned 🍓
ERRORS 💀
NFS4ERR_ACCESNFS4ERR_BADHANDLENFS4ERR_FHEXPIREDNFS4ERR_INVALNFS4ERR_IONFS4ERR_MOVEDNFS4ERR_NAMETOOLONGNFS4ERR_NOENTNFS4ERR_NOFILEHANDLENFS4ERR_NOTDIRNFS4ERR_RESOURCENFS4ERR_SERVERFAULTNFS4ERR_STALENFS4ERR_SYMLINKNFS4ERR_WRONGSEC
14.2.14. Operation 16: LOOKUPP - Lookup Parent Directory 🌷
SYNOPSIS 🌸
(cfh) -> (cfh)ARGUMENT 🎀
/* CURRENT_FH: object */
void;RESULT 🦋
struct LOOKUPP4res {
/* CURRENT_FH: directory */
nfsstat4 status;
};DESCRIPTION ✨
the current filehandle is assumed to refer to a regular directory or a named attribute directory. LOOKUPP sets the filehandle for its parent directory as the current filehandle. if there is no parent directory, an NFS4ERR_ENOENT error must be returned 😭. so the server will return NFS4ERR_ENOENT when the current filehandle is at the root or top of the server's file tree. it's giving "you've reached the top, sweetie" 🫶
IMPLEMENTATION 💅
as for LOOKUP, LOOKUPP will also cross mountpoints 🌈
if the current filehandle is not a directory or named attribute directory, the error NFS4ERR_NOTDIR is returned 🍓
ERRORS 💀
NFS4ERR_ACCESNFS4ERR_BADHANDLENFS4ERR_FHEXPIREDNFS4ERR_INVALNFS4ERR_IONFS4ERR_MOVEDNFS4ERR_NOENTNFS4ERR_NOFILEHANDLENFS4ERR_NOTDIRNFS4ERR_RESOURCENFS4ERR_SERVERFAULTNFS4ERR_STALENFS4ERR_WRONGSEC
14.2.15. Operation 17: NVERIFY - Verify Difference in Attributes 🔥
SYNOPSIS 🧚
(cfh), fattr -> -ARGUMENT 💖
struct NVERIFY4args {
/* CURRENT_FH: object */
fattr4 obj_attributes;
};RESULT 🎀
struct NVERIFY4res {
nfsstat4 status;
};DESCRIPTION 🌈
this operation is used to prefix a sequence of operations that get performed if one or more attributes have changed on some filesystem object. if all the attributes match, then the error NFS4ERR_SAME must be returned 🙈
on success, the current filehandle keeps its value ✨
IMPLEMENTATION 🦋
this operation is super useful as a cache validation operator 🧠. if the object the attributes belong to has changed, the following operations can get new data for that object. for example, to check whether a file has changed and grab new data if it has:
PUTFH (public)
LOOKUP "pub" "foo" "bar"
NVERIFY attrbits attrs
READ 0 32767and if a recommended attribute is specified in the NVERIFY operation but the server doesn't support that attribute for the file system object, the error NFS4ERR_NOTSUPP is returned to the client 🥲
ERRORS 💀
NFS4ERR_ACCESNFS4ERR_BADHANDLENFS4ERR_DELAYNFS4ERR_FHEXPIREDNFS4ERR_INVALNFS4ERR_IONFS4ERR_MOVEDNFS4ERR_NOFILEHANDLENFS4ERR_NOTSUPPNFS4ERR_RESOURCENFS4ERR_SAMENFS4ERR_SERVERFAULTNFS4ERR_STALENFS4ERR_WRONGSEC
14.2.16. Operation 18: OPEN - Open a Regular File 🌟
SYNOPSIS 👑
(cfh), claim, openhow, owner, seqid, access, deny -> (cfh),
stateid, cinfo, rflags, open_confirm, delegationARGUMENT 🎀
struct OPEN4args {
open_claim4 claim;
openflag4 openhow;
nfs_lockowner4 owner;
seqid4 seqid;
uint32_t share_access;
uint32_t share_deny;
};
enum createmode4 {
UNCHECKED4 = 0,
GUARDED4 = 1,
EXCLUSIVE4 = 2
};
union createhow4 switch (createmode4 mode) {
case UNCHECKED4:
case GUARDED4:
fattr4 createattrs;
case EXCLUSIVE4:
verifier4 createverf;
};
enum opentype4 {
OPEN4_NOCREATE = 0,
OPEN4_CREATE = 1
};
union openflag4 switch (opentype4 opentype) {
case OPEN4_CREATE:
createhow4 how;
default:
void;
};
/* Next definitions used for OPEN delegation */
enum limit_by4 {
NFS_LIMIT_SIZE = 1,
NFS_LIMIT_BLOCKS = 2
/* others as needed */
};
struct nfs_modified_limit4 {
uint32_t num_blocks;
uint32_t bytes_per_block;
};
union nfs_space_limit4 switch (limit_by4 limitby) {
/* limit specified as file size */
case NFS_LIMIT_SIZE:
uint64_t filesize;
/* limit specified by number of blocks */
case NFS_LIMIT_BLOCKS:
nfs_modified_limit4 mod_blocks;
} ;
enum open_delegation_type4 {
OPEN_DELEGATE_NONE = 0,
OPEN_DELEGATE_READ = 1,
OPEN_DELEGATE_WRITE = 2
};
enum open_claim_type4 {
CLAIM_NULL = 0,
CLAIM_PREVIOUS = 1,
CLAIM_DELEGATE_CUR = 2,
CLAIM_DELEGATE_PREV = 3
};
struct open_claim_delegate_cur4 {
pathname4 file;
stateid4 delegate_stateid;
};
union open_claim4 switch (open_claim_type4 claim) {
/*
* No special rights to file. Ordinary OPEN of the specified file.
*/
case CLAIM_NULL:
/* CURRENT_FH: directory */
pathname4 file;
/*
* Right to the file established by an open previous to server
* reboot. File identified by filehandle obtained at that time
* rather than by name.
*/
case CLAIM_PREVIOUS:
/* CURRENT_FH: file being reclaimed */
uint32_t delegate_type;
/*
* Right to file based on a delegation granted by the server.
* File is specified by name.
*/
case CLAIM_DELEGATE_CUR:
/* CURRENT_FH: directory */
open_claim_delegate_cur4 delegate_cur_info;
/* Right to file based on a delegation granted to a previous boot
* instance of the client. File is specified by name.
*/
case CLAIM_DELEGATE_PREV:
/* CURRENT_FH: directory */
pathname4 file_delegate_prev;
};RESULT 🌸
struct open_read_delegation4 {
stateid4 stateid; /* Stateid for delegation*/
bool recall; /* Pre-recalled flag for
delegations obtained
by reclaim
(CLAIM_PREVIOUS) */
nfsace4 permissions; /* Defines users who don't
need an ACCESS call to
open for read */
};
struct open_write_delegation4 {
stateid4 stateid; /* Stateid for delegation*/
bool recall; /* Pre-recalled flag for
delegations obtained
by reclaim
(CLAIM_PREVIOUS) */
nfs_space_limit4 space_limit; /* Defines condition that
the client must check to
determine whether the
file needs to be flushed
to the server on close.
*/
nfsace4 permissions; /* Defines users who don't
need an ACCESS call as
part of a delegated
open. */
};
union open_delegation4
switch (open_delegation_type4 delegation_type) {
case OPEN_DELEGATE_NONE:
void;
case OPEN_DELEGATE_READ:
open_read_delegation4 read;
case OPEN_DELEGATE_WRITE:
open_write_delegation4 write;
};
const OPEN4_RESULT_MLOCK = 0x00000001;
const OPEN4_RESULT_CONFIRM= 0x00000002;
struct OPEN4resok {
stateid4 stateid; /* Stateid for open */
change_info4 cinfo; /* Directory Change Info */
uint32_t rflags; /* Result flags */
verifier4 open_confirm; /* OPEN_CONFIRM verifier */
open_delegation4 delegation; /* Info on any open
delegation */
}; union OPEN4res switch (nfsstat4 status) {
case NFS4_OK:
/* CURRENT_FH: opened file */
OPEN4resok resok4;
default:
void;
};WARNING TO CLIENT IMPLEMENTORS 🚨
Okay bestie, listen up 👂 OPEN is kind of like LOOKUP because it also hands you a filehandle to use 🦋 BUT the big difference is that OPEN creates server state on that filehandle, and that's where it gets dramatic 😭 Normally the only way to let that state go is a CLOSE, and CLOSE figures out which file to close by looking at the current filehandle. So the client MUST follow every OPEN operation with a GETFH operation in the same COMPOUND procedure 💅 That gives CLOSE the filehandle it needs to work. No skipping!
Just waiting for the lease on the file to expire is not enough, ngl 💀 The server can keep that state around indefinitely as long as no other client tries to make a conflicting access to the same file.
DESCRIPTION 📝
Okay so OPEN is the operation that creates and/or opens a regular file in a directory using the name you give it 🎀 If the file doesn't exist on the server and you want it created, the openhow parameter is where you spell out how to create it. You get to pick from three creation methods: UNCHECKED, GUARDED, or EXCLUSIVE ✨
UNCHECKED is the chill option 🌸 It creates the file if nothing with that name exists, and running into an existing regular file with that name is not an error. For this one, createattrs sets the starting attributes for the file. The attributes can include any writable attribute that's valid for regular files. If the file already exists, createattrs isn't used, except that when an object_size of zero is specified, the existing file gets truncated ✂️
GUARDED is more of a bouncer 🙅♀️ The server checks for a duplicate object by name before doing the create. If a duplicate exists, you get NFS4ERR_EXIST as the status. If the object doesn't exist, the request gets handled just like UNCHECKED.
EXCLUSIVE is the serious one 🔐 The server follows exclusive creation semantics and uses the verifier to make sure the target is created exclusively. The server should check for a duplicate object by name. If the object doesn't exist, the server creates it and stores the verifier with it. If the object does exist and the stored verifier matches the verifier the client sent, the server uses the existing object as the newly created one 🥰 If the stored verifier doesn't match, you get NFS4ERR_EXIST. No attributes may be provided in this case, since the server may use an attribute of the target object to store the verifier.
For the target directory, the server returns change_info4 information in cinfo 📦 With the atomic field of the change_info4 struct, the server indicates whether the before and after change attributes were obtained atomically with respect to the link creation 🔗
Upon successful creation, the current filehandle gets swapped out for the new object's 🔄
OPEN also gives you DOS SHARE capability through the access and deny fields of the OPEN arguments 🪟 You tell it the access and deny modes you need up front. For clients that don't directly support SHAREs (hi, Unix 👋), the expected deny value is DENY_NONE. If there's an existing SHARE reservation that clashes with your OPEN request, the server hands back NFS4ERR_DENIED. For a complete SHARE request, you have to provide values for the owner and seqid fields of the OPEN argument. For more on SHARE semantics, check the section on 'Share Reservations' 📚
If the client is recovering state after a server failure, the reclaim field of the OPEN argument says the request is trying to reclaim state that was held before 🔁
The claim field of the OPEN argument says which file to open and what state the client claims to already have 🧾 There are four basic claim types that cover the different OPEN situations:
CLAIM_NULL: a brand new OPEN request, with no previous state for this file on this client 🌱CLAIM_PREVIOUS: the client is claiming basic OPEN state for a file it held before a server reboot. This is usually used when the server returns persistent filehandles, since the client may not have the filename to reclaim the OPEN with 🔙CLAIM_DELEGATE_CUR: the client is claiming a delegation the server granted for OPEN. This usually happens as part of recalling a delegation 📞CLAIM_DELEGATE_PREV: the client is claiming a delegation granted to a previous client instance, used after the client reboots 🥾
For OPEN requests whose claim type is other than CLAIM_PREVIOUS (basically anything besides reclaiming opens after a server reboot) that reach the server during its grace or lease expiration period, the server returns NFS4ERR_GRACE ⏳
For any OPEN request, the server may return an open delegation, which lets further opens and closes get handled locally on the client, as described in the section Open Delegation 🏠 Delegation is totally up to the server to decide, so the client should never assume it will or won't be granted. Always be ready for either case! A partial exception is the reclaim (CLAIM_PREVIOUS) case, where a delegation type is claimed. In that case, delegation will always be granted, although the server may specify an immediate recall in the delegation structure 🙃
The rflags returned by a successful OPEN let the server tell you how the open file should be handled 🎁 OPEN4_RESULT_MLOCK tells the caller that mandatory locking is in effect for this file, so the client should act appropriately with regard to data cached on the client 🔒 OPEN4_RESULT_CONFIRM means the client MUST execute an OPEN_CONFIRM operation before using the open file ⚠️
If the file is a zero length array, if any component doesn't obey the UTF-8 definition, or if any component in the path is of zero length, you get NFS4ERR_INVAL 🙅
When an OPEN is done and the specified lockowner already has the resulting filehandle open, the result is to "OR" together the new share and deny status with the existing status 🧩 In this case, only a single CLOSE needs to be done, even though multiple OPEN's were completed.
IMPLEMENTATION 🛠️
The OPEN procedure supports EXCLUSIVE create 🎉 The mechanism is similar to the support in NFS version 3 [RFC1813], and just like there, it provides reliable exclusive creation ✅ Exclusive create is invoked when the how parameter is EXCLUSIVE. In this case, the client provides a verifier that can reasonably be expected to be unique. A combination of a client identifier, perhaps the client network address, and a unique number generated by the client, perhaps the RPC transaction identifier, may be appropriate.
If the object doesn't exist, the server creates it and stores the verifier in stable storage 💾 For file systems that don't provide a mechanism for storing arbitrary file attributes, the server may use one or more elements of the object meta-data to store the verifier. The verifier must be stored in stable storage to prevent erroneous failure on retransmission of the request ‼️ It's assumed that an exclusive create is being performed because exclusive semantics are critical to the application. Because of that expected usage, exclusive CREATE doesn't rely solely on the normally volatile duplicate request cache for storing the verifier 🥶 The duplicate request cache in volatile storage doesn't survive a crash and may actually flush on a long network partition, which opens up failure windows. In the UNIX local file system environment, the expected storage location for the verifier on creation is the meta-data (time stamps) of the object. For this reason, an exclusive object create may not include initial attributes, because the server would have nowhere to store the verifier 🕰️
If the server can't support these exclusive create semantics, possibly because of the requirement to commit the verifier to stable storage, it should fail the OPEN request with the error NFS4ERR_NOTSUPP ❌
During an exclusive CREATE request, if the object already exists, the server reconstructs the object's verifier and compares it with the verifier in the request. If they match, the server treats the request as a success. The request is presumed to be a duplicate of an earlier, successful request whose reply was lost, and that the server's duplicate request cache mechanism didn't detect 🔁 If the verifiers don't match, the request is rejected with the status NFS4ERR_EXIST.
Once the client has performed a successful exclusive create, it must issue a SETATTR to set the correct object attributes. Until it does so, it should not rely upon any of the object attributes, since the server implementation may need to overload object meta-data to store the verifier. The subsequent SETATTR must not occur in the same COMPOUND request as the OPEN. This separation will guarantee that the exclusive create mechanism keeps functioning properly in the face of retransmission of the request 🧠
Use of the GUARDED attribute doesn't provide exactly-once semantics 🙃 In particular, if a reply is lost and the server doesn't detect the retransmission of the request, the procedure can fail with NFS4ERR_EXIST, even though the create was performed successfully.
For SHARE reservations, the client must specify a value for access that is one of READ, WRITE, or BOTH. For deny, the client must specify one of NONE, READ, WRITE, or BOTH. If the client fails to do this, the server must return NFS4ERR_INVAL 📏
If the final component provided to OPEN is a symbolic link, the error NFS4ERR_SYMLINK will be returned to the client. If an intermediate component of the pathname provided to OPEN is a symbolic link, the error NFS4ERR_NOTDIR will be returned to the client 🔗
ERRORS 💥
NFS4ERR_ACCES
NFS4ERR_BAD_SEQID
NFS4ERR_DELAY
NFS4ERR_DQUOT
NFS4ERR_EXIST
NFS4ERR_FHEXPIRED
NFS4ERR_GRACE
NFS4ERR_IO
NFS4ERR_ISDIR
NFS4ERR_LEASE_MOVED
NFS4ERR_MOVED
NFS4ERR_NAMETOOLONG
NFS4ERR_NOFILEHANDLE
NFS4ERR_NOSPC
NFS4ERR_NOTDIR
NFS4ERR_NOTSUPP
NFS4ERR_RESOURCE
NFS4ERR_ROFS
NFS4ERR_SERVERFAULT
NFS4ERR_SHARE_DENIED
NFS4ERR_STALE_CLIENTID
NFS4ERR_SYMLINK14.2.17. Operation 19: OPENATTR - Open Named Attribute Directory ✨
SYNOPSIS
(cfh) -> (cfh)ARGUMENT
/* CURRENT_FH: file or directory */
void;RESULT
struct OPENATTR4res {
/* CURRENT_FH: name attr directory*/
nfsstat4 status;
};DESCRIPTION
okay so OPENATTR is the op you use to grab the filehandle for the named attribute directory tied to your current filehandle 📂 it's giving "folder of extra stuff hiding behind the file." The result is a filehandle pointing at an object of type NF4ATTRDIR. Once you've got that, you can run READDIR and LOOKUP on it to get filehandles for each of the named attributes on the original file system object 💅 And any filehandles that come back from inside that named attribute directory are typed as NF4NAMEDATTR. Iconic structure, no notes 🫶
IMPLEMENTATION
if the server doesn't support named attributes for the current filehandle, the client gets an error of NFS4ERR_NOTSUPP back. Rude, but valid 💀
ERRORS
NFS4ERR_ACCES
NFS4ERR_BADHANDLE
NFS4ERR_DELAY
NFS4ERR_FHEXPIRED
NFS4ERR_INVAL
NFS4ERR_IO
NFS4ERR_MOVED
NFS4ERR_NOENT
NFS4ERR_NOFILEHANDLE
NFS4ERR_NOTSUPP
NFS4ERR_RESOURCE
NFS4ERR_SERVERFAULT
NFS4ERR_STALE
NFS4ERR_WRONGSEC14.2.18. Operation 20: OPEN_CONFIRM - Confirm Open 🎀
SYNOPSIS
(cfh), seqid, open_confirm-> stateidARGUMENT
struct OPEN_CONFIRM4args {
/* CURRENT_FH: opened file */
seqid4 seqid;
verifier4 open_confirm; /* OPEN_CONFIRM verifier */
};RESULT
struct OPEN_CONFIRM4resok {
stateid4 stateid;
};
union OPEN_CONFIRM4res switch (nfsstat4 status) {
case NFS4_OK:
OPEN_CONFIRM4resok resok4;
default:
void;
};DESCRIPTION
this op is how you confirm sequence id usage the first time a nfs_lockowner shows up from a client 🦋 The OPEN op hands back an opaque confirmation verifier, and you pass that verifier into this op along with the next seqid for the nfs_lockowner. The seqid you send to OPEN_CONFIRM must be exactly one more than the seqid you sent to the OPEN that gave you the open_confirm value. If the server gets a seqid that doesn't line up with the original open, it assumes the client isn't going to confirm that OPEN, and it releases all the state tied to the original OPEN. Yikes, but make sense 😭
On success, the current filehandle keeps its value ✨
IMPLEMENTATION
a single client can make a ton of nfs_lockowner structures for one clientid. The client will eventually either throw them out or just stop using them for ages 🧹 That's exactly why NFS version 4 doesn't have an explicit op to exit an nfs_lockowner, since that op would be useless in that situation. So instead, to keep memory from growing forever, the server needs a plan for getting rid of nfs_lockowners that have no current lock, open, or delegation state on any file and haven't been used in a while. How long the server waits before cleaning up is up to the implementation 🌸 It really should be at least the lease time plus whatever grace period the server adds on top of the lease. OPEN_CONFIRM is what lets the server safely toss out nfs_lockowner data structures that aren't in use.
If a client sends an OPEN and the server has no record of that nfs_lockowner, the server has to make sure this is a brand new OPEN and not a replay or a retransmission 👀
A lazy server might ask for confirmation for every nfs_lockowner it has no record of. That's not required, though, until the server has actually recorded that it threw away an nfs_lockowner for that clientid 💅
The server must hold onto unconfirmed OPEN state until one of three things happens. First, the client sends an OPEN_CONFIRM with the right seqid and confirmation verifier inside the lease period. When that happens, the OPEN state on the server becomes confirmed, and the nfs_lockowner is fully set up 🎉
Second, the client sends another OPEN with a seqid that's out of sequence for that nfs_lockowner. Here the server assumes the new OPEN is the valid one and the first one was a replay. So it cancels the OPEN state from the first OPEN, sets up unconfirmed OPEN state for the second one, and replies to the second OPEN saying an OPEN_CONFIRM is needed. Then the whole thing starts over. There's a possible denial of service risk on the client side, but it gets softened if the client and server both require a security flavor based on Kerberos V5, LIPKEY, or some other flavor that uses cryptography 🔐
What if the server is in the unconfirmed OPEN state for a given nfs_lockowner, and it gets an operation on that nfs_lockowner that carries a stateid, but the operation isn't OPEN? Or it's OPEN_CONFIRM with the wrong confirmation verifier? Then the server returns NFS4ERR_BAD_STATEID, even if the seqid looks correct. The server treats it as a replay, because if there's no established OPEN state, then a LOCK operation, for example, can't possibly be valid 🙅♀️
Third, neither of the first two things happens for that nfs_lockowner within the lease period. In that case the OPEN state gets cancelled, and the nfs_lockowner can be disposed of 🧚
ERRORS
NFS4ERR_BADHANDLE
NFS4ERR_BAD_SEQID
NFS4ERR_EXPIRED
NFS4ERR_FHEXPIRED
NFS4ERR_GRACE
NFS4ERR_INVAL
NFS4ERR_MOVED
NFS4ERR_NOENT
NFS4ERR_NOFILEHANDLE
NFS4ERR_NOTSUPP
NFS4ERR_RESOURCE
NFS4ERR_SERVERFAULT
NFS4ERR_STALE
NFS4ERR_WRONGSEC14.2.19. Operation 21: OPEN_DOWNGRADE - Reduce Open File Access 🌈
SYNOPSIS
(cfh), stateid, seqid, access, deny -> stateidARGUMENT
struct OPEN_DOWNGRADE4args {
/* CURRENT_FH: opened file */
stateid4 stateid;
seqid4 seqid;
uint32_t share_access;
uint32_t share_deny;
};RESULT
struct OPEN_DOWNGRADE4resok {
stateid4 stateid;
};
union OPEN_DOWNGRADE4res switch(nfsstat4 status) {
case NFS4_OK:
OPEN_DOWNGRADE4resok resok4;
default:
void;
};This op is used to tweak the access and deny bits for a given open 🔧 It comes in handy when one lockowner opens the same file a bunch of times with different access and deny flags. Closing one of those opens might mean some access and deny flags no longer apply, so you use this to strip out the bits for opens that aren't in effect anymore 💖
The access and deny bits you put in this op replace the current ones for that open file. If either the access or the deny mode includes bits that aren't in effect for the open, the server should return the error NFS4ERR_INVAL. And since access and deny bits are just subsets of what's already been granted, this request can't get denied because of conflicting share reservations 🙌
On success, the current filehandle keeps its value ✨
ERRORS
NFS4ERR_BADHANDLE NFS4ERR_BAD_SEQID NFS4ERR_BAD_STATEID
NFS4ERR_EXPIRED NFS4ERR_FHEXPIRED NFS4ERR_INVAL NFS4ERR_MOVED
NFS4ERR_NOFILEHANDLE NFS4ERR_OLD_STATEID NFS4ERR_RESOURCE
NFS4ERR_SERVERFAULT NFS4ERR_STALE NFS4ERR_STALE_STATEID14.2.20. Operation 22: PUTFH - Set Current Filehandle 🍓
SYNOPSIS
filehandle -> (cfh)ARGUMENT
struct PUTFH4args {
nfs4_fh object; };RESULT
struct PUTFH4res {
/* CURRENT_FH: */
nfsstat4 status; };DESCRIPTION
this one swaps out the current filehandle for whatever filehandle you pass in as the argument 🔄
IMPLEMENTATION
it's super common to use this as the very first operator in an NFS request, so it sets the context for everything that follows 🫶
ERRORS
NFS4ERR_BADHANDLE
NFS4ERR_FHEXPIRED
NFS4ERR_MOVED
NFS4ERR_RESOURCE
NFS4ERR_SERVERFAULT
NFS4ERR_STALE
NFS4ERR_WRONGSEC14.2.21. Operation 23: PUTPUBFH - Set Public Filehandle 🌸
SYNOPSIS
- -> (cfh)ARGUMENT
void;RESULT
struct PUTPUBFH4res {
/* CURRENT_FH: public fh */
nfsstat4 status;
};DESCRIPTION
this op replaces the current filehandle with the one that represents the public filehandle of the server's name space 🌍 Heads up, this one might not be the same as the "root" filehandle, which could be tied to some other directory on the server 👀
IMPLEMENTATION
used as the first operator in an NFS request to set the context for the operations that follow 💅
ERRORS
NFS4ERR_RESOURCE
NFS4ERR_SERVERFAULT
NFS4ERR_WRONGSEC14.2.22. Operation 24: PUTROOTFH - Set Root Filehandle 👑
SYNOPSIS
- -> (cfh)ARGUMENT
void;RESULT
struct PUTROOTFH4res {
/* CURRENT_FH: root fh */
nfsstat4 status;
};DESCRIPTION
this op replaces the current filehandle with the one that represents the root of the server's name space 🌳 From that filehandle, a LOOKUP op can find any other filehandle on the server, which is lowkey amazing ✨ This filehandle might be different from the "public" filehandle, which could be tied to some other directory on the server.
IMPLEMENTATION
commonly used as the first operator in an NFS request to set the context for whatever comes next 🧚
ERRORS
NFS4ERR_RESOURCE
NFS4ERR_SERVERFAULT
NFS4ERR_WRONGSEC14.2.23. Operation 25: READ - Read from File 📖
SYNOPSIS
(cfh), offset, count, stateid -> eof, dataARGUMENT
struct READ4args {
/* CURRENT_FH: file */
stateid4 stateid;
offset4 offset;
count4 count;
};RESULT
struct READ4resok {
bool eof;
opaque data<>;
};
union READ4res switch (nfsstat4 status) {
case NFS4_OK:
READ4resok resok4;
default:
void;
};DESCRIPTION
okay so READ is the op that actually pulls bytes out of a regular file, the one the current filehandle points at 💅
the client gives the server an offset, which is where the READ starts, and a count, which is how many bytes it wants. an offset of 0 (zero) means start at the very beginning of the file 🌸 if the offset is greater than or equal to the size of the file, you get NFS4_OK back with a data length of 0 (zero) and eof set to TRUE. and READ is always subject to access permissions checking, no skipping that part 🔒
if the client asks for a count of 0 (zero), the READ still succeeds and returns 0 (zero) bytes, and that is also subject to access permissions checking ✨ the server may choose to hand back fewer bytes than the client asked for. so the client needs to check for this and handle it properly, bestie 🫶
the stateid for a READ is the value that came back from an earlier record lock or share reservation request. the server uses it to check that the lock is still valid, and it also uses it to update the client's lease timeouts ⏱️
now eof. it comes back as TRUE if the read ended right at the end-of-file. formally, in a correctly formed READ request, that means offset + count is equal to the size of the file. it also comes back as TRUE if the read goes past the end, meaning offset + count is greater than the size of the file. otherwise it is FALSE. and a successful READ of an empty file will always give you eof as TRUE 🥹
on success, the current filehandle keeps its value. no change there 🎀
IMPLEMENTATION
the server is allowed to return fewer than count bytes, and this is a real thing that happens 👀 if the server returns less than the count requested and eof is FALSE, the client should send another READ to grab the rest of the data 🔁
there are a few reasons a server might return less. the file could have been truncated by another client, or by the server itself, so the size is not what the requesting client thinks it is. that shrinks the amount of data the client can actually get. the server might also back off the transfer size and return a smaller read because it wants to. and server resource exhaustion can force a smaller read return too 😭
if the file is locked, the server returns an NFS4ERR_LOCKED error. the lock might only be held for a short moment, so the client may choose to retransmit the READ request, using exponential backoff, until the operation finally succeeds 🦋
ERRORS
NFS4ERR_ACCES
NFS4ERR_BADHANDLE
NFS4ERR_BAD_STATEID
NFS4ERR_DELAY
NFS4ERR_DENIED
NFS4ERR_EXPIRED
NFS4ERR_FHEXPIRED
NFS4ERR_GRACE
NFS4ERR_INVAL
NFS4ERR_IO
NFS4ERR_LOCKED
NFS4ERR_LEASE_MOVED
NFS4ERR_MOVED
NFS4ERR_NOFILEHANDLE
NFS4ERR_NXIO
NFS4ERR_OLD_STATEID
NFS4ERR_RESOURCE
NFS4ERR_SERVERFAULT
NFS4ERR_STALE
NFS4ERR_STALE_STATEID
NFS4ERR_WRONGSEC14.2.24. Operation 26: READDIR - Read Directory 📂
SYNOPSIS
(cfh), cookie, cookieverf, dircount, maxcount, attrbits ->
cookieverf { cookie, filename, attrbits, attributes }ARGUMENT
struct READDIR4args {
/* CURRENT_FH: directory */
nfs_cookie4 cookie;
verifier4 cookieverf;
count4 dircount;
count4 maxcount;
bitmap4 attr_request;
};RESULT
struct entry4 {
nfs_cookie4 cookie;
component4 name;
fattr4 attrs;
entry4 *nextentry;
};
struct dirlist4 {
entry4 *entries;
bool eof;
};
struct READDIR4resok {
verifier4 cookieverf;
dirlist4 reply;
};
union READDIR4res switch (nfsstat4 status) {
case NFS4_OK:
READDIR4resok resok4;
default:
void;
};DESCRIPTION
READDIR is how you browse a directory 🗂️ it grabs a variable number of entries from a file system directory, returns whatever attributes the client asked for on each entry, and also gives back the info needed to ask for more entries in a follow-up READDIR. main character energy 👑
the arguments include a cookie, which tells READDIR where to start in the directory. a cookie of 0 (zero) means start at the beginning. for any later READDIR, the client passes along a cookie that the server gave it on a previous READDIR 🔖
the cookieverf should be set to 0 (zero) when the cookie is 0 (zero), which is the first directory read. on later requests, it should be the cookieverf the server returned. the cookieverf must match the one returned by the READDIR where the cookie was acquired. no mixing and matching 🙅♀️
dircount is a hint for the maximum number of bytes of directory info the server should return. it counts the names of the entries and the cookie values for those entries. this length is the XDR encoding of that data, not the length in the server's native format. the server may return less data 🧮
maxcount is the maximum number of bytes allowed for the whole result. this covers all the data being returned, and it includes the XDR overhead. the server may return less data here too. if the server can't fit even one directory entry inside maxcount, the error NFS4ERR_READDIR_NOSPC gets returned to the client 😬
attrbits is the list of attributes the server should return for each directory entry 📋
on a successful return, the server's response gives a list of directory entries. each entry has the name of the directory entry, a cookie value for that entry, and the associated attributes the client requested 💖
the cookie is only meaningful to the server. it works like a "bookmark" for the directory entry. the client uses it in later READDIR operations so it can keep reading the directory from that spot. it's kind of like a READ offset, but the client should not interpret it as one 🙈 ideally, the cookie value should not change if the directory gets modified, since the client may be caching these values 🧠
sometimes the server runs into an error while getting the attributes for one directory entry. instead of failing the whole READDIR operation, the server can return the attribute 'fattr4_rdattr_error' instead. that way it can tell the client about the failure without failing everything, which is great when the problem is just a transient blip ✨ the client has to request the fattr4_rdattr_error attribute for this to work, though. if the client doesn't ask for it, the server has no choice but to return failure for the entire READDIR operation 😭
the directory entries "." and ".." have special meaning in some file system environments, and no meaning at all in others. if the server supports these special entries within a directory, they should not be returned to the client as part of the READDIR response. to make things work smoothly for some client environments, the cookie values 0, 1, and 2 are reserved. fun fact: the Unix client uses these values when it combines the server's response with its own local representation, so it can build a fully formed Unix directory view for the application 🍓
so for READDIR arguments, cookie values of 1 and 2 should not be used. and for READDIR results, cookie values of 0, 1, and 2 should not be returned.
on success, the current filehandle keeps its value 🌈
IMPLEMENTATION
server directory representations can look really different from each other 🌍 and a client's programming interfaces might be tied to its local operating environment in a way that doesn't map neatly onto NFS. that's why the dircount and maxcount fields exist. they let the client give the server some guidelines. if the client is aggressive about collecting attributes during a READDIR, the server gets a sense of how to limit the encoded response. dircount is a hint about how many entries to return, based only on the entry names. because it's just a hint, dircount could be zero. when that happens, the server is free to ignore dircount and return directory information based on the maxcount value instead 🧚
the cookieverf can help the server manage cookie values that might go stale. it should be really rare for a server to be unable to keep reading a directory with the provided cookie/cookieverf pair. the server should try its hardest to avoid that situation, because the application at the client may not be able to handle that kind of failure 😤
the cookieverf also protects the client from using READDIR cookie values that have gone stale. say the file system got migrated. the new server may or may not be able to use the same cookie values the previous server used. since the client provides the cookieverf, the server can send back the right response. this stops the bad case where the server accepts a cookie but the underlying directory has changed, which would make the response invalid from the client's point of view for its previous READDIR 🫶
some servers won't return "." and ".." entries the way previous versions of the NFS protocol did. so a client that needs those entries to be present in READDIR responses has to make them up itself 🎀
ERRORS
NFS4ERR_ACCES
NFS4ERR_BADHANDLE
NFS4ERR_BAD_COOKIE
NFS4ERR_DELAY
NFS4ERR_FHEXPIRED
NFS4ERR_INVAL
NFS4ERR_IO
NFS4ERR_MOVED
NFS4ERR_NOFILEHANDLE
NFS4ERR_NOTDIR
NFS4ERR_NOTSUPP
NFS4ERR_READDIR_NOSPC
NFS4ERR_RESOURCE
NFS4ERR_SERVERFAULT
NFS4ERR_STALE
NFS4ERR_TOOSMALL
NFS4ERR_WRONGSEC14.2.25. Operation 27: READLINK - Read Symbolic Link 🔗
SYNOPSIS 🌸
(cfh) -> linktextARGUMENT 💌
/* CURRENT_FH: symlink */
void;RESULT ✨
struct READLINK4resok {
linktext4 link;
};
union READLINK4res switch (nfsstat4 status) {
case NFS4_OK:
READLINK4resok resok4;
default:
void;
};DESCRIPTION 📖
okay so READLINK is how you read the data inside a symbolic link 🦋 The data is a UTF-8 string, and it's opaque to the server, which means the server doesn't try to understand it. Whether an NFS client made the link or someone made it locally on the server, the data just gets stored and never interpreted at creation time. Literally just stored, no questions asked 🫶
On success, the current filehandle keeps its value. Main character energy, it doesn't move 💅
IMPLEMENTATION 🛠️
ngl this part is so good. A symbolic link is basically a pointer to another file 🎀 The server doesn't have to interpret the data, it just stores it in the file. A client can totally put a path name in there that means nothing to the server's operating system 🙈 When READLINK runs, the data gets handed back to the client, and the client is the one who figures out what it means. If different implementations want to share symbolic links, they've gotta agree on how to read the data in the link. Otherwise it's chaos 💀
The READLINK operation is only allowed on objects of type NF4LNK. If the object isn't of type NF4LNK, the server should return the error, NFS4ERR_INVAL 🌈
ERRORS 🚨
NFS4ERR_ACCES
NFS4ERR_BADHANDLE
NFS4ERR_DELAY
NFS4ERR_FHEXPIRED
NFS4ERR_INVAL
NFS4ERR_IO
NFS4ERR_MOVED
NFS4ERR_NOFILEHANDLE
NFS4ERR_NOTSUPP
NFS4ERR_RESOURCE
NFS4ERR_SERVERFAULT
NFS4ERR_STALE
NFS4ERR_WRONGSEC14.2.26. Operation 28: REMOVE - Remove Filesystem Object 🗑️
SYNOPSIS 🌸
(cfh), filename -> change_infoARGUMENT 💌
struct REMOVE4args {
/* CURRENT_FH: directory */
component4 target;
};RESULT ✨
struct REMOVE4resok {
change_info4 cinfo;
}
union REMOVE4res switch (nfsstat4 status) {
case NFS4_OK:
REMOVE4resok resok4;
default:
void;
}DESCRIPTION 📖
so REMOVE deletes a directory entry named by filename, from the directory the current filehandle points to 😭 If that entry was the last reference to the file system object, the object may be destroyed. (Lowercase may, it's giving maybe, not a promise 🙈)
For the directory where the filename got removed, the server hands back change_info4 info in cinfo. The atomic field in that change_info4 struct tells you whether the before and after change attributes were obtained atomically with respect to the removal. Very important, very iconic 🔥
If the target has a length of 0 (zero), or if target doesn't follow the UTF-8 definition, the server returns NFS4ERR_INVAL. No cap, that's a hard no 🙅♀️
On success, the current filehandle keeps its value ✨
IMPLEMENTATION 🛠️
bestie, history lesson 🧚 NFS versions 2 and 3 needed a totally different operator, RMDIR, for removing directories. NFS version 4 REMOVE can delete any directory entry no matter what file type it is. Iconic glow up 👑
"Last reference" is server specific, so it depends on the server. Here's the tea though: if the numlinks field in the previous attributes of the object was 1, the client should not rely on using a file handle to reach the object. Also, the client should not count on the resources (disk space, directory entry, and so on) that used to belong to the object becoming free right away 💀 So if a client needs to keep accessing a file after using REMOVE on it, it should take steps to make sure the file is still reachable. The usual move is to RENAME the file from its old name to a new hidden name 🦋
ERRORS 🚨
NFS4ERR_ACCES
NFS4ERR_BADHANDLE
NFS4ERR_DELAY
NFS4ERR_FHEXPIRED
NFS4ERR_IO
NFS4ERR_MOVED
NFS4ERR_NAMETOOLONG
NFS4ERR_NOENT
NFS4ERR_NOFILEHANDLE
NFS4ERR_NOTDIR
NFS4ERR_NOTEMPTY
NFS4ERR_NOTSUPP
NFS4ERR_RESOURCE
NFS4ERR_ROFS
NFS4ERR_SERVERFAULT
NFS4ERR_STALE
NFS4ERR_WRONGSEC14.2.27. Operation 29: RENAME - Rename Directory Entry ✏️
SYNOPSIS 🌸
(sfh), oldname (cfh), newname -> source_change_info,
target_change_infoARGUMENT 💌
struct RENAME4args {
/* SAVED_FH: source directory */
component4 oldname;
/* CURRENT_FH: target directory */
component4 newname;
};RESULT ✨
struct RENAME4resok {
change_info4 source_cinfo;
change_info4 target_cinfo;
};
union RENAME4res switch (nfsstat4 status) {
case NFS4_OK:
RENAME4resok resok4;
default:
void;
};DESCRIPTION 📖
okay the RENAME operation renames the object called oldname, sitting in the source directory (the one saved by SAVEFH), to newname in the target directory (the current filehandle) 🎀 It's required to be atomic to the client, so the client sees it as one clean move ✨ Source and target directories must live on the same file system on the server. On success, the current filehandle stays pointed at the target directory 💅
If the target directory already has an entry with newname, the source object has to match the target. Either both are non-directories, or both are directories and the target has to be empty. If they match, the existing target gets removed before the rename happens. If they don't match, or if the target is a directory that isn't empty, the server returns NFS4ERR_EXIST 😭
If oldname and newname both refer to the same file (they might be hard links of each other), then RENAME should perform no action and return success. Lowkey so chill 🦋
For both directories in the RENAME, the server returns change_info4 info. The atomic field in the change_info4 struct says whether the before and after change attributes were obtained atomically with respect to the rename 🌈
If oldname or newname has a length of 0 (zero), or if either one doesn't follow the UTF-8 definition, the error NFS4ERR_INVAL gets returned 🙅♀️
IMPLEMENTATION 🛠️
RENAME must be atomic to the client. The line "source and target directories must reside on the same file system on the server" means the fsid fields in the attributes for the directories are the same 🧠 If they live on different file systems, the server returns NFS4ERR_XDEV.
A filehandle may or may not go stale or expire on a rename. Still, server implementors are strongly encouraged to try to keep file handles from going stale or expiring this way. Protect those handles, queen 👑
On some servers, the filenames "." and ".." are illegal as either oldname or newname. Also, neither oldname nor newname can be an alias for the source directory. On those servers, these cases get NFS4ERR_INVAL 🙈
ERRORS 🚨
NFS4ERR_ACCES
NFS4ERR_BADHANDLE
NFS4ERR_DELAY
NFS4ERR_DQUOT
NFS4ERR_EXIST
NFS4ERR_FHEXPIRED
NFS4ERR_INVAL
NFS4ERR_IO
NFS4ERR_ISDIR
NFS4ERR_MOVED
NFS4ERR_NAMETOOLONG
NFS4ERR_NOENT
NFS4ERR_NOFILEHANDLE
NFS4ERR_NOSPC
NFS4ERR_NOTDIR
NFS4ERR_NOTEMPTY
NFS4ERR_NOTSUPP
NFS4ERR_RESOURCE
NFS4ERR_ROFS
NFS4ERR_SERVERFAULT
NFS4ERR_STALE
NFS4ERR_WRONGSEC
NFS4ERR_XDEV14.2.28. Operation 30: RENEW - Renew a Lease ⏳
SYNOPSIS 🌸
stateid -> ()ARGUMENT 💌
struct RENEW4args {
stateid4 stateid;
};RESULT ✨
struct RENEW4res {
nfsstat4 status;
};DESCRIPTION 📖
the RENEW operation is how a client keeps its leases alive at a server 🦋 When the server processes a RENEW request, it renews all the leases tied to that client. Those leases are figured out from the client id given through the SETCLIENTID procedure. Keeping things fresh, bestie 🌸
The stateid for RENEW may not be one of the special stateids made of all bits 0 (zero) or all bits 1. So those reserved stateids are a no-go here 🙅♀️
IMPLEMENTATION 🛠️
nothing to add here, this one's empty ✨
ERRORS 🚨
NFS4ERR_BAD_STATEID
NFS4ERR_EXPIRED
NFS4ERR_GRACE
NFS4ERR_INVAL
NFS4ERR_LEASE_MOVED
NFS4ERR_MOVED
NFS4ERR_OLD_STATEID
NFS4ERR_RESOURCE
NFS4ERR_SERVERFAULT
NFS4ERR_STALE_STATEID
NFS4ERR_WRONGSEC14.2.29. Operation 31: RESTOREFH - Restore Saved Filehandle ⏪
SYNOPSIS 🌸
(sfh) -> (cfh)ARGUMENT 💌
/* SAVED_FH: */
void;RESULT ✨
struct RESTOREFH4res {
/* CURRENT_FH: value of saved fh */
nfsstat4 status;
};DESCRIPTION 📖
RESTOREFH sets the current filehandle back to whatever the saved filehandle holds 🧚 If there's no saved filehandle, it returns the error NFS4ERR_NOFILEHANDLE. Tragic, but that's the rule 💀
IMPLEMENTATION 🛠️
ok the big idea 💡 Operations like OPEN and LOOKUP use the current filehandle to represent a directory, then swap it for a new filehandle. If the previous filehandle was saved with SAVEFH, you can restore it as the current filehandle. This is super common for grabbing post-operation attributes for the directory, like this:
PUTFH (directory filehandle)
SAVEFH
GETATTR attrbits (pre-op dir attrs)
CREATE optbits "foo" attrs
GETATTR attrbits (file attributes)
RESTOREFH
GETATTR attrbits (post-op dir attrs)It's giving a clean before-and-after snapshot 📸🫶
ERRORS 🚨
NFS4ERR_BADHANDLE
NFS4ERR_FHEXPIRED
NFS4ERR_MOVED
NFS4ERR_NOFILEHANDLE
NFS4ERR_RESOURCE
NFS4ERR_SERVERFAULT
NFS4ERR_STALE
NFS4ERR_WRONGSEC14.2.30. Operation 32: SAVEFH - Save Current Filehandle 💾
SYNOPSIS 🌸
(cfh) -> (sfh)ARGUMENT 💌
/* CURRENT_FH: */
void;RESULT ✨
struct SAVEFH4res {
/* SAVED_FH: value of current fh */
nfsstat4 status;
};DESCRIPTION 📖
SAVEFH saves the current filehandle 🦋 If a filehandle was saved before, it's no longer accessible after this, so the old one is gone 😭 The saved filehandle can get restored as the current filehandle using the RESTOREFH operator, which is literally its best friend 🫶
On success, the current filehandle keeps its value ✨
IMPLEMENTATION 🛠️
nothing extra for this one, it's a clean little operation 💅
ERRORS 🚨
NFS4ERR_BADHANDLE
NFS4ERR_FHEXPIRED
NFS4ERR_MOVED
NFS4ERR_NOFILEHANDLE
NFS4ERR_RESOURCE
NFS4ERR_SERVERFAULT
NFS4ERR_STALE
NFS4ERR_WRONGSEC14.2.31. Operation 33: SECINFO - Obtain Available Security ✨
SYNOPSIS 💖
(cfh), name -> { secinfo }ARGUMENT 🎀
struct SECINFO4args {
/* CURRENT_FH: */
component4 name;
};RESULT 👑
enum rpc_gss_svc_t {
RPC_GSS_SVC_NONE = 1,
RPC_GSS_SVC_INTEGRITY = 2,
RPC_GSS_SVC_PRIVACY = 3
};
struct rpcsec_gss_info {
sec_oid4 oid;
qop4 qop;
rpc_gss_svc_t service;
};
struct secinfo4 {
uint32_t flavor;
opaque flavor_info<>; /* null for AUTH_SYS, AUTH_NONE;
contains rpcsec_gss_info for
RPCSEC_GSS. */
};
typedef secinfo4 SECINFO4resok<>;
union SECINFO4res switch (nfsstat4 status) {
case NFS4_OK:
SECINFO4resok resok4;
default:
void;
};DESCRIPTION 🦋
okay so SECINFO is how the client asks the server, "hey, which security flavors can I use for this file handle and file name pair? 🥺" 💅 The answer comes back as an array, and every entry in that array is a secinfo4 structure describing one security mechanism the server is cool with. The flavor field will be AUTH_NONE, AUTH_SYS (as defined in [RFC1831]), or RPCSEC_GSS (as defined in [RFC2203]). Iconic, right? 🌈
For AUTH_NONE and AUTH_SYS, there's no extra security info to send back. For RPCSEC_GSS, though, the server returns a security triple: the mechanism object id (as defined in [RFC2078]), the quality of protection (also as defined in [RFC2078]), and the service type (as defined in [RFC2203]). It's totally possible for SECINFO to hand back several RPCSEC_GSS entries, each with a different triple. Main character energy 👑
On success, the current filehandle keeps its value. No cap, it just stays put 🫶
IMPLEMENTATION 🔥
ngl this is the operation the NFS client reaches for when another NFS operation comes back with NFS4ERR_WRONGSEC. That error is the server going "bestie, my security policy is different from what you're using rn 💀" So the client is expected to grab the list of possible security flavors and pick whichever one best fits its policies. Very responsible, very slay ✨
It is recommended that the client issue the SECINFO call protected by a security triple that uses either rpc_gss_svc_integrity or rpc_gss_svc_privacy service. Here's why it matters 😭 If the client used rpc_gss_svc_none, an attacker in the middle could mess with the SECINFO results, tricking the client into picking a weaker algorithm from the set the server allows. That would leave the client and/or server open to further attacks. Absolutely not, we are not doing that 🙈
ERRORS 💀
NFS4ERR_BADHANDLE
NFS4ERR_FHEXPIRED
NFS4ERR_MOVED
NFS4ERR_NAMETOOLONG
NFS4ERR_NOENT
NFS4ERR_NOFILEHANDLE
NFS4ERR_NOTDIR
NFS4ERR_RESOURCE
NFS4ERR_SERVERFAULT
NFS4ERR_STALE
NFS4ERR_WRONGSEC14.2.32. Operation 34: SETATTR - Set Attributes 🦋
SYNOPSIS 💖
(cfh), attrbits, attrvals -> -ARGUMENT 🎀
struct SETATTR4args {
/* CURRENT_FH: target object */
stateid4 stateid;
fattr4 obj_attributes;
};RESULT 👑
struct SETATTR4res {
nfsstat4 status;
bitmap4 attrsset;
};DESCRIPTION 🌸
SETATTR is the operation that lets you change one or more attributes of a file system object. ✨ You tell it which attributes you want by using a bitmap, and the values come right after, in bit order. Simple, clean, iconic 💅
The stateid matters a lot here. It's required for any SETATTR that changes the size of a file (the object_size attribute). That stateid stands for a record lock, a share reservation, or a delegation, and it has to be valid before SETATTR is allowed to touch the file data. Always send a valid stateid. If the file size isn't changing, you use the special stateid made up of all zero bits instead 🧚
Whether the operation succeeds or fails, the server hands back the attrsset bitmask, which shows what (if anything) actually got set. Receipts, bestie 🧾
On success, the current filehandle keeps its value. 🫶
IMPLEMENTATION 🔥
The file size attribute is how you ask for a file to change size. Here's the tea ☕ A size of 0 (zero) truncates the file. A size smaller than the current one throws away everything from the new size to the end of the file. A size bigger than the current one adds logically zeroed data bytes to the end. The server gets to decide how it does that, using holes or actual zero data bytes, and clients shouldn't assume anything about how the server handles it beyond the fact that the bytes come back zeroed. Servers must support extending the file size via SETATTR. Non-negotiable 😤
SETATTR is not guaranteed to be atomic. A failed SETATTR might partially change a file's attributes, so watch out 👀
Changing the size of a file with SETATTR also indirectly changes time_modify. The client has to account for this, since size changes can lead to data getting deleted. 💀
If server and client times don't match, programs that compare client time against file times can break 🙈 A time maintenance protocol should be used to keep client/server time skew limited.
If the server can't set every attribute successfully, it must return NFS4ERR_INVAL. And if the server can only handle 32-bit offsets and sizes, a SETATTR that tries to set a file size bigger than 32 bits can hold will get rejected with that same error. Rude, but fair 😭
ERRORS 💀
NFS4ERR_ACCES
NFS4ERR_BADHANDLE
NFS4ERR_BAD_STATEID
NFS4ERR_DELAY
NFS4ERR_DENIED
NFS4ERR_DQUOT
NFS4ERR_EXPIRED
NFS4ERR_FBIG
NFS4ERR_FHEXPIRED
NFS4ERR_GRACE
NFS4ERR_INVAL
NFS4ERR_IO
NFS4ERR_MOVED
NFS4ERR_NOFILEHANDLE
NFS4ERR_NOSPC
NFS4ERR_NOTSUPP
NFS4ERR_OLD_STATEID
NFS4ERR_PERM
NFS4ERR_RESOURCE
NFS4ERR_ROFS
NFS4ERR_SERVERFAULT
NFS4ERR_STALE
NFS4ERR_STALE_STATEID
NFS4ERR_WRONGSEC14.2.33. Operation 35: SETCLIENTID - Negotiate Clientid 🎀
SYNOPSIS 💖
client, callback -> clientid, setclientid_confirmARGUMENT 🎀
struct SETCLIENTID4args {
nfs_client_id4 client;
cb_client4 callback;
};RESULT 👑
struct SETCLIENTID4resok {
clientid4 clientid;
verifier4 setclientid_confirm;
};
union SETCLIENTID4res switch (nfsstat4 status) {
case NFS4_OK:
SETCLIENTID4resok resok4;
case NFS4ERR_CLID_INUSE:
clientaddr4 client_using;
default:
void;
};DESCRIPTION 🌸
SETCLIENTID is the client's way of saying "hi server, I'm about to use this particular client identifier and verifier pair, just so you know 💅" If that goes through, the server sends back a clientid, which gets used in later file locking requests, plus a confirmation verifier. The client then sends that verifier back using SETCLIENTID_CONFIRM. Once that's done, the client is good to go and can use the clientid in any later operations that need an nfs_lockowner. Main character moment ✨
The callback info given in this operation is what gets used if the client is handed an open delegation down the road. So the client has to be super accurate about the program and port numbers for the callback program at the moment it calls SETCLIENTID. No typos allowed, bestie 🙈
IMPLEMENTATION 🔥
Here's how the server handles it. It takes the verifier and client identification from the nfs_client_id4 and looks for a match on the client identification. If there's no match, the server saves the principal/uid info along with the verifier and client identification, then returns a unique clientid as a shorthand for all of that. Brand new client, who dis? 🧚
If the server finds a matching client identification and also a matching principal/uid, it releases all the locking state for that client and returns a new clientid. A reset, basically 🧹
The principal, or the principal to user-identifier mapping, comes from the credential presented in the RPC. Like I said earlier, the server uses that credential and its principal to match against existing clientids. If the client is a traditional host-based one, like a Unix NFS client, the credential might be the host credential. If it's a user-level or lightweight client, the credential could be the end user's credential. ngl, the client needs to pick its credential carefully, because a rogue client that has access to the credential could try denial of service attacks. Not it, not on my watch 😤🔥
ERRORS 💀
NFS4ERR_CLID_INUSE
NFS4ERR_INVAL
NFS4ERR_RESOURCE
NFS4ERR_SERVERFAULT14.2.34. Operation 36: SETCLIENTID_CONFIRM - Confirm Clientid 🌈
SYNOPSIS 💖
setclientid_confirm -> -ARGUMENT 🎀
struct SETCLIENTID_CONFIRM4args {
verifier4 setclientid_confirm;
};RESULT 👑
struct SETCLIENTID_CONFIRM4res {
nfsstat4 status;
};DESCRIPTION 🌸
This operation is how the client confirms the results of an earlier SETCLIENTID call. The client hands back the opaque confirmation verifier the server gave it in the SETCLIENTID response, and the server replies with a simple success or failure status. Short and sweet 🎀
IMPLEMENTATION 🔥
The client must use SETCLIENTID_CONFIRM to confirm its use of the client identifier. Here's the vibe: if the server is holding state for a client that sent a new verifier through SETCLIENTID, that state doesn't get released until a valid SETCLIENTID_CONFIRM comes in. This is the part described in the section "Client Failure and Recovery", so go check that out too 👀 Once the confirmation succeeds, the server releases the previous state it was holding for that client. The server should pick a confirmation cookie value that's reasonably unique for the client. Unique energy only 🦋💖
ERRORS 💀
NFS4ERR_CLID_INUSE
NFS4ERR_INVAL
NFS4ERR_RESOURCE
NFS4ERR_SERVERFAULT
NFS4ERR_STALE_CLIENTID14.2.35. Operation 37: VERIFY - Verify Same Attributes ✨
SYNOPSIS (the quick vibe check) 👀
(cfh), fattr -> -ARGUMENT 🎀
struct VERIFY4args {
/* CURRENT_FH: object */
fattr4 obj_attributes;
};RESULT 🍓
struct VERIFY4res {
nfsstat4 status;
};DESCRIPTION 💅
okay so VERIFY is basically the client checking the vibes before it moves on 🧐 It makes sure the attributes actually have the values the client assumed they had, before the next operations in the compound request run. If even one attribute doesn't match, the error NFS4ERR_NOT_SAME must be returned, no skipping! The current filehandle keeps its value once the operation finishes successfully. 💖
IMPLEMENTATION 🔥
one possible use of VERIFY is this compound sequence. The client is trying to confirm that the file it's about to remove is the exact file it expects, so an unintended deletion doesn't happen. It's giving safety 🫶
PUTFH (directory filehandle)
LOOKUP (file name)
VERIFY (filehandle == fh)
PUTFH (directory filehandle)
REMOVE (file name)ngl this doesn't stop a second client from removing a file and creating a new one in the middle of the sequence 😭 but it does help avoid the unintended result, which is still a W 🦋
Also, if a recommended attribute is specified in the VERIFY operation and the server doesn't support that attribute for the file system object, the error NFS4ERR_NOTSUPP is returned to the client. not supported, not vibing 💀
ERRORS 🌸
NFS4ERR_ACCES
NFS4ERR_BADHANDLE
NFS4ERR_DELAY
NFS4ERR_FHEXPIRED
NFS4ERR_INVAL
NFS4ERR_MOVED
NFS4ERR_NOFILEHANDLE
NFS4ERR_NOTSUPP
NFS4ERR_NOT_SAME
NFS4ERR_RESOURCE
NFS4ERR_SERVERFAULT
NFS4ERR_STALE
NFS4ERR_WRONGSEC14.2.36. Operation 38: WRITE - Write to File ✨
SYNOPSIS 👀
(cfh), offset, count, stability, stateid, data -> count, committed,
verifierARGUMENT 🎀
enum stable_how4 {
UNSTABLE4 = 0,
DATA_SYNC4 = 1,
FILE_SYNC4 = 2
};
struct WRITE4args {
/* CURRENT_FH: file */
stateid4 stateid;
offset4 offset;
stable_how4 stable;
opaque data<>;
};RESULT 🍓
struct WRITE4resok {
count4 count;
stable_how4 committed;
verifier4 writeverf;
};
union WRITE4res switch (nfsstat4 status) {
case NFS4_OK:
WRITE4resok resok4;
default:
void;
};DESCRIPTION 💅
WRITE is how the client puts data into a regular file 📝 The target is whatever the current filehandle points at. The offset says where the data goes, and an offset of 0 (zero) means start at the very beginning of the file. The count is how many bytes of data you want written. If the count is 0 (zero), the WRITE will succeed and return a count of 0 (zero), subject to permissions checking. The server may choose to write fewer bytes than the client asked for, so don't be shocked 🙈
Now the stable parameter is where it gets spicy 🌶️ It tells the server how the data should be handled. If stable is FILE_SYNC4, the server must commit the written data plus all file system metadata to stable storage before it returns results. That matches the NFS version 2 protocol semantics. Anything else is a protocol violation, no cap 🚫
If stable is DATA_SYNC4, the server must commit all of the data to stable storage, plus enough of the metadata to retrieve the data, before it returns. The server implementor is free to do DATA_SYNC4 the same way as FILE_SYNC4, just with a possible performance drop 🐢
If stable is UNSTABLE4, the server is free to commit any part of the data and metadata to stable storage, including all of it or none of it, before replying to the client. There's no guarantee whether or when uncommitted data will get committed later. The only guarantees are that the server won't destroy any data without changing the value of verf, and that it won't commit the data and metadata at a level less than what the client requested. bestie, that's the deal 🤝
The stateid from a previous record lock or share reservation request gets passed in as part of the argument. The server uses it to check that the associated lock is still valid and to update the client's lease timeouts ⏰
On success, here's what comes back ✨ The count result is the number of bytes of data written to the file. The server may write fewer bytes than requested, and if it does, the actual number of bytes written starting at location, offset, is what gets returned.
The server also sends back the level of commitment through committed. If the server committed all data and metadata to stable storage, committed should be set to FILE_SYNC4. If the level of commitment was at least as strong as DATA_SYNC4, then committed should be set to DATA_SYNC4. Otherwise, committed must be returned as UNSTABLE4. If stable was FILE4_SYNC, then committed must also be FILE_SYNC4, and anything else is a protocol violation. If stable was DATA_SYNC4, then committed may be FILE_SYNC4 or DATA_SYNC4, and anything else is a protocol violation. If stable was UNSTABLE4, then committed may be FILE_SYNC4, DATA_SYNC4, or UNSTABLE4. the rules are a lot but they're so precise 😤
The last piece of the result is the write verifier, verf. This is a cookie the client uses to figure out whether the server changed state between a WRITE call and a later WRITE or COMMIT call 🕵️♀️ It must stay consistent during a single instance of the NFS version 4 protocol service, and it must be unique between instances of the NFS version 4 protocol server, since uncommitted data may be lost between those instances.
If a client writes data with the stable argument set to UNSTABLE4, and the reply comes back with committed set to DATA_SYNC4 or UNSTABLE4, the client will follow up sometime later with a COMMIT operation. That COMMIT synchronizes the outstanding async data and metadata with the server's stable storage, barring client error. It's possible that, because of a client crash or some other error, a subsequent COMMIT never gets received by the server 😭
On success, the current filehandle retains its value ✨
IMPLEMENTATION 🔥
It's possible for the server to write fewer than count bytes. In that case, the server should not return an error unless no data was written at all. If the server writes less than count bytes, the client should issue another WRITE to write the remaining data 🔁
It's assumed that writing data to a file will cause the time_modified of the file to be updated. However, the time_modified of the file should not be changed unless the contents of the file are actually changed. So a WRITE request with count set to 0 should not cause the time_modified of the file to be updated. no phantom updates allowed 👻
Stable storage has historically been a point of contention, lowkey a whole debate 🥊 These expected properties may help sort out design issues during implementation. Stable storage is persistent storage that survives:
- Repeated power failures.
- Hardware failures (of any board, power supply, etc.).
- Repeated software crashes, including reboot cycle.
This definition does not address failure of the stable storage module itself.
The verifier exists so a client can detect different instances of an NFS version 4 protocol server, the kind over which cached, uncommitted data may be lost. In the most likely case, the verifier lets the client detect server reboots. This info is needed so the client can safely determine whether the server could have lost cached data. If the server fails unexpectedly and the client has uncommitted data from previous WRITE requests (done with the stable argument set to UNSTABLE4, and where the result committed came back as UNSTABLE4 as well), that data may not have been flushed to stable storage. The burden of recovery is on the client, and the client will need to retransmit the data to the server 🫠
A suggested verifier would be the time the server was booted, or the time the server was last started (if restarting the server without a reboot results in lost buffers) ⏱️
The committed field lets the client do more effective caching 🧠 If the server is committing all WRITE requests to stable storage, then it should return committed set to FILE_SYNC4, regardless of the value of the stable field in the arguments. A server that uses an NVRAM accelerator may choose to implement this policy. The client can use this to make its cache more effective by discarding cached data that's already been committed on the server 🧹
Some implementations may return NFS4ERR_NOSPC instead of NFS4ERR_DQUOT when a user's quota is exceeded. the quota drama continues 🙃
ERRORS 🌸
NFS4ERR_ACCES
NFS4ERR_BADHANDLE
NFS4ERR_BAD_STATEID
NFS4ERR_DELAY
NFS4ERR_DENIED
NFS4ERR_DQUOT
NFS4ERR_EXPIRED
NFS4ERR_FBIG
NFS4ERR_FHEXPIRED
NFS4ERR_GRACE
NFS4ERR_INVAL
NFS4ERR_IO
NFS4ERR_LEASE_MOVED
NFS4ERR_LOCKED
NFS4ERR_MOVED
NFS4ERR_NOFILEHANDLE
NFS4ERR_NOSPC
NFS4ERR_OLD_STATEID
NFS4ERR_RESOURCE
NFS4ERR_ROFS
NFS4ERR_SERVERFAULT
NFS4ERR_STALE
NFS4ERR_STALE_STATEID
NFS4ERR_WRONGSEC15. NFS Version 4 Callback Procedures ✨
The procedures used for callbacks are defined in the following sections 🔁 Quick heads-up: in this part, "client" and "server" mean NFS clients and servers, even though for an individual callback RPC those terms would be the exact opposite, since the roles are flipped 🙃
15.1. Procedure 0: CB_NULL - No Operation 🌈
SYNOPSIS 👀
<null>ARGUMENT 🎀
void;RESULT 🍓
void;DESCRIPTION 💅
Standard NULL procedure, which means void argument and void response, no drama 🫶 Even though this procedure doesn't do any direct work, the server uses CB_NULL to confirm that a path exists for RPCs from the server to the client. it's giving "are you there?" energy 📡
ERRORS 🌸
None.
15.2. Procedure 1: CB_COMPOUND - Compound Operations ✨
SYNOPSIS
compoundargs -> compoundres
ARGUMENT
enum nfs_cb_opnum4 {
OP_CB_GETATTR = 3,
OP_CB_RECALL = 4 };
union nfs_cb_argop4 switch (unsigned argop) {
case OP_CB_GETATTR: CB_GETATTR4args opcbgetattr;
case OP_CB_RECALL: CB_RECALL4args opcbrecall; };
struct CB_COMPOUND4args {
utf8string tag;
uint32_t minorversion;
nfs_cb_argop4 argarray<>; };RESULT
union nfs_cb_resop4 switch (unsigned resop){
case OP_CB_GETATTR: CB_GETATTR4res opcbgetattr;
case OP_CB_RECALL: CB_RECALL4res opcbrecall; };
struct CB_COMPOUND4res {
nfsstat4 status;
utf8string tag;
nfs_cb_resop4 resarray<>; };DESCRIPTION
okay so CB_COMPOUND is the bestie that bundles one or more of the callback procedures into a single RPC request 🎀 The main callback RPC program only has two main procedures, CB_NULL and CB_COMPOUND, and every other operation rides along inside CB_COMPOUND as a wrapper 💅
now, the client might run out of resources while it's processing CB_COMPOUND, which means it can't run some or all of the operations in the sequence 😭 When that happens, NFS4ERR_RESOURCE gets returned for the exact operation where the exhaustion happened. This assumes every operation before it in the sequence was evaluated successfully ✨
the CB_COMPOUND results also contain a status field, and that status has to be the same as the status of the last operation that was executed in the CB_COMPOUND procedure 🔥 So if an operation hit an error, the status value is that same error value, the one being returned for the operation that failed 💖
IMPLEMENTATION
the CB_COMPOUND procedure combines individual operations into a single RPC request, and the client works through them one at a time 🧚 If an operation runs on the client and its status is NFS4_OK, the next operation in the CB_COMPOUND procedure runs too. This keeps going until there are no operations left or one of them returns a status other than NFS4_OK 🌈
ERRORS
NFS4ERR_BADHANDLENFS4ERR_BAD_STATEIDNFS4ERR_RESOURCE
15.2.1. Operation 3: CB_GETATTR - Get Attributes 🦋
SYNOPSIS
fh, attrbits -> attrbits, attrvals
ARGUMENT
struct CB_GETATTR4args {
nfs_fh4 fh;
bitmap4 attr_request;
};RESULT
struct CB_GETATTR4resok {
fattr4 obj_attributes;
};
union CB_GETATTR4res switch (nfsstat4 status) {
case NFS4_OK:
CB_GETATTR4resok resok4;
default:
void;
};DESCRIPTION
CB_GETATTR is how the server gets the attributes an open delegate has modified 🌸 That way the server can answer GETATTR requests for a file that's under an open delegation. It's giving teamwork, honestly 🫶
if the handle isn't one where the client holds a write open delegation, you get NFS4ERR_BADHANDLE back 🙈
IMPLEMENTATION
the client only sends back attrbits and their values for the attributes it may change, which are change, time_modify, and object_size ✨
ERRORS
NFS4ERR_BADHANDLENFS4ERR_RESOURCE
15.2.2. Operation 4: CB_RECALL - Recall an Open Delegation 👑
SYNOPSIS
stateid, truncate, fh -> status
ARGUMENT
struct CB_RECALL4args {
stateid4 stateid;
bool truncate;
nfs_fh4 fh;
};RESULT
struct CB_RECALL4res {
nfsstat4 status;
};DESCRIPTION
CB_RECALL is the one that starts the process of recalling an open delegation and sending it back to the server 🌷
the truncate flag is an optimization for a file that's about to be truncated to zero. When it's set, the client is off the hook for pushing modified data for that file to the server, because that data is irrelevant anyway 💀
if the handle isn't one where the client holds an open delegation, NFS4ERR_BADHANDLE is returned. And if the stateid doesn't correspond to an open delegation for the file the filehandle points to, NFS4ERR_BAD_STATEID is returned 😭
IMPLEMENTATION
the client should reply to the callback right away. Replying doesn't finish the recall, though 👀 The recall isn't complete until the delegation gets returned using DELEGRETURN ✨
ERRORS
NFS4ERR_BADHANDLENFS4ERR_BAD_STATEIDNFS4ERR_RESOURCE
16. Security Considerations 🔐
okay the big one, bestie 😤 The main security feature to think about is authenticating the user who's making the request for NFS service. Integrity and privacy of that NFS request also deserve a lot of attention, no cap 🧐 These specific issues are covered in the section on "RPC and Security Flavor" 💜
17. IANA Considerations 📋
17.1. Named Attribute Definition 🎀
so NFS version 4 lets you attach named attributes to files, and those names are defined as string names 🌈 The protocol doesn't assign a specific namespace for these file attributes. The application developer or system vendor gets to define the attribute, what it means, and its name 💅 Nothing controls this namespace to stop collisions, but the app developer or system vendor is strongly encouraged to publish the name assignment and semantics for attributes in an Informational RFC 🫶 That's how interoperability happens wherever common interests exist ✨
18. RPC definition file 🧚
ngl, here's the actual .x file, copied exactly so every implementer gets the real thing 🤓
/*
* Copyright (C) The Internet Society (1998,1999,2000).
* All Rights Reserved.
*/
/*
* nfs4_prot.x
*
*/
%#pragma ident "@(#)nfs4_prot.x 1.97 00/06/12"
/*
* Basic typedefs for RFC 1832 data type definitions
*/
typedef int int32_t;
typedef unsigned int uint32_t;
typedef hyper int64_t;
typedef unsigned hyper uint64_t;
/*
* Sizes
*/
const NFS4_FHSIZE = 128;
const NFS4_VERIFIER_SIZE = 8;
/*
* File types
*/
enum nfs_ftype4 {
NF4REG = 1, /* Regular File */
NF4DIR = 2, /* Directory */
NF4BLK = 3, /* Special File - block device */
NF4CHR = 4, /* Special File - character device */
NF4LNK = 5, /* Symbolic Link */
NF4SOCK = 6, /* Special File - socket */
NF4FIFO = 7, /* Special File - fifo */
NF4ATTRDIR = 8, /* Attribute Directory */
NF4NAMEDATTR = 9 /* Named Attribute */
};
/*
* Error status
*/
enum nfsstat4 {
NFS4_OK = 0,
NFS4ERR_PERM = 1,
NFS4ERR_NOENT = 2,
NFS4ERR_IO = 5,
NFS4ERR_NXIO = 6,
NFS4ERR_ACCES = 13,
NFS4ERR_EXIST = 17,
NFS4ERR_XDEV = 18,
NFS4ERR_NODEV = 19,
NFS4ERR_NOTDIR = 20,
NFS4ERR_ISDIR = 21,
NFS4ERR_INVAL = 22,
NFS4ERR_FBIG = 27,
NFS4ERR_NOSPC = 28,
NFS4ERR_ROFS = 30,
NFS4ERR_MLINK = 31,
NFS4ERR_NAMETOOLONG = 63,
NFS4ERR_NOTEMPTY = 66,
NFS4ERR_DQUOT = 69,
NFS4ERR_STALE = 70,
NFS4ERR_BADHANDLE = 10001,
NFS4ERR_BAD_COOKIE = 10003,
NFS4ERR_NOTSUPP = 10004,
NFS4ERR_TOOSMALL = 10005,
NFS4ERR_SERVERFAULT = 10006,
NFS4ERR_BADTYPE = 10007,
NFS4ERR_DELAY = 10008,
NFS4ERR_SAME = 10009,/* nverify says attrs same */
NFS4ERR_DENIED = 10010,/* lock unavailable */
NFS4ERR_EXPIRED = 10011,/* lock lease expired */
NFS4ERR_LOCKED = 10012,/* I/O failed due to lock */
NFS4ERR_GRACE = 10013,/* in grace period */
NFS4ERR_FHEXPIRED = 10014,/* file handle expired */
NFS4ERR_SHARE_DENIED = 10015,/* share reserve denied */
NFS4ERR_WRONGSEC = 10016,/* wrong security flavor */
NFS4ERR_CLID_INUSE = 10017,/* clientid in use */
NFS4ERR_RESOURCE = 10018,/* resource exhaustion */
NFS4ERR_MOVED = 10019,/* filesystem relocated */
NFS4ERR_NOFILEHANDLE = 10020,/* current FH is not set */
NFS4ERR_MINOR_VERS_MISMATCH = 10021,/* minor vers not supp */
NFS4ERR_STALE_CLIENTID = 10022,
NFS4ERR_STALE_STATEID = 10023,
NFS4ERR_OLD_STATEID = 10024,
NFS4ERR_BAD_STATEID = 10025,
NFS4ERR_BAD_SEQID = 10026,
NFS4ERR_NOT_SAME = 10027,/* verify - attrs not same */
NFS4ERR_LOCK_RANGE = 10028,
NFS4ERR_SYMLINK = 10029,
NFS4ERR_READDIR_NOSPC = 10030,
NFS4ERR_LEASE_MOVED = 10031
};
/*
* Basic data types
*/
typedef uint32_t bitmap4<>;
typedef uint64_t offset4;
typedef uint32_t count4;
typedef uint64_t length4;
typedef uint64_t clientid4;
typedef uint64_t stateid4;
typedef uint32_t seqid4;
typedef opaque utf8string<>;
typedef utf8string component4;
typedef component4 pathname4<>;
typedef uint64_t nfs_lockid4;
typedef uint64_t nfs_cookie4;
typedef utf8string linktext4;
typedef opaque sec_oid4<>;
typedef uint32_t qop4;
typedef uint32_t mode4;
typedef uint64_t changeid4;
typedef opaque verifier4[NFS4_VERIFIER_SIZE];
/*
* Timeval
*/
struct nfstime4 {
int64_t seconds;
uint32_t nseconds;
};
enum time_how4 {
SET_TO_SERVER_TIME4 = 0,
SET_TO_CLIENT_TIME4 = 1
};
union settime4 switch (time_how4 set_it) {
case SET_TO_CLIENT_TIME4:
nfstime4 time;
default:
void;
};
/*
* File access handle
*/
typedef opaque nfs_fh4<NFS4_FHSIZE>;
/*
* File attribute definitions
*/
/*
* FSID structure for major/minor
*/
struct fsid4 {
uint64_t major;
uint64_t minor;
};
/*
* Filesystem locations attribute for relocation/migration
*/
struct fs_location4 {
utf8string server<>;
pathname4 rootpath;
};
struct fs_locations4 {
pathname4 fs_root;
fs_location4 locations<>;
};
/*
* Various Access Control Entry definitions
*/Okay bestie, this is the part where the RFC lays out all the ACL bits, the attribute IDs, and the ACCESS, CLOSE, COMMIT, CREATE, and DELEG stuff. Let's gooo 💅✨
/*
* Mask that indicates which Access Control Entries are supported.
* Values for the fattr4_aclsupport attribute.
*/
const ACL4_SUPPORT_ALLOW_ACL = 0x00000001;
const ACL4_SUPPORT_DENY_ACL = 0x00000002;
const ACL4_SUPPORT_AUDIT_ACL = 0x00000004;
const ACL4_SUPPORT_ALARM_ACL = 0x00000008;
typedef uint32_t acetype4;
/*
* acetype4 values, others can be added as needed.
*/
const ACE4_ACCESS_ALLOWED_ACE_TYPE = 0x00000000;
const ACE4_ACCESS_DENIED_ACE_TYPE = 0x00000001;
const ACE4_SYSTEM_AUDIT_ACE_TYPE = 0x00000002;
const ACE4_SYSTEM_ALARM_ACE_TYPE = 0x00000003;
/*
* ACE flag
*/
typedef uint32_t aceflag4;
/*
* ACE flag values
*/
const ACE4_FILE_INHERIT_ACE = 0x00000001;
const ACE4_DIRECTORY_INHERIT_ACE = 0x00000002;
const ACE4_NO_PROPAGATE_INHERIT_ACE = 0x00000004;
const ACE4_INHERIT_ONLY_ACE = 0x00000008;
const ACE4_SUCCESSFUL_ACCESS_ACE_FLAG = 0x00000010;
const ACE4_FAILED_ACCESS_ACE_FLAG = 0x00000020;
const ACE4_IDENTIFIER_GROUP = 0x00000040;
/*
* ACE mask
*/
typedef uint32_t acemask4;
/*
* ACE mask values
*/
const ACE4_READ_DATA = 0x00000001;
const ACE4_LIST_DIRECTORY = 0x00000001;
const ACE4_WRITE_DATA = 0x00000002;
const ACE4_ADD_FILE = 0x00000002;
const ACE4_APPEND_DATA = 0x00000004;
const ACE4_ADD_SUBDIRECTORY = 0x00000004;
const ACE4_READ_NAMED_ATTRS = 0x00000008;
const ACE4_WRITE_NAMED_ATTRS = 0x00000010;
const ACE4_EXECUTE = 0x00000020;
const ACE4_DELETE_CHILD = 0x00000040;
const ACE4_READ_ATTRIBUTES = 0x00000080;
const ACE4_WRITE_ATTRIBUTES = 0x00000100;
const ACE4_DELETE = 0x00010000;
const ACE4_READ_ACL = 0x00020000;
const ACE4_WRITE_ACL = 0x00040000;
const ACE4_WRITE_OWNER = 0x00080000;
const ACE4_SYNCHRONIZE = 0x00100000;
/*
* ACE4_GENERIC_READ -- defined as combination of
* ACE4_READ_ACL |
* ACE4_READ_DATA |
* ACE4_READ_ATTRIBUTES |
* ACE4_SYNCHRONIZE
*/
const ACE4_GENERIC_READ = 0x00120081;
/*
* ACE4_GENERIC_WRITE -- defined as combination of
* ACE4_READ_ACL |
* ACE4_WRITE_DATA |
* ACE4_WRITE_ATTRIBUTES |
* ACE4_WRITE_ACL |
* ACE4_APPEND_DATA |
* ACE4_SYNCHRONIZE
*/
const ACE4_GENERIC_WRITE = 0x00160106;
/*
* ACE4_GENERIC_EXECUTE -- defined as combination of
* ACE4_READ_ACL
* ACE4_READ_ATTRIBUTES
* ACE4_EXECUTE
* ACE4_SYNCHRONIZE
*/
const ACE4_GENERIC_EXECUTE = 0x001200A0;
/*
* Access Control Entry definition
*/
struct nfsace4 {
acetype4 type;
aceflag4 flag;
acemask4 access_mask;
utf8string who;
};
/*
* Special data/attribute associated with
* file types NF4BLK and NF4CHR.
*/
struct specdata4 {
uint32_t specdata1;
uint32_t specdata2;
};
/*
* Values for fattr4_fh_expire_type
*/
const FH4_PERSISTENT = 0x00000000;
const FH4_NOEXPIRE_WITH_OPEN = 0x00000001;
const FH4_VOLATILE_ANY = 0x00000002;
const FH4_VOL_MIGRATION = 0x00000004;
const FH4_VOL_RENAME = 0x00000008;
typedef bitmap4 fattr4_supported_attrs;
typedef nfs_ftype4 fattr4_type;
typedef uint32_t fattr4_fh_expire_type;
typedef changeid4 fattr4_change;
typedef uint64_t fattr4_size;
typedef bool fattr4_link_support;
typedef bool fattr4_symlink_support;
typedef bool fattr4_named_attr;
typedef fsid4 fattr4_fsid;
typedef bool fattr4_unique_handles;
typedef uint32_t fattr4_lease_time;
typedef nfsstat4 fattr4_rdattr_error;
typedef nfsace4 fattr4_acl<>;
typedef uint32_t fattr4_aclsupport;
typedef bool fattr4_archive;
typedef bool fattr4_cansettime;
typedef bool fattr4_case_insensitive;
typedef bool fattr4_case_preserving;
typedef bool fattr4_chown_restricted;
typedef uint64_t fattr4_fileid;
typedef uint64_t fattr4_files_avail;
typedef nfs_fh4 fattr4_filehandle;
typedef uint64_t fattr4_files_free;
typedef uint64_t fattr4_files_total;
typedef fs_locations4 fattr4_fs_locations;
typedef bool fattr4_hidden;
typedef bool fattr4_homogeneous;
typedef uint64_t fattr4_maxfilesize;
typedef uint32_t fattr4_maxlink;
typedef uint32_t fattr4_maxname;
typedef uint64_t fattr4_maxread;
typedef uint64_t fattr4_maxwrite;
typedef utf8string fattr4_mimetype;
typedef mode4 fattr4_mode;
typedef bool fattr4_no_trunc;
typedef uint32_t fattr4_numlinks;
typedef utf8string fattr4_owner;
typedef utf8string fattr4_owner_group;
typedef uint64_t fattr4_quota_avail_hard;
typedef uint64_t fattr4_quota_avail_soft;
typedef uint64_t fattr4_quota_used;
typedef specdata4 fattr4_rawdev;
typedef uint64_t fattr4_space_avail;
typedef uint64_t fattr4_space_free;
typedef uint64_t fattr4_space_total;
typedef uint64_t fattr4_space_used;
typedef bool fattr4_system;
typedef nfstime4 fattr4_time_access;
typedef settime4 fattr4_time_access_set;
typedef nfstime4 fattr4_time_backup;
typedef nfstime4 fattr4_time_create;
typedef nfstime4 fattr4_time_delta;
typedef nfstime4 fattr4_time_metadata;
typedef nfstime4 fattr4_time_modify;
typedef settime4 fattr4_time_modify_set;
/*
* Mandatory Attributes
*/
const FATTR4_SUPPORTED_ATTRS = 0;
const FATTR4_TYPE = 1;
const FATTR4_FH_EXPIRE_TYPE = 2;
const FATTR4_CHANGE = 3;
const FATTR4_SIZE = 4;
const FATTR4_LINK_SUPPORT = 5;
const FATTR4_SYMLINK_SUPPORT = 6;
const FATTR4_NAMED_ATTR = 7;
const FATTR4_FSID = 8;
const FATTR4_UNIQUE_HANDLES = 9;
const FATTR4_LEASE_TIME = 10;
const FATTR4_RDATTR_ERROR = 11;
/*
* Recommended Attributes
*/
const FATTR4_ACL = 12;
const FATTR4_ACLSUPPORT = 13;
const FATTR4_ARCHIVE = 14;
const FATTR4_CANSETTIME = 15;
const FATTR4_CASE_INSENSITIVE = 16;
const FATTR4_CASE_PRESERVING = 17;
const FATTR4_CHOWN_RESTRICTED = 18;
const FATTR4_FILEHANDLE = 19;
const FATTR4_FILEID = 20;
const FATTR4_FILES_AVAIL = 21;
const FATTR4_FILES_FREE = 22;
const FATTR4_FILES_TOTAL = 23;
const FATTR4_FS_LOCATIONS = 24;
const FATTR4_HIDDEN = 25;
const FATTR4_HOMOGENEOUS = 26;
const FATTR4_MAXFILESIZE = 27;
const FATTR4_MAXLINK = 28;
const FATTR4_MAXNAME = 29;
const FATTR4_MAXREAD = 30;
const FATTR4_MAXWRITE = 31;
const FATTR4_MIMETYPE = 32;
const FATTR4_MODE = 33;
const FATTR4_NO_TRUNC = 34;
const FATTR4_NUMLINKS = 35;
const FATTR4_OWNER = 36;
const FATTR4_OWNER_GROUP = 37;
const FATTR4_QUOTA_AVAIL_HARD = 38;
const FATTR4_QUOTA_AVAIL_SOFT = 39;
const FATTR4_QUOTA_USED = 40;
const FATTR4_RAWDEV = 41;
const FATTR4_SPACE_AVAIL = 42;
const FATTR4_SPACE_FREE = 43;
const FATTR4_SPACE_TOTAL = 44;
const FATTR4_SPACE_USED = 45;
const FATTR4_SYSTEM = 46;
const FATTR4_TIME_ACCESS = 47;
const FATTR4_TIME_ACCESS_SET = 48;
const FATTR4_TIME_BACKUP = 49;
const FATTR4_TIME_CREATE = 50;
const FATTR4_TIME_DELTA = 51;
const FATTR4_TIME_METADATA = 52;
const FATTR4_TIME_MODIFY = 53;
const FATTR4_TIME_MODIFY_SET = 54;
typedef opaque attrlist4<>;
/*
* File attribute container
*/
struct fattr4 {
bitmap4 attrmask;
attrlist4 attr_vals;
};
/*
* Change info for the client
*/
struct change_info4 {
bool atomic;
changeid4 before;
changeid4 after;
};
struct clientaddr4 {
/* see struct rpcb in RFC 1833 */
string r_netid<>; /* network id */
string r_addr<>; /* universal address */
};
/*
* Callback program info as provided by the client
*/
struct cb_client4 {
unsigned int cb_program;
clientaddr4 cb_location;
};
/*
* Client ID
*/
struct nfs_client_id4 {
verifier4 verifier;
opaque id<>;
};
struct nfs_lockowner4 {
clientid4 clientid;
opaque owner<>;
};
enum nfs_lock_type4 {
READ_LT = 1,
WRITE_LT = 2,
READW_LT = 3, /* blocking read */
WRITEW_LT = 4 /* blocking write */
};
/*
* ACCESS: Check access permission
*/
const ACCESS4_READ = 0x00000001;
const ACCESS4_LOOKUP = 0x00000002;
const ACCESS4_MODIFY = 0x00000004;
const ACCESS4_EXTEND = 0x00000008;
const ACCESS4_DELETE = 0x00000010;
const ACCESS4_EXECUTE = 0x00000020;
struct ACCESS4args {
/* CURRENT_FH: object */
uint32_t access;
};
struct ACCESS4resok {
uint32_t supported;
uint32_t access;
};
union ACCESS4res switch (nfsstat4 status) {
case NFS4_OK:
ACCESS4resok resok4;
default:
void;
};
/*
* CLOSE: Close a file and release share locks
*/
struct CLOSE4args {
/* CURRENT_FH: object */
seqid4 seqid;
stateid4 stateid;
};
union CLOSE4res switch (nfsstat4 status) {
case NFS4_OK:
stateid4 stateid;
default:
void;
};
/*
* COMMIT: Commit cached data on server to stable storage
*/
struct COMMIT4args {
/* CURRENT_FH: file */
offset4 offset;
count4 count;
};
struct COMMIT4resok {
verifier4 writeverf;
};
union COMMIT4res switch (nfsstat4 status) {
case NFS4_OK:
COMMIT4resok resok4;
default:
void;
};
/*
* CREATE: Create a file
*/
union createtype4 switch (nfs_ftype4 type) {
case NF4LNK:
linktext4 linkdata;
case NF4BLK:
case NF4CHR:
specdata4 devdata;
case NF4SOCK:
case NF4FIFO:
case NF4DIR:
void;
};
struct CREATE4args {
/* CURRENT_FH: directory for creation */
component4 objname;
createtype4 objtype;
};
struct CREATE4resok {
change_info4 cinfo;
};
union CREATE4res switch (nfsstat4 status) {
case NFS4_OK:
CREATE4resok resok4;
default:
void;
};
/*
* DELEGPURGE: Purge Delegations Awaiting Recovery
*/
struct DELEGPURGE4args {
clientid4 clientid;
};
struct DELEGPURGE4res {
nfsstat4 status;
};
/*
* DELEGRETURN: Return a delegation
*/
struct DELEGRETURN4args {
stateid4 stateid;
};That's the whole chunk, babe! The attribute IDs and ACE bits are literally the building blocks everything else leans on, so they're worth a slow scroll 🦋🌸
okay bestie, we're deep in the XDR now and it's giving the full operation lineup 💅✨
struct DELEGRETURN4res {
nfsstat4 status;
};
/*
* GETATTR: Get file attributes
*/
struct GETATTR4args {
/* CURRENT_FH: directory or file */
bitmap4 attr_request;
};
struct GETATTR4resok {
fattr4 obj_attributes;
};
union GETATTR4res switch (nfsstat4 status) {
case NFS4_OK:
GETATTR4resok resok4;
default:
void;
};
/*
* GETFH: Get current filehandle
*/
struct GETFH4resok {
nfs_fh4 object;
};
union GETFH4res switch (nfsstat4 status) {
case NFS4_OK:
GETFH4resok resok4;
default:
void;
};
/*
* LINK: Create link to an object
*/
struct LINK4args {
/* SAVED_FH: source object */
/* CURRENT_FH: target directory */
component4 newname;
};
struct LINK4resok {
change_info4 cinfo;
};
union LINK4res switch (nfsstat4 status) {
case NFS4_OK:
LINK4resok resok4;
default:
void;
};
/*
* LOCK/LOCKT/LOCKU: Record lock management
*/
struct LOCK4args {
/* CURRENT_FH: file */
nfs_lock_type4 locktype;
seqid4 seqid;
bool reclaim;
stateid4 stateid;
offset4 offset;
length4 length;
};
struct LOCK4denied {
nfs_lockowner4 owner;
offset4 offset;
length4 length;
};
union LOCK4res switch (nfsstat4 status) {
case NFS4_OK:
stateid4 stateid;
case NFS4ERR_DENIED:
LOCK4denied denied;
default:
void;
};
struct LOCKT4args {
/* CURRENT_FH: file */
nfs_lock_type4 locktype;
nfs_lockowner4 owner;
offset4 offset;
length4 length;
};
union LOCKT4res switch (nfsstat4 status) {
case NFS4ERR_DENIED:
LOCK4denied denied;
case NFS4_OK:
void;
default:
void;
};
struct LOCKU4args {
/* CURRENT_FH: file */
nfs_lock_type4 locktype;
seqid4 seqid;
stateid4 stateid;
offset4 offset;
length4 length;
};
union LOCKU4res switch (nfsstat4 status) {
case NFS4_OK:
stateid4 stateid;
default:
void;
};
/*
* LOOKUP: Lookup filename
*/
struct LOOKUP4args {
/* CURRENT_FH: directory */
pathname4 path;
};
struct LOOKUP4res {
/* CURRENT_FH: object */
nfsstat4 status;
};
/*
* LOOKUPP: Lookup parent directory
*/
struct LOOKUPP4res {
/* CURRENT_FH: directory */
nfsstat4 status;
};
/*
* NVERIFY: Verify attributes different
*/
struct NVERIFY4args {
/* CURRENT_FH: object */
fattr4 obj_attributes;
};
struct NVERIFY4res {
nfsstat4 status;
};
/*
* Various definitions for OPEN
*/
enum createmode4 {
UNCHECKED4 = 0,
GUARDED4 = 1,
EXCLUSIVE4 = 2
};
union createhow4 switch (createmode4 mode) {
case UNCHECKED4:
case GUARDED4:
fattr4 createattrs;
case EXCLUSIVE4:
verifier4 createverf;
};
enum opentype4 {
OPEN4_NOCREATE = 0,
OPEN4_CREATE = 1
};
union openflag4 switch (opentype4 opentype) {
case OPEN4_CREATE:
createhow4 how;
default:
void;
};
/* Next definitions used for OPEN delegation */
enum limit_by4 {
NFS_LIMIT_SIZE = 1,
NFS_LIMIT_BLOCKS = 2
/* others as needed */
};
struct nfs_modified_limit4 {
uint32_t num_blocks;
uint32_t bytes_per_block;
};
union nfs_space_limit4 switch (limit_by4 limitby) {
/* limit specified as file size */
case NFS_LIMIT_SIZE:
uint64_t filesize;
/* limit specified by number of blocks */
case NFS_LIMIT_BLOCKS:
nfs_modified_limit4 mod_blocks;
} ;
/*
* Share Access and Deny constants for open argument
*/
const OPEN4_SHARE_ACCESS_READ = 0x00000001;
const OPEN4_SHARE_ACCESS_WRITE = 0x00000002;
const OPEN4_SHARE_ACCESS_BOTH = 0x00000003;
const OPEN4_SHARE_DENY_NONE = 0x00000000;
const OPEN4_SHARE_DENY_READ = 0x00000001;
const OPEN4_SHARE_DENY_WRITE = 0x00000002;
const OPEN4_SHARE_DENY_BOTH = 0x00000003;
enum open_delegation_type4 {
OPEN_DELEGATE_NONE = 0,
OPEN_DELEGATE_READ = 1,
OPEN_DELEGATE_WRITE = 2
};
enum open_claim_type4 {
CLAIM_NULL = 0,
CLAIM_PREVIOUS = 1,
CLAIM_DELEGATE_CUR = 2,
CLAIM_DELEGATE_PREV = 3
};
struct open_claim_delegate_cur4 {
pathname4 file;
stateid4 delegate_stateid;
};
union open_claim4 switch (open_claim_type4 claim) {
/*
* No special rights to file. Ordinary OPEN of the specified file.
*/
case CLAIM_NULL:
/* CURRENT_FH: directory */
pathname4 file;
/*
* Right to the file established by an open previous to server
* reboot. File identified by filehandle obtained at that time
* rather than by name.
*/
case CLAIM_PREVIOUS:
/* CURRENT_FH: file being reclaimed */
uint32_t delegate_type;
/*
* Right to file based on a delegation granted by the server.
* File is specified by name.
*/
case CLAIM_DELEGATE_CUR:
/* CURRENT_FH: directory */
open_claim_delegate_cur4 delegate_cur_info;
/* Right to file based on a delegation granted to a previous boot
* instance of the client. File is specified by name.
*/
case CLAIM_DELEGATE_PREV:
/* CURRENT_FH: directory */
pathname4 file_delegate_prev;
};
/*
* OPEN: Open a file, potentially receiving an open delegation
*/
struct OPEN4args {
open_claim4 claim;
openflag4 openhow;
nfs_lockowner4 owner;
seqid4 seqid;
uint32_t share_access;
uint32_t share_deny;
};
struct open_read_delegation4 {
stateid4 stateid; /* Stateid for delegation*/
bool recall; /* Pre-recalled flag for
delegations obtained
by reclaim
(CLAIM_PREVIOUS) */
nfsace4 permissions; /* Defines users who don't
need an ACCESS call to
open for read */
};
struct open_write_delegation4 {
stateid4 stateid; /* Stateid for delegation */
bool recall; /* Pre-recalled flag for
delegations obtained
by reclaim
(CLAIM_PREVIOUS) */
nfs_space_limit4 space_limit; /* Defines condition that
the client must check to
determine whether the
file needs to be flushed
to the server on close.
*/
nfsace4 permissions; /* Defines users who don't
need an ACCESS call as
part of a delegated
open. */
};
union open_delegation4
switch (open_delegation_type4 delegation_type) {
case OPEN_DELEGATE_NONE:
void;
case OPEN_DELEGATE_READ:
open_read_delegation4 read;
case OPEN_DELEGATE_WRITE:
open_write_delegation4 write;
};
/*
* Result flags
*/
/* Mandatory locking is in effect for this file. */
const OPEN4_RESULT_MLOCK = 0x00000001;
/* Client must confirm open */
const OPEN4_RESULT_CONFIRM = 0x00000002;
struct OPEN4resok {
stateid4 stateid; /* Stateid for open */
change_info4 cinfo; /* Directory Change Info */
uint32_t rflags; /* Result flags */
verifier4 open_confirm; /* OPEN_CONFIRM verifier */
open_delegation4 delegation; /* Info on any open
delegation */
};
union OPEN4res switch (nfsstat4 status) {
case NFS4_OK:
/* CURRENT_FH: opened file */
OPEN4resok resok4;
default:
void;
};
/*
* OPENATTR: open named attributes directory
*/
struct OPENATTR4res {
/* CURRENT_FH: name attr directory*/
nfsstat4 status;
};
/*
* OPEN_CONFIRM: confirm the open
*/
struct OPEN_CONFIRM4args {
/* CURRENT_FH: opened file */
seqid4 seqid;
verifier4 open_confirm; /* OPEN_CONFIRM verifier */
};
struct OPEN_CONFIRM4resok {
stateid4 stateid;
};
union OPEN_CONFIRM4res switch (nfsstat4 status) {
case NFS4_OK:
OPEN_CONFIRM4resok resok4;
default:
void;
};
/*
* OPEN_DOWNGRADE: downgrade the access/deny for a file
*/
struct OPEN_DOWNGRADE4args {
/* CURRENT_FH: opened file */
stateid4 stateid;
seqid4 seqid;
uint32_t share_access;
uint32_t share_deny;
};
struct OPEN_DOWNGRADE4resok {
stateid4 stateid;
};
union OPEN_DOWNGRADE4res switch(nfsstat4 status) {
case NFS4_OK:
OPEN_DOWNGRADE4resok resok4;
default:
void;
};
/*
* PUTFH: Set current filehandle
*/
struct PUTFH4args {
nfs_fh4 object;
};
struct PUTFH4res {
/* CURRENT_FH: */
nfsstat4 status;
};
/*
* PUTPUBFH: Set public filehandle
*/
struct PUTPUBFH4res {
/* CURRENT_FH: public fh */
nfsstat4 status;
};
/*
* PUTROOTFH: Set root filehandle
*/
struct PUTROOTFH4res {
/* CURRENT_FH: root fh */
nfsstat4 status;
};
/*
* READ: Read from file
*/
struct READ4args {
/* CURRENT_FH: file */
stateid4 stateid;
offset4 offset;
count4 count;
};
struct READ4resok {
bool eof;
opaque data<>;
};
union READ4res switch (nfsstat4 status) {
case NFS4_OK:
READ4resok resok4;
default:
void;
};
/*
* READDIR: Read directory
*/
struct READDIR4args {
/* CURRENT_FH: directory */
nfs_cookie4 cookie;
verifier4 cookieverf;
count4 dircount;
count4 maxcount;
bitmap4 attr_request;
};
struct entry4 {
nfs_cookie4 cookie;
component4 name;
fattr4 attrs;
entry4 *nextentry;
};Okay bestie, the XDR keeps going and it's giving perfectly organized ✨🎀
struct dirlist4 {
entry4 *entries;
bool eof;
};
struct READDIR4resok {
verifier4 cookieverf;
dirlist4 reply;
};
union READDIR4res switch (nfsstat4 status) {
case NFS4_OK:
READDIR4resok resok4;
default:
void;
};
/*
* READLINK: Read symbolic link
*/
struct READLINK4resok {
linktext4 link;
};
union READLINK4res switch (nfsstat4 status) {
case NFS4_OK:
READLINK4resok resok4;
default:
void;
};
/*
* REMOVE: Remove filesystem object
*/
struct REMOVE4args {
/* CURRENT_FH: directory */
component4 target;
};
struct REMOVE4resok {
change_info4 cinfo;
};
union REMOVE4res switch (nfsstat4 status) {
case NFS4_OK:
REMOVE4resok resok4;
default:
void;
};
/*
* RENAME: Rename directory entry
*/
struct RENAME4args {
/* SAVED_FH: source directory */
component4 oldname;
/* CURRENT_FH: target directory */
component4 newname;
};
struct RENAME4resok {
change_info4 source_cinfo;
change_info4 target_cinfo;
};
union RENAME4res switch (nfsstat4 status) {
case NFS4_OK:
RENAME4resok resok4;
default:
void;
};
/*
* RENEW: Renew a Lease
*/
struct RENEW4args {
stateid4 stateid;
};
struct RENEW4res {
nfsstat4 status;
};
/*
* RESTOREFH: Restore saved filehandle
*/
struct RESTOREFH4res {
/* CURRENT_FH: value of saved fh */
nfsstat4 status;
};
/*
* SAVEFH: Save current filehandle
*/
struct SAVEFH4res {
/* SAVED_FH: value of current fh */
nfsstat4 status;
};
/*
* SECINFO: Obtain Available Security Mechanisms
*/
struct SECINFO4args {
/* CURRENT_FH: */
component4 name;
};
/*
* From RFC 2203
*/
enum rpc_gss_svc_t {
RPC_GSS_SVC_NONE = 1,
RPC_GSS_SVC_INTEGRITY = 2,
RPC_GSS_SVC_PRIVACY = 3
};
struct rpcsec_gss_info {
sec_oid4 oid;
qop4 qop;
rpc_gss_svc_t service;
};
struct secinfo4 {
uint32_t flavor;
/* null for AUTH_SYS, AUTH_NONE;
contains rpcsec_gss_info for
RPCSEC_GSS. */
opaque flavor_info<>;
};
typedef secinfo4 SECINFO4resok<>;
union SECINFO4res switch (nfsstat4 status) {
case NFS4_OK:
SECINFO4resok resok4;
default:
void;
};
/*
* SETATTR: Set attributes
*/
struct SETATTR4args {
/* CURRENT_FH: target object */
stateid4 stateid;
fattr4 obj_attributes;
};
struct SETATTR4res {
nfsstat4 status;
bitmap4 attrsset;
};
/*
* SETCLIENTID
*/
struct SETCLIENTID4args {
nfs_client_id4 client;
cb_client4 callback;
};
struct SETCLIENTID4resok {
clientid4 clientid;
verifier4 setclientid_confirm;
};
union SETCLIENTID4res switch (nfsstat4 status) {
case NFS4_OK:
SETCLIENTID4resok resok4;
case NFS4ERR_CLID_INUSE:
clientaddr4 client_using;
default:
void;
};
struct SETCLIENTID_CONFIRM4args {
verifier4 setclientid_confirm;
};
struct SETCLIENTID_CONFIRM4res {
nfsstat4 status;
};
/*
* VERIFY: Verify attributes same
*/
struct VERIFY4args {
/* CURRENT_FH: object */
fattr4 obj_attributes;
};
struct VERIFY4res {
nfsstat4 status;
};
/*
* WRITE: Write to file
*/
enum stable_how4 {
UNSTABLE4 = 0,
DATA_SYNC4 = 1,
FILE_SYNC4 = 2
};
struct WRITE4args {
/* CURRENT_FH: file */
stateid4 stateid;
offset4 offset;
stable_how4 stable;
opaque data<>;
};
struct WRITE4resok {
count4 count;
stable_how4 committed;
verifier4 writeverf;
};
union WRITE4res switch (nfsstat4 status) {
case NFS4_OK:
WRITE4resok resok4;
default:
void;
};
/*
* Operation arrays
*/
enum nfs_opnum4 {
OP_ACCESS = 3,
OP_CLOSE = 4,
OP_COMMIT = 5,
OP_CREATE = 6,
OP_DELEGPURGE = 7,
OP_DELEGRETURN = 8,
OP_GETATTR = 9,
OP_GETFH = 10,
OP_LINK = 11,
OP_LOCK = 12,
OP_LOCKT = 13,
OP_LOCKU = 14,
OP_LOOKUP = 15,
OP_LOOKUPP = 16,
OP_NVERIFY = 17,
OP_OPEN = 18,
OP_OPENATTR = 19,
OP_OPEN_CONFIRM = 20,
OP_OPEN_DOWNGRADE = 21,
OP_PUTFH = 22,
OP_PUTPUBFH = 23,
OP_PUTROOTFH = 24,
OP_READ = 25,
OP_READDIR = 26,
OP_READLINK = 27,
OP_REMOVE = 28,
OP_RENAME = 29,
OP_RENEW = 30,
OP_RESTOREFH = 31,
OP_SAVEFH = 32,
OP_SECINFO = 33,
OP_SETATTR = 34,
OP_SETCLIENTID = 35,
OP_SETCLIENTID_CONFIRM = 36,
OP_VERIFY = 37,
OP_WRITE = 38
};
union nfs_argop4 switch (nfs_opnum4 argop) {
case OP_ACCESS: ACCESS4args opaccess;
case OP_CLOSE: CLOSE4args opclose;
case OP_COMMIT: COMMIT4args opcommit;
case OP_CREATE: CREATE4args opcreate;
case OP_DELEGPURGE: DELEGPURGE4args opdelegpurge;
case OP_DELEGRETURN: DELEGRETURN4args opdelegreturn;
case OP_GETATTR: GETATTR4args opgetattr;
case OP_GETFH: void;
case OP_LINK: LINK4args oplink;
case OP_LOCK: LOCK4args oplock;
case OP_LOCKT: LOCKT4args oplockt;
case OP_LOCKU: LOCKU4args oplocku;
case OP_LOOKUP: LOOKUP4args oplookup;
case OP_LOOKUPP: void;
case OP_NVERIFY: NVERIFY4args opnverify;
case OP_OPEN: OPEN4args opopen;
case OP_OPENATTR: void;
case OP_OPEN_CONFIRM: OPEN_CONFIRM4args opopen_confirm;
case OP_OPEN_DOWNGRADE: OPEN_DOWNGRADE4args opopen_downgrade;
case OP_PUTFH: PUTFH4args opputfh;
case OP_PUTPUBFH: void;
case OP_PUTROOTFH: void;
case OP_READ: READ4args opread;
case OP_READDIR: READDIR4args opreaddir;
case OP_READLINK: void;
case OP_REMOVE: REMOVE4args opremove;
case OP_RENAME: RENAME4args oprename;
case OP_RENEW: RENEW4args oprenew;
case OP_RESTOREFH: void;
case OP_SAVEFH: void;
case OP_SECINFO: SECINFO4args opsecinfo;
case OP_SETATTR: SETATTR4args opsetattr;
case OP_SETCLIENTID: SETCLIENTID4args opsetclientid;
case OP_SETCLIENTID_CONFIRM: SETCLIENTID_CONFIRM4args
opsetclientid_confirm;
case OP_VERIFY: VERIFY4args opverify;
case OP_WRITE: WRITE4args opwrite;
};
union nfs_resop4 switch (nfs_opnum4 resop){
case OP_ACCESS: ACCESS4res opaccess;
case OP_CLOSE: CLOSE4res opclose;
case OP_COMMIT: COMMIT4res opcommit;
case OP_CREATE: CREATE4res opcreate;
case OP_DELEGPURGE: DELEGPURGE4res opdelegpurge;
case OP_DELEGRETURN: DELEGRETURN4res opdelegreturn;
case OP_GETATTR: GETATTR4res opgetattr;
case OP_GETFH: GETFH4res opgetfh;
case OP_LINK: LINK4res oplink;
case OP_LOCK: LOCK4res oplock;
case OP_LOCKT: LOCKT4res oplockt;
case OP_LOCKU: LOCKU4res oplocku;
case OP_LOOKUP: LOOKUP4res oplookup;
case OP_LOOKUPP: LOOKUPP4res oplookupp;
case OP_NVERIFY: NVERIFY4res opnverify;
case OP_OPEN: OPEN4res opopen;
case OP_OPENATTR: OPENATTR4res opopenattr;
case OP_OPEN_CONFIRM: OPEN_CONFIRM4res opopen_confirm;
case OP_OPEN_DOWNGRADE: OPEN_DOWNGRADE4res opopen_downgrade;
case OP_PUTFH: PUTFH4res opputfh;
case OP_PUTPUBFH: PUTPUBFH4res opputpubfh;
case OP_PUTROOTFH: PUTROOTFH4res opputrootfh;
case OP_READ: READ4res opread;
case OP_READDIR: READDIR4res opreaddir;
case OP_READLINK: READLINK4res opreadlink;
case OP_REMOVE: REMOVE4res opremove;
case OP_RENAME: RENAME4res oprename;
case OP_RENEW: RENEW4res oprenew;
case OP_RESTOREFH: RESTOREFH4res oprestorefh;
case OP_SAVEFH: SAVEFH4res opsavefh;
case OP_SECINFO: SECINFO4res opsecinfo;
case OP_SETATTR: SETATTR4res opsetattr;
case OP_SETCLIENTID: SETCLIENTID4res opsetclientid;
case OP_SETCLIENTID_CONFIRM: SETCLIENTID_CONFIRM4res
opsetclientid_confirm;
case OP_VERIFY: VERIFY4res opverify;
case OP_WRITE: WRITE4res opwrite;
};
struct COMPOUND4args {
utf8string tag;
uint32_t minorversion;
nfs_argop4 argarray<>;
};
struct COMPOUND4res {
nfsstat4 status;
utf8string tag;
nfs_resop4 resarray<>;
};
/*
* Remote file service routines
*/
program NFS4_PROGRAM {
version NFS_V4 {
void
NFSPROC4_NULL(void) = 0;
COMPOUND4res
NFSPROC4_COMPOUND(COMPOUND4args) = 1;
} = 4;
} = 100003;
/*
* NFS4 Callback Procedure Definitions and Program
*/
/*
* CB_GETATTR: Get Current Attributes
*/
struct CB_GETATTR4args {
nfs_fh4 fh;
bitmap4 attr_request;
};
struct CB_GETATTR4resok {
fattr4 obj_attributes;
};
union CB_GETATTR4res switch (nfsstat4 status) {
case NFS4_OK:
CB_GETATTR4resok resok4;
default:
void;
};
/*
* CB_RECALL: Recall an Open Delegation
*/
struct CB_RECALL4args {
stateid4 stateid;
bool truncate;
nfs_fh4 fh;
};
struct CB_RECALL4res {
nfsstat4 status;
};
/*
* Various definitions for CB_COMPOUND
*/
enum nfs_cb_opnum4 {
OP_CB_GETATTR = 3,
OP_CB_RECALL = 4
};
union nfs_cb_argop4 switch (unsigned argop) {
case OP_CB_GETATTR: CB_GETATTR4args opcbgetattr;
case OP_CB_RECALL: CB_RECALL4args opcbrecall;
};
union nfs_cb_resop4 switch (unsigned resop){
case OP_CB_GETATTR: CB_GETATTR4res opcbgetattr;
case OP_CB_RECALL: CB_RECALL4res opcbrecall;
};
struct CB_COMPOUND4args {
utf8string tag;
uint32_t minorversion;
nfs_cb_argop4 argarray<>;
};
struct CB_COMPOUND4res {
nfsstat4 status;
utf8string tag;
nfs_cb_resop4 resarray<>;
};That's the whole XDR block, and every struct, enum and union is exactly as the spec wrote it 🫶💜
/*
* Program number is in the transient range since the client
* will assign the exact transient program number and provide
* that to the server via the SETCLIENTID operation.
*/
program NFS4_CALLBACK {
version NFS_CB {
void
CB_NULL(void) = 0;
CB_COMPOUND4res
CB_COMPOUND(CB_COMPOUND4args) = 1;
} = 1;
} = 40000000;19. Bibliography 📚✨
okay bestie, these are the receipts 🧾 every source the spec leans on, so you can go dig up the originals if you're curious 🤓
- [Floyd] S. Floyd, V. Jacobson, "The Synchronization of Periodic Routing Messages," IEEE/ACM Transactions on Networking, 2(2), pp. 122-136, April 1994.
- [Gray] C. Gray, D. Cheriton, "Leases: An Efficient Fault-Tolerant Mechanism for Distributed File Cache Consistency," Proceedings of the Twelfth Symposium on Operating Systems Principles, p. 202-210, December 1989.
- [ISO10646] "ISO/IEC 10646-1:1993. International Standard -- Information technology -- Universal Multiple-Octet Coded Character Set (UCS) -- Part 1: Architecture and Basic Multilingual Plane."
- [Juszczak] Juszczak, Chet, "Improving the Performance and Correctness of an NFS Server," USENIX Conference Proceedings, USENIX Association, Berkeley, CA, June 1990, pages 53-63. Describes reply cache implementation that avoids work in the server by handling duplicate requests. More important, though listed as a side-effect, the reply cache aids in the avoidance of destructive non-idempotent operation re-application -- improving correctness.
- [Kazar] Kazar, Michael Leon, "Synchronization and Caching Issues in the Andrew File System," USENIX Conference Proceedings, USENIX Association, Berkeley, CA, Dallas Winter 1988, pages 27-36. A description of the cache consistency scheme in AFS. Contrasted with other distributed file systems.
- [Macklem] Macklem, Rick, "Lessons Learned Tuning the 4.3BSD Reno Implementation of the NFS Protocol," Winter USENIX Conference Proceedings, USENIX Association, Berkeley, CA, January 1991. Describes performance work in tuning the 4.3BSD Reno NFS implementation. Describes performance improvement (reduced CPU loading) through elimination of data copies.
- [Mogul] Mogul, Jeffrey C., "A Recovery Protocol for Spritely NFS," USENIX File System Workshop Proceedings, Ann Arbor, MI, USENIX Association, Berkeley, CA, May 1992. Second paper on Spritely NFS proposes a lease-based scheme for recovering state of consistency protocol.
- [Nowicki] Nowicki, Bill, "Transport Issues in the Network File System," ACM SIGCOMM newsletter Computer Communication Review, April 1989. A brief description of the basis for the dynamic retransmission work.
- [Pawlowski] Pawlowski, Brian, Ron Hixon, Mark Stein, Joseph Tumminaro, "Network Computing in the UNIX and IBM Mainframe Environment," Uniforum `89 Conf. Proc., (1989) Description of an NFS server implementation for IBM's MVS operating system.
- [RFC1094] Sun Microsystems, Inc., "NFS: Network File System Protocol Specification", RFC 1094, March 1989.
- [RFC1345] Simonsen, K., "Character Mnemonics & Character Sets", RFC 1345, June 1992.
- [RFC1700] Reynolds, J. and J. Postel, "Assigned Numbers", STD 2, RFC 1700, October 1994.
- [RFC1813] Callaghan, B., Pawlowski, B. and P. Staubach, "NFS Version 3 Protocol Specification", RFC 1813, June 1995.
- [RFC1831] Srinivasan, R., "RPC: Remote Procedure Call Protocol Specification Version 2", RFC 1831, August 1995.
- [RFC1832] Srinivasan, R., "XDR: External Data Representation Standard", RFC 1832, August 1995.
- [RFC1833] Srinivasan, R., "Binding Protocols for ONC RPC Version 2", RFC 1833, August 1995.
- [RFC2025] Adams, C., "The Simple Public-Key GSS-API Mechanism (SPKM)", RFC 2025, October 1996.
- [RFC2054] Callaghan, B., "WebNFS Client Specification", RFC 2054, October 1996.
- [RFC2055] Callaghan, B., "WebNFS Server Specification", RFC 2055, October 1996.
- [RFC2078] Linn, J., "Generic Security Service Application Program Interface, Version 2", RFC 2078, January 1997.
- [RFC2152] Goldsmith, D., "UTF-7 A Mail-Safe Transformation Format of Unicode", RFC 2152, May 1997.
- [RFC2203] Eisler, M., Chiu, A. and L. Ling, "RPCSEC_GSS Protocol Specification", RFC 2203, August 1995.
- [RFC2277] Alvestrand, H., "IETF Policy on Character Sets and Languages", BCP 18, RFC 2277, January 1998.
- [RFC2279] Yergeau, F., "UTF-8, a transformation format of ISO 10646", RFC 2279, January 1998.
- [RFC2623] Eisler, M., "NFS Version 2 and Version 3 Security Issues and the NFS Protocol's Use of RPCSEC_GSS and Kerberos V5", RFC 2623, June 1999.
- [RFC2624] Shepler, S., "NFS Version 4 Design Considerations", RFC 2624, June 1999.
- [RFC2847] Eisler, M., "LIPKEY - A Low Infrastructure Public Key Mechanism Using SPKM", RFC 2847, June 2000.
- [Sandberg] Sandberg, R., D. Goldberg, S. Kleiman, D. Walsh, B. Lyon, "Design and Implementation of the Sun Network Filesystem," USENIX Conference Proceedings, USENIX Association, Berkeley, CA, Summer 1985. The basic paper describing the SunOS implementation of the NFS version 2 protocol, and discusses the goals, protocol specification and trade-offs.
- [Srinivasan] Srinivasan, V., Jeffrey C. Mogul, "Spritely NFS: Implementation and Performance of Cache Consistency Protocols", WRL Research Report 89/5, Digital Equipment Corporation Western Research Laboratory, 100 Hamilton Ave., Palo Alto, CA, 94301, May 1989. This paper analyzes the effect of applying a Sprite-like consistency protocol applied to standard NFS. The issues of recovery in a stateful environment are covered in [Mogul].
- [Unicode1] The Unicode Consortium, "The Unicode Standard, Version 3.0", Addison-Wesley Developers Press, Reading, MA, 2000. ISBN 0-201-61633-5. More information available at: http://www.unicode.org/
- [Unicode2] "Unsupported Scripts" Unicode, Inc., The Unicode Consortium, P.O. Box 700519, San Jose, CA 95710-0519 USA, September 1999 http://www.unicode.org/unicode/standard/unsupported.html
- [XNFS] The Open Group, Protocols for Interworking: XNFS, Version 3W, The Open Group, 1010 El Camino Real Suite 380, Menlo Park, CA 94025, ISBN 1-85912-184-5, February 1998. HTML version available: http://www.opengroup.org
20. Authors 🫶
the squad that wrote this, in their own words 💜
20.1. Editor's Address 🖋️
- Spencer Shepler
- Sun Microsystems, Inc.
- 7808 Moonflower Drive
- Austin, Texas 78750
- Phone: +1 512-349-9376
- EMail: spencer.shepler@sun.com
20.2. Authors' Addresses 🌸
- Carl Beame
- Hummingbird Ltd.
- EMail: beame@bws.com
- Brent Callaghan
- Sun Microsystems, Inc.
- 901 San Antonio Road
- Palo Alto, CA 94303
- Phone: +1 650-786-5067
- EMail: brent.callaghan@sun.com
- Mike Eisler
- 5565 Wilson Road
- Colorado Springs, CO 80919
- Phone: +1 719-599-9026
- EMail: mike@eisler.com
- David Noveck
- Network Appliance
- 375 Totten Pond Road
- Waltham, MA 02451
- Phone: +1 781-895-4949
- E-mail: dnoveck@netapp.com
- David Robinson
- Sun Microsystems, Inc.
- 901 San Antonio Road
- Palo Alto, CA 94303
- Phone: +1 650-786-5088
- EMail: david.robinson@sun.com
- Robert Thurlow
- Sun Microsystems, Inc.
- 901 San Antonio Road
- Palo Alto, CA 94303
- Phone: +1 650-786-5096
- EMail: robert.thurlow@sun.com
20.3. Acknowledgements 🙏
The author thanks and acknowledges:
Neil Brown for his extensive review and comments of various drafts. 💖
21. Full Copyright Statement ⚖️
the legal bit, kept word-for-word so nothing gets lost in translation 🤍
Copyright (C) The Internet Society (2000). All Rights Reserved.
This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English.
The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns.
This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Acknowledgement 💅
Funding for the RFC Editor function is currently provided by the Internet Society. that's a wrap, bestie! 🌈👑
📦 where this page comes from
- ietf: rfc3010 2000-12
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms