RFC 1813
NFS Version 3 Protocol Specification
Part 1 of 22 🎀
NFS Version 3 Protocol Specification 💖
RFC 1813, Network Working Group 🧚 Authors: B. Callaghan, B. Pawlowski, P. Staubach (Sun Microsystems, Inc.) 👑 Category: Informational 📚 Date: June 1995 📅 Obsoletes and updates: none listed in the title block 🙈
Status of this Memo ✨
okay so this memo is just here to give the internet some info 💅 it does NOT specify an internet standard of any kind, and you're free to share it because distribution is unlimited 🦋
IESG Note 📝
the IETF isn't involved in creating or maintaining this spec 🙈 that's the whole point of it not being on the standards track 💀
Abstract 🌸
this paper describes the NFS version 3 protocol 🌷 it's written so that people can build compatible implementations ✨
Table of Contents 📖
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . 3
1.1 Scope of the NFS version 3 protocol . . . . . . . . . . 4
1.2 Useful terms . . . . . . . . . . . . . . . . . . . . . . 5
1.3 Remote Procedure Call . . . . . . . . . . . . . . . . . 5
1.4 External Data Representation . . . . . . . . . . . . . . 5
1.5 Authentication and Permission Checking . . . . . . . . . 7
1.6 Philosophy . . . . . . . . . . . . . . . . . . . . . . . 8
1.7 Changes from the NFS version 2 protocol . . . . . . . . 11
2. RPC Information . . . . . . . . . . . . . . . . . . . . . 14
2.1 Authentication . . . . . . . . . . . . . . . . . . . . . 14
2.2 Constants . . . . . . . . . . . . . . . . . . . . . . . 14
2.3 Transport address . . . . . . . . . . . . . . . . . . . 14
2.4 Sizes . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.5 Basic Data Types . . . . . . . . . . . . . . . . . . . . 15
2.6 Defined Error Numbers . . . . . . . . . . . . . . . . . 17
3. Server Procedures . . . . . . . . . . . . . . . . . . . . 27
3.1 General comments on attributes . . . . . . . . . . . . . 29
3.2 General comments on filenames . . . . . . . . . . . . . 30
3.3.0 NULL: Do nothing . . . . . . . . . . . . . . . . . . . . 31
3.3.1 GETATTR: Get file attributes . . . . . . . . . . . . . . 32
3.3.2 SETATTR: Set file attributes . . . . . . . . . . . . . . 33
3.3.3 LOOKUP: Lookup filename . . . . . . . . . . . . . . . . 37
3.3.4 ACCESS: Check access permission . . . . . . . . . . . . 40
3.3.5 READLINK: Read from symbolic link . . . . . . . . . . . 44
3.3.6 READ: Read from file . . . . . . . . . . . . . . . . . . 46
3.3.7 WRITE: Write to file . . . . . . . . . . . . . . . . . . 49
3.3.8 CREATE: Create a file . . . . . . . . . . . . . . . . . 54
3.3.9 MKDIR: Create a directory . . . . . . . . . . . . . . . 58
3.3.10 SYMLINK: Create a symbolic link . . . . . . . . . . . . 61
3.3.11 MKNOD: Create a special device . . . . . . . . . . . . . 63
3.3.12 REMOVE: Remove a file . . . . . . . . . . . . . . . . . 67
3.3.13 RMDIR: Remove a directory . . . . . . . . . . . . . . . 69
3.3.14 RENAME: Rename a file or directory . . . . . . . . . . . 71
3.3.15 LINK: Create link to an object . . . . . . . . . . . . . 74
3.3.16 READDIR: Read From directory . . . . . . . . . . . . . . 76
3.3.17 READDIRPLUS: Extended read from directory . . . . . . . 80
3.3.18 FSSTAT: Get dynamic file system information . . . . . . 84
3.3.19 FSINFO: Get static file system information . . . . . . . 86
3.3.20 PATHCONF: Retrieve POSIX information . . . . . . . . . . 90
3.3.21 COMMIT: Commit cached data on a server to stable storage 92
4. Implementation issues . . . . . . . . . . . . . . . . . . 96
4.1 Multiple version support . . . . . . . . . . . . . . . . 96
4.2 Server/client relationship . . . . . . . . . . . . . . . 96
4.3 Path name interpretation . . . . . . . . . . . . . . . . 97
4.4 Permission issues . . . . . . . . . . . . . . . . . . . 98
4.5 Duplicate request cache . . . . . . . . . . . . . . . . 99
4.6 File name component handling . . . . . . . . . . . . . . 101
4.7 Synchronous modifying operations . . . . . . . . . . . . 101
4.8 Stable storage . . . . . . . . . . . . . . . . . . . . . 101
4.9 Lookups and name resolution . . . . . . . . . . . . . . 102
4.10 Adaptive retransmission . . . . . . . . . . . . . . . . 102
4.11 Caching policies . . . . . . . . . . . . . . . . . . . . 102
4.12 Stable versus unstable writes. . . . . . . . . . . . . . 103
4.13 32 bit clients/servers and 64 bit clients/servers. . . . 104
5. Appendix I: Mount protocol . . . . . . . . . . . . . . . . 106
5.1 RPC Information . . . . . . . . . . . . . . . . . . . . 106
5.1.1 Authentication . . . . . . . . . . . . . . . . . . . . 106
5.1.2 Constants . . . . . . . . . . . . . . . . . . . . . . 106
5.1.3 Transport address . . . . . . . . . . . . . . . . . . 106
5.1.4 Sizes . . . . . . . . . . . . . . . . . . . . . . . . 106
5.1.5 Basic Data Types . . . . . . . . . . . . . . . . . . . 106
5.2 Server Procedures . . . . . . . . . . . . . . . . . . . 107
5.2.0 NULL: Do nothing . . . . . . . . . . . . . . . . . . . 108
5.2.1 MNT: Add mount entry . . . . . . . . . . . . . . . . . 109
5.2.2 DUMP: Return mount entries . . . . . . . . . . . . . . 110
5.2.3 UMNT: Remove mount entry . . . . . . . . . . . . . . . 111
5.2.4 UMNTALL: Remove all mount entries . . . . . . . . . . 112
5.2.5 EXPORT: Return export list . . . . . . . . . . . . . . 113
6. Appendix II: Lock manager protocol . . . . . . . . . . . . 114
6.1 RPC Information . . . . . . . . . . . . . . . . . . . . 114
6.1.1 Authentication . . . . . . . . . . . . . . . . . . . . 114
6.1.2 Constants . . . . . . . . . . . . . . . . . . . . . . 114
6.1.3 Transport Address . . . . . . . . . . . . . . . . . . 115
6.1.4 Basic Data Types . . . . . . . . . . . . . . . . . . . 115
6.2 NLM Procedures . . . . . . . . . . . . . . . . . . . . . 118
6.2.0 NULL: Do nothing . . . . . . . . . . . . . . . . . . . 120
6.3 Implementation issues . . . . . . . . . . . . . . . . . 120
6.3.1 64-bit offsets and lengths . . . . . . . . . . . . . . 120
6.3.2 File handles . . . . . . . . . . . . . . . . . . . . . 120
7. Appendix III: Bibliography . . . . . . . . . . . . . . . . 122
8. Security Considerations . . . . . . . . . . . . . . . . . 125
9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . 125
10. Authors' Addresses . . . . . . . . . . . . . . . . . . . . 1261. Introduction 🌈
okay so Sun's NFS protocol gives you transparent remote access to shared file systems across networks 🗂️✨ it's giving "my files are right here" even when they're somewhere else entirely 🫶
the NFS protocol is designed to be machine, operating system, network architecture, and transport protocol independent 💅 how it pulls that off is with Remote Procedure Call (RPC) primitives built on top of an eXternal Data Representation (XDR) 🧚 which is honestly iconic
NFS version 2 implementations exist for a bunch of machines, from personal computers all the way up to supercomputers 🔥 the first version of the NFS protocol is specified in the Network File System Protocol Specification [RFC1094] 🌸 and a description of the initial implementation can be found in [Sandberg] 📚
the supporting MOUNT protocol handles the operating system-specific stuff that lets clients attach remote directory trees to a point within the local file system 🎀 the mount process also lets the server grant remote access privileges to a restricted set of clients through export control 👑
the Lock Manager gives you file locking when you're in the NFS environment 🔒 and the Network Lock Manager (NLM) protocol takes the inherently stateful parts of file locking and isolates them into their own separate protocol 🦋
a complete description of all the above protocols and how they're implemented is to be found in [X/OpenNFS] 💜
the purpose of this document is to:
- Specify the NFS version 3 protocol. 💖
- Describe semantics of the protocol through annotation and description of intended implementation. ✨
- Specify the MOUNT version 3 protocol. 🌷
- Briefly describe the changes between the NLM version 3 protocol and the NLM version 4 protocol. 🍓
the normative text is the description of the RPC procedures and arguments and results, which defines the over-the-wire protocol, and the semantics of those procedures 🧠 the material describing implementation practice aids your understanding of the protocol specification and describes some possible implementation issues and solutions 🫶
it's not possible to describe every implementation, so the UNIX operating system implementation of the NFS version 3 protocol is mostly used to give examples 🐧 given that, the implementation discussion does not carry the authority of the description of the over-the-wire protocol itself 🙈
1.1 Scope of the NFS version 3 protocol 🎀
this revision of the NFS protocol addresses new requirements 💅 the need to support larger files and file systems has prompted extensions that allow 64 bit file sizes and offsets 📈 the revision also enhances security by adding support for an access check to be done on the server 🔐 performance changes come in three flavours:
- the number of over-the-wire packets for a given set of file operations goes down, because file attributes get returned on every operation, which means fewer calls to get modified attributes 🚀
- the write throughput bottleneck caused by the synchronous definition of write in the NFS version 2 protocol has been addressed, by adding support so that the NFS server can do unsafe writes 🔥 unsafe writes are writes which have not been committed to stable storage before the operation returns 💀 this specification defines a method for committing these unsafe writes to stable storage in a reliable way 🫶
- limitations on transfer sizes have been relaxed 🦋
the ability to support multiple versions of a protocol in RPC lets implementors of the NFS version 3 protocol define clients and servers that stay backwards compatible with the existing installed base of NFS version 2 protocol implementations 👑
the extensions described here are an evolution of the existing NFS protocol, and most of the design features of the NFS protocol described in [Sandberg] stick around ✨ see Changes from the NFS version 2 protocol on page 11 for a more detailed summary of the changes this revision brings 📖
1.2 Useful terms 📚
in this specification, a "server" is a machine that provides resources to the network 🖥️ a "client" is a machine that accesses resources over the network 💻 a "user" is a person logged in on a client 🧑💻 and an "application" is a program that executes on a client 🌈
1.3. Remote Procedure Call ✨
okay so the Sun Remote Procedure Call specification gives you a procedure-oriented interface to remote services 🧚 each server supplies a program (basically a set of procedures), and the NFS service is one of those programs. the combo of host address, program number, version number, and procedure number pinpoints exactly one remote service procedure 💅 servers can even support multiple versions of a program by using different protocol version numbers, which is so organized of them 🎀
the NFS protocol was designed to not require any specific level of reliability from its lower levels, so it could potentially run on many underlying transport protocols 🦋 the NFS service is built on RPC, and RPC is what gives you that abstraction above the lower level network and transport protocols. iconic architecture, no cap 🔥
this document assumes the NFS environment is implemented on top of Sun RPC, which is specified in [RFC1057]. if you want the full tea, it's covered in [Corbin] 📚🌸
1.4. External Data Representation 💜
the eXternal Data Representation (XDR) specification gives everyone a standard way of representing a set of data types on a network. this fixes the chaos of different byte orders, structure alignment, and data type representation across machines that are talking to each other 🫶
in this document, the RPC Data Description Language is used to specify the XDR format parameters and results for each of the RPC service procedures that an NFS server provides. it's giving C declarations, with a few new constructs added on top ✨ so this notation:
string name[SIZE];
string data<DSIZE>;defines name, which is a fixed size block of SIZE bytes, and data, which is a variable sized block of up to DSIZE bytes. so square brackets mean fixed-length arrays, and angle brackets mean arrays with a variable number of elements up to a fixed maximum. and if a variable-length definition has no size specified, there's no maximum size for that field 🍓
now the discriminated union definition, which is lowkey my favourite part 😭
union example switch (enum status) {
case OK:
struct {
filename file1;
filename file2;
integer count;
}
case ERROR:
struct {
errstat error;
integer errno;
}
default:
void;
}this defines a structure where the first thing over the network is an enumeration type called status. if the value of status is OK, the next thing on the network is the structure containing file1, file2, and count. else, if the value of status is ERROR, the next thing on the network is a structure containing error and errno. and if the value of status is neither OK nor ERROR, then there's no more data in the structure 💀
the XDR typehyperis an 8 byte (64 bit) quantity, and it's used the same way as theintegertype. for example:
hyper foo;
unsigned hyper bar;foo is an 8 byte signed value, while bar is an 8 byte unsigned value 🌈
although RPC/XDR compilers exist to generate client and server stubs from RPC Data Description Language input, NFS implementations do not require their use. any software that provides equivalent encoding and decoding to the canonical network order of data defined by XDR can be used to interoperate with other NFS implementations. main character energy for any implementation that gets it right 👑
XDR is described in [RFC1014] 🎀
1.5. Authentication and Permission Checking 🔐
the RPC protocol includes a slot for authentication parameters on every call 🧚 what goes in that slot depends on the type of authentication the server and client use, and a server may support several different flavors of authentication at once. here's the lineup 💅
AUTH_NONEgives you null authentication, meaning no authentication information is passed.AUTH_UNIXgives you UNIX-style user ID, group ID, and groups with each call.AUTH_DESgives you DES-encrypted authentication parameters based on a network-wide name, with session keys exchanged via a public key scheme.AUTH_KERBgives you DES encrypted authentication parameters based on a network-wide name, with session keys exchanged via Kerberos secret keys.
the NFS server checks permissions by taking the credentials from the RPC authentication information in each remote request 🦋 for example, with AUTH_UNIX, the server grabs the user's effective user ID, effective group ID, and groups on each call, then uses them to check access. using user IDs and group IDs means the client and server either share the same ID list or do local user and group ID mapping 🫶 servers and clients must agree on the mapping from user to uid and from group to gid, for those sites that don't have a consistent user ID and group ID space. in practice, that mapping is usually done on the server, either through a static mapping scheme or a mapping the user sets up from a client at mount time ✨
AUTH_DES and AUTH_KERB are based on a network-wide name, and they give you way more security 🔥 AUTH_DES uses DES encryption and public keys, while AUTH_KERB uses DES encryption and Kerberos secret keys (and tickets). again, the server and client have to agree on the identity of a particular name on the network. but the name-to-identity mapping is more operating system independent than the uid and gid mapping in AUTH_UNIX 🌸 and because the authentication parameters are encrypted, a malicious user has to know another user's network password or private key to masquerade as that user 😤 the server also returns a verifier that's encrypted too, so masquerading as a server requires knowing a network password 🙈
the NULL procedure typically requires no authentication. 🌟
1.6 Philosophy
okay so this section is the vibe check for the whole NFS version 3 protocol 💖. It defines the over-the-wire protocol that a client uses to access a server 🎀, and it gives you a well-defined interface to a server's file resources ✨. A client or server implements the protocol and maps its local file system semantics and actions onto the ones the NFS version 3 protocol defines 🦋. Implementations MAY differ to varying degrees, depending on how much a given environment can support all the operations and semantics in the protocol 🌸. Implementations exist and help illustrate different parts of the protocol, but the protocol specification itself is the final word on how clients access server resources 👑.
Since the NFS version 3 protocol is designed to be operating-system independent, it doesn't necessarily match the semantics of any existing system 💅. Server implementations are expected to make a best effort at supporting it. If a server can't support a particular procedure, it MAY return the error NFS3ERR_NOTSUP, which means the operation is not supported 🙅♀️. For example, lots of operating systems don't do hard links. A server that can't support hard links SHOULD return NFS3ERR_NOTSUP in response to a LINK request 🔗. FSINFO describes the most commonly unsupported procedures in the properties bit map 🍓. Or, a server MAY not natively support an operation but can still emulate it in its NFS version 3 implementation to give extra functionality ✨.
Sometimes a server can handle most of the semantics but not all of them 🥺. Take the ctime field in the fattr structure, which gives the time a file's attributes were last modified. Many systems don't keep that info. Instead of refusing the GETATTR operation, a server could fake it by returning the last modified time in place of ctime. But servers MUST be careful when simulating attribute information, because it can have side effects on clients 🚨. For example, many clients use file modification times as the basis for their cache consistency scheme 💀.
NFS servers are dumb and NFS clients are smart 💁♀️ (lowkey iconic, honestly). The clients do the heavy lifting to turn the generalized file access that servers provide into something apps and users can actually use. In the LINK example, a UNIX client that gets an NFS3ERR_NOTSUP error from a server would do the recovery needed to either make the link request look like it succeeded to the app, or return a reasonable error. In general, recovery is the client's burden 🫶.
The NFS version 3 protocol assumes a stateless server implementation 🧚. Statelessness means the server doesn't need to keep state about any of its clients to work correctly. Stateless servers have a distinct advantage over stateful ones when it comes to crashes 💜. A client only needs to keep retrying a request until the server responds, and it doesn't even need to know the server crashed. See additional comments in Duplicate request cache on page 99 📖.
A server does need to hold nonvolatile state, which is the data stored in the file system 🗂️. The design assumptions about flushing modified data to stable storage cut down on the failure modes where data loss can happen. That's how NFS version 3 implementations can tolerate transient failures, including transient network failures 🌈. Generally, server implementations of the NFS version 3 protocol can't tolerate a non-transient failure of the stable storage itself. However, there are fault tolerant implementations that try to address those problems 🛡️.
That doesn't mean an NFS version 3 server can't keep noncritical state 🤓. Servers often keep state (a cache) about previous operations to boost performance. For example, a client READ request might trigger a read-ahead of the next block of the file into the server's data cache. The server expects a sequential read, so the next READ request gets satisfied from the cache instead of the disk. Read-ahead increases performance by overlapping server disk I/O with client requests 🏎️. The important point is that the read-ahead block isn't necessary for correct server behavior. If the server crashes and loses its memory cache of read buffers, recovery is simple on reboot, since clients just keep reading from the server disk ✨.
Most data-modifying operations in the NFS protocol are synchronous 🔒. When a data-modifying procedure returns to the client, the client can assume the operation completed and any modified data tied to the request is now on stable storage. For example, a synchronous client WRITE request may cause the server to update data blocks, file system information blocks, and file attribute information (usually called metadata 📦). Once the WRITE completes, the client can assume the write data is safe and throw it away. This is a huge part of the server's stateless nature 👑. If the server didn't flush dirty data to stable storage before replying, the client would have no way of knowing when it's safe to discard modified data 😱. The following data-modifying procedures are synchronous: WRITE (with stable flag set to FILE_SYNC), CREATE, MKDIR, SYMLINK, MKNOD, REMOVE, RMDIR, RENAME, LINK, and COMMIT.
The NFS version 3 protocol introduces safe asynchronous writes on the server, when the WRITE procedure is used together with the COMMIT procedure ✨. COMMIT gives the client a way to flush data from previous asynchronous WRITE requests on the server to stable storage, and to detect whether the data needs to be retransmitted. Check the procedure descriptions of WRITE on page 49 and COMMIT on page 92 📚.
The LOOKUP procedure is how the client traverses multicomponent file names (pathnames) 🛤️. Each LOOKUP call resolves one segment of a pathname. There are two reasons it's limited to a single segment: it's hard to standardize a common format for hierarchical file names, and the client and server may have different mappings of pathnames to file systems 🙃. That would mean either the client has to break the path name at file system attachment points, or the server has to know about the client's attachment points. In NFS version 3 implementations, the client is the one that builds the hierarchical file name space, using mounts to put it together 🏗️. Support utilities like the Automounter give you a way to manage a shared, consistent image of the file name space while still being driven by the client mount process 💫.
Clients can do caching in all sorts of ways 🗃️. The general practice with the NFS version 2 protocol was a time-based client-server cache consistency mechanism, and it's expected that NFS version 3 implementations will use something similar. The NFS version 3 protocol has some explicit support in the form of additional attribute information, which helps eliminate explicit attribute checks 🔍. However, caching is not required, and the protocol doesn't define any caching policy 🙅. Neither NFS version 2 nor NFS version 3 provides a way to maintain strict client-server consistency (and, by implication, consistency across client caches) 💔.
1.7 Changes from the NFS Version 2 Protocol ✨
okay bestie, so the ROOT and WRITECACHE procedures are gone 💀 (RIP). A MKNOD procedure was added so you can make special files, which means CREATE isn't overloaded anymore 🙈. Caching on the client? NFS version 3 doesn't define it or dictate it, BUT they added extra info and hints so clients that do cache can manage their caches way better 🧠✨. Procedures that change a file's or directory's attributes can now return the new attributes once they're done, so the client can skip a follow-up GETATTR when validating attribute caches (so efficient, I'm obsessed 😭). Operations that modify the directory where the target object lives also return the old and new attributes of that directory, so clients can do smarter cache invalidation 💅. Then there are the new procedures: ACCESS checks permissions on the server, FSSTAT returns dynamic info about a file system, FSINFO returns static info about the file system and server, READDIRPLUS returns file handles and attributes along with directory entries, and PATHCONF returns POSIX pathconf info about a file 🧚.
Here's the list of the important changes between NFS version 2 and NFS version 3 💖:
- File handle size 🎀: the file handle went from a fixed array of 32 bytes to a variable-length array of 64 bytes max. This handles some known needs for a slightly bigger handle. It's variable length now so systems that don't use all 64 bytes can save local storage and network bandwidth.
- Maximum data sizes 📦: the max size of a READ or WRITE transfer is now set by values in the FSINFO return structure, and FSINFO also returns preferred transfer sizes. The protocol doesn't place any artificial limits on the maximum transfer sizes. Filenames and pathnames are variable-length strings now, and the real length restrictions are decided by the client and server implementations. No artificial limit there either. The
NFS3ERR_NAMETOOLONGerror lets the server tell the client it got a pathname that was too long for it to handle. - Error return ❌: some error returns now carry data (like attributes). nfsstat3 defines the full set of errors a server can return, and no other values are allowed.
- File type 📁: the file type now includes
NF3CHRandNF3BLKfor special files. Attributes for these types have subfields for UNIX major and minor device numbers.NF3SOCKandNF3FIFOare now defined for sockets and fifos in the file system. - File attributes 🧾: the blocksize field is gone. The mode field no longer contains file type info. The size and fileid fields were widened to eight-byte unsigned integers from four-byte integers. Major and minor device info now lives in a distinct structure. The blocks field was renamed to used, and it holds the total number of bytes used by the file. It's also an eight-byte unsigned integer.
- Set file attributes 🎯: in NFS version 2, the settable attributes were a subset of the file attributes structure, and the client marked the attributes it didn't want to change by setting the field to -1, which overloaded some unsigned fields. Now each field uses a discriminated union to say whether or how to set it. The atime and mtime fields can be set to either the server's current time or a time supplied by the client.
- LOOKUP 🔍: the LOOKUP return structure now includes the attributes for the directory searched.
- ACCESS 🔐: an ACCESS procedure was added for an explicit over-the-wire permissions check. This fixes known problems with the superuser ID mapping feature in many server implementations, where unexpected permission denied errors could pop up while reading or writing a file because of root user mapping. It also drops the NFS version 2 assumption that file access was based solely on UNIX style mode bits.
- READ 📖: the reply includes a Boolean that is TRUE if end-of-file was hit during the READ, so the client can detect end-of-file correctly.
- WRITE ✍️: the beginoffset and totalcount fields were removed from the WRITE arguments. The reply now includes a count, so the server can write less than the requested amount of data if it needs to. An indicator was added to the arguments to tell the server what level of cache synchronization the client needs.
- CREATE 🧱: an exclusive flag and a create verifier were added for exclusive creation of regular files.
- MKNOD 🛠️: added to support creating special files. This avoids overloading fields of CREATE, which some NFS version 2 implementations did.
- READDIR 🗂️: the READDIR arguments now include a verifier so the server can validate the cookie. The cookie is a 64-bit unsigned integer now instead of the 4-byte array from NFS version 2. This should help reduce interoperability problems.
- READDIRPLUS ➕: added to return file handles and attributes in an extended directory list.
- FSINFO ℹ️: added to provide nonvolatile info about a file system. The reply includes the preferred and maximum read transfer size, the preferred and maximum write transfer size, and flags saying whether links or symbolic links are supported. It also returns the preferred transfer size for READDIR replies, server time granularity, and whether times can be set in a SETATTR request.
- FSSTAT 💾: added to provide volatile info about a file system, for use by utilities like the Unix system
dfcommand. The reply includes the total size and free space in the file system in bytes, the total number of files and the number of free file slots, and an estimate of the time between file system modifications (for use in cache consistency checking algorithms). - COMMIT 🔄: provides the synchronization mechanism used with asynchronous WRITE operations.
2. RPC Information 🧩
2.1 Authentication 🪪
The NFS service uses AUTH_NONE in the NULL procedure. AUTH_UNIX, AUTH_DES, or AUTH_KERB are used for all other procedures. Other authentication types may be supported in the future 🔮.
2.2 Constants 🔢
These are the RPC constants you need to call the NFS Version 3 service. They're given in decimal 💕.
PROGRAM 100003
VERSION 32.3 Transport address 🚚
The NFS protocol is normally supported over the TCP and UDP protocols. It uses port 2049, the same as the NFS version 2 protocol 🔌.
2.4 Sizes 📏
These are the sizes, given in decimal bytes, of the various XDR structures used in the NFS version 3 protocol:
NFS3_FHSIZE 64: the maximum size in bytes of the opaque file handle.NFS3_COOKIEVERFSIZE 8: the size in bytes of the opaque cookie verifier passed by READDIR and READDIRPLUS.NFS3_CREATEVERFSIZE 8: the size in bytes of the opaque verifier used for exclusive CREATE.NFS3_WRITEVERFSIZE 8: the size in bytes of the opaque verifier used for asynchronous WRITE.
2.5 Basic Data Types 🧱
These are the basic XDR definitions that get used in other structures. Get ready, it's a lot 🥹.
uint64
typedef unsigned hyper uint64;
int64
typedef hyper int64;
uint32
typedef unsigned long uint32;
int32
typedef long int32;
filename3
typedef string filename3<>;
nfspath3
typedef string nfspath3<>;
fileid3
typedef uint64 fileid3;
cookie3
typedef uint64 cookie3;
cookieverf3
typedef opaque cookieverf3[NFS3_COOKIEVERFSIZE];
createverf3
typedef opaque createverf3[NFS3_CREATEVERFSIZE];
writeverf3
typedef opaque writeverf3[NFS3_WRITEVERFSIZE];
uid3
typedef uint32 uid3;
gid3
typedef uint32 gid3;
size3
typedef uint64 size3;
offset3
typedef uint64 offset3;
mode3
typedef uint32 mode3;
count3
typedef uint32 count3;
nfsstat3
enum nfsstat3 {
NFS3_OK = 0,
NFS3ERR_PERM = 1,
NFS3ERR_NOENT = 2,
NFS3ERR_IO = 5,
NFS3ERR_NXIO = 6,
NFS3ERR_ACCES = 13,
NFS3ERR_EXIST = 17,
NFS3ERR_XDEV = 18,
NFS3ERR_NODEV = 19,
NFS3ERR_NOTDIR = 20,
NFS3ERR_ISDIR = 21,
NFS3ERR_INVAL = 22,
NFS3ERR_FBIG = 27,
NFS3ERR_NOSPC = 28,
NFS3ERR_ROFS = 30,
NFS3ERR_MLINK = 31,
NFS3ERR_NAMETOOLONG = 63,
NFS3ERR_NOTEMPTY = 66,
NFS3ERR_DQUOT = 69,
NFS3ERR_STALE = 70,
NFS3ERR_REMOTE = 71,
NFS3ERR_BADHANDLE = 10001,
NFS3ERR_NOT_SYNC = 10002,
NFS3ERR_BAD_COOKIE = 10003,
NFS3ERR_NOTSUPP = 10004,
NFS3ERR_TOOSMALL = 10005,
NFS3ERR_SERVERFAULT = 10006,
NFS3ERR_BADTYPE = 10007,
NFS3ERR_JUKEBOX = 10008
};The nfsstat3 type comes back with every procedure's results except for the NULL procedure 🔙. NFS3_OK means the call completed successfully. Any other value means some error happened on the call, and the error code tells you which one. Note that the precise numeric encoding must be followed 🔢. No other values may be returned by a server. Servers are expected to do a best effort mapping of error conditions to the set of error codes defined. This spec doesn't specify any error precedences either. Error precedences decide which error value gets returned when more than one error applies in a given situation, and the individual server implementation determines them. If the client needs specific error precedences, it should check for the specific errors itself 🫶.
2.6 Defined Error Numbers 🚨
Here's a description of each defined error:
NFS3_OK: indicates the call completed successfully ✅.NFS3ERR_PERM: not owner. The operation wasn't allowed because the caller is either not a privileged user (root) or not the owner of the target of the operation 🙅♀️.NFS3ERR_NOENT: okay so this is the classic 💀 No such file or directory. The file or directory name you sent just doesn't exist. 🙈NFS3ERR_IO: I/O error. Something went really wrong, like a hard error (a disk error, for example) while the server was handling your request. Not a vibe 😭NFS3ERR_NXIO: also an I/O error, but this time it's no such device or address. 🔌NFS3ERR_ACCES: Permission denied. The caller doesn't have the right permission for what they're trying to do. Lowkey don't mix this up withNFS3ERR_PERM, which is only for owner or privileged-user permission failures 🔐NFS3ERR_EXIST: File exists. The file you specified is already there, bestie 🙅♀️NFS3ERR_XDEV: you tried to do a cross-device hard link, which is not it 🚫NFS3ERR_NODEV: No such device. 🎈NFS3ERR_NOTDIR: Not a directory. You put a non-directory where a directory was needed in a directory operation 📁NFS3ERR_ISDIR: Is a directory. It's the opposite drama: you gave a directory to an operation that wanted a non-directory 📂NFS3ERR_INVAL: Invalid argument, or an unsupported argument for an operation. Two examples: trying a READLINK on something that isn't a symbolic link, or trying to SETATTR a time field on a server that doesn't support that. 🚫🕰️NFS3ERR_FBIG: File too large. The operation would have made the file grow past the server's limit 🐘NFS3ERR_NOSPC: No space left on device. The operation would've pushed the server's file system over its limit 📦NFS3ERR_ROFS: Read-only file system. You tried a modifying operation on a read-only file system. Hard pass 📖NFS3ERR_MLINK: Too many hard links. 🔗NFS3ERR_NAMETOOLONG: The filename in an operation was too long. Shorten it, periodt ✂️NFS3ERR_NOTEMPTY: you tried to remove a directory that was not empty. You can't delete a full closet, babe 👗NFS3ERR_DQUOT: Resource (quota) hard limit exceeded. The user's resource limit on the server has been hit. Main character energy but make it a quota problem 📉NFS3ERR_STALE: Invalid file handle. The file handle in the arguments was invalid, because the file it points to doesn't exist anymore or its access got revoked. Ghosted 👻NFS3ERR_REMOTE: Too many levels of remote in path. The file handle referred to a file on a non-local file system on the server 🌍NFS3ERR_BADHANDLE: Illegal NFS file handle. The file handle failed the server's internal consistency checks. Something's giving corrupted 🫠NFS3ERR_NOT_SYNC: Update synchronization mismatch was detected during a SETATTR operation 🔄NFS3ERR_BAD_COOKIE: the READDIR or READDIRPLUS cookie is stale. 🍪NFS3ERR_NOTSUPP: Operation is not supported. Not on this server, sorry bestie 🙅NFS3ERR_TOOSMALL: Buffer or request is too small. Make it bigger 📏NFS3ERR_SERVERFAULT: an error happened on the server that doesn't map to any of the legal NFS version 3 protocol error values. The client should translate it into an appropriate error. UNIX clients may choose to translate this one to EIO. 🛠️NFS3ERR_BADTYPE: you tried to create an object of a type the server doesn't support. 🧩NFS3ERR_JUKEBOX: okay this one's giving patience 🎵. The server took the request but couldn't finish it in a timely fashion. The client should wait and then try again with a new RPC transaction ID. For example, a server with hierarchical storage gets a request for a file that's been migrated. In that case the server should start the immigration process and reply with this error ⏳
ftype3
The ftype3 enum tells you what kind of file something is. NF3REG is a regular file, NF3DIR is a directory, NF3BLK is a block special device file, NF3CHR is a character special device file, NF3LNK is a symbolic link, NF3SOCK is a socket, and NF3FIFO is a named pipe. Note that the precise enum encoding must be followed. Not optional, bestie 🧚
enum ftype3 {
NF3REG = 1,
NF3DIR = 2,
NF3BLK = 3,
NF3CHR = 4,
NF3LNK = 5,
NF3SOCK = 6,
NF3FIFO = 7
};specdata3
struct specdata3 {
uint32 specdata1;
uint32 specdata2;
};The meaning of those two words depends on the file system object type. For a block special (NF3BLK) or character special (NF3CHR) file, specdata1 and specdata2 are the major and minor device numbers, respectively. (Obviously a UNIX-specific interpretation 🐧.) For all other file types, these two elements should either be set to 0 or the values should be agreed upon by the client and server. If the client and server don't agree on the values, the client should treat these fields as if they are set to 0. This data is returned as part of the fattr3 structure, so it's available from every reply that returns attributes. Since these fields are otherwise unused for non-device objects, out-of-band information can be passed from server to client. Again though, both sides have to agree on what the values mean 🤝
nfs_fh3
struct nfs_fh3 {
opaque data<NFS3_FHSIZE>;
};The nfs_fh3 is the variable-length opaque object the server returns on LOOKUP, CREATE, SYMLINK, MKNOD, LINK, or READDIRPLUS operations. The client uses it in later operations to reference the file. The file handle has everything the server needs to tell one file apart from another. To the client it's a total mystery box 📦. The client can store file handles for later and compare two handles from the same server byte-by-byte for equality, but it can't interpret what's inside. If two file handles from the same server are equal, they must refer to the same file. If they're not equal, no conclusions can be drawn. Servers should try to maintain a one-to-one correspondence between file handles and files, but this is not required. Clients should use file handle comparisons only to improve performance, not for correct behavior. 🏎️
Servers can revoke the access a file handle provides at any time 🚪. If the file handle in a call refers to an object that no longer exists on the server, or its access has been revoked, the server should return NFS3ERR_STALE.
nfstime3
struct nfstime3 {
uint32 seconds;
uint32 nseconds;
};nfstime3 gives the number of seconds and nanoseconds since midnight, January 1, 1970 Greenwich Mean Time. It's how time and date info gets passed around 🕰️. The times on files are all server times, except in a SETATTR operation where the client can explicitly set the file time. A server converts to and from local time when it processes time values, keeping as much accuracy as it can. If the precision of the timestamps stored for a file is lower than what NFS version 3 defines, you can lose precision. An adjunct time maintenance protocol is recommended to reduce client and server time skew ⏰.
fattr3
struct fattr3 {
ftype3 type;
mode3 mode;
uint32 nlink;
uid3 uid;
gid3 gid;
size3 size;
size3 used;
specdata3 rdev;
uint64 fsid;
fileid3 fileid;
nfstime3 atime;
nfstime3 mtime;
nfstime3 ctime;
};This structure defines the attributes of a file system object 🗂️. Most operations return it for an object. When an operation affects two objects (like a MKDIR that changes the target directory's attributes and defines new attributes for the new directory), attributes for both may come back. Sometimes attributes come back inside the wcc_data structure (defined below), and sometimes they come back on their own. The big glow-up from NFS version 2 is that a lot of fields got wider, and the major/minor device info now lives in its own structure instead of being packed into one word 💅
Every server should support this set of attributes, even if it has to simulate some of the fields. Here's what each field means:
typeis the type of the file.modeis the protection mode bits.nlinkis the number of hard links to the file, which is the number of different names for the same file.uidis the user ID of the file's owner.gidis the group ID of the file's group.sizeis the size of the file in bytes.usedis the number of bytes of disk space the file actually uses. It can be smaller thansizebecause the file may have holes, or bigger because of fragmentation 🧩rdevdescribes the device file if the type isNF3CHRorNF3BLK. See specdata3 on page 20.fsidis the file system identifier.fileidis a number that uniquely identifies the file within its file system. On UNIX this would be the inumber 🔢atimeis when the file data was last accessed.mtimeis when the file data was last modified.ctimeis when the file's attributes were last changed. Writing to the file changesctimeas well asmtime✏️
The mode bits are defined like this:
0x00800 Set user ID on execution.
0x00400 Set group ID on execution.
0x00200 Save swapped text (not defined in POSIX).
0x00100 Read permission for owner.
0x00080 Write permission for owner.
0x00040 Execute permission for owner on a file. Or lookup
(search) permission for owner in directory.
0x00020 Read permission for group.
0x00010 Write permission for group.
0x00008 Execute permission for group on a file. Or lookup
(search) permission for group in directory.
0x00004 Read permission for others.
0x00002 Write permission for others.
0x00001 Execute permission for others on a file. Or lookup
(search) permission for others in directory.post_op_attr
union post_op_attr switch (bool attributes_follow) {
case TRUE:
fattr3 attributes;
case FALSE:
void;
};This structure returns attributes in operations that aren't directly about manipulating attributes. One of the big principles of this NFS revision is to return the real result of the operation you asked for, not an error from some incidental side quest 🎯. The post_op_attr structure lets the server recover from errors it hits while getting attributes.
This makes returning attributes look optional. But server implementors are strongly encouraged to make a best effort to return attributes whenever they can, even when returning an error 💜
wcc_attr
struct wcc_attr {
size3 size;
nfstime3 mtime;
nfstime3 ctime;
};This is the subset of pre-operation attributes needed to support weak cache consistency semantics ✨. size is the file size in bytes before the operation. mtime is the last modification time of the object before the operation. ctime is the last time the object's attributes changed before the operation. See the discussion in wcc_attr on page 24 📖
The way clients use mtime to detect changes to file system objects on a server depends on the granularity of the server's time base ⏱️
pre_op_attr
union pre_op_attr switch (bool attributes_follow) { ...
Okay, I'm going to write it out in full so you can see the shape of the thing. Here's the union, exactly as the spec has it:
union pre_op_attr switch (bool attributes_follow) {
case TRUE:
wcc_attr attributes;
case FALSE:
void;
};wcc_data
struct wcc_data {
pre_op_attr before;
post_op_attr after;
};okay so bestie, here's the problem 😭 when a client does something that changes a file or directory on the server, it can't immediately tell from the post-operation attributes whether that operation was the only thing that touched the object since the client last got its attributes. that matters a LOT, because if something else changed the object in the meantime, the client has to invalidate any cached data for it (except the data it literally just wrote) 🫶
so enter wcc_data, aka weak cache consistency data 🦋 it's basically certain key fields from the object's attributes before the operation, bundled together with the attributes after it. this lets the client manage its cache way more accurately than NFS version 2 implementations could ✨ the word "weak" is doing real work here though, because this is not the strict server-client consistency you'd get from a true cache consistency protocol. don't oversell it, periodt 💅
to make the weak cache consistency model work, the server will need to be able to get the pre-operation attributes, perform the intended modify, and then get the post-operation attributes atomically 👑 if there's a window where the object could get modified between the operation and either of the get-attributes steps, the client can't tell whether it was the only entity to touch the object. some info just got lost, and that weakens the guarantees 😬
post_op_fh3
union post_op_fh3 switch (bool handle_follows) {
case TRUE:
nfs_fh3 handle;
case FALSE:
void;
};one of the big principles of this revision of NFS is to return the real value from the operation you asked for, and NOT an error from some incidental side operation 🙅♀️ that's why post_op_fh3 exists: it lets the server recover from errors it runs into while building a file handle, without wrecking the actual result 🧚
this is the structure used to return a file handle from the CREATE, MKDIR, SYMLINK, MKNOD, and READDIRPLUS requests. in each case the client could get the file handle by issuing a LOOKUP after a successful return from one of those operations. returning the handle directly is just an optimization, so the client doesn't have to immediately fire off a LOOKUP to get it 🍓
sattr3
enum time_how {
DONT_CHANGE = 0,
SET_TO_SERVER_TIME = 1,
SET_TO_CLIENT_TIME = 2
};
union set_mode3 switch (bool set_it) {
case TRUE:
mode3 mode;
default:
void;
};
union set_uid3 switch (bool set_it) {
case TRUE:
uid3 uid;
default:
void;
};
union set_gid3 switch (bool set_it) {
case TRUE:
gid3 gid;
default:
void;
};
union set_size3 switch (bool set_it) {
case TRUE:
size3 size;
default:
void;
};
union set_atime switch (time_how set_it) {
case SET_TO_CLIENT_TIME:
nfstime3 atime;
default:
void;
};
union set_mtime switch (time_how set_it) {
case SET_TO_CLIENT_TIME:
nfstime3 mtime;
default:
void;
};
struct sattr3 {
set_mode3 mode;
set_uid3 uid;
set_gid3 gid;
set_size3 size;
set_atime atime;
set_mtime mtime;
};the sattr3 structure holds the file attributes a client is allowed to set 💖 the fields match the similarly named ones in fattr3. in NFS version 3, the settable attributes are described by a bunch of discriminated unions, and each union tells you whether that attribute is getting updated and, if so, how ✨
there are two flavours of discriminated union in here. for mode, uid, gid, or size, the union is switched on a boolean called set_it. if it's TRUE, then a value of the right type gets encoded after it 🎀
for atime or mtime, the union is switched on an enum called set_it instead. if it's DONT_CHANGE, that attribute just stays the same 🙌 if it's SET_TO_SERVER_TIME, the server sets it to its own local time, and the client doesn't send any data for it. and if it's SET_TO_CLIENT_TIME, the attribute gets set to the time the client passed in an nfstime3 structure. (check FSINFO on page 86 for how time granularity is handled, it's a whole thing 🕰️)
diropargs3
struct diropargs3 {
nfs_fh3 dir;
filename3 name;
};diropargs3 is used in directory operations. the file handle dir identifies the directory where you want to manipulate or access the file called name. see also the extra notes in File name component handling on page 101 💅
3. Server Procedures ✨
the following sections define the RPC procedures that an NFS version 3 protocol server supplies 🧚 each procedure's RPC number sits at the top of its page, next to its name. the SYNOPSIS gives the procedure name, the names of its arguments, the names of its results, and then the XDR argument and result declarations. that info is written in RPC Data Description Language as defined in [RFC1014] 🦋 the DESCRIPTION tells you what the procedure is supposed to do and how its arguments and results get used. the ERRORS section lists errors returned for specific kinds of failure. but those lists are NOT the final word on every error a procedure could return. they're more of a guide to the common ones, so clients should be prepared to handle unexpected errors coming back from a server 😅 and the IMPLEMENTATION field explains how the procedure is expected to work and how clients should use it 💜
program NFS_PROGRAM {
version NFS_V3 {
void
NFSPROC3_NULL(void) = 0;
GETATTR3res
NFSPROC3_GETATTR(GETATTR3args) = 1;
SETATTR3res
NFSPROC3_SETATTR(SETATTR3args) = 2;
LOOKUP3res
NFSPROC3_LOOKUP(LOOKUP3args) = 3;
ACCESS3res
NFSPROC3_ACCESS(ACCESS3args) = 4;
READLINK3res
NFSPROC3_READLINK(READLINK3args) = 5;
READ3res
NFSPROC3_READ(READ3args) = 6;
WRITE3res
NFSPROC3_WRITE(WRITE3args) = 7;
CREATE3res
NFSPROC3_CREATE(CREATE3args) = 8;
MKDIR3res
NFSPROC3_MKDIR(MKDIR3args) = 9;
SYMLINK3res
NFSPROC3_SYMLINK(SYMLINK3args) = 10;
MKNOD3res
NFSPROC3_MKNOD(MKNOD3args) = 11;
REMOVE3res
NFSPROC3_REMOVE(REMOVE3args) = 12;
RMDIR3res
NFSPROC3_RMDIR(RMDIR3args) = 13;
RENAME3res
NFSPROC3_RENAME(RENAME3args) = 14;
LINK3res
NFSPROC3_LINK(LINK3args) = 15;
READDIR3res
NFSPROC3_READDIR(READDIR3args) = 16;
READDIRPLUS3res
NFSPROC3_READDIRPLUS(READDIRPLUS3args) = 17;
FSSTAT3res
NFSPROC3_FSSTAT(FSSTAT3args) = 18;
FSINFO3res
NFSPROC3_FSINFO(FSINFO3args) = 19;
PATHCONF3res
NFSPROC3_PATHCONF(PATHCONF3args) = 20;
COMMIT3res
NFSPROC3_COMMIT(COMMIT3args) = 21;
} = 3;
} = 100003;okay so if a client asks for a procedure number that's out of range (undefined), that gets you an RPC error 🙈 for more detail, check [RFC1057] 📖
3.1 General comments on attributes and consistency data on failure
for procedures that return either post_op_attr or wcc_data structures on failure, the discriminated union might contain the pre-operation attributes of the object or of its parent directory 🌸 which one you get depends on the error that happened, and it can also depend on the particular server implementation. server implementors are strongly encouraged to return as much attribute data as possible when something fails 🫶 but client implementors need to be aware that their implementation must correctly handle the variant where NO attributes or consistency data comes back at all. be ready for that one, bestie 😭
3.2 General comments on filenames
these comments apply to all NFS version 3 procedures where the client passes in one or more filenames in the arguments: LOOKUP, CREATE, MKDIR, SYMLINK, MKNOD, REMOVE, RMDIR, RENAME, and LINK 🍓
- the filename must not be null, nor may it be the null string. if the server gets one of these, it should return the error
NFS3ERR_ACCES. on some clients, the filename''(or a null string) is treated as an alias for the current directory. clients that need that behaviour should implement it themselves and NOT depend on the server to support that semantic 💅 - a filename with the value "." is treated as an alias for the current directory. clients that need this should implement it themselves and not lean on the server for it. that said, the server should be able to handle a "." filename correctly 🧚
- a filename with the value ".." is treated as an alias for the parent of the current directory, meaning the directory that contains the current one. if the server supports directories, it should be prepared to handle this semantic, even if those directories don't contain UNIX-style "." or ".." entries 🌈
- if the filename is longer than the maximum for the file system (see PATHCONF on page 90, specifically name_max), what happens depends on the value of the PATHCONF flag
no_trunc👑 ifno_truncis FALSE, the filename gets silently truncated to name_max bytes. ifno_truncis TRUE and the filename is longer than the server's file system maximum, the operation fails with the errorNFS3ERR_NAMETOOLONG😬 - in general there will be characters a server can't handle as part of a filename. that set of characters varies from server to server and implementation to implementation 🦋 usually the server is the one controlling the client's view of the file system. if the server gets a filename containing characters it can't handle, it should return the error
NFS3ERR_EACCES. client implementations should be prepared to deal with this side effect of heterogeneity 🌸
see also the comments in File name component handling on page 101 💖
3.3.0 Procedure 0: NULL - Do nothing 💖
SYNOPSIS
void NFSPROC3_NULL(void) = 0;DESCRIPTION
okay so NULL literally does nothing 🫶 It's there so you can check that a server is responding and time how long it takes, and that's it. no cap, it's giving minimalism ✨
IMPLEMENTATION
It is important that this procedure do no work at all so that it can be used to measure the overhead of processing a service request. By convention, the NULL procedure should never require any authentication. A server may choose to ignore this convention, in a more secure implementation, where responding to the NULL procedure call acknowledges the existence of a resource to an unauthenticated client. 🙈 the vibe is "stay empty, stay fast, and don't tell strangers the server exists unless you have to."
ERRORS
Since the NULL procedure takes no NFS version 3 protocol arguments and returns no NFS version 3 protocol response, it can not return an NFS version 3 protocol error. However, it is possible that some server implementations may return RPC errors based on security and authentication requirements. 💀 no NFS errors here, but RPC errors can still show up depending on security stuff.
3.3.1 Procedure 1: GETATTR - Get file attributes 🎀
SYNOPSIS
GETATTR3res NFSPROC3_GETATTR(GETATTR3args) = 1;
struct GETATTR3args {
nfs_fh3 object;
};
struct GETATTR3resok {
fattr3 obj_attributes;
};
union GETATTR3res switch (nfsstat3 status) {
case NFS3_OK:
GETATTR3resok resok;
default:
void;
};DESCRIPTION
okay so GETATTR is how you ask "hey, what are this file's attributes?" 👑 You give it the file handle that the server handed you back from LOOKUP, CREATE, MKDIR, SYMLINK, MKNOD, or READDIRPLUS (or from the MOUNT service, which is described somewhere else). The only argument is:
object: the file handle of the object whose attributes you want.
If it works, GETATTR3res.status is NFS3_OK and GETATTR3res.resok gives you:
obj_attributes: the attributes for the object.
If it doesn't work, GETATTR3res.status holds the error and you get no other results. 😭
IMPLEMENTATION
ngl, attributes are where operating systems really fight 💅 Servers should make a best attempt to support all of the attributes in the fattr3 structure so clients have a common ground to count on. Some mapping may be required to map local attributes to those in the fattr3 structure. 🔥
Today, most client NFS version 3 protocol implementations implement a time-bounded attribute caching scheme to reduce over-the-wire attribute checks. it's giving "don't re-ask every five seconds, the answer probably didn't change" 🌸
ERRORS
NFS3ERR_IO
NFS3ERR_STALE
NFS3ERR_BADHANDLE
NFS3ERR_SERVERFAULTSEE ALSO
ACCESS. 🦋
3.3.2 Procedure 2: SETATTR - Set file attributes 🍓
SYNOPSIS
SETATTR3res NFSPROC3_SETATTR(SETATTR3args) = 2;
union sattrguard3 switch (bool check) {
case TRUE:
nfstime3 obj_ctime;
case FALSE:
void;
};
struct SETATTR3args {
nfs_fh3 object;
sattr3 new_attributes;
sattrguard3 guard;
};
struct SETATTR3resok {
wcc_data obj_wcc;
};
struct SETATTR3resfail {
wcc_data obj_wcc;
};
union SETATTR3res switch (nfsstat3 status) {
case NFS3_OK:
SETATTR3resok resok;
default:
SETATTR3resfail resfail;
};DESCRIPTION
okay so SETATTR is the procedure where you change stuff about a file or directory on the server ✨ The new attributes come in a sattr3 structure, which is basically a bunch of booleans and enums saying what to set and what the new values are. The arguments in SETATTR3args are:
object: the file handle for the object.new_attributes: a sattr3 structure containing booleans and enumerations describing the attributes to be set and the new values for those attributes.guard: a sattrguard3 union:check: TRUE if the server is to verify that guard.obj_ctime matches the ctime for the object; FALSE otherwise.
A client can ask the server to check that the object is in the state it expects before doing the SETATTR. To do that, it sets guard.check to TRUE and passes a time value in guard.obj_ctime. If guard.check is TRUE, the server must compare the value of guard.obj_ctime to the current ctime of the object. If the values are different, the server must preserve the object attributes and must return a status of NFS3ERR_NOT_SYNC. If guard.check is FALSE, the server will not perform this check. 🙅♀️ basically "don't touch it if it changed under me."
If it works, SETATTR3res.status is NFS3_OK and SETATTR3res.resok contains:
obj_wcc: a wcc_data structure containing the old and new attributes for the object.
Otherwise, SETATTR3res.status contains the error on failure and SETATTR3res.resfail contains the following:
obj_wcc: a wcc_data structure containing the old and new attributes for the object.
IMPLEMENTATION
the guard.check thing lets the client avoid changing attributes based on stale info, but it does NOT give you exactly-once semantics 😭 If a reply gets lost and the server doesn't notice the retransmitted request, the procedure can fail with NFS3ERR_NOT_SYNC even though the attribute change already happened. The client can try to recover by getting fresh attributes from the server and sending a new SETATTR request using the new ctime. The client can optionally check the attributes to avoid that second SETATTR request if the new attributes show that the attributes have already been set as desired (though it may not have been the issuing client that set the attributes). it's a whole plot twist, honestly 🎭
The new_attributes.size field is used to request changes to the size of a file. A value of 0 causes the file to be truncated, a value less than the current size of the file causes data from new size to the end of the file to be discarded, and a size greater than the current size of the file causes logically zeroed data bytes to be added to the end of the file. Servers are free to implement this using holes or actual zero data bytes. Clients should not make any assumptions regarding a server's implementation of this feature, beyond that the bytes returned will be zeroed. Servers must support extending the file size via SETATTR. 🌈 so the server can do it however it wants, but the zeros have to be zeros.
SETATTR is not guaranteed atomic. A failed SETATTR may partially change a file's attributes. 💀 so it might be half-done, babe. be warned.
Changing the size of a file with SETATTR indirectly changes the mtime. A client must account for this as size changes can result in data deletion. ⚠️
If server and client times differ, programs that compare client time to file times can break. A time maintenance protocol should be used to limit client/server time skew. ⏰ clocks being out of sync is a whole mood.
In a heterogeneous environment, it is quite possible that the server will not be able to support the full range of SETATTR requests. The error, NFS3ERR_INVAL, may be returned if the server can not store a uid or gid in its own representation of uids or gids, respectively. If the server can only support 32 bit offsets and sizes, a SETATTR request to set the size of a file to larger than can be represented in 32 bits will be rejected with this same error. 🧚 mixed systems are chaotic, and sometimes the server just can't fit what you asked for.
ERRORS
NFS3ERR_PERM
NFS3ERR_IO
NFS3ERR_ACCES
NFS3ERR_INVAL
NFS3ERR_NOSPC
NFS3ERR_ROFS
NFS3ERR_DQUOT
NFS3ERR_NOT_SYNC
NFS3ERR_STALE
NFS3ERR_BADHANDLE
NFS3ERR_SERVERFAULTSEE ALSO
CREATE, MKDIR, SYMLINK, and MKNOD. 🫶
3.3.3 Procedure 3: LOOKUP - Lookup filename 🔍
SYNOPSIS
LOOKUP3res NFSPROC3_LOOKUP(LOOKUP3args) = 3;
struct LOOKUP3args {
diropargs3 what;
};
struct LOOKUP3resok {
nfs_fh3 object;
post_op_attr obj_attributes;
post_op_attr dir_attributes;
};
struct LOOKUP3resfail {
post_op_attr dir_attributes;
};
union LOOKUP3res switch (nfsstat3 status) {
case NFS3_OK:
LOOKUP3resok resok;
default:
LOOKUP3resfail resfail;
};DESCRIPTION
okay so LOOKUP is basically "search this folder for a name and give me back the file handle" 🦋 The arguments in LOOKUP3args are:
what: the object to look up:dir: the file handle for the directory to search.name: the filename to be searched for. Refer to General comments on filenames on page 30.
If it works, LOOKUP3res.status is NFS3_OK and LOOKUP3res.resok contains:
object: the file handle of the object corresponding to what.name.obj_attributes: the attributes of the object corresponding to what.name.dir_attributes: the post-operation attributes of the directory, what.dir.
If it doesn't work, LOOKUP3res.status contains the error on failure and LOOKUP3res.resfail contains:
dir_attributes: the post-operation attributes for the directory, what.dir.
IMPLEMENTATION
okay this one had me like 🤨 At first glance, if what.name is a mount point on the server, two different replies seem possible: the server could return the file handle for the underlying directory that is mounted on, or the file handle of the root of the mounted directory. This ambiguity is simply resolved. A server will not allow a LOOKUP operation to cross a mountpoint to the root of a different filesystem, even if the filesystem is exported. This does not prevent a client from accessing a hierarchy of filesystems exported by a server, but the client must mount each of the filesystems individually so that the mountpoint crossing takes place on the client. A given server implementation may refine these rules given capabilities or limitations particular to that implementation. Refer to [X/OpenNFS] for a discussion on exporting file systems. 🗂️
Two filenames are distinguished, as in the NFS version 2 protocol. The name, ".", is an alias for the current directory and the name, "..", is an alias for the parent directory; that is, the directory that includes the specified directory as a member. There is no facility for dealing with a multiparented directory and the NFS protocol assumes a hierarchical organization, organized as a single-rooted tree. 🌳 one root, no tangled family trees.
Note that this procedure does not follow symbolic links. The client is responsible for all parsing of filenames including filenames that are modified by symbolic links encountered during the lookup process. 🧠 the client does the heavy lifting on symlinks, not the server.
ERRORS
NFS3ERR_IO
NFS3ERR_NOENT
NFS3ERR_ACCES
NFS3ERR_NOTDIR
NFS3ERR_NAMETOOLONG
NFS3ERR_STALE
NFS3ERR_BADHANDLE
NFS3ERR_SERVERFAULTSEE ALSO
CREATE, MKDIR, SYMLINK, MKNOD, READDIRPLUS, and PATHCONF. 💖
3.3.4 Procedure 4: ACCESS - Check Access Permission 🔐✨
SYNOPSIS
ACCESS3res NFSPROC3_ACCESS(ACCESS3args) = 4;
const ACCESS3_READ = 0x0001;
const ACCESS3_LOOKUP = 0x0002;
const ACCESS3_MODIFY = 0x0004;
const ACCESS3_EXTEND = 0x0008;
const ACCESS3_DELETE = 0x0010;
const ACCESS3_EXECUTE = 0x0020;
struct ACCESS3args {
nfs_fh3 object;
uint32 access;
};
struct ACCESS3resok {
post_op_attr obj_attributes;
uint32 access;
};
struct ACCESS3resfail {
post_op_attr obj_attributes;
};
union ACCESS3res switch (nfsstat3 status) {
case NFS3_OK:
ACCESS3resok resok;
default:
ACCESS3resfail resfail;
};DESCRIPTION
Okay bestie, ACCESS is the procedure that figures out what a user is allowed to do to a file system object 💅 The user is identified by the credentials in the request. The client packs the permissions it wants checked into a bit mask 🎀, the server checks them, and then it sends back a bit mask with the permissions the client is allowed. A status of NFS3_OK comes along for the ride 🦋
⚠️ The results here are necessarily advisory, so don't get too attached! NFS3_OK plus the right bit set does not promise that access will still be allowed later, because the server can revoke access rights at any time. 🙈
On entry, the arguments in ACCESS3args are:
- object: The file handle for the file system object to which access is to be checked.
- access: A bit mask of access permissions to check.
These are the access permissions that may be requested:
ACCESS3_READ: Read data from file or read a directory.ACCESS3_LOOKUP: Look up a name in a directory (no meaning for non-directory objects).ACCESS3_MODIFY: Rewrite existing file data or modify existing directory entries.ACCESS3_EXTEND: Write new data or add directory entries.ACCESS3_DELETE: Delete an existing directory entry.ACCESS3_EXECUTE: Execute file (no meaning for a directory).
On successful return, ACCESS3res.status is NFS3_OK. The server should return a status of NFS3_OK if no errors occurred that prevented the server from making the required access checks. The results in ACCESS3res.resok are:
- obj_attributes: The post-operation attributes of object.
- access: A bit mask of access permissions indicating access rights for the authentication credentials provided with the request.
Otherwise, ACCESS3res.status contains the error on failure and ACCESS3res.resfail contains the following:
- obj_attributes: The attributes of object - if access to attributes is permitted.
IMPLEMENTATION
Real talk, the client shouldn't try to figure out its access by poking at the uid, gid, and mode fields in the file attributes 🙅♀️ In general, that's not enough, since the server may do uid or gid mapping or enforce extra access control restrictions. It's also possible that the NFS version 3 protocol server isn't in the same ID space as the NFS version 3 protocol client. In these cases (and maybe others), the client can not reliably run an access check with only the current file attributes.
In the NFS version 2 protocol, the only reliable way to find out whether an operation was allowed was to try it and see if it succeeded or failed 😭 With the ACCESS procedure in the NFS version 3 protocol, the client can ask the server whether one or more classes of operations are permitted. The ACCESS operation is there so clients can check before doing a series of operations ✨ This is super useful in operating systems (such as UNIX) where permission checking only happens when a file or directory is opened. This procedure is also invoked by the NFS client access procedure (called possibly through access(2)). The intent is to make opening a remote file behave more consistently with opening a local file, which is iconic behavior 👑
The information the server returns in response to an ACCESS call is not permanent. It was correct at the exact time the server did the checks, but not necessarily afterwards. The server can revoke access permission at any time 🥀
The NFS version 3 protocol client should use the effective credentials of the user to build the authentication information in the ACCESS request used to determine access rights. The effective user and group credentials are the ones used in subsequent read and write operations. See the comments in Permission issues on page 98 for more information on this topic 📖
Many implementations don't directly support the ACCESS3_DELETE permission. Operating systems like UNIX will ignore the ACCESS3_DELETE bit if it's set on an access request for a non-directory object. In these systems, delete permission on a file is determined by the access permissions on the directory the file lives in, instead of by the permissions of the file itself 📂 So the bit mask returned for such a request will have the ACCESS3_DELETE bit set to 0, which tells the client it does not have this permission 🚫
ERRORS
NFS3ERR_IO
NFS3ERR_STALE
NFS3ERR_BADHANDLE
NFS3ERR_SERVERFAULTSEE ALSO
GETATTR. 🧚
3.3.5 Procedure 5: READLINK - Read from symbolic link 🔗
SYNOPSIS
READLINK3res NFSPROC3_READLINK(READLINK3args) = 5;
struct READLINK3args {
nfs_fh3 symlink;
};
struct READLINK3resok {
post_op_attr symlink_attributes;
nfspath3 data;
};
struct READLINK3resfail {
post_op_attr symlink_attributes;
};
union READLINK3res switch (nfsstat3 status) {
case NFS3_OK:
READLINK3resok resok;
default:
READLINK3resfail resfail;
};DESCRIPTION
Procedure READLINK reads the data associated with a symbolic link 🍓 The data is an ASCII string that is opaque to the server. So whether it was created by NFS version 3 protocol software on a client or created locally on the server, the data in a symbolic link isn't interpreted when it's created. It's just stored 💾 On entry, the arguments in READLINK3args are:
- symlink: The file handle for a symbolic link (file system object of type NF3LNK).
On successful return, READLINK3res.status is NFS3_OK and READLINK3res.resok contains:
- data: The data associated with the symbolic link.
- symlink_attributes: The post-operation attributes for the symbolic link.
Otherwise, READLINK3res.status contains the error on failure and READLINK3res.resfail contains the following:
- symlink_attributes: The post-operation attributes for the symbolic link.
IMPLEMENTATION
A symbolic link is nominally a pointer to another file 👀 The data isn't necessarily interpreted by the server, it's just stored in the file. It's possible for a client implementation to store a path name that doesn't mean anything to the server's operating system inside a symbolic link 🤷♀️ A READLINK operation hands the data back to the client so the client can interpret it. If different implementations want to share access to symbolic links, then they must agree on how to interpret the data in the symbolic link 🫶
The READLINK operation is only allowed on objects of type NF3LNK. The server should return the error NFS3ERR_INVAL if the object is not of type NF3LNK. (Note: The X/Open XNFS Specification for the NFS version 2 protocol defined the error status in this case as NFSERR_NXIO. This is inconsistent with existing server practice.) 🙃
ERRORS
NFS3ERR_IO
NFS3ERR_INVAL
NFS3ERR_ACCES
NFS3ERR_STALE
NFS3ERR_BADHANDLE
NFS3ERR_NOTSUPP
NFS3ERR_SERVERFAULTSEE ALSO
READLINK, SYMLINK. 💅
3.3.6 Procedure 6: READ - Read From file ✨
SYNOPSIS
READ3res NFSPROC3_READ(READ3args) = 6;
struct READ3args {
nfs_fh3 file;
offset3 offset;
count3 count;
};
struct READ3resok {
post_op_attr file_attributes;
count3 count;
bool eof;
opaque data<>;
};
struct READ3resfail {
post_op_attr file_attributes;
};
union READ3res switch (nfsstat3 status) {
case NFS3_OK:
READ3resok resok;
default:
READ3resfail resfail;
};DESCRIPTION
Okay bestie, READ is the procedure that pulls data out of a file 📖🫶. Here's what you send in READ3args:
file: the file handle of the file you want to read from. This must identify a file system object of typeNF3REG. 🎀offset: where the read starts inside the file. An offset of0means you start right at the beginning. If offset is greater than or equal to the size of the file, you get back the statusNFS3_OKwithcountset to0andeofset toTRUE, subject to access permissions checking. 🌸count: how many bytes you want. If count is0, the READ succeeds and returns0bytes of data, subject to access permissions checking. count must be less than or equal to thertmaxfield in the FSINFO reply for the file system that contains file. If it's bigger, the server may return onlyrtmaxbytes, which gives you a short read. 💅
When it works, READ3res.status is NFS3_OK and READ3res.resok holds:
file_attributes: the attributes of the file once the read is done. ✨count: the number of bytes the read actually returned. 🦋eof: if the read ended at the end-of-file (formally, in a correctly formed READ request, if READ3args.offset plus READ3resok.count is equal to the size of the file), eof comes backTRUE. Otherwise it'sFALSE. A successful READ of an empty file will always return eof asTRUE. 🥹data: the counted data read from the file. 🍓
If it doesn't work out, READ3res.status holds the error, and READ3res.resfail contains:
file_attributes: the post-operation attributes of the file. 🧚
IMPLEMENTATION
Okay, the big glow-up here is that the nfsdata type used for READ and WRITE in NFS version 2 (the part that holds the data in a request or reply) is now a variable-length opaque byte array 💖. The maximum size is now only limited by what XDR and the underlying transports allow. NFS version 3 doesn't add any artificial limits of its own, so check the FSINFO procedure description for the details 👑.
Servers can return fewer than count bytes, and when that happens with eof set to FALSE, the client should send another READ to grab the rest 🔥. A server might give back less data for a few reasons. The file could have been truncated by another client, or by the server itself, so the size changed from what the requesting client thinks it is. That shrinks how much data is actually available. The server might also back off the transfer size and shorten the read reply, and server resource exhaustion can force a smaller read return too 🙈.
Some NFS version 2 client implementations decided a short read meant EOF. The eof flag added in NFS version 3 gives everyone a correct way to handle EOF, so no more guessing 🌈.
Some NFS version 2 server implementations wrongly returned NFSERR_ISDIR when the file system object wasn't a regular file. The correct return value for NFS version 3 is NFS3ERR_INVAL 💜.
ERRORS
NFS3ERR_IO
NFS3ERR_NXIO
NFS3ERR_ACCES
NFS3ERR_INVAL
NFS3ERR_STALE
NFS3ERR_BADHANDLE
NFS3ERR_SERVERFAULTSEE ALSO
READLINK. 🦋
3.3.7. Procedure 7: WRITE - Write to file ✨
SYNOPSIS (the shape of it) 💅
WRITE3res NFSPROC3_WRITE(WRITE3args) = 7;
enum stable_how {
UNSTABLE = 0,
DATA_SYNC = 1,
FILE_SYNC = 2
};
struct WRITE3args {
nfs_fh3 file;
offset3 offset;
count3 count;
stable_how stable;
opaque data<>;
};
struct WRITE3resok {
wcc_data file_wcc;
count3 count;
stable_how committed;
writeverf3 verf;
};
struct WRITE3resfail {
wcc_data file_wcc;
};
union WRITE3res switch (nfsstat3 status) {
case NFS3_OK:
WRITE3resok resok;
default:
WRITE3resfail resfail;
};DESCRIPTION (the main tea) 🫶
Okay bestie, Procedure WRITE is how you put data into a file 📝 On entry, the arguments in WRITE3args are:
file: The file handle for the file you want to write to. This MUST identify a file system object of type,NF3REG. 🗂️offset: Where in the file the write begins ✍️ An offset of0means start writing right at the very beginning of the file.count: How many bytes of data to write 🔢 Ifcountis0, the WRITE will succeed and return a count of0, barring errors due to permissions checking. The size ofdataMUST be less than or equal to the value of thewtmaxfield in the FSINFO reply structure for the file system that containsfile. If it's bigger, the server MAY write onlywtmaxbytes, which gives you a short write.stable: this one is the big drama 😭 IfstableisFILE_SYNC, the server MUST commit the data written plus all file system metadata to stable storage before returning results. This matches the NFS version 2 protocol semantics. Any other behavior is a protocol violation, no debate. IfstableisDATA_SYNC, then the server MUST commit all of the data to stable storage, plus enough of the metadata to retrieve the data, before returning. The server implementor is free to handleDATA_SYNCthe same way asFILE_SYNC, just with a possible performance drop. IfstableisUNSTABLE, the server is free to commit any part of the data and the metadata to stable storage, including all or none of it, before returning a reply to the client. There's no guarantee whether or when uncommitted data will later be committed to stable storage. The only guarantees the server makes are that it won't destroy any data without changing the value ofverf, and that it won't commit the data and metadata at a level lower than what the client asked for. Check out the discussion on COMMIT on page 92 for more on if and when data gets committed to stable storage. 🔐data: The data to be written to the file. 💖
On successful return, WRITE3res.status is NFS3_OK and WRITE3res.resok contains:
file_wcc: Weak cache consistency data for the file. If a client only needs the post-write file attributes, those are infile_wcc.after. ✨count: The number of bytes of data written to the file. The server MAY write fewer bytes than requested. If it does, the actual number of bytes written, starting at locationoffset, is what gets returned. 📦committed: The server SHOULD return an indication of how committed the data and metadata are, viacommitted. If the server committed all data and metadata to stable storage,committedSHOULD be set toFILE_SYNC. If the level of commitment was at least as strong asDATA_SYNC, thencommittedSHOULD be set toDATA_SYNC. Otherwise,committedMUST be returned asUNSTABLE. IfstablewasFILE_SYNC, thencommittedMUST also beFILE_SYNC, and anything else is a protocol violation. IfstablewasDATA_SYNC, thencommittedMAY beFILE_SYNCorDATA_SYNC, and anything else is a protocol violation. IfstablewasUNSTABLE, thencommittedMAY be eitherFILE_SYNC,DATA_SYNC, orUNSTABLE. 🧾verf: This is a cookie the client can use to figure out whether the server has changed state between a WRITE call and a later call to either WRITE or COMMIT. This cookie MUST be consistent during a single instance of the NFS version 3 protocol service, and MUST be unique between instances of the NFS version 3 protocol server, where uncommitted data MAY be lost. 🍪
Otherwise, WRITE3res.status holds the error on failure, and WRITE3res.resfail contains the following:
file_wcc: Weak cache consistency data for the file. If a client only needs the post-write file attributes, those are infile_wcc.after. Even though the write failed, fullwcc_datais returned so the client can tell whether the failed write changed anything in the file. 🕵️
If a client writes data to the server with the stable argument set to UNSTABLE, and the reply comes back with a committed response of DATA_SYNC or UNSTABLE, the client will follow up later with a COMMIT operation to sync the outstanding asynchronous data and metadata with the server's stable storage, barring client error. It's possible that, due to a client crash or some other error, a subsequent COMMIT never gets received by the server. 😬
IMPLEMENTATION (the behind-the-scenes) 🛠️
The nfsdata type used for the READ and WRITE operations in the NFS version 2 protocol, which defines the data portion of a request or reply, has been changed to a variable-length opaque byte array. The maximum size the protocol allows is now limited by what XDR and the underlying transports will allow. There are no artificial limits imposed by the NFS version 3 protocol. Consult the FSINFO procedure description for details. 📏
It's possible for the server to write fewer than count bytes of data. In that case, the server SHOULD NOT return an error unless no data was written at all. If the server writes less than count bytes, the client SHOULD issue another WRITE to write the remaining data. 🔁
It's assumed that the act of writing data to a file will cause the mtime of the file to be updated. However, the mtime of the file SHOULD NOT be changed unless the contents of the file are changed. So, a WRITE request with count set to 0 SHOULD NOT cause the mtime of the file to be updated. 🕰️
The NFS version 3 protocol introduces safe asynchronous writes. The combination of WRITE with stable set to UNSTABLE, followed by a COMMIT, addresses the performance bottleneck found in the NFS version 2 protocol, which needed to synchronously commit all writes to stable storage. 🚀
The definition of stable storage has historically been a point of contention 🥊 The following expected properties of stable storage MAY help in resolving design issues in the implementation. Stable storage is persistent storage that survives:
- Repeated power failures.
- Hardware failures (of any board, power supply, and so on.).
- Repeated software crashes, including reboot cycle.
This definition does not address failure of the stable storage module itself.
A cookie, verf, is defined to let a client detect different instances of an NFS version 3 protocol server over which cached, uncommitted data MAY be lost. In the most likely case, verf lets the client detect server reboots. This information is required so the client can safely determine whether the server could have lost cached data. If the server fails unexpectedly and the client has uncommitted data from previous WRITE requests (done with the stable argument set to UNSTABLE, and in which the result committed was returned as UNSTABLE as well), it MAY not have flushed cached data to stable storage. The burden of recovery is on the client, which will need to resend the data to the server. 🧠
A suggested verf cookie would be to use the time the server was booted, or the time the server was last started (if restarting the server without a reboot results in lost buffers). ⏰
The committed field in the results lets the client do more effective caching. If the server is committing all WRITE requests to stable storage, then it SHOULD return with committed set to FILE_SYNC, regardless of the value of the stable field in the arguments. A server that uses an NVRAM accelerator MAY choose to implement this policy. The client can use this to make its cache more effective by discarding cached data that has already been committed on the server. 🧹
Some implementations MAY return NFS3ERR_NOSPC instead of NFS3ERR_DQUOT when a user's quota is exceeded. 💾
Some NFS version 2 protocol server implementations incorrectly returned NFSERR_ISDIR if the file system object type was not a regular file. The correct return value for the NFS version 3 protocol is NFS3ERR_INVAL. 🙅♀️
ERRORS 🚨
NFS3ERR_IO
NFS3ERR_ACCES
NFS3ERR_FBIG
NFS3ERR_DQUOT
NFS3ERR_NOSPC
NFS3ERR_ROFS
NFS3ERR_INVAL
NFS3ERR_STALE
NFS3ERR_BADHANDLE
NFS3ERR_SERVERFAULTSEE ALSO 👀
COMMIT.
3.3.8 Procedure 8: CREATE - Create a file ✨
SYNOPSIS
CREATE3res NFSPROC3_CREATE(CREATE3args) = 8;
enum createmode3 {
UNCHECKED = 0,
GUARDED = 1,
EXCLUSIVE = 2
};
union createhow3 switch (createmode3 mode) {
case UNCHECKED:
case GUARDED:
sattr3 obj_attributes;
case EXCLUSIVE:
createverf3 verf;
};
struct CREATE3args {
diropargs3 where;
createhow3 how;
};
struct CREATE3resok {
post_op_fh3 obj;
post_op_attr obj_attributes;
wcc_data dir_wcc;
};
struct CREATE3resfail {
wcc_data dir_wcc;
};
union CREATE3res switch (nfsstat3 status) {
case NFS3_OK:
CREATE3resok resok;
default:
CREATE3resfail resfail;
};DESCRIPTION
Okay so, CREATE is the procedure that makes a regular file 📄🎀. The arguments in CREATE3args on entry are:
where: the location of the file to be created.dir: the file handle for the directory where the file is going to live.name: the name that gets attached to the new file. Refer to General comments on filenames on page 30.
When you're creating a regular file, there are three different ways to do it, and they're set up with how. how is a discriminated union that says how the server should handle the creation, plus the matching attributes:
mode: one ofUNCHECKED,GUARDED, orEXCLUSIVE. 💅UNCHECKEDmeans the file gets created without checking whether a duplicate already exists in the same directory. Herehow.obj_attributesis asattr3with the file's starting attributes.GUARDEDmeans the server checks for a duplicate before doing the create, and if one exists it fails the request withNFS3ERR_EXIST. If the file doesn't exist, the request goes ahead exactly likeUNCHECKED.EXCLUSIVEmeans the server follows exclusive creation semantics and uses the verifier to make sure the target is created exclusively. No attributes may be provided in this case, since the server may use the target file's metadata to store thecreateverf3verifier. 🔒
On a successful return, CREATE3res.status is NFS3_OK, and the results in CREATE3res.resok are:
obj: the file handle of the newly created regular file. 🦋obj_attributes: the attributes of the regular file that was just created.dir_wcc: weak cache consistency data for the directory,where.dir. A client that needs the post-CREATE directory attributes can find them indir_wcc.after.
Otherwise, CREATE3res.status holds the error when something goes wrong, and CREATE3res.resfail contains the following:
dir_wcc: weak cache consistency data for the directory,where.dir. A client that only needs the post-CREATE directory attributes can find them indir_wcc.after. Even though the CREATE failed, the fullwcc_datastill comes back, so the client can tell whether the failed CREATE changed the directory at all. 🙈
IMPLEMENTATION
Unlike the NFS version 2 protocol, where some fields in the initial attributes structure were overloaded to indicate creating devices and FIFOs as well as regular files, this procedure only supports regular files. The MKNOD procedure was introduced in the NFS version 3 protocol to handle devices and FIFOs, so implementations should have no reason to overload CREATE semantics in NFS version 3. 🧚
One part of the NFS version 3 CREATE procedure needs extra-careful thought: the mechanism added to support reliable exclusive creation of regular files. This kicks in when how.mode is EXCLUSIVE. In that case, how.verf contains a verifier that can reasonably be expected to be unique. A combination of a client identifier (perhaps the client network address) and a unique number generated by the client (perhaps the RPC transaction identifier) may be appropriate. ✨
If the file doesn't exist, the server creates it and stores the verifier in stable storage. For file systems that don't provide a way to store arbitrary file attributes, the server may use one or more elements of the file metadata to store the verifier. The verifier must be stored in stable storage so that a retransmitted request doesn't fail by mistake. An exclusive create is assumed to be happening because exclusive semantics are critical to the application. Because of that expected usage, exclusive CREATE does not rely only on the normally volatile duplicate request cache to store the verifier. The duplicate request cache lives in volatile storage, so it doesn't survive a crash, and it may actually get flushed during a long network partition, which would open up failure windows. 😭 In the UNIX local file system environment, the expected place to store the verifier on creation is the file's metadata (its time stamps). For this reason, an exclusive file create may not include initial attributes, because the server would have nowhere to put the verifier.
If the server can't support these exclusive create semantics (perhaps because it can't commit the verifier to stable storage), it should fail the CREATE request with the error NFS3ERR_NOTSUPP. 💀
During an exclusive CREATE request, if the file already exists, the server reconstructs the file's verifier and compares it with the verifier in the request. If they match, the server treats the request as a success. The request is presumed to be a duplicate of an earlier, successful request whose reply was lost, and the server's duplicate request cache mechanism didn't catch it. If the verifiers don't match, the request is rejected with the status NFS3ERR_EXIST. 🔥
Once the client has completed a successful exclusive create, it must issue a SETATTR to set the correct file attributes. Until it does, it should not rely on any of the file's attributes, since the server implementation may need to overload the file metadata to store the verifier. 👑
Using the GUARDED mode does not provide exactly-once semantics. In particular, if a reply is lost and the server doesn't detect the retransmission of the request, the procedure can fail with NFS3ERR_EXIST even though the create actually went through. 😬
Refer to General comments on filenames on page 30.
ERRORS
NFS3ERR_IO
NFS3ERR_ACCES
NFS3ERR_EXIST
NFS3ERR_NOTDIR
NFS3ERR_NOSPC
NFS3ERR_ROFS
NFS3ERR_NAMETOOLONG
NFS3ERR_DQUOT
NFS3ERR_STALE
NFS3ERR_BADHANDLE
NFS3ERR_NOTSUPP
NFS3ERR_SERVERFAULTSEE ALSO
MKDIR, SYMLINK, MKNOD, and PATHCONF. 🌸
3.3.9 Procedure 9: MKDIR - Create a directory 🍓
SYNOPSIS
MKDIR3res NFSPROC3_MKDIR(MKDIR3args) = 9;
struct MKDIR3args {
diropargs3 where;
sattr3 attributes;
};
struct MKDIR3resok {
post_op_fh3 obj;
post_op_attr obj_attributes;
wcc_data dir_wcc;
};
struct MKDIR3resfail {
wcc_data dir_wcc;
};
union MKDIR3res switch (nfsstat3 status) {
case NFS3_OK:
MKDIR3resok resok;
default:
MKDIR3resfail resfail;
};DESCRIPTION
MKDIR makes a brand-new subdirectory 📁💖. The arguments in MKDIR3args on entry are:
where: the location of the subdirectory to be created.dir: the file handle for the directory where the subdirectory is going to be created.name: the name that gets attached to the new subdirectory. Refer to General comments on filenames on page 30.
attributes: the initial attributes for the subdirectory.
On a successful return, MKDIR3res.status is NFS3_OK, and the results in MKDIR3res.resok are:
obj: the file handle for the newly created directory. 🦋obj_attributes: the attributes for the newly created subdirectory.dir_wcc: weak cache consistency data for the directory,where.dir. A client that only needs the post-MKDIR directory attributes can find them indir_wcc.after.
Otherwise, MKDIR3res.status holds the error on failure, and MKDIR3res.resfail contains the following:
dir_wcc: weak cache consistency data for the directory,where.dir. A client that only needs the post-MKDIR directory attributes can find them indir_wcc.after. Even though the MKDIR failed, the fullwcc_datastill comes back, so the client can figure out whether the failed MKDIR changed the directory at all. 🙈
IMPLEMENTATION
Many server implementations won't allow the filenames.or..to be used as targets in a MKDIR operation. In that case, the server should returnNFS3ERR_EXIST. Refer to General comments on filenames on page 30. 🔥
ERRORS
NFS3ERR_IO
NFS3ERR_ACCES
NFS3ERR_EXIST
NFS3ERR_NOTDIR
NFS3ERR_NOSPC
NFS3ERR_ROFS
NFS3ERR_NAMETOOLONG
NFS3ERR_DQUOT
NFS3ERR_STALE
NFS3ERR_BADHANDLE
NFS3ERR_NOTSUPP
NFS3ERR_SERVERFAULTSEE ALSO
CREATE, SYMLINK, MKNOD, and PATHCONF. 🧚
3.3.10 Procedure 10: SYMLINK - Create a symbolic link 🔗✨
SYNOPSIS 💅
SYMLINK3res NFSPROC3_SYMLINK(SYMLINK3args) = 10;
struct symlinkdata3 {
sattr3 symlink_attributes;
nfspath3 symlink_data;
};
struct SYMLINK3args {
diropargs3 where;
symlinkdata3 symlink;
};
struct SYMLINK3resok {
post_op_fh3 obj;
post_op_attr obj_attributes;
wcc_data dir_wcc;
};
struct SYMLINK3resfail {
wcc_data dir_wcc;
};
union SYMLINK3res switch (nfsstat3 status) {
case NFS3_OK:
SYMLINK3resok resok;
default:
SYMLINK3resfail resfail;
};DESCRIPTION 🎀
Okay so, procedure SYMLINK is how you make a brand new symbolic link 🦋 On the way in, the arguments in SYMLINK3args are:
- where: the location of the symbolic link you want to create
- dir: the file handle for the directory where the symbolic link is going to live 🏡
- name: the name that gets attached to the new symbolic link. Refer to General comments on filenames on page 30.
- symlink: the symbolic link itself
- symlink_attributes: the starting attributes for the symbolic link 💖
- symlink_data: the string holding the symbolic link data
If it works, SYMLINK3res.status comes back as NFS3_OK and SYMLINK3res.resok gives you:
- obj: the file handle for the brand new symbolic link ✨
- obj_attributes: the attributes for that new symbolic link
- dir_wcc: weak cache consistency data for the directory, where.dir. If you only care about the post-SYMLINK directory attributes, you can find them in dir_wcc.after. Lowkey the cutest shortcut 🫶
If it doesn't work out, SYMLINK3res.status holds the error, and SYMLINK3res.resfail gives you:
- dir_wcc: weak cache consistency data for the directory, where.dir. Same deal, if you just want the post-SYMLINK directory attributes, look in dir_wcc.after. Even though the SYMLINK failed, you still get the full wcc_data 🙈 so you can check whether the failed SYMLINK changed the directory anyway. Detective energy 🕵️♀️
IMPLEMENTATION 🌸
Refer to General comments on filenames on page 30.
For symbolic links, the actual file system node and its contents are expected to be created in a single atomic operation. That is, once the symbolic link is visible, there must not be a window where a READLINK would fail or return incorrect data. No window, no drama, no READLINK crying in the corner 😭🧚
ERRORS 💀
NFS3ERR_IO
NFS3ERR_ACCES
NFS3ERR_EXIST
NFS3ERR_NOTDIR
NFS3ERR_NOSPC
NFS3ERR_ROFS
NFS3ERR_NAMETOOLONG
NFS3ERR_DQUOT
NFS3ERR_STALE
NFS3ERR_BADHANDLE
NFS3ERR_NOTSUPP
NFS3ERR_SERVERFAULTSEE ALSO 🍓
READLINK, CREATE, MKDIR, MKNOD, FSINFO, and PATHCONF.
3.3.11 Procedure 11: MKNOD - Create a special device 🧚✨
SYNOPSIS 💜
MKNOD3res NFSPROC3_MKNOD(MKNOD3args) = 11;
struct devicedata3 {
sattr3 dev_attributes;
specdata3 spec;
};
union mknoddata3 switch (ftype3 type) {
case NF3CHR:
case NF3BLK:
devicedata3 device;
case NF3SOCK:
case NF3FIFO:
sattr3 pipe_attributes;
default:
void;
};
struct MKNOD3args {
diropargs3 where;
mknoddata3 what;
};
struct MKNOD3resok {
post_op_fh3 obj;
post_op_attr obj_attributes;
wcc_data dir_wcc;
};
struct MKNOD3resfail {
wcc_data dir_wcc;
};
union MKNOD3res switch (nfsstat3 status) {
case NFS3_OK:
MKNOD3resok resok;
default:
MKNOD3resfail resfail;
};DESCRIPTION 🌈
Procedure MKNOD makes a brand new special file of the type what.type. Special files can be device files or named pipes, so it's giving toolbox energy 🧰 On the way in, the arguments in MKNOD3args are:
- where: the location of the special file you want to create
- dir: the file handle for the directory where the special file is going to be created 🏠
- name: the name that gets attached to the created special file. Refer to General comments on filenames on page 30.
- what: a discriminated union that says which type of special file to make, plus the data and attributes that match that type
- type: the type of the object being created
When you're making a character special file (what.type is NF3CHR) or a block special file (what.type is NF3BLK), what includes:
- device: a structure devicedata3 with these components:
- dev_attributes: the starting attributes for the special file
- spec: the major number stored in device.spec.specdata1 and the minor number stored in device.spec.specdata2 🔢
When you're making a socket (what.type is NF3SOCK) or a FIFO (what.type is NF3FIFO), what includes:
- pipe_attributes: the starting attributes for the special file
If it works, MKNOD3res.status comes back as NFS3_OK and MKNOD3res.resok gives you:
- obj: the file handle for the newly created special file ✨
- obj_attributes: the attributes for that special file
- dir_wcc: weak cache consistency data for the directory, where.dir. If you only need the post-MKNOD directory attributes, check dir_wcc.after 💅
If it fails, MKNOD3res.status holds the error and MKNOD3res.resfail gives you:
- dir_wcc: weak cache consistency data for the directory, where.dir. If you only need the post-MKNOD directory attributes, check dir_wcc.after. Even though the MKNOD failed, you still get the full wcc_data 🕵️♀️ so you can tell whether the failing MKNOD changed the directory.
IMPLEMENTATION 🦋
Refer to General comments on filenames on page 30.
Without explicit support for special file type creation in the NFS version 2 protocol, fields in the CREATE arguments were overloaded to indicate creation of certain types of objects. That was kinda messy 🙃 This overloading is not necessary in the NFS version 3 protocol. Glow-up unlocked 🌟
If the server does not support any of the defined types, the error, NFS3ERR_NOTSUPP, should be returned. Otherwise, if the server does not support the target type or the target type is illegal, the error, NFS3ERR_BADTYPE, should be returned. Note that NF3REG, NF3DIR, and NF3LNK are illegal types for MKNOD. The procedures, CREATE, MKDIR, and SYMLINK should be used to create these file types, respectively, instead of MKNOD. So regular files, directories, and symlinks each get their own procedure, no mixing it up 🙅♀️
ERRORS 🎀
NFS3ERR_IO
NFS3ERR_ACCES
NFS3ERR_EXIST
NFS3ERR_NOTDIR
NFS3ERR_NOSPC
NFS3ERR_ROFS
NFS3ERR_NAMETOOLONG
NFS3ERR_DQUOT
NFS3ERR_STALE
NFS3ERR_BADHANDLE
NFS3ERR_NOTSUPP
NFS3ERR_SERVERFAULT
NFS3ERR_BADTYPESEE ALSO 🍓
CREATE, MKDIR, SYMLINK, and PATHCONF.
3.3.12 Procedure 12: REMOVE - Remove a File 🗑️💖
SYNOPSIS 🌸
REMOVE3res NFSPROC3_REMOVE(REMOVE3args) = 12;
struct REMOVE3args {
diropargs3 object;
};
struct REMOVE3resok {
wcc_data dir_wcc;
};
struct REMOVE3resfail {
wcc_data dir_wcc;
};
union REMOVE3res switch (nfsstat3 status) {
case NFS3_OK:
REMOVE3resok resok;
default:
REMOVE3resfail resfail;
};DESCRIPTION 🌷
Procedure REMOVE removes (deletes) an entry from a directory. If that entry was the last reference to the matching file system object, the object may be destroyed. Bye bye 👋 On the way in, the arguments in REMOVE3args are:
- object: a diropargs3 structure that identifies the entry to be removed
- dir: the file handle for the directory where the entry is to be removed from
- name: the name of the entry to be removed. Refer to General comments on filenames on page 30.
If it works, REMOVE3res.status comes back as NFS3_OK and REMOVE3res.resok gives you:
- dir_wcc: weak cache consistency data for the directory, object.dir. If you only want the post-REMOVE directory attributes, look in dir_wcc.after ✨
If it fails, REMOVE3res.status holds the error and REMOVE3res.resfail gives you:
- dir_wcc: weak cache consistency data for the directory, object.dir. If you only want the post-REMOVE directory attributes, look in dir_wcc.after. Even though the REMOVE failed, you still get the full wcc_data so you can check whether the failing REMOVE changed the directory 🔍
IMPLEMENTATION 🧚
In general, REMOVE is meant to remove non-directory file objects, and RMDIR is for removing directories. However, REMOVE can be used to remove directories too, subject to restrictions imposed by either the client or server interfaces. This had been a source of confusion in the NFS version 2 protocol 😩
The concept of last reference is server specific. However, if the nlink field in the previous attributes of the object had the value 1, the client should not rely on referring to the object via a file handle. Likewise, the client should not rely on the resources (disk space, directory entry, and so on.) formerly associated with the object becoming immediately available. Thus, if a client needs to be able to continue to access a file after using REMOVE to remove it, the client should take steps to make sure that the file will still be accessible. The usual mechanism used is to use RENAME to rename the file from its old name to a new hidden name. 🫣
Refer to General comments on filenames on page 30.
ERRORS 💀
NFS3ERR_NOENT
NFS3ERR_IO
NFS3ERR_ACCES
NFS3ERR_NOTDIR
NFS3ERR_NAMETOOLONG
NFS3ERR_ROFS
NFS3ERR_STALE
NFS3ERR_BADHANDLE
NFS3ERR_SERVERFAULTSEE ALSO 🍓
RMDIR and RENAME.
3.3.13 Procedure 13: RMDIR - Remove a Directory ✨
SYNOPSIS
RMDIR3res NFSPROC3_RMDIR(RMDIR3args) = 13;
struct RMDIR3args {
diropargs3 object;
};
struct RMDIR3resok {
wcc_data dir_wcc;
};
struct RMDIR3resfail {
wcc_data dir_wcc;
};
union RMDIR3res switch (nfsstat3 status) {
case NFS3_OK:
RMDIR3resok resok;
default:
RMDIR3resfail resfail;
};DESCRIPTION
Okay so RMDIR is the procedure that removes (deletes) a subdirectory from a directory 🍓. If the directory entry of the subdirectory is the last reference to the subdirectory, the subdirectory may be destroyed 💀. On entry, the arguments in RMDIR3args are:
- object: a diropargs3 structure identifying the directory entry to be removed:
- dir: the file handle for the directory the subdirectory is to be removed from.
- name: the name of the subdirectory to be removed. Refer to General comments on filenames on page 30.
On successful return, RMDIR3res.status is NFS3_OK and RMDIR3res.resok contains:
- dir_wcc: weak cache consistency data for the directory, object.dir. For a client that only needs the post-RMDIR directory attributes, these can be found in dir_wcc.after. 🫶
Otherwise, RMDIR3res.status contains the error on failure, and RMDIR3res.resfail contains the following:
- dir_wcc: weak cache consistency data for the directory, object.dir. For a client that only needs the post-RMDIR directory attributes, these can be found in dir_wcc.after. Note that even though the RMDIR failed, full wcc_data is returned so the client can tell whether the failing RMDIR changed the directory. 👀
IMPLEMENTATION
Note that on some servers, removal of a non-empty directory is disallowed. 🚫
On some servers, the filename, ".", is illegal. These servers will return the error, NFS3ERR_INVAL. On some servers, the filename, "..", is illegal. These servers will return the error, NFS3ERR_EXIST. This might seem inconsistent, but it lets these servers comply with their own specific interface definitions. Clients should be prepared to handle both cases 🧚.
The client should not rely on the resources (disk space, directory entry, and so on.) formerly associated with the directory becoming immediately available. ⏳
ERRORS
NFS3ERR_NOENT
NFS3ERR_IO
NFS3ERR_ACCES
NFS3ERR_INVAL
NFS3ERR_EXIST
NFS3ERR_NOTDIR
NFS3ERR_NAMETOOLONG
NFS3ERR_ROFS
NFS3ERR_NOTEMPTY
NFS3ERR_STALE
NFS3ERR_BADHANDLE
NFS3ERR_NOTSUPP
NFS3ERR_SERVERFAULTSEE ALSO
REMOVE.
3.3.14 Procedure 14: RENAME - Rename a File or Directory 💜
SYNOPSIS
RENAME3res NFSPROC3_RENAME(RENAME3args) = 14;
struct RENAME3args {
diropargs3 from;
diropargs3 to;
};
struct RENAME3resok {
wcc_data fromdir_wcc;
wcc_data todir_wcc;
};
struct RENAME3resfail {
wcc_data fromdir_wcc;
wcc_data todir_wcc;
};
union RENAME3res switch (nfsstat3 status) {
case NFS3_OK:
RENAME3resok resok;
default:
RENAME3resfail resfail;
};DESCRIPTION
Okay, RENAME is the glow-up procedure 🦋. It renames the file identified by from.name in the directory, from.dir, to to.name in the directory, to.dir. The operation is required to be atomic to the client. to.dir and from.dir must reside on the same file system and server. On entry, the arguments in RENAME3args are:
- from: a diropargs3 structure identifying the source (the file system object to be re-named):
- from.dir: the file handle for the directory from which the entry is to be renamed.
- from.name: the name of the entry that identifies the object to be renamed. Refer to General comments on filenames on page 30.
- to: a diropargs3 structure identifying the target (the new name of the object):
- to.dir: the file handle for the directory to which the object is to be renamed.
- to.name: the new name for the object. Refer to General comments on filenames on page 30.
If the directory, to.dir, already contains an entry with the name, to.name, the source object must be compatible with the target: either both are non-directories, or both are directories and the target must be empty. If they're compatible, the existing target is removed before the rename happens. If they're not compatible, or if the target is a directory but not empty, the server should return the error, NFS3ERR_EXIST. 🙅♀️
On successful return, RENAME3res.status is NFS3_OK and RENAME3res.resok contains:
- fromdir_wcc: weak cache consistency data for the directory, from.dir.
- todir_wcc: weak cache consistency data for the directory, to.dir.
Otherwise, RENAME3res.status contains the error on failure, and RENAME3res.resfail contains the following:
- fromdir_wcc: weak cache consistency data for the directory, from.dir.
- todir_wcc: weak cache consistency data for the directory, to.dir.
IMPLEMENTATION
The RENAME operation must be atomic to the client. The message "to.dir and from.dir must reside on the same file system on the server, [or the operation will fail]" means that the fsid fields in the attributes for the directories are the same. If they reside on different file systems, the error, NFS3ERR_XDEV, is returned. Even though the operation is atomic, the status, NFS3ERR_MLINK, may be returned if the server used an "unlink/link/unlink" sequence internally. 🧩
A file handle may or may not become stale on a rename. However, server implementors are strongly encouraged to attempt to keep file handles from becoming stale in this fashion. 🥹
On some servers, the filenames, "." and "..", are illegal as either from.name or to.name. In addition, neither from.name nor to.name can be an alias for from.dir. These servers will return the error, NFS3ERR_INVAL, in these cases.
If from and to both refer to the same file (they might be hard links of each other), then RENAME should perform no action and return NFS3_OK. ✅
Refer to General comments on filenames on page 30.
ERRORS
NFS3ERR_NOENT
NFS3ERR_IO
NFS3ERR_ACCES
NFS3ERR_EXIST
NFS3ERR_XDEV
NFS3ERR_NOTDIR
NFS3ERR_ISDIR
NFS3ERR_INVAL
NFS3ERR_NOSPC
NFS3ERR_ROFS
NFS3ERR_MLINK
NFS3ERR_NAMETOOLONG
NFS3ERR_NOTEMPTY
NFS3ERR_DQUOT
NFS3ERR_STALE
NFS3ERR_BADHANDLE
NFS3ERR_NOTSUPP
NFS3ERR_SERVERFAULTSEE ALSO
REMOVE and LINK.
3.3.15 Procedure 15: LINK - Create Link to an object 🎀
SYNOPSIS
LINK3res NFSPROC3_LINK(LINK3args) = 15;
struct LINK3args {
nfs_fh3 file;
diropargs3 link;
};
struct LINK3resok {
post_op_attr file_attributes;
wcc_data linkdir_wcc;
};
struct LINK3resfail {
post_op_attr file_attributes;
wcc_data linkdir_wcc;
};
union LINK3res switch (nfsstat3 status) {
case NFS3_OK:
LINK3resok resok;
default:
LINK3resfail resfail;
};DESCRIPTION
LINK is giving hard link energy 🔗. It creates a hard link from file to link.name, in the directory, link.dir. file and link.dir must reside on the same file system and server. On entry, the arguments in LINK3args are:
- file: the file handle for the existing file system object.
- link: the location of the link to be created:
- link.dir: the file handle for the directory in which the link is to be created.
- link.name: the name that is to be associated with the created link. Refer to General comments on filenames on page 17.
On successful return, LINK3res.status is NFS3_OK and LINK3res.resok contains:
- file_attributes: the post-operation attributes of the file system object identified by file.
- linkdir_wcc: weak cache consistency data for the directory, link.dir.
Otherwise, LINK3res.status contains the error on failure, and LINK3res.resfail contains the following:
- file_attributes: the post-operation attributes of the file system object identified by file.
- linkdir_wcc: weak cache consistency data for the directory, link.dir.
IMPLEMENTATION
Changes to any property of the hard-linked files are reflected in all of the linked files. When a hard link is made to a file, the attributes for the file should have a value for nlink that is one greater than the value before the LINK. 📈
The comments under RENAME regarding object and target residing on the same file system apply here as well. The comments regarding the target name apply as well. Refer to General comments on filenames on page 30.
ERRORS
NFS3ERR_IO
NFS3ERR_ACCES
NFS3ERR_EXIST
NFS3ERR_XDEV
NFS3ERR_NOTDIR
NFS3ERR_INVAL
NFS3ERR_NOSPC
NFS3ERR_ROFS
NFS3ERR_MLINK
NFS3ERR_NAMETOOLONG
NFS3ERR_DQUOT
NFS3ERR_STALE
NFS3ERR_BADHANDLE
NFS3ERR_NOTSUPP
NFS3ERR_SERVERFAULTSEE ALSO
SYMLINK, RENAME and FSINFO.
3.3.16 Procedure 16: READDIR - Read From Directory ✨
SYNOPSIS 🎀
READDIR3res NFSPROC3_READDIR(READDIR3args) = 16;
struct READDIR3args {
nfs_fh3 dir;
cookie3 cookie;
cookieverf3 cookieverf;
count3 count;
};
struct entry3 {
fileid3 fileid;
filename3 name;
cookie3 cookie;
entry3 *nextentry;
};
struct dirlist3 {
entry3 *entries;
bool eof;
};
struct READDIR3resok {
post_op_attr dir_attributes;
cookieverf3 cookieverf;
dirlist3 reply;
};
struct READDIR3resfail {
post_op_attr dir_attributes;
};
union READDIR3res switch (nfsstat3 status) {
case NFS3_OK:
READDIR3resok resok;
default:
READDIR3resfail resfail;
};DESCRIPTION 💖
Okay so READDIR is the procedure you use to read a directory, bestie 📂. Each call grabs a chunk of entries, in order, and hands back each entry's name and file identifier. It also gives you what you need to ask for the next chunk in a later READDIR, so you can keep going until the whole directory is read. Main character energy 👑. Here's what goes into READDIR3args:
dir: the file handle for the directory you want to read.cookie: should be set to0in the first request to read the directory. On later requests, it should be a cookie the server gave you.cookieverf: should be set to0in the first request. On later requests, it should be a cookieverf the server gave you. The cookieverf must match the one returned by the READDIR where you got that cookie, or it's not it 🙅♀️.count: the maximum size of the READDIR3resok structure, in bytes. The size must include all XDR overhead. The server is free to return less thancountbytes of data, so don't be shocked if it does 🫶.
When it works, READDIR3res.status is NFS3_OK and READDIR3res.resok holds:
dir_attributes: the attributes of the directory,dir.cookieverf: the cookie verifier.reply: the directory list, which has:entries: zero or more directory (entry3) entries.eof:TRUEif the last member ofreply.entriesis the last entry in the directory, or ifreply.entriesis empty and the cookie pointed at the end of the directory. If it'sFALSE, there may be more entries to read 🌸.
If something goes wrong, READDIR3res.status holds the error, and READDIR3res.resfail contains:
dir_attributes: the attributes of the directory,dir.
IMPLEMENTATION 🧚
Here's the tea ☕. In NFS version 2, every directory entry came with a cookie that marked a spot in the directory. Clients could pass that cookie back in a later READDIR to pick up from that exact spot. The problem was that servers had no easy way to check whether a cookie was still valid 😬. If the directory got changed between two READDIRs, say by being reordered or compressed, the second READDIR could skip entries or process them twice. And if the cookie was no longer usable, like one pointing into the middle of an entry, the server had to round it down to the previous entry's cookie or up to the next one's. Either way could give wrong results, and the client would have no clue anything was off. Iconic chaos 💀.
NFS version 3 fixed this with a cookie verifier 🔥. Every READDIR request includes both a cookie and a cookie verifier, and both are 0 on the first call. The response comes with a new cookie verifier, plus a cookie for each entry. For later READDIRs, the client must send both the cookie and the matching cookie verifier. If the server notices the cookie is no longer valid, it rejects the READDIR with the status NFS3ERR_BAD_COOKIE. Clients should be careful not to hang onto directory entry cookies across operations that change the directory's contents, like REMOVE and CREATE 🙈.
One way to build the cookie verifier is for the server to use the directory's modification time. That might be overly restrictive, though 🥺. A better approach is to record the time of the last directory modification that changed the directory's organization in a way that makes a cookie impossible to interpret reliably. Servers where directory cookies are always valid are free to use zero as the verifier every time ✨.
The server may return fewer than count bytes of XDR-encoded entries. The count the client sends in the request should be greater than or equal to FSINFO dtpref 💅.
Since UNIX clients give a special meaning to the fileid value zero, UNIX clients should be careful to map zero fileid values to some other value, and servers should try to avoid sending a zero fileid 🦋.
ERRORS 💀
NFS3ERR_IO
NFS3ERR_ACCES
NFS3ERR_NOTDIR
NFS3ERR_BAD_COOKIE
NFS3ERR_TOOSMALL
NFS3ERR_STALE
NFS3ERR_BADHANDLE
NFS3ERR_SERVERFAULTSEE ALSO 🌈
READDIRPLUS and FSINFO 🍓.
3.3.17 Procedure 17: READDIRPLUS - Extended read from directory 🦋✨
SYNOPSIS 💅
READDIRPLUS3res NFSPROC3_READDIRPLUS(READDIRPLUS3args) = 17;
struct READDIRPLUS3args {
nfs_fh3 dir;
cookie3 cookie;
cookieverf3 cookieverf;
count3 dircount;
count3 maxcount;
};
struct entryplus3 {
fileid3 fileid;
filename3 name;
cookie3 cookie;
post_op_attr name_attributes;
post_op_fh3 name_handle;
entryplus3 *nextentry;
};
struct dirlistplus3 {
entryplus3 *entries;
bool eof;
};
struct READDIRPLUS3resok {
post_op_attr dir_attributes;
cookieverf3 cookieverf;
dirlistplus3 reply;
};
struct READDIRPLUS3resfail {
post_op_attr dir_attributes;
};
union READDIRPLUS3res switch (nfsstat3 status) {
case NFS3_OK:
READDIRPLUS3resok resok;
default:
READDIRPLUS3resfail resfail;
};DESCRIPTION 🌸
okay so READDIRPLUS is basically READDIR's glow-up 💖 It grabs a bunch of entries from a directory and hands back the full details for each one, plus the info the client needs to ask for the next batch later. The only real difference is how much info comes back per entry. READDIR gives you just the filename and the fileid, but READDIRPLUS gives you the name, the fileid, the attributes (fileid included, bestie), and the file handle. Main character energy, honestly 👑
Here's what goes in READDIRPLUS3args on the way in:
dir: the file handle for the directory you want to read 📂cookie: set this to0on your very first request. After that, use whatever cookie the server gave you back 🍪cookieverf: also0on the first request. Later on, it should be the cookieverf the server returned. It has to match the one that came back from the READDIRPLUS call where you got the cookie, or it's a no-go 🙅♀️dircount: the max number of bytes of directory information you want back. This number should not count the attributes and file handle parts of the result ✨maxcount: the max size of the whole READDIRPLUS3resok structure, in bytes. This size has to include all the XDR overhead. The server is free to send back fewer thanmaxcountbytes of data, so don't be shocked if it does 🫶
When it works out, READDIRPLUS3res.status is NFS3_OK and READDIRPLUS3res.resok holds:
dir_attributes: the attributes of the directory,dir📋cookieverf: the cookie verifier 🔐reply: the directory list, which has:entries: zero or more directory (entryplus3) entries 📝eof: this isTRUEif the last member ofreply.entriesis the last entry in the directory, OR ifreply.entriesis empty and the cookie pointed to the end of the directory. If it'sFALSE, there might be more entries to read 🏁
If something goes wrong, READDIRPLUS3res.status holds the error, and READDIRPLUS3res.resfail only has:
dir_attributes: the attributes of the directory,dir💜
IMPLEMENTATION 🧚
Some things to keep in mind with this procedure: the client's caches get flushed a lot more, since new file handles come back with names that get stored in the caches 🌊 There's also more over-the-wire overhead, which you have to weigh against skipping the LOOKUPs you'd otherwise need later. It's thought that this procedure might speed up directory browsing where attributes are always needed, like on the Apple Macintosh operating system and for MS-DOS 🍎💾
The dircount and maxcount fields are an optimisation, and here's the tea ☕ Imagine a READDIRPLUS call on a UNIX implementation asking for 1048 bytes. The reply doesn't have many entries, because all the attributes and file handles eat up the space. So the client might try asking for 8192 bytes and just use the first 1048 bytes of directory info. But the server doesn't know you only need 1048 bytes of directory info (the amount READDIR would return). It sees the 8192 byte request and runs a VOP_READDIR for 8192 bytes. Then it goes through every one of those entries, grabbing attributes and file handles for each. When it encodes the result, it stops once it hits 8192 bytes of results, attributes and file handles included. So it did a bigger VOP_READDIR and way more attribute fetches than it needed 😭 The ratio of directory entry size to attributes plus file handle size is usually at least 8 to 1, so the server ends up doing way more work than necessary 💀
The fix is for the client to send two counts. The first is dircount, the number of bytes of directory information the client actually wants. The second is maxcount, the max number of bytes in the result, attributes and file handles included. That way the server only issues a VOP_READDIR for the amount the client really wants, not an inflated number 🎯 This should shrink the VOP_READDIR requests on the server, which means less work there, and cut down on how many VOP_LOOKUP, VOP_GETATTR, and other calls the server makes to build attributes and file handles. Iconic efficiency, no notes 🔥
ERRORS 🙈
NFS3ERR_IO
NFS3ERR_ACCES
NFS3ERR_NOTDIR
NFS3ERR_BAD_COOKIE
NFS3ERR_TOOSMALL
NFS3ERR_STALE
NFS3ERR_BADHANDLE
NFS3ERR_NOTSUPP
NFS3ERR_SERVERFAULTSEE ALSO 🔗
READDIR.
3.3.18 Procedure 18: FSSTAT - Get dynamic file system information 🌈
SYNOPSIS 💅
FSSTAT3res NFSPROC3_FSSTAT(FSSTAT3args) = 18;
struct FSSTAT3args {
nfs_fh3 fsroot;
};
struct FSSTAT3resok {
post_op_attr obj_attributes;
size3 tbytes;
size3 fbytes;
size3 abytes;
size3 tfiles;
size3 ffiles;
size3 afiles;
uint32 invarsec;
};
struct FSSTAT3resfail {
post_op_attr obj_attributes;
};
union FSSTAT3res switch (nfsstat3 status) {
case NFS3_OK:
FSSTAT3resok resok;
default:
FSSTAT3resfail resfail;
};DESCRIPTION 🎀
So FSSTAT is the procedure that tells you the volatile state of a file system, like how full it is right now 💖 On the way in, FSSTAT3args just has:
fsroot: a file handle identifying an object in the file system. This is usually the file handle for a file system's mount point, the one you originally got from the MOUNT service on the server 📍
When it succeeds, FSSTAT3res.status is NFS3_OK and FSSTAT3res.resok holds:
obj_attributes: the attributes of the file system object named infsroot📋tbytes: the total size of the file system, in bytes 📦fbytes: the amount of free space in the file system, in bytes 🫧abytes: the amount of free space, in bytes, that's available to the user in the authentication info of the RPC. (This is space the file system has reserved. It doesn't reflect any quota system the server might have, so don't mix them up 🙅♀️)tfiles: the total number of file slots in the file system. (On a UNIX server, this often matches the number of inodes configured 🗂️)ffiles: the number of free file slots in the file system 🆓afiles: the number of free file slots available to the user in the authentication info of the RPC. (Again, this is slots the file system has reserved, not a quota system the server runs 💅)invarsec: a measure of how volatile the file system is. It's the number of seconds the file system isn't expected to change. For a volatile, constantly updated file system this is0. For an immutable one, like a CD-ROM, it'd be the largest unsigned integer. For file systems that barely get modified, like one with local executables and online docs, you might use a value of a few hours or days ⏰ The client can use this as a hint when tuning its cache management. BUT, and this is important, this measure is assumed to be dynamic and can change at any time, so don't get too attached 🥺
If it fails, FSSTAT3res.status has the error and FSSTAT3res.resfail only has:
obj_attributes: the attributes of the file system object named infsroot💜
IMPLEMENTATION 🧚
Not every implementation can support the whole list of attributes, and that's okay! Servers are expected to make a best effort at supporting all of them 🫶
ERRORS 🙈
NFS3ERR_IO
NFS3ERR_STALE
NFS3ERR_BADHANDLE
NFS3ERR_SERVERFAULTSEE ALSO 🔗
FSINFO.
3.3.19 Procedure 19: FSINFO - Get static file system Information ✨
SYNOPSIS 🎀
FSINFO3res NFSPROC3_FSINFO(FSINFO3args) = 19;
const FSF3_LINK = 0x0001;
const FSF3_SYMLINK = 0x0002;
const FSF3_HOMOGENEOUS = 0x0008;
const FSF3_CANSETTIME = 0x0010;
struct FSINFOargs {
nfs_fh3 fsroot;
};
struct FSINFO3resok {
post_op_attr obj_attributes;
uint32 rtmax;
uint32 rtpref;
uint32 rtmult;
uint32 wtmax;
uint32 wtpref;
uint32 wtmult;
uint32 dtpref;
size3 maxfilesize;
nfstime3 time_delta;
uint32 properties;
};
struct FSINFO3resfail {
post_op_attr obj_attributes;
};
union FSINFO3res switch (nfsstat3 status) {
case NFS3_OK:
FSINFO3resok resok;
default:
FSINFO3resfail resfail;
};DESCRIPTION 💅
okay so FSINFO is the procedure that grabs the nonvolatile file system state info, plus general details about the NFS version 3 server implementation. it's the "what even is this place" call, bestie 🌸 On entry, the arguments in FSINFO3args are:
fsroot A file handle identifying a file object. Normal usage is to provide a file handle for a mount point for a file system, as originally obtained from the MOUNT service on the server.
On successful return, FSINFO3res.status is NFS3_OK and FSINFO3res.resok contains:
obj_attributes The attributes of the file system object specified in fsroot.
rtmax The maximum size in bytes of a READ request supported by the server. Any READ with a number greater than rtmax will result in a short read of rtmax bytes or less. 🦋
rtpref The preferred size of a READ request. This should be the same as rtmax unless there is a clear benefit in performance or efficiency.
rtmult The suggested multiple for the size of a READ request.
wtmax The maximum size of a WRITE request supported by the server. In general, the client is limited by wtmax since there is no guarantee that a server can handle a larger write. Any WRITE with a count greater than wtmax will result in a short write of at most wtmax bytes. 😭
wtpref The preferred size of a WRITE request. This should be the same as wtmax unless there is a clear benefit in performance or efficiency.
wtmult The suggested multiple for the size of a WRITE request.
dtpref The preferred size of a READDIR request.
maxfilesize The maximum size of a file on the file system.
time_delta The server time granularity. When setting a file time using SETATTR, the server guarantees only to preserve times to this accuracy. If this is {0, 1}, the server can support nanosecond times, {0, 1000000} denotes millisecond precision, and {1, 0} indicates that times are accurate only to the nearest second. ⏰
properties A bit mask of file system properties. The following values are defined:
FSF_LINK If this bit is 1 (TRUE), the file system supports hard links. 🔗
FSF_SYMLINK If this bit is 1 (TRUE), the file system supports symbolic links. 🌈
FSF_HOMOGENEOUS If this bit is 1 (TRUE), the information returned by PATHCONF is identical for every file and directory in the file system. If it is 0 (FALSE), the client should retrieve PATHCONF information for each file and directory as required.
FSF_CANSETTIME If this bit is 1 (TRUE), the server will set the times for a file via SETATTR if requested (to the accuracy indicated by time_delta). If it is 0 (FALSE), the server cannot set times as requested. 🙈
ngl the properties bit mask is where the real drama is 👑 each bit is its own little yes-or-no flag
Otherwise, FSINFO3res.status contains the error on failure and FSINFO3res.resfail contains the following:
attributes The attributes of the file system object specified in fsroot.
IMPLEMENTATION 🧚
Not all implementations can support the entire list of attributes. It is expected that a server will make a best effort at supporting all the attributes.
The file handle provided is expected to be the file handle of the file system root, as returned to the MOUNT operation. Since mounts may occur anywhere within an exported tree, the server should expect FSINFO requests specifying file handles within the exported file system. A server may export different types of file systems with different attributes returned to the FSINFO call. The client should retrieve FSINFO information for each mount completed. Though a server may return different FSINFO information for different files within a file system, there is no requirement that a client obtain FSINFO information for other than the file handle returned at mount.
The maxfilesize field determines whether a server's particular file system uses 32 bit sizes and offsets or 64 bit file sizes and offsets. This may affect a client's processing. 🍓
The preferred sizes for requests are nominally tied to an exported file system mounted by a client. A surmountable issue arises in that the transfer size for an NFS version 3 protocol request is not only dependent on characteristics of the file system but also on characteristics of the network interface, particularly the maximum transfer unit (MTU). A server implementation can advertise different transfer sizes (for the fields, rtmax, rtpref, wtmax, wtpref, and dtpref) depending on the interface on which the FSINFO request is received. This is an implementation issue. 💜
ERRORS 💀
NFS3ERR_STALE
NFS3ERR_BADHANDLE
NFS3ERR_SERVERFAULTSEE ALSO 🌸
READLINK, WRITE, READDIR, FSSTAT and PATHCONF.
3.3.20 Procedure 20: PATHCONF - Retrieve POSIX information 🦋
SYNOPSIS 🎀
PATHCONF3res NFSPROC3_PATHCONF(PATHCONF3args) = 20;
struct PATHCONF3args {
nfs_fh3 object;
};
struct PATHCONF3resok {
post_op_attr obj_attributes;
uint32 linkmax;
uint32 name_max;
bool no_trunc;
bool chown_restricted;
bool case_insensitive;
bool case_preserving;
};
struct PATHCONF3resfail {
post_op_attr obj_attributes;
};
union PATHCONF3res switch (nfsstat3 status) {
case NFS3_OK:
PATHCONF3resok resok;
default:
PATHCONF3resfail resfail;
};DESCRIPTION 💅
okay so PATHCONF is the procedure for pulling the pathconf info for a file or directory 🌈 If the FSF_HOMOGENEOUS bit is set in FSFINFO3resok.properties, the pathconf info will be the same for all files and directories in the exported file system where this file or directory lives. On entry, the arguments in PATHCONF3args are:
object The file handle for the file system object.
On successful return, PATHCONF3res.status is NFS3_OK and PATHCONF3res.resok contains:
obj_attributes The attributes of the object specified by object.
linkmax The maximum number of hard links to an object. 🔗
name_max The maximum length of a component of a filename.
no_trunc If TRUE, the server will reject any request that includes a name longer than name_max with the error, NFS3ERR_NAMETOOLONG. If FALSE, any length name over name_max bytes will be silently truncated to name_max bytes. 😭
chown_restricted If TRUE, the server will reject any request to change either the owner or the group associated with a file if the caller is not the privileged user. (Uid 0.) 👑
case_insensitive If TRUE, the server file system does not distinguish case when interpreting filenames. 🙈
case_preserving If TRUE, the server file system will preserve the case of a name during a CREATE, MKDIR, MKNOD, SYMLINK, RENAME, or LINK operation. 💖
Otherwise, PATHCONF3res.status contains the error on failure and PATHCONF3res.resfail contains the following:
obj_attributes The attributes of the object specified by object.
IMPLEMENTATION 🧚
In some implementations of the NFS version 2 protocol, pathconf information was obtained at mount time through the MOUNT protocol. The proper place to obtain it is, as here, in the NFS version 3 protocol itself. ✨
ERRORS 💀
NFS3ERR_STALE
NFS3ERR_BADHANDLE
NFS3ERR_SERVERFAULTSEE ALSO 🌸
LOOKUP, CREATE, MKDIR, SYMLINK, MKNOD, RENAME, LINK and FSINFO.
3.3.21 Procedure 21: COMMIT - Commit cached data on a server to stable storage
SYNOPSIS ✨
COMMIT3res NFSPROC3_COMMIT(COMMIT3args) = 21;
struct COMMIT3args {
nfs_fh3 file;
offset3 offset;
count3 count;
};
struct COMMIT3resok {
wcc_data file_wcc;
writeverf3 verf;
};
struct COMMIT3resfail {
wcc_data file_wcc;
};
union COMMIT3res switch (nfsstat3 status) {
case NFS3_OK:
COMMIT3resok resok;
default:
COMMIT3resfail resfail;
};DESCRIPTION 💅
Okay bestie, so COMMIT is the procedure that forces or flushes data to stable storage 🧚 that was previously written with a WRITE procedure call where the stable field was set to UNSTABLE. It's giving "make it permanent now, please" energy. On entry, the arguments in COMMIT3args are:
file: the file handle for the file that the data gets flushed (committed) to. This must identify a file system object of type,NF3REG. 📁offset: the position within the file where the flush is supposed to begin. An offset of0means the flush starts at the very beginning of the file. 🌸count: the number of bytes of data to flush. Ifcountis0, the flush goes fromoffsetall the way to the end of the file. 🦋
On a successful return, COMMIT3res.status is NFS3_OK and COMMIT3res.resok contains:
file_wcc: weak cache consistency data for the file. If a client only needs the post-operation file attributes, those can be found infile_wcc.after. ✨verf: a cookie the client can use to figure out whether the server rebooted between a call to WRITE and a later call to COMMIT. This cookie must be consistent during a single boot session and must be unique between instances of the NFS version 3 protocol server where uncommitted data may be lost. 🔥
Otherwise, COMMIT3res.status holds the error if something went wrong, and COMMIT3res.resfail contains the following:
file_wcc: weak cache consistency data for the file. If a client only needs the post-write file attributes, those can be found infile_wcc.after. Even though the COMMIT failed, the fullwcc_datastill comes back so the client can tell whether the file changed on the server between the WRITE and COMMIT calls. 👀
IMPLEMENTATION 🎀
Procedure COMMIT is similar in operation and semantics to the POSIX fsync(2) system call, which syncs a file's state with the disk. Like, it flushes the file's data and metadata to disk. COMMIT does the same thing for a client, flushing any unsynchronized data and metadata on the server to the server's disk for the specified file. Just like fsync(2), there may be some modified data or no modified data to synchronize at all. The data might already have been synchronized by the server's normal periodic buffer synchronization activity. COMMIT will always return NFS3_OK, unless there has been an unexpected error. 💖
COMMIT is different from fsync(2) in one way: the client is allowed to flush just a range of the file. That's most likely triggered by a buffer-reclamation scheme on the client, before the file has been completely written. 🫶
The server implementation of COMMIT is reasonably simple, honestly lowkey easy. If the server receives a full file COMMIT request, meaning one that starts at offset 0 with count 0, it should do the equivalent of fsync()'ing the file. Otherwise, it should arrange for the cached data in the range given by offset and count to be flushed to stable storage. In both cases, any metadata associated with the file must be flushed to stable storage before returning. It's not an error for there to be nothing to flush on the server. That just means the data and metadata that needed flushing were already flushed, or got lost during the last server failure. 🍓
The client implementation of COMMIT is a little more complex, ngl. There are two reasons a client would want to commit a client buffer to stable storage. The first is that the client wants to reuse a buffer. In that case, the offset and count of the buffer are sent to the server in the COMMIT request. The server then flushes any cached data based on that offset and count, and flushes any metadata associated with the file. Then it returns the status of the flush and the verf verifier. The other reason is a full file flush, such as the one that may happen at close. In that case, the client gathers all of the buffers for this file that contain uncommitted data, does the COMMIT operation with an offset of 0 and count of 0, and then frees all of those buffers. Any other dirty buffers are sent to the server in the normal fashion. 💅
This implementation will require some modifications to the buffer cache on the client. After a buffer is written with stable UNSTABLE, it must be considered dirty by the client system until it is either flushed via a COMMIT operation or written via a WRITE operation with stable set to FILE_SYNC or DATA_SYNC. This is done so the buffer doesn't get freed and reused before the data can be flushed to stable storage on the server. 🔒
When a response comes back from either a WRITE or a COMMIT operation that contains an unexpected verf, the client will need to retransmit all of the buffers containing uncommitted cached data to the server. How this is done is up to the implementor. If there's only one buffer of interest, it should probably be sent back over in a WRITE request with the appropriate stable flag. If there's more than one, it might be worthwhile to retransmit all of the buffers in WRITE requests with stable set to UNSTABLE, and then retransmit the COMMIT operation to flush all of the data on the server to stable storage. The timing of these retransmissions is left to the implementor. ⏰
The above description applies to page-cache-based systems as well as buffer-cache-based systems. In page-cache-based systems, the virtual memory system will need to be modified instead of the buffer cache. 🌈
See additional comments on WRITE on page 49. 📝
ERRORS 🙈
NFS3ERR_IO
NFS3ERR_STALE
NFS3ERR_BADHANDLE
NFS3ERR_SERVERFAULTSEE ALSO 🦋
WRITE.
4. Implementation issues
The NFS version 3 protocol was designed so different operating systems could share files. But since it was designed in a UNIX environment, a lot of operations have semantics similar to the operations of the UNIX file system. This section talks about some of the general implementation-specific details and semantic issues. Procedure descriptions have implementation comments specific to that procedure. 👑
A bunch of papers have been written about the issues people ran into when building an NFS version 2 protocol implementation. The best overview paper is still [Sandberg]. [Israel], [Macklem], and [Pawlowski] describe other implementations. [X/OpenNFS] gives a complete description of the NFS version 2 protocol and its supporting protocols, plus a discussion of implementation issues and procedure and error semantics. Lots of the issues you hit when building an NFS version 2 protocol implementation will show up again when you build an NFS version 3 protocol implementation. 🤓
4.1 Multiple version support
The RPC protocol provides explicit support for versioning of a service. Client and server implementations of the NFS version 3 protocol should support both versions, for full backwards compatibility, when possible. The default behavior of the RPC binding protocol is that the client and server bind using the highest version number they both support. Client or server implementations that can't easily support both versions (for example, because of memory restrictions) will have to choose which version to support. The NFS version 2 protocol would be a safe choice, since fully capable clients and servers should support both versions. However, that choice would need to be made keeping all requirements in mind. 🧠
4.2 Server/client relationship
The NFS version 3 protocol is designed to let servers be as simple and general as possible. Sometimes the simplicity of the server can be a problem, if the client implements complicated file system semantics. 🌸
For example, some operating systems allow removal of open files. A process can open a file and, while it's open, remove it from the directory. The file can be read and written as long as the process keeps it open, even though the file has no name in the file system. It's impossible for a stateless server to implement these semantics. The client can do some tricks, like renaming the file on remove (to a hidden name) and only physically deleting it on close. The NFS version 3 protocol provides enough functionality to implement most file system semantics on a client. 🎀
Every NFS version 3 protocol client can also potentially be a server, and remote and local mounted file systems can be freely mixed. This causes some problems when a client travels down the directory tree of a remote file system and reaches the mount point on the server for another remote file system. Letting the server follow the second remote mount would require loop detection, server lookup, and user revalidation. So instead, both NFS version 2 protocol and NFS version 3 protocol implementations don't typically let clients cross a server's mount point. When a client does a LOOKUP on a directory where the server has mounted a file system, the client sees the underlying directory instead of the mounted directory. 🚫
For example, if a server has a file system called /usr and mounts another file system on /usr/src, and a client mounts /usr, it does not see the mounted version of /usr/src. A client could do remote mounts that match the server's mount points to keep the server's view. In this example, the client would also have to mount /usr/src in addition to /usr, even if they're from the same server. 🦋
4.3 Path name interpretation
There are a few complications to the rule that path names are always parsed on the client. For example, symbolic links could have different interpretations on different clients. There is no answer to this problem in this specification. 🤷♀️
Another common problem for non-UNIX implementations is the special interpretation of the pathname, .., to mean the parent of a given directory. A future revision of the protocol may use an explicit flag to indicate the parent instead. However, it's not a huge problem, since plenty of working non-UNIX implementations already exist. ✨
4.4 Permission issues 🔐
okay bestie, so strictly speaking, the NFS version 3 protocol does not define the permission checking that servers use 🙈 but it is expected that a server will do normal operating system permission checking, using AUTH_UNIX style authentication as the basis of its protection mechanism, or another stronger form of authentication such as AUTH_DES or AUTH_KERB ✨. With AUTH_UNIX, the server gets the client's effective uid, effective gid, and groups on each call and uses them to check permission. These are the so-called UNIX credentials 💅. AUTH_DES and AUTH_KERB use a network name, or netname, as the basis for identification, and a UNIX server derives the standard UNIX credentials from that 🦋. There are problems with this method, and they have been solved (iconic, honestly).
Using uid and gid implies that the client and server share the same uid list. Every server and client pair must have the same mapping from user to uid and from group to gid 💜. Since every client can also be a server, this tends to imply that the whole network shares the same uid/gid space 😭. If this is not the case, then it usually falls upon the server to perform some custom mapping of credentials from one authentication domain into another. A discussion of techniques for managing a shared user space, or for providing mechanisms for user ID mapping, is beyond the scope of this specification 🌸.
Another problem comes from the usually stateful open operation. Most operating systems check permission at open time, and then check that the file is open on each read and write request. With stateless servers, the server cannot detect that the file is open, so it must do permission checking on each read and write call 💀. UNIX client semantics of access permission checking on open can be provided with the ACCESS procedure call in this revision, which lets a client explicitly check access permissions without resorting to trying the operation ✨. On a local file system, a user can open a file, change the permissions so that no one is allowed to touch it, and still write to the file because it is open 🤯. On a remote file system, by contrast, the write would fail. To get around this problem, the server's permission checking algorithm should allow the owner of a file to access it regardless of the permission setting. This is needed in a practical NFS version 3 protocol server implementation, but it does depart from correct local file system semantics 🫶. This should not affect the return result of access permissions as returned by the ACCESS procedure, however.
A similar problem has to do with paging in an executable program over the network 🧚. The operating system usually checks for execute permission before opening a file for demand paging, and then reads blocks from the open file. In a local UNIX file system, an executable file does not need read permission to execute (pagein). An NFS version 3 protocol server cannot tell the difference between a normal file read, where the read permission bit is meaningful, and a demand pagein read, where the server should allow access to the executable file if the execute bit is set for that user, group, or public 🙈. To make this work, the server allows reading of files if the uid given in the call has either execute or read permission on the file, through ownership, group membership, or public access. Again, this departs from correct local file system semantics 🌈.
In most operating systems, a particular user (on UNIX, uid 0) has access to all files, no matter what permission and ownership they have 👑. This superuser permission may not be allowed on the server, since anyone who can become superuser on their client could gain access to all remote files 😱. By default, a UNIX server maps uid 0 to a distinguished value (UID_NOBODY), and it also maps the groups list, before doing its access checking. A server implementation may provide a mechanism to change this mapping. This works except for NFS version 3 protocol root file systems (required for diskless NFS version 3 protocol client support), where superuser access cannot be avoided 🥲. Export options are used, on the server, to restrict the set of clients allowed superuser access 🔒.
4.5 Duplicate request cache 🔁
The typical NFS version 3 protocol failure recovery model uses client time-out and retry to handle server crashes, network partitions, and lost server replies 🍓. A retried request is called a duplicate of the original.
When used in a file server context, the term idempotent can be used to distinguish between operation types ✨. An idempotent request is one that a server can perform more than once with equivalent results (though it may in fact change, as a side effect, the access time on a file, say for READ). Some NFS operations are obviously non-idempotent 💅. They cannot be reprocessed without special attention, simply because they may fail if tried a second time. The CREATE request, for example, can be used to create a file for which the owner does not have write permission. A duplicate of this request cannot succeed if the original succeeded. Likewise, a file can be removed only once 😭.
The side effects caused by performing a duplicate non-idempotent request can be destructive (for example, a truncate operation causing lost writes) 💀. The combination of a stateless design with the common choice of an unreliable network transport (UDP) implies the possibility of destructive replays of non-idempotent requests. To be more accurate, it is the inherent stateless design of the NFS version 3 protocol on top of an unreliable RPC mechanism that yields the possibility of destructive replays of non-idempotent requests 🧐. Even in an implementation of the NFS version 3 protocol over a reliable connection-oriented transport, a connection break with automatic reestablishment requires duplicate request processing, because the client will retransmit the request, and the server needs to deal with a potential duplicate non-idempotent request 🔄.
Most NFS version 3 protocol server implementations use a cache of recent requests (called the duplicate request cache) for the processing of duplicate non-idempotent requests 🧠. The duplicate request cache provides a short-term memory mechanism in which the original completion status of a request is remembered, and the operation is attempted only once. If a duplicate copy of this request is received, then the original completion status is returned 🎀.
The duplicate-request cache mechanism has been useful in reducing destructive side effects caused by duplicate NFS version 3 protocol requests 👏. This mechanism, however, does not guarantee against these destructive side effects in all failure modes ⚠️. Most servers store the duplicate request cache in RAM, so the contents are lost if the server crashes 😩. The exception to this may possibly occur in a redundant server approach to high availability, where the file system itself may be used to share the duplicate request cache state 🦋. Even if the cache survives server reboots (or failovers in the high availability case), its effectiveness is a function of its size 📏. A network partition can cause a cache entry to be reused before a client receives a reply for the corresponding request. If this happens, the duplicate request will be processed as a new one, possibly with destructive side effects 💔.
A good description of the implementation and use of a duplicate request cache can be found in [Juszczak] 📚.
4.6 File name component handling 🏷️
Server implementations of the NFS version 3 protocol will frequently impose restrictions on the names that can be created 🙅. Many servers will also forbid the use of names that contain certain characters, such as the path component separator used by the server operating system. For example, the UFS file system will reject a name which contains "/", while "." and ".." are distinguished in UFS, and may not be specified as the name when creating a file system object 🚫. The exact error status values returned for these errors are specified in the description of each procedure argument. The values (which conform to NFS version 2 protocol server practice) are not necessarily obvious, nor are they consistent from one procedure to the next 😵💫.
4.7 Synchronous modifying operations ⏱️
Data-modifying operations in the NFS version 3 protocol are synchronous 💖. When a procedure returns to the client, the client can assume that the operation has completed and that any data associated with the request is now on stable storage 💾.
4.8 Stable storage 🧊
NFS version 3 protocol servers must be able to recover without data loss from multiple power failures (including cascading power failures, that is, several power failures in quick succession), operating system failures, and hardware failure of components other than the storage medium itself (for example, disk, nonvolatile RAM) ⚡.
Some examples of stable storage that are allowable for an NFS server include:
- Media commit of data, that is, the modified data has been successfully written to the disk media, for example, the disk platter 💿.
- An immediate reply disk drive with battery-backed on-drive intermediate storage or uninterruptible power system (UPS) 🔋.
- Server commit of data with battery-backed intermediate storage and recovery software 🛠️.
- Cache commit with uninterruptible power system (UPS) and recovery software 🌟.
Conversely, the following are not examples of stable storage:
- An immediate reply disk drive without battery-backed on-drive intermediate storage or uninterruptible power system (UPS) 🙅♀️.
- Cache commit without both uninterruptible power system (UPS) and recovery software 🚩.
The only exception to this (introduced in this protocol revision) is as described under the WRITE procedure on the handling of the stable bit, and the use of the COMMIT procedure 👀. It is the use of the synchronous COMMIT procedure that provides the necessary semantic support in the NFS version 3 protocol 🎯.
4.9 Lookups and name resolution 🔍
A common objection to the NFS version 3 protocol is the philosophy of component-by-component LOOKUP by the client when resolving a name 🙄. The objection is that this is inefficient, as latencies for component-by-component LOOKUP would be unbearable 🐌.
Implementation practice solves this issue 🙌. A name cache, providing component to file-handle mapping, is kept on the client to short circuit actual LOOKUP invocations over the wire ⚡. The cache is subject to cache timeout parameters that bound attributes ⏳.
4.10 Adaptive retransmission 📡
Most client implementations use either an exponential back-off strategy to some maximum retransmission value, or a more adaptive strategy that attempts congestion avoidance 🧘. Congestion avoidance schemes in NFS request retransmission are modelled on the work presented in [Jacobson]. [Nowicki] and [Macklem] describe congestion avoidance schemes to be applied to the NFS protocol over UDP 🌊.
4.11 Caching policies 🗂️
The NFS version 3 protocol does not define a policy for caching on the client or server 🤷♀️. In particular, there is no support for strict cache consistency between a client and server, nor between different clients. See [Kazar] for a discussion of the issues of cache synchronization and mechanisms in several distributed file systems 🦄.
4.12 Stable versus unstable writes
okay so the stable field in the WRITE args (aka whether you're doing an async WRITE or not) is lowkey straightforward on a UNIX client 💅 If the NFS version 3 protocol client gets a write request that is not marked as asynchronous, it should generate the RPC with stable set to TRUE. If the request is marked as asynchronous, the RPC should be generated with stable set to FALSE. When the response comes back with committed set to TRUE, the client should just mark the write request as done and that's it, no further action needed ✨ If committed is FALSE, meaning the buffer was not synchronized with the server's disk, the client will need to mark the buffer somehow to show that a copy of it lives on the server and a new copy doesn't need to be sent, but a commit is still required 🫶
Note that this algorithm introduces a new state for buffers, so there are now three states for buffers 🔥 The three states are dirty, done but needs to be committed, and done. This extra state on the client will likely require modifications to the system outside of the NFS version 3 protocol client 💀
One proposal that got rejected was adding a boolean commit argument to the WRITE operation 🙈 It would be used to indicate whether the server should do a full file commit after doing the write. It sounds useful if the client knew it was doing the last write on the file, but it's hard to see how that could be used with existing client architectures, tbh.
The asynchronous write opens up a whole window of problems tied to write sharing 😭 For example: client A writes some data asynchronously. Client A is still holding the buffers cached, waiting to commit them later. Client B reads the modified data and writes it back to the server. Then the server crashes. When it comes back up, client A issues a COMMIT operation, which returns a different cookie as well as changed attributes. In this case, the correct action may or may not be to retransmit the cached buffers. Unfortunately, client A can't tell for sure, so it will need to retransmit the buffers, which overwrites the changes from client B. Fortunately, write sharing is rare and the solution matches the current write sharing situation. Without using locking for synchronization, the behaviour will be indeterminate.
In a high availability (redundant system) server implementation, there are two cases that relate to the verf changing 🦋 If the high availability server implementation does not use a shared-memory scheme, then the verf should change on failover. The unsynchronized data isn't available to the second processor, and there's no guarantee that the system which had the data cached was able to flush it to stable storage before going down. The client will need to retransmit the data to be safe. In a shared-memory high availability server implementation, the verf wouldn't need to change, because the server would still have the cached data available to flush. The exact policy regarding the verf in a shared-memory high availability implementation, however, is up to the server implementor 🌸
4.13 32 bit clients/servers and 64 bit clients/servers
the 64 bit nature of the NFS version 3 protocol causes several compatibility problems 🎀 The two most notable are mismatched clients and servers, meaning a 32 bit client with a 64 bit server, or a 64 bit client with a 32 bit server.
the problems of a 64 bit client with a 32 bit server are easy to handle ✨ The client will never encounter a file it can not handle. If it sends a request the server can not handle, the server should reject the request with an appropriate error.
the problems of a 32 bit client with a 64 bit server are way harder to handle 😭 Here the server has no problem, because it can handle anything the client can generate. However, the client may encounter a file it can not handle. The client won't be able to handle a file whose size can not be expressed in 32 bits. So the client won't be able to properly decode the size of the file into its local attributes structure. Also, a file can grow beyond the limit of the client while the client is accessing it.
The solutions to these problems are left up to the individual implementor 🧚 There are two common approaches used to resolve this situation, and the implementor can choose between them or even invent a new solution altogether.
The most common solution is for the client to deny access to any file whose size can not be expressed in 32 bits. This is probably the safest, but it does introduce some strange semantics when the file grows beyond the limit of the client while it is being accessed by that client. The file becomes inaccessible even while it is being accessed 🙈
The second solution is for the client to map any size greater than it can handle to the maximum size it can handle. Effectively, it is lying to the application program 💀 This lets the application access as much of the file as possible given the 32 bit offset restriction. It gets rid of the strange semantic of the file effectively disappearing after it has been accessed, but it does introduce other problems. The client will not be able to access the entire file.
Currently, the first solution is the recommended solution. However, client implementors are encouraged to do the best that they can to reduce the effects of this situation 🫶
5.0 Appendix I: Mount protocol 🌈
the changes from the NFS version 2 protocol to the NFS version 3 protocol have required some changes to be made in the MOUNT protocol 👑 To meet the needs of the NFS version 3 protocol, a new version of the MOUNT protocol has been defined. This new protocol satisfies the requirements of the NFS version 3 protocol and addresses several other current market requirements 💅
5.1 RPC Information
5.1.1 Authentication
The MOUNT service uses AUTH_NONE in the NULL procedure. AUTH_UNIX, AUTH_SHORT, AUTH_DES, or AUTH_KERB are used for all other procedures. Other authentication types may be supported in the future 🦋
5.1.2 Constants
these are the RPC constants needed to call the MOUNT service. They are given in decimal 🔢
PROGRAM 100005
VERSION 35.1.3 Transport address
the MOUNT service is normally supported over the TCP and UDP protocols. The rpcbind daemon should be queried for the correct transport address 🌸
5.1.4 Sizes
const MNTPATHLEN = 1024; /* Maximum bytes in a path name */
const MNTNAMLEN = 255; /* Maximum bytes in a name */
const FHSIZE3 = 64; /* Maximum bytes in a V3 file handle */5.1.5 Basic Data Types
typedef opaque fhandle3<FHSIZE3>;
typedef string dirpath<MNTPATHLEN>;
typedef string name<MNTNAMLEN>;
enum mountstat3 {
MNT3_OK = 0, /* no error */
MNT3ERR_PERM = 1, /* Not owner */
MNT3ERR_NOENT = 2, /* No such file or directory */
MNT3ERR_IO = 5, /* I/O error */
MNT3ERR_ACCES = 13, /* Permission denied */
MNT3ERR_NOTDIR = 20, /* Not a directory */
MNT3ERR_INVAL = 22, /* Invalid argument */
MNT3ERR_NAMETOOLONG = 63, /* Filename too long */
MNT3ERR_NOTSUPP = 10004, /* Operation not supported */
MNT3ERR_SERVERFAULT = 10006 /* A failure on the server */
};5.2 Server Procedures
the following sections define the RPC procedures supplied by a MOUNT version 3 protocol server 💖 The RPC procedure number is given at the top of the page with the name and version. The SYNOPSIS gives the name of the procedure, the list of the names of the arguments, the list of the names of the results, followed by the XDR argument declarations and results declarations. The information in the SYNOPSIS is specified in RPC Data Description Language as defined in [RFC1014]. The DESCRIPTION section tells what the procedure is expected to do and how its arguments and results are used. The ERRORS section lists the errors returned for specific types of failures. The IMPLEMENTATION field describes how the procedure is expected to work and how it should be used by clients 🧠
program MOUNT_PROGRAM {
version MOUNT_V3 {
void MOUNTPROC3_NULL(void) = 0;
mountres3 MOUNTPROC3_MNT(dirpath) = 1;
mountlist MOUNTPROC3_DUMP(void) = 2;
void MOUNTPROC3_UMNT(dirpath) = 3;
void MOUNTPROC3_UMNTALL(void) = 4;
exports MOUNTPROC3_EXPORT(void) = 5;
} = 3;
} = 100005;5.2.0 Procedure 0: Null - Do nothing
SYNOPSIS
void MOUNTPROC3_NULL(void) = 0;DESCRIPTION
Procedure NULL does not do any work. It is made available to allow server response testing and timing 🍓
IMPLEMENTATION
It is important that this procedure do no work at all, so that it can be used to measure the overhead of processing a service request. By convention, the NULL procedure should never require any authentication. A server may choose to ignore this convention in a more secure implementation, where responding to the NULL procedure call acknowledges the existence of a resource to an unauthenticated client 🔥
ERRORS
Since the NULL procedure takes no MOUNT protocol arguments and returns no MOUNT protocol response, it can not return a MOUNT protocol error. However, it is possible that some server implementations may return RPC errors based on security and authentication requirements 😭
5.2.1 Procedure 1: MNT - Add mount entry
SYNOPSIS
mountres3 MOUNTPROC3_MNT(dirpath) = 1;
struct mountres3_ok {
fhandle3 fhandle;
int auth_flavors<>;
};
union mountres3 switch (mountstat3 fhs_status) {
case MNT_OK:
mountres3_ok mountinfo;
default:
void;
};DESCRIPTION
Procedure MNT maps a pathname on the server to a file handle 🗺️ The pathname is an ASCII string that describes a directory on the server. If the call is successful (MNT3_OK), the server returns an NFS version 3 protocol file handle and a vector of RPC authentication flavors that are supported with the client's use of the file handle (or any file handles derived from it). The authentication flavors are defined in Section 7.2 and section 9 of [RFC1057] 🦋
IMPLEMENTATION
If mountres3.fhs_status is MNT3_OK, then mountres3.mountinfo contains the file handle for the directory and a list of acceptable authentication flavors. This file handle may only be used in the NFS version 3 protocol. This procedure also results in the server adding a new entry to its mount list recording that this client has mounted the directory. AUTH_UNIX authentication or better is required 💜
ERRORS
MNT3ERR_NOENTMNT3ERR_IOMNT3ERR_ACCESMNT3ERR_NOTDIRMNT3ERR_NAMETOOLONG
5.2.2 Procedure 2: DUMP - Return mount entries ✨
SYNOPSIS
mountlist MOUNTPROC3_DUMP(void) = 2;
typedef struct mountbody *mountlist;
struct mountbody {
name ml_hostname;
dirpath ml_directory;
mountlist ml_next;
};DESCRIPTION
okay so DUMP is the procedure that hands back the list of remotely mounted file systems 📋 each entry is one client host name and directory pair. that's the whole vibe 💅
IMPLEMENTATION
this list is built from a list the server keeps of clients that asked for file handles with the MNT procedure 🌸 entries only get removed when a client calls UMNT or UMNTALL. so ngl, entries can go stale if a client crashes and never sends UMNT for every file system it mounted, or never sends a UMNTALL to clear out everything it had on that server 💀
ERRORS
there are no MOUNT protocol errors that this procedure can return 🙌 however, RPC errors may still come back for authentication or other RPC failures.
5.2.3 Procedure 3: UMNT - Remove mount entry 🦋
SYNOPSIS
void MOUNTPROC3_UMNT(dirpath) = 3;DESCRIPTION
UMNT removes the mount list entry for the directory that this client previously got through a MNT call 🧹 AUTH_UNIX authentication or better is required.
IMPLEMENTATION
servers usually keep a list of the clients that have file systems mounted 📝 back in the day, that list got used to tell clients when the server was about to shut down 🕰️
ERRORS
no MOUNT protocol errors can come back from this one either 🫶 RPC errors may still be returned for authentication or other RPC failures.
5.2.4 Procedure 4: UMNTALL - Remove all mount entries 🧼
SYNOPSIS
void MOUNTPROC3_UMNTALL(void) = 4;DESCRIPTION
UMNTALL wipes out every mount entry this client previously recorded through MNT calls 💫 AUTH_UNIX authentication or better is required.
IMPLEMENTATION
clients should use this when they're recovering after a system shutdown 🔄 if a client couldn't unmount everything before going down, or it crashed because of a software or hardware problem, some servers might still have mount entries for it 😭 UMNTALL is an easy way to tell every server at once that the client has nothing mounted anymore 🎀 BUT it's usually implemented with broadcast RPC, so it's only of limited usefulness 🙈
ERRORS
no MOUNT protocol errors come back from this procedure 💖 RPC errors may still be returned for authentication or other RPC failures.
5.2.5 Procedure 5: EXPORT - Return export list 🌈
SYNOPSIS
exports MOUNTPROC3_EXPORT(void) = 5;
typedef struct groupnode *groups;
struct groupnode {
name gr_name;
groups gr_next;
};
typedef struct exportnode *exports;
struct exportnode {
dirpath ex_dir;
groups ex_groups;
exports ex_next;
};DESCRIPTION
EXPORT gives back a list of every exported file system, plus which clients are allowed to mount each one 🗂️ the names in the group list are implementation-specific, so clients can't directly read them 🙃 these names can stand for single hosts or whole groups of hosts.
IMPLEMENTATION
this procedure usually returns the contents of the list of shared or exported file systems 🌟 these are the ones made available to NFS version 3 protocol clients.
ERRORS
no MOUNT protocol errors can be returned from this procedure 🎉 RPC errors may still come back for authentication or other RPC failures.
6.0 Appendix II: Lock manager protocol 🔐
okay bestie, here's the twist: NFS version 2 and NFS version 3 are both stateless 🫠 so an additional Network Lock Manager (NLM) protocol is required to support locking of NFS-mounted files. the NLM version 3 protocol, which goes with NFS version 2, is documented in [X/OpenNFS] 📚
some of the changes in NFS version 3 need a new NLM version too 👑 that's the NLM version 4 protocol. this table shows which NFS and NLM versions go together ✨
NFS and NLM protocol compatibility
+---------+---------+
| NFS | NLM |
| Version | Version |
+===================+
| 2 | 1,3 |
+---------+---------+
| 3 | 4 |
+---------+---------+this appendix only talks about the differences between NLM version 3 and NLM version 4 🎀 almost all the names in NLM version 4 got a version number added, just like in NFS version 3 🏷️ this appendix doesn't cover changes that are only a name change.
6.1 RPC Information 🧾
6.1.1 Authentication 🔑
the NLM service uses AUTH_NONE in the NULL procedure 🫧 AUTH_UNIX, AUTH_SHORT, AUTH_DES, and AUTH_KERB are used for all other procedures. other authentication types might be supported in the future 🔮
6.1.2 Constants 🔢
these are the RPC constants you need to call the NLM service 💜 they're given in decimal.
PROGRAM 100021
VERSION 46.1.3 Transport Address 🚚
the NLM service is normally supported over TCP and UDP 🌐 the rpcbind daemon should be queried for the correct transport address.
6.1.4 Basic Data Types 🧩
uint64
typedef unsigned hyper uint64;
int64
typedef hyper int64;
uint32
typedef unsigned long uint32;
int32
typedef long int32;these types are new for the NLM version 4 protocol 🆕 they're the same as the ones in the NFS version 3 protocol.
nlm4_stats
enum nlm4_stats {
NLM4_GRANTED = 0,
NLM4_DENIED = 1,
NLM4_DENIED_NOLOCKS = 2,
NLM4_BLOCKED = 3,
NLM4_DENIED_GRACE_PERIOD = 4,
NLM4_DEADLCK = 5,
NLM4_ROFS = 6,
NLM4_STALE_FH = 7,
NLM4_FBIG = 8,
NLM4_FAILED = 9
};nlm4_stats tells you whether a call worked or failed 💅 this version adds several new error codes, so clients can give applications more precise failure info 🔥
NLM4_GRANTED : the call completed successfully ✅
NLM4_DENIED : the call failed. for lock attempts, this status means that if the client retries later, it might succeed 🔁
NLM4_DENIED_NOLOCKS : the call failed because the server couldn't allocate the resources it needed 😩
NLM4_BLOCKED : a blocking request can't be granted right away. the server will send an NLMPROC4_GRANTED callback to the client once the lock is granted 🔔
NLM4_DENIED_GRACE_PERIOD : the call failed because the server is re-establishing old locks after a reboot and isn't ready for normal service yet 🌅
NLM4_DEADLCK : the request can't be granted, and blocking would cause a deadlock 💀
NLM4_ROFS : the call failed because the remote file system is read-only. for example, some server implementations might not support exclusive locks on read-only file systems 🔒
NLM4_STALE_FH : the call failed because it uses an invalid file handle. this can happen if the file was removed or if access to it was revoked on the server 👻
NLM4_FBIG : the call failed because it specified a length or offset that's beyond the range the server supports 📏
NLM4_FAILED : the call failed for a reason not already listed 🫥 the client should treat this status as a strong hint not to retry the request.
nlm4_holder
struct nlm4_holder {
bool exclusive;
int32 svid;
netobj oh;
uint64 l_offset;
uint64 l_len;
};this structure tells you who's holding a lock 🕵️♀️ the exclusive field says whether the holder has an exclusive lock or a shared lock. the svid field identifies the process holding the lock. the oh field is an opaque object that identifies the host or process holding the lock. l_len and l_offset identify the region that's locked. the only difference between NLM version 3 and NLM version 4 here is that in version 3, l_len and l_offset are 32 bits wide, while in version 4 they're 64 bits wide. 🧠
nlm4_lock
struct nlm4_lock {
string caller_name<LM_MAXSTRLEN>;
netobj fh;
netobj oh;
int32 svid;
uint64 l_offset;
uint64 l_len;
};this structure describes a lock request 📨 the caller_name field identifies the host making the request. the fh field identifies the file to lock. the oh field is an opaque object that identifies the host or process making the request, and svid identifies the process making the request. l_offset and l_len identify the region of the file the lock controls. an l_len of 0 means "to end of file".
there are two differences between the NLM version 3 and NLM version 4 versions of this structure 👀 first, in version 3 the length and offset are 32 bits wide, while they're 64 bits wide in version 4. second, in version 3 the file handle is a fixed-length NFS version 2 file handle, encoded as a byte count followed by a byte array. in NFS version 3, the file handle is already variable-length, so it's copied straight into the fh field. that means the first four bytes of the fh field match the byte count in an NFS version 3 nfs_fh3, and the rest of the fh field holds the byte array from that nfs_fh3 🧵
nlm4_share
struct nlm4_share {
string caller_name<LM_MAXSTRLEN>;
netobj fh;
netobj oh;
fsh4_mode mode;
fsh4_access access;
};this structure is used to support DOS file sharing 🗃️ the caller_name field identifies the host making the request. the fh field identifies the file being operated on. the oh field is an opaque object that identifies the host or process making the request. the mode and access fields specify the file-sharing and access modes. the encoding of fh is a byte count, followed by the file handle byte array. see the description of nlm4_lock for more details 💜
6.2 NLM Procedures ✨
Okay bestie, the NLM version 4 procedures are basically the same as version 3 💅 The only real difference is the new NULL procedure, which lets you check if a server is responsive 🫶 The procedures with _MSG and _RES suffixes are asynchronous messages, and for those a void response means no reply 🦋 They also got renamed so they don't clash with the values of nlm4_stats 🍓 Here's the full definition:
version NLM4_VERS {
void
NLMPROC4_NULL(void) = 0;
nlm4_testres
NLMPROC4_TEST(nlm4_testargs) = 1;
nlm4_res
NLMPROC4_LOCK(nlm4_lockargs) = 2;
nlm4_res
NLMPROC4_CANCEL(nlm4_cancargs) = 3;
nlm4_res
NLMPROC4_UNLOCK(nlm4_unlockargs) = 4;
nlm4_res
NLMPROC4_GRANTED(nlm4_testargs) = 5;
void
NLMPROC4_TEST_MSG(nlm4_testargs) = 6;
void
NLMPROC4_LOCK_MSG(nlm4_lockargs) = 7;
void
NLMPROC4_CANCEL_MSG(nlm4_cancargs) = 8;
void
NLMPROC4_UNLOCK_MSG(nlm4_unlockargs) = 9;
void
NLMPROC4_GRANTED_MSG(nlm4_testargs) = 10;
void
NLMPROC4_TEST_RES(nlm4_testres) = 11;
void
NLMPROC4_LOCK_RES(nlm4_res) = 12;
void
NLMPROC4_CANCEL_RES(nlm4_res) = 13;
void
NLMPROC4_UNLOCK_RES(nlm4_res) = 14;
void
NLMPROC4_GRANTED_RES(nlm4_res) = 15;
nlm4_shareres
NLMPROC4_SHARE(nlm4_shareargs) = 20;
nlm4_shareres
NLMPROC4_UNSHARE(nlm4_shareargs) = 21;
nlm4_res
NLMPROC4_NM_LOCK(nlm4_lockargs) = 22;
void
NLMPROC4_FREE_ALL(nlm4_notify) = 23;
} = 4;6.2.0 Procedure 0: NULL - Do nothing 🌸
SYNOPSIS
void NLMPROC4_NULL(void) = 0;DESCRIPTION
The NULL procedure does no work 🧚 It's available in every RPC service so you can test server response and timing.
IMPLEMENTATION
It's super important that this procedure does absolutely no work, so it can be used to measure the overhead of handling a service request 👑 By convention, the NULL procedure should never require any authentication 🔐
ERRORS
Some server implementations may return RPC errors based on their security and authentication requirements. Not shocking, but worth knowing 🙈
6.3 Implementation issues 🎀
6.3.1 64-bit offsets and lengths 🔥
Some NFS version 3 protocol servers can only handle requests where the file offset or length fits in 32 bits or fewer 😭 For those servers, the lock manager gets the same restriction. If a lock manager gets a request it can't handle (because the offset or length uses more than 32 bits), it should return the error NLM4_FBIG 💅
6.3.2 File handles 🦋
The file handle format changed from NFS version 2 to NFS version 3, and that makes the lock manager's life harder 💀 First, the lock manager needs a way to tell when an NFS version 2 file handle refers to the same file as an NFS version 3 file handle. (This assumes the lock manager supports both NLM version 3 and NLM version 4 clients.) Second, if the lock manager runs file handles through a hashing function, that hashing function might need retuning so it works with NFS version 3 file handles as well as NFS version 2 file handles 🧠
7.0 Appendix III: Bibliography 📚
Here's the reading list, with every citation exactly as written 🍓
[Corbin] Corbin, John, "The Art of Distributed
Programming-Programming Techniques for Remote
Procedure Calls." Springer-Verlag, New York, New
York. 1991. Basic description of RPC and XDR
and how to program distributed applications
using them.
[Glover] Glover, Fred, "TNFS Protocol Specification,"
Trusted System Interest Group, Work in
Progress.
[Israel] Israel, Robert K., Sandra Jett, James Pownell,
George M. Ericson, "Eliminating Data Copies in
UNIX-based NFS Servers," Uniforum Conference
Proceedings, San Francisco, CA,
February 27 - March 2, 1989. Describes two
methods for reducing data copies in NFS server
code.
[Jacobson] Jacobson, V., "Congestion Control and
Avoidance," Proc. ACM SIGCOMM `88, Stanford, CA,
August 1988. The paper describing improvements
to TCP to allow use over Wide Area Networks and
through gateways connecting networks of varying
capacity. This work was a starting point for the
NFS Dynamic Retransmission work.
[Juszczak] Juszczak, Chet, "Improving the Performance and
Correctness of an NFS Server," USENIX Conference
Proceedings, USENIX Association, Berkeley, CA,
June 1990, pages 53-63. Describes reply cache
implementation that avoids work in the server by
handling duplicate requests. More important,
though listed as a side-effect, the reply cache
aids in the avoidance of destructive
non-idempotent operation re-application --
improving correctness.
[Kazar] Kazar, Michael Leon, "Synchronization and Caching
Issues in the Andrew File System," USENIX Conference
Proceedings, USENIX Association, Berkeley, CA,
Dallas Winter 1988, pages 27-36. A description
of the cache consistency scheme in AFS.
Contrasted with other distributed file systems.
[Macklem] Macklem, Rick, "Lessons Learned Tuning the
4.3BSD Reno Implementation of the NFS Protocol,"
Winter USENIX Conference Proceedings, USENIX
Association, Berkeley, CA, January 1991.
Describes performance work in tuning the 4.3BSD
Reno NFS implementation. Describes performance
improvement (reduced CPU loading) through
elimination of data copies.
[Mogul] Mogul, Jeffrey C., "A Recovery Protocol for Spritely
NFS," USENIX File System Workshop Proceedings,
Ann Arbor, MI, USENIX Association, Berkeley, CA,
May 1992. Second paper on Spritely NFS proposes
a lease-based scheme for recovering state of
consistency protocol.
[Nowicki] Nowicki, Bill, "Transport Issues in the Network
File System," ACM SIGCOMM newsletter Computer
Communication Review, April 1989. A brief
description of the basis for the dynamic
retransmission work.
[Pawlowski] Pawlowski, Brian, Ron Hixon, Mark Stein, Joseph
Tumminaro, "Network Computing in the UNIX and
IBM Mainframe Environment," Uniforum `89 Conf.
Proc., (1989) Description of an NFS server
implementation for IBM's MVS operating system.
[RFC1014] Sun Microsystems, Inc., "XDR: External Data
Representation Standard", RFC 1014,
Sun Microsystems, Inc., June 1987.
Specification for canonical format for data
exchange, used with RPC.
[RFC1057] Sun Microsystems, Inc., "RPC: Remote Procedure
Call Protocol Specification", RFC 1057,
Sun Microsystems, Inc., June 1988.
Remote procedure protocol specification.
[RFC1094] Sun Microsystems, Inc., "Network Filesystem
Specification", RFC 1094, Sun Microsystems, Inc.,
March 1989. NFS version 2 protocol
specification.
[Sandberg] Sandberg, R., D. Goldberg, S. Kleiman, D. Walsh,
B. Lyon, "Design and Implementation of the Sun
Network Filesystem," USENIX Conference
Proceedings, USENIX Association, Berkeley, CA,
Summer 1985. The basic paper describing the
SunOS implementation of the NFS version 2
protocol, and discusses the goals, protocol
specification and trade-offs.
[Srinivasan] Srinivasan, V., Jeffrey C. Mogul, "Spritely
NFS: Implementation and Performance of Cache
Consistency Protocols", WRL Research Report
89/5, Digital Equipment Corporation Western
Research Laboratory, 100 Hamilton Ave., Palo
Alto, CA, 94301, May 1989. This paper analyzes
the effect of applying a Sprite-like consistency
protocol applied to standard NFS. The issues of
recovery in a stateful environment are covered
in [Mogul].
[X/OpenNFS] X/Open Company, Ltd., X/Open CAE Specification:
Protocols for X/Open Internetworking: XNFS,
X/Open Company, Ltd., Apex Plaza, Forbury Road,
Reading Berkshire, RG1 1AX, United Kingdom,
1991. This is an indispensable reference for
NFS version 2 protocol and accompanying
protocols, including the Lock Manager and the
Portmapper.
[X/OpenPCNFS] X/Open Company, Ltd., X/Open CAE Specification:
Protocols for X/Open Internetworking: (PC)NFS,
Developer's Specification, X/Open Company, Ltd.,
Apex Plaza, Forbury Road, Reading Berkshire, RG1
1AX, United Kingdom, 1991. This is an
indispensable reference for NFS version 2
protocol and accompanying protocols, including
the Lock Manager and the Portmapper.8. Security Considerations 🔐
Okay, this part is serious, no cap 😭 Since sensitive file data may be transmitted to or received from a server over the NFS protocol, implementations of this protocol should address authentication, privacy, and data integrity issues.
As with the previous protocol revision (version 2), NFS version 3 defers to the authentication provisions of the supporting RPC protocol [RFC1057], and assumes that data privacy and integrity are provided by the underlying transport layers available in each implementation of the protocol. See section 4.4 for a discussion relating to file access permissions 🫶
9. Acknowledgements 🌈
This description of the protocol is derived from an original document written by Brian Pawlowski and revised by Peter Staubach. This protocol is the result of a co-operative effort that brings together the contributions of Geoff Arnold, Brent Callaghan, John Corbin, Fred Glover, Chet Juszczak, Mike Eisler, John Gillono, Dave Hitz, Mike Kupfer, Rick Macklem, Ron Minnich, Brian Pawlowski, David Robinson, Rusty Sandberg, Craig Schamp, Spencer Shepler, Carl Smith, Mark Stein, Peter Staubach, Tom Talpey, Rob Thurlow, and Mark Wittle 👑
10. Authors' Addresses ✨
okay bestie, this is the last stop 🦋 and it's the people behind the whole thing! if you wanna send comments about this protocol, the spec says to email the group at:
nfs3@eng.sun.comhere's the squad that wrote it 🫶
Brent Callaghan
Sun Microsystems, Inc.
2550 Garcia Avenue
Mailstop UMTV05-44
Mountain View, CA 94043-1100
Phone: 1-415-336-1051
Fax: 1-415-336-6015
EMail: brent.callaghan@eng.sun.comBrian Pawlowski
Network Appliance Corp.
319 North Bernardo Ave.
Mountain View, CA 94043
Phone: 1-415-428-5136
Fax: 1-415-428-5151
EMail: beepy@netapp.comPeter Staubach
Sun Microsystems, Inc.
2550 Garcia Avenue
Mailstop UMTV05-44
Mountain View, CA 94043-1100
Phone: 1-415-336-5615
Fax: 1-415-336-6015
EMail: peter.staubach@eng.sun.commain character energy 👑 these three are literally the reason NFS version 3 exists 💅🌸
📦 where this page comes from
- ietf: rfc1813 1995-06
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms