RFC 8276

File System Extended Attributes in NFSv4

📅 published December 2017, from ietf

RFC 8276: File System Extended Attributes in NFSv4 ✨

Request for Comments: 8276 🎀 Category: Standards Track 👑 Stream: Internet Engineering Task Force (IETF) 💜 Date: December 2017 🌸 ISSN: 2070-1721 Authors: M. Naik (Nutanix) and M. Eshel (IBM Almaden) 🫶 Obsoletes / Updates: The title block doesn't list any RFCs this one obsoletes or updates. 🦋

Abstract

Okay bestie, this one is an optional feature that extends NFSv4 🧚. It lets NFSv4 clients read and change extended attributes (xattrs for short) 💅. Xattrs are key/value metadata that a file system stores with files and directories, and the file system doesn't interpret them. Lots of modern local file systems already support them ✨. To check whether a server supports xattrs, clients use new file attributes, and the support itself comes from new operations for getting and setting xattrs on file system objects 🔥.

Status of This Memo

This is an Internet Standards Track document 👑.

It's a product of the Internet Engineering Task Force (IETF) 💜, so it reflects the consensus of the IETF community 🫶. It went through public review and got approved for publication by the Internet Engineering Steering Group (IESG) 🎀. If you want more on Internet Standards, check Section 2 of RFC 7841 🌈.

For the current status, any errata, and how to send feedback, go to https://www.rfc-editor.org/info/rfc8276 🦋.

Copyright (c) 2017 IETF Trust and the persons identified as the document authors 🌸. All rights reserved 💖.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as they stand on the publication date 📜. Please read these carefully, because they spell out your rights and restrictions for this document 🙈. Code Components pulled out of this document must include Simplified BSD License text, as described in Section 4.e of the Trust Legal Provisions, and they come without warranty, as described in the Simplified BSD License ⚠️.

Table of Contents

1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   4
  1.1.  Terminology . . . . . . . . . . . . . . . . . . . . . . .   5
2.  Uses of Extended Attributes . . . . . . . . . . . . . . . . .   5
3.  Functional Gaps Due to Lack of NFSv4 Extended Attribute
    Support . . . . . . . . . . . . . . . . . . . . . . . . . . .   5
4.  File System Support for Extended Attributes . . . . . . . . .   6
5.  Namespaces  . . . . . . . . . . . . . . . . . . . . . . . . .   7
6.  Relationship with Named Attributes  . . . . . . . . . . . . .   7
7.  XDR Description . . . . . . . . . . . . . . . . . . . . . . .   8
  7.1.  Code Components Licensing Notice  . . . . . . . . . . . .   9
  7.2.  XDR for Xattr Extension . . . . . . . . . . . . . . . . .  11
8.  Protocol Extensions . . . . . . . . . . . . . . . . . . . . .  11
  8.1.  New Definitions . . . . . . . . . . . . . . . . . . . . .  11
  8.2.  New Attribute . . . . . . . . . . . . . . . . . . . . . .  12
    8.2.1.  xattr_support . . . . . . . . . . . . . . . . . . . .  12
  8.3.  New Error Definitions . . . . . . . . . . . . . . . . . .  12
    8.3.1.  NFS4ERR_NOXATTR (Error Code 10095)  . . . . . . . . .  12
    8.3.2.  NFS4ERR_XATTR2BIG (Error Code 10096)  . . . . . . . .  13
  8.4.  New Operations  . . . . . . . . . . . . . . . . . . . . .  13
    8.4.1.  GETXATTR - Get an Extended Attribute of a File  . . .  14
    8.4.2.  SETXATTR - Set an Extended Attribute of a File  . . .  15
    8.4.3.  LISTXATTRS - List Extended Attributes of a File . . .  17
    8.4.4.  REMOVEXATTR - Remove an Extended Attribute of a File   18
    8.4.5.  Valid Errors  . . . . . . . . . . . . . . . . . . . .  19
  8.5.  Modifications to Existing Operations  . . . . . . . . . .  21
  8.6.  Numeric Values Assigned to Protocol Extensions  . . . . .  22
  8.7.  Caching . . . . . . . . . . . . . . . . . . . . . . . . .  23
  8.8.  Xattrs and File Locking . . . . . . . . . . . . . . . . .  25
  8.9.  pNFS Considerations . . . . . . . . . . . . . . . . . . .  25
9.  Security Considerations . . . . . . . . . . . . . . . . . . .  25
10. IANA Considerations . . . . . . . . . . . . . . . . . . . . .  25
11. References  . . . . . . . . . . . . . . . . . . . . . . . . .  26
  11.1.  Normative References . . . . . . . . . . . . . . . . . .  26
  11.2.  Informative References . . . . . . . . . . . . . . . . .  27
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . .  28
Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  28

1. Introduction 🌟

Okay so, extended attributes (a.k.a. xattrs) are a way to attach opaque metadata to file system objects, stored as key/value pairs 🦋. They're especially handy when they hold info that the object itself doesn't have, or can't have 💖. User-space apps can make, look up, and change those key/value pairs whenever they want 🧚.

Here's the thing: xattrs are file system agnostic 🫶. Apps use one interface that doesn't care which file system is underneath, so they never have to think about how the pairs are stored 💅. Every major operating system has facilities for accessing and changing xattrs 🌈. A lot of user-space tools even bring xattrs along with the regular attributes that need to survive when an object gets updated, moved, or copied 🔥.

Now, the NFSv4 spec has never included xattrs 🙈. There are some issues to sort out first. Like named attributes, some parts of how xattrs get handled aren't precisely defined, and no standard like POSIX [POSIX] formally documents them 📜. Still, xattrs are widely deployed, and modern disk-based file systems support them almost everywhere 💜.

Right now there's no spec for mapping xattrs onto the existing file attributes in the NFSv4 protocol [RFC5661] [RFC7530] [RFC7862] 😭. So most NFSv4 client implementations just ignore app-specified xattrs 💀. That causes data loss when someone copies a file with xattrs over NFSv4 from one file system to another that also supports xattrs 😱.

We need a way to stop that data loss 🛡️. That means exposing xattrs inside NFSv4, even though not every file system implements them in a fully compatible way 🎀.

This document explains (in Section 5) why the NFSv4 named attributes standardized in [RFC5661] aren't a good fit for representing xattrs 🙅‍♀️. Instead, it describes a separate protocol mechanism for xattrs 🦋. As a result, xattrs and named attributes are both OPTIONAL features, and servers are free to support either one, both, or neither ✨.

1.1. Terminology 📖

Okay, here are the BCP 14 key words you'll spot in capitals: "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" 🔥. When they show up in all capitals like that, they mean exactly what BCP 14 [RFC2119] [RFC8174] says, and only then 🫶.

2. Uses of Extended Attributes 💖

Apps can stash tracking info in xattrs 🕵️‍♀️. For example, they can record which app created the file, add a tag for when a data integrity scrubber last verified it, or keep a checksum/crypto hash of the contents along with the date of that signature 🔐. Xattrs are also great for decorations and annotations 🎀. Say you download a file from a web server: you can tag it with the URL so you can track down where it came from later 🌐. Same deal with an email attachment: when you save it, you can tag it with the email's message-id so you can trace back the original message 💌.

Apps sometimes need to act differently depending on the type of file 🧚. Take file managers like GNOMEs: they give each file type its own icon, its own click behavior, and its own list of operations, all based on the file format 🌈. The file type can come from the file extension (Windows) or from inspecting the file itself (Unix MIME type) 👀. Some file managers figure this out on the fly 💅. Others work it out once and cache it, usually in a custom database 🗃️. Those caching ones have a real job keeping that database in sync with the files, since files can change without the file manager knowing 😩. A better move is to skip the custom database and store that metadata in xattrs instead 👑. That's easier to maintain, gives faster access, and makes the info easy for other applications to reach [Love] ✨.

3. Functional Gaps Due to Lack of NFSv4 Extended Attribute Support 🚧

Beyond the data loss from Section 1 that happens when you use xattrs on local file systems, apps that use xattrs run into more trouble, because NFSv4 doesn't support them yet 😤. So some apps either can't run over NFSv4 or only work in clunky ways 💔. Here are a few examples 👇.

Swift, the OpenStack distributed object store, uses xattrs to keep an object's metadata together with its data in one file 📦. Swift-on-File [Swift] hands off responsibility for object durability and availability to the underlying file system 🏰. When this was written, you needed a native file system client to mount those volumes 🙄. Xattr support in NFSv4 would open the door to storing and reading data from other storage systems, and it would make moving data between different backend storage systems much easier 🚀.

Baloo, the file indexing and search framework for Key Distribution Exchange (KDE), moved its tags, ratings, and comments into file system xattrs instead of a custom database, because it's simpler 🧠. Starting with KDE Plasma 5.1, NFS is no longer supported because it lacks xattr support [KDE] 😭.

Part 1 of 5, bestie! More of the spec is coming up next 🫶💅

4. File System Support for Extended Attributes ✨

Okay so extended attributes are supported by basically every modern file system, bestie 💾🫶

On Linux, the file systems that support them include ext3, ext4, JFS, XFS, and Btrfs 🌸 You can use the getfattr and setfattr utilities to retrieve and set xattrs. Every xattr name has to start with its category name and a dot, which is why these categories are called namespaces 🎀 Right now Linux has four of them: user, trusted, security, and system [Linux]. People have also published recommendations on how to use them [freedesktop] 📚

FreeBSD supports xattrs in two universal namespaces, user and system, but individual file systems are allowed to add their own namespaces on top of that [FreeBSD] 🦋

Some file systems can store both xattrs and named attributes. For more on how those two relate, see Section 5 💅 Solaris 9 and later have file "forks", which show up as files inside a hidden directory attached to the target file [fsattr] 🧚 And in NTFS, xattrs can live inside "file streams" [NTFS] ✨

Xattrs can be read and written through system calls or shell commands, and most user-space tools that keep other file attributes around also keep them 🍓 For example, the "rsync" remote copy program will correctly preserve user-extended attributes between Linux/ext4 and OSX/hfs, as long as it strips off the Linux-specific user. prefix 💖

5. Namespaces 🌈

Operating systems can define multiple "namespaces" where xattrs can be set, and they are so much more than just organizational buckets 👑 The OS can enforce different access policies and give different capabilities depending on which namespace you're in. Linux, for example, has "security", "system", "trusted", and "user" namespaces, and the first three are Linux-specific [freedesktop] 💜

Implementations generally agree on what the "user" namespace means. It lets applications stash arbitrary user attribute data on file system objects, and access is controlled by the normal file system attributes 🫶 Because of that, reading and setting xattrs in the user namespace can be treated as interoperable across platforms and vendors 🙈 Attributes from other namespaces are usually platform specific.

This document only covers namespaces for user-managed metadata, so it doesn't have to spell out how system-interpreted xattrs should behave 💅 Xattr values are treated as application data, the same way the contents of named attributes, files, and symbolic links are. Servers are responsible for storing whatever value the client specifies and returning it on demand 🎀 Xattr keys and values MUST NOT be interpreted by NFS clients and servers, because that would lead to non-interoperable implementations 😭 If there's ever a need for attributes that servers have to act on, the right semantics would be defined by adding a new attribute for that purpose, as allowed by [RFC5661] and [RFC8178] ✨

6. Relationship with Named Attributes 🦋

[RFC7530] defines named attributes as opaque byte streams that are tied to a directory or file and referred to by a string name. Clients use them to attach application-specific data to a regular file or directory 🌸 Xattrs are similar in concept and use, but there are some important semantic differences, bestie 👀

File systems usually define get and set operations on individual xattrs as atomic, though collectively they may be independent 💖 Xattrs usually have size limits somewhere from a few bytes to several kilobytes, and the maximum isn't universally defined. It's usually restricted by the file system itself 🍓 Like Access Control Lists (ACLs), the amount of xattr data exchanged between client and server for get/set operations can be considered to fit in a single COMPOUND request, bounded by the channel's negotiated maximum size for requests 🧚 Named attributes are different because they're unbounded data streams and don't impose atomicity requirements 🌈

Individual named attributes are basically treated like files, so they get opened and closed just like files do 📁 Caching their data has to be handled the same way data caching works for ordinary files following close-to-open semantics. Xattrs, on the other hand, have caching requirements that look more like other file attributes 💅

Named attributes and xattrs have different semantics and applications treat them as belonging to disjoint namespaces, so mapping one onto the other would be a compromise at best 😬 Even so, the underlying file system structure used to store named attributes can usually store xattrs too. The reverse usually doesn't work, though, because of the size limits on xattrs 🙈

It might be possible to write guidance on using the named attribute mechanism to act like xattrs, like carving out a namespace and specifying locking primitives to enforce atomicity on individual get/set operations. But that approach is problematic enough that this document won't attempt it ✋ A client trying to use xattrs through named attributes with a server that supports xattrs directly would get a lower level of service, and it could fail to cooperate with a local application running on the server, unless the server file system defined its own interoperability constraints 💀 File systems that already implement xattrs and named attributes natively would need extra guidance, like reserving a named attribute namespace specifically for implementation purposes 🔥

7. XDR Description 🧁

This document includes the External Data Representation (XDR) [RFC4506] description of the extended attributes ✨ The XDR is embedded in the document in a way that makes it easy to extract into a ready-to-compile form. You can feed the document into this shell script to get the machine-readable XDR description of extended attributes:

<CODE BEGINS>

#! /bin/sh
grep '^ *///' $* | sed 's?^ */// ??' | sed 's?^ *///$??'

<CODE ENDS>

So if the script above is saved in a file called "extract.sh", and this document is saved in a file called "spec.txt", you can run:

sh extract.sh < spec.txt > xattr_prot.x

What the script does is strip leading white space from each line, along with the sentinel sequence of "///" 🎀 The first section of the embedded XDR file header comes next. More XDR descriptions, each with the sentinel sequence, are embedded throughout the document 💖

Heads up: the XDR code in this document depends on types from the NFSv4.2 nfs4_prot.x file [RFC7863] 👑 That includes nfs types ending in a 4, like nfs_cookie4 and count4, plus more generic types like opaque and bool 🫶

To produce a compilable XDR file, here's the suggested procedure:

  • Extract the file nfs4_prot.x as described in [RFC7863].
  • Extract xattr_prot.x from this document as described above.
  • Apply any changes required for other extensions to be included together with the xattr extension.
  • Perform modifications to nfs4_prot.x as described by comments within xattr_prot.x.
  • Extend the unions nfs_argop4 and nfs_resop4 to include cases for the new operations defined in this document.
  • Combine the XDR files for the base NFSv4.2 protocol and all needed extensions, either by concatenating the relevant XDR files or by using file inclusion 🌸

7.1. Code Components Licensing Notice 📜

Both the XDR description and the scripts used for extracting the XDR description are Code Components as described in "Legal Provisions Relating to IETF Documents", Section 4 of [LEGAL]. These Code Components are licensed according to the terms of that document 🦋

   <CODE BEGINS>

   /// /*
   ///  * Copyright (c) 2017 IETF Trust and the persons identified
   ///  * as authors of the code.  All rights reserved.
   ///  *
   ///  * Redistribution and use in source and binary forms, with
   ///  * or without modification, are permitted provided that the
   ///  * following conditions are met:
   ///  *
   ///  * o Redistributions of source code must retain the above
   ///  *   copyright notice, this list of conditions and the
   ///  *   following disclaimer.
   ///  *
   ///  * o Redistributions in binary form must reproduce the above
   ///  *   copyright notice, this list of conditions and the
   ///  *   following disclaimer in the documentation and/or other
   ///  *   materials provided with the distribution.
   ///  *
   ///  * o Neither the name of Internet Society, IETF or IETF
   ///  *   Trust, nor the names of specific contributors, may be
   ///  *   used to endorse or promote products derived from this
   ///  *   software without specific prior written permission.
   ///  *
   ///  *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS
   ///  *   AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED
   ///  *   WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
   ///  *   IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
   ///  *   FOR A PARTICULAR PURPOSE ARE DISCLAIMED.  IN NO
   ///  *   EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
   ///  *   LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
   ///  *   EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
   ///  *   NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
   ///  *   SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
   ///  *   INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
   ///  *   LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
   ///  *   OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
   ///  *   IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
   ///  *   ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
   ///  *
   ///  * This code was derived from RFC 8276.
   ///  * Please reproduce this note if possible.
   ///  */

   <CODE ENDS>

7.2. XDR for Xattr Extension 🧩

     <CODE BEGINS>

     /// /*
     ///  * xattr_prot.x
     ///  */

     /// /*
     ///  * The following includes statements that are for example only.
     ///  * The actual XDR definition files are generated separately
     ///  * and independently and are likely to have a different name.
     ///  * %#include <rpc_prot.x>
     ///  * %#include <nfsv42.x>
     ///  */

     <CODE ENDS>

8. Protocol Extensions 🚀

This section documents extensions to the NFSv4 protocol operations so clients can query and modify xattrs 💖 A new attribute lets clients figure out whether the file system being accessed supports xattrs. New operations are defined to query and set xattr keys and values. The ACCESS operation also gets new mask bits that give access information about xattrs 🦋

These changes follow the applicable guidelines for valid NFSv4 XDR protocol extensions, as specified in [RFC8178], and they follow the rules for extensions that can be made without changing the minor version number ✨🫶

8.1. New Definitions

okay bestie, the new definitions are here 💅✨ Every xattr is a key/value pair. xattrkey4 is a string that denotes the xattr key name, and attrvalue4 is a variable-length string that identifies the value of the xattr. The way xattrkey4 handles internationalization is the same as NFSv4 file names and named attribute names, as described in [RFC7530]. Any regular file or directory can have a set of extended attributes, each one a key plus its value 🎀

/// typedef component4     xattrkey4;
/// typedef opaque         xattrvalue4<>;

The NFS client or server MUST NOT interpret the contents of xattrkey4 or xattrvalue4. Hands off, no peeking 🙈

8.2. New Attribute

the per-fs read-only attribute below is how you check whether xattrs are supported 🔍 Servers don't have to support this attribute, and some NFSv4.2 servers might not even know it exists 💀 Before you interrogate it with GETATTR, a client should check whether it's a supported attribute by looking at the supported_attrs attribute first. Do your homework, babe 📚

8.2.1. xattr_support

xattr_support is set to True if the object's file system supports extended attributes ✨

Since xattr_support is not a REQUIRED attribute, the server need not support it. However, a client may reasonably assume that a server (or file system) that does not support the xattr_support attribute does not provide xattr support, and it acts on that basis. Logic, iykyk 🧠

Note that the protocol does not enforce any limits on the number of keys, the length of a key, the size of a value, or the total size of xattrs that are allowed for a file. The server file system MAY impose additional limits. In addition, a single xattr key or value exchanged between the client and server for get/set operations is limited by the channel's negotiated maximum size for requests and responses. So no hard caps from the protocol, but the pipe has a size limit 🚰

8.3. New Error Definitions

Here are the error codes you need to know 🚨

/// /* Following lines are to be added to enum nfsstat4 */
/// /*
///  NFS4ERR_NOXATTR        = 10095, /* xattr does not exist    */
///  NFS4ERR_XATTR2BIG      = 10096  /* xattr value is too big  */
/// */

8.3.1. NFS4ERR_NOXATTR (Error Code 10095)

The specified xattr does not exist or the server is unable to retrieve it. Not there, or can't grab it. Either way, it's a no 🙅‍♀️

8.3.2. NFS4ERR_XATTR2BIG (Error Code 10096)

The size of the xattr value specified as part of a SETXATTR operation, or the collective size of all xattrs of the file resulting from the SETXATTR operation, is bigger than that supported by the underlying file system. It's giving "too much, too soon" 😭

8.4. New Operations

Applications need to perform the following operations on a given file's extended attributes [Love]:

  • Given a file, return a list of all of the file's assigned extended attribute keys.
  • Given a file and a key, return the corresponding value.
  • Given a file, a key, and a value, assign that value to the key.
  • Given a file and a key, remove that extended attribute from the file.

In order to meet these requirements, this section introduces four new OPTIONAL operations: GETXATTR, SETXATTR, LISTXATTRS and REMOVEXATTR 🦋 They query, set, list, and remove xattrs, respectively. A server MUST support all four operations when they are directed to a file system that supports the xattr_support attribute and returns TRUE when it is interrogated. For file systems that either do not support the xattr_support attribute or return FALSE when the xattr_support attribute is interrogated, all of the above operations MUST NOT be supported. So it's all or nothing, no in-between vibes 💅 GETXATTR gets a key's value, SETXATTR creates or replaces a key with a value, LISTXATTRS lists all the xattr names, and REMOVEXATTR deletes a single xattr.

Note that some server implementations may not be aware of the existence of these operations, so a client can't always expect that issuing one of them will either succeed or return NFS4ERR_NOTSUPP. In some cases, NFS4ERR_OP_ILLEGAL may be returned, or the request may run into an XDR decode error on the server. As a result, clients should only issue these operations after determining that support is present. Check the vibe before you send the request 🫶

8.4.1. GETXATTR - Get an Extended Attribute of a File

8.4.1.1. ARGUMENTS

/// struct GETXATTR4args {
///         /* CURRENT_FH: file */
///         xattrkey4     gxa_name;
/// };

8.4.1.2. RESULTS

/// union GETXATTR4res switch (nfsstat4 gxr_status) {
///  case NFS4_OK:
///         xattrvalue4   gxr_value;
///  default:
///         void;
/// };

8.4.1.3. DESCRIPTION

The GETXATTR operation will obtain the value for the given extended attribute key for the file system object specified by the current filehandle. Main character energy, it's literally fetching the value you asked for 🌸

The server will fetch the xattr value for the key that the client requests if xattrs are supported by the server for the target file system. If the server does not support xattrs on the target file system, then it MUST NOT return a value and MUST return the NFS4ERR_NOTSUPP error or another error indicating the request was not understood. The server also MUST return NFS4ERR_NOXATTR if it supports xattrs on the target but cannot obtain the requested data. If the xattr value contained in the server response is such as to cause the channel's negotiated maximum response size to be exceeded, then the server MUST return NFS4ERR_REP_TOO_BIG in gxr_status. Too big for the pipe, so it gets bounced 🚫

8.4.1.4. IMPLEMENTATION

Clients that have cached an xattr may avoid the need to do a GETXATTR by determining if the change attribute is the same as it was when the xattr was fetched. If the client does not hold a delegation for the file in question, it can obtain the change attribute with a GETATTR request and compare that change attribute's value to the change attribute value fetched when the xattr value was obtained. This handling is similar to how a client would revalidate other file attributes such as ACLs. Cache smart, not hard 🧚

When responding to such a GETATTR, the server will, if there is an OPEN_DELEGATE_WRITE delegation held by another client for the file in question, either obtain the actual current value of these attributes from the client holding the delegation by using the CB_GETATTR callback or revoke the delegation. See Section 18.7.4 of [RFC5661] for details. Delegations are a whole thing, so go read that section 🤓

8.4.2. SETXATTR - Set an Extended Attribute of a File

8.4.2.1. ARGUMENTS

/// enum setxattr_option4 {
///         SETXATTR4_EITHER      = 0,
///         SETXATTR4_CREATE      = 1,
///         SETXATTR4_REPLACE     = 2
/// };

/// struct SETXATTR4args {
///         /* CURRENT_FH: file */
///         setxattr_option4 sxa_option;
///         xattrkey4        sxa_key;
///         xattrvalue4      sxa_value;
/// };

8.4.2.2. RESULTS

/// union SETXATTR4res switch (nfsstat4 sxr_status) {
///  case NFS4_OK:
///         change_info4      sxr_info;
///  default:
///         void;
/// };

8.4.2.3. DESCRIPTION

The SETXATTR operation changes one extended attribute of a file system object. The change you want is specified by sxa_option. SETXATTR4_CREATE is used to associate the given value with the given extended attribute key for the file system object specified by the current filehandle. The server MUST return NFS4ERR_EXIST if the attribute key already exists. SETXATTR4_REPLACE is also used to set an xattr, but the server MUST return NFS4ERR_NOXATTR if the attribute key does not exist. By default (SETXATTR4_EITHER), the extended attribute will be created if need be, or its value will be replaced if the attribute exists. Three modes, one job 💖

If the xattr key and value contained in the client request are such that the request would exceed the channel's negotiated maximum request size, then the server MUST return NFS4ERR_REQ_TOO_BIG in sxr_status. If the server file system imposes additional limits on the size of the key name or value, it MAY return NFS4ERR_XATTR2BIG. Oversized requests get the boot 🥾

A successful SETXATTR MUST change the file time_metadata and change attributes if the xattr is created or the value assigned to xattr changes. However, it is not necessary to change these attributes if there has been no actual change in the xattr value. Avoiding attribute change in such situations is desirable as it avoids unnecessary cache invalidation. No pointless changes, no pointless cache drama 🙌

On success, the server returns the change_info4 information in sxr_info. With the atomic field of the change_info4 data type, the server will indicate if the before and after change attributes were obtained atomically with respect to the SETXATTR operation. This allows the client to determine if its cached xattrs are still valid after the operation. See Section 8.7 for a discussion on xattr caching. The atomic field is the tea on whether your cache is still good ☕

8.4.2.4. IMPLEMENTATION

If the object whose xattr is being changed has a file delegation that is held by a client other than the one doing the SETXATTR, the delegation(s) must be recalled, and the operation cannot proceed to actually change the xattr until each such delegation is returned or revoked. In all cases in which delegations are recalled, the server is likely to return one or more NFS4ERR_DELAY errors while the delegation(s) remains outstanding, although it might not do that if the delegations are returned quickly. Somebody's delegation is in the way, so hold tight, it's a wait 🕰️

8.4.3. LISTXATTRS - List Extended Attributes of a File

8.4.3.1. ARGUMENTS

/// struct LISTXATTRS4args {
///         /* CURRENT_FH: file */
///         nfs_cookie4    lxa_cookie;
///         count4         lxa_maxcount;
/// };

8.4.3.2. RESULTS

/// struct LISTXATTRS4resok {
///         nfs_cookie4    lxr_cookie;
///         xattrkey4      lxr_names<>;
///         bool           lxr_eof;
/// };

/// union LISTXATTRS4res switch (nfsstat4 lxr_status) {
///  case NFS4_OK:
///         LISTXATTRS4resok  lxr_value;
///  default:
///         void;
/// };

8.4.3.3. DESCRIPTION 💖

okay so LISTXATTRS grabs a chunk of extended attribute keys off whatever file system object the current filehandle points at 🦋, and it also hands back what the client needs to ask for MORE keys in a follow-up call ✨

the arguments carry a cookie that says where the listing should start in the xattr list 🎀. setting lxa_cookie to 0 (zero) means "start at the very beginning" 🌸. for every follow-up LISTXATTRS, the client passes back a cookie the server gave it on an earlier LISTXATTRS request, so it can keep going from where it left off 💅

the lxa_maxcount value is the max number of bytes allowed for the result 📦. that limit covers everything inside the LISTXATTRS4resok structure, XDR overhead included 🤓. the server may return less data than that, totally its call 💁‍♀️. and if it can't fit even ONE xattr name inside the maxcount limit, it returns NFS4ERR_TOOSMALL to the client 😭

on success, the server sends back a list of extended attribute keys 🌈. the lxr_eof flag is TRUE when there are no more keys left for the object, so the client knows it's done bestie 🎉

the cookie is only meaningful to the server 🔮. it works like a "bookmark" for the xattr key, and the client uses it on later LISTXATTRS calls so it can keep listing keys 📖. it's kinda like a READDIR cookie or the READ offset, but the client should not interpret it as either of those 🙅‍♀️

and one more thing: on success, the current filehandle keeps its value 🔥

8.4.3.4. IMPLEMENTATION ⭐

the way a cookie gets handled is similar to how READDIR does it 🧚. it should be a rare occurrence for a server to be unable to keep listing xattrs properly with the cookie it was given 🙈. the server should try super hard to avoid that situation, because the app on the client side may not be able to handle that kind of failure at all 😬

8.4.4. REMOVEXATTR - Remove an Extended Attribute of a File 🗑️

8.4.4.1. ARGUMENTS 🍓

/// struct REMOVEXATTR4args {
///         /* CURRENT_FH: file */
///         xattrkey4      rxa_name;
/// };

8.4.4.2. RESULTS 🌸

/// union REMOVEXATTR4res switch (nfsstat4 rxr_status) {
///  case NFS4_OK:
///         change_info4      rxr_info;
///  default:
///         void;
/// };

8.4.4.3. DESCRIPTION 💜

REMOVEXATTR deletes one extended attribute of a file system object, the one named by rxa_name 🗑️. the server MUST return NFS4ERR_NOXATTR if that attribute key doesn't exist 🚫

a successful REMOVEXATTR MUST change the file time_metadata and change attributes ⏰

just like SETXATTR, the server reports the value of the change attribute right before AND right after a successful REMOVEXATTR, both inside rxr_info 🎀. that way the client can tell whether its cached xattrs are still valid after the operation 🧠. see Section 8.7 for the full talk on xattr caching 💅

8.4.4.4. IMPLEMENTATION 🔥

if the object whose xattr is being removed has a file delegation held by a client that isn't the one doing the REMOVEXATTR, the delegation(s) must be recalled 🚨. the operation can't delete the xattr until each of those delegations has been returned or revoked 😤

in all the cases where delegations get recalled, the server is likely to return one or more NFS4ERR_DELAY errors while the delegation(s) is still outstanding 🙃. it might not do that if the delegations come back quickly though, so no guarantees either way 🤷‍♀️

8.4.5. Valid Errors 📋

this section has a table of the valid error returns for each new protocol operation 💖. the error code NFS4_OK (no error) isn't listed, but every new operation can return it, so it's implied 🫶. the error values for all the other operations are defined in Section 13.2 of [RFC7530] and Section 11.2 of [RFC7862] 📚

+-------------+-----------------------------------------------------+
| Operation   | Errors                                              |
+-------------+-----------------------------------------------------+
| GETXATTR    | NFS4ERR_ACCESS, NFS4ERR_BADXDR,                     |
|             | NFS4ERR_DEADSESSION, NFS4ERR_DELAY,                 |
|             | NFS4ERR_FHEXPIRED, NFS4ERR_INVAL, NFS4ERR_IO,       |
|             | NFS4ERR_MOVED, NFS4ERR_NAMETOOLONG,                 |
|             | NFS4ERR_NOFILEHANDLE, NFS4ERR_NOTSUPP,              |
|             | NFS4ERR_NOXATTR, NFS4ERR_OP_NOT_IN_SESSION,         |
|             | NFS4ERR_PERM, NFS4ERR_REP_TOO_BIG,                  |
|             | NFS4ERR_REP_TOO_BIG_TO_CACHE, NFS4ERR_REQ_TOO_BIG,  |
|             | NFS4ERR_RETRY_UNCACHED_REP, NFS4ERR_SERVERFAULT,    |
|             | NFS4ERR_STALE, NFS4ERR_TOO_MANY_OPS,                |
|             | NFS4ERR_WRONG_TYPE                                  |
| SETXATTR    | NFS4ERR_ACCESS, NFS4ERR_BADCHAR, NFS4ERR_BADXDR,    |
|             | NFS4ERR_DEADSESSION, NFS4ERR_DELAY, NFS4ERR_DQUOT,  |
|             | NFS4ERR_EXIST, NFS4ERR_FHEXPIRED, NFS4ERR_INVAL,    |
|             | NFS4ERR_IO, NFS4ERR_MOVED, NFS4ERR_NAMETOOLONG,     |
|             | NFS4ERR_NOFILEHANDLE, NFS4ERR_NOSPC,                |
|             | NFS4ERR_NOXATTR, NFS4ERR_OP_NOT_IN_SESSION,         |
|             | NFS4ERR_PERM, NFS4ERR_REP_TOO_BIG,                  |
|             | NFS4ERR_REP_TOO_BIG_TO_CACHE, NFS4ERR_REQ_TOO_BIG,  |
|             | NFS4ERR_RETRY_UNCACHED_REP, NFS4ERR_ROFS,           |
|             | NFS4ERR_SERVERFAULT, NFS4ERR_STALE,                 |
|             | NFS4ERR_TOO_MANY_OPS, NFS4ERR_WRONG_TYPE,           |
|             | NFS4ERR_XATTR2BIG                                   |
| LISTXATTRS  | NFS4ERR_ACCESS, NFS4ERR_DEADSESSION, NFS4ERR_DELAY, |
|             | NFS4ERR_INVAL, NFS4ERR_IO, NFS4ERR_MOVED,           |
|             | NFS4ERR_NAMETOOLONG, NFS4ERR_NOFILEHANDLE,          |
|             | NFS4ERR_NOTSUPP, NFS4ERR_NOXATTR,                   |
|             | NFS4ERR_OP_NOT_IN_SESSION, NFS4ERR_PERM,            |
|             | NFS4ERR_REP_TOO_BIG, NFS4ERR_REP_TOO_BIG_TO_CACHE,  |
|             | NFS4ERR_REQ_TOO_BIG, NFS4ERR_RETRY_UNCACHED_REP,    |
|             | NFS4ERR_SERVERFAULT, NFS4ERR_STALE,                 |
|             | NFS4ERR_TOO_MANY_OPS, NFS4ERR_WRONG_TYPE            |
| REMOVEXATTR | NFS4ERR_ACCESS, NFS4ERR_BADCHAR, NFS4ERR_BADXDR,    |
|             | NFS4ERR_DEADSESSION, NFS4ERR_DELAY, NFS4ERR_DQUOT,  |
|             | NFS4ERR_EXIST, NFS4ERR_INVAL, NFS4ERR_IO,           |
|             | NFS4ERR_LOCKED, NFS4ERR_MOVED, NFS4ERR_NAMETOOLONG, |
|             | NFS4ERR_NOFILEHANDLE, NFS4ERR_NOSPC,                |
|             | NFS4ERR_NOXATTR,, NFS4ERR_OLD_STATEID,              |
|             | NFS4ERR_OPENMODE, NFS4ERR_OP_NOT_IN_SESSION,        |
|             | NFS4ERR_PERM, NFS4ERR_RETRY_UNCACHED_REP,           |
|             | NFS4ERR_ROFS, NFS4ERR_SERVERFAULT, NFS4ERR_STALE,   |
|             | NFS4ERR_TOO_MANY_OPS, NFS4ERR_WRONG_TYPE            |
+-------------+-----------------------------------------------------+

            Valid Error Returns for Each New Protocol Operation

8.5. Modifications to Existing Operations 🔐

to get fine-grained access control over querying or modifying extended attributes, there are new access rights that can be checked to see if the client is allowed to do the xattr operation ✨

heads up: in general, as explained in Section 18.1.4 of [RFC5661], a client can't reliably do an access check using only the current file attributes, so it must verify access with the server 👀

this section extends the ACCESS semantics documented in Section 18.1 of [RFC5661] 🧁. three new access permissions can be requested:

ACCESS4_XAREAD     Query a file or directory for its xattr value
                   given a key.

ACCESS4_XAWRITE    Modify xattr keys and/or values of a file or
                   directory.

ACCESS4_XALIST     Query a file or directory to list its xattr keys.

just like the existing access permissions, the results of ACCESS are advisory, with no promise that access will be allowed or denied in the future 🤞

the rules for client and server go like this:

  • if the client is sending ACCESS to find out whether the user can read an xattr of the file with GETXATTR, the client should set ACCESS4_XAREAD in the request's access field 📖
  • if the client is sending ACCESS to find out whether the user can modify an xattr of the file with SETXATTR or REMOVEXATTR, the client should set ACCESS4_XAWRITE in the request's access field ✏️
  • if the client is sending ACCESS to find out whether the user can list the xattr keys of the file with LISTXATTRS, the client should set ACCESS4_XALIST in the request's access field 📝

8.6. Numeric Values Assigned to Protocol Extensions 🔢

this section lists the numeric values assigned to the new attributes and operations that implement the xattr feature 🎀. to avoid inconsistent assignments, these were checked against the most recent protocol version [RFC5661] and the current minor version [RFC7862] 💅. you can build interoperable prototypes with these values 🛠️

/// /*
///  * ACCESS - Check Access Rights
///  */
/// const ACCESS4_XAREAD    = 0x00000040;
/// const ACCESS4_XAWRITE   = 0x00000080;
/// const ACCESS4_XALIST    = 0x00000100;

/// /*
///  * New NFSv4 attribute
///  */
/// typedef bool            fattr4_xattr_support;

/// /*
///  * New RECOMMENDED Attribute
///  */
/// const FATTR4_XATTR_SUPPORT = 82;

/// /*
///  * New NFSv4 operations
///  */
/// /* Following lines are to be added to enum nfs_opnum4 */
/// /*
/// OP_GETXATTR                = 72,
/// OP_SETXATTR                = 73,
/// OP_LISTXATTRS              = 74,
/// OP_REMOVEXATTR             = 75,
/// */

/// /*
///  * New cases for Operation arrays
///  */
/// /* Following lines are to be added to nfs_argop4 */
/// /*
/// case OP_GETXATTR:      GETXATTR4args opgetxattr;
/// case OP_SETXATTR:      SETXATTR4args opsetxattr;
/// case OP_LISTXATTRS:    LISTXATTRS4args oplistxattrs;
/// case OP_REMOVEXATTR:   REMOVEXATTR4args opremovexattr;
/// */

/// /* Following lines are to be added to nfs_resop4 */
/// /*
/// case OP_GETXATTR:      GETXATTR4res opgetxattr;
/// case OP_SETXATTR:      SETXATTR4res opsetxattr;
/// case OP_LISTXATTRS:    LISTXATTRS4res oplistxattrs;
/// case OP_REMOVEXATTR:   REMOVEXATTR4res opremovexattr;
/// */

8.7. Caching ✨

okay so caching for xattrs is lowkey giving the same energy as other file attributes like ACLs 💅 Whether a client gets to cache them depends on whether the server has granted it an OPEN delegation 🦋

Xattrs that come back from the server, or get sent to it, can be cached 🧁 Clients can use those cached copies to skip future GETXATTR requests, but only if they can make sure nobody else has changed the cached value 👀 That assurance can come from holding a delegation for the file, or from checking the change attribute to see if the cached value is still good. The caching can be read-only or write-through 🌸

When a delegation is in effect, some operations by a second client to a delegated file will make the server recall the delegation through a callback 📞 For each individual operation, IMPLEMENTATION spells out when those operations are required to trigger a recall.

Here's the tea 🫖 Because of local caching, the individual xattrs on clients might not be up to date. Changes made in one order on the server might show up in a different order on one client and a third order on another client 😭 To cut down on problems from grabbing individual xattrs separately from other file attributes, a client should treat xattrs just like other file attributes when it comes to caching, as detailed in Section 10.6 of [RFC7530] 📚 A client can validate its cached copy of an xattr for a file by fetching the change attribute. If the change attribute still has the same value it had when the attributes were cached, the client can assume the xattrs haven't changed. And if the client holds a delegation that makes sure no other client can modify the change attribute, it can skip actually checking the change attribute at all 🙌

When a client changes xattrs on a file, it needs to figure out whether other clients have changed the file too 🕵️‍♀️ It does this by using the change attribute as reported before and after the change operation (SETXATTR or REMOVEXATTR), which comes back in the associated change_info4 value for that operation. The server can tell the client whether the change_info4 data was provided atomically with respect to the change operation. When the change values are atomic, the client has a basis for figuring out, with proper care, whether other clients are modifying the file 🧐

A simple way for the client to make that call is to serialize all xattr changes made to a specific file 🎀 When it does this, and the server gives before and after values of the change attribute atomically, the client can just compare the after value from one operation with the before value from the next change operation on that file. If they match, the client knows for sure that no other client is modifying the file 💖

If the comparison shows another client updated the file, the client purges the xattr cache associated with that file 🗑️ If the comparison shows no modification, the client can update its xattr cache to reflect the file operation and extend the associated timeout ⏰ The post-operation change value needs to be saved as the basis for future change_info4 comparisons 💾

Xattr caching requires the client to revalidate its xattr cache data by inspecting the change attribute of a file at the point when an xattr was cached 🔍 That means the server has to update the change attribute whenever xattrs get modified 🛠️ For a client to use the change_info4 information properly and correctly, the server must report the pre- and post-operation change attribute values atomically 🫶 When the server can't report the before and after values atomically with respect to the xattr update operation, it must say so in the change_info4 return value 🚩 If the information isn't reported atomically, the client should not assume that other clients haven't changed the xattrs 😬

The protocol doesn't support write-back caching of xattrs at all 🙅‍♀️ So every modification to an xattr should be done through requests to the server, and the server should perform those updates synchronously ⚡

8.8. Xattrs and File Locking 🔐

Xattr operations mostly work independently of operations related to file locking state 🦋 For example, xattrs can be interrogated and modified without a corresponding OPEN operation 💅 The server doesn't need to check for locks that conflict with xattr access or modify operations. Like, another OPEN specified with OPEN4_SHARE_DENY_READ or OPEN4_SHARE_DENY_BOTH doesn't stop anyone from accessing or modifying xattrs 🙈 Still, the server MUST verify that the client is allowed to perform the xattr operation based on access permissions. No skipping that step, bestie 🛡️

Delegations can change how xattr operations interact with the state-related logic though 🎭 Xattrs can't be modified while another client holds a delegation for the corresponding file 🚫 On the flip side, xattrs can be interrogated even when another client is holding a write delegation, since updates are write-through to the server 🌈

8.9. pNFS Considerations 🧚

All xattr operations get sent to the metadata server 🏰 That server is responsible for fetching data from persistent storage and making any necessary changes to it 💾 It's the one in charge of the whole vibe ✨

9. Security Considerations 💀

Okay, this section is serious business, so listen up 👂 Since xattrs are application data, the security issues are exactly the same as the ones for storing file data and named attributes 🔒 Clients MUST NOT accord any system-interpreted semantics to xattrs 🚨 Their use is restricted to user-managed metadata only, as explained in Section 5 🙅‍♀️ Extended attributes are various sorts of application data, and the fact that the way you reference them is a little different in each case shouldn't be treated as security relevant 🙄 So the additions to the NFS protocol for supporting extended attributes don't change the security considerations of the NFSv4 protocol [RFC7530] 🫡 Stay safe out there, bestie 🥹

10. IANA Considerations 📋

Adding xattr support to the NFSv4 protocol doesn't require any actions from IANA 🎉 This document limits xattr names to the user namespace, where application developers are free to define and use attributes as needed 🌷 Unlike named attributes, there's no namespace identifier tied to xattrs that would need registering 💅

11. References 📚

11.1. Normative References 🧾

11.2. Informative References 🌟

Acknowledgments 🫶

This document tried to capture the discussion on adding xattrs to the NFSv4 protocol from lots of participants on the IETF NFSv4 mailing list 💌 The people who gave valuable input and comments on draft versions of this document include: Tom Haynes, Christoph Hellwig, Nico Williams, Dave Noveck, Benny Halevy, and Andreas Gruenbacher. Iconic squad 👑

Authors' Addresses 🖊️

Manoj Naik
Nutanix
1740 Technology Drive, Suite 150
San Jose, CA 95110
United States of America

Email: manoj.naik@nutanix.com


Marc Eshel
IBM Almaden
650 Harry Road
San Jose, CA 95120
United States of America

Phone: +1 408-927-1894
Email: eshel@us.ibm.com
📦 where this page comes from