RFC 8620

The JSON Meta Application Protocol (JMAP)

📅 published July 2019, from ietf

RFC 8620: The JSON Meta Application Protocol (JMAP) 💖

Internet Engineering Task Force (IETF) 🌸 Request for Comments: 8620 Authors: N. Jenkins (Fastmail) and C. Newman (Oracle) 🫶 Category: Standards Track 📜 ISSN: 2070-1721 Date: July 2019 Obsoletes/Updates: none listed in the title block 🙈

Abstract

okay so this doc is about a protocol that lets clients query, fetch, and modify JSON data objects super efficiently ✨ It also has push notifications for when stuff changes, fast resynchronisation, and out-of-band uploads and downloads for binary data 🦋

Status of This Memo

This is an Internet Standards Track document. 📜

It's a product of the Internet Engineering Task Force (IETF), so it represents the consensus of the IETF community 👑 It got public review and was approved for publication by the Internet Engineering Steering Group (IESG). If you want more info on Internet Standards, check out Section 2 of RFC 7841 🎀

For the current status, any errata, and how to send feedback, head to https://www.rfc-editor.org/info/rfc8620 🔗

Copyright (c) 2019 IETF Trust and the persons identified as the document authors. All rights reserved 💜

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as they stand on the publication date. Please actually read these, bestie, because they spell out your rights and restrictions for this document 🙈 Any Code Components pulled out of this document must include the Simplified BSD License text, as described in Section 4.e of the Trust Legal Provisions, and they come with no warranty, as described in the Simplified BSD License ⚠️

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   4
     1.1.  Notational Conventions  . . . . . . . . . . . . . . . . .   4
     1.2.  The Id Data Type  . . . . . . . . . . . . . . . . . . . .   6
     1.3.  The Int and UnsignedInt Data Types  . . . . . . . . . . .   6
     1.4.  The Date and UTCDate Data Types . . . . . . . . . . . . .   7
     1.5.  JSON as the Data Encoding Format  . . . . . . . . . . . .   7
     1.6.  Terminology . . . . . . . . . . . . . . . . . . . . . . .   7
       1.6.1.  User  . . . . . . . . . . . . . . . . . . . . . . . .   7
       1.6.2.  Accounts  . . . . . . . . . . . . . . . . . . . . . .   7
       1.6.3.  Data Types and Records  . . . . . . . . . . . . . . .   8
     1.7.  The JMAP API Model  . . . . . . . . . . . . . . . . . . .   8
     1.8.  Vendor-Specific Extensions  . . . . . . . . . . . . . . .   9
   2.  The JMAP Session Resource . . . . . . . . . . . . . . . . . .   9
     2.1.  Example . . . . . . . . . . . . . . . . . . . . . . . . .  14
     2.2.  Service Autodiscovery . . . . . . . . . . . . . . . . . .  15
   3.  Structured Data Exchange  . . . . . . . . . . . . . . . . . .  16
     3.1.  Making an API Request . . . . . . . . . . . . . . . . . .  16
     3.2.  The Invocation Data Type  . . . . . . . . . . . . . . . .  16
     3.3.  The Request Object  . . . . . . . . . . . . . . . . . . .  16
       3.3.1.  Example Request . . . . . . . . . . . . . . . . . . .  18
     3.4.  The Response Object . . . . . . . . . . . . . . . . . . .  18
       3.4.1.  Example Response  . . . . . . . . . . . . . . . . . .  19
     3.5.  Omitting Arguments  . . . . . . . . . . . . . . . . . . .  19
     3.6.  Errors  . . . . . . . . . . . . . . . . . . . . . . . . .  19
       3.6.1.  Request-Level Errors  . . . . . . . . . . . . . . . .  20
       3.6.2.  Method-Level Errors . . . . . . . . . . . . . . . . .  21
     3.7.  References to Previous Method Results . . . . . . . . . .  22
     3.8.  Localisation of User-Visible Strings  . . . . . . . . . .  27
     3.9.  Security  . . . . . . . . . . . . . . . . . . . . . . . .  28
     3.10. Concurrency . . . . . . . . . . . . . . . . . . . . . . .  28
   4.  The Core/echo Method  . . . . . . . . . . . . . . . . . . . .  28
     4.1.  Example . . . . . . . . . . . . . . . . . . . . . . . . .  28
   5.  Standard Methods and Naming Convention  . . . . . . . . . . .  29
     5.1.  /get  . . . . . . . . . . . . . . . . . . . . . . . . . .  29
     5.2.  /changes  . . . . . . . . . . . . . . . . . . . . . . . .  30
     5.3.  /set  . . . . . . . . . . . . . . . . . . . . . . . . . .  34
     5.4.  /copy . . . . . . . . . . . . . . . . . . . . . . . . . .  40
     5.5.  /query  . . . . . . . . . . . . . . . . . . . . . . . . .  42
     5.6.  /queryChanges . . . . . . . . . . . . . . . . . . . . . .  48
     5.7.  Examples  . . . . . . . . . . . . . . . . . . . . . . . .  51
     5.8.  Proxy Considerations  . . . . . . . . . . . . . . . . . .  58
   6.  Binary Data . . . . . . . . . . . . . . . . . . . . . . . . .  58
     6.1.  Uploading Binary Data . . . . . . . . . . . . . . . . . .  59
     6.2.  Downloading Binary Data . . . . . . . . . . . . . . . . .  60
     6.3.  Blob/copy . . . . . . . . . . . . . . . . . . . . . . . .  61

   7.  Push  . . . . . . . . . . . . . . . . . . . . . . . . . . . .  62
     7.1.  The StateChange Object  . . . . . . . . . . . . . . . . .  63
       7.1.1.  Example . . . . . . . . . . . . . . . . . . . . . . .  64
     7.2.  PushSubscription  . . . . . . . . . . . . . . . . . . . .  64
       7.2.1.  PushSubscription/get  . . . . . . . . . . . . . . . .  67
       7.2.2.  PushSubscription/set  . . . . . . . . . . . . . . . .  68
       7.2.3.  Example . . . . . . . . . . . . . . . . . . . . . . .  69
     7.3.  Event Source  . . . . . . . . . . . . . . . . . . . . . .  71
   8.  Security Considerations . . . . . . . . . . . . . . . . . . .  73
     8.1.  Transport Confidentiality . . . . . . . . . . . . . . . .  73
     8.2.  Authentication Scheme . . . . . . . . . . . . . . . . . .  73
     8.3.  Service Autodiscovery . . . . . . . . . . . . . . . . . .  73
     8.4.  JSON Parsing  . . . . . . . . . . . . . . . . . . . . . .  74
     8.5.  Denial of Service . . . . . . . . . . . . . . . . . . . .  74
     8.6.  Connection to Unknown Push Server . . . . . . . . . . . .  74
     8.7.  Push Encryption . . . . . . . . . . . . . . . . . . . . .  75
     8.8.  Traffic Analysis  . . . . . . . . . . . . . . . . . . . .  76
   9.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  76
     9.1.  Assignment of jmap Service Name . . . . . . . . . . . . .  76
     9.2.  Registration of Well-Known URI Suffix for JMAP  . . . . .  76
     9.3.  Registration of the jmap URN Sub-namespace  . . . . . . .  77
     9.4.  Creation of "JMAP Capabilities" Registry  . . . . . . . .  77
       9.4.1.  Preliminary Community Review  . . . . . . . . . . . .  77
       9.4.2.  Submit Request to IANA  . . . . . . . . . . . . . . .  78
       9.4.3.  Designated Expert Review  . . . . . . . . . . . . . .  78
       9.4.4.  Change Procedures . . . . . . . . . . . . . . . . . .  78
       9.4.5.  JMAP Capabilities Registry Template . . . . . . . . .  79
       9.4.6.  Initial Registration for JMAP Core  . . . . . . . . .  79
       9.4.7.  Registration for JMAP Error Placeholder in JMAP
               Capabilities Registry . . . . . . . . . . . . . . . .  80
     9.5.  Creation of "JMAP Error Codes" Registry . . . . . . . . .  80
       9.5.1.  Expert Review . . . . . . . . . . . . . . . . . . . .  80
       9.5.2.  JMAP Error Codes Registry Template  . . . . . . . . .  81
       9.5.3.  Initial Contents for the JMAP Error Codes Registry  .  81
   10. References  . . . . . . . . . . . . . . . . . . . . . . . . .  86
     10.1.  Normative References . . . . . . . . . . . . . . . . . .  86
     10.2.  Informative References . . . . . . . . . . . . . . . . .  89
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  90

1. Introduction ✨

okay so JMAP (the JSON Meta Application Protocol) is what clients and servers use to sync data like mail, calendars, or contacts 📬 It's optimised for mobile and web, and the big goal is one consistent interface across different data types 🌈

This spec is only for the generic sync mechanism 🧚 Other specs come along later and define the data models for each data type you can sync with JMAP.

JMAP is built to be really efficient with limited network resources 💅 You can batch multiple API calls into one request to the server, which cuts down on round trips and saves battery on phones 🔋 Push connections mean no more polling, and the efficient delta updates make sure only a minimum amount of data gets transferred ⭐

JMAP is also designed to scale horizontally to a massive number of users 👑 How? Separate endpoints for users after login, binary and structured data kept apart, and a sharing model that doesn't allow data dependencies between accounts 🦋

1.1. Notational Conventions 📜

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.

The format underneath all of this is JSON 🍓 So the terms "object" and "array", plus the four primitive types (strings, numbers, booleans, and null), are read the way Section 1 of [RFC8259] describes them. Unless noted otherwise, all property names and values are case sensitive, so be careful, bestie 🙈

Some examples are "partial" JSON documents, just for illustration. In those, three periods "..." mean a chunk of the document has been cut out to keep things short ✂️

For publishing reasons, line breaks have been added inside long JSON strings, and the continuation lines are indented. To turn an example back into valid JSON, any line break inside a string must be replaced with a space, and any other white space after the line break must be removed 💜

Unless otherwise specified, API exchange examples only show the methodCalls array of the Request object or the methodResponses array of the Response object. The rest of the Request/Response object is left out to keep things compact 🧚

Every JSON value gets a type signature in this document. Here's the cheat sheet 💖

  • "*" - The type is undefined. The value could be any type, though the context may limit what's allowed.
  • "String" - The JSON string type.
  • "Number" - The JSON number type.
  • "Boolean" - The JSON boolean type.
  • "A[B]" - A JSON object where all the keys are type "A" and all the values are type "B".
  • "A[]" - An array of values of type "A".
  • "A|B" - The value is either type "A" or type "B".

Other types can show up too, and their representation is defined somewhere else in this document 🎀

Object properties can also come with attributes next to their type signature. Here's what they mean 🫶

  • "server-set" - Only the server can set the value for this property. The client MUST NOT send this property when creating a new object of this type. No sending it, periodt! 🚫
  • "immutable" - The value MUST NOT change after the object is created. 🔒
  • "default" - (This is followed by a JSON value.) This is the value that gets used for this property if it's left out of an argument or when a new object of this type is created 🌸

Part 1 of 17 is done, bestie! 💜 Part 2 is coming up next, and it's going to be iconic 🔥

1.2. The Id Data Type 🎀

okay so every record id is assigned by the server, and once it's given, it's locked in forever (immutable) 🔒✨

When the spec says "Id" as a data type, it means a "String" that's at least 1 and at most 255 octets long. It MUST only use characters from the "URL and Filename Safe" base64 alphabet (see Section 5 of [RFC4648]), and the pad character (=) is NOT allowed. So you're only working with ASCII letters and numbers (A-Za-z0-9), plus hyphen (-) and underscore (_) 💅

These characters are safe basically everywhere: filesystems, URIs, IMAP atoms, the works 🌈 But for maximum safety, servers SHOULD also use defensive allocation strategies so nothing gets weird with glob completion or data type detection (like on filesystems or in spreadsheets). Things to avoid, bestie:

  • Ids starting with a dash
  • Ids starting with digits
  • Ids that contain only digits
  • Ids that differ only by ASCII case (like A vs. a)
  • the specific sequence of three characters "NIL" (because it can get mixed up with the IMAP way of writing the null value) 🙈

A good fix for all of this? Just prefix every id with a single alphabetical character. Easy, iconic 🧚

1.3. The Int and UnsignedInt Data Types 🔢

When the spec says "Int", it means an integer in the range -2^53+1 <= value <= 2^53-1. That's the safe range for integers stored in a floating-point double, and it gets represented as a JSON "Number" ✨

"UnsignedInt" is an "Int" but with extra rules: the value MUST be in the range 0 <= value <= 2^53-1. No negatives for you 🙅‍♀️

1.4. The Date and UTCDate Data Types 🗓️

When "Date" shows up as a type, it means a string in "date-time" format ([RFC3339]). To keep things normalised, the "time-secfrac" MUST always be omitted if it's zero, and any letters in the string (like "T" and "Z") MUST be uppercase. Example: "2014-10-30T14:12:00+08:00" 🍓

"UTCDate" is a "Date" where the "time-offset" component MUST be "Z", meaning it has to be in UTC time 🌸 Example: "2014-10-30T06:12:00Z"

1.5. JSON as the Data Encoding Format 📦

JSON is a text-based data interchange format, as specified in [RFC8259]. The Internet JSON (I-JSON) format from [RFC7493] is a strict subset of it, with extra restrictions to avoid confusing scenarios. For example, an object MUST NOT have two members with the same name. Strict, but make it make sense 💜

All data sent between client and server (in either direction), except binary file upload/download, MUST be valid I-JSON according to the RFC. That means it's case sensitive and encoded in UTF-8 [RFC3629] 🫶

1.6. Terminology 📖

1.6.1. User 👤

A user is a person accessing data via JMAP. A user has a set of permissions that decide what data they can see 👀

1.6.2. Accounts 🗂️

An account is a collection of data. One account can hold a bunch of different data types, like mail, contacts, and calendars all together 💖 Most JMAP methods take a mandatory accountId argument, which says which account the operations should happen on.

An account is NOT the same thing as a user, even though a user's primary account usually belongs to them directly. For example, you could have an account for a group or business that multiple users can get into 🧑‍🤝‍🧑

One set of credentials can unlock multiple accounts too. Like, if someone shares their work calendar with you, or there's a group mailbox for a support-desk inbox, you get access to both 🦋

Okay, this one is serious. If the server runs into a severe internal error, it might have to reallocate ids or do something else that breaks standard JMAP data constraints for an account. When that happens, the data on the server doesn't match what the client has cached anymore. The server MUST treat this as though the account was deleted and then recreated with a new account id. Clients will then have to throw away any data with the old account id and refetch everything from scratch 😭

1.6.3. Data Types and Records 🧩

JMAP gives you one uniform interface for creating, retrieving, updating, and deleting different types of objects ✨ A "data type" is a collection of named, typed properties, kind of like the schema for a database table. Each instance of a data type is called a "record".

The id of a record is immutable and gets assigned by the server. The id MUST be unique among all records of the same type within the same account. Ids are allowed to clash across different accounts, or for two records of different types in the same account 🌈

1.7. The JMAP API Model 🚀

JMAP uses HTTP [RFC7230] to expose API, push, upload, and download resources. All HTTP requests MUST use the https:// scheme (HTTP over TLS [RFC2818]), and All HTTP requests MUST be authenticated 🔐

Once a client is authenticated, it can fetch the user's Session object, which has details about the data and capabilities the server can provide (see Section 2). After that, the client can exchange data with the server in three main ways:

  1. The client may make an API request to get or set structured data. This request is an ordered series of method calls. The server processes them and sends back an ordered series of responses. This is covered in Sections 3, 4, and 5 💅
  2. The client may download or upload binary files from/to the server. That's in Section 6 📁
  3. The client may connect to a push channel on the server, so it gets notified when data changes. That's explained in Section 7 🔔

1.8. Vendor-Specific Extensions 🎀

Individual services are going to have custom features they want to show off over JMAP 💅 This can look like extra data types and/or methods that aren't in the spec, extra arguments to JMAP methods, or extra properties on existing data types (which can also show up in arguments to methods that take property names).

The server can advertise the custom extensions it supports by including their identifiers in the capabilities object. Identifiers for vendor extensions MUST be a URL belonging to a domain the vendor owns, so nobody collides with anyone else. The URL SHOULD resolve to documentation explaining the changes the extension makes 📚

The client MUST opt in to an extension by passing the right capability identifier in the "using" array of the Request object, as described in Section 3.3. The server MUST only follow the specifications that were opted into, and it has to act as though it doesn't implement anything else when processing a request. This keeps things compatible with clients that don't know about a specific custom extension, and with future versions of JMAP too 🦋✨

2. The JMAP Session Resource ✨

okay bestie, to connect to a JMAP server you need two things 🎀

  1. The URL for the JMAP Session resource. You can get this straight from the user, or it can be discovered automatically from a username domain (see Section 2.2 below) 🔍
  2. Credentials to authenticate with. How you get those is totally out of scope for this doc 🙈

A successful authenticated GET request to the JMAP Session resource MUST return a JSON-encoded Session object 💖. It gives details about the data and capabilities the server can offer the client, based on those credentials. Here are its properties:

  • capabilities: "String[Object]"
    • An object listing the capabilities this server has. Each key is a URI for a capability the server supports, and each value is an object with more info about how the server handles that capability ✨
    • The client MUST ignore any properties it does not understand 🫶
    • The capabilities object MUST include a property called "urn:ietf:params:jmap:core". Its value is an object that MUST contain the following info on server capabilities. Suggested minimum limits are also included so clients can use the network efficiently 🦋
      • maxSizeUpload: "UnsignedInt"
        • The maximum file size, in octets, that the server will accept for a single file upload (for any purpose). Suggested minimum: 50,000,000.
      • maxConcurrentUpload: "UnsignedInt"
        • The maximum number of concurrent requests the server will accept to the upload endpoint. Suggested minimum: 4.
      • maxSizeRequest: "UnsignedInt"
        • The maximum size, in octets, that the server will accept for a single request to the API endpoint. Suggested minimum: 10,000,000.
      • maxConcurrentRequests: "UnsignedInt"
        • The maximum number of concurrent requests the server will accept to the API endpoint. Suggested minimum: 4.
      • maxCallsInRequest: "UnsignedInt"
        • The maximum number of method calls the server will accept in a single request to the API endpoint. Suggested minimum: 16.
      • maxObjectsInGet: "UnsignedInt"
        • The maximum number of objects the client may request in a single /get type method call. Suggested minimum: 500.
      • maxObjectsInSet: "UnsignedInt"
        • The maximum number of objects the client may send to create, update, or destroy in a single /set type method call. This is the combined total. Say the max is 10: you couldn't create 7 objects and destroy 6, because that's 13 actions and it goes over the limit 🙅‍♀️. Suggested minimum: 500.
      • collationAlgorithms: "String[]"
        • A list of identifiers for algorithms registered in the collation registry, as defined in [RFC4790], that the server supports for sorting when querying records.
    • Specs for future capabilities will define their own properties on the capabilities object 🌈
    • Servers MAY advertise vendor-specific JMAP extensions, as described in Section 1.8 💅. To avoid conflicts, an identifier for a vendor-specific extension MUST be a URL with a domain owned by the vendor. Clients MUST opt in to any capability it wishes to use (see Section 3.3) 👑.
  • accounts: "Id[Account]"
    • A map of an account id to an Account object for each account (see Section 1.6.2) the user can access 🗂️. An Account object has these properties:
      • name: "String"
        • A user-friendly string to show when presenting content from this account, like the email address of the account's owner 💌.
      • isPersonal: "Boolean"
        • This is true if the account belongs to the authenticated user, rather than a group account or someone else's personal account that's been shared with them 🥰.
      • isReadOnly: "Boolean"
        • This is true if the entire account is read-only 🔒.
      • accountCapabilities: "String[Object]"
        • The set of capability URIs for the methods supported in this account. Each key is a URI for a capability that has methods you can use with this account. Each value is an object with more info on the account's permissions and restrictions for that capability, as defined in the capability's spec 📋.
        • The client MUST ignore any properties it does not understand 🙃.
        • The server advertises the full list of capabilities it supports in the capabilities object, as defined above. If a capability defines new methods, the server MUST include it in the accountCapabilities object if the user may use those methods with this account. It MUST NOT include it in the accountCapabilities object if the user cannot use those methods with this account 🚦.
        • For example, you might have your own account with mail, calendars, and contacts, plus a shared account that only has contacts (like a business address book). In that case, the accountCapabilities property on the first account would include something like "urn:ietf:params:jmap:mail", "urn:ietf:params:jmap:calendars", and "urn:ietf:params:jmap:contacts", while the second account would only have the last of these 📇.
        • Trying to use the methods of a capability with an account that doesn't support that capability gets rejected with an "accountNotSupportedByMethod" error (see "Method-Level Errors", Section 3.6.2) 🚫.
  • primaryAccounts: "String[Id]"
    • A map of capability URIs (as found in accountCapabilities) to the account id that counts as the user's main or default account for data about that capability 🏠. If none of the returned accounts belong to the user, or there's no good way to pick a default, there MAY be no entry for a particular URI, even though the server supports that capability (and it's in the capabilities object). "urn:ietf:params:jmap:core" SHOULD NOT be present 🤷‍♀️.
  • username: "String"
    • The username tied to the given credentials, or the empty string if there isn't one 🫥.
  • apiUrl: "String"
    • The URL to use for JMAP API requests 🚀.
  • downloadUrl: "String"
    • The URL endpoint to use when downloading files, in URI Template (level 1) format [RFC6570]. The URL MUST contain variables called "accountId", "blobId", "type", and "name". How these variables get used is described in Section 6.2. Because slashes in content types can cause encoding issues, it is RECOMMENDED to put the "type" variable in the query section of the URL 🧁.
  • uploadUrl: "String"
    • The URL endpoint to use when uploading files, in URI Template (level 1) format [RFC6570]. The URL MUST contain a variable called "accountId". How this variable gets used is described in Section 6.1 ⬆️.
  • eventSourceUrl: "String"
    • The URL to connect to for push events, as described in Section 7.3, in URI Template (level 1) format [RFC6570]. The URL MUST contain variables called "types", "closeafter", and "ping". How these variables get used is described in Section 7.3 📣.
  • state: "String"
    • A (preferably short) string that represents the state of this object on the server. If any other property on the Session object changes, this string changes too. The current value is also returned on the API Response object (see Section 3.4), so clients can quickly check whether the session info has changed (like if an account got added or removed) and know they need to refetch the object 🔄.

To keep things future-proof, other properties MAY be included on the Session object. Clients MUST ignore any properties they are not expecting 🧚.

Implementors must take care to avoid inappropriate caching of the Session object at the HTTP layer 👀. Since the client should only refetch when it detects a change (via the sessionState property of an API response), it is RECOMMENDED to disable HTTP caching altogether. For example, you can set Cache-Control: no-cache, no-store, must-revalidate on the response 🛑.

2.1. Example

In this example Session object, the user has access to their own mail and contacts through JMAP, plus read-only access to shared mail from another user 💌. The server is also advertising a custom "https://example.com/apis/foobar" capability 🌸.

{
  "capabilities": {
    "urn:ietf:params:jmap:core": {
      "maxSizeUpload": 50000000,
      "maxConcurrentUpload": 8,
      "maxSizeRequest": 10000000,
      "maxConcurrentRequest": 8,
      "maxCallsInRequest": 32,
      "maxObjectsInGet": 256,
      "maxObjectsInSet": 128,
      "collationAlgorithms": [
        "i;ascii-numeric",
        "i;ascii-casemap",
        "i;unicode-casemap"
      ]
    },
    "urn:ietf:params:jmap:mail": {}
    "urn:ietf:params:jmap:contacts": {},
    "https://example.com/apis/foobar": {
      "maxFoosFinangled": 42
    }
  },
  "accounts": {
    "A13824": {
      "name": "john@example.com",
      "isPersonal": true,
      "isReadOnly": false,
      "accountCapabilities": {
        "urn:ietf:params:jmap:mail": {
          "maxMailboxesPerEmail": null,
          "maxMailboxDepth": 10,
          ...
        },
        "urn:ietf:params:jmap:contacts": {
          ...
        }
      }
    },
    "A97813": {
      "name": "jane@example.com",
      "isPersonal": false,
      "isReadOnly": true,
      "accountCapabilities": {
        "urn:ietf:params:jmap:mail": {
          "maxMailboxesPerEmail": 1,
          "maxMailboxDepth": 10,
          ...
        }
      }
    }
  },
  "primaryAccounts": {
    "urn:ietf:params:jmap:mail": "A13824",
    "urn:ietf:params:jmap:contacts": "A13824"
  },
  "username": "john@example.com",
  "apiUrl": "https://jmap.example.com/api/",
  "downloadUrl": "https://jmap.example.com
    /download/{accountId}/{blobId}/{name}?accept={type}",
  "uploadUrl": "https://jmap.example.com/upload/{accountId}/",
  "eventSourceUrl": "https://jmap.example.com
    /eventsource/?types={types}&closeafter={closeafter}&ping={ping}",
  "state": "75128aab4b1b"
}

2.2. Service Autodiscovery

There are two standardised autodiscovery methods in use for Internet protocols 🕵️‍♀️:

A JMAP-supporting host for the domain example.com SHOULD publish a SRV record _jmap._tcp.example.com that gives a hostname and port (usually port 443) 📡. The JMAP Session resource is then https://${hostname}[:${port}]/.well-known/jmap (following any redirects) 🌈.

If the client has a username in the form of an email address, it MAY use the domain portion of that address to try autodiscovering the JMAP server 💜.

3. Structured Data Exchange

The client may make an API request to the server to get or set structured data 🎀. This request is made up of an ordered series of method calls. The server processes these, then sends back an ordered series of responses 💖.

3.1. Making an API Request ✨

okay so, to make an API request, the client sends an authenticated POST request to the API resource. That's the one defined by the apiUrl property in the Session object (see Section 2) 🦋

The request MUST be of type application/json and consist of a single JSON-encoded Request object, as defined in Section 3.3. If it's successful, the response MUST also be of type application/json and consist of a single Response object, as defined in Section 3.4. 💅

3.2. The Invocation Data Type 🎀

Method calls and responses are represented by the Invocation data type. It's a tuple, which in JSON is an array with three elements:

  1. A String name of the method to call or of the response.
  2. A String[*] object containing named arguments for that method or response.
  3. A String method call id: an arbitrary string from the client that gets echoed back with the responses emitted by that method call. A method may return 1 or more responses, since it may make implicit calls to other methods, and all responses initiated by one method call get the same method call id in the response. 🧚

3.3. The Request Object 🍓

A Request object has the following properties:

  • using: String[]. The set of capabilities the client wishes to use. The client MAY include capability identifiers even if the method calls it makes don't actually use those capabilities. The server advertises the set of specifications it supports in the Session object (see Section 2), as keys on the capabilities property. 🌸
  • methodCalls: Invocation[]. An array of method calls to process on the server. The method calls MUST be processed sequentially, in order. 🔥
  • createdIds: Id[Id] (optional). A map of a (client-specified) creation id to the id the server assigned when a record was successfully created.

As described later in this specification, some records may have a property that contains the id of another record. To make network usage more efficient, you can set that property to reference a record created earlier in the same API request. Since the real id isn't known yet when the request is created, the client can instead specify the creation id it assigned, prefixed with a # (see Section 5.3 for more details). 🫶

As the server processes API requests, any time it successfully creates a new record, it adds the creation id to this map (see the create argument to /set in Section 5.3), with the server-assigned real id as the value. If it comes across a reference to a creation id in a create/update, it looks it up in the map and replaces the reference with the real id, if found. 👑

The client can pass an initial value for this map as the createdIds property of the Request object. This may be an empty object. If it's given in the request, the response will also include a createdIds property. This lets proxy servers split a JMAP request into multiple JMAP requests to send to different servers. For example, a proxy could send the first two method calls to server A, then the third to server B, before sending the fourth to server A again. By passing the createdIds of the previous response to the next request, it can make sure all of these still resolve. See Section 5.8 for further discussion of proxy considerations. 🦋

Future specifications MAY add further properties to the Request object to extend the semantics. To keep things forwards compatible, a server MUST ignore any other properties it doesn't understand on the JMAP Request object. 🌈

3.3.1. Example Request ✨

{
  "using": [ "urn:ietf:params:jmap:core", "urn:ietf:params:jmap:mail" ],
  "methodCalls": [
    [ "method1", {
      "arg1": "arg1data",
      "arg2": "arg2data"
    }, "c1" ],
    [ "method2", {
      "arg1": "arg1data"
    }, "c2" ],
    [ "method3", {}, "c3" ]
  ]
}

3.4. The Response Object 🌈

A Response object has the following properties:

  • methodResponses: Invocation[]. An array of responses, in the same format as the methodCalls on the Request object. The output of the methods MUST be added to the methodResponses array in the same order that the methods are processed. 💖
  • createdIds: Id[Id] (optional; only returned if given in the request). A map of a (client-specified) creation id to the id the server assigned when a record was successfully created. This MUST include all creation ids passed in the original createdIds parameter of the Request object, as well as any additional ones added for newly created records. ✨
  • sessionState: String. The current value of the state string on the Session object, as described in Section 2. Clients may use this to detect whether that object has changed and needs to be refetched. 🔥

Unless otherwise specified, if the method call completed successfully, its response name is the same as the method name in the request. 💅

3.4.1. Example Response ✨

                   {
                     "methodResponses": [
                       [ "method1", {
                         "arg1": 3,
                         "arg2": "foo"
                       }, "c1" ],
                       [ "method2", {
                         "isBlah": true
                       }, "c2" ],
                       [ "anotherResponseFromMethod2", {
                         "data": 10,
                         "yetmoredata": "Hello"
                       }, "c2"],
                       [ "error", {
                         "type":"unknownMethod"
                       }, "c3" ]
                     ],
                     "sessionState": "75128aab4b1b"
                   }

3.5. Omitting Arguments 🙈

An argument to a method may be specified to have a default value. If the client leaves it out, the server MUST treat the method call the same as if the default value had been specified. Similarly, the server MAY omit any argument in a response that has the default value. 🍓

Unless a method description says otherwise, null is the default value for any argument in a request or response where the type signature allows it. Other arguments may only be omitted if the method description defines an explicit default value. 🧚

3.6. Errors 💀

There are three different levels of granularity at which an error may be returned in JMAP. Buckle up, bestie 😭

When an API request is made, the request as a whole may be rejected because of rate limiting, malformed JSON, a request for an unknown capability, and so on. In that case, the entire request is rejected with an appropriate HTTP error response code and an additional JSON body with more detail for the client. 🚫

Provided the request itself is syntactically valid (the JSON is valid and, when decoded, matches the type signature of a Request object), the methods inside it are executed sequentially by the server. Each method may fail individually, for example if invalid arguments are given or an unknown method name is called. 🌸

Finally, methods that change server state often act upon a number of different records within a single call. Each record change may be rejected separately with a SetError, as described in Section 5.3. 🎀

3.6.1. Request-Level Errors 🦋

When an HTTP error response is returned to the client, the server SHOULD return a JSON "problem details" object as the response body, as per [RFC7807]. 💜

The following problem types are defined:

  • urn:ietf:params:jmap:error:unknownCapability. The client included a capability in the using property of the request that the server does not support.
  • urn:ietf:params:jmap:error:notJSON. The content type of the request was not application/json, or the request did not parse as I-JSON.
  • urn:ietf:params:jmap:error:notRequest. The request parsed as JSON but did not match the type signature of the Request object.
  • urn:ietf:params:jmap:error:limit. The request was not processed because it would have exceeded one of the request limits defined on the capability object, such as maxSizeRequest, maxCallsInRequest, or maxConcurrentRequests. A limit property MUST also be present on the "problem details" object, containing the name of the limit being applied. 🚦

3.6.1.1. Example ✨

       {
         "type": "urn:ietf:params:jmap:error:unknownCapability",
         "status": 400,
         "detail": "The Request object used capability
           'https://example.com/apis/foobar', which is not supported
           by this server."
       }

Another example:

     {
       "type": "urn:ietf:params:jmap:error:limit",
       "limit": "maxSizeRequest",
       "status": 400,
       "detail": "The request is larger than the server is willing to
                  process."
     }

3.6.2. Method-Level Errors 💅

Okay so, if a method runs into an error 💀, the right error response MUST be inserted at the current spot in the methodResponses array, and unless the spec says otherwise, further processing MUST NOT happen within that method call. Lowkey it's just "stop, report, move on" 🫶

Any further method calls in the request MUST then be processed as normal. Method-level errors MUST NOT generate an HTTP-level error, so the rest of the request keeps going even when one call goes sideways ✨

An error response looks like this:

[ "error", {
  "type": "unknownMethod"
}, "call-id" ]

The response name is error, and it MUST have a type property. Other properties may be present with more info, and those get detailed in the error type descriptions where they make sense 🧚

Okay, the big rule here: unless the serverPartialFail error is the one coming back, the externally visible state of the server MUST NOT have changed when an error is returned at the method level. Nothing should look different to the outside world 🙈

These error types are defined, and any method call can return them where it's relevant:

  • serverUnavailable: Some internal server resource was temporarily unavailable. Trying the same operation later (maybe after a backoff with a random factor) may work 🌸
  • serverFail: An unexpected or unknown error happened while the call was being processed. A description property should give more details about the error. The method call made no changes to the server's state, and trying the same operation again is expected to fail again. If it keeps happening, you'll likely need to contact the service administrator to fix it 😭
  • serverPartialFail: Some, but not all, of the expected changes described by the method actually happened. The client MUST resynchronise the impacted data to figure out the server's state. Using this error is strongly discouraged, bestie 🚫
  • unknownMethod: The server doesn't recognise this method name 🤷‍♀️
  • invalidArguments: One of the arguments is the wrong type or is otherwise invalid, or a required argument is missing. A description property MAY be present to help with debugging and explain what went wrong. This is a non-localised string, and it is not meant to be shown directly to end users 🍓
  • invalidResultReference: The method used a result reference for one of its arguments (see Section 3.7), but it failed to resolve 🔗
  • forbidden: The method and arguments are valid, but running the method would break an Access Control List (ACL) or other permissions policy 🔒
  • accountNotFound: The accountId doesn't match a valid account 👀
  • accountNotSupportedByMethod: The accountId given matches a valid account, but that account doesn't support this method or data type 🫠
  • accountReadOnly: This method changes state, but the account is read-only (as returned on the corresponding Account object in the JMAP Session resource) 📖

Further possible errors for a particular method are listed in that method's description 💖

Further general errors MAY be defined in future RFCs. Should a client get an error type it doesn't understand, it MUST treat it the same as the serverFail type. Being chill about unknown errors is iconic 👑

3.7. References to Previous Method Results 🦋

Okay, this is the efficiency trick ✨ To save on network trips, an argument to one method can be pulled from the result of a previous method call in the same request. Literally no extra round trip needed 🏃‍♀️💨

To do this, the client puts a # (an octothorpe) in front of the argument name. The value is a ResultReference object, which is described below. When processing a method call, the server MUST first check the arguments object for any names that begin with #. If it finds one, the result reference should be resolved and its value used as the "real" argument. After that, the method is processed as normal. If any result reference fails to resolve, the whole method MUST be rejected with an invalidResultReference error. And if an arguments object has the same argument name in both normal and referenced form (for example foo and #foo), the method MUST return an invalidArguments error. Don't do that, periodt 🙅‍♀️

A ResultReference object has these properties:

  • resultOf ("String"): The method call id (see Section 3.2) of a previous method call in the current request 🎯
  • name ("String"): The required name of a response to that method call 🏷️
  • path ("String"): A pointer into the arguments of the response picked out by the name and resultOf properties. This is a JSON Pointer [RFC6901], except it also lets you use * to map through an array (see the description below) 🗺️

To resolve one, follow these steps:

  1. Find the first response with a method call id identical to the resultOf property of the ResultReference, in the methodResponses array from the method calls already processed in the same request. If there isn't one, evaluation fails 🕵️‍♀️
  2. If the response name is not identical to the name property of the ResultReference, evaluation fails 🚫
  3. Apply the path to the arguments object of the response (the second item in the response array), following the JSON Pointer algorithm [RFC6901]. The one difference is this addition in "Evaluation" (see Section 4):

If the currently referenced value is a JSON array, the reference token may be exactly the single character *, which makes the new referenced value the result of applying the rest of the JSON Pointer tokens to every item in the array, with the results returned in the same order in a new array. If the result of applying the rest of the pointer tokens to each item was itself an array, the contents of that array are added to the output rather than the array itself (that is, the result is flattened from an array of arrays to a single array). If the result of applying the rest of the pointer tokens to a value was itself an array, its items should be included individually in the output rather than including the array itself (that is, the result is flattened from an array of arrays to a single array) 🌈

Let's do a simple example 🍭 Say we've got this API request methodCalls:

[[ "Foo/changes", {
    "accountId": "A1",
    "sinceState": "abcdef"
}, "t0" ],
[ "Foo/get", {
    "accountId": "A1",
    "#ids": {
        "resultOf": "t0",
        "name": "Foo/changes",
        "path": "/created"
    }
}, "t1" ]]

After the first method call runs, the methodResponses array looks like this:

[[ "Foo/changes", {
    "accountId": "A1",
    "oldState": "abcdef",
    "newState": "123456",
    "hasMoreChanges": false,
    "created": [ "f1", "f4" ],
    "updated": [],
    "destroyed": []
}, "t0" ]]

To run the Foo/get call, we look through the arguments and find one with a # prefix. To resolve it, we use the algorithm above:

  1. Find the first response with method call id t0. The Foo/changes response matches ✅
  2. Check that the response name is the same as in the result reference. It is, so we're good 👌
  3. Apply the path as a JSON Pointer to the arguments object. This just selects the created property, so evaluating it gives us [ "f1", "f4" ] 💫

The JMAP server then carries on with the Foo/get call as if the arguments were:

{
    "accountId": "A1",
    "ids": [ "f1", "f4" ]
}

Now for a more complicated one, using the JMAP Mail data model 📬 The goal is to fetch the from, date and subject for every Email in the first 10 Threads in the inbox, sorted newest first:

[[ "Email/query", {
  "accountId": "A1",
  "filter": { "inMailbox": "id_of_inbox" },
  "sort": [{ "property": "receivedAt", "isAscending": false }],
  "collapseThreads": true,
  "position": 0,
  "limit": 10,
  "calculateTotal": true
}, "t0" ],
[ "Email/get", {
  "accountId": "A1",
  "#ids": {
    "resultOf": "t0",
    "name": "Email/query",
    "path": "/ids"
  },
  "properties": [ "threadId" ]
}, "t1" ],
[ "Thread/get", {
  "accountId": "A1",
  "#ids": {
    "resultOf": "t1",
    "name": "Email/get",
    "path": "/list/*/threadId"
  }
}, "t2" ],
[ "Email/get", {
  "accountId": "A1",
  "#ids": {
    "resultOf": "t2",
    "name": "Thread/get",
    "path": "/list/*/emailIds"
  },
  "properties": [ "from", "receivedAt", "subject" ]
}, "t3" ]]

After the first 3 method calls run, the methodResponses array might look like this:

[[ "Email/query", {
    "accountId": "A1",
    "queryState": "abcdefg",
    "canCalculateChanges": true,
    "position": 0,
    "total": 101,
    "ids": [ "msg1023", "msg223", "msg110", "msg93", "msg91",
        "msg38", "msg36", "msg33", "msg11", "msg1" ]
}, "t0" ],
[ "Email/get", {
    "accountId": "A1",
    "state": "123456",
    "list": [{
        "id": "msg1023",
        "threadId": "trd194"
    }, {
        "id": "msg223",
        "threadId": "trd114"
    },
    ...
    ],
    "notFound": []
}, "t1" ],
[ "Thread/get", {
    "accountId": "A1",
    "state": "123456",
    "list": [{
        "id": "trd194",
        "emailIds": [ "msg1020", "msg1021", "msg1023" ]
    }, {
        "id": "trd114",
        "emailIds": [ "msg201", "msg223" ]
    },
    ...
    ],
    "notFound": []
}, "t2" ]]

To run the final Email/get call, we look through the arguments and find one with a # prefix. To resolve it, we apply the algorithm:

  1. Find the first response with method call id t2. The Thread/get response matches 🎀
  2. Thread/get is the name given in the result reference, so this is fine ✅
  3. Apply the path as a JSON Pointer to the arguments object, going token by token:
    1. list: get the array of thread objects 🧵
    2. *: for each of the items in that array:
      1. emailIds: get the array of Email ids 📨
      2. Concatenate these into a single array of all the ids in the result 🧩

The JMAP server then carries on with the Email/get call as if the arguments were:

{
    "accountId": "A1",
    "ids": [ "msg1020", "msg1021", "msg1023", "msg201", "msg223", ... ],
    "properties": [ "from", "receivedAt", "subject" ]
}

A ResultReference does a similar job to a creation id 🫶 In both cases, a chained method call can point at information that wasn't around yet when the request was made. But they're different things and totally not interchangeable. The only thing they have in common is the octothorpe used to mark them 🐙

3.8. Localisation of User-Visible Strings 🌍

Okay bestie, so if the server is sending back a custom string that the user actually sees (like an error message 😭), it SHOULD look at the Accept-Language header of the request (as defined in Section 5.3.5 of [RFC7231]) and pick the best localisation it has. The Content-Language header of the response (see Section 3.1.3.2 of [RFC7231]) SHOULD say which language the user-visible strings are in. It's giving organised. 🎀

Here's a little story to make it click. Say a request came in with this header:

       Accept-Language: fr-CH, fr;q=0.9, de;q=0.8, en;q=0.7, *;q=0.5

and a method generated an error to show the user. The server only has translations of that error in English and German. Looking at the Accept-Language header, the user's favourite language is French 🥖. Since there's no French translation, we check the next most preferred language, which is German. We have a German translation, so the server sends that back and tells everyone which language it picked using a Content-Language header, like so:

                           Content-Language: de

3.9. Security 🔒

Okay, this one is serious bestie, so pay attention 🙈. As always, the server must be strict about data received from the client. Arguments need to be checked for validity, because a malicious user could attempt to find an exploit through the API 💀. If the arguments are invalid (unknown, insufficient, or the wrong type for the data, etc.), the method MUST return a "invalidArguments" error and terminate. No exceptions, no vibes-based validation. 😤

3.10. Concurrency ⏳

Method calls within a single request MUST be executed in order. However, method calls from different concurrent API requests may be interleaved. This means the data on the server may change between two method calls within a single API request, so don't assume the world stays still while you're mid-request 👀.

4. The Core/echo Method 🔁

The Core/echo method returns exactly the same arguments it is given. It's super useful for checking whether you have a valid authenticated connection to a JMAP API endpoint. Think of it as the "hello, are you there?" of the whole API 🫶.

4.1. Example

Request:

                             [[ "Core/echo", {
                               "hello": true,
                               "high": 5
                             }, "b3ff" ]]

Response:

                             [[ "Core/echo", {
                               "hello": true,
                               "high": 5
                             }, "b3ff" ]]

5. Standard Methods and Naming Convention 📚

JMAP gives you a uniform interface for creating, retrieving, updating, and deleting objects of a particular type, and it's honestly so clean 💅. For a "Foo" data type, records of that type get fetched with a Foo/get call and modified with a Foo/set call. Delta updates can be fetched with a Foo/changes call. All of these methods follow a standard format, which we'll walk through below.

Some types may not have all of these methods. Specifications defining types MUST specify which methods are available for the type. Read the fine print, bestie 📝.

5.1. /get

Objects of type Foo are fetched with a call to Foo/get.

It takes the following arguments:

  • accountId: "Id"

The id of the account to use.

  • ids: "Id[]|null"

The ids of the Foo objects to return. If null, then all records of the data type are returned, if this is supported for that data type and the number of records does not exceed the "maxObjectsInGet" limit.

  • properties: "String[]|null"

If supplied, only the properties listed in the array are returned for each Foo object. If null, all properties of the object are returned. The id property of the object is always returned, even if it isn't explicitly requested 🆔. If an invalid property is requested, the call MUST be rejected with an "invalidArguments" error.

The response has the following arguments:

  • accountId: "Id"

The id of the account used for the call.

  • state: "String"

A (preferably short) string representing the state on the server for all the data of this type in the account (not just the objects returned in this call). If the data changes, this string MUST change. If the Foo data is unchanged, servers SHOULD return the same state string on subsequent requests for this data type. When a client receives a response with a different state string to a previous call, it MUST either throw away all currently cached objects for the type or call Foo/changes to get the exact changes. This is the state string doing its whole job 🧠.

  • list: "Foo[]"

An array of the Foo objects requested. This is the empty array if no objects were found or if the "ids" argument passed in was also an empty array. The results MAY be in a different order to the "ids" in the request arguments. If an identical id is included more than once in the request, the server MUST only include it once in either the "list" or the "notFound" argument of the response. No double-dipping allowed 🙅‍♀️.

  • notFound: "Id[]"

This array contains the ids passed to the method for records that do not exist. The array is empty if all requested ids were found, or if the "ids" argument passed in was either null or an empty array.

The following additional error may be returned instead of the Foo/get response:

"requestTooLarge": The number of ids requested by the client exceeds the maximum number the server is willing to process in a single method call. Too many ids, too much drama 😩.

5.2. /changes

When the state of the set of Foo records in an account changes on the server (whether due to creation, updates, or deletion), the "state" property of the Foo/get response will change. The Foo/changes method lets a client efficiently update the state of its Foo cache to match the new state on the server ✨. It takes the following arguments:

  • accountId: "Id"

The id of the account to use.

  • sinceState: "String"

The current state of the client. This is the string that was returned as the "state" argument in the Foo/get response. The server will return the changes that have occurred since this state.

  • maxChanges: "UnsignedInt|null"

The maximum number of ids to return in the response. The server MAY choose to return fewer than this value but MUST NOT return more. If not given by the client, the server may choose how many to return. If supplied by the client, the value MUST be a positive integer greater than 0. If a value outside of this range is given, the server MUST reject the call with an "invalidArguments" error.

The response has the following arguments:

  • accountId: "Id"

The id of the account used for the call.

  • oldState: "String"

This is the "sinceState" argument echoed back. It's the state the server is returning changes from.

  • newState: "String"

This is the state the client will be in after applying the set of changes to the old state.

  • hasMoreChanges: "Boolean"

If true, the client may call Foo/changes again with the "newState" returned to get further updates. If false, "newState" is the current server state.

  • created: "Id[]"

An array of ids for records that have been created since the old state.

  • updated: "Id[]"

An array of ids for records that have been updated since the old state.

  • destroyed: "Id[]"

An array of ids for records that have been destroyed since the old state.

If a record has been created AND updated since the old state, the server SHOULD just return the id in the "created" list but MAY return it in the "updated" list as well.

If a record has been updated AND destroyed since the old state, the server SHOULD just return the id in the "destroyed" list but MAY return it in the "updated" list as well.

If a record has been created AND destroyed since the old state, the server SHOULD remove the id from the response entirely. However, it MAY include it in just the "destroyed" list or in both the "destroyed" and "created" lists. It's a choose-your-own-adventure, but the top choice is "remove it" 🦋.

If a "maxChanges" is supplied, or set automatically by the server, the server MUST ensure the number of ids returned across "created", "updated", and "destroyed" does not exceed this limit. If there are more changes than this between the client's state and the current server state, the server SHOULD generate an update to take the client to an intermediate state, from which the client can keep calling Foo/changes until it is fully up to date. If it is unable to calculate an intermediate state, it MUST return a "cannotCalculateChanges" error response instead.

When generating intermediate states, the server may choose how to divide up the changes. For many types, it will give a better user experience to return the more recent changes first, since that's more likely to be what the user is most interested in 👑. The client can then keep paging in the older changes while the user is viewing the newer data. For example, suppose a server went through the following states:

                           A -> B -> C -> D -> E

And a client asks for changes from state "B". The server might first get the ids of records created, updated, or destroyed between states D and E, returning them with:

                           state: "B-D-E"
                           hasMoreChanges: true

The client will then ask for the changes from state "B-D-E", and the server can return the changes between states C and D, returning:

                           state: "B-C-E"
                           hasMoreChanges: true

Finally, the client will request the changes from "B-C-E", and the server can return the changes between states B and C, returning:

                           state: "E"
                           hasMoreChanges: false

Should the state on the server be modified in the middle of all this (to "F"), the server still does the same thing, but now when the update to state "E" is returned, it would indicate that it still has more changes for the client to fetch 🌀.

Where multiple changes to a record are split across different intermediate states, the server MUST NOT return a record as created after a response that deems it as updated or destroyed, and it MUST NOT return a record as destroyed before a response that deems it as created or updated. The server may have to coalesce multiple changes to a record to satisfy this requirement. Order matters, babe ⏰.

The following additional errors may be returned instead of the Foo/changes response:

"cannotCalculateChanges": The server cannot calculate the changes from the state string given by the client. Usually, this is due to the client's state being too old, or the server being unable to produce an update to an intermediate state when there are too many updates. The client MUST invalidate its Foo cache. Start fresh, no shame 🧹.

Maintaining state to allow calculation of Foo/changes can be expensive for the server, but always returning "cannotCalculateChanges" severely increases network traffic and resource usage for the client. So to allow efficient sync, servers SHOULD be able to calculate changes from any state string that was given to a client within the last 30 days (but of course they may support calculating updates from states older than this). That's the sweet spot, honestly iconic 🌈✨.

5.3. /set ✨

Okay bestie, changing Foo objects on the server is all done through the Foo/set method 💅 It covers creating, updating, and destroying Foo records. The server gets to be the traffic cop for ordering and dependencies when you send a bunch of operations at once 🚦 (like making sure there's always a minimum number of some record type 🧚).

Foo/set takes these arguments 🎀:

  • accountId: "Id"

The id of the account to use.

  • ifInState: "String|null"

This is a state string as returned by the Foo/get method (it represents the state of all objects of this type in the account). If you supply it, the string must match the current state. Otherwise the method gets aborted and a "stateMismatch" error comes back 🙅‍♀️ If it's null, any changes just get applied to the current state.

  • create: "Id[Foo]|null"

A map of a creation id (a temporary id the client picks 🌸) to Foo objects, or null if nothing is being created.

The Foo object type definition can set default values for properties. The client is allowed to leave out any property that has one of those defaults.

The client MUST omit any properties that may only be set by the server (for example, the "id" property on most object types) 🙈

  • update: "Id[PatchObject]|null"

A map of an id to a Patch object, which gets applied to the current Foo object with that id, or null if nothing is being updated.

A PatchObject is of type "String[*]" and it's an unordered set of patches 🧩 The keys are paths in JSON Pointer format [RFC6901], with an implicit leading /. So you prefix each key with / before running the JSON Pointer evaluation algorithm.

Every path MUST also follow these restrictions. If there's any violation, the update MUST be rejected with an "invalidPatch" error 💀

  • The pointer MUST NOT reference inside an array. You MUST NOT insert or delete from an array. The array MUST be replaced in its entirety instead 🍓
  • All parts prior to the last one (that is, the value after the final slash) MUST already exist on the object being patched.
  • There MUST NOT be two patches in the PatchObject where the pointer of one is the prefix of the pointer of the other. For example, "alerts/1/offset" and "alerts" would clash 🙅‍♀️

The value tied to each pointer decides how that patch gets applied:

  • If it's null, the property gets set to its default value if one is specified for it. If there's no default, the property gets removed from the patched object. If the key isn't present in the parent, this is a no-op 🫧
  • Anything else: that's the value to set for this property. It can be a replacement or an addition to the object being patched 🎨

Any server-set properties MAY be included in the patch, but only if their value is identical to the current server value (checked before the patches get applied to the object). Otherwise, the update MUST be rejected with an "invalidProperties" SetError 😭

This patch definition is designed so that an entire Foo object is also a valid PatchObject 🙌 The client can choose to save network usage by only sending the diff, or it can send the whole object. The server processes it the same either way ✨

  • destroy: "Id[]|null"

A list of ids for Foo objects to permanently delete, or null if nothing is being destroyed 🗑️

Each creation, modification, or destruction of an object counts as one atomic unit ⚛️ The server is allowed to commit changes to some objects but not others. However, it MUST NOT only commit part of an update to a single record. For example, it can't update a "name" property but skip a "count" property if both were supplied in the update object 🙅‍♀️

The final state MUST be valid once the Foo/set is finished 💖 The server may have to pass through invalid intermediate states while it works through the individual create, update, and destroy requests, but those states are never exposed to the client 🫣 Say a "name" property has to be unique. One method call could rename object A to B and, at the same time, rename object B to A. If the final state is valid, that's allowed! Otherwise, each creation, modification, or destruction of an object should be processed sequentially and accepted or rejected based on the current server state 🔍

If a create, update, or destroy gets rejected, the appropriate error MUST be added to the notCreated, notUpdated, or notDestroyed property of the response. The server MUST continue on to the next create, update, or destroy. It doesn't terminate the method 🏃‍♀️

If an id you give can't be found, the update or destroy MUST be rejected with a "notFound" set error 🔎

The server MAY skip an update (rejecting it with a "willDestroy" SetError) if that object is destroyed in the same /set request 🪦

Some records hold references to other records (foreign keys) 🔗 That reference can be set, via create or update, in the same request that creates the referenced record. To do this, the client refers to the new record by its creation id, prefixed with a #. The order of the method calls in the request by the client MUST be such that the record being referenced is created in the same call or an earlier one. That way, the server never has to look ahead 👀 Instead, while processing a request, the server MUST keep a simple map for the duration of the request. It maps each creation id to the record id of each newly created record, so it can substitute the correct value in later method calls when needed 🗺️ For records with references to the same type, the server MUST order the creates and updates within a single method call so that creates happen before their creation ids get referenced by another create, update, or destroy in the same call 🧠

Creation ids aren't scoped by type. They're one single map for all types 🌈 A client SHOULD NOT reuse a creation id anywhere in the same API request. If a creation id is reused, the server MUST map that creation id to the most recently created item with that id. To make proxying API requests easy, an initial set of creation id to real id values can be passed with a request (see "The Request Object", Section 3.3), and the final state of the map gets passed back out with the response (see "The Response Object", Section 3.4) 🎀

The response has these arguments 💌:

  • accountId: "Id"

The id of the account used for the call.

  • oldState: "String|null"

The state string that would have been returned by Foo/get before the requested changes were made, or null if the server doesn't know what the previous state string was.

  • newState: "String"

The state string that will now be returned by Foo/get.

  • created: "Id[Foo]|null"

A map of the creation id to an object containing any properties of the created Foo object that the client didn't send. This includes all server-set properties (like the "id" in most object types) and any properties the client left out, which the server then set to a default 🧁

This argument is null if no Foo objects were successfully created.

  • updated: "Id[Foo|null]|null"

The keys in this map are the ids of all the Foos that were successfully updated.

The value for each id is a Foo object containing any property that changed in a way not explicitly requested by the PatchObject sent to the server, or null if there are none. This lets the client know about any changes to server-set or computed properties 👑

This argument is null if no Foo objects were successfully updated.

  • destroyed: "Id[]|null"

A list of Foo ids for records that were successfully destroyed, or null if none were.

  • notCreated: "Id[SetError]|null"

A map of the creation id to a SetError object for each record that failed to be created, or null if everything succeeded.

  • notUpdated: "Id[SetError]|null"

A map of the Foo id to a SetError object for each record that failed to be updated, or null if everything succeeded.

  • notDestroyed: "Id[SetError]|null"

A map of the Foo id to a SetError object for each record that failed to be destroyed, or null if everything succeeded.

A SetError object has these properties 🚨:

  • type: "String"

The type of error.

  • description: "String|null"

A description of the error to help with debugging, including an explanation of what the problem was. It's a non-localised string and it's not meant to be shown directly to end users 🙈

The following SetError types are defined, and they may be returned for set operations on any record type where it makes sense 💅:

  • "forbidden": (create; update; destroy). The create, update, or destroy would violate an ACL or other permissions policy 🔒
  • "overQuota": (create; update). The create would exceed a server-defined limit on the number or total size of objects of this type 📦
  • "tooLarge": (create; update). The create or update would result in an object that exceeds a server-defined limit for the maximum size of a single object of this type 🐘
  • "rateLimit": (create). Too many objects of this type have been created recently, and a server-defined rate limit has been reached. It may work if you try again later ⏳
  • "notFound": (update; destroy). The id given to update or destroy can't be found 🔎
  • "invalidPatch": (update). The PatchObject given to update the record wasn't a valid patch (see the patch description) 🩹
  • "willDestroy": (update). The client asked for an object to be both updated and destroyed in the same /set request, and the server has decided to ignore the update 🫡
  • "invalidProperties": (create; update). The record given is invalid in some way. For example:
    • It contains properties that are invalid according to the type specification of this record type.
    • It contains a property that may only be set by the server (for example, "id") and is different to the current value. Note, to let clients pass whole objects back, it's not an error to include a server-set property in an update as long as the value is identical to the current value on the server 💡
    • There's a reference to another record (a foreign key), and the given id doesn't correspond to a valid record 🔗

The SetError object SHOULD also have a property called "properties" of type "String[]" that lists all the properties that were invalid 📝

Individual methods MAY specify more specific errors for certain conditions that would otherwise result in an invalidProperties error. If the condition for one of those is met, it MUST be returned instead of the invalidProperties error 🎯

  • "singleton": (create; destroy). This is a singleton type, so you can't create another one or destroy the existing one 👑

Other possible SetError types MAY be given in specific method descriptions. Other properties MAY also be present on the SetError object, as described in the relevant methods 🌸

The following additional errors may be returned instead of the Foo/set response 🙃:

  • "requestTooLarge": The total number of objects to create, update, or destroy is more than the maximum number the server is willing to process in a single method call 📚
  • "stateMismatch": An ifInState argument was supplied, and it doesn't match the current state 😬

5.4. /copy ✨

okay bestie, so the only way to move Foo records between two different accounts is to copy them with the Foo/copy method, and once that copy has worked, you delete the original 💅 the onSuccessDestroyOriginal argument lets you try to do both in one method call 🦋 BUT ngl, those two actions are not atomic 🙈 so it's possible for the copy to succeed while the original doesn't get destroyed for some reason 😭

the copy is basically three phases 🎀:

  1. Reading the current values from the from account.
  2. Writing the new copies to the other account.
  3. Destroying the originals in the from account, if requested.

data may change in between phases because of concurrent requests 👀

the Foo/copy method takes these arguments 💖:

  • fromAccountId: "Id": The id of the account to copy records from.
  • ifFromInState: "String|null": This is a state string as returned by the Foo/get method. If supplied, the string must match the current state of the account referenced by the fromAccountId when reading the data to be copied. Otherwise, the method will be aborted and a "stateMismatch" error returned. If null, the data will be read from the current state.
  • accountId: "Id": The id of the account to copy records to. This MUST be different to the fromAccountId.
  • ifInState: "String|null": This is a state string as returned by the Foo/get method. If supplied, the string must match the current state of the account referenced by the accountId. Otherwise, the method will be aborted and a "stateMismatch" error returned. If null, any changes will be applied to the current state.
  • create: "Id[Foo]": A map of the creation id to a Foo object. The Foo object MUST contain an id property, which is the id (in the fromAccount) of the record to be copied. When creating the copy, any other properties included are used instead of the current value for that property on the original.
  • onSuccessDestroyOriginal: "Boolean" (default: false): If true, an attempt will be made to destroy the original records that were successfully copied. After emitting the Foo/copy response, but before processing the next method, the server MUST make a single call to Foo/set to destroy the original of each successfully copied record. The output of this is added to the responses as normal, to be returned to the client. 🌸
  • destroyFromIfInState: "String|null": This argument is passed on as the ifInState argument to the implicit Foo/set call, if made at the end of this request to destroy the originals that were successfully copied.

each record copy is considered an atomic unit that may succeed or fail individually ✨

the response has these arguments 🌈:

  • fromAccountId: "Id": The id of the account records were copied from.
  • accountId: "Id": The id of the account records were copied to.
  • oldState: "String|null": The state string that would have been returned by Foo/get on the account records that were copied to before making the requested changes, or null if the server doesn't know what the previous state string was.
  • newState: "String": The state string that will now be returned by Foo/get on the account records were copied to.
  • created: "Id[Foo]|null": A map of the creation id to an object containing any properties of the copied Foo object that are set by the server (such as the id in most object types). Note, the id is likely to be different to the id of the object in the account it was copied from. This argument is null if no Foo objects were successfully copied.
  • notCreated: "Id[SetError]|null": A map of the creation id to a SetError object for each record that failed to be copied, or null if none.

the SetError may be any of the standard set errors returned for a create or update 🧚 on top of those, the following SetError is defined:

"alreadyExists": The server forbids duplicates, and the record already exists in the target account. An "existingId" property of type "Id" MUST be included on the SetError object with the id of the existing record. 💜

the following additional errors may be returned instead of the Foo/copy response 💀:

  • "fromAccountNotFound": The "fromAccountId" does not correspond to a valid account.
  • "fromAccountNotSupportedByMethod": The "fromAccountId" given corresponds to a valid account, but the account does not support this data type.
  • "stateMismatch": An "ifInState" argument was supplied and it does not match the current state, or an "ifFromInState" argument was supplied and it does not match the current state in the from account. 😭🫶

5.5. /query 🔍✨

Okay bestie, if the total data is super tiny, the client can just grab the whole thing and sort or filter it locally. Easy peasy 💅. But for huge sets (think multi-gigabyte mailboxes 😭), the client really needs to search, sort, and window the data on the server. Otherwise it's just not it.

A query on the set of Foos in an account is made by calling Foo/query. It takes a bunch of arguments to decide which records get included, how they're sorted, and which slice of the results comes back (the full list might be very long 🙈). The result is a list of Foo ids 🌸.

A call to Foo/query takes the following arguments:

  • accountId: Id

The id of the account to use. 🎀

  • filter: FilterOperator|FilterCondition|null

This decides which Foos show up in the results. If it's null, every object of this type in the account gets included 🦋. A FilterOperator object has these properties:

  • operator: String

This MUST be one of the following strings:

  • "AND": All of the conditions must match for the filter to match.
  • "OR": At least one of the conditions must match for the filter to match.
  • "NOT": None of the conditions must match for the filter to match.
  • conditions: (FilterOperator|FilterCondition)[]

The conditions to evaluate against each record.

A FilterCondition is an object whose allowed properties and semantics depend on the data type. Each type's own /query method spec defines them 🧚. It MUST NOT have an "operator" property.

  • sort: Comparator[]|null

This lists the names of properties to compare between two Foo records, plus how to compare them, to figure out which one comes first 👑. If two Foo records have an identical value for the first comparator, the next comparator gets a look, and so on. If all the comparators are the same (this includes the case where an empty array or null is given as the sort argument), the sort order is server dependent, but it MUST be stable between calls to Foo/query. A Comparator has these properties:

  • property: String

The name of the property on the Foo objects to compare.

  • isAscending: Boolean (optional; default: true)

If true, sort in ascending order. If false, reverse the comparator's results to sort in descending order.

  • collation: String (optional; default is server-dependent)

This is the identifier, as registered in the collation registry defined in [RFC4790], for the algorithm used to compare the order of strings. The server's supported algorithms are advertised in the capabilities object returned with the Session object (see Section 2) ✨.

If it's omitted, the default algorithm is server dependent, but:

  1. It MUST be unicode-aware.
  2. It MAY be selected based on an Accept-Language header in the request (as defined in [RFC7231], Section 5.3.5) or out-of-band information about the user's language/locale.
  3. It SHOULD be case insensitive where such a concept makes sense for a language/locale. Where the user's language is unknown, it is RECOMMENDED to follow the advice in Section 5.2.3 of [RFC8264].

The i;unicode-casemap collation [RFC5051] and the Unicode Collation Algorithm (<http://www.unicode.org/reports/tr10/>) are two examples that fulfil these criteria and give reasonable behaviour for a large number of languages 🌈.

When the property being compared is not a string, the collation property is ignored and the following comparison rules apply based on the type. In ascending order:

  • "Boolean": false comes before true.
  • "Number": A lower number comes before a higher number.
  • "Date"/"UTCDate": The earlier date comes first.

The Comparator object may also have additional properties as required for specific sort operations defined in a type's /query method 💜.

  • position: Int (default: 0)

The zero-based index of the first id in the full list of results to return.

If a negative value is given, it's an offset from the end of the list. Specifically, the negative value MUST be added to the total number of results given the filter, and if it's still negative, it gets clamped to 0. This is now the zero-based index of the first id to return 🍓.

If the index is greater than or equal to the total number of objects in the results list, the ids array in the response will be empty. That's not an error, bestie 🫶.

  • anchor: Id|null

A Foo id. If supplied, the position argument is ignored. The index of this id in the results gets used together with the anchorOffset argument to work out the index of the first result to return (more details below) 🎯.

  • anchorOffset: Int (default: 0)

The index of the first result to return, relative to the index of the anchor (if an anchor is given). This MAY be negative. For example, -1 means the Foo immediately before the anchor is the first result in the returned list 🦋.

  • limit: UnsignedInt|null

The maximum number of results to return. If null, no limit is presumed. The server MAY choose to enforce a maximum limit argument. In that case, if a greater value is given (or if it's null), the limit is clamped to the maximum, and the new limit is returned with the response so the client knows 👀. If a negative value is given, the call MUST be rejected with an invalidArguments error 💀.

  • calculateTotal: Boolean (default: false)

Does the client want to know the total number of results in the query? This can be slow and expensive for servers to calculate, especially with complex filters, so clients should take care to only request the total when they really need it 🙏.

If an anchor argument is given, the anchor is looked for in the results after filtering and sorting. If it's found, the anchorOffset is added to its index. If the resulting index is now negative, it's clamped to 0. This index is then used exactly as though it were supplied as the position argument. If the anchor isn't found, the call is rejected with an "anchorNotFound" error 😵.

If an anchor is specified, any position argument supplied by the client MUST be ignored. If no anchor is supplied, any anchorOffset argument MUST be ignored.

A client can use anchor instead of position to find the index of an id within a large set of results 🧭.

The response has the following arguments:

  • accountId: Id

The id of the account used for the call.

  • queryState: String

A string encoding the current state of the query on the server. This string MUST change if the results of the query (i.e., the matching ids and their sort order) have changed. The queryState string MAY change if something has changed on the server, which means the results may have changed but the server doesn't know for sure 🤷‍♀️.

The queryState string only represents the ordered list of ids that match the particular query (including its sort/filter). There is no requirement for it to change if a property on an object matching the query changes but the query results are unaffected (indeed, it's more efficient if the queryState string doesn't change in this case) ✨. The queryState string only has meaning when compared to future responses to a query with the same type/sort/filter, or when used with /queryChanges to fetch changes.

Should a client receive a response with a different queryState string to a previous call, it MUST either throw away the currently cached query and fetch it again (note: this doesn't require fetching the records again, just the list of ids) or call Foo/queryChanges to get the difference 🔄.

  • canCalculateChanges: Boolean

This is true if the server supports calling Foo/queryChanges with these filter/sort parameters. Note, this does not guarantee that the Foo/queryChanges call will succeed, as it may only be possible for a limited time afterwards due to server internal implementation details 🌙.

  • position: UnsignedInt

The zero-based index of the first result in the ids array within the complete list of query results.

  • ids: Id[]

The list of ids for each Foo in the query results, starting at the index given by the position argument of this response and continuing until it hits the end of the results or reaches the limit number of ids. If position is >= total, this MUST be the empty list 🫧.

  • total: UnsignedInt (only if requested)

The total number of Foos in the results (given the filter). This argument MUST be omitted if the calculateTotal request argument is not true 🧮.

  • limit: UnsignedInt (if set by the server)

The limit enforced by the server on the maximum number of results to return. This is only returned if the server set a limit or used a different limit than the one given in the request 📏.

The following additional errors may be returned instead of the Foo/query response:

  • "anchorNotFound": An anchor argument was supplied, but it can't be found in the results of the query 😭.
  • "unsupportedSort": The sort is syntactically valid, but it includes a property the server doesn't support sorting on, or a collation method it doesn't recognise 🙃.
  • "unsupportedFilter": The filter is syntactically valid, but the server can't process it. If the filter was the result of a user's search input, the client SHOULD suggest that the user simplify their search 🥺💖.

5.6. /queryChanges ✨

okay bestie, this is the method that lets a client update its cached query results to match what the server has now, without re-downloading the whole thing 🦋 it's giving efficiency, no cap 💅 The Foo/queryChanges method takes these arguments:

  • accountId: "Id" 🎀 The id of the account to use.
  • filter: "FilterOperator|FilterCondition|null" 🌸 The filter argument that was used with Foo/query.
  • sort: "Comparator[]|null" 💜 The sort argument that was used with Foo/query.
  • sinceQueryState: "String" ⭐ This is the current state of the query in the client. It's the string that came back as the queryState argument in the Foo/query response with the same sort/filter. The server will return the changes made to the query since this state, so it's the main clue for the whole thing 🔥
  • maxChanges: "UnsignedInt|null" 🍓 The maximum number of changes to return in the response. Check the error descriptions below for more details 👀
  • upToId: "Id|null" 🧚 The last (highest-index) id the client currently has cached from the query results. When there are a huge number of results, the client has often only downloaded and cached a small chunk from the beginning, and that's the common case 🙈 If the sort and filter are both only on immutable properties, this lets the server skip changes after this point in the results, which can seriously boost efficiency ✨ If they aren't immutable, this argument gets ignored 💀
  • calculateTotal: "Boolean" (default: false) 🌈 Does the client want to know the total number of results now in the query? This can be slow and expensive for servers to calculate, especially with complex filters, so clients should take care to only ask for the total when they actually need it 🫶

The response has these arguments:

  • accountId: "Id" 🎀 The id of the account used for the call.
  • oldQueryState: "String" 🦋 This is the sinceQueryState argument echoed back. In other words, it's the state the server is returning changes from 💖
  • newQueryState: "String" 🌟 This is the state the query will be in after applying the set of changes to the old state ✨
  • total: "UnsignedInt" (only if requested) 🧮 The total number of Foos in the results (given the filter). This argument MUST be omitted if the calculateTotal request argument is not true 🙅‍♀️
  • removed: "Id[]" 🗑️ The id for every Foo that was in the query results in the old state and is not in the results in the new state.

If the server can't calculate this exactly, the server MAY return the ids of extra Foos too, ones that may have been in the old results but aren't in the new results 🤷‍♀️

If the sort and filter are both only on immutable properties and an upToId is supplied and exists in the results, any ids that were removed but have a higher index than upToId SHOULD be omitted 🙈

If the filter or sort includes a mutable property, the server MUST include all Foos in the current results for which this property may have changed. Their positions may have moved in the results, so the client has to reinsert them to keep its query cache correct 😭

  • added: "AddedItem[]" 🌈 The id and index in the query results (in the new state) for every Foo that has been added to the results since the old state AND every Foo in the current results that was included in the removed array (because of a filter or sort based on a mutable property) 🧚

If the sort and filter are both only on immutable properties and an upToId is supplied and exists in the results, any ids that were added but have a higher index than upToId SHOULD be omitted 🫶

The array MUST be sorted in order of index, with the lowest index first 👑

An AddedItem object has the following properties:

  • id: "Id" 🍓
  • index: "UnsignedInt" 🍓

The result of all this is that if the client has a cached sparse array of Foo ids that matches the results in the old state, then it looks like this 🧸

fooIds = [ "id1", "id2", null, null, "id3", "id4", null, null, null ]

If the client splices out all the ids in the removed array that it has in its cached results, then:

removed = [ "id2", "id31", ... ];
fooIds => [ "id1", null, null, "id3", "id4", null, null, null ]

and then splices in (one by one, in order, starting with the lowest index) all of the ids in the added array:

added = [{ id: "id5", index: 0, ... }];
fooIds => [ "id5", "id1", null, null, "id3", "id4", null, null, null ]

and finally truncates or extends to the new total length. Then the results are in the new state, and we are done 🎉

Note: splicing in adds the item at the given index and increments the index of all items previously at that index or higher. Splicing out is the inverse: it removes the item and decrements the index of every item after it in the array 🧠

The following additional errors may be returned instead of the Foo/queryChanges response:

  • "tooManyChanges": There are more changes than the client's maxChanges argument. Each item in the removed or added array counts as one change. The client can retry with a higher max changes value or invalidate its cache of the query results. The client is allowed to retry, so it's totally up to them 💅
  • "cannotCalculateChanges": The server can't calculate the changes from the queryState string the client gave, usually because the client's state is too old. The client MUST invalidate its cache of the query results. No cap, this one's non-negotiable 😭🔥

5.7. Examples 🎀

okay so let's say we have a type Todo with these properties 📝

  • id: "Id" (immutable; server-set)
    • The id of the object. 🆔
  • title: "String"
    • A brief summary of what is to be done. ✍️
  • keywords: "String[Boolean]" (default: {})
    • A set of keywords that apply to the Todo. The set is represented as an object, with the keys being the "keywords". The value for each key in the object MUST be true. (This format allows you to update an individual key using patch syntax rather than having to update the whole set of keywords as one, which a "String[]" representation would require.) 🏷️✨
  • neuralNetworkTimeEstimation: "Number" (server-set)
    • The title and keywords are fed into the server's state-of-the-art neural network to get an estimation of how long this Todo will take, in seconds. 🧠⏱️
  • subTodoIds: "Id[]|null"
    • The ids of a list of other Todos to complete as part of this Todo. 🌸

suppose also that all the standard methods are defined for this type and the FilterCondition object supports a "hasKeyword" property to match Todos with the given keyword. the girlies are literally set up 💅

a client might want to display the list of Todos with either a "music" keyword or a "video" keyword, so it makes the following method call 🎶🎬

[[ "Todo/query", {
  "accountId": "x",
  "filter": {
    "operator": "OR",
    "conditions": [
      { "hasKeyword": "music" },
      { "hasKeyword": "video" }
    ]
  },
  "sort": [{ "property": "title" }],
  "position": 0,
  "limit": 10
}, "0" ],
[ "Todo/get", {
  "accountId": "x",
  "#ids": {
    "resultOf": "0",
    "name": "Todo/query",
    "path": "/ids"
  }
}, "1" ]]

this would query the server for the set of Todos with a keyword of either "music" or "video", sorted by title, and limited to the first 10 results. 💖 it then fetches the full object for each of these Todos using back-references to reference the result of the query. the response might look something like this 👀

[[ "Todo/query", {
  "accountId": "x",
  "queryState": "y13213",
  "canCalculateChanges": true,
  "position": 0,
  "ids": [ "a", "b", "c", "d", "e", "f", "g", "h", "i", "j" ]
}, "0" ],
[ "Todo/get", {
  "accountId": "x",
  "state": "10324",
  "list": [{
    "id": "a",
    "title": "Practise Piano",
    "keywords": {
      "music": true,
      "beethoven": true,
      "mozart": true,
      "liszt": true,
      "rachmaninov": true
    },
    "neuralNetworkTimeEstimation": 3600
  }, {
    "id": "b",
    "title": "Watch Daft Punk music video",
    "keywords": {
      "music": true,
      "video": true,
      "trance": true
    },
    "neuralNetworkTimeEstimation": 18000
  },
  ...
  ]
}, "1" ]]

now, suppose the user adds a keyword "chopin" and removes the keyword "mozart" from the "Practise Piano" task 🎹✨ the client may send the whole object to the server, as this is a valid PatchObject 🩷

[[ "Todo/set", {
  "accountId": "x",
  "ifInState": "10324",
  "update": {
    "a": {
      "id": "a",
      "title": "Practise Piano",
      "keywords": {
        "music": true,
        "beethoven": true,
        "chopin": true,
        "liszt": true,
        "rachmaninov": true
      },
      "neuralNetworkTimeEstimation": 360
    }
  }
}, "0" ]]

or it may send a minimal patch, which is very giving efficiency 🦋

[[ "Todo/set", {
  "accountId": "x",
  "ifInState": "10324",
  "update": {
    "a": {
      "keywords/chopin": true,
      "keywords/mozart": null
    }
  }
}, "0" ]]

the effect is exactly the same on the server in either case, and presuming the server is still in state "10324", it will probably return success 🥹

[[ "Todo/set", {
  "accountId": "x",
  "oldState": "10324",
  "newState": "10329",
  "updated": {
    "a": {
      "neuralNetworkTimeEstimation": 5400
    }
  }
}, "0" ]]

the server changed the "neuralNetworkTimeEstimation" property on the object 🙈 as this changed in a way not explicitly requested by the PatchObject sent to the server, it is returned with the "updated" confirmation. the server really said "I'm doing my own thing" 💅

let us now add a sub-Todo to our new "Practise Piano" Todo 🎀 in this example, we can see the use of a reference to a creation id to allow us to set a foreign key reference to a record created in the same request. main character energy 👑

[[ "Todo/set", {
  "accountId": "x",
  "create": {
    "k15": {
      "title": "Warm up with scales"
    }
  },
  "update": {
    "a": {
      "subTodoIds": [ "#k15" ]
    }
  }
}, "0" ]]

now, suppose another user deleted the "Listen to Daft Punk" Todo 💀 the first user will receive a push notification (see Section 7) with the changed state string for the "Todo" type. since the new string does not match its current state, it knows it needs to check for updates. it may make a request like 🔔

[[ "Todo/changes", {
  "accountId": "x",
  "sinceState": "10324",
  "maxChanges": 50
}, "0" ],
[ "Todo/queryChanges", {
  "accountId": "x",
  "filter": {
    "operator": "OR",
    "conditions": [
      { "hasKeyword": "music" },
      { "hasKeyword": "video" }
    ]
  },
  "sort": [{ "property": "title" }],
  "sinceQueryState": "y13213",
  "maxChanges": 50
}, "1" ]]

and receive in response 📥

[[ "Todo/changes", {
  "accountId": "x",
  "oldState": "10324",
  "newState": "871903",
  "hasMoreChanges": false,
  "created": [],
  "updated": [],
  "destroyed": ["b"]
}, "0" ],
[ "Todo/queryChanges", {
  "accountId": "x",
  "oldQueryState": "y13213",
  "newQueryState": "y13218",
  "removed": ["b"],
  "added": null
}, "1" ]]

suppose the user has access to another account "y", for example, a team account shared between multiple users. 🫶 to move an existing Todo from account "x", the client would call 🚚

[[ "Todo/copy", {
  "fromAccountId": "x",
  "accountId": "y",
  "create": {
    "k5122": {
      "id": "a"
    }
  },
  "onSuccessDestroyOriginal": true
}, "0" ]]

the server successfully copies the Todo to a new account (where it receives a new id) and deletes the original. due to the implicit call to "Todo/set", there are two responses to the single method call, both with the same method call id 🧚 so yes, two replies, one id, zero drama

[[ "Todo/copy", {
  "fromAccountId": "x",
  "accountId": "y",
  "created": {
    "k5122": {
      "id": "DAf97"
    }
  },
  "oldState": "c1d64ecb038c",
  "newState": "33844835152b"
}, "0" ],
[ "Todo/set", {
  "accountId": "x",
  "oldState": "871903",
  "newState": "871909",
  "destroyed": [ "a" ],
  ...
}, "0" ]]

5.8. Proxy Considerations 🌈

JMAP has been designed to allow an API endpoint to easily proxy through to one or more JMAP servers. this may be useful for load balancing, augmenting capabilities, or presenting a single endpoint to accounts hosted on different JMAP servers (splitting the request based on each method's "accountId" argument). the proxy need only understand the general structure of a JMAP Request object; it does not need to know anything specifically about the methods and arguments it will pass through to other servers. so the proxy can stay chill and not get into everyone's business 🫧

if splitting up the methods in a request to call them on different backend servers, the proxy must do two things to ensure back-references and creation-id references resolve the same as if the entire request were processed on a single server 🧩

  1. it must pass a "createdIds" property with each subrequest. if this is not given by the client, an empty object should be used for the first subrequest. the "createdIds" property of each subresponse should be passed on in the next subrequest. 📦
  2. it must resolve back-references to previous method results that were processed on a different server. this is a relatively simple syntactic substitution, described in Section 3.7. 🔁
when splitting a request based on accountId, proxy implementors do need to be aware of "/copy" methods that copy between accounts. if the accounts are on different servers, the proxy will have to implement this functionality directly. this is the one that gets people 😭

6. Binary Data 📦✨

Okay bestie, so binary data is referenced by a blobId in JMAP 💖 and it gets uploaded and downloaded separately through the core API. The blobId ONLY represents the raw bytes 🫶, not the file name or content type. That metadata lives next to the blobId in whatever object is pointing at it. And the data behind a blobId is immutable, so it never changes. Iconic 🔒

Any blobId that exists within an account can be used when you create or update another object in that same account 🦋. Like, an Email object can have a blobId that stands for the whole message in Internet Message Format [RFC5322]. So you could make a brand new Email with an attachment and reuse that blobId, which basically attaches the old message to the new one 💅. You can also grab any existing attachment from an old message without downloading and re-uploading it. Lowkey genius.

When the client uses a blobId in a create or update, the server MAY give a new blobId to refer to the same binary data inside the new or updated object 🌈. If it does, it MUST return any properties that contain a changed blobId in the created or updated response, so the client actually gets the new ids. No surprises, periodt ✨

A blob that no JMAP object references (like a message attachment) MAY be deleted by the server to free up resources 🧹. Uploads are unreferenced blobs to start with. To keep things interoperable:

  • The server SHOULD use a separate quota for unreferenced blobs, not the account's usual quota. For shared accounts, this quota SHOULD be separate per user 👑.
  • This quota SHOULD be at least the maximum total size that a single object can reference on this server. For example, if you're supporting JMAP Mail, it should be at least the maximum total attachments size for a message 📎.
  • When an upload would take the user over quota, the server MUST delete unreferenced blobs in date order, oldest first, until there's room for the new blob 🗑️.
  • Except where quota restrictions force early deletion, an unreferenced blob MUST NOT be deleted for at least 1 hour from the time of upload. If it gets reuploaded, the same blobId MAY be returned, but this SHOULD reset the expiry time ⏰.
  • A blob MUST NOT be deleted during the method call that removed the last reference. That way a client can issue a create and a destroy that both reference the blob within the same method call. Main character planning 🎀

6.1. Uploading Binary Data 🚀

There's one single endpoint that handles all file uploads for an account 🌸, no matter what they're for. The Session object (see Section 2) has an "uploadUrl" property in URI Template (level 1) format [RFC6570], and it MUST contain a variable called "accountId". The client can use that template along with an "accountId" to get the URL of the file upload resource 🧚.

To upload a file, the client sends an authenticated POST request to the file upload resource 💌.

A successful request MUST return a single JSON object with the following properties as the response:

  • accountId: "Id"

The id of the account used for the call.

  • blobId: "Id"

The id representing the binary data uploaded. The data for this id is immutable. The id only refers to the binary data, not any metadata 🔐.

  • type: "String"

The media type of the file (as specified in [RFC6838], Section 4.2), as set in the Content-Type header of the upload HTTP request 🏷️.

  • size: "UnsignedInt"

The size of the file in octets 📏.

If identical binary content to an existing blob in the account gets uploaded, the existing blobId MAY be returned 🪞.

Clients should use the blobId they get back in a timely manner. In rare circumstances the server may have deleted the blob before the client uses it, so the client should keep a reference to the local file so it can upload it again in that situation 🫠.

When an HTTP error response goes back to the client, the server SHOULD return a JSON "problem details" object as the response body, as per [RFC7807] 🚨.

As access controls are often decided by the object holding the reference to a blob, unreferenced blobs MUST only be accessible to the uploader, even in shared accounts 🙈. Private era, no cap.

6.2. Downloading Binary Data 📥

The Session object (see Section 2) has a "downloadUrl" property, which is in URI Template (level 1) format [RFC6570]. The URL MUST contain variables called "accountId", "blobId", "type", and "name" 💫.

To download a file, the client makes an authenticated GET request to the download URL with the right variables filled in:

  • "accountId": The id of the account to which the record with the blobId belongs.
  • "blobId": The blobId representing the data of the file to download.
  • "type": The type for the server to set in the "Content-Type" header of the response. The blobId only represents the binary data and doesn't have a content-type built into it, so this is the vibe check 🎨.
  • "name": The name for the file. The server MUST return this as the filename if it sets a "Content-Disposition" header 📁.

Since the data for a particular blobId is immutable, the response from the generated download URL is too 🧊. So implementors are recommended to set long cache times and use the "immutable" Cache-Control extension [RFC8246] for successful responses. For example: Cache-Control: private, immutable, max-age=31536000 🍓.

When an HTTP error response goes back to the client, the server SHOULD return a JSON "problem details" object as the response body, as per [RFC7807] 💀.

6.3. Blob/copy 🫧

Binary data can be copied between two different accounts using the "Blob/copy" method, so the client doesn't have to download and then re-upload it 🚫⬇️. Efficient bestie behaviour.

The "Blob/copy" method takes the following arguments:

  • fromAccountId: "Id"

The id of the account to copy blobs from.

  • accountId: "Id"

The id of the account to copy blobs to.

  • blobIds: "Id[]"

A list of ids of blobs to copy to the other account.

The response has the following arguments:

  • fromAccountId: "Id"

The id of the account blobs were copied from.

  • accountId: "Id"

The id of the account blobs were copied to.

  • copied: "Id[Id]|null"

A map of the blobId in the fromAccount to the id for the blob in the account it was copied to, or null if none were successfully copied 🍬.

  • notCopied: "Id[SetError]|null"

A map of blobId to a SetError object for each blob that failed to be copied, or null if none 🥺.

The SetError may be any of the standard set errors that can come back for a create, as defined in Section 5.3. On top of that, the "notFound" SetError may come back if the blobId to be copied can't be found 🔍.

The following extra method-level error might come back instead of the "Blob/copy" response:

"fromAccountNotFound": The "fromAccountId" included with the request does not correspond to a valid account. Not the vibe 😤.

7. Push 🔔

Push notifications let clients update (almost) instantly so they stay in sync with data changes on the server ⚡. The general model is simple and sends minimal data over the push channel: just enough for the client to know whether it needs to resync 🧠. The format lets multiple changes get coalesced into one push update, and the server can rate limit how often pushes happen 🐢. If some push events get dropped before they reach the client, that's fine. The next time the client gets or sets any records of a changed type, it'll notice the data changed and still sync everything 🌷.

There are two different mechanisms a client can use to receive push notifications, because clients can live in lots of different environments 🌍. An event source resource (see Section 7.3) lets clients that can hold transport connections open receive push notifications straight from the JMAP server. That's simple and skips third parties, but it's often not doable on constrained platforms like mobile devices 📱. The other option is that clients use whatever push service their environment supports. A URL for that push service is registered with the JMAP server (see Section 7.2), and then the server POSTs each notification to that URL. The push service is responsible for routing them to the client from there 🛎️.

7.1. The StateChange Object 🎈

When something changes on the server, the server pushes a StateChange object to the client. A StateChange object has the following properties:

  • @type: "String"

This MUST be the string "StateChange" 💜.

  • changed: "Id[TypeState]"

A map of an "account id" to an object encoding the state of data types that have changed for that account since the last StateChange object was pushed. There's one entry for each of the accounts the user has access to where something has changed 👯.

A TypeState object is a map. The keys are the type name "Foo" (e.g., "Mailbox" or "Email"), and the value is the "state" property that a call to "Foo/get" would return right now 🗂️.

The client can compare the new state strings with its current values to see whether it already has the current data for these types. If not, the changes can then be efficiently fetched in a single standard API request (using the /changes type methods) 🏃‍♀️💨.

7.1.1. Example 🧾

In this example, the server has combined a few changes across two different accounts the user has access to, before pushing the following StateChange object to the client 🍭:

                  {
                    "@type": "StateChange",
                    "changed": {
                      "a3123": {
                        "Email": "d35ecb040aab",
                        "EmailDelivery": "428d565f2440",
                        "CalendarEvent": "87accfac587a"
                      },
                      "a43461d": {
                        "Mailbox": "0af7a512ce70",
                        "CalendarEvent": "7a4297cecd76"
                      }
                    }
                  }

The client can compare the state strings with its current state for the Email, CalendarEvent, and so on, object types in the right accounts, to see if it needs to fetch changes 🔎.

If the client is making changes itself, it may receive a StateChange object while the /set API call is still in flight. It can wait until the call finishes and then check whether the new state string after the /set matches what was pushed in the StateChange object. If it does, and the old state in the /set response matches the client's previous state, then the client doesn't need to waste a request asking for changes it already knows about 🫶✨

7.2. PushSubscription 💖

Okay bestie, clients can make a PushSubscription to register a URL with the JMAP server 🦋. After that, the server just sends an HTTP POST to that URL for every push notification it wants to deliver to the client ✨

ngl this one is a little dangerous 💀 A push subscription makes the server send requests to an endpoint it has never met before, so it could be used as a vector for a denial-of-service attack 🙈 To stop that, the server immediately sends a PushVerification object to that URL when the subscription is created (see Section 7.2.2). The JMAP server MUST NOT make any further requests to the URL until the client receives the push and updates the subscription with the correct verification code 🔒

A PushSubscription object has these properties:

  • id: "Id" (immutable; server-set) 🌸 The id of the push subscription.
  • deviceClientId: "String" (immutable) 🎀 An id that uniquely identifies the client + device it's running on. The point is to let clients spot which PushSubscription objects they made, even if they lose their local state, so they can revoke or update them. This string MUST be different on different devices, and it MUST be different from apps from other vendors. It SHOULD be easy to regenerate and shouldn't depend on persisted state. It is RECOMMENDED to use a secure hash of a string that contains:
    1. A unique identifier associated with the device where the JMAP client is running, normally supplied by the device's operating system.
    2. A custom vendor/app id, including a domain controlled by the vendor of the JMAP client.

To protect the user's privacy, the deviceClientId MUST NOT contain an unobfuscated device id 🫶

  • url: "String" (immutable) 🦋 An absolute URL where the JMAP server will POST the data for the push message. This MUST begin with "https://" 🔐
  • keys: "Object|null" (immutable) 🗝️ Client-generated encryption keys. If they're supplied, the server MUST use them as specified in [RFC8291] to encrypt all data sent to the push subscription. The object MUST have the following properties:
    • p256dh: "String" The P-256 Elliptic Curve Diffie-Hellman (ECDH) public key as described in [RFC8291], encoded in URL-safe base64 representation as defined in [RFC4648].
    • auth: "String" The authentication secret as described in [RFC8291], encoded in URL-safe base64 representation as defined in [RFC4648].
  • verificationCode: "String|null" ✅ This MUST be null (or omitted) when the subscription is created. The JMAP server then generates a verification code and sends it in a push message, and the client updates the PushSubscription object with the code; see Section 7.2.2 for details.
  • expires: "UTCDate|null" ⏰ The time this push subscription expires. If it's specified, the JMAP server MUST NOT make further requests to this resource after this time. It MAY automatically destroy the push subscription at or after this time.

The server MAY choose to set an expiry if the client doesn't give one, or MAY modify the expiry time the client gave to a shorter duration 💅

  • types: "String[]|null" 👑 A list of types the client is interested in (using the same names as the keys in the TypeState object defined in the previous section). A StateChange notification only gets sent if the data for one of these types changes. Other types are left out of the TypeState object. If it's null, changes will be pushed for all types.

The POST request MUST have a content type of "application/json" and contain the UTF-8 JSON-encoded object as the body. The request MUST have a "TTL" header and MAY have "Urgency" and/or "Topic" headers, as specified in Section 5 of [RFC8030]. The JMAP server is expected to understand and handle HTTP status responses in a reasonable manner. A "429" (Too Many Requests) response MUST cause the JMAP server to reduce the frequency of pushes. The JMAP push structure lets multiple changes get coalesced into a single minimal StateChange object, so this is a glow-up for everyone 🌈 For a look at the risks of connecting to unknown servers, check the security considerations in Section 8.6. Seriously, read it 😭

The JMAP server acts as an application server as defined in [RFC8030]. A client MAY use the rest of [RFC8030] together with its own push service to make a complete end-to-end solution, or it MAY rely on alternative mechanisms to make sure the pushed data gets delivered after it leaves the JMAP server 🚀

The push subscription is tied to the credentials used to authenticate the API request that created it. If those credentials expire or get revoked, the push subscription MUST be destroyed by the JMAP server. Only subscriptions created by these credentials get returned when the client fetches existing subscriptions 🧚

When these credentials have their own expiry (i.e., it's a session with a timeout), the server SHOULD NOT set or bound the expiry time for the push subscription given by the client, but it MUST expire the subscription when the session expires ⏳

When these credentials aren't time bounded (e.g., Basic authentication [RFC7617]), the server SHOULD set an expiry time for the push subscription if none is given, and SHOULD limit the expiry time if it's set too far in the future. This maximum expiry time MUST be at least 48 hours in the future and SHOULD be at least 7 days in the future. An app running on a mobile device may only be able to refresh the push subscription lifetime while it's in the foreground, so this gives a reasonable time frame for that to happen 📱💜

In the case of separate access and refresh credentials, as in Oauth 2.0 [RFC6749], the server SHOULD tie the push subscription to the validity of the refresh token rather than the access token, and behave according to whether that token is time-limited or not 🎟️

When a push subscription is destroyed, the server MUST securely erase the URL and encryption keys from memory and storage as soon as possible. This one is SERIOUS bestie, no cap 🚨

7.2.1. PushSubscription/get

This is the standard /get method described in Section 5.1, except it does not take or return an "accountId" argument, because push subscriptions aren't tied to specific accounts. It also does not return a "state" argument. The "ids" argument may be null to fetch everything at once 🍓

The server MUST only return push subscriptions that were created using the same authentication credentials as this "PushSubscription/get" request 🔑

Since the "url" and "keys" properties may contain data that's private to a particular device, the values for these properties MUST NOT be returned. If the "properties" argument is null or omitted, the server MUST default to all properties excluding these two. If one of them is explicitly requested, the method call MUST be rejected with a "forbidden" error 🙅‍♀️

7.2.2. PushSubscription/set

This is the standard /set method described in Section 5.3, except it does not take an "accountId" argument or return one, because push subscriptions aren't tied to specific accounts. It also does not take an "ifInState" argument or return "oldState" or "newState" arguments 💅

The "url" and "keys" properties are immutable. If the client wants to change them, it must destroy the current push subscription and create a new one 🔄

When a PushSubscription is created, the server MUST immediately push a PushVerification object to the URL. It has these properties:

  • @type: "String" This MUST be the string "PushVerification" 🏷️
  • pushSubscriptionId: "String" The id of the push subscription that was created.
  • verificationCode: "String" The verification code to add to the push subscription. This MUST contain sufficient entropy so the client can't guess the code by brute force 🧠

The client MUST update the push subscription with the correct verification code before the server makes any further requests to the subscription's URL. Attempts to update the subscription with an invalid verification code MUST be rejected by the server with an "invalidProperties" SetError 🚫

The client may update the "expires" property to extend (or, less commonly, shorten) the lifetime of a push subscription. The server MAY modify the proposed new expiry time to enforce server-defined limits. Extending the lifetime does not require the subscription to be verified again 🌟

Clients SHOULD NOT update or destroy a push subscription they didn't create (i.e., one with a "deviceClientId" they don't recognise) 🙈

7.2.3. Example

At "2018-07-06T02:14:29Z", a client with deviceClientId "a889-ffea-910" fetches the set of push subscriptions currently on the server, making an API request with:

                       [[ "PushSubscription/get", {
                         "ids": null
                       }, "0" ]]

Which returns:

    [[ "PushSubscription/get", {
      "list": [{
          "id": "e50b2c1d-9553-41a3-b0a7-a7d26b599ee1",
          "deviceClientId": "b37ff8001ca0",
          "verificationCode": "b210ef734fe5f439c1ca386421359f7b",
          "expires": "2018-07-31T00:13:21Z",
          "types": [ "Todo" ]
      }, {
          "id": "f2d0aab5-e976-4e8b-ad4b-b380a5b987e4",
          "deviceClientId": "X8980fc",
          "verificationCode": "f3d4618a9ae15c8b7f5582533786d531",
          "expires": "2018-07-12T05:55:00Z",
          "types": [ "Mailbox", "Email", "EmailDelivery" ]
      }],
      "notFound": []
    }, "0" ]]

Since neither of the returned push subscription objects has the client's deviceClientId, it knows there's no current push subscription active on the server for it. So it makes one, sending this request:

[[ "PushSubscription/set", {
  "create": {
    "4f29": {
      "deviceClientId": "a889-ffea-910",
      "url": "https://example.com/push/?device=X8980fc&client=12c6d086",
      "types": null
    }
  }
}, "0" ]]

Okay so the server makes the push subscription but caps the expiry at 7 days from now 🕰️ 💅 (no, not forever, bestie). Here's its reply:

[[ "PushSubscription/set", {
  "created": {
    "4f29": {
      "id": "P43dcfa4-1dd4-41ef-9156-2c89b3b19c60",
      "keys": null,
      "expires": "2018-07-13T02:14:29Z"
    }
  }
}, "0" ]]

The server also immediately POSTs to https://example.com/push/?device=X8980fc&client=12c6d086 with this data 📨:

{
  "@type": "PushVerification",
  "pushSubscriptionId": "P43dcfa4-1dd4-41ef-9156-2c89b3b19c60",
  "verificationCode": "da1f097b11ca17f06424e30bf02bfa67"
}

The client gets this and updates the subscription with the verification code. ⚠️ Heads up, there's a potential race condition here 😭, so the client MUST be able to handle getting the push while the request that creates the subscription is still in progress 🫶:

[[ "PushSubscription/set", {
  "update": {
    "P43dcfa4-1dd4-41ef-9156-2c89b3b19c60": {
      "verificationCode": "da1f097b11ca17f06424e30bf02bfa67"
    }
  }
}, "0" ]]

The server confirms the update worked, and now it'll hit the registered URL whenever the state changes 🎀✨.

Two days later, the client wants to extend the subscription's life, so it sends this 🌸:

[[ "PushSubscription/set", {
  "update": {
    "P43dcfa4-1dd4-41ef-9156-2c89b3b19c60": {
      "expires": "2018-08-13T00:00:00Z"
    }
  }
}, "0" ]]

The server extends the expiry, but again only up to its max of 7 days from now 🙈, so it replies with:

[[ "PushSubscription/set", {
  "updated": {
    "P43dcfa4-1dd4-41ef-9156-2c89b3b19c60": {
      "expires": "2018-07-15T02:22:50Z"
    }
  }
}, "0" ]]

7.3. Event Source 🦋

Clients that can keep transport connections open can connect straight to the JMAP server and get push notifications over a text/event-stream resource, as described in [EventSource] 🧚. It's a long-running HTTP request where the server keeps shoving data at the client by adding to the response without ever ending it. Iconic behaviour, honestly 😭.

When something changes in the data on the server, it pushes an event called state to every connected client, with the StateChange object as the data 💜.

The server SHOULD also send a new event id that encodes the entire server state visible to the user right after it sends a state event 🌈. When a new connection is made to the event-source endpoint, a client that follows the server-sent events spec sends a Last-Event-ID HTTP header field with the last id it saw. The server can use that to figure out whether the client missed some changes. If it did, the server SHOULD send those changes right away on connection 🔥.

The Session object (see Section 2) has an eventSourceUrl property, which is in URI Template (level 1) format [RFC6570]. The URL MUST contain variables called types, closeafter, and ping 🍓.

To connect, the client makes an authenticated GET request to the event-source URL with those variables filled in 💅:

  • "types": This MUST be either:
    • A comma-separated list of type names, e.g., "Email,CalendarEvent". The server MUST only push changes for the types in this list.
    • The single character: "*". Changes to all types are pushed.
  • "closeafter": This MUST be one of the following values:
    • "state": The server MUST end the HTTP response after pushing a state event. This can be used by clients in environments where buffering proxies stop the pushed data from arriving immediately, or indeed at all, when operating in the usual mode 🙃.
    • "no": The connection is persisted by the server as a standard event-source resource.
  • "ping": A positive integer value representing a length of time in seconds, e.g., "300". If it's non-zero, the server MUST send an event called "ping" whenever this time elapses since the previous event was sent. This MUST NOT set a new event id. If the value is "0", the server MUST NOT send ping events 🫠.

The server MAY modify a requested ping interval so it's subject to a minimum and/or maximum value. For interoperability, servers MUST NOT have a minimum allowed value higher than 30 or a maximum allowed value less than 300 🎀.

The data for the ping event MUST be a JSON object containing an "interval" property, with the value (type "UnsignedInt") being the interval in seconds the server is using to send pings. This might be different from the requested value if the server clamped it to be within a min/max value 👑.

Clients can watch for the ping event to help figure out when closeafter mode might be needed ✨.

A client MAY hold open multiple connections to the event-source resource, although it SHOULD try to use a single connection for efficiency 🧚💜.

8. Security Considerations 🚨

8.1. Transport Confidentiality 🔐

To keep data sent and received via JMAP confidential and intact, all requests MUST use TLS 1.2 [RFC5246] [RFC8446] or later, following the recommendations in [RFC7525] 🛡️. Servers SHOULD support TLS 1.3 [RFC8446] or later.

Clients MUST validate TLS certificate chains to protect against man-in-the-middle attacks [RFC5280] 😤.

8.2. Authentication Scheme 🗝️

A number of HTTP authentication schemes have been standardised (see https://www.iana.org/assignments/http-authschemes/). Servers should take care to assess the security characteristics of different schemes in relation to their needs when deciding what to implement 🤔.

Use of the Basic authentication scheme is NOT RECOMMENDED 💀. Services that choose to use it are strongly recommended to require generation of a unique "app password" via some external mechanism for each client they want to connect. This lets the server tell connections from different devices apart and revoke access one device at a time 🙅‍♀️.

8.3. Service Autodiscovery 🕵️‍♀️

Unless it's secured by something like DNSSEC, autodiscovering server details with SRV DNS records is vulnerable to a DNS poisoning attack 😱. That can lead to the client talking to an attacker's server instead of the real JMAP server. The attacker could then intercept requests to run man-in-the-middle attacks and, depending on the authentication scheme, steal credentials to make its own requests.

Clients that don't support SRV lookups will probably just try the /.well-known/jmap path directly against the domain of the username over HTTPS. Servers SHOULD ensure this path resolves or redirects to the correct JMAP Session resource so this works. If that isn't feasible, servers MUST ensure this path can't be controlled by an attacker, because again it could be used to steal credentials 🚫.

8.4. JSON Parsing 🧾

The Security Considerations of [RFC8259] apply to the use of JSON as the data interchange format 📜.

As with any serialization format, parsers need to thoroughly check the syntax of the supplied data. JSON uses opening and closing tags for several types and structures, and it's possible that the end of the supplied data gets reached while scanning for a matching closing tag. That's an error condition, and implementations need to stop scanning at the end of the supplied data 🛑.

JSON also uses a string encoding with some escape sequences to encode special characters within a string. Care is needed when processing these escape sequences, so they're fully formed before the special processing kicks in. Be extra careful when escape sequences sit next to other (non-escaped) special characters or next to the end of the data (like in the previous paragraph) 🧐.

If you're parsing JSON into a non-textual structured data format, implementations may need to allocate storage to hold JSON string elements. JSON doesn't use explicit string lengths, so the risk of denial of service from resource exhaustion is small. Implementations may still want to set limits on the size of allocations they're willing to make in a given context, so untrusted data can't cause excessive memory allocation 🍰.

8.5. Denial of Service 🌪️

A small request can lead to a very large response and take a lot of work on the server if resource limits aren't enforced. JMAP has ways to advertise and enforce a wide variety of limits to mitigate this, including limits on the number of objects fetched in a single method call, the number of methods in a single request, the number of concurrent requests, and so on 🧱.

JMAP servers MUST implement sensible limits to mitigate against resource exhaustion attacks 🙅‍♀️.

8.6. Connection to Unknown Push Server 🚩

When a push subscription is registered, the application server will make POST requests to the given URL. There are a bunch of security considerations that MUST be considered when implementing this 😭.

The server MUST ensure the URL is externally resolvable to avoid server-side request forgery, where the server makes a request to a resource on its own internal network 🏠.

A malicious client could use the push subscription to flood a third-party server with requests. That's a denial-of-service attack, and it masks the attacker's true identity 🥷. There's no guarantee that the URL given to the JMAP server is actually a valid push server. When a push subscription is created, the JMAP server sends a PushVerification object to the URL and MUST NOT send any further requests until the client verifies it has received the initial push. The verification code MUST contain sufficient entropy to stop the client from being verified by brute force 🔑.

The verification code doesn't guarantee the URL is a valid push server, only that the client can access the data submitted to it. Verification does shrink the set of potential targets a lot, but there's still a risk that the server is unrelated to the client and is being targeted for a denial-of-service attack 😬.

The server MUST limit the number of push subscriptions any one user may have, so the user can't make the server send a huge number of push notifications at once. That could also be used as part of a denial-of-service attack 💥. The rate of creation MUST also be limited to minimise the ability to abuse the verification request as an attack vector 🚧.

8.7. Push Encryption 🔒

When data changes, a small object gets pushed with the new state strings for the types that changed. The data here is minimal, but a passive man-in-the-middle attacker may still be able to gain useful information 👀. To ensure confidentiality and integrity, if the push is sent via a third party outside the control of the client and JMAP server, the client MUST specify encryption keys when establishing the PushSubscription. It must also ignore any push notification that isn't encrypted with those keys 🫶.

The privacy and security considerations of [RFC8030] and [RFC8291] also apply to the use of the PushSubscription mechanism 🧁.

Since there's no crypto algorithm agility in Web Push Encryption [RFC8291], a new specification will be needed to provide this if new algorithms are required in the future 🔮.

8.8. Traffic Analysis 👀

Okay so even though everything is encrypted 🔐, a sneaky passive observer who can just watch network traffic might still learn stuff from the timing of API requests and push notifications. Picture this: an email or calendar invite gets sent from User A (hosted on Server X) to User B (hosted on Server Y). If Server X hosts data for a TON of users, the observer can see that the two servers connected, but they still don't know who the data was actually for 🤷. BUT, if a push notification goes to User B right away and the attacker sees that too, they may reasonably conclude that someone on Server X is connecting to User B. Iykyk, timing says a lot 💀

9. IANA Considerations 📋

9.1. Assignment of jmap Service Name

IANA has assigned the 'jmap' service name in the "Service Name and Transport Protocol Port Number Registry" [RFC6335] 🎀. Here's the entry:

Service Name: jmap

Transport Protocol(s): tcp

Assignee: IESG

Contact: IETF Chair

Description: JSON Meta Application Protocol

Reference: RFC 8620

Assignment Notes: This service name was previously assigned under the
name "JSON Mail Access Protocol".  This has been de-assigned and
re-assigned with the approval of the previous assignee.

The name used to belong to "JSON Mail Access Protocol", but it got de-assigned and then re-assigned with the previous assignee's approval. Very tidy 🫶.

9.2. Registration of Well-Known URI Suffix for JMAP

IANA registered this suffix in the "Well-Known URIs" registry for JMAP, as described in [RFC8615] ✨:

URI Suffix: jmap

Change Controller: IETF

Specification Document: RFC 8620, Section 2.2.

9.3. Registration of the jmap URN Sub-namespace

IANA also registered a URN sub-namespace in the "IETF URN Sub-namespace for Registered Protocol Parameter Identifiers" registry, which lives inside the "Uniform Resource Name (URN) Namespace for IETF Use" registry, as described in [RFC3553] 🌈:

Registered Parameter Identifier: jmap

Reference: RFC 8620, Section 9.4

IANA Registry Reference: http://www.iana.org/assignments/jmap

9.4. Creation of "JMAP Capabilities" Registry

IANA created the "JMAP Capabilities" registry, as described in Section 2 🔥. Capabilities get advertised in the "capabilities" property of the JMAP Session resource, and they're how a JMAP server gets its functionality extended 👑. Each capability is referenced by a URI. That URI can be a URN that starts with "urn:ietf:params:jmap:" plus a unique suffix, which is the index value in the jmap URN sub-namespace. Registering a capability with some other kind of URI has no impact on the jmap URN sub-namespace 🙅.

This registry follows the expert review process, unless the "intended use" field is "common" or "placeholder". In that case, registration follows the specification required process.

A JMAP capability registration can have an intended use of "common", "placeholder", "limited", or "obsolete". IANA lists common-use registrations prominently and separately from the ones with other intended use values 🌸.

The JMAP capability registration procedure is not a formal standards process. It's more of an administrative thing, meant to get community comment and a sanity check without a ton of waiting ⏳.

A "placeholder" registration reserves part of the jmap URN namespace for another purpose. It's typically not included in the "capabilities" property of the JMAP Session resource 🧚.

9.4.1. Preliminary Community Review

Notice of a potential JMAP common-use registration SHOULD be sent to the JMAP mailing list <jmap@ietf.org> for review 📬. That list is the right place to solicit community feedback on a proposed JMAP capability. Registrations that are not intended for common use MAY be sent to the list for review as well. Doing so is entirely OPTIONAL, but it's encouraged 💖.

The whole point of posting publicly is to collect comments and feedback on the choice of capability name, whether the specification document is unambiguous, and any interoperability or security considerations 🔍. The submitter may submit a revised registration proposal or abandon the registration completely at any time 🫶.

9.4.2. Submit Request to IANA

Registration requests can be sent to <iana@iana.org> 💌.

9.4.3. Designated Expert Review

For a limited-use registration, the main thing the designated expert (DE) cares about is preventing name collisions and encouraging the submitter to document security and privacy considerations. A published specification is not required for that 🧐. For a common-use registration, the DE is expected to confirm that suitable documentation, as described in Section 4.6 of [RFC8126], is available. The DE should also verify that the capability doesn't conflict with work that is active or already published within the IETF.

Before a period of 30 days has passed, the DE will either approve or deny the registration request. They'll publish a notice of the decision to the JMAP WG mailing list or its successor, and inform IANA too. A denial notice must be justified with an explanation. Where it's possible, concrete suggestions on how to modify the request so it becomes acceptable should be provided ✏️.

If the DE doesn't respond within 30 days, the registrant may request that the IESG take action, so the request gets processed in a timely manner ⏰.

9.4.4. Change Procedures

Once a JMAP capability has been published by IANA, the change controller may request a change to its definition. The same procedure that would be appropriate for the original registration request is used to process a change request 🔄.

JMAP capability registrations may not be deleted. Capabilities that are no longer believed appropriate for use can be declared obsolete by changing their "intended use" field, and they'll be clearly marked in the lists IANA publishes 🪦.

Significant changes to a capability's definition should be requested only when there are serious omissions or errors in the published specification. When review is required, a change request may be denied if it makes entities that were valid under the previous definition invalid under the new one 🚫.

The owner of a JMAP capability can pass responsibility to another person or agency by informing IANA. This can be done without discussion or review 🤝.

The IESG may reassign responsibility for a JMAP capability. The most common case is when the registration's author has died, moved out of contact, or is otherwise unable to make changes that matter to the community. That's the situation this is built for 🕊️.

9.4.5. JMAP Capabilities Registry Template

Here's the template:

Capability name: (see capability property in Section 2)

Specification document:

Intended use: (one of common, limited, placeholder, or obsolete)

Change controller: ("IETF" for Standards Track / BCP RFCs)

Security and privacy considerations:

9.4.6. Initial Registration for JMAP Core

The very first registration, for JMAP Core, looks like this 🍓:

Capability Name: "urn:ietf:params:jmap:core"

Specification document: RFC 8620, Section 2

Intended use: common

Change Controller: IETF

Security and privacy considerations: RFC 8620, Section 8.

9.4.7. Registration for JMAP Error Placeholder in JMAP Capabilities Registry

This one reserves the error placeholder in the JMAP Capabilities registry 🧸:

Capability Name: "urn:ietf:params:jmap:error:"

Specification document: RFC 8620, Section 9.5

Intended use: placeholder

Change Controller: IETF

Security and privacy considerations: RFC 8620, Section 8.

9.5. Creation of "JMAP Error Codes" Registry

IANA created the "JMAP Error Codes" registry 🚨. JMAP error codes show up in the "type" member of a JSON problem details object (as described in Section 3.6.1), in the "type" member of a JMAP error object (as described in Section 3.6.2), or in the "type" member of a JMAP method-specific error object (such as SetError in Section 5.3). When an error code is used in a problem details object, the prefix "urn:ietf:params:jmap:error:" is always included. When it's used in JMAP objects, the prefix is always omitted 🎯.

This registry follows the expert review process. Preliminary community review for this registry follows the same procedures as the "JMAP Capabilities" registry, but it is optional. The change procedures are the same as the change procedures for the "JMAP Capabilities" registry 🔁.

9.5.1. Expert Review

The designated expert should review the following aspects of the registration 🔎:

  1. Check that the error code doesn't conflict with existing names.
  2. Check that the error code follows the syntax limitations, so it doesn't require URI encoding.
  3. Encourage the submitter to follow the naming convention of previously registered errors.
  4. Encourage the submitter to describe the client behaviours recommended in response to the error code. These may help distinguish it from other error codes.
  5. Encourage the submitter to describe when the server should issue this error instead of some other error code.
  6. Encourage the submitter to note any security considerations associated with the error, if any (e.g., an error code that might disclose the existence of data the authenticated user doesn't have permission to know about) 🔐.

Steps 3-6 are meant to promote a higher-quality registry. However, the expert is encouraged to approve any registration that would not actively harm JMAP interoperability, so the whole process stays relatively lightweight 💅.

9.5.2. JMAP Error Codes Registry Template

Here's the template for new error codes 📝:

JMAP Error Code:

Intended use: (one of "common", "limited", "obsolete")

Change Controller: ("IETF" for Standards Track / BCP RFCs)

Reference: (Optional.  Only required if defined in an RFC.)

Description:

Okay bestie, this is the big list of starter error codes for the registry, and every single one is going in exactly as written 💅🎀

9.5.3. Initial Contents for the JMAP Error Codes Registry ✨

   o  JMAP Error Code: accountNotFound
      Intended Use: Common
      Change Controller: IETF
      Reference: RFC 8620, Section 3.6.2
      Description: The accountId does not correspond to a valid account.

   o  JMAP Error Code: accountNotSupportedByMethod
      Intended Use: Common
      Change Controller: IETF
      Reference: RFC 8620, Section 3.6.2
      Description: The accountId given corresponds to a valid account,
      but the account does not support this method or data type.

   o  JMAP Error Code: accountReadOnly
      Intended Use: Common
      Change Controller: IETF
      Reference: RFC 8620, Section 3.6.2
      Description: This method modifies state, but the account is read-
      only (as returned on the corresponding Account object in the JMAP
      Session resource).

   o  JMAP Error Code: anchorNotFound
      Intended Use: Common
      Change Controller: IETF
      Reference: RFC 8620, Section 5.5
      Description: An anchor argument was supplied, but it cannot be
      found in the results of the query.

   o  JMAP Error Code: alreadyExists
      Intended Use: Common
      Change Controller: IETF
      Reference: RFC 8620, Section 5.4
      Description: The server forbids duplicates, and the record already
      exists in the target account.  An existingId property of type Id
      MUST be included on the SetError object with the id of the
      existing record.

   o  JMAP Error Code: cannotCalculateChanges
      Intended Use: Common
      Change Controller: IETF
      Reference: RFC 8620, Sections 5.2 and 5.6
      Description: The server cannot calculate the changes from the
      state string given by the client.

   o  JMAP Error Code: forbidden
      Intended Use: Common
      Change Controller: IETF
      Reference: RFC 8620, Sections 3.6.2, 5.3, and 7.2.1
      Description: The action would violate an ACL or other permissions
      policy.

   o  JMAP Error Code: fromAccountNotFound
      Intended Use: Common
      Change Controller: IETF
      Reference: RFC 8620, Sections 5.4 and 6.3
      Description: The fromAccountId does not correspond to a valid
      account.

   o  JMAP Error Code: fromAccountNotSupportedByMethod
      Intended Use: Common
      Change Controller: IETF
      Reference: RFC 8620, Section 5.4
      Description: The fromAccountId given corresponds to a valid
      account, but the account does not support this data type.

   o  JMAP Error Code: invalidArguments
      Intended Use: Common
      Change Controller: IETF
      Reference: RFC 8620, Section 3.6.2
      Description: One of the arguments is of the wrong type or
      otherwise invalid, or a required argument is missing.

   o  JMAP Error Code: invalidPatch
      Intended Use: Common
      Change Controller: IETF
      Reference: RFC 8620, Section 5.3
      Description: The PatchObject given to update the record was not a
      valid patch.

   o  JMAP Error Code: invalidProperties
      Intended Use: Common
      Change Controller: IETF
      Reference: RFC 8620, Section 5.3
      Description: The record given is invalid.

   o  JMAP Error Code: notFound
      Intended Use: Common
      Change Controller: IETF
      Reference: RFC 8620, Section 5.3
      Description: The id given cannot be found.

   o  JMAP Error Code: notJSON
      Intended Use: Common
      Change Controller: IETF
      Reference: RFC 8620, Section 3.6.1
      Description: The content type of the request was not application/
      json, or the request did not parse as I-JSON.

   o  JMAP Error Code: notRequest
      Intended Use: Common
      Change Controller: IETF
      Reference: RFC 8620, Section 3.6.1
      Description: The request parsed as JSON but did not match the type
      signature of the Request object.

   o  JMAP Error Code: overQuota
      Intended Use: Common
      Change Controller: IETF
      Reference: RFC 8620, Section 5.3
      Description: The create would exceed a server-defined limit on the
      number or total size of objects of this type.

   o  JMAP Error Code: rateLimit
      Intended Use: Common
      Change Controller: IETF
      Reference: RFC 8620, Section 5.3
      Description: Too many objects of this type have been created
      recently, and a server-defined rate limit has been reached.  It
      may work if tried again later.

   o  JMAP Error Code: requestTooLarge
      Intended Use: Common
      Change Controller: IETF
      Reference: RFC 8620, Sections 5.1 and 5.3
      Description: The total number of actions exceeds the maximum
      number the server is willing to process in a single method call.

   o  JMAP Error Code: invalidResultReference
      Intended Use: Common
      Change Controller: IETF
      Reference: RFC 8620, Section 3.6.2
      Description: The method used a result reference for one of its
      arguments, but this failed to resolve.

   o  JMAP Error Code: serverFail
      Intended Use: Common
      Change Controller: IETF
      Reference: RFC 8620, Section 3.6.2
      Description: An unexpected or unknown error occurred during the
      processing of the call.  The method call made no changes to the
      server's state.

   o  JMAP Error Code: serverPartialFail
      Intended Use: Limited
      Change Controller: IETF
      Reference: RFC 8620, Section 3.6.2
      Description: Some, but not all, expected changes described by the
      method occurred.  The client MUST resynchronise impacted data to
      determine the server state.  Use of this error is strongly
      discouraged.

   o  JMAP Error Code: serverUnavailable
      Intended Use: Common
      Change Controller: IETF
      Reference: RFC 8620, Section 3.6.2
      Description: Some internal server resource was temporarily
      unavailable.  Attempting the same operation later (perhaps after a
      backoff with a random factor) may succeed.

   o  JMAP Error Code: singleton
      Intended Use: Common
      Change Controller: IETF
      Reference: RFC 8620, Section 5.3
      Description: This is a singleton type, so you cannot create
      another one or destroy the existing one.

   o  JMAP Error Code: stateMismatch
      Intended Use: Common
      Change Controller: IETF
      Reference: RFC 8620, Section 5.3
      Description: An ifInState argument was supplied, and it does not
      match the current state.

   o  JMAP Error Code: tooLarge
      Intended Use: Common
      Change Controller: IETF
      Reference: RFC 8620, Section 5.3
      Description: The action would result in an object that exceeds a
      server-defined limit for the maximum size of a single object of
      this type.

   o  JMAP Error Code: tooManyChanges
      Intended Use: Common
      Change Controller: IETF
      Reference: RFC 8620, Section 5.6
      Description: There are more changes than the client's maxChanges
      argument.

   o  JMAP Error Code: unknownCapability
      Intended Use: Common
      Change Controller: IETF
      Reference: RFC 8620, Section 3.6.1
      Description: The client included a capability in the "using"
      property of the request that the server does not support.

   o  JMAP Error Code: unknownMethod
      Intended Use: Common
      Change Controller: IETF
      Reference: RFC 8620, Section 3.6.2
      Description: The server does not recognise this method name.

   o  JMAP Error Code: unsupportedFilter
      Intended Use: Common
      Change Controller: IETF
      Reference: RFC 8620, Section 5.5
      Description: The filter is syntactically valid, but the server
      cannot process it.

   o  JMAP Error Code: unsupportedSort
      Intended Use: Common
      Change Controller: IETF
      Reference: RFC 8620, Section 5.5
      Description: The sort is syntactically valid but includes a
      property the server does not support sorting on or a collation
      method it does not recognise.

   o  JMAP Error Code: willDestroy
      Intended Use: Common
      Change Controller: IETF
      Reference: RFC 8620, Section 5.3
      Description: The client requested an object be both updated and
      destroyed in the same /set request, and the server has decided to
      therefore ignore the update.

10. References 📚

10.1. Normative References 📚✨

These are the must-reads, bestie. Anything in here that the spec leans on is in this list. 💅

10.2. Informative References 🦋

These one's extra background reading, so it's not strictly required to get the spec, but it's still a good one to know about.

Authors' Addresses 🌸

Meet the people who wrote this whole thing! 🫶

Neil Jenkins
Fastmail
PO Box 234, Collins St. West
Melbourne, VIC  8007
Australia

Email: neilj@fastmailteam.com
URI:   https://www.fastmail.com


Chris Newman
Oracle
440 E. Huntington Dr., Suite 400
Arcadia, CA  91006
United States of America

Email: chris.newman@oracle.com
📦 where this page comes from