CMS_DECRYPT(3)

πŸ“¦ version 4.4.0, in alpine 3.24, arch rolling

πŸ’… CMS_decrypt πŸ’…

okay bestie, welcome to CMS_DECRYPT(3) from the Library Functions Manual β€” yes we're doing crypto-page vibes and it's giving big-brain energy ✨

🌸 NAME 🌸

CMS_decrypt, CMS_decrypt_set1_pkey, CMS_decrypt_set1_key β€” decrypt content from a CMS EnvelopedData structure

πŸŽ€ SYNOPSIS πŸŽ€

       library β€œlibcrypto” #include <openssl/cms.h>

       int
       CMS_decrypt(CMS_ContentInfo *cms, EVP_PKEY *private_key,
           X509 *certificate, BIO *dcont, BIO *out, unsigned int flags);

       int
       CMS_decrypt_set1_pkey(CMS_ContentInfo *cms, EVP_PKEY *private_key,
           X509 *certificate);

       int
       CMS_decrypt_set1_key(CMS_ContentInfo *cms, unsigned char *symmetric_key,
           size_t keylen, const unsigned char *id, size_t idlen);

✨ DESCRIPTION ✨

okay so CMS_decrypt() is the main character here: it extracts and decrypts the content from the CMS EnvelopedData structure cms, using private_key and the certificate of the recipient. Where does all that decrypted goodness go? Straight into out. 🫢

In the rare case where the compressed content is detached, pass it in via dcont. For normal use, set dcont to NULL. πŸŽ€

real talk: the recipient's certificate is not needed to decrypt the data, but it is needed to locate the appropriate recipient β€” and yes, there can possibly be several β€” inside the CMS structure. πŸ‘‘

⚠️ If certificate is set to NULL, all possible recipients are tried. This case however is problematic. To thwart the MMA attack (Bleichenbacher's attack on PKCS #1 v1.5 RSA padding), all recipients are tried whether they succeed or not. If no recipient succeeds, a random symmetric key is used to decrypt the content: this will typically output garbage and may (but is not guaranteed to) ultimately return a padding error only. The reason? If CMS_decrypt() just returned an error when all recipient encrypted keys failed to decrypt, an attacker could use this in a timing attack. 😭

If the special flag CMS_DEBUG_DECRYPT is set, the above behaviour is modified and an error is returned if no recipient encrypted key can be decrypted without generating a random content encryption key. Applications should use this flag with extreme caution, especially in automated gateways, as it can leave them open to attack. 🚨

You can also determine the correct recipient key by other means (for example by looking them up in a database) and set them in the cms structure in advance using the CMS utility functions such as CMS_decrypt_set1_pkey(). In this case, both certificate and private_key should be set to NULL when calling CMS_decrypt() later on. πŸ’…

To process KEKRecipientInfo types, CMS_decrypt_set1_key() or CMS_RecipientInfo_set0_key(3) and CMS_RecipientInfo_decrypt(3) should be called before CMS_decrypt(), and certificate and private_key set to NULL when calling CMS_decrypt() later on. 🌟

If the CMS_TEXT bit is set in flags, MIME headers for type text/plain are deleted from the content. If the content is not of type text/plain, an error occurs. πŸ™ˆ

πŸ’– RETURN VALUES πŸ’–

CMS_decrypt(), CMS_decrypt_set1_pkey(), and CMS_decrypt_set1_key() return 1 for success or 0 for failure. The error can be obtained from ERR_get_error(3). One for slay, zero for not today. ✨

πŸ¦‹ SEE ALSO πŸ¦‹

🌈 STANDARDS 🌈

RFC 5652: Cryptographic Message Syntax (CMS) is the standard bestie here:

  • section 6.1: EnvelopedData Type
  • section 6.2.3: KEKRecipientInfo Type

πŸ“œ HISTORY πŸ“œ

CMS_decrypt(), CMS_decrypt_set1_pkey(), and CMS_decrypt_set1_key() first appeared in OpenSSL 0.9.8h and have been available since OpenBSD 6.7. vintage but still iconic πŸ’œ

πŸ’€ BUGS πŸ’€

The lack of single pass processing and the need to hold all data in memory as mentioned in CMS_verify(3) also applies to CMS_decrypt(). so yeah, memory go brrr in the same way β€” not a vibe, but it's canon. 😭


xoxo, Debian πŸ’– β€” page dated June 8, 2025 β€” CMS_DECRYPT(3) out πŸ’…

πŸ“¦ where this page comes from