PKCS7_ENCRYPT(3ossl)

πŸ’Œ PKCS7_ENCRYPT (3ossl) β€” bestie's guide to sealing the tea πŸ”

okay so this is the OpenSSL manpage for PKCS7_ENCRYPT, part of the OpenSSL docs (yes, that OpenSSL), and we're about to make envelopedData make sense. sit down, hydrate, let's go 🫢


✨ NAME ✨

PKCS7_encrypt_ex, PKCS7_encrypt β€” create a PKCS#7 envelopedData structure. that's it. that's the job. πŸ’…


🌸 SYNOPSIS 🌸

#include <openssl/pkcs7.h>

PKCS7 *PKCS7_encrypt_ex(STACK_OF(X509) *certs, BIO *in,
                        const EVP_CIPHER *cipher, int flags,
                        OSSL_LIB_CTX *libctx, const char *propq);
PKCS7 *PKCS7_encrypt(STACK_OF(X509) *certs, BIO *in, const EVP_CIPHER *cipher,
                     int flags);

two functions, one glow-up. the _ex one is the newer, more customizable sibling πŸ’–


πŸŽ€ DESCRIPTION πŸŽ€

PKCS7_encrypt_ex() creates and returns a PKCS#7 envelopedData structure. iconic behavior fr.

here's the cast of characters:

  • certs β€” a list of recipient certificates πŸ‘‘
  • in β€” the content you want to encrypt πŸ“
  • cipher β€” the symmetric cipher to use πŸ”
  • flags β€” an optional set of flags 🚩
  • libctx and propq β€” the library context and property query, used when retrieving algorithms from providers ✨

big important note: only RSA keys are supported in PKCS#7 and envelopedData, so every recipient certificate you hand this function must contain an RSA public key. they do not have to be signed using the RSA algorithm though β€” so like, the signing can be whatever, but the key itself has to be RSA. no exceptions bestie 😭

EVP_des_ede3_cbc() (triple DES) is the algorithm of choice for S/MIME use, because basically every client out there supports it. it's giving "safe bet, always invited to the party."

now, some old "export grade" clients (vintage, retro, lowkey ancient) may only support weak encryption using 40 or 64 bit RC2. you can serve those by passing EVP_rc2_40_cbc() and EVP_rc2_64_cbc() respectively. yes they're weak. yes they're still in the API. we don't gatekeep history πŸ’€

the algorithm you pass in the cipher parameter must support ASN1 encoding of its parameters. non-negotiable, it's the rules.

a lot of browsers ship a "sign and encrypt" option, which is honestly just an S/MIME envelopedData holding an S/MIME signed message inside. very nesting-doll of them πŸͺ†. you can make one of these super easily: store the S/MIME signed message in a memory BIO and pass it to PKCS7_encrypt().


🚩 the flags you can pass in flags 🚩

  • if the PKCS7_TEXT flag is set, MIME headers for type text/plain get prepended to the data. cute little header moment.
  • normally the supplied content gets translated into MIME canonical format (as required by the S/MIME specifications). if PKCS7_BINARY is set, no translation occurs. use this one if your supplied data is in binary format, otherwise the translation will corrupt it 😭 β€” and heads up, if PKCS7_BINARY is set, then PKCS7_TEXT is ignored. sorry text, binary ate πŸ’…
  • if the PKCS7_STREAM flag is set, a partial PKCS7 structure is output, suitable for streaming I/O: no data is read from the BIO in. streaming girlies rise 🌊
⚠️ if the flag PKCS7_STREAM is set, the returned PKCS7 structure is not complete, and outputting its contents via a function that does not properly finalize the PKCS7 structure will give unpredictable results. do not play with this one, it will not play nice back πŸ’€

several functions do finalize the structure, including:

  • SMIME_write_PKCS7()
  • i2d_PKCS7_bio_stream()
  • PEM_write_bio_PKCS7_stream()

alternatively, you can finalize it by obtaining the streaming ASN1 BIO directly using BIO_new_PKCS7(). options, options, we love choices ✨

lastly: PKCS7_encrypt() is basically PKCS7_encrypt_ex() but it uses default values of NULL for the library context libctx and the property query propq. the chill, no-extra-args version πŸ¦‹


🫢 RETURN VALUES 🫢

PKCS7_encrypt_ex() and PKCS7_encrypt() return either a PKCS7 structure, or NULL if an error occurred. if it's NULL, you can go get the error from ERR_get_error(3). don't just sit there wondering what went wrong, the info is right there bestie πŸ₯Ί


πŸ‘€ SEE ALSO πŸ‘€

(the whole decrypt-and-encrypt duo, a matched set, obsessed πŸŽ€)


πŸ“œ HISTORY πŸ“œ

  • the function PKCS7_encrypt_ex() was added in OpenSSL 3.0.
  • the PKCS7_STREAM flag was added in OpenSSL 1.0.0.

ancient lore, but we respect it πŸ›οΈ


Copyright 2002-2020 The OpenSSL Project Authors. All Rights Reserved.

Licensed under the Apache License 2.0 (the "License"). You may not use this file except in compliance with the License. You can obtain a copy in the file LICENSE in the source distribution or at <https://www.openssl.org/source/license.html>.


and that's the whole page, no cap 🌈 β€” this was PKCS7_ENCRYPT(3ossl) from OpenSSL 3.5.9, dated 2026-09-29. go forth and encrypt responsibly, my love, and remember: RSA keys only, PKCS7_BINARY for binaries, and always finalize your streams πŸ’…βœ¨πŸ’œ

πŸ“¦ where this page comes from