EVP_PKEY_ENCAPSULATE(3ossl)

📦
version 15.1 · 2 versions
, in freebsd 15.1

💖 EVP_PKEY_ENCAPSULATE(3ossl) — key encapsulation with a KEM algorithm 💖

okay so this is from the OpenSSL manual, section 3ossl, and bestie… it's giving cryptography main character energy ✨. It's all about wrapping up a secret with a public key so only the private-key holder can unwrap it. Let's go 🫶

✨ NAME ✨

EVP_PKEY_encapsulate_init, EVP_PKEY_auth_encapsulate_init, EVP_PKEY_encapsulate — key encapsulation using a KEM algorithm with a public key 💅

🌸 SYNOPSIS 🌸

#include <openssl/evp.h>

int EVP_PKEY_encapsulate_init(EVP_PKEY_CTX *ctx, const OSSL_PARAM params[]);
int EVP_PKEY_auth_encapsulate_init(EVP_PKEY_CTX *ctx, EVP_PKEY *authpriv,
                                  const OSSL_PARAM params[]);
int EVP_PKEY_encapsulate(EVP_PKEY_CTX *ctx,
                         unsigned char *wrappedkey, size_t *wrappedkeylen,
                         unsigned char *genkey, size_t *genkeylen);

🦋 DESCRIPTION 🦋

okay so first up: EVP_PKEY_encapsulate_init() initializes a public key algorithm context ctx for an encapsulation operation, and then sets the params on that context in exactly the same way as calling EVP_PKEY_CTX_set_params(3). Little note: ctx is usually produced using EVP_PKEY_CTX_new_from_pkey(3), where you specify the public key you want to use 🔑

EVP_PKEY_auth_encapsulate_init() is basically the same vibe as EVP_PKEY_encapsulate_init() but it also passes an authpriv authentication private key that gets used during encapsulation. authenticated encapsulation, slay 💫

EVP_PKEY_encapsulate() is the one doing the actual work — it performs a public key encapsulation operation using ctx. The shared secret gets written to genkey, and bestie, that secret can be used as an input for key derivation, typically for various symmetric algorithms 🔥. Its size gets written to genkeylen, which must be initialised to the size of the provided buffer.

The ciphertext written to wrappedkey is an encapsulated form, and it's expected to be only usable by the holder of the private key corresponding to the public key associated with ctx 👀. That ciphertext then gets communicated to the private-key holder, who can use EVP_PKEY_decapsulate(3) to securely recover the same shared secret. matching secrets, iconic 🎀

Now the buffer-size situations, because this part matters fr fr:

  • If wrappedkey is NULL, then the maximum size of the output buffer is written to the *wrappedkeylen parameter — unless wrappedkeylen is NULL — and the maximum size of the generated key buffer is written to *genkeylen unless genkeylen is NULL.
  • If wrappedkey is not NULL and the call succeeds, then the generated shared secret is written to genkey and its size is written to *genkeylen (which must be non-NULL) 💖. The encapsulated ciphertext is written to wrappedkey and its size is written to *wrappedkeylen (must also be non-NULL). The value pointed to by wrappedlen initially holds the size of the unwrapped buffer, so that its size can be validated by the call, ensuring it's large enough to hold the result written to wrapped.
🙈 yes bestie, the page really says wrappedlen and wrapped there — that's literally what's written, I'm not fixing it, no cap.

And here's the big caution, listen up 📢: absent detailed prior knowledge of the internals of the specific KEM algorithm, callers SHOULD NOT assume that the returned shared secret and ciphertext are necessarily of the maximum possible length. The lengths returned via *wrappedkeylen and *genkeylen SHOULD be used to determine the actual lengths of the outputs. don't assume, check the lengths, periodt 💅

🌈 NOTES 🌈

After you call EVP_PKEY_encapsulate_init(), algorithm-specific parameters for the operation may be set or modified using EVP_PKEY_CTX_set_params(3) ⭐

🍓 RETURN VALUES 🍓

EVP_PKEY_encapsulate_init(), EVP_PKEY_auth_encapsulate_init() and EVP_PKEY_encapsulate() all return 1 for success, and 0 or a negative value for failure. And specifically: a return value of -2 means the operation is not supported by the public key algorithm 💀

🧚 EXAMPLES 🧚

Encapsulate an RSASVE key (for RSA keys).

#include <openssl/evp.h>

/*
 * NB: assumes rsa_pub_key is an public key of another party.
 */

EVP_PKEY_CTX *ctx = NULL;
size_t secretlen = 0, outlen = 0;
unsigned char *out = NULL, *secret = NULL;

ctx = EVP_PKEY_CTX_new_from_pkey(libctx, rsa_pub_key, NULL);
if (ctx == NULL)
    /* Error */
if (EVP_PKEY_encapsulate_init(ctx, NULL) <= 0)
    /* Error */

/* Set the mode - only 'RSASVE' is currently supported */
 if (EVP_PKEY_CTX_set_kem_op(ctx, "RSASVE") <= 0)
    /* Error */
/* Determine buffer length */
if (EVP_PKEY_encapsulate(ctx, NULL, &outlen, NULL, &secretlen) <= 0)
    /* Error */

out = OPENSSL_malloc(outlen);
secret = OPENSSL_malloc(secretlen);
if (out == NULL || secret == NULL)
    /* malloc failure */

/*
 * The generated 'secret' can be used as key material.
 * The encapsulated 'out' can be sent to another party who can
 * decapsulate it using their private key to retrieve the 'secret'.
 */
if (EVP_PKEY_encapsulate(ctx, out, &outlen, secret, &secretlen) <= 0)
    /* Error */

👑 SEE ALSO 👑

EVP_PKEY_CTX_new_from_pkey(3), EVP_PKEY_decapsulate(3), EVP_KEM-RSA(7), EVP_KEM-X25519(7), EVP_KEM-EC(7), EVP_KEM-ML-KEM-512(7), EVP_KEM-ML-KEM-768(7), EVP_KEM-ML-KEM-1024(7)

💜 HISTORY 💜

The functions EVP_PKEY_encapsulate_init() and EVP_PKEY_encapsulate() were added in OpenSSL 3.0. The function EVP_PKEY_auth_encapsulate_init() was added in OpenSSL 3.2.

Support for ML-KEM was added in OpenSSL 3.5 🌟

Copyright 2020-2025 The OpenSSL Project Authors. All Rights Reserved.

Licensed under the Apache License 2.0 (the "License"). You may not use this file except in compliance with the License. You can obtain a copy in the file LICENSE in the source distribution or at <https://www.openssl.org/source/license.html>.

okay that's the whole page bestie, you ate that 🫶✨ — this was OpenSSL 3.5.6, dated 2026-04-07, and the page is EVP_PKEY_ENCAPSULATE(3ossl). go encapsulate something iconic 💅🌸

📦 where this page comes from