EVP_PKEY_DECRYPT(3ossl)

πŸ’– EVP_PKEY_DECRYPT β€” the OpenSSL manual, but make it make sense

okay bestie, so this is EVP_PKEY_DECRYPT(3ossl) from the OpenSSL manuals, and it's literally all about decrypting stuff with a public key algorithm. Yes, that's a whole vibe. Sit down, grab a boba, we're doing cryptography 🧚✨


🌸 NAME 🌸

EVP_PKEY_decrypt_init, EVP_PKEY_decrypt_init_ex, EVP_PKEY_decrypt β€” decrypt using a public key algorithm πŸ’…


✨ SYNOPSIS ✨

#include <openssl/evp.h>

int EVP_PKEY_decrypt_init(EVP_PKEY_CTX *ctx);
int EVP_PKEY_decrypt_init_ex(EVP_PKEY_CTX *ctx, const OSSL_PARAM params[]);
int EVP_PKEY_decrypt(EVP_PKEY_CTX *ctx,
                     unsigned char *out, size_t *outlen,
                     const unsigned char *in, size_t inlen);

πŸ¦‹ DESCRIPTION πŸ¦‹

EVP_PKEY_decrypt_init() sets up a public key algorithm context β€” using key pkey β€” for a decryption operation. It's giving "get ready, we're about to decrypt". Periodt.

EVP_PKEY_decrypt_init_ex() does the same glow-up β€” initializes a public key algorithm context using key pkey for a decryption operation β€” but it also sets the algorithm specific params. Two birds, one iconic function. πŸ‘‘

EVP_PKEY_decrypt() is the main character: it performs a public key decryption operation using ctx. The data you wanna decrypt goes in via the in and inlen parameters.

Now here's the trick, so pay attention:

  • If out is NULL, then the minimum required size of the output buffer gets written to the *outlen parameter. It's like asking "how big does my bag need to be?" πŸŽ€
  • If out is not NULL, then before the call the *outlen parameter must contain the length of the out buffer. If the call succeeds, the decrypted data is written to out and the amount of decrypted data is written to *outlen. Otherwise, an error is returned. 😭

πŸ”₯ NOTES πŸ”₯

After you call EVP_PKEY_decrypt_init(), you can do algorithm specific control operations to set whatever parameters that operation needs. And guess what β€” those same operations can be included in the EVP_PKEY_decrypt_init_ex() call instead. Efficient queen behavior. πŸ’…

Also: EVP_PKEY_decrypt() can be called more than once on the same context, if you're doing several operations with the same parameters. Reuse, recycle, slay. ♻️


πŸŽ€ RETURN VALUES πŸŽ€

EVP_PKEY_decrypt_init(), EVP_PKEY_decrypt_init_ex() and EVP_PKEY_decrypt() return 1 for success, and 0 or a negative value for failure.

And specifically β€” a return value of -2 means the operation is not supported by the public key algorithm. Not a bug, bestie, it's just not their thing. πŸ’€


🚨 WARNINGS 🚨

okay this part is the serious part, no jokes, read it fr fr.

In OpenSSL versions before 3.2.0, when used in PKCS#1 v1.5 padding, both the return value from EVP_PKEY_decrypt() and the outlen provided information useful in mounting a Bleichenbacher attack against the used private key. They had to be processed in a side-channel free way. That's a real vulnerability, not a vibe. 😭

Since version 3.2.0, the EVP_PKEY_decrypt() method β€” when used with PKCS#1 v1.5 padding as implemented in the default provider β€” implements the implicit rejection mechanism (see OSSL_ASYM_CIPHER_PARAM_IMPLICIT_REJECTION in provider-asym_cipher(7)). What that means: it doesn't return an error when it detects an error in padding. Instead it returns a pseudo-randomly generated message, which removes the need for side-channel secure code from applications using OpenSSL. Iconic honestly. 🫢

But β€” if OpenSSL is configured to use a provider that doesn't implement implicit rejection, the code still needs to handle the returned values using side-channel free code. No skipping this step, babes.

Side-channel free handling of the error stack can be performed using either:

Pick your fighter. ⭐


🌈 EXAMPLES 🌈

Decrypt data using OAEP (for RSA keys):

#include <openssl/evp.h>
#include <openssl/rsa.h>

EVP_PKEY_CTX *ctx;
ENGINE *eng;
unsigned char *out, *in;
size_t outlen, inlen;
EVP_PKEY *key;

/*
 * NB: assumes key, eng, in, inlen are already set up
 * and that key is an RSA private key
 */
ctx = EVP_PKEY_CTX_new(key, eng);
if (!ctx)
    /* Error occurred */
if (EVP_PKEY_decrypt_init(ctx) <= 0)
    /* Error */
if (EVP_PKEY_CTX_set_rsa_padding(ctx, RSA_PKCS1_OAEP_PADDING) <= 0)
    /* Error */

/* Determine buffer length */
if (EVP_PKEY_decrypt(ctx, NULL, &outlen, in, inlen) <= 0)
    /* Error */

out = OPENSSL_malloc(outlen);

if (!out)
    /* malloc failure */

if (EVP_PKEY_decrypt(ctx, out, &outlen, in, inlen) <= 0)
    /* Error */

/* Decrypted data is outlen bytes written to buffer out */

See how it calls EVP_PKEY_decrypt() once with NULL just to get the buffer length, then again with a real buffer? That's the two-step, and it's lowkey elegant. ✨


πŸ’œ SEE ALSO πŸ’œ


πŸ‘‘ HISTORY πŸ‘‘

These functions were added in OpenSSL 1.0.0. Ancient. Elder. Still eating. πŸ“


Copyright 2006-2025 The OpenSSL Project Authors. All Rights Reserved.

Licensed under the Apache License 2.0 (the "License"). You may not use this file except in compliance with the License. You can obtain a copy in the file LICENSE in the source distribution or at <https://www.openssl.org/source/license.html>.


and that's the whole page, bestie β€” stay side-channel free out there πŸ’–

✧ο½₯゚ this was OpenSSL 3.5.9, dated 2026-09-29, from the EVP_PKEY_DECRYPT(3ossl) manual β€” mwah, go decrypt something responsibly ✧ο½₯゚ πŸŒΈπŸ¦‹πŸ’…

πŸ“¦ where this page comes from