RSA_PKEY_CTX_CTRL(3)

💖 RSA_PKEY_CTX_CTRL(3) 💖

okay so bestie, this is a Library Functions Manual page and it's literally all about RSA private key control operations ✨ — like, all the little knobs and dials you can turn on an EVP_PKEY_CTX when you're doing RSA stuff. no cap, it's a lot of macros. let's go 🫶

🌸 NAME 🌸

RSA_pkey_ctx_ctrl, EVP_PKEY_CTX_set_rsa_padding, EVP_PKEY_CTX_get_rsa_padding, EVP_PKEY_CTX_set_rsa_keygen_bits, EVP_PKEY_CTX_set_rsa_keygen_pubexp, EVP_PKEY_CTX_set_rsa_mgf1_md, EVP_PKEY_CTX_get_rsa_mgf1_md, EVP_PKEY_CTX_set_rsa_oaep_md, EVP_PKEY_CTX_get_rsa_oaep_md, EVP_PKEY_CTX_set0_rsa_oaep_label, EVP_PKEY_CTX_get0_rsa_oaep_label, EVP_PKEY_CTX_set_rsa_pss_saltlen, EVP_PKEY_CTX_get_rsa_pss_saltlen, EVP_PKEY_CTX_set_rsa_pss_keygen_md, EVP_PKEY_CTX_set_rsa_pss_keygen_mgf1_md, EVP_PKEY_CTX_set_rsa_pss_keygen_saltlen — RSA private key control operations 💅 (yes that is a LOT of names, she's an icon with many aliases 👑)

✨ SYNOPSIS ✨

#include <openssl/rsa.h>

int
RSA_pkey_ctx_ctrl(EVP_PKEY_CTX *ctx, int optype, int cmd, int p1,
    void *p2);

int
EVP_PKEY_CTX_set_rsa_padding(EVP_PKEY_CTX *ctx, int pad);

int
EVP_PKEY_CTX_get_rsa_padding(EVP_PKEY_CTX *ctx, int *ppad);

int
EVP_PKEY_CTX_set_rsa_keygen_bits(EVP_PKEY_CTX *ctx, int mbits);

int
EVP_PKEY_CTX_set_rsa_keygen_pubexp(EVP_PKEY_CTX *ctx, BIGNUM *pubexp);

int
EVP_PKEY_CTX_set_rsa_mgf1_md(EVP_PKEY_CTX *ctx, const EVP_MD *md);

int
EVP_PKEY_CTX_get_rsa_mgf1_md(EVP_PKEY_CTX *ctx, const EVP_MD **pmd);

int
EVP_PKEY_CTX_set_rsa_oaep_md(EVP_PKEY_CTX *ctx, const EVP_MD *md);

int
EVP_PKEY_CTX_get_rsa_oaep_md(EVP_PKEY_CTX *ctx, const EVP_MD **pmd);

int
EVP_PKEY_CTX_set0_rsa_oaep_label(EVP_PKEY_CTX *ctx,
    unsigned char *label, int len);

int
EVP_PKEY_CTX_get0_rsa_oaep_label(EVP_PKEY_CTX *ctx,
    unsigned char **plabel);

int
EVP_PKEY_CTX_set_rsa_pss_saltlen(EVP_PKEY_CTX *ctx, int len);

int
EVP_PKEY_CTX_get_rsa_pss_saltlen(EVP_PKEY_CTX *ctx, int *plen);

int
EVP_PKEY_CTX_set_rsa_pss_keygen_md(EVP_PKEY_CTX *pctx,
    const EVP_MD *md);

int
EVP_PKEY_CTX_set_rsa_pss_keygen_mgf1_md(EVP_PKEY_CTX *pctx,
    const EVP_MD *md);

int
EVP_PKEY_CTX_set_rsa_pss_keygen_saltlen(EVP_PKEY_CTX *pctx,
    int saltlen);

that's the whole squad, assembled 💜

🦋 DESCRIPTION 🦋

so RSA_pkey_ctx_ctrl() is like, a shallow little wrapper around EVP_PKEY_CTX_ctrl(3) — and the way it only succeeds if ctx matches either EVP_PKEY_RSA or EVP_PKEY_RSA_PSS? that's the whole vibe check right there. if it's not one of those two, she's not interested 🙅‍♀️

and here's the plot twist: all the remaining "functions" are implemented as macros. the quotes around "functions" are doing so much work, iykyk 😭

🎀 the padding one

EVP_PKEY_CTX_set_rsa_padding() sets the RSA padding mode for ctx. the pad parameter can take these values, and yes we're listing all of them:

RSA_PKCS1_PADDING — for PKCS#1 padding
RSA_NO_PADDING — for no padding
RSA_PKCS1_OAEP_PADDING — for OAEP padding (encrypt and decrypt only)
RSA_X931_PADDING — for X9.31 padding (signature operations only)
RSA_PKCS1_PSS_PADDING — sign and verify only

and lowkey important: only that last one, RSA_PKCS1_PSS_PADDING, can be used with keys of the type EVP_PKEY_RSA_PSS. don't try the others, she will not cooperate 💀

🌈 two padding modes are the ✨special✨ ones

two RSA padding modes behave differently if EVP_PKEY_CTX_set_signature_md(3) is used — and this part is real, bestie, pay attention:

if this macro is called for PKCS#1 padding, the plaintext buffer is an actual digest value and is encapsulated in a DigestInfo structure according to PKCS#1 when signing, and this structure is expected (and stripped off) when verifying.

if this control is not used with RSA and PKCS#1 padding, then the supplied data is used directly and not encapsulated.

in the case of X9.31 padding for RSA, the algorithm identifier byte is added or checked and removed if this control is called. and if it isn't called? then the first byte of the plaintext buffer is expected to be the algorithm identifier byte. the confidence of that expectation. anyway 🫶

🍓 the rest of the getters and setters

EVP_PKEY_CTX_get_rsa_padding() retrieves the RSA padding mode for ctx. cute, short, does what it says 💅

EVP_PKEY_CTX_set_rsa_keygen_bits() sets the RSA key length for RSA or RSA-PSS key generation to mbits. the smallest supported value is 512 bits, and if you don't specify anything, 1024 bits is used. periodt.

EVP_PKEY_CTX_set_rsa_keygen_pubexp() sets the public exponent value for RSA or RSA-PSS key generation to pubexp. currently, it should be an odd integer. ⚠️ and this is the important bit: the pubexp pointer is used internally by this function, so it should not be modified or freed after the call. like, hands off, she's holding it now 🙈 if this macro is not called, then 65537 is used.

EVP_PKEY_CTX_set_rsa_mgf1_md() sets the MGF1 digest for RSA padding schemes to md. unless explicitly specified, the signing digest is used. the padding mode must have been set to RSA_PKCS1_OAEP_PADDING or RSA_PKCS1_PSS_PADDING. and if the key is of the type EVP_PKEY_RSA_PSS and has usage restrictions, an error occurs if an attempt is made to set the digest to anything other than the restricted value. no sneaking around the restrictions, bestie ✋

EVP_PKEY_CTX_get_rsa_mgf1_md() retrieves the MGF1 digest for ctx. unless explicitly specified, the signing digest is used. the padding mode must have been set to RSA_PKCS1_OAEP_PADDING or RSA_PKCS1_PSS_PADDING. same rules, other direction 🌸

💜 Optimal asymmetric encryption padding

the following macros require that the padding mode was set to RSA_PKCS1_OAEP_PADDING. prerequisites, bestie, we love a prerequisite.

EVP_PKEY_CTX_set_rsa_oaep_md() — sets the message digest type used in RSA OAEP to md. ✨
EVP_PKEY_CTX_get_rsa_oaep_md() — gets the message digest type used in RSA OAEP to *pmd. 👀
EVP_PKEY_CTX_set0_rsa_oaep_label() — sets the RSA OAEP label to label and its length to len. if label is NULL or len is 0, the label is cleared. ⚠️ the library takes ownership of the label so the caller should not free the original memory pointed to by label. she owns it now, it's hers 💅
EVP_PKEY_CTX_get0_rsa_oaep_label() — gets the RSA OAEP label to *plabel. the return value is the label length. the resulting pointer is owned by the library and should not be freed by the caller. again. do not free. i'm serious 😤

🔥 Probabilistic signature scheme

the following macros require that the padding mode was set to RSA_PKCS1_PSS_PADDING. another prerequisite, another slay.

EVP_PKEY_CTX_set_rsa_pss_saltlen() sets the RSA PSS salt length to len. three special values are supported, and they're giving main character energy:

RSA_PSS_SALTLEN_DIGEST — sets the salt length to the digest length ⭐
RSA_PSS_SALTLEN_MAX — sets the salt length to the maximum permissible value 🔥
RSA_PSS_SALTLEN_AUTO — when signing, sets the salt length to the maximum permissible value. when verifying, causes the salt length to be automatically determined based on the PSS block structure 🦋

if this macro is not called, a salt length value of RSA_PSS_SALTLEN_AUTO is used by default. the auto girl is the default girl, we love that for her 💖

ok important caveat time: if the key has usage restrictions and an attempt is made to set the salt length below the minimum value, an error occurs. also, if the key has usage restrictions, RSA_PSS_SALTLEN_AUTO is not supported for verification. no cap, that's a real limitation 😭

EVP_PKEY_CTX_get_rsa_pss_saltlen() retrieves the RSA PSS salt length for ctx. short king. we respect her. 👑

🧚 PSS keygen restrictions

optional parameter restrictions can be specified when generating a PSS key. and here's the catch: if any restrictions are set using the macros described below, then all parameters are restricted. like, all of them. no picking and choosing. for example, setting a minimum salt length also restricts the digest and MGF1 algorithms. the way that one restriction just drags the whole group chat into it 😭

if any restrictions are in place, then they are reflected in the corresponding parameters of the public key when (for example) a certificate request is signed. so everyone can see your settings. transparent queen ✨

EVP_PKEY_CTX_set_rsa_pss_keygen_md() — restricts the digest algorithm the generated key can use to md. 🎀
EVP_PKEY_CTX_set_rsa_pss_keygen_mgf1_md() — restricts the MGF1 algorithm the generated key can use to md. 💜
EVP_PKEY_CTX_set_rsa_pss_keygen_saltlen() — restricts the minimum salt length to saltlen. 🌸

⭐ RETURN VALUES ⭐

these functions return a positive value for success or 0 or a negative value for failure. and specifically — this is the detail everybody forgets — a return value of -2 indicates the operation is not supported by the public key algorithm. that's the "not my department" error code fr fr 💀

the whole friend group. go read them. 🫶

🌈 HISTORY 🌈

lore drop time, bestie 🎀

EVP_PKEY_CTX_set_rsa_padding(), EVP_PKEY_CTX_set_rsa_keygen_bits(), EVP_PKEY_CTX_set_rsa_keygen_pubexp(), and EVP_PKEY_CTX_set_rsa_pss_saltlen() first appeared in OpenSSL 1.0.0 and have been available since OpenBSD 4.9. the OGs 💅
EVP_PKEY_CTX_get_rsa_padding(), EVP_PKEY_CTX_set_rsa_mgf1_md(), EVP_PKEY_CTX_get_rsa_mgf1_md(), and EVP_PKEY_CTX_get_rsa_pss_saltlen() first appeared in OpenSSL 1.0.1 and have been available since OpenBSD 5.3. the glow-up era ✨
EVP_PKEY_CTX_set_rsa_oaep_md(), EVP_PKEY_CTX_get_rsa_oaep_md(), EVP_PKEY_CTX_set0_rsa_oaep_label(), and EVP_PKEY_CTX_get0_rsa_oaep_label() first appeared in OpenSSL 1.0.2 and have been available since OpenBSD 6.7. the OAEP renaissance 🦋
RSA_pkey_ctx_ctrl() first appeared in OpenSSL 1.1.1 and has been available since OpenBSD 6.7. late to the party but literally the wrapper for the whole party 💖

and that's the page, bestie 🫶 this cutie was brought to you by Debian, dated December 6, 2024, and it's filed under RSA_PKEY_CTX_CTRL(3) in the Library Functions Manual — go forth and control your RSA keys responsibly 💅✨

📦 where this page comes from

opensuse-leap 16.0: libressl-devel-doc 4.1.0-bp160.1.1 · OpenSSL
💖 girlified by deepseek-ai/DeepSeek-V4.1-Flash-1017 on 2026-11-04: a language model's rewrite of the original, under its terms

RSA_PKEY_CTX_CTRL(3)