RSA_SET_METHOD(3)

💅 RSA_SET_METHOD(3) — select RSA method 💅

okay bestie, this is the RSA_SET_METHOD(3) page from the Library Functions Manual, and it's literally all about picking which RSA method does the work. No cap, it's a whole little ecosystem of function pointers and flags, so let's get into it 💖

🎀 NAME 🎀

Meet the whole squad: RSA_set_default_method, RSA_get_default_method, RSA_set_method, RSA_get_method, RSA_PKCS1_SSLeay, RSA_flags, and RSA_new_method — their collective job is select RSA method ✨.

🌸 SYNOPSIS 🌸

#include <openssl/rsa.h>

void
RSA_set_default_method(const RSA_METHOD *meth);

const RSA_METHOD *
RSA_get_default_method(void);

int
RSA_set_method(RSA *rsa, const RSA_METHOD *meth);

const RSA_METHOD *
RSA_get_method(const RSA *rsa);

const RSA_METHOD *
RSA_PKCS1_SSLeay(void);

int
RSA_flags(const RSA *rsa);

RSA *
RSA_new_method(ENGINE *engine);

🦋 DESCRIPTION 🦋

An RSA_METHOD object is basically a little bag of function pointers — it contains pointers to the functions used for RSA operations. By default, the internal implementation returned by RSA_PKCS1_SSLeay() is the one in charge. But if you pick another method, alternative implementations such as hardware accelerators may be used 👑.

• RSA_set_default_method() — selects meth as the default method for all RSA structures created later. Future keys, this is your new normal 💫
• RSA_get_default_method() — returns a pointer to the current default method. Peek at the main character 👀
• RSA_set_method() — selects meth to perform all operations using the key rsa. This replaces the previous RSA_METHOD used by the RSA key, calling the finish function set up with RSA_meth_set_finish(3) if any. And if meth contains an init function set up with RSA_meth_set_init(3), that function is called just before returning from RSA_set_method() 🛠️
• RSA_get_method() — returns a pointer to the RSA_METHOD being used by rsa. Slay, now you know who's actually doing the work 💅

Okay important: it is possible to have RSA keys that only work with certain RSA_METHOD implementations, and in such cases attempting to change the RSA_METHOD for the key can have unexpected results 😭.

The misleadingly named function RSA_flags() returns the flags that are set for the current RSA_METHOD of rsa. The flags used by rsa itself can instead be tested with RSA_test_flags(3). See the BUGS section for more details — it's a whole thing 🙈.

RSA_new_method() allocates and initializes an RSA structure. The engine argument is ignored and the default method controlled by RSA_set_default_method() is used. The initial flags are copied from the RSA_METHOD object used and will not be affected by later changes to that object, but may be modified by the optional init function which may have been set up with RSA_meth_set_init(3) and which is called just before returning from RSA_new_method() 🌟.

🌈 RETURN VALUES 🌈

• RSA_PKCS1_SSLeay(), RSA_get_default_method(), and RSA_get_method() return pointers to the respective RSA_METHOD.
• RSA_set_method() returns 1 on success or 0 on failure. Currently, it cannot fail. Iconic, no notes 💅
• RSA_new_method() returns NULL and sets an error code that can be obtained by ERR_get_error(3) if the allocation fails. Otherwise it returns a pointer to the newly allocated structure.

🫶 SEE ALSO 🫶

• RSA_meth_new(3)
• RSA_new(3)

⭐ HISTORY ⭐

• RSA_set_default_method(), RSA_PKCS1_SSLeay(), and RSA_new_method() first appeared in SSLeay 0.8.0.
• RSA_flags() first appeared in SSLeay 0.9.0.
• These functions have been available since OpenBSD 2.4.
• RSA_get_default_method(), RSA_set_method(), and RSA_get_method() as well as the rsa_sign and rsa_verify components of RSA_METHOD first appeared in OpenSSL 0.9.4 and have been available since OpenBSD 2.6.

💀 BUGS 💀

🚨 real talk, bestie: the behaviour of RSA_flags() is a misfeature that is left as-is for now to avoid creating compatibility problems.

RSA functionality, such as the encryption functions, are controlled by the flags value in the RSA key itself — not by the flags value in the RSA_METHOD attached to the RSA key, which is what this function returns. So if the flags element of an RSA key is changed, the changes will be honoured by RSA functionality but will not be reflected in the return value of the RSA_flags() function. In effect, RSA_flags() behaves more like an RSA_default_flags() function, which does not currently exist 😵‍💫.

kisses and clean builds, bestie — this page is from Debian, dated November 19, 2023, and it's literally the RSA_SET_METHOD(3) page 💖🫶💅