LIBRESSL-SSL_CTX_SET_ALPN_SELECT_CB(3)

📦 version 4.4.0, in alpine 3.24, arch rolling

🌸 SSL_CTX_SET_ALPN_SELECT_(3) 🌸

okay so welcome to the Library Functions Manual page for SSL_CTX_SET_ALPN_SELECT_(3), bestie. This one is all about ALPN — application layer protocol negotiation — and it’s lowkey iconic for clients and servers agreeing on which protocol to speak. 🦋

🎀 NAME 🎀

SSL_CTX_set_alpn_protos, SSL_set_alpn_protos, SSL_CTX_set_alpn_select_cb, SSL_select_next_proto, SSL_get0_alpn_selected — handle application layer protocol negotiation (ALPN).

✨ SYNOPSIS ✨

library “libssl” libcrypto #include <openssl/ssl.h>

int
SSL_CTX_set_alpn_protos(SSL_CTX *ctx, const unsigned char *protos,
    unsigned int protos_len);

int
SSL_set_alpn_protos(SSL *ssl, const unsigned char *protos,
    unsigned int protos_len);

void
SSL_CTX_set_alpn_select_cb(SSL_CTX *ctx,
    int (*cb)(SSL *ssl, const unsigned char **out, unsigned char *outlen, const unsigned char *in, unsigned int inlen, void *arg),
    void *arg);

int
SSL_select_next_proto(unsigned char **out, unsigned char *outlen,
    const unsigned char *peer_list, unsigned int peer_list_len,
    const unsigned char *supported_list,
    unsigned int supported_list_len);

void
SSL_get0_alpn_selected(const SSL *ssl, const unsigned char **data,
    unsigned int *len);

💅 DESCRIPTION 💅

okay so here’s the tea, bestie:

  • SSL_CTX_set_alpn_protos() and SSL_set_alpn_protos() — these are the client-side besties. The client uses them to set the list of protocols available to be negotiated. The protos must be in protocol-list format, described below. The length of protos is specified in protos_len. 🫶
  • SSL_CTX_set_alpn_select_cb() — sets the application callback cb used by a server to select which protocol to use for the incoming connection. When cb is NULL, ALPN is not used. The arg value is a pointer which is passed to the application callback.
  • cb — this is the application-defined callback. The in, inlen parameters are a vector in protocol-list format. The value of the out, outlen vector should be set to the value of a single protocol selected from the in, inlen vector. The out buffer may point directly into in, or to a buffer that outlives the handshake. The arg parameter is the pointer set via SSL_CTX_set_alpn_select_cb().
  • SSL_select_next_proto() — a helper function used to select protocols. It is expected that this function is called from the application callback cb. If SSL_select_next_proto() returns OPENSSL_NPN_NO_OVERLAP, cb should ignore out and fail by returning SSL_TLSEXT_ERR_ALERT_FATAL. The protocol data in peer_list, peer_list_len and supported_list, supported_list_len must be two non-empty lists, validly encoded in the protocol-list format described below. The first item in the peer_list that matches an item in the supported_list is selected, and returned in out, outlen. The out value will point into either peer_list or supported_list, so it must not be modified and should be copied immediately. If no match is found, the first item in supported_list is returned in out, outlen.
  • SSL_get0_alpn_selected() — returns a pointer to the selected protocol in data with length len. It is not NUL-terminated. data is set to NULL and len is set to 0 if no protocol has been selected. data must not be freed. 🙈

The protocol-lists must be in wire-format, which is defined as a vector of non-empty, 8-bit length-prefixed byte strings. The length-prefix byte is not included in the length. Each string is limited to 255 bytes. A byte-string length of 0 is invalid. The length of the vector is not in the vector itself, but in a separate variable.

For example:

const unsigned char *vector = "\x06" "spdy/1" "\x08" "http/1.1";
unsigned int length = strlen(vector);

The ALPN callback is executed after the servername callback; as that servername callback may update the SSL_CTX, and subsequently, the ALPN callback. 👀

If there is no ALPN proposed in the ClientHello, the ALPN callback is not invoked.

🔥 RETURN VALUES 🔥

  • SSL_CTX_set_alpn_protos() and SSL_set_alpn_protos() — return 0 on success or non-zero on failure. WARNING: these functions reverse the return value convention. 🚨
  • SSL_select_next_proto() — returns one of the following:
    • OPENSSL_NPN_NEGOTIATED — a match was found and is returned in out, outlen.
    • OPENSSL_NPN_NO_OVERLAP — no match was found. The first item in supported_list, supported_list_len is returned in out, outlen.
  • The ALPN select callback cb must return one of the following:
    • SSL_TLSEXT_ERR_OK — ALPN protocol selected.
    • SSL_TLSEXT_ERR_ALERT_FATAL — there was no overlap between the client's supplied list and the server configuration.
    • SSL_TLSEXT_ERR_NOACK — ALPN protocol not selected, e.g., because no ALPN protocols are configured for this connection.

🌈 SEE ALSO 🌈

⭐ STANDARDS ⭐

🦋 HISTORY 🦋

  • SSL_select_next_proto() first appeared in OpenSSL 1.0.1 and has been available since OpenBSD 5.3.
  • SSL_CTX_set_alpn_protos(), SSL_set_alpn_protos(), SSL_CTX_set_alpn_select_cb(), and SSL_get0_alpn_selected() first appeared in OpenSSL 1.0.2 and have been available since OpenBSD 5.7.

😭 CAVEATS 😭

The fallback to the first supported protocol in SSL_select_next_proto() comes from the opportunistic fallback mechanism in the NPN extension. This behavior does not make sense for ALPN, where missing protocol overlap should result in a handshake failure. To avoid accidental selection of a protocol that the server does not support, it is recommended to pass the locally configured protocols as second pair of protocols in the ALPN callback. 💀

🐛 BUGS 🐛

The out argument of SSL_select_next_proto() should have been const.

xoxo, Debian · June 8, 2025 · SSL_CTX_SET_ALPN_SELECT_CB(3) 💖

📦 where this page comes from