SSL_CTX_SET_...AME_CALLBACK(3ossl)

📦 version 3.5.0, in opensuse-leap 16.0

💅 SSL_CTX_set_tlsext_servername_callback — OpenSSL 3ossl, bestie edition 🎀

okay so this one is all about SNI (server name indication) — like, how a server figures out which of its many little website children a client is asking for. it's from the OpenSSL 3ossl manual family, and yes it's giving "load-bearing TLS plumbing" ✨

🌸 NAME 🌸

SSL_CTX_set_tlsext_servername_callback, SSL_CTX_set_tlsext_servername_arg, SSL_get_servername_type, SSL_get_servername, SSL_set_tlsext_host_name — handle server name indication (SNI). that's the whole vibe 🦋

✨ SYNOPSIS ✨

#include <openssl/ssl.h>

long SSL_CTX_set_tlsext_servername_callback(SSL_CTX *ctx,
                                  int (*cb)(SSL *s, int *al, void *arg));
long SSL_CTX_set_tlsext_servername_arg(SSL_CTX *ctx, void *arg);

const char *SSL_get_servername(const SSL *s, const int type);
int SSL_get_servername_type(const SSL *s);

int SSL_set_tlsext_host_name(const SSL *s, const char *name);

💖 DESCRIPTION 💖

so first things first, bestie: the servername callback is mostly been superseded by the ClientHello callback, which you set with SSL_CTX_set_client_hello_cb(). that's the glow-up. but — and this is important — even when you're using the ClientHello callback, the servername callback is still necessary so you can acknowledge the servername the client asked for. so it's not dead, it's just not the main character anymore 👑

🫶 the setter functions

  • SSL_CTX_set_tlsext_servername_callback() — sets the application callback cb, which a server uses to do literally whatever actions or configuration it needs based on the servername extension it got in the incoming connection. and when cb is NULL? SNI is not used. periodt.
  • SSL_CTX_set_tlsext_servername_arg() — sets a context-specific argument that gets passed into the callback via the arg parameter, for this SSL_CTX. it's like handing your callback a little snack 🍓

🌈 what the callback should return

the servername callback should return one of these values, no cap:

SSL_TLSEXT_ERR_OK

this one means: the servername the client requested has been accepted. typically a server will call SSL_set_SSL_CTX() inside the callback here, to set up a different configuration for the servername it just selected. slay 💅

SSL_TLSEXT_ERR_ALERT_FATAL

nope, not accepted — and the handshake gets aborted. the alert value you want to use should be stored in the location that al (the callback's parameter) points to. by default that value is initialised to SSL_AD_UNRECOGNIZED_NAME. so if you don't touch it, that's what you get.

SSL_TLSEXT_ERR_ALERT_WARNING

the servername is not accepted by the server… but the handshake keeps going and it sends a warning alert instead. store the alert value in the location al points to, same as for SSL_TLSEXT_ERR_ALERT_FATAL above.

⚠️ heads up bestie: TLSv1.3 does not support warning alerts. so if TLSv1.3 got negotiated, this return value gets treated exactly the same way as SSL_TLSEXT_ERR_NOACK. no drama, just facts 😭

SSL_TLSEXT_ERR_NOACK

this return value means the servername is not accepted by the server. no alerts are sent, and the server will not acknowledge the requested servername. it's the quiet-quitting option 💀

🦋 SSL_get_servername() — the whole vibe check

the behaviour of SSL_get_servername() depends on a bunch of different factors. and in particular — note this — in TLSv1.3 the servername is negotiated in every handshake, but in TLSv1.2 the servername is only negotiated on initial handshakes and not on resumption handshakes. got it? got it. ✨

💜 on the client, before the handshake

  • if a servername has been set via a call to SSL_set_tlsext_host_name(), it returns that servername.
  • if one has not been set, but a TLSv1.2 resumption is being attempted and the session from the original handshake had a servername accepted by the server, then it returns that servername.
  • otherwise, it returns NULL.

💜 on the client, during or after the handshake and a TLSv1.2 (or below) resumption occurred

  • if the session from the original handshake had a servername accepted by the server, it returns that servername.
  • otherwise, it returns the servername set via SSL_set_tlsext_host_name(), or NULL if that was never called.

💜 on the client, during or after the handshake and a TLSv1.2 (or below) resumption did not occur

  • it returns the servername set via SSL_set_tlsext_host_name(), or NULL if it was not called.

🌸 on the server, before the handshake

  • the function always returns NULL before the handshake. that's it. that's the whole bullet.

🌸 on the server, after the servername extension has been processed and a TLSv1.2 (or below) resumption occurred

  • if a servername was accepted by the server in the original handshake, it returns that servername; otherwise NULL.

🌸 on the server, after the servername extension has been processed and a TLSv1.2 (or below) resumption did not occur

  • the function returns the servername the client requested in this handshake, or NULL if none was requested.

⭐ callback ordering tea ⭐

note that the ClientHello callback happens before a servername extension from the client is processed. meanwhile the servername, certificate and ALPN callbacks happen after a servername extension from the client is processed. order matters, iykyk 🧚

🎀 the other two

  • SSL_get_servername_type() — returns the servername type, or -1 if there's no servername present. currently the only supported type (defined in RFC3546) is TLSEXT_NAMETYPE_host_name.
  • SSL_set_tlsext_host_name() — sets the server name indication ClientHello extension to contain the value name. the type of the server name indication extension gets set to TLSEXT_NAMETYPE_host_name (defined in RFC3546). iconic.

👑 NOTES 👑

several callbacks get executed during ClientHello processing, including the ClientHello, ALPN, and servername callbacks. and the order is, in fact:

  1. the ClientHello callback runs first
  2. then the servername callback
  3. followed by the ALPN callback

also important: SSL_set_tlsext_host_name() should only be called on SSL objects that are going to act as clients. if you call it on something else, the configured name will just be ignored. 🙈

🔥 RETURN VALUES 🔥

  • SSL_CTX_set_tlsext_servername_callback() and SSL_CTX_set_tlsext_servername_arg() — both always return 1, indicating success. always. it's giving unbothered. 💅
  • SSL_set_tlsext_host_name() — returns 1 on success, and 0 in case of error.

🌈 SEE ALSO 🌈

ssl(7), SSL_CTX_set_alpn_select_cb(3), SSL_get0_alpn_selected(3), SSL_CTX_set_client_hello_cb(3) — go read these, they're all in the same friend group 🫶

🦋 HISTORY 🦋

SSL_get_servername() historically gave some genuinely unexpected results in certain corner cases. that's been fixed from OpenSSL 1.1.1e. let's get into it:

  • prior to 1.1.1e, if the client requested a servername in an initial TLSv1.2 handshake, the server accepted it, and then the client successfully resumed but set a different explicit servername in the second handshake — then when called by the client, it returned the servername from the second handshake. this has now been changed to return the servername requested in the original handshake. 🙃
  • also prior to 1.1.1e, if the client sent a servername in the first handshake but the server did not accept it, and then a second handshake occurred where TLSv1.2 resumption was successful — then when called by the server, it returned the servername requested in the original handshake. this has now been changed to NULL. love that for us 💀

Copyright 2017-2020 The OpenSSL Project Authors. All Rights Reserved.

Licensed under the Apache License 2.0 (the "License"). You may not use this file except in compliance with the License. You can obtain a copy in the file LICENSE in the source distribution or at <https://www.openssl.org/source/license.html>.

okay that's the whole page bestie, go configure your SNI and thrive 🌸💖✨

mwah 💋 — this was OpenSSL 3.5.0, dated 2026-09-25, signing off 🎀

📦 where this page comes from