OSSL-GUIDE-TLS-INTRODUCTION(7ossl)

📦 version 3.5.0, in opensuse-leap 16.0

🦋 OSSL-GUIDE-TLS-INTRODUCTION 💅

okay bestie so this is ossl-guide-tls-introduction(7ossl), the OpenSSL Guide's little intro chapter to SSL/TLS shenanigans ✨ part 1 of the series, and it's literally the glow-up your brain needs before you touch any of this stuff.

🌸 INTRODUCTION 🌸

so like, this page is gonna walk you through some basic SSL/TLS concepts and the whole background vibe of how it works inside OpenSSL 🫶. It does assume you already have a basic understanding of TCP/IP and sockets — like, nobody's explaining sockets to you today, that's a you-and-the-socket-man-page situation, iykyk.

🔥 WHAT IS TLS? 🔥

TLS stands for Transport Layer Security, and no cap, it's the moment 💅. TLS lets applications communicate securely with each other across a network so that the confidentiality of the info being exchanged is protected — literally it stops eavesdroppers from listening in on the convo 👀. On top of that it protects the integrity of the info, so an attacker can't just sneak in and change it. And finally it gives you authentication, so one or both parties can be sure they're talking to who they actually think they're talking to and not some imposter 🎭.

so sometimes TLS gets called by its predecessor's name, SSL (Secure Sockets Layer). OpenSSL dates from a time when the SSL name was still in common use, which is why so many of the functions and names in OpenSSL have the "SSL" abbreviation in them. But bestie, OpenSSL contains a fully fledged TLS implementation, fr fr. it's giving legacy naming, modern icon behavior 👑.

TLS is based on a client/server model:

  • The application that initiates a communication is the client.
  • The application that responds to a remotely initiated communication is the server.
  • The term "endpoint" refers to either the client or the server in a communication.
  • The term "peer" refers to the endpoint at the other side of the communication that we're currently referring to. So if we're currently talking about the client, then the peer would be the server 🫶.

TLS is a standardised protocol and there are numerous different implementations of it. Which is the slay part: because of the standards, an OpenSSL client or server can communicate seamlessly with an application using some different implementation of TLS 🌈. TLS (and its predecessor SSL) have been around for a significant period of time and the protocol has undergone various changes over the years — consequently there are different versions of the protocol available. TLS includes the ability to perform version negotiation so that the highest protocol version that the client and server share in common is what gets used ✨.

TLS acts as a security layer over some lower level transport protocol. Typically the transport layer will be TCP.

⭐ SSL AND TLS VERSIONS ⭐

history lesson time, bestie, stay with me 📖✨

SSL was initially developed by Netscape Communications and its first publicly released version was SSLv2 in 1995. Note that SSLv1 was never publicly released 💀. SSLv3 came along quickly afterwards in 1996. Subsequently development of the protocol moved to the IETF, which released the first version of TLS (TLSv1.0) in 1999 as RFC2246. TLSv1.1 was released in 2006 as RFC4346 and TLSv1.2 came along in 2008 as RFC5246. The most recent version of the standard is TLSv1.3, released in 2018 as RFC8446 👑.

Today TLSv1.3 and TLSv1.2 are the most commonly deployed versions of the protocol. The IETF have formally deprecated TLSv1.1 and TLSv1.0, so anything below TLSv1.2 should be avoided since the older protocol versions are susceptible to security problems 😭🔒.

bestie do NOT be out here running anything below TLSv1.2. that's not a vibe, that's a security problem.

Now the OpenSSL-specific tea ☕:

  • OpenSSL does not support SSLv2 — it was removed in OpenSSL 1.1.0.
  • Support for SSLv3 is available as a compile time option — but it is not built by default 🙈.
  • Support for TLSv1.0, TLSv1.1, TLSv1.2 and TLSv1.3 are all available by default in a standard build of OpenSSL.
  • However, special run-time configuration is required in order to make TLSv1.0 and TLSv1.1 work successfully. like, they exist but they need help to show up, main character energy with a whole pre-show routine 💅.

OpenSSL will always try to negotiate the highest protocol version that it has been configured to support. In most cases this will mean either TLSv1.3 or TLSv1.2 is chosen ✨.

🎀 CERTIFICATES 🎀

okay so this is the part where the server has to prove it's actually that girl 💅.

In order for a client to establish a connection to a server it must authenticate the identity of that server — i.e. it needs to confirm that the server is really the server it claims to be and not some imposter. To do this, the server will send to the client a digital certificate (also commonly referred to as an X.509 certificate). The certificate contains various information about the server including its full DNS hostname. Also within the certificate is the server's public key. The server operator will have a private key which is linked to the public key and must not be published 🔒🙅‍♀️.

Along with the certificate, the server will also send to the client proof that it knows the private key associated with the public key in the certificate. It does this by digitally signing a message to the client using that private key. The client can verify the signature using the public key from the certificate. If the signature verifies successfully, then the client knows that the server is in possession of the correct private key ✅👑.

The certificate that the server sends will also be signed by a Certificate Authority. The Certificate Authority (commonly known as a CA) is a third party organisation responsible for verifying the information in the server's certificate (including its DNS hostname). The CA should only sign the certificate if it has been able to confirm that the server operator does indeed have control of the server associated with its DNS hostname and that the server operator has control of the private key 🫶.

So in this way, if the client trusts the CA that has signed the server's certificate and it can verify that the server has the right private key, then it can trust that the server truly does represent the DNS hostname given in the certificate. The client must also verify that the hostname given in the certificate matches the hostname that it originally sent the request to ⭐.

Once all of these checks have been done, the client has successfully verified the identity of the server. OpenSSL can perform all of these checks automatically ✨ — but it must be provided with certain information in order to do so, i.e.:

  • the set of CAs that the client trusts, and
  • the DNS hostname for the server that this client is trying to connect to.

🦋 the chain of trust moment 🦋

note that it's common for certificates to be built up into a chain 🔗. For example, a server's certificate may be signed by a key owned by an intermediate CA. That intermediate CA also has a certificate containing its public key, which is in turn signed by a key owned by a root CA. The client may only trust the root CA, but if the server sends both its own certificate and the certificate for the intermediate CA, then the client can still successfully verify the identity of the server. There is a chain of trust between the root CA and the server 💖. it's giving found family but make it cryptography.

💜 client authentication 💜

by default it is only the client that authenticates the server using this method. However it is also possible to set things up such that the server additionally authenticates the client — this is known as "client authentication" ✨. In this approach the client will still authenticate the server in the same way, but the server will request a certificate from the client. The client sends the server its certificate and the server authenticates it in the same way that the client does 🫶.

and that's where part 1 wraps, bestie — more incoming, don't go anywhere 🌈

✨ TRUSTED CERTIFICATE STORE ✨

okay so bestie, here's the tea: everything we just described only works if a chain of trust can actually be built between the set of CAs your endpoint trusts and the certificate the peer is presenting 💅. Which means — plot twist — your endpoint has to already have a set of certificates for CAs it trusts before any communication can even happen. And OpenSSL itself does not ship you that set 🙈. So you gotta make sure you have them before you start verifying certificates (and btw that means always if your endpoint is a client, and only if client authentication is in use for a server).

Good news though, other organisations totally maintain that set for you 🫶. If you got your copy of OpenSSL from an Operating System (OS) vendor (like, say, a Linux distribution), then normally the set of CA certificates comes distributed with that copy too.

You can check this by running the OpenSSL command line application like so:

openssl version -d

That'll display a value for OPENSSLDIR. Now go look in the certs sub directory of OPENSSLDIR and check its contents. For example, if OPENSSLDIR is "/usr/local/ssl", then check the contents of the "/usr/local/ssl/certs" directory.

You're expecting to see a list of files, typically with the suffix ".pem" or ".0" ✨. If they're there, congrats, you already have a suitable trusted certificate store, slay.

If you're running your version of OpenSSL on Windows then OpenSSL (from version 3.2 onwards) will use the default Windows set of trusted CAs 👑

Now, if you built your version of OpenSSL from source, or got it from some other location and it does not have a set of trusted CA certificates, then you're gonna have to get them yourself. One such source is the Curl project — see the page <https://curl.se/docs/caextract.html> where you can download trusted certificates in a single file. Rename that file to "cert.pem" and store it directly in OPENSSLDIR. For example if OPENSSLDIR is "/usr/local/ssl", then save it as "/usr/local/ssl/cert.pem" 🎀.

You can also use environment variables to override the default location that OpenSSL looks for its trusted certificate store 👀. Set the SSL_CERT_PATH environment variable to give the directory where OpenSSL should look for its certificates, or the SSL_CERT_FILE environment variable to give the name of a single file containing all of the certificates. See openssl-env(7) for further details about OpenSSL environment variables. Like, you could literally use this to have multiple versions of OpenSSL all installed on the same system using different values for OPENSSLDIR but all sharing the same trusted certificate store. iconic.

Wanna test that your trusted certificate store is set up correctly? Just use it via the OpenSSL command line. Run this to connect to a TLS server:

openssl s_client www.openssl.org:443

Once the command has connected, type the letter "Q" followed by <enter> to exit the session. It'll print a lot of info on the screen about the connection. Look for a block of text like this:

SSL handshake has read 4584 bytes and written 403 bytes
Verification: OK

Hopefully if everything worked, the "Verification" line will say "OK" 🎉. If it's not working as expected, you might see output like this instead:

SSL handshake has read 4584 bytes and written 403 bytes
Verification error: unable to get local issuer certificate

The "unable to get local issuer certificate" error means OpenSSL couldn't find a trusted CA for the chain of certificates provided by the server in its trusted certificate store 😭. Go check your trusted certificate store configuration again.

heads up bestie: s_client is a testing tool and it will still let you connect to the TLS server regardless of the verification error 💀. Most applications should not do this and should abort the connection when a verification error happens. don't be that app.

🌈 IMPORTANT OBJECTS FOR AN OPENSSL TLS APPLICATION 🌈

A TLS connection is represented by the SSL object in an OpenSSL based application ✨. Once a connection with a remote peer has been established, an endpoint can "write" data to the SSL object to send data to the peer, or "read" data from it to receive data from the server 👑.

A new SSL object is created from an SSL_CTX object. Think of an SSL_CTX as a "factory" for creating SSL objects 🏭. You can create a single SSL_CTX object and then create multiple connections (i.e. SSL objects) from it. Typically you can set up common configuration options on the SSL_CTX so that all the SSL objects created from it inherit the same configuration options. inheritance but make it TLS 💅.

ngl, internally to OpenSSL various items that are shared between multiple SSL objects are cached in the SSL_CTX for performance reasons. So it's considered best practice to create one SSL_CTX for use by multiple SSL objects instead of having one SSL_CTX for each SSL object you create 🫶.

Each SSL object is also associated with two BIO objects. A BIO object is used for sending or receiving data from the underlying transport layer. For example you might create a BIO to represent a TCP socket 🧚. The SSL object uses one BIO for reading data and one BIO for writing data. In most cases you'd use the same BIO for each direction, but there could be some circumstances where you want them to be different 👀.

It is up to the application programmer to create the BIO objects that are needed and supply them to the SSL object. See ossl-guide-tls-client-block(7) and ossl-guide-tls-server-block(7) for usage examples 📚.

Finally, an endpoint can establish a "session" with its peer. The session holds various TLS parameters about the connection between the client and the server. Those session details can then be reused in a subsequent connection attempt to speed up the process of connecting — this is known as "resumption" ⚡. Sessions are represented in OpenSSL by the SSL_SESSION object. In TLSv1.2 there is always exactly one session per connection. In TLSv1.3 there can be any number per connection — including none 🫠.

🦋 PHASES OF A TLS CONNECTION 🦋

A TLS connection starts with an initial "set up" phase. The endpoint creates the SSL_CTX (if one hasn't already been created) and configures it 🎀.

A client then creates an SSL object to represent the new TLS connection. Any connection specific configuration parameters get applied, and then the underlying socket is created and associated with the SSL via BIO objects ✨.

A server will create a socket for listening for incoming connection attempts from clients. Once a connection attempt is made, the server creates an SSL object in the same way as for a client and associates it with a BIO for the newly created incoming socket 🌸.

After set up is complete, the TLS "handshake" phase begins 🤝. A TLS handshake consists of the client and server exchanging a series of TLS handshake messages to establish the connection. The client starts by sending a "ClientHello" handshake message and the server responds with a "ServerHello". The handshake is complete once an endpoint has sent its last message (known as the "Finished" message) and received a Finished message from its peer.

And here's the messy part 💀 — this might occur at slightly different times for each peer. For example in TLSv1.3 the server always sends its Finished message before the client. The client later responds with its Finished message. At this point the client has completed the handshake because it has both sent and received a Finished message. The server has sent its Finished message but the Finished message from the client may still be in-flight, so the server is still in the handshake phase 😭. It's even possible that the server will fail to complete the handshake (if it considers there's some problem with the messages sent from the client), even though the client may have already progressed to sending application data. In TLSv1.2 this can happen the other way around, i.e. the server finishes first and the client finishes second 🙈.

Once the handshake is complete, the application data transfer phase begins 🔥. Strictly speaking there are some situations where the client can start sending application data even earlier (using the TLSv1.3 "early data" capability) — but we're gonna skip over that for this basic introduction, iykyk.

During application data transfer the client and server can read and write data to the connection freely. The details of this are typically left to some higher level application protocol (for example HTTP) 🌐. And no, not all information exchanged during this phase is application data — some protocol level messages may still be exchanged, so it's not necessarily the case that, just because the underlying socket is "readable", application data will be available to read 👀.

When the connection is no longer required then it should be shutdown 👋. A shutdown may be initiated by either the client or the server via a message known as a "close_notify" alert. The client or server that receives a close_notify may respond with one, and then the connection is fully closed and application data can no longer be sent or received 🚫.

Once shutdown is complete a TLS application must clean up by freeing the SSL object 🧹.

📖 FURTHER READING 📖

👑 SEE ALSO 👑

ossl-guide-introduction(7), ossl-guide-libraries-introduction(7), ossl-guide-libssl-introduction(7), ossl-guide-tls-client-block(7), ossl-guide-tls-server-block(7), ossl-guide-quic-introduction(7)

Copyright 2023-2024 The OpenSSL Project Authors. All Rights Reserved.

Licensed under the Apache License 2.0 (the "License"). You may not use this file except in compliance with the License. You can obtain a copy in the file LICENSE in the source distribution or at <https://www.openssl.org/source/license.html>.

okay that's the whole guide bestie, go build something iconic with it 🌈✨ this was version 3.5.0, dated 2026-09-25 — mwah, study up and slay 💅🎀

📦 where this page comes from