A walkthrough of a TLS 1.3 handshake
A while back, I wrote up a walkthrough of a real TLS 1.2 handshake,
detailing what each byte contributed to the SSL connection establishment process. Since the latest revision of TLS, 1.3, is now almost a year old,
and since it's a radical change from the TLS versions that came before it, this is probably a good time to go through the same exercise
for it. A lot more of the handshake is encrypted in this revision than in previous ones, so it's not as helpful to use tcpdump to examine the protocol
exchange as it was when I went through TLS 1.2 — in 1.3, everything after the server hello is now encrypted, including the certificate exchange.
This time, then, instead of capturing packets with tcpdump, I'll
use openssl's helpful s_client command with the -msg option to see the actual exchange that occurred
between client and server.
As always, TLS begins with a client hello. Example 1 illustrates the header of the client hello, which is sent in plaintext to the server to kick off the handshake process. I'll step through the whole message in parts; I'll indent to show which parts are subordinate to which other parts as well.
16 TLS handshake protocol
03 01 SSL version 3.1 (TLS 1.0)
01 37 Length of payload (311 bytes)
Example 1: TLS header
So far, this isn't too surprising, if you're familiar with older TLS protocols - I said this was TLS 1.3, but the second and third bytes indicate TLS 1.0. This is the TLS record protocol (not the handshake protocol), which itself hasn't changed since TLS 1.0. (Not too shocking since the record protocol just includes the version and the length of the data contained within it).
01 Handshake type Client Hello
00 01 33 Length of payload (307 bytes)
03 03 SSL version 3.3 (TLS 1.2)
Example 2: Client Hello Header
Now this bit is a little more unexpected. This seems to suggest that the client is requesting a TLS 1.2 handshake. In fact, it is — if the server doesn't understand TLS 1.3, anyway. TLS 1.3 is so radically different from its predecessors, and TLS implementations have been shown to be so version intolerant, that a TLS 1.3 client hello looks superficially exactly like a TLS 1.2 handshake, right down to the version. If the server only understands TLS 1.2, it will just negotiate a TLS 1.2 handshake as before. But if that's the case, how does the client advertise to the server that it actually does understand TLS 1.3? Well, read on for the answer (if you really can't wait, the answer is "a new client hello extension").
0d 19 8c 01 36 7a b0 a8 0a 1b bc 5e 33 e9 a4 e8 6e
2a ea 74 ec 19 0b 78 9c 25 4f 45 12 f3 1b bc client random
20 Session ID length (32 bytes)
c1 63 ad d1 41 8b b4 75 8d ec ed b5 28 47 13 52 64
60 14 ff ad 7c 2b 6c b4 8c 7d 4c 53 c8 2d 58 session ID
00 3e length of cipher suites (62 bytes)
13 02 AES_256_GCM_SHA384
13 03 CHACHA20_POLY1305_SHA256
13 01 AES_128_GCM_SHA256
c0 2c ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
c0 30 ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
00 9f DHE_RSA_WITH_AES_256_GCM_SHA384
cc a9 ECDHE_ECDSA_WITH_CHACHA_20_POLY1305_SHA256
cc a8 ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
cc aa DHE_RSA_WITH_CHACHA20_POLY1305_SHA256
c0 2b ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
c0 2f ECDHE_RSA_WITH_AES_128_GCM_SHA256
00 9e DHE_RSA_WITH_AES_128_GCM_SHA256
c0 24 ECDHE_ECDSA_WITH_AES_256_CBC_SHA384
c0 28 ECDHE_RSA_WITH_AES_256_CBC_SHA384
00 6b DHE_RSA_WITH_AES_256_CBC_SHA384
c0 23 ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
c0 27 ECDHE_RSA_WITH_AES_128_CBC_SHA256
00 67 DHE_RSA_WITH_AES_128_CBC_SHA256
c0 0a ECDHE_ECDSA_WITH_AES_256_CBC_SHA
c0 14 ECDHE_RSA_WITH_AES_256_CBC_SHA
00 39 DHE_RSA_WITH_AES_256_CBC_SHA
c0 09 ECDHE_ECDSA_WITH_AES_128_CBC_SHA
c0 13 ECDHE_RSA_WITH_AES_128_CBC_SHA
00 33 DHE_RSA_WITH_AES_128_CBC_SHA
00 9d RSA_WITH_AES_256_GCM_SHA384
00 9c RSA_WITH_AES_128_GCM_SHA256
00 3d RSA_WITH_AES_256_CBC_SHA256
00 3c RSA_WITH_AES_128_CBC_SHA256
00 35 RSA_WITH_AES_256_CBC_SHA
00 2f RSA_WITH_AES_128_CBC_SHA
00 ff psuedo-cipher-suite "renegotiation SCSV supported"
01 compression methods length (1 byte)
00 compression method (no compression)
Example 3: Ciphers and compression methods
So far, this continues to look like a TLS 1.2 client hello with the standard client random, session ID, cipher suites and compression
methods (which are required by the protocol to be listed but are always turned off since compressed TLS data is susceptible to the
CRIME and BREACH attacks). There are a lot of cipher suites listed here, but that's mostly because openssl's command-
line mode is mostly a testing tool. If you were to use Chrome to negotiate a connection, you'd see a much smaller list (17 at my last
count). The session ID is included for backward compatibility with TLS 1.2, but is ignored if TLS 1.3 is negotiated,
as it is in this case. Technically, this could be left empty with a 0 length indicator, but OpenSSL negotiates in "compatibility mode".
00 ac length of extensions (172 bytes)
00 00 Extension Server name
00 13 length of server name (19 bytes)
00 11 length of list of server names
00 type host name
00 0e length of server name
77 77 77 2e 67 6f 6f 67 6c 65 2e 63 6f 6d www.google.com
Example 4: Start of extensions
The client hello preamble is followed by a variable-length list of extensions, shown in example 4. The first is the required server name indicator which has been around for quite a while and is used by multi-homed hosts to select the correct certificate to present; it's the exact DNS name that the client requested which would otherwise be unavailable to the server which is only listening to a particular IP address. Nothing new here.
One of the original goals of the TLS working group for 1.3 was to encrypt the SNI part so that eavesdroppers can't also tell which hosts a target is visiting. Although this is desirable from a privacy perspective, it turned out to be difficult to implement in practice, so was removed from TLS 1.3. However, there is ongoing work in the form of the ESNI (Encyrpted Server Name Interface) effort to make this capability available as an extension.
00 0b Extension EC points format
00 04 length of formats (4 bytes)
03 length of list
00 uncompressed
01 ANSI X.962 compressed prime
02 ANSI X.962 compressed char2
00 0a Extension Supported groups
00 0c length of supported groups (12 bytes)
00 0a length of list
00 1d x25519 RFC 7748
00 17 secp256r1
00 1e x448
00 19 secp521r1
00 18 secp348r1
Example 5: ECC points and groups
Although DHE and RSA are presented as supported key exchange methods, it's pretty likely that the server will selected an ECC-based key exchange/signature method, and if they do, the client and server need to agree of point format and supported groups. This is pretty much the same as you would see in a TLS 1.2 handshake, although x25519 and x448 aren't always supported in older TLS 1.2 implementations.
00 23 Extension session ticket
00 00 length (0 bytes)
00 16 Extension encrypt then MAC
00 00 length (0 bytes)
00 17 Extension extended master secret
00 00 length (0 bytes)
Example 6: Marker extensions
There are a few "marker" extensions here. The session ticket extension is defined in RFC 5077, the encrypt then mac in 7366, and extended master secret in 7627. These actually apply to TLS 1.2 if the server ends up selecting it; each extension is either mandatory or not applicable in TLS 1.3.
00 0d Extension signature algorithms
00 30 length (48 bytes)
00 2e length of list
04 03 ecdsa_secp256r1_sha256
05 03 ecdsa_secp384r1_sha384
06 03 ecdsa_secp521r1_sha512
08 07 ed25519
08 08 ed448
08 09 rsa_pss_pss_sha256
08 0a rsa_pss_pss_sha384
08 0b rsa_pss_pss_sha512
08 04 rsa_pss_rsae_sha256
08 05 rsa_pss_rsae_sha384
08 06 rsa_pss_rsae_sha512
04 01 rsa_pkcs1_sha256
05 01 rsa_pkcs1_sha384
06 01 rsa_pkcs1_sha512
03 03 sha224_ecdsa
02 03 ecdsa_sha1
03 01 sha224_rsa
02 01 rsa_pkcs1_sha1
03 02 sha224_dsa
02 02 sha1_dsa
04 02 sha256_dsa
05 02 sha384_dsa
06 02 sha512_dsa
Example 7: Signature algorithms
Again, because I'm using openssl's s_client tool, I see an unnaturally long list of supported signature
algorithms; Chrome would only list nine or so.
00 2b Extension supported versions
00 09 length of extension (9 bytes)
08 length of list
03 04 TLS 1.3
03 03 TLS 1.2
03 02 TLS 1.1
03 01 TLS 1.0
Example 8: Supported versions
Finally, as promised, example 8 shows how the server is notified that the client is willing and able to negotiate a TLS 1.3 handshake: the list of supported versions extension. Prior versions of TLS were required to be backward compatible all the way down to SSLv2; the client would indicate the highest version it was willing to support, but the server was considered free to negotiate any prior version. This has been replaced with an explicit supported list so that older protocols can be more easily deprecated in the future.
00 2d Extension PSK exchange modes
00 02 length of extension (2 bytes)
01 length of list
01 PSK with DHE
Example 9: Pre-shared key modes
Example 9 is the key share extension. This is completely new to TLS 1.3 — the client and the server can establish a key offline and omit the key-share portion of the handshake entirely (in theory, anyway). In practice, this pre-shared-key is negotiated after a "normal" (EC)DHE key establishment and takes the place of what prior revisions of TLS called session resumption.
00 33 Extension key share
00 26 length of extension (38 bytes)
00 24 length of key share (36 bytes)
00 1d Group x25519
00 20 Length of key exchange (32 bytes) RFC 7748
e7 62 a7 5a 88 e1 e9 21 c9 4a 80 0a 0e 65 51 d7 fc c8
55 df f7 b2 e8 9d a6 b5 60 6b ba c0 0f 2f key
Example 10: Key share
The final client hello extension is the key share: in prior versions of TLS, this was a separate message called the Client Key Exchange. It's inclusion as a hello extension in 1.3 might surprise you — after all, how can the client begin a key exchange without letting the server select a key exchange method from the available list? Well, technically the answer is "it can't", but the client can go ahead and assume that the server will select its preferred key exchange method and just start a key exchange using it. If the server selects a different key exchange method, it (the server) will respond with a Retry Hello Request message (not shown here) which will restart the key share. In most cases, though, since the server will support the preferred key exchange method, this optimization saves a round-trip and speeds up the handshake quite a bit. Notice that there are two lengths indicated here: the length of the extension and then the length of the key shares. That might seem redundant, but TLS 1.3 allows for multiple key shares to be offered (in descending order of preference). Here there's only one, which is the most common case.
Once digested by the server, the client hello is followed — again in cleartext — by the corresponding server hello as show in example 11.
16 TLS Handshake protocol
03 03 SSL 3.3 (TLS 1.2)
00 7a Length of handshake message (122 bytes)
02 Server hello message
00 00 76 Length of server hello (118 bytes)
03 03 SSL 3.3 (TLS 1.2)
11 62 86 81 d9 e8 89 2f d7 ed 62 8c 74 28 21 17 3b c2
8d a5 65 8d f3 83 e6 0e 69 ad d3 49 54 42 server random
20 Length of session ID (32 bytes)
c1 63 ad d1 41 8b b4 75 8d ec ed b5 28 47 13 52 64 60
14 ff ad 7c 2b 6c b4 8c 7d 4c 53 c8 2d 58
13 02 Selected cipher suite (AES 256 w/GCM SHA 384)
00 Compression method (none)
00 2e Length of extensions
00 33 Key share extension
00 24 length of extension (36 bytes)
00 1d Group (x25519)
00 20 length of key share
c6 c6 78 37 8d 84 bb fb 6e 97 a7 63 76 92 33 65 6f 95 8a 4a
8f 6c aa 29 48 fd 4b b7 f2 de 5a 6f key share
00 2b Supported versions extension
00 02 Length of supported versions
03 04 SSL 3.4 (TLS 1.3)
Example 11: Server Hello
Now this still looks a bit suspicious - the client hello advertised itself as a TLS 1.2 handshake message for easy backwards compatibility. So why, now that we've established that this is a TLS 1.3 handshake, does the server do the same? This is done to allow certain (broken) proxies to continue to work correctly; some of them were found to drop the server hello completely if it was of a higher version than they themselves supported. The TLS 1.2 indicator lets them pass the message through without error. Otherwise this message isn't too surprising; the server selected one of the available cipher suites as expected. The supported versions extension (at the end) indicates that this is, indeed, a TLS 1.3 handshake. The key exchange is completed here in the key share extension; this takes the place of the older Server Key Exchange message in prior versions of TLS. It's interesting to note that, since the client key exchange came first in this case, the semantics have been inverted — in all prior revisions of TLS, the server started the key exchange and the client finished it.
14 TLS Change Cipher Spec protocol
03 03 SSL 3.3 (TLS 1.2)
00 01 length of change cipher spec message
01 payload (irrelevant)
Example 12: Change cipher spec
Again, for middlebox and proxy compatibility, TLS 1.3 includes the now irrelevant Change Cipher Spec message. From this point on, everything else from the server is encrypted. The next message, which you'd expect to be sent in cleartext if you were familiar with TLS 1.2 and before, is the server certificate. The first five bytes are the "encrypted data" header:
17 TLS application data
03 03 version 1.2
08 df length of payload (2271 bytes)
Example 13: TLS Header with encrypted data
You'll see these same five bytes if you snoop on the wire using, say Wireshark or tcpdump. The remaining 2,271 bytes, though, are encrypted using the new key share. If you were to decrypt them, though, the contents begin as show in example 14:
16 handshake message
08 encrypted extensions
00 00 02 2 bytes
00 00 (empty)
Example 14: start of encrypted handshake message
The first byte is the "standard" TLS handshake indicator 0x16. In previous versions of TLS, handshake messages could be batched together in a single record, but were preceded by the handshake indicator, a version number, and the length. In this case, since the handshake message is contained within a TLS application data record and the version and payload size have already been established, the next byte is a message type and the version and length are omitted. In this case, the message type is 0x08 indicating encrypted extensions. TLS 1.3 allows for additional sensitive extensions to be exchanged after the cryptographic context has been established. In this case, none were selected, but the message is required by the protocol so it's exchanged empty in this case.
0b certificate message
00 08 41 length of certificate chain (2,113 bytes)
00 certificate request context length
00 08 3d (length of certificate chain)
00 03 d3 (length of certificate)
30 82 03 cf 30 82 02 b7 a0 03 ...
00 00 Certificate extensions
00 04 60 length of certificate (1,120 bytes)
30 82 04 5c 30 82 03 44 a0 03 ...
00 00 Certificate extensions
Example 15: Encrypted handshake message continued (server certificate exchange)
The encrypted extensions message is immediately succeeded by a certificate message. The certificates themselves are identical to the certificates that would be exchanged if you had performed a TLS 1.2 handshake — this makes sense; the available signatures algorithms are essentially the same, and the certificate is the same whether you negotiate TLS 1.2 or not. I've covered this format before and won't repeat it here.
However, although the certificate message is similar to that of TLS 1.2 (and before), there are two differences worth pointing out here, bolded in example 15. The first is the new certificate request context indicator. This is used for client certificates; if the server requested a certificate from the client, it could supply a context which the client could return to bind the certificate properly to the request. It's included in all certificate messages to simplify the parser, even though it's mandated to be empty (0 length) in the server certificate message. The second difference is the optional certificate extensions. The OCSP status response, which used to be a separate handshake message, is defined as a certificate extension in TLS 1.3. Here they're just an empty list signified by two 0 bytes.
0f Certificate verify message
00 00 4b length of certificate verify message (75 bytes)
04 03 Signature scheme (ECDSA SECP256r1 SHA 256)
00 47 Length of signature (71 bytes)
30 45 02 21 00 8c f7 fc 1c 53 c3 c6 fd b4 0e a5 17 dc b3 72 4a 88 90 63 1d
84 ef 31 51 ba a2 44 af 26 b1 7c 5a 02 20 6e 47 02 c1 56 31 02 38 fe 62 1f
bd b6 83 69 73 ff c3 34 a2 c8 ea f6 d2 fd 7c a3 aa 62 8f 91 58
Example 16: Certificate Verify
The Certificate Verify message is how the server proves ownership of the certificate's private key without revealing the private key itself: the entire handshake up to this point is hashed, and that hash is signed using the private key. The client can then compute the same hash and verify the signature using the public key.
14 Finished message
00 00 30 Length of finished message (48 bytes)
39 05 4a c7 b5 2d a4 8e 2e bd 0a e5 78 11 6a 96 81 8b a2 75 4f e2 5c c3 77
41 6f 6c a9 04 0e 37 93 d4 15 d4 8f 0f 5c e6 08 10 5a fe 44 1e 04 9e
Example 17: Server finished
The server's finished message payload is a hash of the entire handshake up to this point. The hash algorithm is SHA-384, as selected by the server in the server hello message, so the hash itself is 48 bytes.
If you've been paying close attention (really, really close attention), you may have noticed that the application data declared 2,271 bytes, but the payload only contained 2,255. The final 16 bytes are the AEAD cipher's tag (effectively the AEAD equivalent of a MAC). If you're familiar with earlier TLS implementations that mandated CBC mode, you might be surprised by the 2,271-byte data length: that's not a 16-byte boundary. In TLS 1.3 that's OK because AEAD ciphers, which are block-ciphers-turned-stream-ciphers (like GCM mode here), are mandatory.
If you're familiar with AEAD ciphers and earlier versions of TLS, you may be wondering... where is the nonce/IV? TLS 1.3 specifies that the AEAD nonce be derived from the sequence number and the negotiated master secret, so is never explicitly transmitted by either side. This might seem surprising, since explicit IV's were the only addition TLS 1.1 made to the protocol; however, since the IV's are securely derived from secret values, there's no vulnerability here.
Finally, the client sends (in the clear), a change cipher spec message as required by previous revisions:
14 TLS Change Cipher Spec protocol
03 03 SSL 3.3 (TLS 1.2)
00 01 length of change cipher spec message
01 payload (irrelevant)
Example 18: Client change cipher spec
Followed by an encrypted application data indicator:
17 TLS Application Data
03 03 version 1.2
00 45 length of encrypted data (69 bytes)
Example 19: Client encrypted data
16 handshake message
14 Finished message
00 00 30 Length of finished message (48 bytes)
eb 7a ee 13 74 cb 9d f2 5e 0d b2 e1 ac fa 48 37 c1 e2 f1 2f 80
6e fb 55 ee 25 88 86 73 17 e3 72 a7 87 f8 36 2c 9c 37 f3 83 51
80 10 1c f8 4d a5
Example 20: Client finished
Just as in the case of the server, the client includes a batch of handshake messages (in this case, just one), declared by the handshake indicator 0x16. The lone message is the client's finished message hashing everything that it has sent up to this point. Again, the length of the encrypted data is declared as 69 bytes, but only 53 are shown here - the last 16 bytes of the payload are the GCM tag. At this point, the handshake is complete, and the participants may safely begin exchanging messages.
Add a comment:
Completely off-topic or spam comments will be removed at the discretion of the moderator.
You may preserve formatting (e.g. a code sample) by indenting with four spaces preceding the formatted line(s)
Thanks for this good explanation. I implemented the TLS1.3 mechanism in my controller and i am able to access google page. But when i use my own servers, i get back this 7 byte hex instead of ServerHello "0x15 0x03 0x03 0x00 0x02 0x02 0x46"
Could you please help with what this means ?. But i get proper ServerHello response when connecting to google servers.
Thanks a lot for your quick reply. In the supported versions i just had (0x0304) TLS1.3 mentioned, and when i added (0x0303) TLS1.2, i do not get this error alert.
Now i have a new problem that the server does not reply back with Server Hello and ends up with a receive timeout. I have tried many ways of modifying the client hello (referring to sample hello message send from browsers), but ended up still with no response from the server.
But the same hello message works with google or amazon servers and i get back Server hello. I have problems accessing for example "iot.evergreeninternet.in" where i finally have to post my data. I am completely puzzled why it works with google or amazon sites but not any other normal servers.
I have placed the wireshark hexdump and screenshot of the hello message i am sending in the below link. Could you please help me in solving this puzzle on what might be wrong in this ?.
drive.google.com/file/d/1NuRjQRncU7xCXfQxXcGG4ZGfIOiM1IjP/view?usp=sharing
1. client handshake
2. server handshake
.
11. certificate
etc. I've tried reading the TLS RFC but i haven't found any listing of them or even a C/C++ header file anywhere that would list the proper operators.
Thanks in advance.
Göran
I happen to come across this and another article of yours on TLS 1.2/1.3.
I needed some updated information on TLS 1.3 in book form. I checked your book on Amazon India site, it looks the book covers only till TLS 1.2 only.
Is there any plan to come out update of your book, 'Implementing SSL / TLS Using Cryptography and PKI', with TLS 1.3 information?
Regards,
Abu.
PS: Its not a question for forum, but could not get your email ID so posting it here.
I happen to come across this and another article of yours on TLS 1.2/1.3.
I needed some updated information on TLS 1.3 in book form. I checked your book on Amazon India site, it looks the book covers only till TLS 1.2 only.
Is there any plan to come out update of your book, 'Implementing SSL / TLS Using Cryptography and PKI', with TLS 1.3 information?
Regards,
Abu.
PS: Its not a question for forum, but could not get your email ID so posting it here.
Can client directly send the encrytped data once server hello is received ?
I’ve read your paper book “Implementing TLS1.2” some years ago, which helped me a lot in throughly understand TLS and its sourroundings like dissecting the the DER certificate e.g.
If you’d bring out an updated version “Implementing TLS 1.3” on the same manner, I’d buy it as well.
Do you have any plans in this direction?
Cheers
-Alex
I’ve read your paper book “Implementing TLS1.2” some years ago, which helped me a lot in throughly understand TLS and its sourroundings like dissecting the the DER certificate e.g.
If you’d bring out an updated version “Implementing TLS 1.3” on the same manner, I’d buy it as well.
Do you have any plans in this direction?
Cheers
-Alex
I have a question about TLS1.3 and *External* Pre-shared Keys. I have tried to do my homework and read around, but in general, the topic of pre-shared keys isn't given a whole lot of air-time, because it is sort of a niche topic, and when it is mentioned, it is only discussed in the context of session resumption, which sort of obfuscates the answer I am looking for.
CONTEXT:
I have an IoT application where computational power and code space on the clients are very limited limited, but pre-established pre-shared keys (e.g. configured at the factory and securely copied to the back-office; or provisioned by a handheld installation tool that can communicate with a back-office using a full TLS suite) are 100% feasible.
QUESTION:
The first thing I'm trying to establish is: Assuming that I am willing to accept the loss of forward secrecy associated with external PSKs, is it possible to have a TLS1.3 client communicate with my back-office, without the client needing any asymmetric key or elliptic-curve algorithms to be compiled-in? The answer *seems* to be yes, from what I can understand of the RFCs, but I'm having a hard time finding an SSL stack that will let me do it. I would like to use TLS_AES_256_GCM_SHA_384 if possible, since I already have efficient AES and SHA implementations available.
Second question is: if it *is* possible to do what I'm asking, what options, exactly, should I look for in SSL libraries to figure out if they will work for my use case? For example, I *think* I need a client and server that supports the plain PSK_Key_Exchange_Mode(mode 0) - but so far it seems the servers I have examined (e.g. cryto/tls in the Golang standard library) only support mode 1 (psk_dhe?).
Thanks for any pointers you can provide...
I read the spec the same as you - that this should be possible, and supported. For example, 2.2 “TLS PSKs can be established out of band” “PSKs can be … be used alone, at the cost of losing forward secrecy for the application data.” “When using an out-of-band provisioned pre-shared secret, a critical consideration is using sufficient entropy during the key generation, as discussed in [RFC4086].”
According to section 4.2.9, as you noted, the authentication method you're looking for is psk_ke. Reading through the 1.1.1d source doe, openssl does appear to support this authentication method, so that might be a good starting place.
This is probably a good question to ask on the IETF TLS working group mailing list - I've been following it for a while, and they're usually pretty helpful and receptive to this sort of question.