TCP Three-Way Handshake

The scenario (Why you care)

When a “web page doesn’t load” or an API client reports connection timed out, the very first question is: did the TCP connection get established at all? The TCP three-way handshake is the minimal exchange that turns a destination IP:port into an actual connection state on both ends. If the handshake fails, everything above it (HTTP, TLS, DNS-over-TCP, database protocols) will look broken even if the application is healthy.

The good news is that the handshake is easy to read in a capture once you know what to look for. A successful handshake shows a clean sequence of SYN → SYN-ACK → ACK. Most real-world outages show one of a small number of repeatable patterns: repeated SYNs with no answer, immediate RSTs, or a SYN-ACK that never receives the final ACK. This page focuses on turning those patterns into quick, reliable decisions about what to check next.

What “good” looks like (Success pattern)

What you see

A client sends a TCP SYN to the server’s listening port. The server replies with SYN,ACK. The client responds with the final ACK. After that, application data can flow (for example, TLS ClientHello or HTTP request bytes).

Why it happens

TCP uses the handshake to synchronize sequence numbers and confirm that both sides can send and receive. The client’s SYN proposes an initial sequence number, the server’s SYN-ACK acknowledges it and proposes its own, and the final ACK confirms the server’s proposal. Only then do both stacks allocate full connection state and deliver data reliably.

Key clue

You see all three packets for the same 4-tuple (client IP/port, server IP/port) in a short time window, and the SYN-ACK’s ACK number matches the client SYN’s sequence number + 1. This simple arithmetic alignment is a strong indicator that you are observing a real handshake (not unrelated packets).

What goes wrong (Failure pattern)

What you see

One of the handshake steps is missing or replaced: (1) repeated SYN retransmissions with no SYN-ACK, (2) an immediate RST after SYN (or instead of SYN-ACK), or (3) a SYN-ACK is sent but the final ACK never arrives and the server retransmits SYN-ACK.

Likely causes

The missing packet usually means the path (or policy) blocks it: wrong destination, routing blackhole, firewall drop, server not listening, or asymmetric routing where replies take a different path. Immediate RST often indicates “nothing is listening on that port” or an intermediate device actively rejecting the connection. A SYN-ACK with no final ACK can point to return-path issues, stateful filtering, or NAT/load balancer behavior changing the flow midstream.

Key clue

Handshake failures are mostly about which direction is missing. If SYN leaves but SYN-ACK never returns, the problem is “server-side reachability or return path.” If SYN-ACK returns but ACK never reaches the server, the problem is “client-side return path, filtering, or translation.” Capturing on both sides of a boundary (client vs gateway, or server vs load balancer) often turns this into a yes/no decision quickly.

Signals & decision table

Use the table below as a fast decision map. The handshake is small, so you do not need dozens of signals. The goal is to classify what you see into one of a few branches, then pick the next observation point (or related field) that confirms the hypothesis.

How it looks in Wireshark

Display filter example:

tcp.flags.syn == 1 or tcp.flags.reset == 1

• Successful handshakes show a SYN, then SYN-ACK, then ACK with matching sequence/acknowledgment progression.
• Failures often show repeated SYN retransmissions (same destination) or repeated SYN-ACK retransmissions (server side).
• RST packets are easy to spot and often immediately explain “connection refused” style errors.

Quick read tip: Follow one 4-tuple and ask “which direction is missing?” before thinking about the application.

Fast triage checklist

1. Pick one attempt: select a single client port and server port pair (one connection try), not the whole capture.
2. Find the SYN: confirm the client is actually sending SYNs to the intended destination IP and port.
3. Check for SYN-ACK: if none return, classify it as “no response” and move your capture point closer to the server path.
4. If SYN-ACK exists, look for the final ACK: missing ACK strongly suggests return-path, stateful filtering, or NAT mismatch.
5. If handshake completes, stop blaming TCP: switch to the next layer (TLS/HTTP) or to later TCP behavior (window/PMTU) as needed.

Common pitfalls

Mistaking “SYN seen” for “server received it”

Seeing a SYN on the client capture only proves the client transmitted it. It does not prove the server received it, or that the server could respond back. That’s why handshake debugging often requires two vantage points: one near the client and one near the server or gateway. The decisive evidence is where the packet disappears: if it is visible at the client but not at the next hop, it’s a local/link problem; if it is visible near the server but no SYN-ACK leaves, it’s a server-side listener/policy problem.

Assuming RST always means “the server is down”

RST usually means “actively rejected,” which can be a server with no listener, but it can also be a firewall, proxy, or load balancer generating resets. Correlate the source IP/MAC (if local), TTL patterns, and capture location. If the “RST source” changes when you move the capture point, it’s likely not the end host.

Overlooking middleboxes that terminate TCP

In many environments the TCP handshake you see is with a VIP (load balancer) rather than the real backend server. That is not wrong, but it changes what “success” means: the handshake can succeed even when the backend is unhealthy. In those cases, a successful handshake plus an immediate application failure may indicate the middlebox accepted the connection but could not complete upstream routing or health checks. Use the decision table to pivot to the right next check.

Related pages (internal links)

• Back to Topics Index
• Dictionary: TCP overview
• Field: TCP SYN flag
• Dictionary Index