TCP Deep Dive with Wireshark

TCP is so foundational that many engineers only notice it when something breaks: slow file transfers, intermittent app timeouts, mysterious “connection reset” errors, or security alerts that don’t line up with what the application team is seeing.、

This article summarizes the most practical concepts from that discussion, while framing them for day-to-day work in modern environments: hybrid networks, cloud traffic patterns, high-latency links, and security analysis workflows.

Why TCP Analysis Still Matters for Modern Networking and Security

Even with QUIC and the broader shift toward encrypted, user-space transport protocols, TCP remains the default for a large part of enterprise traffic: internal APIs, legacy apps, many database sessions, management planes, and a long tail of operational tools. More importantly, TCP is often the first layer where symptoms become measurable.

When users say “the app is slow,” your fastest path to narrowing the blast radius is to start at the transport layer:

● If the TCP handshake is slow or failing, you’re dealing with reachability, firewall state, SYN drops, resource exhaustion, or congestion.

● If the handshake is fast but data exchange is delayed, you may be looking at window limitations, retransmissions, application backpressure, or path MTU/MSS issues.

● If TCP behaves normally but the app is still slow, you can pivot upward confidently.

That “transport-first” workflow is why Wireshark trace literacy is a core skill not only for network engineers, but also for SREs, SOC analysts, and pentesters.

TCP Three-Way Handshake:

More Than SYN, SYN-ACK, ACK

Most people memorize the handshake as a simple sequence:

● Client sends SYN

● Server responds SYN-ACK

● Client replies ACK

The handshake really establishes: synchronized byte-counting and negotiated capabilities. The handshake exchanges Initial Sequence Numbers (ISN)—large, random-looking values—so both sides agree on where each direction’s byte stream “starts.”

Why ISNs are big random numbers

ISNs aren’t small counters because TCP connections must not be confused with other connections. Randomized ISNs reduce the chance of collisions and make spoofing harder. The key point for analysts: sequence numbers are per-direction, and they represent byte positions, not packet numbers.

Why Wireshark shows “relative sequence numbers”

In real captures, ISNs are huge. Wireshark often displays them as relative sequence numbers (starting at 0) to make analysis human-friendly. This doesn’t change TCP behavior—it’s just an interpretation layer to help you reason about flow.

Sequence Numbers and Acknowledgments:

TCP reliability is enforced through two numbers that show up in almost every segment:

● Sequence Number: “Here’s the byte position of the first byte in this segment.”

● Acknowledgment Number: “I have received everything up to byte N-1; I expect byte N next.”

A critical detail highlighted in the video: each byte advances the sequence number by 1. So when you see a segment with “Len=1460,” the next expected sequence will be previous sequence + 1460 (assuming no options changing payload semantics). That’s the mechanical basis for understanding:

● Out-of-order delivery

● Retransmissions

● Duplicate ACKs

● Fast retransmit behavior

● Stalls caused by missing segments

Retransmissions: the TCP “receipt” didn’t arrive

If a sender doesn’t receive an ACK within its timeout window, it retransmits. In real troubleshooting, retransmissions often correlate with:

● Congestion and queue drops

● Wi-Fi interference and loss

● Overloaded middleboxes

● MTU black holes

● Asymmetric routing or capture limitations

In Wireshark, retransmissions can be spotted quickly via analysis flags and by following the sequence number continuity.

TCP Ports and the 4-Tuple: 

Another “simple but critical” concept: a TCP connection is uniquely identified by the 4-tuple:

● Source IP

● Source port

● Destination IP

● Destination port

In practice:

● Client portsare usually ephemeral (random high ports, typically >1023).

● Server portsare usually well-known (e.g., 80 for HTTP, 443 for HTTPS).

This matters in troubleshooting because it explains why multiple sessions to the same server can coexist, and why NAT tables, firewall state entries, and load balancer flows are keyed the way they are.

For security teams, the 4-tuple is also the anchor for correlating logs: firewall accepts/denies, IDS alerts, proxy records, and application telemetry.

TCP Flags That Actually Matter in Most Traces

TCP has many flags, but Bombal and Chris focus on the ones that consistently explain real-world behavior:

● SYN: open/synchronize sequence numbers

● ACK: acknowledge received bytes (present on most segments after handshake)

● FIN: graceful close (I’m done sending)

● RST: immediate reset/abort (often policy or error-driven)

● PSH: “push” data to the application (frequently misunderstood)

RST vs FIN: why resets are so diagnostic

A FIN is orderly shutdown. A RST often indicates:

● The destination port is closed (no listener)

● A firewall/middlebox actively rejected the flow

● An application crashed or closed abruptly

● A policy device injected resets (common in some enterprise environments)

Seeing who sends the RST, and at what point in the flow, can dramatically shorten root-cause time.

TCP Window Size and Flow Control

One of the most operationally useful sections in the deep dive is receive window behavior. TCP doesn’t let a sender transmit unlimited data. The receiver advertises how much buffer space it has—its receive window—which caps in-flight unacknowledged bytes.

The classic window limit: 65,535 bytes

Historically, the TCP window field is 16-bit, so the maximum advertised window is 65,535 bytes.

On modern high-bandwidth, high-latency links, that limit is painfully small. Even a modest RTT can throttle throughput if the window is too small, because the sender can’t keep enough data “in the pipe.”

Window Scaling: negotiated during the handshake

To solve that, TCP uses Window Scaling, a TCP option negotiated during the handshake. The scaling factor effectively multiplies the window value (commonly by powers of two). The video notes examples like multiplying by 64 (or other scale values depending on the peer OS and tuning).

Practical takeaway:

● If you’re diagnosing “good bandwidth but low throughput,” you mustcheck window scaling negotiation and steady-state window advertisements.

● Window size is dynamic. It can shrink when the receiving application can’t drain buffers quickly.

Zero Window: when the receiver is the bottleneck

A Zero Window advertisement means: “My buffer is full. Stop sending.” That’s not a network congestion issue—it’s usually application backpressure, host resource constraints, or a slow consumer (e.g., a server thread pool saturated).

If you see Zero Window in traces, stop blaming the WAN first. Validate the receiving host’s CPU, disk, memory pressure, and application behavior.

Handshake TCP Options: MSS, SACK

The deep dive also emphasizes that important TCP capabilities are negotiated early, primarily in the SYN and SYN-ACK:

MSS (Maximum Segment Size)

MSS defines the largest TCP payload per segment to avoid fragmentation. MSS is tightly connected to MTU and path characteristics. A wrong MSS (or broken PMTUD) can lead to:

● Fragmentation and loss

● Performance collapse

● “Works for small requests, fails for large transfers” symptoms

SACK (Selective Acknowledgment)

SACK allows a receiver to say: “I received these blocks, I’m missing that block.” This improves recovery when there’s packet loss because the sender can retransmit only the missing segments, not entire windows of data.

For analysts, SACK blocks in Wireshark are a goldmine: they tell you exactly what the receiver has and what it’s still missing.

RTT and Latency

Another practical technique: use the timing between handshake packets to estimate RTT. Bombal and Chris reference an example where the handshake implies an RTT around 20 ms.

Why this matters:

● RTT sets the baseline for application responsiveness.

● RTT interacts with window size to determine achievable throughput.

● Sudden RTT spikes in captures can correlate with routing changes, congestion events, or overloaded devices.

In Wireshark, adding “delta time” columns makes latency patterns obvious without needing external tooling.

Wireshark Workflow Tips for Faster TCP Troubleshooting

The video isn’t just theory—it’s about making traces readable.

Turn on relative sequence numbers (if not already)

Relative numbers reduce cognitive load. You care about movement and continuity more than absolute ISNs.

Add columns that reveal behavior

Useful additions include:

● Time delta(difference between packets)

● TCP segment length

● Retransmission indicators (Wireshark analysis)

These make throughput limits, stalls, and retransmit storms stand out.

Use targeted display filters

To isolate handshake behavior:

● flags.syn == 1

You can similarly filter for FIN/RST events or focus on a specific stream to reduce noise.

Use visualization to spot retransmissions and ACK patterns

Wireshark can highlight retransmissions and duplicate ACKs—especially helpful when explaining issues to non-network stakeholders.

Real-World Impact: What This Helps You Diagnose

Putting it all together, TCP deep analysis helps answer the questions that matter in production:

● Is the issue connectivity(handshake delays/failures) or data transfer (retransmits/windows)?

● Is the bottleneck network loss/congestionor receiver backpressure (Zero Window)?

● Are large transfers failing due to MTU/MSSmismatches?

● Are security alerts consistent with TCP state.

The video also points out an architectural limitation: TCP headers and options have fixed constraints (including limited option space), which is part of why newer transports like QUIC emerged. Even so, for the bulk of enterprise troubleshooting and many security investigations, TCP remains the protocol you must read fluently.

Conclusion

The key takeaway is that TCP isn’t just “reliable transport.” It’s an observable protocol with byte-accurate accounting, negotiated capabilities, and explicit flow control. When you learn to interpret sequence numbers, acknowledgments, window sizes, and handshake options in Wireshark, you gain a consistent diagnostic method that works across vendors, cloud environments, and applications.

Whether you’re troubleshooting performance issues, validating security events, or performing packet-level forensics, deeper TCP literacy turns packet captures from “noise” into evidence.

Get started for free

Sign up with Google