Networking and transport — extended reference

This reference preserves the earlier detailed chapter and its citations. Start with the guided explanation, then use these sections for additional mechanisms and qualifications. References to the original lesson sequence are historical.

Part 1: Foundations: understanding work, time, and resources

Prerequisites: Lesson 03 and the preceding foundations.

Learning outcomes

Trace connection setup and data delivery across DNS, transport, encryption, and application boundaries.
Distinguish flow control, congestion control, loss recovery, and head-of-line blocking.
Calculate bandwidth-delay product and explain why transport changes cannot eliminate propagation time.

Separate connection setup from an established conversation

A database responds in a few milliseconds inside its data center, yet a distant client waits hundreds of milliseconds for its first query. The discrepancy may arise before the database executes anything. Resolving a name, reaching a network address, establishing transport state, negotiating encryption, authenticating, and opening a database session all contribute to the first operation. An established pooled connection takes a different path.

Networking is a layered division of responsibilities. The database designer does not need to implement every layer, but must understand which guarantees and costs each layer supplies. Otherwise a timeout can be mistaken for a failed transaction, an ordered stream can be mistaken for independent messages, or bandwidth can be mistaken for low propagation delay.

All numerical values in this lesson are illustrative unless explicitly identified as protocol constants. The relevant specifications describe semantics; production performance depends on topology, implementations, and operating conditions.

Names, addresses, routes, and endpoints

DNS maps a name to information such as network addresses. A name may identify several addresses, and caches can retain answers until their configured lifetime expires. Changing a DNS answer therefore does not instantly move every established client. Existing connections may remain attached to an old endpoint while new connections use another.

IP delivers packets between addresses across a sequence of network hops. Routing determines which paths packets take. A path can change, become congested, or stop delivering. The application generally observes these conditions indirectly through delay, loss, connection failure, and retry behavior. Two machines in the same named cloud region do not necessarily have identical network paths or failure dependencies.

A socket is an operating-system interface to a communication endpoint. Ports distinguish services at an address. A connection can traverse load balancers, proxies, gateways, firewalls, and network-address translation. Those intermediaries can impose idle timeouts, buffering, payload limits, and additional queues. The complete database path includes them even when the engine process itself is fast.

For a persistent client, reconnect behavior matters as much as initial connection behavior. A failover may change the authoritative server while a client retains a stale route or session. Connection draining, endpoint discovery, and retry deadlines must cooperate. An application should not infer that a reachable endpoint is authorized to serve the latest database state merely because it accepts a socket.

TCP supplies a reliable ordered byte stream

TCP orders bytes and retransmits lost data under its protocol rules. It does not preserve an application’s original write boundaries as message boundaries. If the sender writes a header and a body separately, a receiver may read them together or in partial pieces. The application protocol must frame messages and validate their lengths.

Reliability is not instantaneous delivery. Suppose bytes 1 through 1,000 arrive, bytes 1,001 through 2,000 are lost, and later bytes arrive. TCP cannot present the later portion of the ordered stream as if the gap were absent. Loss recovery delays delivery. If several logical database messages share that stream, a missing segment can delay later messages that are logically independent.

This is transport head-of-line blocking. It differs from an engine queue in which one long transaction delays the next transaction. Both appear as waiting to a user, but they require different remedies. Shortening a transaction does not recover a lost segment faster. Choosing another transport does not fix an overloaded execution queue.

TCP also detects duplicates, tracks acknowledgments, manages retransmission, and responds to network conditions. A successful application write to a socket can mean the kernel accepted bytes into a buffer, not that the peer application processed them. Even an acknowledgment of transport bytes does not establish database commit. The database protocol must supply its own semantic result.

Congestion control and flow control solve different problems

Flow control protects a receiver from receiving more data than it can buffer or process through the transport interface. Congestion control limits the sender’s use of the network based on observed conditions. A small receiving window and a congested network can both limit throughput, but for different reasons.

A database client that stops reading results can fill transport buffers. Eventually the server may be unable to enqueue more output for that client. If the server retains unlimited result state while waiting, a slow consumer becomes a memory problem. If a shared worker blocks while writing to that client, other clients can suffer too. Network backpressure must reach the appropriate application boundary.

The bandwidth-delay product estimates how many bytes must be in flight to fill a path at a specified rate. At 1 Gbit/s and a 40-millisecond round-trip time, the product is 1,000,000,000 bit/s × 0.040 s = 40,000,000 bits, or 5,000,000 bytes. A sender limited to a much smaller effective in-flight window cannot achieve that nominal rate on this illustrative path.

Increasing the window does not reduce the path’s propagation delay. A 100-byte request still needs time to reach a distant server. Bandwidth helps large transfers and overlapping work; latency dominates small dependent exchanges. Database protocols with many sequential request/response steps are especially exposed to that distinction.

UDP and application responsibility

UDP sends datagrams without providing TCP’s reliable ordered stream abstraction. Datagram boundaries are preserved by the interface, subject to size and delivery constraints. Datagrams can be lost, duplicated, or reordered. An application that needs reliable transactions must add appropriate request identity, retransmission, ordering, congestion behavior, and security rather than assuming the transport supplies them.

Some realtime information becomes obsolete quickly. A newer position update may be more useful than retransmitting every old position. That makes different delivery policies attractive for movement and for durable inventory ownership. It does not imply that every game action should use unreliable delivery. The application’s correctness and presentation requirements determine what must arrive, in what order, and by when.

Datagram size also matters. Large messages may be fragmented or require application-level segmentation, increasing exposure to loss and path constraints. A protocol should account for the path’s effective maximum transmission size rather than treating a large logical message as one indivisible network operation. Exact usable sizes depend on encapsulation and transport headers.

QUIC changes stream and connection behavior

QUIC runs over UDP while providing secure connections and reliable, flow-controlled streams. Independent streams can avoid the cross-stream delivery blocking inherent in putting all messages into one TCP byte stream. Ordering still applies within a stream. The connection still shares network capacity and congestion behavior, and the receiver still needs resource limits.

This distinction is important for database protocol design. Assigning independent operations to independent streams can reduce one form of interference. A transaction whose second step depends on its first step remains sequential. An engine that executes all operations behind one saturated queue gains no execution parallelism merely because requests arrive on multiple transport streams.

QUIC also supports connection behavior such as migration across network paths. Whether that helps a particular database depends on client environments and protocol implementation. A browser, a mobile client, an internal replication link, and a colocated application server have different operational needs. Transport choice should follow those needs rather than a general assumption that newer means universally faster.

TLS and replay-sensitive operations

TLS protects confidentiality and integrity in transit and authenticates endpoints under its configured trust model. It does not decide whether an authenticated user may purchase an item or read another tenant’s rows. Those authorization decisions occur at the application and database layers.

Connection establishment and cryptographic negotiation contribute to first-request latency. Reusing an authenticated encrypted connection can amortize setup. Session resumption can also change setup behavior, but it must be considered alongside replay protections and version-specific protocol requirements. Early-data mechanisms are especially important to understand before carrying non-idempotent mutations: reducing a handshake delay must not create duplicate irreversible actions.

The course cites RFC 8446 as the historical TLS 1.3 specification. Standards can be updated or superseded; an implementation must follow the current applicable specifications and security guidance. The teaching point is the boundary between transport authentication and database authorization, not a recommendation to implement cryptography from an old document.

Connection pooling changes which cost is visible

A connection pool retains established sessions for reuse. It can avoid repeated DNS, transport, encryption, and authentication setup. It can also limit database-side concurrency. However, waiting for a pooled connection becomes part of request latency. A database may report fast execution while the application spends most of its time waiting before sending the query.

A pool also manages state. Session variables, prepared statements, transaction status, and temporary objects may affect reuse. A transaction must not accidentally resume on an incompatible session or leak a previous tenant’s state to a new borrower. Pooling modes differ in the lifetime of session affinity and therefore in which features are safe.

Increasing pool size can move queueing into the database rather than remove it. If the database can complete 5,000 operations per second under the required contract, permitting 50,000 requests to enter concurrently does not establish a higher sustainable rate. It may increase memory usage and tail latency. The pool is an admission mechanism as well as a setup optimization.

Trace the critical network path

Consider a simplified first connection to a distant service with a 50-millisecond RTT. Suppose setup requires several sequential exchanges before the first query can be processed. Each dependent exchange contributes another opportunity to wait for that RTT. The exact count depends on protocol versions, resumption, proxies, authentication, and whether messages can be combined; it should be measured rather than assumed.

Now compare the established connection. A single request and response might require approximately one network RTT plus execution and queueing. If application logic then performs four dependent exchanges with another remote database, that second path adds further delay. A useful trace identifies every boundary independently:

client -> gateway -> application -> database
                  <-             <-
client <---------------------------------

The diagram is schematic. It illustrates that client-to-service distance and application-to-database distance are different costs. Moving application logic next to its data can remove the latter while leaving the former. A colocated architecture should therefore be evaluated with both regional execution measurements and the actual client’s network conditions.

Trace loss through an ordered stream

Consider an established connection carrying three framed responses, A, B, and C. A’s final bytes are lost while B and C arrive at the receiver. TCP can acknowledge received ranges and recover loss, but the application cannot consume B’s bytes from that stream before the preceding gap is repaired. B’s database computation may have finished promptly. Its observed delay nevertheless includes loss recovery caused by earlier stream bytes.

Opening separate connections can isolate that ordering dependency, but adds setup, state, and congestion interactions. Multiplexing B and C into independent QUIC streams changes cross-stream delivery behavior, while still sharing the path and connection-level resources. If the application serializes all three responses through one own output queue, it can recreate head-of-line blocking above the transport. A transport feature is effective only if higher layers preserve the intended independence.

Loss also affects congestion behavior. Retransmission sends additional bytes, and congestion control can reduce the sending rate. The resulting delay can outlast the recovery of one missing segment. A latency investigation should separate recovery of the affected message from reduced capacity for subsequent traffic. Increasing application concurrency can aggravate the queue instead of restoring the previous operating point.

Packet budgets, serialization time, and propagation time

A network message takes time to place onto a link. At an illustrative 100 Mbit/s bottleneck, transmitting 1 MB of decimal payload alone takes 8,000,000 / 100,000,000 = 0.08 seconds, or 80 milliseconds, before protocol overhead. Even with negligible server execution, a large result cannot arrive faster than the bottleneck permits. A 100-byte response has negligible serialization time on that link, but still incurs propagation and processing delays.

A packet has headers and may pass through encapsulation layers. Dividing a result into packets adds per-packet work at hosts and intermediaries. Very small application writes can create avoidable scheduling and packetization overhead unless combined by suitable buffering. Excessive buffering, however, can delay interactive messages while a large transfer occupies queues. The desired behavior depends on priority, batching thresholds, and whether applications share a connection.

Path maximum transmission size matters because a message large enough for one local network may be too large after tunnel or encryption overhead. A robust transport discovers or accommodates usable packet sizes rather than relying on one environment’s Ethernet setting. For an application designer, the practical lesson is to test actual production paths and avoid assuming one logical record maps to one network packet.

Receive-side limits must reach the producer

Imagine a realtime server publishing 2 MB per second to one client that can consume only 500 KB per second. The deficit is 1.5 MB per second. An unbounded output queue accumulates about 90 MB in one minute under these decimal assumptions. Compression might alter the rates but does not establish a bound. If thousands of clients fall behind simultaneously, the server can exhaust memory despite a healthy transaction engine.

The system needs a semantic policy. A state feed may replace obsolete intermediate positions with a newer position. A durable audit stream may require replay from a persisted offset, making disconnection and later catch-up preferable to retaining all data in RAM. A transactional response cannot be silently discarded if the caller needs to determine the outcome; it may instead use an operation-result lookup after reconnecting. Flow control is therefore not only a socket-buffer setting. It must propagate to a policy appropriate to the data.

The producer also needs to distinguish one slow client from global congestion. Blocking all publication behind the slowest subscriber makes that subscriber control everyone else’s progress. Maintaining separate bounded queues improves isolation while increasing per-client metadata. Shared immutable message buffers can reduce copies, provided reference lifetimes prevent one slow client from pinning unlimited history. Delivery architecture is a resource-ownership problem as much as a transport problem.

Connection establishment is a dependency graph

A simplified new-session path might resolve a name, establish transport, negotiate encryption, authenticate, and exchange a query. Some operations can overlap or combine; others depend on prior results. Counting stages as independent fixed RTTs is therefore only an initial model. An implementation trace should identify which messages actually wait for which replies.

For a constructed example, suppose a service’s measured setup dependency path is three 40-millisecond network round trips plus 5 milliseconds of local work. Its first-query setup component is about 125 milliseconds. Reusing an established connection can remove much of that setup, but it does not eliminate the RTT for the query itself. If the query then returns a large result, transfer time adds another distinct component. These components explain why “connection latency” and “query latency” should be reported separately.

Authentication can introduce another remote dependency, such as validating credentials against a separate authority. Repeated validation may be cached under a defined expiry and revocation policy. The cache improves latency while changing when revoked privileges take effect. A transport connection surviving for hours must not silently imply that its application privileges can never change. Security state has its own lifetime, independent of the socket.

Retransmission, retransmission timeouts, and application deadlines

Transport recovery and application retries operate at different layers. TCP retransmits bytes to preserve its stream. An application retry submits another logical request, perhaps on another connection. The original request can still execute. If the deadline is shorter than the network’s transient recovery time, an aggressive application may create duplicate work precisely when the path is already impaired.

A deadline should represent the total remaining time for the operation, including connection acquisition, transmission, execution, and retry delay. Giving every nested RPC its own full timeout can make the composed operation exceed the user’s intended budget. Canceling a local wait does not retract bytes already delivered or undo a committed transaction. Request identity and stored outcomes remain necessary for mutations even on reliable transports.

Retries need bounded frequency and jitter so thousands of clients do not reconnect simultaneously after a brief outage. Reconnection storms spend CPU on encryption, authentication, session creation, and subscription snapshots before useful steady-state traffic resumes. Reserving capacity for recovery can matter more than maximizing normal-state admission. The networking layer exposes the failure; application protocol and admission policies determine whether recovery is stable.

Scenario: nearby compute and distant authority

An application runs at an edge location near its user, while authoritative inventory remains in a distant region. The user-to-application RTT is 5 milliseconds, but application-to-database RTT is 90 milliseconds. If a purchase requires four dependent database exchanges, communication on that internal path contributes about 360 milliseconds. Marketing the application as edge deployed does not change the location of the authoritative decision.

Moving computation into one transaction or a server-side function beside inventory can reduce that dependency count. Moving a read-only catalog copy near users can accelerate browsing. Moving purchase authority requires a stronger architectural change: partition ownership, distributed coordination, or a revised inventory contract. A cached stock display can be useful while remaining insufficient to authorize the final sale.

This scenario will recur in the SpacetimeDB discussion. Co-locating logic and state targets avoidable application/database exchanges. It does not remove user-to-authority distance, packet loss, fan-out bytes, or the need to place authority appropriately. The optimization is substantial when those removed exchanges dominate; it is modest when the remaining network path dominates.

Compare setup and stream ordering visually

new session: DNS result -> transport setup -> encryption -> auth -> query
pooled session:                                      existing -> query

TCP application delivery:
  A bytes [missing portion] | B bytes received | C bytes received
  A gap blocks delivery of later stream bytes until repaired

QUIC application delivery:
  stream A [missing portion] -> A waits
  stream B [complete bytes]  -> B may proceed
  stream C [complete bytes]  -> C may proceed
  all streams still share path capacity and congestion effects

The setup row is a dependency illustration, not a claim that every protocol uses one separate exchange per labeled step. Resumption, cached DNS, proxies, and combined messages change the actual path. A measurement should record the messages and waiting it observes.

The stream rows distinguish a delivery guarantee from a scheduling guarantee. Independent QUIC streams can isolate an ordering gap, but the receiving application must still schedule B and C. A saturated worker, a global transaction queue, or a full connection-level buffer can delay them. Transport independence does not imply unlimited processing independence.

For a database result, the application also needs a completion marker indicating the query succeeded. Receiving some bytes on B’s stream does not establish that all rows are complete, that a mutation committed, or that a subscription snapshot has reached a coherent watermark. Those meanings belong to the protocol above transport, which is why the next lesson follows message framing into operation identity and uncertainty.

Case analysis and design conclusion

TCP and QUIC specifications define different communication abstractions. Their implications for a database follow from the shape of the application dependency graph. A chat history query, a replication stream, a long analytical result, and an interactive mutation do not automatically belong on the same scheduling or transport policy.

A robust architecture separates setup from steady state, byte delivery from commit acknowledgment, receiver limits from network congestion, and independent streams from independent transactions. Once those distinctions are explicit, a latency investigation can identify whether time is spent establishing connections, traversing distance, recovering loss, waiting for output space, or executing database work. Lesson 5 builds the semantic request protocol on top of these transport guarantees.

Sources and further reading

Course index · Lesson plan