Application protocols and remote calls — extended reference

This reference preserves the earlier detailed chapter and its citations. Start with the guided explanation, then use these sections for additional mechanisms and qualifications. References to the original lesson sequence are historical.

Part 1: Foundations: understanding work, time, and resources

Prerequisites: Lesson 04 and the preceding foundations.

Learning outcomes

Design a framed request protocol with bounded messages and explicit serialization choices.
Follow timeout, retry, cancellation, and lost-reply cases without confusing uncertainty with failure.
State the atomicity and retention requirements of operation identifiers used for retry deduplication.

The purchase that succeeded without a reply

A buyer submits a purchase. The server validates the request, commits the order, and prepares a success response. The connection fails before the client receives the response. From the buyer’s perspective, the action timed out. From the database’s perspective, the order exists. Treating the timeout as proof of failure creates a duplicate when the client retries.

This is the central problem of remote calls: communication and execution do not fail at the same boundary. A protocol must define request identity, result meaning, deadlines, and resource ownership. Making a remote call resemble a local function is convenient, but its failure behavior remains different.

A database protocol carries more than values. It carries the information needed to interpret uncertain outcomes safely. The goal of this lesson is to connect framing and serialization to transaction semantics rather than treating networking as a transparent pipe.

Framing turns a byte stream into messages

TCP delivers an ordered byte stream, not a sequence of application messages. A receiver may obtain half a length header, several complete messages, or a complete header followed by only part of its body. A decoder must preserve incomplete state and process complete frames without assuming that one read corresponds to one write.

A common binary format places a message type and body length before the payload. The receiver validates the length before allocating or reading the body. A length field from an untrusted peer is not permission to allocate arbitrary memory. The protocol needs maximum sizes, integer-overflow checks, legal message types, and state-dependent validation.

decode(input):
    if fewer than header_bytes available:
        return NEED_MORE
    header = inspect_header(input)
    if header.length < minimum_body or header.length > configured_limit:
        return PROTOCOL_ERROR
    if complete body not yet available:
        return NEED_MORE
    return parse_exactly_one_message(header, body)

The pseudocode shows the ordering of trust-boundary checks. A production parser also validates field encodings, nesting, authentication state, version compatibility, and ownership of retained bytes. Rejecting malformed input early protects both correctness and availability. A parser that handles normal messages quickly but permits unbounded allocation is not a complete performance design.

Serialization is a physical and semantic choice

Serialization maps typed values to bytes. A text format can make diagnostics easy and interoperate with many languages. A binary format can reduce repeated field names and permit compact numeric representations. Actual performance depends on schema, implementation, allocation, compression, and payload shape; the format label alone does not establish speed.

The encoding must define integers, floating-point values, strings, nulls, arrays, and version evolution. Endianness and alignment affect binary readers. Text encodings must define character representation and escaping. A value valid in one language may not fit another language’s numeric range. Generated client bindings can reduce mismatches but do not eliminate the need for validation.

Database semantics also extend beyond types. Decimal money should not accidentally become an imprecise binary floating-point value. Timestamps need a defined interpretation and precision. An absent field and an explicitly null field may represent different update intentions. If an update protocol merges partial values, its semantics must identify which fields are changed and which are retained.

Compression reduces bytes at the cost of CPU, buffering, and sometimes delay. Compressing tiny interactive messages may offer little benefit. Compressing a large result can save network time, but may delay the first useful output if the implementation waits for a large block. Security-sensitive compression also needs care when secrets and attacker-controlled data share contexts. The appropriate decision follows the actual payload and trust model.

RPC and HTTP expose different shapes of conversation

An RPC names an operation and supplies typed arguments. An HTTP API uses the semantics of methods, resources, status codes, and representations. Either can carry a transaction request. Neither automatically determines whether the operation is atomic, durable, idempotent, or authorized.

HTTP’s idempotent method semantics describe the intended effect of repeated requests, not a transport guarantee that the network sends a request only once. A server implementation must preserve the intended semantics. A mutation sent using an operation identifier can also be idempotent even when carried by a method that is not inherently idempotent.

A persistent bidirectional protocol such as WebSocket permits server-initiated messages on an established connection. It helps subscriptions and asynchronous results, but introduces ordering, correlation, and flow-control responsibilities at the application layer. The client needs to distinguish the response to request 73 from a subscription update generated by another user’s transaction.

Correlation identifiers connect requests to responses. Operation identifiers establish business identity across retries. These may be different fields. A new transport attempt may have a new correlation identifier while retaining the original purchase identifier. Confusing the two can make duplicate suppression fail after reconnection.

A minimal idempotent transaction boundary

For the illustrative purchase service, define an operation key scoped to the authenticated actor and a fingerprint of the intended payload. Retain a completed result long enough to cover the supported retry window. If the same key is used with different parameters, reject it rather than silently treating a new request as the old purchase.

A conceptual transactional implementation is:

purchase(actor, operation_key, payload):
    begin transaction with scoped abort/cleanup on any failed step
    prior = find_and_protect_operation(actor, operation_key)
    if prior exists:
        require prior.fingerprint == fingerprint(payload)
        saved_result = copy(prior.saved_result)
        end read-only transaction and release protection
        return saved_result
    reserve_unique_operation_identity(actor, operation_key)
    result = validate_and_apply_purchase(actor, payload)
    save_result_with_purchase(actor, operation_key, result)
    commit under required durability policy
    return result

The protection and reservation steps must be race safe. A unique constraint or equivalent transactional mechanism prevents two simultaneous first attempts from both proceeding. An implementation must handle the losing attempt by retrying the lookup or returning the recorded outcome according to its transaction API. A nontransactional check-then-insert map is insufficient.

Every path ends the transaction: the completed-result branch releases its snapshot and any protection before returning; a failed requirement or other error aborts and cleans up the transaction through the scoped handler. Leaving the duplicate branch open would retain resources and could leak transaction state into the next request on a pooled connection.

Saving the operation result in the same atomic unit as the purchase connects duplicate suppression to actual state. If the transaction aborts, neither should remain. If it commits and the response is lost, a later attempt finds the saved result. Retention and garbage collection are part of the contract: deleting the record before the allowed retry horizon can permit duplication later.

This provides at-most-once business effects within stated assumptions about identity, retention, transaction correctness, and durability. It is not a universal claim of exactly-once delivery across every external system. An email, payment processor, or third-party API needs its own compatible protocol or reconciliation process.

Deadlines and timeouts

A timeout is a local waiting limit. A deadline is an absolute or propagated bound for the whole operation. If each layer independently spends its full timeout, the end-to-end duration can greatly exceed the user’s intended limit. Propagating a remaining budget helps prevent work that cannot finish usefully.

Suppose an illustrative 200-millisecond request budget includes 20 milliseconds of connection waiting, 30 milliseconds of network time, and 50 milliseconds of expected execution. A nested retry that starts with a fresh 200-millisecond timeout does not fit the original contract. The service should consider elapsed time and reserve enough budget for delivering the result or explaining uncertainty.

A deadline also interacts with transaction boundaries. Expiring before execution may allow safe rejection. Expiring while waiting for a lock may trigger transaction cancellation. Expiring after a durable commit cannot undo the committed business event merely because the client stopped waiting. The result should remain discoverable by operation identity.

Cancellation is best effort unless a stronger protocol explicitly establishes completion. A client can request cancellation while the server has already finished. A server can notice a disconnected socket after committing. The application must distinguish “cancellation requested,” “execution stopped before effect,” and “outcome uncertain.”

Retries alter the workload

Retries can recover from transient communication failures. They can also amplify overload. A request that consumes substantial resources before timing out may be repeated while its first attempt is still running. Duplicate suppression preserves business correctness but may still consume parsing, authentication, lookup, and response resources for every attempt.

A retry policy needs an overall deadline, a bounded number or budget of attempts, and randomized delay where appropriate. It should distinguish retryable transport or serialization failures from permanent validation errors. Retrying a sold-out result wastes capacity and degrades the experience without changing the outcome.

A service should report uncertainty honestly. If it cannot determine whether a mutation committed, returning an ordinary definitive failure can prompt unsafe behavior. A status lookup by operation identity can resolve the uncertainty later. This is especially important when clients are mobile, reconnect frequently, or operate across several gateways.

Pooling, pipelining, and multiplexing

A connection pool reuses sessions and limits simultaneous database conversations. Waiting in the pool belongs in end-to-end latency. Session affinity determines whether temporary state and prepared statements remain usable. A pool must also return connections in a clean transaction and authentication state.

Pipelining sends several operations without waiting for each response before sending the next. It can hide network waiting for independent work. It cannot remove a genuine dependency in which request B needs request A’s result. Responses need identifiers or defined order, and the server must bound how much pipelined work one client can accumulate.

Multiplexing allows several logical conversations to share a connection. It reduces connection overhead while creating shared failure and flow-control boundaries. One large result can monopolize output capacity unless the protocol and scheduler support fair interleaving. A failed connection may leave several operations uncertain at once.

Consider four dependent database exchanges at an illustrative 2-millisecond RTT. Communication contributes 8 milliseconds before execution. Pipelining unrelated transactions can raise aggregate throughput but does not remove those four dependencies within the one transaction. Combining the operation into a database-side procedure or changing its data access can shorten the dependent path, with corresponding changes to deployment and execution responsibility.

Case connection: a mature wire protocol exposes state

PostgreSQL’s frontend/backend protocol distinguishes message types and conversation states. It is a useful case because a database connection is not simply a generic request handler: transactions, authentication, errors, prepared execution, and asynchronous notices affect what messages are valid and how clients recover.

The architectural analysis is that client simplicity requires careful server and protocol design. A generated function call or convenient SDK can hide framing and encoding from the application author, but the underlying implementation still needs operation identity, bounded buffers, legal state transitions, and failure interpretation. A shorter API is valuable when those responsibilities are handled correctly rather than forgotten.

Parsing partial input is a state machine

A real decoder cannot discard its state whenever a network read ends. Suppose a frame uses a four-byte unsigned length followed by that many body bytes. The first read returns two length bytes, the second returns the remaining two bytes and five body bytes, and the third contains the rest of that body plus another whole message. Correct processing must produce exactly two messages without treating any read boundary as significant.

One design retains a small header buffer, the validated expected length, and the number of body bytes already collected. Once the header is complete, it rejects forbidden lengths before allocating the body. It then accumulates or references body fragments until the declared length is available. After decoding one message, it continues consuming any remaining input. Each transition has an explicit maximum allocation and must avoid arithmetic overflow when combining header and body sizes.

This state machine also handles EOF. A clean end between messages differs from an end halfway through a frame. The latter is an incomplete message, not permission to parse a truncated body. If the protocol supports streaming rows, each row or chunk needs its own valid termination and error behavior. A client receiving the first half of a large result must know whether it can use partial data and how a later execution error is represented.

Nested values introduce another resource dimension. A tiny message can describe extremely deep arrays or repeated structures. A parser needs depth or complexity bounds in addition to an overall byte limit. Decompression requires an output-size limit as well, because a small compressed input may expand greatly. Availability protection belongs at the decoder boundary before expensive semantic work begins.

Duplicate suppression under simultaneous first attempts

The interesting idempotency race occurs when neither attempt sees a stored outcome. Suppose requests A and B carry the same actor, key, and payload. Both begin transactions. If the implementation simply performs a lookup and then applies a purchase, both could allocate inventory before discovering a duplicate. The reservation step must establish exclusive transactional ownership of the operation identity before its business effect can commit.

An illustrative unique-index implementation inserts (actor, operation_key, fingerprint) as part of the transaction. One insertion succeeds. The competing insertion waits, aborts, or receives a uniqueness outcome under the engine’s rules. If the first transaction commits, the second safely retrieves its recorded result through a new or appropriately retried transaction. If the first aborts, the second may become the legitimate first execution. The design must handle both outcomes rather than interpreting every collision as completed success.

A reserves identity -> A changes stock -> A stores result -> A commits
B competes for identity -------------------------------> B reads A's result

alternative:
A reserves identity -> A fails -> A aborts
B competes -------------------> B may execute the purchase once

The diagram shows logical ordering, not required thread scheduling. It establishes that the unique identity and the business effect share one failure boundary. A separate cache entry created before the transaction can become a stranded reservation after a crash. An entry created after the transaction can leave a duplication window. Such designs need additional recovery protocols if they do not use the shared atomic boundary.

The saved result should contain enough stable information for replay. Returning the order identifier and committed business outcome is often safer than reconstructing a response from mutable current state. An order later canceled remains the same originally completed purchase, with a subsequent cancellation transition. Replaying the original operation should not accidentally execute a replacement purchase because today’s inventory differs.

Retention is part of the idempotency guarantee

Assume a constructed service accepts retries for seven days and completes one million purchases per day. If each retained operation outcome occupies 160 bytes before indexes and allocation overhead, seven days require 7 × 1000000 × 160 = 1.12 GB of payload. Retention is a deliberate capacity cost. Increasing the retry horizon increases stored state; deleting outcomes early weakens duplicate protection.

A timestamp supplied by an untrusted client should not alone determine whether the server accepts an operation as new. Otherwise an old key can be presented with a manipulated time. The server can define its own issuance or first-observation boundary and reject requests outside the supported window. Long-disconnected clients need an explicit recovery path rather than an accidental promise of indefinite duplicate suppression.

The fingerprint binds identity to intent. It should cover semantically relevant fields in a stable representation, including quantities and target entities. Hashing raw JSON bytes can treat harmless field-order changes as different intent, while omitting a relevant field can treat a different purchase as identical. The system needs canonical semantics, not merely a convenient hash function. Security-sensitive adversarial collisions require an appropriate hash and validation strategy; ordinary checksums should not be assumed sufficient.

Errors need scope and recovery rules

A protocol error may invalidate one message, one transaction, or the entire connection. A malformed frame often makes further stream parsing unsafe. A valid query with a type error can leave framing intact. A transaction error may require rollback before later statements are accepted. The client library must know the recovery boundary to avoid interpreting subsequent messages in the wrong state.

Pipelining makes this especially visible. If statements A, B, and C are already sent when B fails, the protocol must define whether C executes, is skipped, or belongs to an aborted transaction. The application cannot infer the answer from a generic exception. Mature protocols expose synchronization points or conversation states so both parties can regain an agreed boundary. PostgreSQL’s protocol documentation provides one implementation reference, rather than a requirement that every protocol adopt the same messages. PostgreSQL protocol

Streaming results require similar clarity. A server can emit several rows and then encounter an I/O error. The client must distinguish a complete result from a truncated one. A final completion message can establish successful termination; transport closure without that message does not. Applications that forward streamed data to other consumers need to carry that distinction rather than labeling partial output as a successful answer.

Scenario: a retry that is correct but still expensive

An inventory request takes 300 milliseconds during a short overload. A client times out after 100 milliseconds and retries twice, each through a different gateway. All three attempts carry one operation key, so only one purchase commits. Business correctness holds. Nevertheless, the service parses three requests, authenticates three sessions or tokens, performs three operation lookups, and eventually formats several responses or notices abandoned connections.

If 50,000 clients follow that pattern, the system can receive 150,000 attempts for 50,000 intentions. Duplicate suppression does not turn the extra 100,000 attempts into zero-cost events. Bounded retries, backoff, admission control, and a sensible end-to-end deadline remain necessary. Retry-safe design and overload-safe design solve related but different problems.

The most efficient recovery can be a status lookup after reconnecting rather than a new execution attempt. The lookup can report completed, still pending under a documented protocol, unknown, or expired. Each state has a defined meaning. In particular, unknown must not be interpreted as safe to repeat when the service has lost the only evidence needed to determine an earlier effect.

A lost-reply sequence preserves one business identity

client                     service                     database
  | purchase key K ---------->| begin transaction -------->|
  |                           | allocate + save K result ->|
  |                           | durable commit <-----------|
  |          reply lost <-----|                            |
  | local timeout             |                            |
  | retry key K ------------->| read recorded outcome ---->|
  | saved order result <------| same business effect        |

The sequence’s authoritative boundary is the committed transaction, not the client’s timeout. The retry retains K but can use another connection and another transport correlation identifier. An actor-scoped uniqueness rule prevents another caller’s operation key from retrieving a result without authorization.

If the database crashes before commit, recovery must establish that no completed purchase exists for K, allowing a later legitimate execution under the protocol. If it crashes after a durable commit, recovery must preserve both the order and K’s saved result. Losing one while retaining the other breaks the claimed retry guarantee. This is why identity storage and business mutation belong in one recoverable atomic unit.

The sequence does not include an external payment charge. Adding that arrow creates a new failure boundary that local atomicity cannot absorb automatically. A payment provider’s idempotency interface, a reservation protocol, or an explicit reconciliation process must connect the two systems. Lesson 37 develops that extension without pretending that reliable transport alone provides exactly-once business effects across services.

Design conclusion

A remote-call interface is a correctness boundary. Its contract must say what a result means, how retries identify the same action, how message sizes are bounded, and what remains uncertain after communication failure. Framing and serialization protect the byte boundary; transactions and idempotency protect the business boundary.

Lesson 6 turns these paths into measurable quantities. The protocol tells us what happened. The measurement model tells us how long it took, how much useful work completed, and whether increasing offered load is producing progress or just larger queues.

Sources and further reading

Course index · Lesson plan