Waiting for a service that may not answer

The payment example leaves a question that applies to many other services: what should the app do while it has no answer? In code, the call may sit beside our own functions. In use, it crosses a network and reaches a system we do not control. Either can delay or fail independently of our app.

Take a small example: a booking page that calls an external availability API before showing free slots, inside the request a visitor is waiting on. Everything below is about that one call.

Give the call an end

Without an explicit deadline you may be relying on a client or platform default that is much longer than your product can tolerate, and every waiting request holds a connection and some memory open for nothing. Set a timeout deliberately, based on how long the visitor can reasonably wait.

Then remember what a timeout means: you stopped waiting. It does not prove the other side failed. The request may have arrived and done its work, with only the answer lost. For a side-effect-free read, like fetching availability, retrying does not create a second booking. For anything that changes state, like making the booking, it does, because retrying blindly can book twice. That is why retries of writes need the idempotency keys from the chapter on double payments.

Retry, but not forever

Retrying is right because many failures are momentary: a dropped connection, or a 503 during someone else's deploy. It stops being right when it never ends, or when you retry an answer that will not change. A 400 means your request was wrong, and asking again is asking a locked door the same question in the same voice. Classify errors for this operation. Timeouts, connection failures and some 5xx responses may be transient. A 429 is a 4xx exception that may invite a later retry; respect Retry-After where supplied. Do not retry unchanged invalid credentials or invalid parameters, and do not assume every 5xx is safe to repeat.

Space retries out with exponential backoff, doubling the wait each time, and cap both the wait and the number of attempts. Then add jitter, a random amount on top. If every client backs off by the same doubling numbers, they have effectively agreed on when to knock, and a service coming back up meets all of them at once. The randomness breaks that agreement.

For a hypothetical read, allow 4 attempts, each with a 3-second timeout. Choose a random delay between zero and 0.2 seconds before the second attempt, then up to 0.4 seconds, then up to 0.8 seconds. The configured attempt deadlines and waits total at most 13.4 seconds, before other overhead. An overall deadline can stop the operation sooner. Your HTTP client must actually honor cancellation; merely stopping your own wait does not stop remote work.

Calculate the same budget for your own call. Is it longer than the interaction can tolerate? If so, reduce the work or use an asynchronous design rather than hiding another retry inside the wait.

Stop calling, and say so

After enough failures in a row, stop calling for a while. That is a circuit breaker, named after the switch that cuts power to protect the house, not the appliance. While the breaker is open, calls fail immediately, and your page can say that availability is temporarily unavailable instead of hanging and saying nothing. After a pause, the breaker lets one call through to test whether the service is back.

A fallback is often better than an error. If you cached the last successful availability response, you can show it, but show its age next to it, as in: last updated 14:05. Stale data presented as fresh is a bug of its own.

A cached availability list may help someone browse, but booking a slot still needs a fresh authoritative decision. Choose the fallback according to what the user will do with it; a timestamp alone cannot make stale inventory safe to sell.

Take slow work out of the request

For work that does not need to finish before the user continues, another design is to move the call out of the request. Put the job in a queue, answer the visitor at once, and let a worker do the slow part with its own retries. Work that fails for good goes to a dead letter queue, which is only worth having if somebody reads it.

Exercise

In a development copy, replace the external service with a test double that first delays, then returns an error, then succeeds. Check the number of attempts, the total wait and the message the user sees. Keep this experiment off real payment and booking endpoints.