#latency

6 latency interview questions, asked at Apple, Decagon, Google and 3 more companies. Last asked Oct 2026.

Check and insert among a million user IDsSystem designA service holds about a million user IDs. Each request carries a user ID: if it is not in the set yet, add it. Decide how to store the IDs and run this check-and-insert, and talk…
Last asked Oct 2026
Design a real-time AI voice agentAI system designDesign a system in which callers talk to an AI agent over the phone or in an app in real time: speech recognition, the language model, speech synthesis, keeping the response delay…
Last asked Feb 2026
Fast expensive tokens versus slow cheap tokensAI system designAn LLM provider offers fast but expensive tokens and slow but cheap ones. How would you decide which to use for a product? Discuss which features need low latency, batch versus…
Last asked Jan 2026
Design large-scale model inference serving with back-of-envelope sizingAI system designDesign an inference system for a model used by 100 million daily active users. Do the back-of-envelope estimates for memory and bandwidth, then explain how you would reduce…
Last asked Nov 2025
Resize a hash map without long pausesCodingExplain how a hash map works and how it grows its bucket array. Resizing all at once can stall a request for a long time on a big map. How would you avoid that? Discuss…
Last asked Oct 2025
Design an answer engine with citationsAI system designDesign an answer engine: a user asks a question in plain language and gets a direct, current answer written by a model, with numbered citations to the web pages it used, streaming…