Design a streaming AI feature for a large audience

A consumer product wants to add an AI feature whose answers stream to millions of users in real time. Design the path from the client to the model: how responses stream, how requests are admitted and queued when capacity runs short, how cost is kept under control, and how the client behaves when a stream fails. Cover what you would monitor.