← Questions
Design a batch job orchestrator over a rate-limited LLM API
0
I was asked this
No reports yet. Sign in to say where you were asked it.
Operators submit jobs of up to 5,000 tasks, stored as JSON documents in MongoDB. Each task is finished by an outside LLM provider: a blocking, rate-limited API that takes no more than 10 tasks per call. Design the job service and its workers: batching tasks into requests, admission and rate limits, idempotency, retries and partial failures, per-task progress that operators can watch, and a final job result. Note that a timed-out request may already have run at the provider.
No official answer yet. Draw yours on the right; publish it and it shows under Community.
Nothing published yet. Publish your canvas and it shows here for others to open and remix.
Draw your answer hereBoxes, arrows and notes, saved for you to come back to.Sign in to start drawing
