Design a scheduler for bursts of short serverless tasks

Tenants submit short serverless tasks, each needing some CPU, memory and maybe a GPU, a container image, possibly data on certain machines, a deadline and a priority. A burst can bring 10,000 tasks in one second. Design the scheduler that places them.

Cover:

• accepting a burst without losing tasks: validation, quotas, durable admission
• fairness between tenants, so one tenant's burst can't starve the rest
• choosing a machine: fit, image and data locality, deadlines, and avoiding fragmented capacity
• cancelling queued or running tasks, and recovering when a scheduler or a machine dies without running a task's result twice