Design a job scheduler

Design a service that runs jobs for other teams: jobs can run once, at a given time or on a recurring schedule. Cover how jobs are submitted and stored, how the scheduler finds jobs that are due without missing or double-running them, how work is handed to workers, retries and failure handling, and how the system scales and survives a scheduler crash.