Design a durable cron job scheduler

Design a service that runs recurring jobs on cron-style schedules and does not lose them when machines fail. Cover how schedules are stored and turned into the next run time, how due runs are handed to workers, how a run that crashed is detected and retried without running twice, and what happens to runs that were missed while the scheduler was down.