Google DeepMind system design interview

Round 5 of 7 · 2 questions reported

See the whole loop

Length: 60 min
Format: Design discussion
They grade: Tradeoffs you can justify, not a list of technologies
AI use: Not allowed or heavily restricted
Heads-up: Research engineers design distributed training and evaluation infrastructure instead of product systems.

Questions Google DeepMind asks in this round 2

Design fault-tolerant training for a frontier model on 16k acceleratorsSystem designA 400B-parameter model trains on about 16,000 accelerators for two months. At that scale hardware fails every few hours and each failure stops the synchronous job. Design the…
Evaluate a new LLM before launchML designYou have a new model checkpoint. Design the evaluation process that decides whether it ships.