Hugging Face domain depth interview

Round 5 of 6 · 3 questions reported

See the whole loop
Format
One or two discussions, depending on the role
They grade
Tradeoffs in ML systems, distributed training, inference serving or library design
Heads-up
Reported prompts include KV cache eviction for very long contexts and dense versus mixture-of-experts at a fixed budget.

Questions Hugging Face asks in this round 3

  • Dense versus mixture-of-experts at fixed computeAI/ML designYour team will train its next model with a fixed budget of activated parameters, the parameters each token actually passes through, which sets the compute per token. Should it be…
  • KV cache eviction for million-token contextsSystem designA model serving system must handle contexts of up to a million tokens, but the KV cache for that many tokens doesn't fit in GPU memory, and it competes with the cache of every…
  • Web-scale crawl cleaning for pretraining dataSystem designDesign a batch pipeline that turns a raw web crawl (petabytes of pages) into a pretraining corpus for a language model: text extracted, deduplicated, filtered for quality and…