#rlhf

8 rlhf interview questions, asked at Google, Amazon, Cohere and 3 more companies. Last asked Sep 2026.

Post-train a 7B model to code better for one enterpriseML designAn enterprise customer wants a 7-billion-parameter model that writes better code for them. Design the post-training pipeline end to end: collecting and cleaning data, supervised…
Last asked Sep 2026
+2
Reward-model RLHF versus direct preference optimizationML designWalk through a classic RLHF pipeline: supervised fine-tuning, a reward model trained on preference pairs, then policy optimization (PPO) with a KL penalty to the reference model…
Last asked Sep 2026
KL divergence in language modelsML designDefine KL divergence and explain what it measures. Then explain where it is used with large language models and why, for example as a penalty that keeps a fine-tuned model close…
Last asked Aug 2026
Detect and prevent reward hackingML designExplain reward hacking when a language model is trained with reinforcement learning: what it looks like, how you would detect it, and what you would change in the reward, the data…
Last asked Aug 2026
Design an LLM chatbot: training data, rewards and deploymentAI system designDesign a chatbot built on a large language model and walk through its whole lifecycle. Go deep on three areas: how you build the training data (sources, filtering, labeling), how…
Last asked Jan 2026
GRPO in NumPy: group-relative advantages, process and shaped rewardsML designIn a notebook, implement the core math of Group Relative Policy Optimization with NumPy. For each prompt the policy samples a group of completions, each completion gets a score…
Last asked Jan 2026
Explain LLM training, reinforcement learning and evaluationML designWalk through how a large language model is trained, from pretraining to fine-tuning, where reinforcement learning comes in after that, and how you evaluate the model at each…
Last asked Dec 2025
RLHF take-home aligning a small open modelCodingA take-home over a few days: align a small open model with preference data and a parameter-efficient adapter on a single GPU, push the result to the Hugging Face Hub, and write up…