#llm evaluation

6 llm evaluation interview questions, asked at Cohere, Mercor, Figma and 1 more company. Last asked Sep 2026.

Post-train a 7B model to code better for one enterpriseML designAn enterprise customer wants a 7-billion-parameter model that writes better code for them. Design the post-training pipeline end to end: collecting and cleaning data, supervised…
Last asked Sep 2026
Design a RAG question-answering agent over a firm's documentsAI system designDesign a system that answers questions from a large, private collection of documents (for example a law firm's memos) with retrieval-augmented generation, citing the passages it…
Last asked Sep 2026
Design an evaluation platform for a design-editing AI agentAI system designAn AI agent edits design files on request. Design the platform that evaluates it: the test cases, how an edited design is judged (automatic checks, model graders, human review)…
Last asked Jul 2026
Design candidate search for a talent marketplaceML designDesign a system that lets a company search for candidates and ranks them for a role: how candidate profiles are represented and indexed, how a query or job description retrieves…
Last asked May 2026
Build an eval harness for local models served by OllamaAI system designA take-home of about four hours (AI coding tools allowed): write an evaluation harness that runs a set of test cases against language models served locally through Ollama, scores…
Last asked Mar 2026
Explain LLM training, reinforcement learning and evaluationML designWalk through how a large language model is trained, from pretraining to fine-tuning, where reinforcement learning comes in after that, and how you evaluate the model at each…
Last asked Dec 2025