#pytorch

17 pytorch interview questions, asked at OpenAI, Amazon, Apple and 11 more companies. Last asked Oct 2026.

Train a model to extract highlight snippets from textML designEach training example is a passage of text with the snippet that should be highlighted and its start and end positions. Complete a PyTorch training pipeline that learns to predict…
Last asked Oct 2026
+1
Find the planted bugs in a small Transformer training scriptCodingA small Transformer, its training script and some tests are provided, with a few bugs hidden in them. Find and fix them so the loss goes down and the outputs are correct, and…
Last asked Oct 2026
+2
Implement multi-head self-attentionML designWrite multi-head self-attention from scratch: project the input to queries, keys and values, split them into heads, compute scaled dot-product attention, combine the heads and…
Last asked Sep 2026
Matrix product scan with autograd and a hand-written backwardCodingGiven a sequence of square matrices, return every prefix product. A first version that updates a buffer in place breaks autograd: say why, and replace it with one that allocates…
Last asked Sep 2026
Minimal reverse-mode autograd engineCodingWith PyTorch available, write the core of an automatic differentiation engine yourself. Build a small Tensor wrapper whose operations record how each result was computed, and a…
Last asked Sep 2026
Review a compact GPT-style transformer implementationML designYou are given a short implementation of a GPT-style decoder-only transformer. Read it and review it: explain what each part does (embeddings, attention with its causal mask…
Last asked Jul 2026
Read and refactor a large PyTorch training fileCodingYou get a few hundred lines of existing PyTorch code that trains a classifier. Answer questions about its time and memory cost, then make a series of changes to it in parts…
Last asked Mar 2026
Debug a shape mismatch in a PyTorch U-NetCodingA U-Net written in PyTorch fails with a tensor shape mismatch. Find the cause (typically where skip connections are concatenated with upsampled features whose size no longer…
Last asked Mar 2026
QKV attention programming with einsum fluencyML designA short programming screen around sequence models and scaled dot-product attention, plus quick ML knowledge questions. Reading equations and expressing the math with einsum-style…
Last asked Feb 2026
Implement multi-head attention and LLM samplingCodingImplement multi-head self-attention from scratch, then the sampling step of a language model: given logits, pick the next token with temperature, top-k and top-p (nucleus)…
Last asked Feb 2026
MLP tensor shapes and PyTorch broadcastingML designGiven a small multi-layer perceptron and a batch of inputs, write down the shape of every weight, bias, activation and gradient. Then explain PyTorch's broadcasting rules, how the…
Last asked Jan 2026
Implement a transformer block with SwiGLUCodingWrite a transformer block from scratch: multi-head self-attention, the residual connections and normalisation, and a feed-forward layer that uses the SwiGLU gate (two parallel…
Last asked Dec 2025
Write a PyTorch training loopCodingWrite a training loop in PyTorch from scratch: build the data loader, run the forward pass, compute the loss, back-propagate, step the optimizer, and evaluate on held-out data…
Last asked Dec 2025
Implement a decoder-only GPT modelCodingWrite a small decoder-only transformer from scratch as four classes: multi-head self-attention with a causal mask, the feed-forward block, a decoder layer that combines them with…
Last asked Nov 2025
Debug a vision transformer notebookCodingA notebook trains a small vision transformer on handwritten digits but performs badly. Find and fix the implementation bugs, paying attention to details like masking and padding…
Last asked Oct 2025
Deep learning frameworks and compilers: from NumPy to PyTorch and JAXML designDiscuss where machine learning frameworks are heading: how they moved from NumPy to PyTorch and JAX, and how PyTorch and JAX differ (eager versus traced execution, functional…
Last asked Sep 2025
+1
Block-wise attention forward pass without the full score matrixCodingImplement scaled dot-product attention that processes queries and keys in blocks with a running softmax normalization, so the quadratic score matrix is never materialized…
Last asked Jul 2025