+2Transformer and attention fundamentalsML designExplain how a transformer works from the inside: how self-attention is computed, why the scores are scaled, how encoder and decoder masking differ, and what multiple heads buy…
Last asked Sep 2026
#attention
- +2
+2Implement multi-head self-attentionML designWrite multi-head self-attention from scratch: project the input to queries, keys and values, split them into heads, compute scaled dot-product attention, combine the heads and…Last asked Sep 2026- Stable softmax and multi-head attention in NumPyCodingIn NumPy, implement a numerically stable sigmoid and softmax, then scaled dot-product attention, then multi-head attention from scratch.Last asked Jul 2026
QKV attention programming with einsum fluencyML designA short programming screen around sequence models and scaled dot-product attention, plus quick ML knowledge questions. Reading equations and expressing the math with einsum-style…Last asked Feb 2026
Implement multi-head attention and LLM samplingCodingImplement multi-head self-attention from scratch, then the sampling step of a language model: given logits, pick the next token with temperature, top-k and top-p (nucleus)…Last asked Feb 2026- Multi-head attention with a KV cache and grouped-query attentionML designImplement the attention layer of a decoder-only transformer from scratch in three steps that build on each other: 1. Standard multi-head attention with a causal mask. 2. A…Last asked Jan 2026
Implement a transformer block with SwiGLUCodingWrite a transformer block from scratch: multi-head self-attention, the residual connections and normalisation, and a feed-forward layer that uses the SwiGLU gate (two parallel…Last asked Dec 2025
Write a 2-D convolution, a biased attention step and a training loopCodingImplement small ML building blocks from scratch: a 2-D convolution in NumPy, an attention step with an added bias term before the softmax, and the missing pieces of a standard…Last asked Dec 2025
Scaled dot-product attention from scratchCodingWrite scaled dot-product attention by hand: from queries, keys and values, compute the scores, scale them by the square root of the key size, apply an optional mask, take a…Last asked Sep 2025
Attention and a small Transformer with a hand-written backward passCodingImplement scaled dot-product attention and a small autoregressive Transformer block from scratch, including the causal mask. Then write the backward pass by hand (no autograd) for…Last asked Aug 2025
+1Block-wise attention forward pass without the full score matrixCodingImplement scaled dot-product attention that processes queries and keys in blocks with a running softmax normalization, so the quadratic score matrix is never materialized…Last asked Jul 2025- KV cache eviction for million-token contextsAI system designDesign an eviction policy for serving very long contexts on limited GPU memory, comparing sliding windows, attention sinks, heavy-hitter eviction, quantization and tiered or…