Minimal reverse-mode autograd engineCodingWith PyTorch available, write the core of an automatic differentiation engine yourself. Build a small Tensor wrapper whose operations record how each result was computed, and a…
Last asked Sep 2026
#backpropagation
Softmax cross-entropy forward and backward, then a training loopCodingImplement from scratch, with NumPy but no autograd, the forward and backward pass of a softmax layer followed by cross-entropy loss. Keep it numerically stable. Then use it in a…Last asked Sep 2026- Sharded matrix multiply: forward, backward and planted bugsCodingSplit a matrix multiplication across several devices and explain how the pieces combine. Write the forward pass and the gradient computation for that layout in NumPy or PyTorch…Last asked Jul 2026
Neural network forward and backward pass from scratchCodingBuild a small neural network and a loss function by hand, without an autodiff library. Write the forward pass, then derive and code the backward pass that computes the gradient of…Last asked Sep 2025
Attention and a small Transformer with a hand-written backward passCodingImplement scaled dot-product attention and a small autoregressive Transformer block from scratch, including the causal mask. Then write the backward pass by hand (no autograd) for…Last asked Aug 2025