Neural network forward and backward pass from scratch

Build a small neural network and a loss function by hand, without an autodiff library. Write the forward pass, then derive and code the backward pass that computes the gradient of the loss with respect to every parameter, and use it to update the weights. Be ready to walk through the derivation step by step and explain how you would check that your gradients are right.