← Questions
Attention and a small Transformer with a hand-written backward pass
0
I was asked this
No reports yet. Sign in to say where you were asked it.
Implement scaled dot-product attention and a small autoregressive Transformer block from scratch, including the causal mask. Then write the backward pass by hand (no autograd) for the pieces the interviewer picks, and check your gradients numerically.
No official answer yet. Write yours on the right and run it.
Nothing published yet.
Sign in to submit your code; every attempt is kept here.
Write and test your solution herePython in your browser, the question's tests level by level, every submission kept for you.Sign in to start coding