← Questions
Implement multi-head self-attention
0
I was asked this
No reports yet. Sign in to say where you were asked it.
Write multi-head self-attention from scratch: project the input to queries, keys and values, split them into heads, compute scaled dot-product attention, combine the heads and apply the output projection. Get the shapes and transposes right, and add a causal mask so a position cannot attend to later positions. Be ready to explain the scaling factor and the cost in sequence length.
No official answer yet. Draw yours on the right; publish it and it shows under Community.
Nothing published yet. Publish your canvas and it shows here for others to open and remix.
Draw your answer hereBoxes, arrows and notes, saved for you to come back to.Sign in to start drawing


