← Questions
Scaled dot-product attention from scratch
0
I was asked this
No reports yet. Sign in to say where you were asked it.
Write scaled dot-product attention by hand: from queries, keys and values, compute the scores, scale them by the square root of the key size, apply an optional mask, take a softmax and weight the values. Start from the formula, keep the tensor shapes straight, and extend to several heads if time allows. The code doesn't need to run.
No official answer yet. Write yours on the right and run it.
Nothing published yet.
Sign in to submit your code; every attempt is kept here.
Write and test your solution herePython in your browser, the question's tests level by level, every submission kept for you.Sign in to start coding