← Questions
Multi-head attention with a KV cache and grouped-query attention
0
I was asked this
No reports yet. Sign in to say where you were asked it.
Implement the attention layer of a decoder-only transformer from scratch in three steps that build on each other:
- Standard multi-head attention with a causal mask.
- A key-value cache so generation can add one token at a time without recomputing earlier keys and values.
- The forward pass of grouped-query attention, where several query heads share one key-value head, using the cache.
Keep the tensor shapes explicit and say how much memory the cache saves in step 3.
No official answer yet. Draw yours on the right; publish it and it shows under Community.
Nothing published yet. Publish your canvas and it shows here for others to open and remix.
Draw your answer hereBoxes, arrows and notes, saved for you to come back to.Sign in to start drawing
