← Questions
Shard an LM head and combine the logits
0
I was asked this
No reports yet. Sign in to say where you were asked it.
The output layer maps a hidden state through an [H, V] weight matrix to vocabulary logits. Compare splitting the weight by vocabulary with splitting it by hidden dimension: what each device computes, which reductions are needed, and whether full logits must be gathered before top-k selection or sampling.
No official answer yet. Draw yours on the right; publish it and it shows under Community.
Nothing published yet. Publish your canvas and it shows here for others to open and remix.
Draw your answer hereBoxes, arrows and notes, saved for you to come back to.Sign in to start drawing
