← Questions
Implement a transformer block with SwiGLU
0
I was asked this
No reports yet. Sign in to say where you were asked it.
Write a transformer block from scratch: multi-head self-attention, the residual connections and normalisation, and a feed-forward layer that uses the SwiGLU gate (two parallel projections, one passed through SiLU, multiplied together, then projected back). Explain the tensor shapes at each step.
No official answer yet. Write yours on the right and run it.
Nothing published yet.
Sign in to submit your code; every attempt is kept here.
Write and test your solution herePython in your browser, the question's tests level by level, every submission kept for you.Sign in to start coding