← Questions
Data-parallel and fully sharded matrix multiply over simulated devices
0
I was asked this
No reports yet. Sign in to say where you were asked it.
Simulated devices can only exchange tensors through send and receive calls. Finish a data-parallel matrix multiply, then implement a fully sharded version where each device holds only a slice of the weights and gathers what it needs, and reason about the communication each one costs.
No official answer yet. Write yours on the right and run it.
Nothing published yet.
Sign in to submit your code; every attempt is kept here.
Write and test your solution herePython in your browser, the question's tests level by level, every submission kept for you.Sign in to start coding