Shard a large input embedding table across devices

A model's [V, H] input embedding table is too big for one device. Compare sharding along the vocabulary with sharding along the hidden dimension: how a lookup works in each, the communication needed, load balance across devices, and how the output feeds the next layers.