Design a model inference pipeline

Billed as ML design but really ML infrastructure: design a pipeline that serves a trained model's predictions. Cover how requests or data arrive, batching, GPU use, model versions and rollout, latency and throughput, and monitoring.