Predict late shipments end to end from raw data

You get a CSV of shipment events with columns order_id, origin, destination, ship_date, promised_date, carrier, weight, item_count, scan_events[] and delivered_date. Write, from scratch and in Python, a pipeline that predicts which shipments will be late and outputs the top K at-risk shipments. A shipment counts as late when delivered_date - promised_date is more than 48 hours. You write the code and talk through each decision as you go.

Requirements

• Functional:
  • Load and validate the data: missing values, outliers, time zones on every timestamp.
  • Build features: day of week, route, carrier statistics, dwell times between scan events.
  • Label each example late or on time, and decide how to treat shipments that are not delivered yet (censoring).
  • Train a baseline (logistic regression or gradient-boosted trees) with cross-validation; report ROC-AUC and PR-AUC; deal with class imbalance.
  • Calibrate the probabilities and explain the most important features.
  • Write a CSV of the top K at-risk shipments with calibrated probabilities and reason codes.
• Non-functional: under 5 minutes end to end on 1M rows, under 4 GB of memory, and reproducible runs.

Clarifying questions

• When is a prediction made: at ship time, or after each scan? That decides which features are allowed.
• Is promised_date in the shipper's or the recipient's time zone?
• How large is K, and who acts on the list?
• Is a false alarm or a missed late shipment more costly?

What a strong answer covers

• Leakage: no feature may use information after the prediction time (later scans, delivered_date), and a time-based split instead of a random one.
• The censoring decision for undelivered shipments: drop those not yet past the promise, label as late those already more than 48 hours past it.
• Carrier and route statistics computed only on training folds (target encoding without leakage).
• Imbalance handled with class weights or threshold choice, judged on PR-AUC and precision at K, not accuracy.
• Calibration (isotonic or Platt on a held-out fold) and reason codes from per-row feature contributions.
• Meeting the budget: vectorised pandas or Polars, typed columns, histogram-based boosting, fixed seeds and pinned versions.

Follow-ups

• The model will run every hour on new scans. What changes in features and serving?
• PR-AUC is good but the top-K list keeps naming the same carrier. Is that a bug?
• How would you check calibration holds a month later, and when would you retrain?