Design Google News: group articles into stories and rank them

Design a news aggregator like Google News. It gathers articles from thousands of publishers, groups articles about the same event into one story, and shows each reader a fresh, personalized front page.

Requirements

• Functional: crawl or receive articles from thousands of publishers; extract title, text, images, time and topic; group articles about the same story; rank stories for a general front page and per reader; sections by topic and region; search.
• Non-functional: new articles appear within minutes; front pages load fast for many millions of readers; ranking favors fresh, important and trustworthy sources; scales to many languages and regions.

Clarifying questions

• Do publishers push feeds, or do we crawl their sites?
• How fresh must breaking news be?
• How much personalization, and what signals may we use?
• How many regions and languages?
• Do we show full articles or link out to publishers?

What a strong answer covers

• Ingestion: feed polling and crawling with per-publisher schedules and politeness, deduplication of the same article, extraction of content.
• Story clustering: text embeddings or similarity, incremental clustering as new articles arrive, and splitting or merging clusters as a story evolves.
• Ranking: freshness, number and quality of sources, and reader interest; a general ranking computed offline or every few minutes, with personalization applied at request time.
• Serving: precomputed front pages per region and topic in a cache, personal layers merged on request.
• Feedback loops from clicks, and avoiding a feed that only shows one viewpoint or clickbait.
• Scaling per language and region, and monitoring of freshness.

Follow-ups

• A major event breaks. How quickly does it reach the front page, and how?
• A publisher starts posting misleading articles.
• Personalize for readers who are not signed in.
• One story splits into two separate stories over a day.