簡介
動態消息是每個社群平台的核心功能。每個用戶都有自己的動態，這些動態由數千名朋友/追蹤者聚合而成，並按相關性排名。這是一個難題，因為規模非常大。
1. 要求與估算
Functional:
- User tạo post (text, image, video)
- News Feed hiển thị posts từ friends/followings
- Feed ranked theo relevance (không chỉ chronological)
- Support likes, comments, shares
- Real-time: post mới xuất hiện trong feed bạn bè
Non-Functional:
- Feed load < 500ms
- 500M DAU
- Average 500 friends per user
Estimation:
Feed requests: 500M × 10 views/day = 5B feed requests/day
QPS: 5B / 86400 ≈ 58K QPS (peak: 150K)
Posts/day: 500M × 2 posts = 1B posts/day
Post size: ~1KB text + pointers → 1TB/day
2. 核心元件
┌──────────────────────────────────────────────────────┐
│ │
│ ┌──────────┐ Post Service │
│ │ User │───► ┌──────────────┐ │
│ │ creates │ │ Post Storage │ │
│ │ post │ │ (posts DB) │ │
│ └──────────┘ └──────┬───────┘ │
│ │ │
│ ┌──────▼───────┐ │
│ │ Fan-out │ │
│ │ Service │ │
│ └──────┬───────┘ │
│ │ │
│ ┌─────────▼──────────┐ │
│ │ Feed Cache │ │
│ │ (per-user feed) │ │
│ └─────────┬──────────┘ │
│ │ │
│ ┌──────────┐ ┌──────▼───────┐ │
│ │ User │───►│ Feed Service │ │
│ │ reads │ │ (retrieve) │ │
│ │ feed │ └──────────────┘ │
│ └──────────┘ │
│ │
└──────────────────────────────────────────────────────┘
3. 扇出策略
3.1 寫入時扇出（推送模型）
User A posts → Write to ALL followers' feed cache
User A has 1000 followers
Post → Fan-out Service:
Feed[follower_1].prepend(post)
Feed[follower_2].prepend(post)
...
Feed[follower_1000].prepend(post)
✅ Feed read: Instant (pre-computed)
❌ Write: Slow for celebrities (10M followers!)
❌ Wasted work: inactive users' feeds updated
❌ Hot key: Celebrity's post → 10M cache writes
3.2 讀取時扇出（拉模型）
User B opens feed → Query all friends' posts, merge, rank
User B has 500 friends
Feed request:
Get posts from friend_1 (last 24h)
Get posts from friend_2 (last 24h)
...
Merge + Rank + Return top 20
✅ Write: Instant (just store post)
❌ Read: Slow (500 queries per feed request!)
❌ High read latency
3.3 混合（Facebook/Twitter 方法）
Normal users (< 10K followers): Fan-out on Write
→ Pre-compute feeds, instant reads
Celebrities (> 10K followers): Fan-out on Read
→ Don't pre-compute, merge at read time
Feed Read:
1. Get pre-computed feed (from cache)
2. Get celebrity posts (from post store)
3. Merge + Re-rank
4. Return top N
┌────────────────────────────────────────┐
│ User B's Feed │
│ │
│ Pre-computed cache: [post5, post3, ...]│
│ + │
│ Celebrity posts: [celeb_post_1, ...] │
│ = │
│ Merged + Ranked: [final feed] │
└────────────────────────────────────────┘
4. Feed 排名
Không chỉ chronological, mà ranked by relevance:
Score = f(affinity, weight, time_decay)
Affinity: Bạn tương tác với author bao nhiêu?
(likes, comments, messages, profile views)
Weight: Post type importance
Video > Photo > Link > Text
Comments > Likes
Time Decay: Post cũ hơn → score giảm
score × (1 / time_since_posted^1.5)
Simplified Ranking:
score = (likes × 1 + comments × 3 + shares × 5)
× affinity_score
× time_decay_factor
ML-based:
Feature engineering → Train model
Features: user engagement history, post features,
social graph, time context
Model: Predict P(user engages with post)
5. Feed 快取設計
Per-user feed cache (Redis):
Key: feed:{user_id}
Value: List of post_ids (last 1000)
feed:user_123 → [post_999, post_998, post_995, ...]
Feed retrieval:
Page 1: LRANGE feed:user_123 0 19 (posts 1-20)
Page 2: LRANGE feed:user_123 20 39 (posts 21-40)
Post details:
Separately cached:
post:999 → { author, text, image_url, likes, ... }
Feed = post_ids from user cache
+ post details from post cache
+ author info from user cache
→ Assemble in API server
Cache Eviction:
- Keep last 1000 posts per user
- TTL: 7 days (re-compute if expired)
- Active users: always fresh (fan-out keeps updating)
- Inactive users: compute on demand
6. 即時更新
Long Polling vs WebSocket vs SSE:
Long Polling: Client polls every 30s
Simple, but delayed, wasteful
WebSocket: Persistent connection
Real-time, but resource-heavy (50M connections!)
SSE (Server-Sent Events): Server push, HTTP-based
Simpler than WebSocket, one-directional
Approach (Facebook-style):
- Initial load: REST API (full feed)
- Updates: Long polling / SSE (new posts notification)
- "3 new posts available" → Click to load
- NOT auto-inject (breaks reading flow)
總結
| 決定 | 選擇 | 原因 |
|---|---|---|
| 扇出 | 混合動力 | 平衡所有使用者類型的寫入/讀取 |
| 儲存 | Cassandra（貼文）+ Redis（提要） | 大量寫入 + 快速讀取 |
| 排行榜 | 基於機器學習的分數 | 相關性 > 新近度 |
| 即時 | 上交所+通知 | 不完整的WebSocket |
| 快取 | 每用戶提要快取 | 預先計算以實現快速閱讀 |
練習
-
名人問題： 用戶擁有 5000 萬粉絲貼文。寫入時扇出 → 50M 快取更新。設計最大延遲 5 秒的解決方案。
-
**提要多樣性：**提要不應是一個人的所有帖子。設計去重+分集演算法。
-
廣告整合： 將贊助貼文插入 Feed（每 5 個貼文 1 個廣告）。設計不影響 Feed 延遲的廣告插入策略。