簡介
這是一堂總結課——您將建立一個端到端的 NLP 平台，整合之前 19 課中的所有知識。選擇一個領域（醫療、法律或電子商務）並建立完整的管道。
1. 專案概況
目標
為電子商務領域建立越南 NLP 平台，具有 3 個功能：
- 情緒分析：將產品評論分類（正面/中性/負面）
- NER：提取實體（產品、品牌、屬性）
- QA：產品描述中的問答
技術堆疊
┌─────────────────────────────────────────────────────────┐
│ NLP PLATFORM ARCHITECTURE │
│ │
│ Frontend: Streamlit / Gradio │
│ │ │
│ ▼ │
│ API Layer: FastAPI │
│ │ │
│ ├──▶ Sentiment Model (PhoBERT fine-tuned) │
│ ├──▶ NER Model (PhoBERT + CRF) │
│ └──▶ QA Pipeline (RAG: BGE-M3 + GPT-4o-mini) │
│ │
│ Infrastructure: │
│ ├── PostgreSQL + pgvector (embeddings) │
│ ├── Redis (caching) │
│ ├── Prometheus + Grafana (monitoring) │
│ └── Docker + Docker Compose │
└─────────────────────────────────────────────────────────┘
2. 第一階段：資料收集與準備
import pandas as pd
from underthesea import word_tokenize
from datasets import Dataset
# 1. Thu thập data (từ Shopee reviews, Tiki, etc.)
reviews = pd.read_csv("ecommerce_reviews.csv")
# 2. Preprocessing pipeline
def preprocess(text: str) -> str:
# Word segmentation cho tiếng Việt
segmented = word_tokenize(text, format="text")
return segmented
reviews["processed"] = reviews["text"].apply(preprocess)
# 3. Label annotation
# Sentiment: 0=negative, 1=neutral, 2=positive
# NER: IOB tagging cho entities
# 4. Train/Val/Test split (70/15/15)
from sklearn.model_selection import train_test_split
train, temp = train_test_split(reviews, test_size=0.3, random_state=42)
val, test = train_test_split(temp, test_size=0.5, random_state=42)
3.第二階段：模型訓練
3.1 情緒分類
from transformers import AutoTokenizer, AutoModelForSequenceClassification, Trainer
tokenizer = AutoTokenizer.from_pretrained("vinai/phobert-base-v2")
model = AutoModelForSequenceClassification.from_pretrained(
"vinai/phobert-base-v2", num_labels=3
)
# Fine-tune với Trainer API (xem Bài 13)
# Target: F1-macro > 0.85
3.2 Custom NER
from transformers import AutoModelForTokenClassification
# Custom entity types cho e-commerce:
# PRODUCT, BRAND, ATTRIBUTE, PRICE
labels = ["O", "B-PRODUCT", "I-PRODUCT", "B-BRAND", "I-BRAND",
"B-ATTRIBUTE", "I-ATTRIBUTE", "B-PRICE", "I-PRICE"]
model = AutoModelForTokenClassification.from_pretrained(
"vinai/phobert-base-v2", num_labels=len(labels)
)
# Fine-tune (xem Bài 14)
# Target: Entity-level F1 > 0.80
3.3 基於 RAG 的 QA
from sentence_transformers import SentenceTransformer
# Embed product descriptions
embedder = SentenceTransformer("BAAI/bge-m3")
# Store embeddings in pgvector
# Retrieve relevant descriptions → GPT-4o-mini answer
4. 第三階段：API 開發
from fastapi import FastAPI
from pydantic import BaseModel
app = FastAPI(title="Vietnamese E-commerce NLP Platform")
class AnalysisRequest(BaseModel):
text: str
class AnalysisResponse(BaseModel):
sentiment: dict # {label, score}
entities: list # [{text, type, score}]
summary: str | None
@app.post("/analyze", response_model=AnalysisResponse)
async def analyze(request: AnalysisRequest):
"""Phân tích toàn diện: sentiment + NER + summary."""
# 1. Sentiment
sentiment_result = sentiment_model(request.text)
# 2. NER
entities = ner_model(request.text)
# 3. Summary (nếu text dài)
summary = None
if len(request.text) > 200:
summary = summarizer(request.text)
return AnalysisResponse(
sentiment=sentiment_result,
entities=entities,
summary=summary,
)
@app.post("/qa")
async def question_answer(question: str, product_id: str):
"""RAG-based QA cho product."""
# 1. Retrieve relevant passages
# 2. Generate answer with LLM
pass
5. 第 4 階段：部署與監控
Docker Compose
# docker-compose.yml
services:
api:
build: .
ports: ["8000:8000"]
environment:
- MODEL_PATH=/models
- DEVICE=cpu
postgres:
image: pgvector/pgvector:pg16
volumes: ["pgdata:/var/lib/postgresql/data"]
redis:
image: redis:alpine
prometheus:
image: prom/prometheus
volumes: ["./prometheus.yml:/etc/prometheus/prometheus.yml"]
grafana:
image: grafana/grafana
ports: ["3000:3000"]
6. 評估清單
| 標準 | 目標 | 指標 |
|---|---|---|
| Sentiment Accuracy | > 85% | F1-macro |
| NER 品質 | > 80% | Entity F1 |
| 品質檢查相關性 | > 90% | 人類評估 |
| API Latency | < 200ms | p95 latency |
| Uptime | > 99.5% | 可用性 |
7. Career Roadmap
NLP Engineer Level Map:
Junior (0-2 năm):
├── Thành thạo Python, PyTorch
├── Hiểu Transformer, BERT, GPT
├── Fine-tune pre-trained models
└── Basic deployment (FastAPI)
Mid-level (2-4 năm):
├── Design NLP pipelines end-to-end
├── RAG architecture
├── Model optimization (quantization, ONNX)
├── MLOps practices
└── Domain expertise (healthcare, legal, finance)
Senior (4+ năm):
├── Architecture decisions (NLP vs LLM vs hybrid)
├── Scale to millions of requests
├── Research & implement latest papers
├── Lead NLP team
└── Cost optimization strategies
系列總結
20 堂課後，您將掌握：
| 部分 | Knowledge |
|---|---|
| 平台 | 預處理、標記化 |
| Performing | BoW、TF-IDF、Word2Vec、句子嵌入 |
| 深度學習 | RNN、LSTM、注意力、變壓器 |
| 預訓練 | BERT、GPT、Hugging Face 生態 |
| 應用 | 分類、NER、QA、摘要、翻譯 |
| 生產 | 越南 NLP、MLOps、LLM 趨勢、Capstone |
🎓 **恭喜您完成 NLP 系列（從基本到高級）！ **