Introduction
This is a summary lesson — you will build an end-to-end NLP Platform integrating all the knowledge from the previous 19 lessons. Choose a domain (medical, legal, or e-commerce) and build a complete pipeline.
1. Project Overview
Goal
Building Vietnamese NLP Platform for domain E-commerce with 3 features:
- Sentiment Analysis: Classify product reviews (positive/neutral/negative)
- NER: Extract entities (products, brands, attributes)
- QA: Q&A from product descriptions
Tech Stack
┌─────────────────────────────────────────────────────────┐
│ NLP PLATFORM ARCHITECTURE │
│ │
│ Frontend: Streamlit / Gradio │
│ │ │
│ ▼ │
│ API Layer: FastAPI │
│ │ │
│ ├──▶ Sentiment Model (PhoBERT fine-tuned) │
│ ├──▶ NER Model (PhoBERT + CRF) │
│ └──▶ QA Pipeline (RAG: BGE-M3 + GPT-4o-mini) │
│ │
│ Infrastructure: │
│ ├── PostgreSQL + pgvector (embeddings) │
│ ├── Redis (caching) │
│ ├── Prometheus + Grafana (monitoring) │
│ └── Docker + Docker Compose │
└─────────────────────────────────────────────────────────┘
2. Phase 1: Data Collection & Preparation
import pandas as pd
from underthesea import word_tokenize
from datasets import Dataset
# 1. Thu thập data (từ Shopee reviews, Tiki, etc.)
reviews = pd.read_csv("ecommerce_reviews.csv")
# 2. Preprocessing pipeline
def preprocess(text: str) -> str:
# Word segmentation cho tiếng Việt
segmented = word_tokenize(text, format="text")
return segmented
reviews["processed"] = reviews["text"].apply(preprocess)
# 3. Label annotation
# Sentiment: 0=negative, 1=neutral, 2=positive
# NER: IOB tagging cho entities
# 4. Train/Val/Test split (70/15/15)
from sklearn.model_selection import train_test_split
train, temp = train_test_split(reviews, test_size=0.3, random_state=42)
val, test = train_test_split(temp, test_size=0.5, random_state=42)
3. Phase 2: Model Training
3.1 Sentiment Classification
from transformers import AutoTokenizer, AutoModelForSequenceClassification, Trainer
tokenizer = AutoTokenizer.from_pretrained("vinai/phobert-base-v2")
model = AutoModelForSequenceClassification.from_pretrained(
"vinai/phobert-base-v2", num_labels=3
)
# Fine-tune với Trainer API (xem Bài 13)
# Target: F1-macro > 0.85
3.2 Custom NER
from transformers import AutoModelForTokenClassification
# Custom entity types cho e-commerce:
# PRODUCT, BRAND, ATTRIBUTE, PRICE
labels = ["O", "B-PRODUCT", "I-PRODUCT", "B-BRAND", "I-BRAND",
"B-ATTRIBUTE", "I-ATTRIBUTE", "B-PRICE", "I-PRICE"]
model = AutoModelForTokenClassification.from_pretrained(
"vinai/phobert-base-v2", num_labels=len(labels)
)
# Fine-tune (xem Bài 14)
# Target: Entity-level F1 > 0.80
3.3 RAG-based QA
from sentence_transformers import SentenceTransformer
# Embed product descriptions
embedder = SentenceTransformer("BAAI/bge-m3")
# Store embeddings in pgvector
# Retrieve relevant descriptions → GPT-4o-mini answer
4. Phase 3: API Development
from fastapi import FastAPI
from pydantic import BaseModel
app = FastAPI(title="Vietnamese E-commerce NLP Platform")
class AnalysisRequest(BaseModel):
text: str
class AnalysisResponse(BaseModel):
sentiment: dict # {label, score}
entities: list # [{text, type, score}]
summary: str | None
@app.post("/analyze", response_model=AnalysisResponse)
async def analyze(request: AnalysisRequest):
"""Phân tích toàn diện: sentiment + NER + summary."""
# 1. Sentiment
sentiment_result = sentiment_model(request.text)
# 2. NER
entities = ner_model(request.text)
# 3. Summary (nếu text dài)
summary = None
if len(request.text) > 200:
summary = summarizer(request.text)
return AnalysisResponse(
sentiment=sentiment_result,
entities=entities,
summary=summary,
)
@app.post("/qa")
async def question_answer(question: str, product_id: str):
"""RAG-based QA cho product."""
# 1. Retrieve relevant passages
# 2. Generate answer with LLM
pass
5. Phase 4: Deployment & Monitoring
Docker Compose
# docker-compose.yml
services:
api:
build: .
ports: ["8000:8000"]
environment:
- MODEL_PATH=/models
- DEVICE=cpu
postgres:
image: pgvector/pgvector:pg16
volumes: ["pgdata:/var/lib/postgresql/data"]
redis:
image: redis:alpine
prometheus:
image: prom/prometheus
volumes: ["./prometheus.yml:/etc/prometheus/prometheus.yml"]
grafana:
image: grafana/grafana
ports: ["3000:3000"]
6. Evaluation Checklist
| Criteria | Target | Metrics |
|---|---|---|
| Sentiment Accuracy | > 85% | F1-macro |
| NER Quality | > 80% | Entity F1 |
| QA Relevance | > 90% | Human eval |
| API Latency | < 200ms | p95 latency |
| Uptime | > 99.5% | Availability |
7. Career Roadmap
NLP Engineer Level Map:
Junior (0-2 năm):
├── Thành thạo Python, PyTorch
├── Hiểu Transformer, BERT, GPT
├── Fine-tune pre-trained models
└── Basic deployment (FastAPI)
Mid-level (2-4 năm):
├── Design NLP pipelines end-to-end
├── RAG architecture
├── Model optimization (quantization, ONNX)
├── MLOps practices
└── Domain expertise (healthcare, legal, finance)
Senior (4+ năm):
├── Architecture decisions (NLP vs LLM vs hybrid)
├── Scale to millions of requests
├── Research & implement latest papers
├── Lead NLP team
└── Cost optimization strategies
Series Summary
After 20 lessons, you will have mastered:
| Part | Knowledge |
|---|---|
| Platform | Preprocessing, Tokenization |
| Performing | BoW, TF-IDF, Word2Vec, Sentence Embeddings |
| Deep Learning | RNN, LSTM, Attention, Transformer |
| Pre-trained | BERT, GPT, Hugging Face ecosystem |
| Application | Classification, NER, QA, Summarization, Translation |
| Production | Vietnamese NLP, MLOps, LLM trends, Capstone |
🎓 Congratulations on completing the NLP series from Basic to Advanced!