簡介
物件偵測為我們提供了一個邊界框－圍繞物件的矩形。但許多應用需要更精確：醫學需要知道腫瘤的正確區域，自動駕駛汽車需要知道正確的像素哪些是道路，哪些是人行道。這就是圖像分割。
🎯 分割 = 將影像中的每個像素分類。比檢測更詳細。
1. 三種細分類型
1.1 比較
Input Image: 2 con mèo + 1 nền cỏ
Semantic Segmentation: Tất cả pixels mèo → "cat" (KHÔNG phân biệt 2 con)
Tất cả pixels cỏ → "grass"
Instance Segmentation: Mèo 1 → "cat #1", Mèo 2 → "cat #2" (PHÂN BIỆT)
Cỏ → không xử lý (chỉ "things")
Panoptic Segmentation: Mèo 1 → "cat #1", Mèo 2 → "cat #2"
Cỏ → "grass" (xử lý CẢ "things" lẫn "stuff")
| 類型 | 區分物體？ | 「東西」（背景）？ | 使用案例 |
|---|---|---|---|
| 語意 | ❌ | ✅ | 自動駕駛（道路、天空）、醫療 |
| 實例 | ✅ | ❌ | 機器人，計數物體 |
| 全景 | ✅ | ✅ | 全面場景了解 |
1.2 細分指標
"""IoU (Intersection over Union) cho Segmentation"""
import numpy as np
def pixel_iou(pred_mask, gt_mask, class_id):
"""IoU cho 1 class cụ thể"""
pred = (pred_mask == class_id)
gt = (gt_mask == class_id)
intersection = np.logical_and(pred, gt).sum()
union = np.logical_or(pred, gt).sum()
return intersection / union if union > 0 else 0
def mean_iou(pred_mask, gt_mask, num_classes):
"""Mean IoU: trung bình IoU tất cả classes"""
ious = []
for c in range(num_classes):
iou = pixel_iou(pred_mask, gt_mask, c)
ious.append(iou)
return np.mean(ious)
# Ví dụ:
# mIoU = 0.75 → trung bình, model cover 75% diện tích đúng
2. 分段架構
2.1 U-Net—醫學影像分割之王
Encoder (downsampling) Decoder (upsampling)
┌─────────────────┐ ┌─────────────────┐
│ Conv 3×3, 64 │────────────│ Conv 3×3, 64 │ ← Skip Connection
│ MaxPool 2×2 │ │ UpConv 2×2 │
├─────────────────┤ ├─────────────────┤
│ Conv 3×3, 128 │────────────│ Conv 3×3, 128 │ ← Skip Connection
│ MaxPool 2×2 │ │ UpConv 2×2 │
├─────────────────┤ ├─────────────────┤
│ Conv 3×3, 256 │────────────│ Conv 3×3, 256 │ ← Skip Connection
│ MaxPool 2×2 │ │ UpConv 2×2 │
├─────────────────┤ ├─────────────────┤
│ Bottleneck (512) │
└─────────────────┘ └─────────────────┘
"""U-Net implementation simplified"""
import torch
import torch.nn as nn
class UNet(nn.Module):
def __init__(self, in_channels=3, num_classes=2):
super().__init__()
# Encoder
self.enc1 = self._double_conv(in_channels, 64)
self.enc2 = self._double_conv(64, 128)
self.enc3 = self._double_conv(128, 256)
self.pool = nn.MaxPool2d(2, 2)
# Bottleneck
self.bottleneck = self._double_conv(256, 512)
# Decoder
self.up3 = nn.ConvTranspose2d(512, 256, 2, stride=2)
self.dec3 = self._double_conv(512, 256) # 256 + 256 = 512 input
self.up2 = nn.ConvTranspose2d(256, 128, 2, stride=2)
self.dec2 = self._double_conv(256, 128)
self.up1 = nn.ConvTranspose2d(128, 64, 2, stride=2)
self.dec1 = self._double_conv(128, 64)
# Output
self.out = nn.Conv2d(64, num_classes, 1)
def _double_conv(self, in_ch, out_ch):
return nn.Sequential(
nn.Conv2d(in_ch, out_ch, 3, padding=1),
nn.BatchNorm2d(out_ch),
nn.ReLU(inplace=True),
nn.Conv2d(out_ch, out_ch, 3, padding=1),
nn.BatchNorm2d(out_ch),
nn.ReLU(inplace=True),
)
def forward(self, x):
# Encoder
e1 = self.enc1(x)
e2 = self.enc2(self.pool(e1))
e3 = self.enc3(self.pool(e2))
# Bottleneck
b = self.bottleneck(self.pool(e3))
# Decoder + Skip Connections
d3 = self.dec3(torch.cat([self.up3(b), e3], dim=1))
d2 = self.dec2(torch.cat([self.up2(d3), e2], dim=1))
d1 = self.dec1(torch.cat([self.up1(d2), e1], dim=1))
return self.out(d1)
model = UNet(in_channels=3, num_classes=5)
x = torch.randn(1, 3, 256, 256)
print(f"Output shape: {model(x).shape}") # (1, 5, 256, 256)
2.2 SegFormer — 用於分段的轉換器
"""SegFormer: model segmentation hiện đại dùng Transformer"""
from transformers import SegformerForSemanticSegmentation, SegformerImageProcessor
from PIL import Image
import torch
# Load pretrained SegFormer
processor = SegformerImageProcessor.from_pretrained(
"nvidia/segformer-b2-finetuned-ade-512-512"
)
model = SegformerForSemanticSegmentation.from_pretrained(
"nvidia/segformer-b2-finetuned-ade-512-512"
)
# Inference
image = Image.open("street_scene.jpg")
inputs = processor(images=image, return_tensors="pt")
with torch.no_grad():
outputs = model(**inputs)
# Upscale to original size
logits = torch.nn.functional.interpolate(
outputs.logits,
size=image.size[::-1], # (H, W)
mode="bilinear",
align_corners=False,
)
# Predicted mask
predicted_mask = logits.argmax(dim=1).squeeze().numpy()
print(f"Mask shape: {predicted_mask.shape}") # (H, W)
print(f"Classes found: {set(predicted_mask.flatten())}")
2.3 YOLO 分割
"""Instance Segmentation với YOLO"""
from ultralytics import YOLO
model = YOLO("yolo11n-seg.pt") # Segmentation model
results = model("people_park.jpg")
for result in results:
# Masks
if result.masks is not None:
masks = result.masks.data.cpu().numpy() # (N, H, W)
print(f"Found {len(masks)} instance masks")
for i, (mask, box) in enumerate(zip(masks, result.boxes)):
class_name = result.names[int(box.cls)]
confidence = box.conf[0].item()
mask_area = mask.sum() # Pixels in mask
print(f" {class_name} ({confidence:.1%}): {mask_area:.0f} pixels")
# Visualize
result.show()
3. 實踐：醫學影像分割
"""Semantic Segmentation cho ảnh y tế — ví dụ: segment tumor"""
import torch
import torch.nn as nn
from torch.utils.data import Dataset, DataLoader
import cv2
import numpy as np
class MedicalDataset(Dataset):
def __init__(self, image_paths, mask_paths, size=256):
self.image_paths = image_paths
self.mask_paths = mask_paths
self.size = size
def __len__(self):
return len(self.image_paths)
def __getitem__(self, idx):
# Read image
img = cv2.imread(self.image_paths[idx])
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
img = cv2.resize(img, (self.size, self.size))
img = img.astype(np.float32) / 255.0
img = torch.from_numpy(img).permute(2, 0, 1) # (C, H, W)
# Read mask
mask = cv2.imread(self.mask_paths[idx], cv2.IMREAD_GRAYSCALE)
mask = cv2.resize(mask, (self.size, self.size), interpolation=cv2.INTER_NEAREST)
mask = torch.from_numpy(mask).long()
return img, mask
# Training với Dice Loss (tốt cho medical segmentation)
class DiceLoss(nn.Module):
def forward(self, pred, target, smooth=1e-6):
pred = torch.softmax(pred, dim=1)
target_one_hot = torch.nn.functional.one_hot(
target, num_classes=pred.shape[1]
).permute(0, 3, 1, 2).float()
intersection = (pred * target_one_hot).sum(dim=(2, 3))
union = pred.sum(dim=(2, 3)) + target_one_hot.sum(dim=(2, 3))
dice = (2 * intersection + smooth) / (union + smooth)
return 1 - dice.mean()
4. 實際應用
🏥 Y tế: Segment khối u, tế bào, cơ quan
🚗 Tự lái: Segment đường, vỉa hè, biển báo, người
🗺️ Bản đồ: Segment buildings, roads từ ảnh vệ tinh
🌾 Nông nghiệp: Segment cây trồng, detect vùng bệnh
📱 Điện thoại: Portrait mode (tách người khỏi nền)
🏭 Công nghiệp: Kiểm tra bề mặt sản phẩm
總結
| 概念 | 記住 |
|---|---|
| 語意 | 將每個像素分類→類別（不論實例為何） |
| 實例 | 區分單一物件 |
| 全景 | 結合語意+實例 |
| 優網 | 編碼器-解碼器 + 跳過連接（醫療） |
| SegFormer | 基於變壓器，SOTA |
| YOLO-Seg | 即時實例分割 |
| 骰子損失 | 適合分割的損失（類別不平衡） |
一般練習
- SegFormer 示範： 在 5 張街道照片上執行預先訓練的 SegFormer。可視化顏色蒙版。
- YOLO-Seg： 短視頻上的實例分割。計算唯一實例。
- U-Net Mini： 在二進位資料集（前景/背景）上訓練簡單的 U-Net。
- Mask Overlay： 在原圖上寫一個函數疊加分割mask（半透明）。
下一篇文章： SAM（分割任何東西）－分割任何物件無需訓練。