Lesson 3: Fine-tuning costs — Calculate ROI before starting

Fine-tuning LLM: The Art of AI Tuning

Part 1: Overview & Strategy — When to Fine-tune?

xdev.asia

Introduction

Fine-tuning isn't free — but it's also not as expensive as you might think. This article helps you accurately calculate costs and ROI before committing.

---

1. Training cost formula

Chi phí Training = (Số tokens trong dataset) × (Số epochs) × (Giá per token)

Quick calculation example

• Dataset: 500 examples × ~500 tokens/example = 250,000 tokens
• Epochs: 3
• Total training tokens: 250,000 × 3 = 750,000 tokens

---

2. Comparative price list (2025–2026)

Provider	Model	Training cost	Inference cost
Google Vertex AI	Gemini 2.0 Flash	~$0.40/1M tokens	By base model
OpenAI	GPT-4o-mini	$3.00/1M tokens	2x base models
OpenAI	GPT-4o	$25.00/1M tokens	2x base models
Self-hosted	LLaMA 3 (LoRA)	GPU cost (~$1–3/hour)	Hosting cost only

Google Vertex AI — The biggest advantage

• Training is cheaper than OpenAI
• Inference NO price increase — this is a game changer
• Free trial $300 credit enough for dozens of fine-tunes

---

3. ROI Calculator

def calculate_roi(
    training_cost: float,
    queries_per_day: int,
    prompt_tokens_saved: int,  # Nhờ FT, system prompt ngắn hơn
    price_per_1m_tokens: float,
    days: int = 30
):
    daily_savings = queries_per_day * prompt_tokens_saved * price_per_1m_tokens / 1_000_000
    total_savings = daily_savings * days
    roi_days = training_cost / daily_savings if daily_savings > 0 else float('inf')
    return {
        "training_cost": f"${training_cost:.2f}",
        "monthly_savings": f"${total_savings:.2f}",
        "break_even_days": f"{roi_days:.0f} ngày",
        "6_month_net": f"${total_savings * 6 - training_cost:.2f}"
    }

# Ví dụ: Fine-tune tiết kiệm 2000 tokens/query system prompt
print(calculate_roi(
    training_cost=50,
    queries_per_day=5000,
    prompt_tokens_saved=2000,
    price_per_1m_tokens=0.15  # GPT-4o-mini input
))
# → Break even: ~33 ngày, 6-month net savings: ~$220

---

4. Hidden Costs — Hidden costs

Cost	Description	Estimate
Data preparation	Collect, clean, label data	2–20 engineering hours
Evaluation	Test, iterate, re-train	5–15 hours engineering
Maintenance	Re-train when data changes	2–5 hours/month
Opportunity costs	Other non-working time	Depends on the team

---

5. Budget Planning Template

┌─────────────────────────────────────┐
│  FINE-TUNING BUDGET PLANNER         │
├─────────────────────────────────────┤
│  Phase 1: Data Prep      $0–$200   │
│  Phase 2: Training v1    $10–$100  │
│  Phase 3: Evaluation     $5–$50   │
│  Phase 4: Iterations ×3  $30–$300 │
│  Phase 5: Production     $0–$100  │
│  ───────────────────────────────── │
│  TOTAL                   $45–$750  │
│  (Typical: ~$200)                  │
└─────────────────────────────────────┘

---

Summary

• Cost = Training tokens × Epochs × Price per token
• Cheapest Google Vertex AI for inference (no price increase)
• OpenAI inference is 2x more expensive for fine-tuned models
• Always calculate hidden costs: data prep, evaluation, maintenance
• Fine-tune can SAVE money if prompt length is reduced

Exercises

1. Calculate fine-tune costs for your use case (use ROI calculator)
2. Comparison: Google vs OpenAI for the same dataset size
3. Calculate break-even point: how long does it take for fine-tuning to "pay back"?
4. Create budget proposal for team/manager