Lesson 17: Deploy Agent to Production — FastAPI, Docker & Cloud

Build AI Agents: From Zero to Production

Part 6: Production & Actual Deployment

xdev.asia

Introduction

Agent running in notebook ≠ agent running in production. This article covers the entire pipeline from wrap agent to API → containerize → deploy → scale.

---

1. FastAPI Wrapper

from fastapi import FastAPI, WebSocket
from pydantic import BaseModel

app = FastAPI()

class AgentRequest(BaseModel):
    message: str
    session_id: str = None

@app.post("/chat")
async def chat(request: AgentRequest):
    agent = get_or_create_agent(request.session_id)
    response = await agent.run(request.message)
    return {"response": response, "session_id": agent.session_id}

@app.websocket("/ws/chat")
async def websocket_chat(websocket: WebSocket):
    await websocket.accept()
    agent = create_agent()
    while True:
        data = await websocket.receive_text()
        async for chunk in agent.stream(data):
            await websocket.send_text(chunk)

2. Docker

FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

3. Scaling Considerations

• Session affinity for stateful agents
• Redis for shared state
• Queue-based processing for long-running tasks
• Cost budgets per user/session

---

Summary

• FastAPI + WebSocket for real-time agent API
• Docker for reproducible deployments
• Cloud deployment: AWS ECS, GCP Cloud Run, Railway
• Session management & scaling are key challenges

Exercises

1. Wrap SimpleAgent into FastAPI app
2. Dockerize and test locally
3. Implement WebSocket streaming
4. Deploy to Railway or Cloud Run