Bài 6: Prompt Engineering Engine — Template System, Chain-of-Thought & Dynamic Prompts

Kiến trúc Enterprise AI Chatbot Platform — Từ Prototype đến Production

Phần 2: Core Chatbot Engine

xdev.asia

1. Prompt as Code — Tại sao cần Prompt Engine?

Trong production, prompt không phải "một chuỗi text hardcode". Prompt cần được quản lý, versioned, A/B tested, và compose dynamically dựa trên context.

┌─────────────────── PROMPT ENGINE ───────────────────────┐
│                                                          │
│  ┌──────────┐  ┌──────────┐  ┌──────────┐  ┌────────┐  │
│  │ Template │  │ Variable │  │ Persona  │  │  A/B   │  │
│  │ Registry │  │ Resolver │  │ Manager  │  │ Tester │  │
│  └────┬─────┘  └────┬─────┘  └────┬─────┘  └───┬────┘  │
│       │              │             │             │       │
│       └──────────────┼─────────────┘             │       │
│                      │                           │       │
│                ┌─────▼─────┐              ┌──────▼────┐  │
│                │  Prompt   │              │  Version  │  │
│                │ Assembler │              │  Manager  │  │
│                └─────┬─────┘              └───────────┘  │
│                      │                                   │
│                ┌─────▼─────┐                             │
│                │ Final     │                             │
│                │ Prompt    │──▶ LLM                      │
│                └───────────┘                             │
└──────────────────────────────────────────────────────────┘

2. Template System

interface PromptTemplate {
  id: string;
  name: string;
  version: number;
  tenantId: string;
  category: 'system' | 'few_shot' | 'instruction' | 'output_format';
  template: string;       // Handlebars template
  variables: VariableDefinition[];
  metadata: {
    author: string;
    description: string;
    createdAt: Date;
    isActive: boolean;
    abTestGroup?: string;
  };
}

interface VariableDefinition {
  name: string;
  type: 'string' | 'array' | 'object' | 'boolean';
  required: boolean;
  defaultValue?: unknown;
  source: 'context' | 'config' | 'runtime' | 'rag' | 'memory';
}

class PromptTemplateEngine {
  private handlebars = Handlebars.create();

  constructor() {
    // Register custom helpers
    this.handlebars.registerHelper('ifEquals', function (arg1, arg2, options) {
      return arg1 === arg2 ? options.fn(this) : options.inverse(this);
    });

    this.handlebars.registerHelper('truncate', function (str: string, len: number) {
      return str.length > len ? str.substring(0, len) + '...' : str;
    });

    this.handlebars.registerHelper('formatDate', function (date: string) {
      return new Date(date).toLocaleDateString('vi-VN');
    });
  }

  compile(template: PromptTemplate, variables: Record<string, unknown>): string {
    const compiled = this.handlebars.compile(template.template);
    return compiled(variables);
  }
}

3. System Prompt Design Pattern

// Production system prompt template
const SYSTEM_PROMPT_TEMPLATE = `
You are {{persona.name}}, {{persona.description}}.

## Your Role
{{persona.role_description}}

## Instructions
{{#each instructions}}
- {{this}}
{{/each}}

## Knowledge Context
{{#if rag_context}}
Use the following knowledge to answer the user's question. Cite sources using [1], [2] notation.
If the knowledge doesn't contain the answer, say you don't have enough information.

{{rag_context}}
{{/if}}

## User Memory
{{#if user_memory}}
What you know about this user:
{{#each user_memory}}
- {{this.content}}
{{/each}}
{{/if}}

## Conversation Summary
{{#if conversation_summary}}
Previous conversation summary: {{conversation_summary}}
{{/if}}

## Output Rules
{{#each output_rules}}
- {{this}}
{{/each}}

## Available Tools
{{#if tools}}
You have access to these tools:
{{#each tools}}
- **{{this.name}}**: {{this.description}}
{{/each}}
Use tools when needed. Do NOT make up information.
{{/if}}

## Language
Always respond in {{language}}.
Current date: {{current_date}}.
`;

4. Persona Manager

interface Persona {
  id: string;
  name: string;
  description: string;
  role_description: string;
  tone: 'formal' | 'friendly' | 'professional' | 'casual';
  language: string;
  instructions: string[];
  output_rules: string[];
  forbidden_topics: string[];
}

const DEFAULT_PERSONAS: Record<string, Persona> = {
  'customer-support': {
    id: 'customer-support',
    name: 'Support Assistant',
    description: 'a helpful customer support agent',
    role_description: 'You help customers with their questions about products, orders, and account issues.',
    tone: 'friendly',
    language: 'vi',
    instructions: [
      'Be empathetic and patient with customers',
      'If you cannot help, offer to escalate to a human agent',
      'Never share internal system details or pricing formulas',
      'Always verify order numbers before making changes',
    ],
    output_rules: [
      'Keep responses concise (under 200 words unless detailed explanation needed)',
      'Use bullet points for lists',
      'Include next steps when applicable',
    ],
    forbidden_topics: ['competitor pricing', 'internal metrics', 'employee information'],
  },
  'knowledge-assistant': {
    id: 'knowledge-assistant',
    name: 'Knowledge Assistant',
    description: 'an internal knowledge base assistant',
    role_description: 'You help employees find information from company documentation and policies.',
    tone: 'professional',
    language: 'vi',
    instructions: [
      'Always cite sources with document names and sections',
      'If unsure, say so and suggest who to contact',
      'Provide step-by-step guides when explaining processes',
    ],
    output_rules: [
      'Format responses with clear headings',
      'Include links to source documents when available',
    ],
    forbidden_topics: ['salary information', 'personal employee data'],
  },
};

5. Dynamic Prompt Assembly

class PromptAssembler {
  constructor(
    private templateEngine: PromptTemplateEngine,
    private personaManager: PersonaManager,
    private templateRegistry: TemplateRegistry,
  ) {}

  async assemble(context: AssemblyContext): Promise<AssembledPrompt> {
    // 1. Get persona
    const persona = await this.personaManager.getPersona(
      context.tenantId,
      context.personaId,
    );

    // 2. Get template (check A/B test)
    const template = await this.templateRegistry.getActiveTemplate(
      context.tenantId,
      'system',
      context.abTestGroup,
    );

    // 3. Resolve variables
    const variables: Record<string, unknown> = {
      persona,
      rag_context: context.ragContext,
      user_memory: context.userMemory,
      conversation_summary: context.conversationSummary,
      tools: context.tools,
      language: persona.language === 'vi' ? 'Tiếng Việt' : 'English',
      current_date: new Date().toLocaleDateString('vi-VN'),
      instructions: persona.instructions,
      output_rules: persona.output_rules,
    };

    // 4. Compile
    const systemPrompt = this.templateEngine.compile(template, variables);

    // 5. Build few-shot examples if available
    const fewShotMessages = await this.buildFewShotExamples(context);

    return {
      systemPrompt,
      fewShotMessages,
      templateVersion: template.version,
      personaId: persona.id,
      abTestGroup: context.abTestGroup,
    };
  }

  private async buildFewShotExamples(context: AssemblyContext): Promise<Message[]> {
    const examples = await this.templateRegistry.getFewShotExamples(
      context.tenantId,
      context.personaId,
      3, // Max 3 examples
    );

    return examples.flatMap(ex => [
      { role: 'user' as const, content: ex.userMessage },
      { role: 'assistant' as const, content: ex.assistantMessage },
    ]);
  }
}

6. Prompt Versioning & Rollback

class PromptVersionManager {
  async createVersion(
    tenantId: string,
    templateId: string,
    newTemplate: string,
    changelog: string,
  ): Promise<PromptTemplate> {
    const current = await this.db.promptTemplate.findActive(tenantId, templateId);
    const newVersion = (current?.version ?? 0) + 1;

    const created = await this.db.promptTemplate.create({
      ...current,
      id: crypto.randomUUID(),
      template: newTemplate,
      version: newVersion,
      metadata: {
        ...current?.metadata,
        changelog,
        createdAt: new Date(),
        isActive: false, // Not active until explicitly activated
      },
    });

    return created;
  }

  async activateVersion(tenantId: string, templateId: string, version: number): Promise<void> {
    // Deactivate current
    await this.db.promptTemplate.updateMany(
      { tenantId, name: templateId, 'metadata.isActive': true },
      { 'metadata.isActive': false },
    );

    // Activate target version
    await this.db.promptTemplate.update(
      { tenantId, name: templateId, version },
      { 'metadata.isActive': true },
    );
  }

  async rollback(tenantId: string, templateId: string): Promise<void> {
    const versions = await this.db.promptTemplate.findAll({
      tenantId, name: templateId,
      orderBy: 'version', order: 'desc', limit: 2,
    });

    if (versions.length < 2) throw new Error('No previous version to rollback to');
    await this.activateVersion(tenantId, templateId, versions[1].version);
  }
}

7. Prompt A/B Testing

class PromptABTester {
  async assignGroup(
    tenantId: string,
    userId: string,
    experimentId: string,
  ): Promise<string> {
    // Consistent hashing — same user always gets same group
    const hash = this.hashString(`${tenantId}:${userId}:${experimentId}`);
    const experiment = await this.db.experiment.findById(experimentId);

    let cumulative = 0;
    for (const variant of experiment.variants) {
      cumulative += variant.trafficPercent;
      if (hash <= cumulative) return variant.group;
    }

    return experiment.variants[0].group; // Fallback to control
  }

  async trackOutcome(
    experimentId: string,
    group: string,
    metrics: {
      responseQuality?: number;  // LLM-judge score 1-5
      userSatisfaction?: number; // Thumbs up/down
      resolutionRate?: boolean;  // Was issue resolved?
      latencyMs?: number;
    },
  ): Promise<void> {
    await this.db.experimentResult.create({
      experimentId,
      group,
      metrics,
      timestamp: new Date(),
    });
  }

  private hashString(str: string): number {
    let hash = 0;
    for (let i = 0; i < str.length; i++) {
      const char = str.charCodeAt(i);
      hash = ((hash << 5) - hash) + char;
      hash = hash & hash; // Convert to 32-bit integer
    }
    return Math.abs(hash) % 100; // 0-99
  }
}

Tổng kết Bài 6

• Prompt engine = Template System + Variable Resolver + Persona Manager + Version Control
• System prompt gồm: persona + instructions + RAG context + memory + tools + output rules
• Persona quản lý tone, language, instructions, forbidden topics per use case
• Prompt versioning cho phép rollback khi prompt mới giảm quality
• A/B testing với consistent hashing — same user luôn thấy same variant

Bài tiếp theo: Streaming & Real-time — SSE/WebSocket streaming, voice agent (STT + TTS), multimodal input, latency optimization.