1. Transform Secrets Engine Overview
Transform Secrets Engine (Enterprise) provides advanced data protection mechanisms, allowing sensitive data to be protected while retaining the original format — unlike Transit Engine, which returns ciphertext in a completely different format.
Three types of Transform
| Type | Description | Reversible | Use case |
|---|---|---|---|
| FPE | Format Preserving Encryption | Yes | Credit card, SSN — keep original format |
| Masking | Mask data with characters | No | Display "****1234" |
| Tokenization | Replace with random token | Yes (lookup) | PCI DSS, de-scope environments |
2. Format Preserving Encryption (FPE)
FPE encrypts the data but retains the original format. For example, a 16-digit credit card number is encoded into 16 other digits. This allows legacy systems to continue operating without schema changes.
Setup FPE
# Enable Transform secrets engine
vault secrets enable transform
# Tạo alphabet (tập ký tự cho phép)
# Built-in: builtin/numeric, builtin/alphanumericlower, builtin/alphanumericupper
# Tạo template cho credit card
vault write transform/template/credit-card-tmpl \
type=regex \
pattern='(\d{4})-(\d{4})-(\d{4})-(\d{4})' \
alphabet=builtin/numeric \
encode_format='$1-$2-$3-$4' \
decode_format='$1-$2-$3-$4'
# Tạo transformation
vault write transform/transformations/fpe/credit-card \
template=credit-card-tmpl \
tweak_source=internal \
allowed_roles=payments
# Tạo role
vault write transform/role/payments \
transformations=credit-card
Use FPE
# Encode (encrypt)
vault write transform/encode/payments \
value="4111-1111-1111-1111" \
transformation=credit-card
# encoded_value: 7492-8372-1938-4726
# Decode (decrypt)
vault write transform/decode/payments \
value="7492-8372-1938-4726" \
transformation=credit-card
# decoded_value: 4111-1111-1111-1111
# Batch operations
vault write transform/encode/payments \
batch_input='[
{"value": "4111-1111-1111-1111", "transformation": "credit-card"},
{"value": "5500-0000-0000-0004", "transformation": "credit-card"}
]'
FPE cho SSN/CCCD
# Template cho SSN (xxx-xx-xxxx)
vault write transform/template/ssn-tmpl \
type=regex \
pattern='(\d{3})-(\d{2})-(\d{4})' \
alphabet=builtin/numeric
# Template cho CCCD Việt Nam (12 chữ số)
vault write transform/template/cccd-tmpl \
type=regex \
pattern='(\d{12})' \
alphabet=builtin/numeric
vault write transform/transformations/fpe/cccd \
template=cccd-tmpl \
tweak_source=internal \
allowed_roles=patient-data
3. Masking
Masking masks data with replacement characters — cannot reverse. Suitable for displaying on UI, logs, reports.
# Tạo masking transformation
vault write transform/transformations/masking/card-mask \
template=credit-card-tmpl \
masking_character="#" \
allowed_roles=display
vault write transform/role/display \
transformations=card-mask
# Mask
vault write transform/encode/display \
value="4111-1111-1111-1111" \
transformation=card-mask
# encoded_value: ####-####-####-1111
4. Tokenization
Tokenization replaces sensitive data with a random token, storing the mapping in a separate store. Tokens have no mathematical relationship with the original data → more secure than FPE.
Tokenization vs FPE
| Criteria | FPE | Tokenization |
|---|---|---|
| Keep original format | Yes | Optional |
| Reversible | Yes (by key) | Yes (by lookup store) |
| Need external store | No | Yes (internal or external) |
| PCI DSS de-scope | No (still needs key management) | Yes (environments without keys) |
| Performance | Fast (crypto operation) | Slower (storage lookup) |
# Tạo tokenization store (internal)
vault write transform/stores/internal-store \
type=internal
# Tạo tokenization transformation
vault write transform/transformations/tokenization/card-token \
allowed_roles=tokenize-role \
stores=internal-store \
max_ttl=8760h
vault write transform/role/tokenize-role \
transformations=card-token
# Tokenize
vault write transform/encode/tokenize-role \
value="4111111111111111" \
transformation=card-token
# encoded_value: Q4hx8AZDhk3jfn9876XYZ123...
# Detokenize
vault write transform/decode/tokenize-role \
value="Q4hx8AZDhk3jfn9876XYZ123..." \
transformation=card-token
# decoded_value: 4111111111111111
# Kiểm tra token metadata
vault write transform/tokeninfo/tokenize-role \
value="Q4hx8AZDhk3jfn9876XYZ123..." \
transformation=card-token
5. Transit vs Transform — When to use which?
| Criteria | Transit | Transform |
|---|---|---|
| License | Community | Enterprise |
| Output format | Base64 ciphertext | Keep original format (FPE) or token |
| Schema changes | Need to change column type | No need to change |
| Use case main | Encrypt at rest | PCI DSS, PII protection |
| Batch performance | Very good | Good (FPE), fair (Tokenization) |
| Key rotation | Yes (rewrap) | Yes |
Practical example
Transit: Encrypt emails, addresses, medical data in the database → long ciphertext, need decrypt to use
Transform FPE: Encode credit card number → legacy billing system can still handle 16-digit format
Transform Tokenization: Replace credit card into token → dev/test environment completely de-scoped from PCI
Transform Masking: Display ****1234 on UI → cannot recover original data
6. Integration with Database Views
-- PostgreSQL view sử dụng Vault Transform
-- Application gọi Vault API để decode khi cần
-- Table gốc lưu tokenized data
CREATE TABLE orders (
id SERIAL PRIMARY KEY,
customer_name TEXT,
card_token TEXT, -- Tokenized value
card_last4 TEXT, -- Masked value cho display
amount DECIMAL(10,2),
created_at TIMESTAMPTZ DEFAULT NOW()
);
-- Application flow:
-- 1. User nhập card number
-- 2. App gửi đến Vault Transform → nhận token + masked
-- 3. Lưu token vào card_token, masked vào card_last4
-- 4. Khi cần charge → app gửi token đến Vault → nhận card number gốc
-- 5. Charge card rồi xóa plaintext khỏi memory
7. Summary
FPE — format-preserving encoding, compatible with legacy systems
Masking — masking data, irreversible, for UI/logs
Tokenization — replace with random token, de-scope environments
Transit — basic encryption, Community edition
Transform — advanced data protection, Enterprise
The next article will explore KMIP, Consul, Nomad Secrets Engines and how to develop Custom Plugins for Vault.