Lesson 18: KMIP, Consul, Nomad Secrets Engines and Custom Plugins

HashiCorp Vault from Basic to Advanced

Part 4: Advanced Secrets Engines

xdev.asia

1. KMIP Secrets Engine (Enterprise)

KMIP (Key Management Interoperability Protocol) is the OASIS standard protocol for cryptographic keys management. KMIP Secrets Engine allows Vault to act as a KMIP Server, providing key management for KMIP-enabled databases, storage systems, and applications.

Use Cases

• MongoDB Enterprise — encryption at rest with KMIP

• MySQL Enterprise — TDE (Transparent Data Encryption)

• VMware vSphere — VM encryption

• NetApp — storage encryption

Setup KMIP

# Enable KMIP
vault secrets enable kmip

# Cấu hình KMIP listener
vault write kmip/config \
  listen_addrs="0.0.0.0:5696" \
  tls_ca_key_type="ec" \
  tls_ca_key_bits=256 \
  default_tls_client_key_type="ec" \
  default_tls_client_key_bits=256

# Tạo scope (logical separation)
vault write -f kmip/scope/mongodb

# Tạo role trong scope
vault write kmip/scope/mongodb/role/admin \
  operation_activate=true \
  operation_create=true \
  operation_destroy=true \
  operation_discover_versions=true \
  operation_get=true \
  operation_locate=true \
  operation_rekey=true

# Sinh client certificate cho MongoDB
vault write -format=json kmip/scope/mongodb/role/admin/credential/generate \
  format=pem > mongodb-kmip-creds.json

# Extract cert và key
cat mongodb-kmip-creds.json | jq -r '.data.certificate' > client.pem
cat mongodb-kmip-creds.json | jq -r '.data.private_key' >> client.pem
cat mongodb-kmip-creds.json | jq -r '.data.ca_chain[]' > ca.pem

MongoDB with KMIP Vault

# mongod.conf
security:
  enableEncryption: true
  kmip:
    serverName: vault.company.com
    port: 5696
    clientCertificateFile: /etc/mongodb/client.pem
    serverCAFile: /etc/mongodb/ca.pem
    keyStatePollingSeconds: 60

2. Consul Secrets Engine

Consul Secrets Engine generates dynamic Consul ACL tokens, allowing applications to access the Consul service mesh and KV store with short-term credentials.

# Enable Consul secrets engine
vault secrets enable consul

# Cấu hình kết nối Consul
vault write consul/config/access \
  address="consul.company.com:8500" \
  token="<consul-management-token>"

# Tạo role
vault write consul/roles/app-readonly \
  policies="app-readonly-policy" \
  ttl=1h \
  max_ttl=4h

# Consul policy (tạo trước trên Consul)
# app-readonly-policy:
#   key_prefix "app/" { policy = "read" }
#   service_prefix "" { policy = "read" }

# Sinh dynamic Consul token
vault read consul/creds/app-readonly
# token: 2f76e8b4-a3d0-...
# lease_duration: 1h

# Token tự động revoke khi hết lease

3. Nomad Secrets Engine

Nomad Secrets Engine sinh dynamic Nomad ACL tokens.

# Enable Nomad secrets engine
vault secrets enable nomad

# Cấu hình
vault write nomad/config/access \
  address="https://nomad.company.com:4646" \
  token="<nomad-management-token>"

# Tạo role
vault write nomad/role/deploy \
  policies="deploy-policy" \
  type="client" \
  ttl=30m

# Sinh Nomad token
vault read nomad/creds/deploy
# secret_id: 5e1c1a...
# accessor_id: 3d2b...

4. Vault Plugin System

Vault has a plugin-based architecture — all secrets engines and auth methods are plugins. You can develop custom plugins using Go.

Plugin Architecture

┌─────────────────────────────────────────────────┐
│                  Vault Server                   │
│                                                 │
│  ┌──────────────┐  ┌──────────────┐            │
│  │ Built-in     │  │ External     │            │
│  │ Plugins      │  │ Plugins      │            │
│  │ (kv, transit,│  │ (custom,     │            │
│  │  pki, aws)   │  │  community)  │            │
│  └──────────────┘  └──────┬───────┘            │
│                           │                     │
│                    gRPC over Unix Socket         │
│                    (mutually authenticated)      │
│                           │                     │
│                    ┌──────┴───────┐             │
│                    │ Plugin Binary│             │
│                    │ (separate    │             │
│                    │  process)    │             │
│                    └──────────────┘             │
└─────────────────────────────────────────────────┘

Plugin Catalog

# Liệt kê built-in plugins
vault plugin list

# Liệt kê chỉ secrets plugins
vault plugin list secret

# Liệt kê auth plugins
vault plugin list auth

# Đăng ký custom plugin
vault plugin register -sha256="$(sha256sum vault-plugin-myengine | cut -d' ' -f1)" \
  secret vault-plugin-myengine

# Enable custom plugin
vault secrets enable -path=myengine vault-plugin-myengine

Developing Custom Secrets Engine

// main.go
package main

import (
    "os"
    "github.com/hashicorp/go-hclog"
    "github.com/hashicorp/vault/api"
    "github.com/hashicorp/vault/sdk/framework"
    "github.com/hashicorp/vault/sdk/logical"
    "github.com/hashicorp/vault/sdk/plugin"
)

func main() {
    apiClientMeta := &api.PluginAPIClientMeta{}
    flags := apiClientMeta.FlagSet()
    flags.Parse(os.Args[1:])

    tlsConfig := apiClientMeta.GetTLSConfig()
    tlsProviderFunc := api.VaultPluginTLSProvider(tlsConfig)

    err := plugin.ServeMultiplex(&plugin.ServeOpts{
        BackendFactoryFunc: Factory,
        TLSProviderFunc:    tlsProviderFunc,
    })
    if err != nil {
        logger := hclog.New(&hclog.LoggerOptions{})
        logger.Error("plugin shutting down", "error", err)
        os.Exit(1)
    }
}

func Factory(ctx context.Context, conf *logical.BackendConfig) (logical.Backend, error) {
    b := &backend{}
    b.Backend = &framework.Backend{
        Help: "My custom secrets engine",
        BackendType: logical.TypeLogical,
        Paths: []*framework.Path{
            b.pathCreds(),
            b.pathConfig(),
        },
        Secrets: []*framework.Secret{
            b.secretCreds(),
        },
    }
    if err := b.Setup(ctx, conf); err != nil {
        return nil, err
    }
    return b, nil
}

type backend struct {
    *framework.Backend
}

Plugin Multiplexing

Plugin multiplexing (Vault 1.12+) allows one plugin process to serve multiple mounts, reducing resource usage:

// Sử dụng plugin.ServeMultiplex thay vì plugin.Serve
err := plugin.ServeMultiplex(&plugin.ServeOpts{
    BackendFactoryFunc: Factory,
    TLSProviderFunc:    tlsProviderFunc,
})

Versioned Plugins

# Đăng ký plugin version mới
vault plugin register \
  -sha256="..." \
  -version="v2.0.0" \
  secret vault-plugin-myengine

# Liệt kê versions
vault plugin info secret vault-plugin-myengine

# Pin mount đến version cụ thể
vault secrets tune -plugin-version="v2.0.0" myengine/

# Reload plugin (zero-downtime)
vault plugin reload -plugin vault-plugin-myengine

5. Notable Community Plugins

Plugin	Description
vault-plugin-secrets-github	Dynamic GitHub tokens
vault-plugin-secrets-kafka	Kafka credentials
vault-plugin-secrets-artifactory	JFrog Artifactory tokens
vault-plugin-auth-kerberos	Kerberos/SPNEGO auth
vault-plugin-secrets-openldap	OpenLDAP credentials

6. Summary

• KMIP — standard key management, MongoDB/MySQL/VMware integration

• Consul/Nomad Engines — dynamic ACL tokens cho HashiCorp ecosystem

• Plugin System — extensible architecture, custom plugins using Go

• Plugin Multiplexing — performance improvement cho multiple mounts

The next section will explore Vault Agent, Vault Proxy, and Kubernetes Integration — how to deliver secrets to applications automatically.