HashiCorp Vault ที่ทำให้ Secrets ปลอดภัยจริงๆ

วันที่ Secrets Management เป็น Nightmare

ก่อนรู้จัก HashiCorp Vault การจัดการ secrets เป็นเรื่องน่ากลัว:

วิธีการเก็บ Secrets แบบเก่า (อันตราย!):

# 1. Environment Variables (ใน production server)
export DATABASE_PASSWORD="supersecretpassword123"
export API_KEY="sk-abcdefghijklmnop1234567890"
export JWT_SECRET="myverysecretjwttoken"

# 2. Configuration Files (committed ใน Git!)
# config/production.yaml
database:
  host: prod-db.example.com
  username: root
  password: "supersecretpassword123"  # 😱

# 3. Docker Environment Files
# .env (ถูก commit เข้า Git!)
DATABASE_URL=postgresql://user:password@db:5432/myapp
STRIPE_SECRET_KEY=sk_live_abcdefghijklmnop

# 4. Kubernetes Secrets (base64 encoded)
apiVersion: v1
kind: Secret
metadata:
  name: app-secrets
data:
  password: c3VwZXJzZWNyZXRwYXNzd29yZDEyMw==  # แค่ base64!

ปัญหาที่เจอบ่อย:

Secrets ใน Git: Commit แล้วลบไม่ได้ (Git history)
Plain Text Storage: ไม่ได้ encrypt จริง
No Rotation: ใช้ password เดิมตลอด
No Audit: ไม่รู้ใครเข้าถึง secrets เมื่อไหร่
Hard to Share: ส่ง secrets ผ่าน Slack/Email
No Access Control: ใครมี access ก็เห็นได้หมด
Environment Drift: secrets แต่ละ env ไม่เหมือนกัน

ตัวอย่างความวุ่นวาย:

# สถานการณ์จริงที่เกิดขึ้น 😅

# Developer commit secrets โดยไม่ตั้งใจ
git add .
git commit -m "Add production config"
git push origin main

# Later...
git log --oneline
# a1b2c3d Add production config  <- Secrets exposed!

# พยายาม "ลบ" 
git reset --hard HEAD~1
git push -f origin main

# แต่... Git history ยังมีอยู่!
git reflog
# a1b2c3d HEAD@{1}: commit: Add production config  <- Still there!

# ผลลัพธ์: Secrets leaked forever in Git history! 😱

Security Incidents ที่เกิดจริง:

Database credentials ใน public GitHub repo
API keys ใน Docker images
Production passwords ใน Slack messages
JWT secrets ใน configuration files
Cloud access keys ใน CI/CD logs

จนวันหนึ่งพบ HashiCorp Vault แล้วชีวิตเปลี่ยนไป! 🔐

HashiCorp Vault Fundamentals

1. Vault Core Concepts

Vault เป็น “Bank for Secrets”:

┌─────────────────────────────────────┐
│            HashiCorp Vault          │
│                                     │
│  ┌─────────────┐  ┌─────────────┐  │
│  │   Secrets   │  │  Encryption │  │
│  │   Engine    │  │   Transit   │  │
│  └─────────────┘  └─────────────┘  │
│                                     │
│  ┌─────────────┐  ┌─────────────┐  │
│  │    Auth     │  │   Audit     │  │
│  │  Methods    │  │    Logs     │  │
│  └─────────────┘  └─────────────┘  │
│                                     │
└─────────────────────────────────────┘

Key Features:

Dynamic Secrets: สร้าง credentials ตาม request
Encryption as a Service: Encrypt/decrypt data
Leases & Renewal: Secrets มี lifetime
Revocation: ยกเลิก secrets ได้ทันที
Audit Logging: บันทึกทุก operation
Fine-grained Access Control: ควบคุม access แบบละเอียด

2. Vault Installation และ Setup

# Download และ Install Vault
wget https://releases.hashicorp.com/vault/1.15.0/vault_1.15.0_linux_amd64.zip
unzip vault_1.15.0_linux_amd64.zip
sudo mv vault /usr/local/bin/

# Verify installation
vault version

# vault.hcl - Production Configuration
ui = true

storage "consul" {
  address = "127.0.0.1:8500"
  path    = "vault/"
}

# Alternative: File storage (for dev)
# storage "file" {
#   path = "/opt/vault/data"
# }

# Alternative: Integrated storage (Raft)
storage "raft" {
  path    = "/opt/vault/data"
  node_id = "node1"
  
  retry_join {
    leader_api_addr = "http://vault-0.vault-internal:8200"
  }
  
  retry_join {
    leader_api_addr = "http://vault-1.vault-internal:8200"
  }
}

listener "tcp" {
  address       = "0.0.0.0:8200"
  cluster_address = "0.0.0.0:8201"
  tls_cert_file = "/opt/vault/tls/vault.crt"
  tls_key_file  = "/opt/vault/tls/vault.key"
  
  # For development (disable TLS)
  # tls_disable = true
}

cluster_addr = "https://vault.example.com:8201"
api_addr = "https://vault.example.com:8200"

# Enable Prometheus metrics
telemetry {
  prometheus_retention_time = "30s"
  disable_hostname = true
}

# Seal configuration (AWS KMS)
seal "awskms" {
  region     = "us-west-2"
  kms_key_id = "12345678-1234-1234-1234-123456789012"
}

# Plugin directory
plugin_directory = "/opt/vault/plugins"

# Performance settings
default_lease_ttl = "768h"
max_lease_ttl = "8760h"

# Log level
log_level = "INFO"

# Start Vault server
vault server -config=vault.hcl

# Initialize Vault (first time only)
vault operator init -key-shares=5 -key-threshold=3

# Sample output:
# Unseal Key 1: AbCdEfGhIjKlMnOpQrStUvWxYz1234567890
# Unseal Key 2: BcDeFgHiJkLmNoPqRsTuVwXyZ2345678901
# Unseal Key 3: CdEfGhIjKlMnOpQrStUvWxYzA3456789012
# Unseal Key 4: DeFgHiJkLmNoPqRsTuVwXyZaB4567890123
# Unseal Key 5: EfGhIjKlMnOpQrStUvWxYzAbC5678901234
#
# Initial Root Token: hvs.AbCdEfGhIjKlMnOpQrStUvWxYz123456

# Unseal Vault (need 3 keys)
vault operator unseal AbCdEfGhIjKlMnOpQrStUvWxYz1234567890
vault operator unseal BcDeFgHiJkLmNoPqRsTuVwXyZ2345678901
vault operator unseal CdEfGhIjKlMnOpQrStUvWxYzA3456789012

# Login with root token
vault auth hvs.AbCdEfGhIjKlMnOpQrStUvWxYz123456

# Check status
vault status

3. Kubernetes Integration

# vault-k8s-deployment.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: vault
  namespace: vault
spec:
  serviceName: vault-internal
  replicas: 3
  selector:
    matchLabels:
      app: vault
  template:
    metadata:
      labels:
        app: vault
    spec:
      serviceAccountName: vault
      containers:
      - name: vault
        image: hashicorp/vault:1.15.0
        ports:
        - containerPort: 8200
          name: vault-port
        - containerPort: 8201
          name: cluster-port
        env:
        - name: VAULT_ADDR
          value: "https://localhost:8200"
        - name: VAULT_API_ADDR
          value: "https://vault.example.com:8200"
        - name: VAULT_CLUSTER_ADDR
          value: "https://vault.example.com:8201"
        - name: VAULT_RAFT_NODE_ID
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        volumeMounts:
        - name: vault-config
          mountPath: /vault/config
        - name: vault-data
          mountPath: /vault/data
        - name: vault-tls
          mountPath: /vault/tls
        resources:
          requests:
            memory: "512Mi"
            cpu: "500m"
          limits:
            memory: "1Gi"
            cpu: "1000m"
        securityContext:
          capabilities:
            add: ["IPC_LOCK"]
      volumes:
      - name: vault-config
        configMap:
          name: vault-config
      - name: vault-tls
        secret:
          secretName: vault-tls
  volumeClaimTemplates:
  - metadata:
      name: vault-data
    spec:
      accessModes: ["ReadWriteOnce"]
      resources:
        requests:
          storage: 10Gi

Secrets Engines และ Management

1. Key-Value Secrets Engine (v2)

# Enable KV secrets engine
vault secrets enable -path=myapp kv-v2

# Store secrets
vault kv put myapp/database \
  username=myapp_user \
  password=supersecret123 \
  host=db.example.com \
  port=5432

vault kv put myapp/api \
  stripe_key=sk_live_abcdef123456 \
  sendgrid_key=SG.abcdef123456 \
  jwt_secret=myverysecretjwtkey

# Read secrets
vault kv get myapp/database
vault kv get -field=password myapp/database

# Get specific version
vault kv get -version=1 myapp/database

# List secrets
vault kv list myapp/

# Delete secrets
vault kv delete myapp/api
vault kv destroy -versions=1,2 myapp/database  # Permanent delete

JSON Output:

# Get secrets as JSON
vault kv get -format=json myapp/database

# Output:
{
  "request_id": "abcd1234-...",
  "lease_id": "",
  "renewable": false,
  "lease_duration": 0,
  "data": {
    "data": {
      "host": "db.example.com",
      "password": "supersecret123",
      "port": 5432,
      "username": "myapp_user"
    },
    "metadata": {
      "created_time": "2024-01-15T10:30:00Z",
      "version": 2
    }
  }
}

2. Dynamic Secrets - Database Engine

# Enable database secrets engine
vault secrets enable database

# Configure MySQL connection
vault write database/config/mysql-prod \
  plugin_name=mysql-database-plugin \
  connection_url="{{username}}:{{password}}@tcp(mysql.prod:3306)/" \
  allowed_roles="myapp-role" \
  username="vault-admin" \
  password="vault-admin-password"

# Create role for dynamic credentials
vault write database/roles/myapp-role \
  db_name=mysql-prod \
  creation_statements="CREATE USER '{{name}}'@'%' IDENTIFIED BY '{{password}}';GRANT SELECT,INSERT,UPDATE,DELETE ON myapp.* TO '{{name}}'@'%';" \
  default_ttl="1h" \
  max_ttl="24h"

# Generate dynamic credentials
vault read database/creds/myapp-role

# Output:
# Key                Value
# ---                -----
# lease_id           database/creds/myapp-role/abcd1234-...
# lease_duration     1h
# lease_renewable    true
# password           A1Bb2Cc3-abcdef123456
# username           v-token-myapp-ro-abcdefgh-1642234567

3. PostgreSQL Dynamic Secrets

# Configure PostgreSQL
vault write database/config/postgres-prod \
  plugin_name=postgresql-database-plugin \
  connection_url="postgresql://{{username}}:{{password}}@postgres.prod:5432/myapp?sslmode=require" \
  allowed_roles="readonly,readwrite" \
  username="vault" \
  password="vault-password"

# Read-only role
vault write database/roles/readonly \
  db_name=postgres-prod \
  creation_statements="CREATE ROLE \"{{name}}\" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}'; \
                      GRANT SELECT ON ALL TABLES IN SCHEMA public TO \"{{name}}\";" \
  default_ttl="1h" \
  max_ttl="24h"

# Read-write role  
vault write database/roles/readwrite \
  db_name=postgres-prod \
  creation_statements="CREATE ROLE \"{{name}}\" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}'; \
                      GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public TO \"{{name}}\";" \
  default_ttl="1h" \
  max_ttl="24h"

# Generate credentials
vault read database/creds/readonly
vault read database/creds/readwrite

# Revoke credentials
vault lease revoke database/creds/readonly/abcd1234-...

# Revoke all credentials for role
vault lease revoke -prefix database/creds/readonly

4. AWS Secrets Engine

# Enable AWS secrets engine
vault secrets enable aws

# Configure AWS credentials
vault write aws/config/root \
  access_key=AKIA123456789EXAMPLE \
  secret_key=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY \
  region=us-west-2

# Create IAM role
vault write aws/roles/s3-readonly \
  credential_type=iam_user \
  policy_document=-<<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::my-bucket/*"
    }
  ]
}
EOF

# Generate AWS credentials
vault read aws/creds/s3-readonly

# Output:
# Key                Value
# ---                -----
# lease_id           aws/creds/s3-readonly/abcd1234-...
# lease_duration     768h
# lease_renewable    true
# access_key         AKIA789012345EXAMPLE
# secret_key         wxyzABC123DEF456GHI789JKL012MNO345PQR678
# security_token     <nil>

Authentication และ Authorization

1. Kubernetes Auth Method

# Enable Kubernetes auth
vault auth enable kubernetes

# Configure Kubernetes auth
vault write auth/kubernetes/config \
  token_reviewer_jwt="$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" \
  kubernetes_host="https://kubernetes.default.svc:443" \
  kubernetes_ca_cert=@/var/run/secrets/kubernetes.io/serviceaccount/ca.crt

# Create role for service account
vault write auth/kubernetes/role/myapp \
  bound_service_account_names=myapp \
  bound_service_account_namespaces=production \
  policies=myapp-policy \
  ttl=24h

2. Vault Policies

# policies/myapp-policy.hcl
# Read secrets
path "myapp/data/database" {
  capabilities = ["read"]
}

path "myapp/data/api" {
  capabilities = ["read"]
}

# Generate database credentials
path "database/creds/readonly" {
  capabilities = ["read"]
}

# Renew leases
path "sys/leases/renew" {
  capabilities = ["create", "update"]
}

# Revoke own leases
path "sys/leases/revoke" {
  capabilities = ["update"]
}

# Write policy
vault policy write myapp-policy policies/myapp-policy.hcl

# List policies
vault policy list

# Read policy
vault policy read myapp-policy

3. AppRole Auth Method

# Enable AppRole auth
vault auth enable approle

# Create AppRole
vault write auth/approle/role/myapp \
  token_policies="myapp-policy" \
  token_ttl=1h \
  token_max_ttl=4h \
  bind_secret_id=true

# Get RoleID
vault read auth/approle/role/myapp/role-id

# Generate SecretID
vault write -f auth/approle/role/myapp/secret-id

# Login with AppRole
vault write auth/approle/login \
  role_id="abcd1234-5678-90ab-cdef-1234567890ab" \
  secret_id="efgh5678-90ab-cdef-1234-567890abcdef"

4. JWT/OIDC Auth

# Enable JWT auth
vault auth enable jwt

# Configure JWT auth (with OIDC Discovery)
vault write auth/jwt/config \
  bound_issuer="https://login.microsoftonline.com/tenant-id/v2.0" \
  oidc_discovery_url="https://login.microsoftonline.com/tenant-id/v2.0"

# Create role for specific groups
vault write auth/jwt/role/developers \
  bound_audiences="vault" \
  bound_claims_type="glob" \
  bound_claims='{"groups":["developers","*-developers"]}' \
  user_claim="email" \
  policies="developer-policy" \
  ttl="1h"

# Login with JWT token
vault write auth/jwt/login role=developers jwt=eyJ0eXAiOiJKV1QiLCJhbGc...

Application Integration

1. Vault Agent สำหรับ Applications

# vault-agent.hcl
pid_file = "/tmp/vault-agent.pid"

vault {
  address = "https://vault.example.com:8200"
}

auto_auth {
  method "kubernetes" {
    mount_path = "auth/kubernetes"
    config = {
      role = "myapp"
    }
  }

  sink "file" {
    config = {
      path = "/tmp/vault-token"
    }
  }
}

template {
  source      = "/tmp/database.tpl"
  destination = "/tmp/database.conf"
  command     = "systemctl reload myapp"
}

template {
  source      = "/tmp/api-keys.tpl"
  destination = "/tmp/api-keys.conf"
  perms       = 0600
}

cache {
  use_auto_auth_token = true
}

listener "tcp" {
  address = "127.0.0.1:8007"
  tls_disable = true
}

# Template files
# /tmp/database.tpl
{{- with secret "myapp/data/database" -}}
[database]
host = {{ .Data.data.host }}
port = {{ .Data.data.port }}
username = {{ .Data.data.username }}
password = {{ .Data.data.password }}
{{- end }}

# /tmp/api-keys.tpl
{{- with secret "myapp/data/api" -}}
export STRIPE_KEY="{{ .Data.data.stripe_key }}"
export SENDGRID_KEY="{{ .Data.data.sendgrid_key }}"
export JWT_SECRET="{{ .Data.data.jwt_secret }}"
{{- end }}

2. Kubernetes Vault Integration

# vault-serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: myapp
  namespace: production
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: myapp-tokenreview-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:auth-delegator
subjects:
- kind: ServiceAccount
  name: myapp
  namespace: production

# app-with-vault-sidecar.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp
  namespace: production
spec:
  replicas: 3
  selector:
    matchLabels:
      app: myapp
  template:
    metadata:
      labels:
        app: myapp
    spec:
      serviceAccountName: myapp
      
      containers:
      # Main application
      - name: app
        image: myapp:v1.2.3
        volumeMounts:
        - name: vault-secrets
          mountPath: /etc/secrets
          readOnly: true
        env:
        - name: DATABASE_CONFIG
          value: "/etc/secrets/database.conf"
        - name: API_KEYS_CONFIG
          value: "/etc/secrets/api-keys.conf"
          
      # Vault Agent sidecar
      - name: vault-agent
        image: hashicorp/vault:1.15.0
        command: ["vault", "agent", "-config=/vault/config/agent.hcl"]
        volumeMounts:
        - name: vault-config
          mountPath: /vault/config
        - name: vault-secrets
          mountPath: /vault/secrets
        env:
        - name: VAULT_ADDR
          value: "https://vault.example.com:8200"
          
      volumes:
      - name: vault-config
        configMap:
          name: vault-agent-config
      - name: vault-secrets
        emptyDir:
          medium: Memory  # Store secrets in memory

3. Vault CSI Driver

# vault-csi-secretproviderclass.yaml
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: myapp-secrets
  namespace: production
spec:
  provider: vault
  parameters:
    vaultAddress: "https://vault.example.com:8200"
    roleName: "myapp"
    objects: |
      - objectName: "database-password"
        secretPath: "myapp/data/database"
        secretKey: "password"
      - objectName: "api-key"
        secretPath: "myapp/data/api"
        secretKey: "stripe_key"
  secretObjects:
  - secretName: myapp-secret
    type: Opaque
    data:
    - objectName: database-password
      key: password
    - objectName: api-key
      key: stripe_key

# app-with-csi-driver.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp
spec:
  template:
    spec:
      serviceAccountName: myapp
      containers:
      - name: app
        image: myapp:v1.2.3
        env:
        - name: DATABASE_PASSWORD
          valueFrom:
            secretKeyRef:
              name: myapp-secret
              key: password
        - name: STRIPE_KEY
          valueFrom:
            secretKeyRef:
              name: myapp-secret
              key: stripe_key
        volumeMounts:
        - name: secrets-store
          mountPath: "/mnt/secrets"
          readOnly: true
      volumes:
      - name: secrets-store
        csi:
          driver: secrets-store.csi.k8s.io
          readOnly: true
          volumeAttributes:
            secretProviderClass: "myapp-secrets"

Transit Encryption Engine

1. Encryption as a Service

# Enable transit engine
vault secrets enable transit

# Create encryption key
vault write -f transit/keys/myapp-data

# Encrypt data
vault write transit/encrypt/myapp-data \
  plaintext=$(echo "sensitive data" | base64)

# Output:
# Key           Value
# ---           -----
# ciphertext    vault:v1:abcd1234efgh5678ijkl90mnop...

# Decrypt data
vault write transit/decrypt/myapp-data \
  ciphertext="vault:v1:abcd1234efgh5678ijkl90mnop..."

# Output (base64 encoded):
# Key          Value
# ---          -----
# plaintext    c2Vuc2l0aXZlIGRhdGE=

# Decode
echo "c2Vuc2l0aXZlIGRhdGE=" | base64 -d
# Output: sensitive data

2. Key Rotation

# Rotate encryption key
vault write -f transit/keys/myapp-data/rotate

# List key versions
vault read transit/keys/myapp-data

# Encrypt with latest version
vault write transit/encrypt/myapp-data \
  plaintext=$(echo "new sensitive data" | base64)

# Re-encrypt old ciphertext with new key
vault write transit/rewrap/myapp-data \
  ciphertext="vault:v1:abcd1234efgh5678ijkl90mnop..."

# Update minimum decryption version
vault write transit/keys/myapp-data/config \
  min_decryption_version=2

# Delete old key version
vault write transit/keys/myapp-data/config \
  deletion_allowed=true

vault delete transit/keys/myapp-data

3. Application Integration

# Python example with hvac
import hvac
import base64

# Initialize Vault client
client = hvac.Client(url='https://vault.example.com:8200')

# Authenticate using AppRole
client.auth.approle.login(
    role_id='abcd1234-5678-90ab-cdef-1234567890ab',
    secret_id='efgh5678-90ab-cdef-1234-567890abcdef'
)

def encrypt_data(data):
    """Encrypt sensitive data using Vault Transit"""
    encoded_data = base64.b64encode(data.encode()).decode()
    
    response = client.secrets.transit.encrypt_data(
        name='myapp-data',
        plaintext=encoded_data
    )
    
    return response['data']['ciphertext']

def decrypt_data(ciphertext):
    """Decrypt data using Vault Transit"""
    response = client.secrets.transit.decrypt_data(
        name='myapp-data',
        ciphertext=ciphertext
    )
    
    decoded_data = base64.b64decode(response['data']['plaintext'])
    return decoded_data.decode()

# Usage
sensitive_info = "credit card: 1234-5678-9012-3456"
encrypted = encrypt_data(sensitive_info)
print(f"Encrypted: {encrypted}")

decrypted = decrypt_data(encrypted)
print(f"Decrypted: {decrypted}")

// Go example
package main

import (
    "context"
    "encoding/base64"
    "fmt"
    "log"
    
    "github.com/hashicorp/vault/api"
)

func main() {
    // Create Vault client
    client, err := api.NewClient(api.DefaultConfig())
    if err != nil {
        log.Fatal(err)
    }
    
    // Authenticate with AppRole
    data := map[string]interface{}{
        "role_id":   "abcd1234-5678-90ab-cdef-1234567890ab",
        "secret_id": "efgh5678-90ab-cdef-1234-567890abcdef",
    }
    
    resp, err := client.Logical().Write("auth/approle/login", data)
    if err != nil {
        log.Fatal(err)
    }
    
    client.SetToken(resp.Auth.ClientToken)
    
    // Encrypt data
    plaintext := "sensitive database connection string"
    encoded := base64.StdEncoding.EncodeToString([]byte(plaintext))
    
    encryptData := map[string]interface{}{
        "plaintext": encoded,
    }
    
    encryptResp, err := client.Logical().Write("transit/encrypt/myapp-data", encryptData)
    if err != nil {
        log.Fatal(err)
    }
    
    ciphertext := encryptResp.Data["ciphertext"].(string)
    fmt.Printf("Encrypted: %s\n", ciphertext)
    
    // Decrypt data
    decryptData := map[string]interface{}{
        "ciphertext": ciphertext,
    }
    
    decryptResp, err := client.Logical().Write("transit/decrypt/myapp-data", decryptData)
    if err != nil {
        log.Fatal(err)
    }
    
    decryptedEncoded := decryptResp.Data["plaintext"].(string)
    decryptedBytes, err := base64.StdEncoding.DecodeString(decryptedEncoded)
    if err != nil {
        log.Fatal(err)
    }
    
    fmt.Printf("Decrypted: %s\n", string(decryptedBytes))
}

PKI Secrets Engine

1. Certificate Authority Setup

# Enable PKI engine
vault secrets enable pki

# Set TTL for root CA
vault secrets tune -max-lease-ttl=87600h pki

# Generate root CA
vault write pki/root/generate/internal \
  common_name="My Company Root CA" \
  ttl=87600h

# Configure CA URLs
vault write pki/config/urls \
  issuing_certificates="https://vault.example.com:8200/v1/pki/ca" \
  crl_distribution_points="https://vault.example.com:8200/v1/pki/crl"

# Create intermediate CA
vault secrets enable -path=pki_int pki
vault secrets tune -max-lease-ttl=43800h pki_int

# Generate intermediate CSR
vault write -format=json pki_int/intermediate/generate/internal \
  common_name="My Company Intermediate CA" \
  | jq -r '.data.csr' > pki_intermediate.csr

# Sign intermediate CSR with root CA
vault write -format=json pki/root/sign-intermediate \
  csr=@pki_intermediate.csr \
  format=pem_bundle ttl="43800h" \
  | jq -r '.data.certificate' > intermediate.cert.pem

# Set signed certificate
vault write pki_int/intermediate/set-signed \
  certificate=@intermediate.cert.pem

2. Certificate Roles และ Issuance

# Create role for server certificates
vault write pki_int/roles/server-cert \
  allowed_domains="example.com,api.example.com" \
  allow_subdomains=true \
  max_ttl="720h" \
  key_bits=2048 \
  key_type=rsa \
  allow_any_name=false \
  enforce_hostnames=true \
  allow_localhost=false \
  server_flag=true \
  client_flag=false

# Create role for client certificates
vault write pki_int/roles/client-cert \
  allowed_domains="example.com" \
  allow_subdomains=true \
  max_ttl="168h" \
  key_bits=2048 \
  key_type=rsa \
  server_flag=false \
  client_flag=true

# Issue server certificate
vault write pki_int/issue/server-cert \
  common_name="api.example.com" \
  alt_names="api.example.com,backend.example.com" \
  ttl="720h"

# Issue client certificate
vault write pki_int/issue/client-cert \
  common_name="client.example.com" \
  ttl="168h"

3. Automatic Certificate Management

# cert-manager integration
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: vault-issuer
  namespace: production
spec:
  vault:
    server: https://vault.example.com:8200
    path: pki_int/sign/server-cert
    auth:
      kubernetes:
        mountPath: /v1/auth/kubernetes
        role: cert-manager
        secretRef:
          name: cert-manager-vault-token
          key: token
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: api-tls
  namespace: production
spec:
  secretName: api-tls-secret
  issuerRef:
    name: vault-issuer
    kind: Issuer
  commonName: api.example.com
  dnsNames:
  - api.example.com
  - backend.example.com
  duration: 720h
  renewBefore: 240h

Audit Logging และ Monitoring

1. Audit Device Configuration

# Enable file audit device
vault audit enable file file_path=/vault/logs/audit.log

# Enable syslog audit device
vault audit enable syslog tag="vault" facility="AUTH"

# Enable socket audit device
vault audit enable socket address="audit.example.com:9090" socket_type="tcp"

# List audit devices
vault audit list

2. Audit Log Analysis

{
  "time": "2024-01-15T10:30:45.123456Z",
  "type": "request",
  "auth": {
    "client_token": "hmac-sha256:abcd1234...",
    "accessor": "hmac-sha256:efgh5678...",
    "display_name": "kubernetes-myapp-production",
    "policies": ["default", "myapp-policy"],
    "token_policies": ["myapp-policy"],
    "metadata": {
      "role": "myapp",
      "service_account_name": "myapp",
      "service_account_namespace": "production"
    }
  },
  "request": {
    "id": "abcd1234-5678-90ab-cdef-1234567890ab",
    "operation": "read",
    "mount_type": "kv",
    "client_token": "hmac-sha256:abcd1234...",
    "client_token_accessor": "hmac-sha256:efgh5678...",
    "namespace": {
      "id": "root"
    },
    "path": "myapp/data/database",
    "data": null,
    "remote_address": "10.0.1.100",
    "wrap_ttl": 0
  },
  "response": {
    "mount_type": "kv",
    "data": {
      "data": "hmac-sha256:ijkl90mn...",
      "metadata": "hmac-sha256:opqr3456..."
    }
  }
}

3. Prometheus Metrics

# vault-servicemonitor.yaml
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: vault-metrics
  namespace: vault
spec:
  selector:
    matchLabels:
      app: vault
  endpoints:
  - port: vault-port
    path: /v1/sys/metrics
    params:
      format: ['prometheus']
    bearerTokenSecret:
      name: vault-metrics-token
      key: token

# vault-alerts.yaml
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: vault-alerts
  namespace: vault
spec:
  groups:
  - name: vault
    rules:
    - alert: VaultSealed
      expr: vault_core_unsealed == 0
      for: 1m
      labels:
        severity: critical
      annotations:
        summary: "Vault is sealed"
        description: "Vault instance {{ $labels.instance }} is sealed"
        
    - alert: VaultHighErrorRate
      expr: rate(vault_audit_log_request_failure[5m]) > 0.1
      for: 2m
      labels:
        severity: warning
      annotations:
        summary: "High Vault error rate"
        description: "Vault error rate is {{ $value }} errors/sec"
        
    - alert: VaultLeaseExpirationHigh
      expr: vault_expire_num_leases > 1000
      for: 5m
      labels:
        severity: warning
      annotations:
        summary: "High number of Vault leases"
        description: "Number of Vault leases is {{ $value }}"
        
    - alert: VaultTokenExpirationSoon
      expr: vault_token_count_by_ttl{creation_ttl="30m"} > 100
      for: 1m
      labels:
        severity: warning
      annotations:
        summary: "Many tokens expiring soon"
        description: "{{ $value }} tokens will expire within 30 minutes"

Security Best Practices

1. Vault Hardening

# vault-production.hcl
# Disable UI in production
ui = false

# Use TLS everywhere
listener "tcp" {
  address       = "0.0.0.0:8200"
  tls_cert_file = "/vault/tls/vault.crt"
  tls_key_file  = "/vault/tls/vault.key"
  tls_min_version = "tls12"
  tls_cipher_suites = "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384"
}

# Enable auto-unseal with cloud KMS
seal "awskms" {
  region     = "us-west-2"
  kms_key_id = "alias/vault-unseal-key"
}

# Secure storage backend
storage "consul" {
  address = "consul.vault.svc.cluster.local:8500"
  path    = "vault/"
  tls_ca_file   = "/vault/tls/consul-ca.crt"
  tls_cert_file = "/vault/tls/consul.crt"
  tls_key_file  = "/vault/tls/consul.key"
}

# Telemetry with authentication
telemetry {
  prometheus_retention_time = "30s"
  disable_hostname = true
  unauthenticated_metrics_access = false
}

# Logging
log_level = "WARN"
log_format = "json"

# Performance
default_lease_ttl = "1h"
max_lease_ttl = "24h"

2. Network Security

# vault-network-policy.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: vault-netpol
  namespace: vault
spec:
  podSelector:
    matchLabels:
      app: vault
  policyTypes:
  - Ingress
  - Egress
  ingress:
  - from:
    # Only allow from specific namespaces
    - namespaceSelector:
        matchLabels:
          name: production
    - namespaceSelector:
        matchLabels:
          name: monitoring
    ports:
    - protocol: TCP
      port: 8200
  egress:
  # Allow to Consul backend
  - to: []
    ports:
    - protocol: TCP
      port: 8500
  # Allow to KMS
  - to: []
    ports:
    - protocol: TCP
      port: 443

3. RBAC และ Least Privilege

# policies/app-readonly.hcl
# Read application secrets only
path "apps/data/myapp/+/config" {
  capabilities = ["read"]
}

# Cannot read secrets from other apps
path "apps/data/otherapp/*" {
  capabilities = ["deny"]
}

# Generate short-lived DB credentials
path "database/creds/readonly" {
  capabilities = ["read"]
}

# Renew own tokens only
path "auth/token/renew-self" {
  capabilities = ["update"]
}

# policies/devops-admin.hcl
# Full access to specific app paths
path "apps/data/myapp/*" {
  capabilities = ["create", "read", "update", "delete", "list"]
}

# Manage database roles
path "database/roles/myapp-*" {
  capabilities = ["create", "read", "update", "delete", "list"]
}

# Cannot access production secrets directly
path "apps/data/*/production/*" {
  capabilities = ["deny"]
}

# Can rotate keys
path "transit/keys/myapp-*/rotate" {
  capabilities = ["update"]
}

# View audit logs
path "sys/audit-hash/*" {
  capabilities = ["update"]
}

Disaster Recovery และ High Availability

1. Vault Cluster Setup

# vault-ha.hcl - Node 1
storage "raft" {
  path    = "/vault/data"
  node_id = "vault-1"
  
  retry_join {
    leader_api_addr = "https://vault-2.vault.svc.cluster.local:8200"
  }
  
  retry_join {
    leader_api_addr = "https://vault-3.vault.svc.cluster.local:8200"
  }
}

listener "tcp" {
  address       = "0.0.0.0:8200"
  cluster_address = "0.0.0.0:8201"
  tls_cert_file = "/vault/tls/vault.crt"
  tls_key_file  = "/vault/tls/vault.key"
}

cluster_addr = "https://vault-1.vault.svc.cluster.local:8201"
api_addr = "https://vault-1.vault.svc.cluster.local:8200"

seal "awskms" {
  region     = "us-west-2"
  kms_key_id = "alias/vault-unseal-key"
}

2. Backup Strategy

#!/bin/bash
# vault-backup.sh

VAULT_ADDR="https://vault.example.com:8200"
BACKUP_PATH="/backups/vault/$(date +%Y%m%d_%H%M%S)"
S3_BUCKET="mycompany-vault-backups"

# Create backup directory
mkdir -p "$BACKUP_PATH"

# Backup Raft data (if using Raft storage)
vault operator raft snapshot save "$BACKUP_PATH/vault-snapshot.snap"

# Backup policies
vault policy list | while read policy; do
  if [ "$policy" != "default" ] && [ "$policy" != "root" ]; then
    vault policy read "$policy" > "$BACKUP_PATH/policy-${policy}.hcl"
  fi
done

# Backup auth methods configuration
vault auth list -format=json > "$BACKUP_PATH/auth-methods.json"

# Backup secrets engines
vault secrets list -format=json > "$BACKUP_PATH/secrets-engines.json"

# Compress backup
cd "$(dirname "$BACKUP_PATH")"
tar -czf "vault-backup-$(date +%Y%m%d_%H%M%S).tar.gz" "$(basename "$BACKUP_PATH")"

# Upload to S3
aws s3 cp "vault-backup-$(date +%Y%m%d_%H%M%S).tar.gz" "s3://$S3_BUCKET/"

# Cleanup local files older than 7 days
find /backups/vault -name "*.tar.gz" -mtime +7 -delete

echo "Vault backup completed: $(date)"

3. Disaster Recovery Procedures

# disaster-recovery.sh
#!/bin/bash

BACKUP_FILE="vault-backup-20240115_103000.tar.gz"
S3_BUCKET="mycompany-vault-backups"

# Download backup from S3
aws s3 cp "s3://$S3_BUCKET/$BACKUP_FILE" .

# Extract backup
tar -xzf "$BACKUP_FILE"
BACKUP_DIR=$(tar -tzf "$BACKUP_FILE" | head -1 | cut -f1 -d"/")

# Initialize new Vault cluster
vault operator init -key-shares=5 -key-threshold=3 > init-keys.txt

# Unseal Vault
UNSEAL_KEY_1=$(grep 'Unseal Key 1:' init-keys.txt | awk '{print $4}')
UNSEAL_KEY_2=$(grep 'Unseal Key 2:' init-keys.txt | awk '{print $4}')
UNSEAL_KEY_3=$(grep 'Unseal Key 3:' init-keys.txt | awk '{print $4}')

vault operator unseal "$UNSEAL_KEY_1"
vault operator unseal "$UNSEAL_KEY_2"
vault operator unseal "$UNSEAL_KEY_3"

# Login with root token
ROOT_TOKEN=$(grep 'Initial Root Token:' init-keys.txt | awk '{print $4}')
vault auth "$ROOT_TOKEN"

# Restore Raft snapshot
vault operator raft snapshot restore "$BACKUP_DIR/vault-snapshot.snap"

# Restore policies
for policy_file in "$BACKUP_DIR"/policy-*.hcl; do
  policy_name=$(basename "$policy_file" .hcl | sed 's/policy-//')
  vault policy write "$policy_name" "$policy_file"
done

echo "Disaster recovery completed: $(date)"

เคสจริง: จาก .env Hell สู่ Vault Heaven

ก่อนใช้ HashiCorp Vault

ปัญหาที่เจอจริง:

# สถานการณ์วุ่นวายในการจัดการ secrets

# 1. Secrets scattered everywhere
# .env files
DATABASE_PASSWORD=supersecret123
API_KEY=sk-live-abcdefghijklmnop1234567890

# Kubernetes secrets (base64)  
kubectl create secret generic app-secrets \
  --from-literal=db-password=supersecret123 \
  --from-literal=api-key=sk-live-abcdef123456

# Configuration files
echo "password: supersecret123" >> config/production.yaml
git add config/production.yaml  # 😱 Secrets in Git!

# 2. Security incidents
# Developer accidentally commits .env to GitHub
git add .
git commit -m "Fix configuration"
git push origin main

# Later discovers the mistake
git log --oneline
# a1b2c3d Fix configuration  <- Contains production secrets!

# 3. Sharing secrets insecurely
# Slack message: "Hey, the DB password is supersecret123"
# Email: "API key is sk-live-abcdef123456"  
# WhatsApp: "JWT secret is myverysecrettoken"

# 4. No rotation strategy
# Same passwords for years
# No way to know who accessed what
# Manual updates across all environments

ผลกระทบที่เกิด:

Security Breaches: Credentials leaked ใน Git history
Compliance Issues: ไม่ผ่าน security audit
Operational Overhead: Manual secret management
No Auditability: ไม่รู้ใครเข้าถึง secrets เมื่อไหร่
Environment Inconsistency: Secrets ไม่ sync กันระหว่าง environments

หลังใช้ HashiCorp Vault

การจัดการ Secrets ใหม่:

# 1. Centralized secret storage
vault kv put myapp/database \
  username=myapp_user \
  password=vault-generated-password \
  host=db.production.com

# 2. Dynamic secrets
vault read database/creds/myapp-readonly
# Gets fresh credentials every time!

# 3. Application integration
# Applications fetch secrets directly from Vault
# No more .env files or hardcoded credentials

# 4. Automatic rotation
vault write -f transit/keys/myapp-data/rotate
# All data automatically re-encrypted with new key

สิ่งที่เปลี่ยนไป:

Aspect	Before Vault	After Vault
Storage	Files, env vars	Encrypted, centralized
Access Control	File permissions	Fine-grained policies
Audit Trail	None	Complete logging
Rotation	Manual	Automatic
Sharing	Insecure channels	Secure API access
Compliance	Fail audits	Pass security reviews

ผลลัพธ์ที่วัดได้:

Metric	Before	After	Improvement
Security Incidents	3/quarter	0/quarter	100% reduction
Secret Rotation Frequency	Never	Weekly	∞% improvement
Audit Compliance	40%	100%	150% improvement
Access Visibility	0%	100%	Complete
Time to Rotate Secrets	2 days	5 minutes	99.8% faster

Real-world Vault Workflow

Development to Production:

# 1. Developer needs database access
# Before: Ask DevOps for password
# After: Vault generates temporary credentials

vault read database/creds/myapp-readonly
# Key                Value
# ---                -----
# lease_id           database/creds/myapp-readonly/abcd1234...
# lease_duration     1h
# password           A1Bb2Cc3-temp-password
# username           v-kubernetes-myapp-readonly-abcdefgh

# 2. Application startup
# Before: Read from .env file
# After: Authenticate with Vault and fetch secrets

# 3. Secret rotation
# Before: Manual coordination across teams
# After: Automatic with zero downtime

vault write -f database/rotate-root/myapp-prod
# Database password rotated automatically
# All applications get new credentials seamlessly

Security Incident Response:

# Before: Panic mode
# "Someone might have seen our Slack message with the password!"
# 1. Change all passwords manually
# 2. Update all applications
# 3. Hope nothing breaks

# After: Controlled response
# Revoke specific access immediately
vault lease revoke database/creds/myapp-readonly/suspicious-lease-id

# Rotate all credentials
vault write -f database/rotate-root/myapp-prod

# Audit who accessed what
vault audit-log | grep "suspicious-user"

Compliance Audit:

# Before: "We store passwords in environment variables"
# Auditor: 😱 "FAIL"

# After: "We use HashiCorp Vault with full audit logging"
# Show complete audit trail:
vault audit-log | head -10
# Every access logged with timestamp, user, action

# Demonstrate access controls:
vault policy read developer-policy
# Fine-grained permissions

# Show encryption in transit and at rest:
vault status
# All data encrypted with strong algorithms

# Auditor: 😊 "PASS"

สรุป: Vault ที่เปลี่ยนวิธีคิดเรื่อง Security

ก่อนรู้จัก Vault:

Secrets = ไฟล์ที่กลัวว่าจะ commit เข้า Git 😰
Password sharing = Slack messages และ Post-it notes
Rotation = “ปีหน้าค่อยเปลี่ยน”
Audit = “ไม่รู้ใครเข้าถึงอะไรเมื่อไหร่”
Compliance = กลัวเจอ security audit

หลังใช้ HashiCorp Vault:

Centralized Secret Management 🏦 - Vault เป็น “ธนาคาร” สำหรับ secrets
Zero Trust Security - Verify ทุกการเข้าถึง
Dynamic Secrets - Generate credentials ตามต้องการ
Automatic Rotation - Secrets เปลี่ยนแปลงอัตโนมัติ
Complete Audit Trail - รู้ทุกการเข้าถึง

ข้อดีที่ได้จริง:

Security เพิ่ม 100x: ไม่มี secrets leak
Compliance: ผ่าน security audit ทุกครั้ง
Operational Efficiency: ไม่ต้องจัดการ secrets manually
Developer Experience: Focus on code, not credential management
Zero Trust: Verify ทุกการเข้าถึง

Vault Principles ที่ทำให้สำเร็จ:

Secure by Default: Encrypt everything
Principle of Least Privilege: ให้ access เท่าที่จำเป็น
Audit Everything: Log ทุกการเข้าถึง
Dynamic Credentials: Generate เมื่อต้องการ
Automatic Rotation: เปลี่ยน secrets อัตโนมัติ

Best Practices ที่เรียนรู้:

Auth Methods: ใช้ Kubernetes/JWT auth แทน static tokens
Policies: Fine-grained access control
Dynamic Secrets: ใช้แทน static credentials
Transit Engine: Encrypt sensitive data
Audit Logging: Monitor ทุก operation
High Availability: Cluster setup สำหรับ production

Anti-patterns ที่หลีกเลี่ยง:

เก็บ Vault tokens ใน environment variables
ใช้ root token สำหรับ applications
ไม่มี secret rotation strategy
ไม่ monitor Vault audit logs
Skip การทำ backup

HashiCorp Vault เหมือน Fort Knox สำหรับ Secrets

มันทำให้ secrets management จาก “เรื่องน่ากลัว” เป็น “ระบบที่เชื่อถือได้”

ตอนนี้ไม่สามารถคิดถึงการทำ production โดยไม่มี proper secrets management ได้เลย!

เพราะมันทำให้ security เป็นระบบจริงๆ แทนที่จะเป็นเรื่องที่ “หวังว่าจะไม่เกิดปัญหา”! 🔐⚡