目的_
このレッスンの後、次のことを行います:
- HA クラスター用に PostgreSQL 構成を最適化
- PgBouncer を使用して接続プーリングをセットアップ
- ロード バランシングを実装するHAProxy
- 複数のレプリカによる読み取りのスケール_
- クエリとインデックスの調整_
- パフォーマンスの問題の監視とトラブルシューティング
1。 PostgreSQL 構成のチューニング
1.1。メモリ設定_
shared_buffers
-- Recommended: 25% of total RAM -- Example for 16GB RAM server: ALTER SYSTEM SET shared_buffers = '4GB';
-- Check current: SHOW shared_buffers;
Effective_cache_size
_CODEBLOCK 1work_mem
-- Per-operation memory (sorting, hashing) -- Careful: per query per operation! -- Example: 10 concurrent queries × 5 operations = 50 × work_mem ALTER SYSTEM SET work_mem = '64MB';
-- For specific query: SET work_mem = '256MB'; SELECT ...;
maintenance_work_mem
-- For VACUUM, CREATE INDEX, ALTER TABLE
ALTER SYSTEM SET maintenance_work_mem = '1GB';
1.2。チェックポイントの調整_
-- How often to checkpoint (time-based) ALTER SYSTEM SET checkpoint_timeout = '15min'; -- Default: 5min-- Maximum size of WAL between checkpoints ALTER SYSTEM SET max_wal_size = '4GB'; -- Default: 1GB ALTER SYSTEM SET min_wal_size = '1GB';
-- Spread checkpoint I/O over time (0.5 = 50% of checkpoint_timeout) ALTER SYSTEM SET checkpoint_completion_target = 0.9;
-- Warn if checkpoints happen too frequently ALTER SYSTEM SET checkpoint_warning = '5min';
1.3。 WAL 設定_
-- WAL buffers (auto-tuned to 1/32 of shared_buffers, max 16MB) ALTER SYSTEM SET wal_buffers = '16MB';-- WAL writer delay ALTER SYSTEM SET wal_writer_delay = '200ms'; -- Default: 200ms
-- Commit delay (group commit optimization) ALTER SYSTEM SET commit_delay = 0; -- Microseconds, 0 = disabled ALTER SYSTEM SET commit_siblings = 5; -- Minimum concurrent transactions
1.4。クエリ プランナー
-- Random page cost (lower for SSD) ALTER SYSTEM SET random_page_cost = 1.1; -- Default: 4.0 (HDD)-- Enable parallel query ALTER SYSTEM SET max_parallel_workers_per_gather = 4; ALTER SYSTEM SET max_parallel_workers = 8; ALTER SYSTEM SET parallel_tuple_cost = 0.1; ALTER SYSTEM SET parallel_setup_cost = 1000;
-- Join optimization ALTER SYSTEM SET enable_hashjoin = on; ALTER SYSTEM SET enable_mergejoin = on; ALTER SYSTEM SET enable_nestloop = on;
1.5。接続設定
-- Maximum connections (balance with work_mem) ALTER SYSTEM SET max_connections = 200;-- Superuser reserved connections ALTER SYSTEM SET superuser_reserved_connections = 5;
-- Statement timeout (prevent runaway queries) ALTER SYSTEM SET statement_timeout = '30min';
-- Lock timeout ALTER SYSTEM SET lock_timeout = '10s';
-- Idle in transaction timeout ALTER SYSTEM SET idle_in_transaction_session_timeout = '5min';
1.6。自動バキューム調整
-- Enable autovacuum ALTER SYSTEM SET autovacuum = on;-- Number of autovacuum workers ALTER SYSTEM SET autovacuum_max_workers = 4;
-- Delay between runs ALTER SYSTEM SET autovacuum_naptime = '1min';
-- Vacuum threshold ALTER SYSTEM SET autovacuum_vacuum_threshold = 50; ALTER SYSTEM SET autovacuum_vacuum_scale_factor = 0.1; -- 10% of table
-- Analyze threshold ALTER SYSTEM SET autovacuum_analyze_threshold = 50; ALTER SYSTEM SET autovacuum_analyze_scale_factor = 0.05; -- 5% of table
-- Vacuum cost delay (throttling) ALTER SYSTEM SET autovacuum_vacuum_cost_delay = '2ms'; ALTER SYSTEM SET autovacuum_vacuum_cost_limit = 400;
1.7。パフォーマンスのログ
-- Log slow queries ALTER SYSTEM SET log_min_duration_statement = '1000'; -- 1 second-- Log checkpoints (monitoring) ALTER SYSTEM SET log_checkpoints = on;
-- Log connections/disconnections ALTER SYSTEM SET log_connections = off; ALTER SYSTEM SET log_disconnections = off;
-- Log lock waits ALTER SYSTEM SET log_lock_waits = on; ALTER SYSTEM SET deadlock_timeout = '1s';
-- Log temp files ALTER SYSTEM SET log_temp_files = 10485760; -- 10MB
1.8。構成
-- Reload configuration (no restart needed for most) SELECT pg_reload_conf();-- Check what requires restart: SELECT name, setting, pending_restart FROM pg_settings WHERE pending_restart = true;
-- Restart if needed:
sudo systemctl restart patroni
2を適用します。 PgBouncer
2.1 による接続プーリング。接続プーリングを行う理由_
プーリングなしの問題:
Application: 1000 concurrent users Each user: 1 PostgreSQL connection PostgreSQL: 1000 connections = HIGH overhead
Each connection = ~10MB RAM + fork overhead 1000 connections = ~10GB RAM wasted!
を使用した解決策PgBouncer:
Application: 1000 concurrent users → PgBouncer PgBouncer: Pool of 50 connections → PostgreSQL PostgreSQL: 50 connections = LOW overhead
50 connections = ~500MB RAM ✅
2.2。 PgBouncer_
# Install sudo apt-get install -y pgbouncerCreate config directory
sudo mkdir -p /etc/pgbouncer
Create log directory
sudo mkdir -p /var/log/pgbouncer sudo chown postgres:postgres /var/log/pgbouncer
2.3 をインストールします。 PgBouncer_
# /etc/pgbouncer/pgbouncer.ini [databases] myapp = host=localhost port=5432 dbname=myapp postgres = host=localhost port=5432 dbname=postgres[pgbouncer]
Listen address
listen_addr = * listen_port = 6432
Authentication
auth_type = md5 auth_file = /etc/pgbouncer/userlist.txt
Admin
admin_users = postgres stats_users = monitoring
Pool settings
pool_mode = transaction # session | transaction | statement max_client_conn = 1000 default_pool_size = 25 min_pool_size = 10 reserve_pool_size = 5 reserve_pool_timeout = 3
Connection limits per user/database
max_db_connections = 50 max_user_connections = 50
Timeouts
server_idle_timeout = 600 server_lifetime = 3600 server_connect_timeout = 15 query_timeout = 0 query_wait_timeout = 120
Logging
log_connections = 1 log_disconnections = 1 log_pooler_errors = 1 logfile = /var/log/pgbouncer/pgbouncer.log
Additional
ignore_startup_parameters = extra_float_digits
プール モードの説明:
session mode:
- Connection assigned to client for entire session
- Most compatible
- Least efficient pooling
transaction mode: ✅ RECOMMENDED
- Connection returned to pool after transaction
- Good balance of compatibility and efficiency
- Some features don't work (temp tables, prepared statements)
statement mode:
- Connection returned after each statement
- Most efficient
Least compatible (no multi-statement transactions)
2.4 を構成します。ユーザー認証
# Create userlist sudo tee /etc/pgbouncer/userlist.txt <<EOF "app_user" "md5hashed_password" "postgres" "md5hashed_password" EOFGenerate MD5 hash:
echo -n "passwordusername" | md5sum
Example: "app_user" "md5abc123..."
Or use PostgreSQL to generate:
sudo -u postgres psql -c "SELECT 'md5' || md5('password' || 'app_user');"
sudo chmod 600 /etc/pgbouncer/userlist.txt sudo chown postgres:postgres /etc/pgbouncer/userlist.txt
2.5。 PgBouncer
# Edit systemd service sudo tee /etc/systemd/system/pgbouncer.service <<EOF [Unit] Description=PgBouncer connection pooler After=network.target[Service] Type=forking User=postgres ExecStart=/usr/sbin/pgbouncer -d /etc/pgbouncer/pgbouncer.ini ExecReload=/bin/kill -HUP $MAINPID KillSignal=SIGINT Restart=on-failure
[Install] WantedBy=multi-user.target EOF
Start
sudo systemctl daemon-reload sudo systemctl start pgbouncer sudo systemctl enable pgbouncer
Verify
sudo systemctl status pgbouncer
2.6 を起動します。接続をテスト
# Connect through PgBouncer psql -h localhost -p 6432 -U app_user -d myappCheck PgBouncer stats
psql -h localhost -p 6432 -U postgres pgbouncer -c "SHOW POOLS;"
database | user | cl_active | cl_waiting | sv_active | sv_idle | sv_used
----------+----------+-----------+------------+-----------+---------+---------
myapp | app_user | 10 | 0 | 5 | 5 | 0
postgres | postgres | 0 | 0 | 0 | 2 | 0
cl_active: Active client connections
sv_active: Active server connections
sv_idle: Idle server connections in pool
2.7。アプリケーション構成
# Python example import psycopg2OLD: Direct connection
conn = psycopg2.connect(
host="10.0.1.11",
port=5432,
database="myapp",
user="app_user",
password="password"
)
NEW: Through PgBouncer ✅
conn = psycopg2.connect( host="10.0.1.11", # PgBouncer host port=6432, # PgBouncer port (not 5432!) database="myapp", user="app_user", password="password" )
2.8。 PgBouncer_
# Admin console psql -h localhost -p 6432 -U postgres pgbouncerUseful commands:
SHOW POOLS; SHOW DATABASES; SHOW CLIENTS; SHOW SERVERS; SHOW STATS; SHOW CONFIG;
Reload config without restart
RELOAD;
Pause all connections
PAUSE;
Resume
RESUME;
3 を監視します。 HAProxy
3.1 による負荷分散。 HAProxy アーキテクチャ
Application Servers ↓ HAProxy (VIP: 10.0.1.100:5432) ↓ ├─→ node1 (Primary - Write) :5432 ├─→ node2 (Replica - Read) :5432 └─→ node3 (Replica - Read) :5432
Write traffic → Primary only Read traffic → Round-robin across replicas
3.2。 HAProxy
sudo apt-get install -y haproxyVerify version
haproxy -v
3.3 をインストールします。 HAProxy を構成します
# /etc/haproxy/haproxy.cfg sudo tee /etc/haproxy/haproxy.cfg <<'EOF' global log /dev/log local0 log /dev/log local1 notice chroot /var/lib/haproxy stats socket /run/haproxy/admin.sock mode 660 level admin stats timeout 30s user haproxy group haproxy daemondefaults log global mode tcp option tcplog option dontlognull timeout connect 5000 timeout client 50000 timeout server 50000
Stats page
listen stats mode http bind *:7000 stats enable stats uri / stats refresh 10s stats admin if TRUE
Frontend for write (primary)
frontend postgres_write bind *:5000 mode tcp default_backend postgres_primary
Backend for primary (writes)
backend postgres_primary mode tcp option httpchk http-check expect status 200 default-server inter 3s fall 3 rise 2 on-marked-down shutdown-sessions server node1 10.0.1.11:5432 check port 8008 check-ssl verify none server node2 10.0.1.12:5432 check port 8008 check-ssl verify none backup server node3 10.0.1.13:5432 check port 8008 check-ssl verify none backup
Frontend for read (replicas)
frontend postgres_read bind *:5001 mode tcp default_backend postgres_replicas
Backend for replicas (reads)
backend postgres_replicas mode tcp balance roundrobin option httpchk http-check expect status 200 http-check send meth GET uri /replica default-server inter 3s fall 3 rise 2 server node2 10.0.1.12:5432 check port 8008 check-ssl verify none server node3 10.0.1.13:5432 check port 8008 check-ssl verify none server node1 10.0.1.11:5432 check port 8008 check-ssl verify none backup EOF
構成の説明:
Port 5000: Write traffic → Primary node
- Health check: Patroni REST API port 8008
- If primary fails, backup (replica) can take over
- Backup = only used if primary down
Port 5001: Read traffic → Replicas (round-robin)
- Health check: /replica endpoint
- Primary as backup (if all replicas down)
- Load balanced across healthy replicas
Port 7000: HAProxy stats page
3.4。ヘルスチェック用の Patroni REST API エンドポイント_
# Check if node is leader
curl http://10.0.1.11:8008/leader
Returns 200 if leader, 503 if not
Check if node is replica
curl http://10.0.1.12:8008/replica
Returns 200 if replica, 503 if not
Check if node is running (any role)
curl http://10.0.1.11:8008/health
Returns 200 if running
Master endpoint (redirects to current leader)
3.5。 HAProxy
# Test configuration sudo haproxy -c -f /etc/haproxy/haproxy.cfgStart
sudo systemctl restart haproxy sudo systemctl enable haproxy
Check status
sudo systemctl status haproxy
View logs
sudo journalctl -u haproxy -f
3.6 を開始します。負荷分散
# Test write endpoint (should connect to primary) psql -h localhost -p 5000 -U app_user -d myapp -c "SELECT pg_is_in_recovery();"pg_is_in_recovery
------------------
f ← false = PRIMARY ✅
Test read endpoint (should connect to replica)
psql -h localhost -p 5001 -U app_user -d myapp -c "SELECT pg_is_in_recovery();"
pg_is_in_recovery
------------------
t ← true = REPLICA ✅
Multiple reads should round-robin:
for i in {1..10}; do psql -h localhost -p 5001 -U app_user -d myapp -c "SELECT inet_server_addr();" -t done
Should see different IPs rotating
3.7 をテストします。アプリケーションの使用法
# Application code with read/write splitWrite connection (primary only)
write_conn = psycopg2.connect( host="haproxy-host", port=5000, # Write port database="myapp", user="app_user" )
Read connection (replicas)
read_conn = psycopg2.connect( host="haproxy-host", port=5001, # Read port database="myapp", user="app_user" )
Writes
write_conn.cursor().execute("INSERT INTO users ...") write_conn.commit()
Reads (load balanced)
cursor = read_conn.cursor() cursor.execute("SELECT * FROM users WHERE ...") results = cursor.fetchall()
3.8。 HAProxy
# Access stats pagehttp://haproxy-host:7000/
Shows:
- Backend status (UP/DOWN)
- Current connections
- Requests per second
- Health check results
- Traffic distribution
4 を監視します。スケーリング戦略
4.1 を参照してください。リードレプリカをさらに追加
# Add 4th node as read replicaOn node4:
Install PostgreSQL + Patroni (same as before)
Configure patroni.yml with tags:
tags: nofailover: true # Don't promote to primary noloadbalance: false # Include in load balancing priority: 0 # Lowest priority
Start Patroni
sudo systemctl start patroni
Verify joined cluster
patronictl list postgres
Before (3 nodes): Write: 100% → Primary Read: 50% → Replica1, 50% → Replica2
After (4 nodes): Write: 100% → Primary Read: 33% → Replica1, 33% → Replica2, 33% → Replica3 ✅
4.2。カスケード レプリケーション
# For geographically distributed replicasnode4 (remote datacenter) replicates from node2 instead of primary
In node4's patroni.yml:
bootstrap: dcs: postgresql: parameters: primary_conninfo: 'host=node2 port=5432 user=replicator...'
Topology:
Primary (node1)
↓
├─→ Replica (node2)
│ ↓
│ └─→ Replica (node4 - cascading) ← Reduces load on primary
└─→ Replica (node3)
4.3。応用イオンレベルの読み取りルーティング
# Smart routing based on query typeclass DatabaseRouter: def init(self): self.write_pool = create_pool(host='haproxy', port=5000) self.read_pool = create_pool(host='haproxy', port=5001)
def execute(self, query): # Parse query to determine read vs write if query.upper().startswith(('SELECT', 'WITH')): return self.read_pool.execute(query) else: return self.write_pool.execute(query)
4.4。読み取り分布のモニタリング_
-- On each replica, check query load SELECT count(*) FROM pg_stat_activity WHERE state = 'active';
-- Track queries per replica SELECT pg_stat_statements.query, calls, total_exec_time, mean_exec_time FROM pg_stat_statements ORDER BY calls DESC LIMIT 20;
5。クエリの最適化_
5.1。 pg_stat_statements_
-- Add to postgresql.conf ALTER SYSTEM SET shared_preload_libraries = 'pg_stat_statements';
-- Restart required
sudo systemctl restart patroni
-- Create extension CREATE EXTENSION IF NOT EXISTS pg_stat_statements;
-- View top queries by time SELECT query, calls, total_exec_time, mean_exec_time, max_exec_time FROM pg_stat_statements ORDER BY total_exec_time DESC LIMIT 20;
5.2 を有効にします。遅いクエリを特定します_
-- Currently running slow queries SELECT pid, now() - query_start AS duration, state, wait_event, query FROM pg_stat_activity WHERE state = 'active' AND now() - query_start > interval '10 seconds' ORDER BY duration DESC;
-- Queries with high mean time SELECT query, calls, mean_exec_time / 1000 AS mean_time_seconds, (total_exec_time / 1000 / 3600) AS total_hours FROM pg_stat_statements WHERE mean_exec_time > 1000 -- > 1 second ORDER BY mean_exec_time DESC LIMIT 20;
5.3。分析
コードブロック_415.4の説明。インデックスを作成します_
-- Index for WHERE clause CREATE INDEX CONCURRENTLY idx_users_created_at ON users(created_at);-- Index for JOIN CREATE INDEX CONCURRENTLY idx_orders_user_id ON orders(user_id);
-- Composite index CREATE INDEX CONCURRENTLY idx_orders_user_date ON orders(user_id, order_date);
-- Partial index (for filtered queries) CREATE INDEX CONCURRENTLY idx_active_users ON users(created_at) WHERE status = 'active';
-- CONCURRENTLY = no table lock ✅
5.5。インデックスのメンテナンス
-- Find unused indexes SELECT schemaname, tablename, indexname, idx_scan FROM pg_stat_user_indexes WHERE idx_scan = 0 AND indexname NOT LIKE '%_pkey' ORDER BY pg_relation_size(indexrelid) DESC;-- Find duplicate indexes SELECT pg_size_pretty(SUM(pg_relation_size(idx))::BIGINT) AS size, (array_agg(idx))[1] AS idx1, (array_agg(idx))[2] AS idx2 FROM ( SELECT indexrelid::regclass AS idx, indrelid, (indcollation, indclass, indkey, indexprs, indpred) AS key FROM pg_index ) sub GROUP BY indrelid, key HAVING COUNT(*) > 1 ORDER BY SUM(pg_relation_size(idx)) DESC;
-- Rebuild bloated indexes REINDEX INDEX CONCURRENTLY idx_name;
6。ベスト プラクティス
✅ DO
- 保守的な設定から開始 - 段階的に調整
- 事前に監視するafter_ - 変更の影響を測定_
- 接続プーリングを使用 - Web アプリケーションに必須
- 読み取りと書き込みを分離トラフィック_ - 読み取りを個別にスケール_
- 適切なインデックスを作成 - クエリパターンに基づいて
- 通常のVACUUM -テーブル統計を最新の状態に保つ
- EXPLAIN ANALYZEを使用 - クエリの実行を理解する
- statement_timeoutを設定 - 暴走を防ぐクエリ_
- プールの飽和状態を監視 - 必要に応じて PgBouncer をスケール
- テスト構成の変更 - ステージング中first
❌ 禁止
- work_mem - 最大値を乗算します接続!
- インデックスを作成しすぎない - 書き込み速度を下げる_
- 自動バキュームを無視しない -肥大化_
- 接続プーリングをスキップしない - 接続オーバーヘッドが問題になる
- セッションプーリングを使用しない_ - トランザクションモードより良い_
- 分析を忘れない - 古い統計 = 悪い計画
- 盲目的に調整しない - 自分の現状を理解する変更_
- shared_buffers を高く設定しすぎないでください - 25% 以上の RAM が無駄になります
7。ラボ演習
ラボ 1: PostgreSQL のチューニング
タスク:
- 現在のパフォーマンスをベンチマークするpgbench
- メモリ設定の調整 (shared_buffers、work_mem)
- チェックポイント設定の調整
- pgbench と c を再実行します結果の比較_
- ドキュメントの改善
ラボ 2: セットアップPgBouncer
タスク:
- プライマリ ノードへの PgBouncer のインストール
- トランザクションの構成プーリング
- PgBouncer を使用するようにアプリケーションを更新
- 接続数の監視 (前/後)
- 負荷テストと改善の測定
ラボ 3: HAProxy 負荷バランス
タスク:
- HAProxyのインストールと構成_
- 書き込みおよび読み取りエンドポイントのセットアップ
- テストルーティング (書き込み→プライマリ、読み取り→レプリカ)
- フェイルオーバーのシミュレーション、HAProxy の適応の確認_
- トラフィック分散の監視
ラボ 4: クエリ最適化_
タスク:
- pg_stat_statements を有効にする
- サンプル ワークロードを実行
- トップを特定最も遅い 10 のクエリ_
- EXPLAIN ANALYZE を使用して計画を理解
- 最適化するためのインデックスを作成
- 改善を測定
8。概要_
パフォーマンス調整チェックリスト
- shared_buffers を調整します (25% RAM)
- effect_cache_size を設定します (50 ~ 75%) RAM)
- work_mem を慎重に調整
- チェックポイントを最適化
- SSD の Random_page_cost を下げる
- 有効にするpg_stat_statements
- PgBouncer 接続プーリングのセットアップ
- HAProxy ロード バランシングの構成
- に基づいてインデックスを作成クエリ_
- 監視と反復_
重要な概念
✅ 接続プール - 接続を削減します大幅なオーバーヘッド
✅ ロードバランシング - レプリカ間で読み取りトラフィックを分散
✅ Readスケーリング - 読み取り負荷を処理するためのレプリカの追加
✅ クエリの最適化 - インデックス + EXPLAIN分析
✅ 構成のチューニング - メモリ、I/O、CPU のバランス
次のステップ
レッスン 19 では、カバー ログとトラブルシューティング:
- PostgreSQL ログ分析
- Patroni ログ解釈_
- etcd のトラブル撮影_
- 一般的な問題と解決策
- デバッグ手法とツール_