vlbsio/keel

By vlbsio

Updated about 1 month ago

KEEL — High-performance PostgreSQL/MySQL connection pooler and proxy

Image
Networking
Developer tools
Databases & storage
1

3.4K

vlbsio/keel repository overview

KEEL — More than a pooler. The missing link between your application and a truly elastic database

KEEL — Database Connection Pooler & Proxy

KEEL is a correctness-first PostgreSQL connection pooler and intelligent database proxy. Its production candidate profile is conservative PostgreSQL transaction pooling with prepared-statement virtualization. Broader routing, MySQL, sharding, and cluster features exist, but are labelled by maturity and should be enabled deliberately.


Why KEEL?

ProblemHow KEEL solves it
Too many connections overloading the databaseTransaction pooling multiplexes hundreds of app threads over a small backend pool
Read replicas sitting idle while the primary is overloadedSmart read/write routing exists in the Hardening bucket and can be enabled deliberately
ORM prepared statements breaking pooling (Hibernate, pgx, GORM, SQLAlchemy)Transparent prepared-statement virtualization — no ORM changes required
Session state (SET, search_path, GUCs) lost when connections are reusedSession-context preservation is being hardened for stateful workloads
Patroni failover causing application errorsRole observation and conservative routing are in the Hardening bucket
Cloud IAM token rotation for RDS/Cloud SQL/AzureCloud auth providers are available as Experimental/Hardening features depending on provider and deployment
Compliance requirements for audit logging and TLS enforcementStructured audit log, mTLS, LDAP/PAM integration, privilege drop, seccomp filter

Available Images

TagBaseSizeUse when
Immutable Debian release tags, debianDebian Trixie Slim~120 MBDefault base, best library compatibility
ubuntu, immutable Ubuntu tagsUbuntu 24.04 LTS~140 MBUbuntu-centric environments
alpine, immutable Alpine tagsAlpine 3.20~40 MBMinimal footprint, musl libc
latestDebian base~120 MBMutable convenience tag for development

All images are multi-arch: linux/amd64 and linux/arm64.

Production deployments should pin an immutable release tag or digest.


Quick Start

Minimal PostgreSQL pool (replacing PgBouncer)
docker run -d \
  -e KEEL_SERVER_HOST=your-postgres-host \
  -e KEEL_SERVER_PORT=5432 \
  -e KEEL_SERVER_USER=app \
  -e KEEL_SERVER_PASSWORD=secret \
  -e KEEL_SERVER_DATABASE=mydb \
  -p 7432:7432 \
  vlbsio/keel:<release-tag>

Then connect your app to localhost:7432 exactly as you would connect to PostgreSQL directly.

With a config file
docker run -d \
  -v /path/to/keel.ini:/etc/keel/keel.ini \
  -p 7432:7432 \
  vlbsio/keel:<release-tag>
Docker Compose
services:
  keel:
    image: vlbsio/keel:<release-tag>
    ports:
      - "7432:7432"   # PostgreSQL proxy
      - "9101:9101"   # Prometheus metrics
    volumes:
      - ./keel.ini:/etc/keel/keel.ini
    restart: unless-stopped

Feature Readiness

KEEL has a broad feature surface. Features are labelled by production maturity. The canonical maturity matrix lives in docs/PRODUCTION_READINESS.md; the summary below mirrors it.

Recommended production deployment: PostgreSQL in mode = pool with prepared_statement = virtualize and experimental_features = false. MySQL support, smart routing, SSV, Patroni failover, and transaction tracking are in the Hardening bucket — enable deliberately with monitoring.

Production candidate

These are the supported production candidate capabilities when configured conservatively and validated against your workload.

FeatureWhat it does
PostgreSQL transaction poolingConnections returned to the pool after each transaction; hundreds of app connections share a small backend pool
PostgreSQL prepared-statement virtualizationNamed prepared statements transparently replayed on any backend — works with Hibernate, pgx, GORM, SQLAlchemy, Prisma
Full TLS + mTLSFrontend TLS termination, backend TLS, optional client certificate verification (kTLS acceleration is Hardening — kernel/cipher support varies)
SCRAM-SHA-256 / MD5 authPostgreSQL authentication; prefer SCRAM-SHA-256 for production
Admin consolepsql-compatible admin interface: SHOW POOLS, SHOW STATS, SHOW SERVERS, RELOAD, and 20+ commands
Prometheus metrics/metrics endpoint with pool, session, query, and TLS counters + P50/P95/P99 histograms
Live config reloadSIGHUP reloads pool sizes, TLS certs, timeouts, and server weights without restart
Graceful drain/shutdownIn-flight queries complete before shutdown; commit-in-doubt sessions are never force-closed
Privilege drop + seccompSupported hardening controls; enable in production where the runtime supports them
🔶 Hardening (works, needs validation in your environment)

These features are implemented and tested but the full failure-mode surface is still being hardened. Enable deliberately with monitoring.

FeatureWhat it does
MySQL transaction poolingWire support, prepared-statement session map, GTID capture, XA pinning — parity with PostgreSQL tracked in docs/MYSQL_PG_PARITY.md
Automatic read/write splittingSQL parser classifies queries; SELECTs go to replicas, writes to primary — with sticky-primary override after writes
Patroni / health probe failoverAutomatic role detection via pg_is_in_recovery(), Patroni REST API, MySQL @@read_only; dead servers removed from rotation
Session-context preservation (SSV)Keeps SET parameters, search_path, and session GUCs consistent across pooled backends
Transaction trackingXID-based commit-in-doubt recovery; read-after-write consistency tokens
NOTIFY / LISTEN proxyingTransparent proxy for PostgreSQL pub-sub; LISTEN sessions pinned automatically
Query rules (declarative routing)INI-based route overrides, blocking, and SQL rewriting — no scripting required
OSC proxyingAuto-detects gh-ost and pt-online-schema-change; pins shadow-table DML to primary
Cloud-native authAWS RDS IAM, GCP Cloud SQL IAM, Azure AD/Entra — validate token renewal and provider-outage behaviour per environment
kTLS accelerationKernel TLS offload — kernel and cipher compatibility vary by deployment
🧪 Experimental (opt-in, not production defaults)

These require experimental_features = true in config. Use in non-critical environments or staging only.

FeatureWhat it does
Horizontal shardingTransparent shard-key extraction from SQL; scatter-merge aggregations across shards
Scatter-merge queriesCOUNT, SUM, AVG, MIN, MAX, GROUP BY, ORDER BY, LIMIT merged globally across shards
WAL LSN / GTID replica catch-upToken-based replica lag checks for cross-service read-after-write safety
Multi-proxy HA cluster2–3 KEEL nodes gossip configuration and monitor each other; zlib/zstd wire compression for WAN
Result cacheQuery result caching framework — Aspirational: infrastructure hooks exist, correctness/invalidation guarantees are not production-supported

Database & Platform Support

The supported PostgreSQL version list and database maturity notes live in docs/COMPATIBILITY.md. PostgreSQL is the production candidate baseline; MySQL, MariaDB, and Percona topologies are in the Hardening bucket.

Cloud databases
CloudServiceAuth method
AWSRDS, AuroraIAM token (SigV4, auto-rotated)
GCPCloud SQLService account JWT + metadata server
AzureDatabase for PostgreSQL/MySQLManaged identity (IMDS + env var fallback)

No password rotation scripts needed — KEEL handles token generation and caching automatically.

Linux kernel requirements
KernelI/O backendNotes
5.6+io_uring (default)Best performance; recommended for production
5.4–5.5epoll (auto-fallback)Works; silent fallback
Any (unprivileged)epoll (auto-fallback)Docker rootless: EPERM on io_uring → epoll

Security Hardening

KEEL supports defense-in-depth controls:

  • Seccomp BPF filter — allowlist-only syscall policy (baseline or strict mode), opt-in by config
  • Privilege drop — drops to unprivileged user after binding ports, opt-in by config
  • mTLS — optional client certificate verification with CN/SAN identity extraction
  • LDAP / PAM integration — enterprise authentication via ldap_bind() or pam_authenticate()
  • Structured audit log — NDJSON audit trail for auth, admin, query, and pool events; filterable by event type
  • Cipher enforcement — configurable TLS 1.2/1.3 cipher suites and minimum version
  • Certificate hot-reloadSIGHUP triggers atomic TLS context swap without dropping connections
  • Known limits surfaced — review docs/LIMITATIONS.md before enabling Hardening or Experimental features

Configuration Example

[keel]
log_level = info

[worker_group.myapp]
protocol          = postgresql
bind_addr         = 0.0.0.0
bind_port         = 7432
num_workers       = 4
mode              = pool
prepared_statement = virtualize     # transparent PS replay for ORMs
min_pool_size     = 10
max_pool_size     = 100
auth_method       = scram-sha-256

[worker_group.myapp.servers]
primary  = host=db1.internal port=5432 dbname=myapp user=app password=secret role=RW weight=100

For the full configuration reference, see docs/DOCKER.md.


Exposed Ports

PortPurpose
7432PostgreSQL proxy
7306MySQL proxy
6433Admin console (psql-compatible)
9100Cluster peer communication
9101Prometheus metrics + web UI

Environment Variables

Common overrides for container deployments (no config file required for simple setups):

KEEL_LOG_LEVEL=2              # 0=trace … 5=fatal
KEEL_SERVER_HOST=db.internal
KEEL_SERVER_PORT=5432
KEEL_SERVER_USER=app
KEEL_SERVER_PASSWORD=secret
KEEL_SERVER_DATABASE=mydb
KEEL_POOL_MIN=10
KEEL_POOL_MAX=100
KEEL_BIND_PORT=7432

Documentation

GuideDescription
Quick start & DockerDocker quick-start, env vars, Compose templates, Kubernetes
Production readinessFeature maturity matrix, failure-mode guide, operator checklist
Admin SQL consoleSHOW POOLS, SHOW STATS, RELOAD, health endpoints
Prepared statementsPooling strategies for ORMs (virtualize / pinning / tracking)
Session contextGUC and SET preservation across pooled backends
Read/write splittingSQL routing, replica selection, sticky-primary override
Cloud authAWS RDS IAM, GCP Cloud SQL, Azure Entra token management
ShardingHorizontal sharding, scatter-merge, admin virtual tables
Tracing (OpenTelemetry)W3C traceparent injection, OTLP export, per-query spans
Runtime modesPROXY / POOL / SMART / FULL tier guide
Hooks (Lua / Python)Scripted routing, query rewriting, and pipeline extensions
OperationsSIGHUP reload, graceful drain, signal reference
Cluster compressionMulti-proxy HA, zlib/zstd WAN compression

Kubernetes

KEEL ships a Helm chart and a Kubernetes operator:

helm install keel oci://ghcr.io/virtlabs-io/helm/keel \
  --set config.server.host=postgres-primary.default.svc \
  --set config.server.password=secret \
  --set monitoring.enabled=true

The operator (KeelPool CRD) handles pool lifecycle, config reconciliation, and HPA integration via the pool_wait_queue_enqueued metric.


License

AGPL-3.0 — free to use, modify, and distribute. Network use (SaaS) requires source disclosure.
See LICENSE for details.


KEEL — the missing link between your application and a truly elastic database.

Tag summary

Content type

Image

Digest

sha256:7a1eb5e87

Size

43.2 MB

Last updated

about 1 month ago

docker pull vlbsio/keel:ubuntu