vadymsemeniuk/saboteur

By vadymsemeniuk

Updated 2 months ago

Image
Networking
Developer tools
34

50K+

vadymsemeniuk/saboteur repository overview

Saboteur — HTTP Fault Injection Proxy

A lightweight HTTP-layer fault injection proxy. Intercepts traffic and injects configurable faults based on rules — useful for testing service resilience in development and CI/CD.

Key differentiator from Toxiproxy: operates at HTTP layer — understands URLs, methods, headers, and response bodies. Not TCP-level.


Quickstart

docker run -p 8080:8080 -p 8081:8081 \
  -e UPSTREAM_URL=http://my-service:3000 \
  yourname/saboteur
  • Port 8080 — proxy (send your real traffic here)
  • Port 8081 — control UI + API (manage rules, view traffic)

Open http://localhost:8081 in your browser to use the web UI.


With a config file

docker run -p 8080:8080 -p 8081:8081 \
  -v $(pwd)/config/example.yaml:/etc/saboteur/config.yaml \
  yourname/saboteur --config /etc/saboteur/config.yaml

Docker Compose

docker compose up

Uses docker-compose.yml in the repo root. Spins up saboteur + a mock upstream (hashicorp/http-echo).


Configuration Reference

All settings can be provided via YAML file or environment variables.

YAML keyEnv varDefaultDescription
proxy.upstream_urlUPSTREAM_URL(required)URL to proxy traffic to
proxy.portPROXY_PORT8080Proxy listener port
proxy.timeout_ms30000Upstream request timeout
proxy.max_idle_conns100Max idle upstream connections
control.portCONTROL_PORT8081Control API + UI port
control.bindCONTROL_BIND0.0.0.0Control bind address
control.api_keyAPI_KEY(none)If set, require X-API-Key header on control API
traffic_log.max_entriesTRAFFIC_LOG_SIZE1000Ring-buffer size
log_levelLOG_LEVELinfodebug / info / warn / error
log_formatLOG_FORMATjsonjson / text

Fault Types

Latency

Add delay before forwarding or returning.

fault:
  type: latency
  mode: fixed       # fixed | uniform | normal
  fixed_ms: 500
  apply_to: request # request | response
Error Response

Return a specific status without hitting upstream.

fault:
  type: error
  status_code: 503
  body: '{"error":"down"}'
  headers:
    Retry-After: "30"
Connection Abort

Drop the connection after N bytes.

fault:
  type: abort
  after_bytes: 0    # 0 = drop immediately
Timeout

Accept connection, forward to upstream, never respond.

fault:
  type: timeout
Body Corruption

Replace the upstream response body.

fault:
  type: body_corrupt
  mode: json_invalid  # empty | random_bytes | json_invalid | truncate
  truncate_bytes: 100 # for truncate mode
Header Injection

Add or override headers on request or response.

fault:
  type: header_inject
  apply_to: response   # request | response
  headers:
    X-Injected: "true"
Bandwidth Throttle

Rate-limit response streaming.

fault:
  type: throttle
  bytes_per_second: 1024

Rule System

Rules are evaluated in priority order (lower number = higher priority). A request matches the first rule where all matcher conditions are satisfied.

rules:
  - id: "payment-errors"
    enabled: true
    priority: 10
    description: "503 on 20% of EU payment POSTs"
    match:
      path_prefix: "/api/payment"
      methods: [POST]
      headers:
        X-Region: "EU"
    percentage: 20        # 0-100; fraction of matching requests that get the fault
    fault:
      type: error
      status_code: 503
      body: '{"error":"unavailable"}'
Matcher fields (all optional; unset = match anything)
FieldDescription
pathExact URL path
path_prefixURL path prefix
path_regexRE2 regex on URL path
methodsHTTP methods (empty = all)
headersAll listed headers must match
query_paramsAll listed query params must match

Control API Overview

Base URL: http://localhost:8081

MethodPathDescription
GET/healthHealth check
GET/metricsPrometheus metrics
GET/api/rulesList all rules
POST/api/rulesCreate rule
GET/api/rules/{id}Get rule
PUT/api/rules/{id}Replace rule
PATCH/api/rules/{id}Update fields (enabled, priority, percentage)
DELETE/api/rules/{id}Delete rule
POST/api/rules/resetDelete all runtime rules (preserves config-file rules)
GET/api/trafficTraffic log (?limit=100&path_filter=&fault_only=true)
DELETE/api/trafficClear traffic log
GET/api/traffic/streamSSE stream of live traffic
GET/api/configCurrent effective configuration

Full spec: openapi.yaml

API Key auth
curl -H "X-API-Key: mysecret" http://localhost:8081/api/rules
Quick examples
# Add an error rule
curl -X POST http://localhost:8081/api/rules \
  -H "Content-Type: application/json" \
  -d '{"id":"test-error","enabled":true,"priority":1,"percentage":100,"match":{},"fault":{"type":"error","status_code":503}}'

# Toggle a rule off
curl -X PATCH http://localhost:8081/api/rules/test-error \
  -H "Content-Type: application/json" \
  -d '{"enabled":false}'

# View recent traffic (faults only)
curl "http://localhost:8081/api/traffic?limit=10&fault_only=true"

# Delete a rule
curl -X DELETE http://localhost:8081/api/rules/test-error

# Reset all runtime rules
curl -X POST http://localhost:8081/api/rules/reset

Hot Reload (SIGHUP)

Send SIGHUP to reload the config file without restarting:

kill -HUP $(pgrep saboteur)

Config-file rules are replaced atomically. Runtime rules (created via API) are preserved.


Prometheus Metrics

Available at http://localhost:8081/metrics.

MetricLabelsDescription
fault_proxy_requests_totalmethod, path_pattern, status_code, fault_typeTotal proxied requests
fault_proxy_faults_totalrule_id, fault_typeRequests where a fault was injected
fault_proxy_request_duration_msquantileEnd-to-end latency (includes injected latency)
fault_proxy_upstream_duration_msquantileUpstream-only latency
fault_proxy_rules_activeNumber of enabled rules
fault_proxy_upstream_healthy1 = reachable, 0 = unreachable

Building from source

go build -o saboteur ./cmd/saboteur
UPSTREAM_URL=http://localhost:3000 ./saboteur
Run tests
go test ./...
go test -race ./...
Build Docker image
docker build -f docker/Dockerfile -t saboteur .

Multi-arch:

docker buildx build --platform linux/amd64,linux/arm64 \
  -f docker/Dockerfile -t yourname/saboteur:latest --push .

Out of scope

  • TCP-level fault injection (use Toxiproxy)
  • gRPC support
  • Persistent rule storage (config file is the persistence layer)
  • TLS termination on the proxy port

Tag summary

Content type

Image

Digest

sha256:87685c040

Size

4.6 MB

Last updated

2 months ago

docker pull vadymsemeniuk/saboteur