kjanat/chatlogger-api-worker

By kjanat

Updated about 1 year ago

Worker for kjanat/chatlogger-api-go, a go backend for the multi-tenant ChatLogger API.

Image
Message queues
API management
0

4.5K

kjanat/chatlogger-api-worker repository overview

ChatLogger API Worker

Go version Docker Pulls GitHub Release License

Overview

The ChatLogger API Worker is a background processing component of the ChatLogger API system. It handles asynchronous tasks, primarily focused on generating exports of chat data in various formats (JSON, CSV). This worker uses Asynq with Redis for reliable job queue management.

Features

  • Asynchronous Export Processing: Handles data export requests without blocking the main API
  • Multiple Export Formats: Supports JSON and CSV formats using the Strategy pattern
  • Resilient Job Handling: Automatic retries, graceful shutdown, and error handling
  • Progress Tracking: Updates export job status in the database throughout processing

Usage

Required Services
  • PostgreSQL: For storing chat data and export job status
  • Redis: For the job queue
Environment Variables
VariableDescriptionDefault
DATABASE_URLPostgreSQL connection stringRequired
REDIS_ADDRRedis address for job queueredis:6379
EXPORT_DIRDirectory to store export filesexports
JWT_SECRETSecret for JWT validationRequired
Running with Docker Compose
version: '3.8'

services:
  # API server component
  api:
    image: kjanat/chatlogger-api-server:latest # or ghcr.io/kjanat/chatlogger-api-server:latest
    ports:
      - "8080:8080"
    depends_on:
      db:
        condition: service_healthy
      redis:
        condition: service_started
    environment:
      - DATABASE_URL=postgresql://dbuser:dbpassword@db:5432/chatlogger
      - REDIS_ADDR=redis:6379
      - HOST=0.0.0.0
      - PORT=8080
      - JWT_SECRET=your-jwt-secret-replace-in-production
      - EXPORT_DIR=/app/exports
      - GIN_MODE=release # Set to `debug` for more verbose logging
    volumes:
      - ./migrations:/app/migrations
      - cl_exports_data:/app/exports
    healthcheck:
      test: ["CMD", "wget", "-q", "--spider", "http://localhost:8080/health"]
      interval: 10s
      timeout: 5s
      retries: 3
      start_period: 5s
    restart: unless-stopped

  # Worker component for background jobs
  worker:
    image: kjanat/chatlogger-api-worker:latest # or ghcr.io/kjanat/chatlogger-api-worker:latest
    depends_on:
      db:
        condition: service_healthy
      redis:
        condition: service_started
    environment:
      - DATABASE_URL=postgresql://dbuser:dbpassword@db:5432/chatlogger
      - REDIS_ADDR=redis:6379
      - EXPORT_DIR=/app/exports
      - JWT_SECRET=your-jwt-secret-replace-in-production
    volumes:
      - cl_exports_data:/app/exports
    healthcheck:
      test: ["CMD", "ps", "aux", "|", "grep", "chatlogger-worker"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 10s
    restart: unless-stopped

  # Database
  db:
    image: postgres:16-alpine
    ports:
      - "5432:5432" # Optional: remove if you don't need direct DB access from host
    environment:
      - POSTGRES_USER=dbuser
      - POSTGRES_PASSWORD=dbpassword
      - POSTGRES_DB=chatlogger
    volumes:
      - cl_postgres_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U dbuser"]
      interval: 5s
      timeout: 5s
      retries: 5
    restart: unless-stopped

  # Redis for job queue
  redis:
    image: redis:alpine
    ports:
      - "6379:6379" # Optional: remove if you don't need direct Redis access from host
    volumes:
      - cl_redis_data:/data
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 5s
      timeout: 5s
      retries: 5
    restart: unless-stopped

volumes:
  cl_postgres_data:
  cl_redis_data:
  cl_exports_data:

Architecture

This worker is part of a clean architecture implementation with:

  • Layered Design: Separation of concerns between handler, service, and repository layers
  • Dependency Injection: Constructor-based DI for improved testability
  • Strategy Pattern: Pluggable exporters for different formats

License

Released under the MIT License.

Tag summary

Content type

Image

Digest

sha256:a900a7418

Size

13.4 MB

Last updated

about 1 year ago

docker pull kjanat/chatlogger-api-worker