tinsaetadesse/data-bus

By tinsaetadesse

•Updated 7 days ago

A lightweight, secure, cloud-native Apache Kafka container image for production deployments.

Image

Message queues

921

Overview Tags

tinsaetadesse/data-bus repository overview

⁠Data Bus Container Image

⁠Introduction

Data Bus is a lightweight, production-ready Apache Kafka container image optimized for secure, cloud-native deployments. The image runs in KRaft mode by default, eliminating the need for ZooKeeper, and supports both AMD64 and ARM64 architectures. It provides a secure listener model, mutual TLS (mTLS) authentication, configurable runtime properties, and JVM Shared Archive (JSA) optimization for improved startup performance.

⁠Features

The Data Bus image comes with the below features that make it standout.

KRaft-based Apache Kafka (ZooKeeper-free)
Multi-architecture images (AMD64 and ARM64)
Secure SSL/TLS client communication
Mutual TLS (mTLS) authentication
Configurable runtime using environment variables
Optimized JVM startup using Java Shared Archive (JSA)
Production-ready container layout
Compatible with Docker and Podman

⁠Security Model

The Data Bus image adopts a layered security model that separates internal cluster communication from external client access while supporting encrypted client connectivity through SSL/TLS and mutual TLS authentication.

⁠SSL/TLS Encryption

All producer, consumer and administrative client traffic can be encrypted using SSL/TLS.

⁠Mutual TLS (mTLS)

The image supports client certificate authentication. This allows only trusted clients possessing valid certificates to connect to the cluster.

ssl.client.auth=required

⁠Listener Architecture

The image separates internal cluster communication from external client communication.

Listener	Purpose	Protocol
BROKER	Broker-to-broker communication	PLAINTEXT
CONTROLLER	KRaft controller quorum	PLAINTEXT
CLIENT	Producers, Consumers, AdminClient	SSL

Internal broker replication and KRaft controller traffic remain isolated from client traffic. Only the CLIENT listener is intended for producer, consumer and administrative connections.

⁠Certificates & Secrets

Data Bus brokers use the following certificates and credential files to establish encrypted connections and authenticate clients.

File	Purpose
server.keystore.jks	Broker certificate
server.truststore.jks	Trusted CA certificates
ssl.keystore.credential	Keystore password
ssl.key.credential	Private key password
ssl.truststore.credential	Truststore password

In production environments these files should be supplied using your container platform's secret management mechanism rather than being embedded in the image.

⁠Supported Architectures

Architecture	Platform
x86_64	linux/amd64
ARM64	linux/arm64/v8

⁠Container Layout

The image exposes a stable internal filesystem contract.

Path	Description
`/usr/local/t-soft/data-bus`	Application home
`/etc/t-soft/data-bus`	Configuration files
`/data/t-soft/data-bus`	Persistent data storage
`/var/log/t-soft/data-bus`	Application logs

⁠Persistent Volumes

The image defines the following persistent storage volumes:

Volume	Purpose
`/data/t-soft/data-bus`	Data-bus data
`/var/log/t-soft/data-bus`	Data-bus logs
`/etc/t-soft/data-bus`	Data-bus configuration

Example:

volumes:
    - dbus-data:/data/t-soft/data-bus
    - dbus-log:/var/log/t-soft/data-bus

⁠Runtime Configuration

All Kafka broker properties used in the Data Bus image can be configured using environment variables prefixed with KAFKA_. During container startup, these variables are translated into the corresponding Kafka configuration properties.

Example:

environment:
    ...
    KAFKA_LOG_DIRS: /custom/path/kraft-logs

Security-related environment variables include:

KAFKA_SSL_KEYSTORE_LOCATION
KAFKA_SSL_KEYSTORE_CREDENTIALS
KAFKA_SSL_KEY_CREDENTIALS
KAFKA_SSL_TRUSTSTORE_LOCATION
KAFKA_SSL_TRUSTSTORE_CREDENTIALS
KAFKA_SSL_CLIENT_AUTH

⁠Deployment Example

The compose script below is an example template you can use to quickly spin-up controller and broker containers.

services:
  controller-1:
    image: docker.io/tinsaetadesse/data-bus:latest
    container_name: controller-1
    hostname: controller-1
    environment:
      KAFKA_NODE_ID: 1
      # This instance acts as a controller
      KAFKA_PROCESS_ROLES: 'controller'
      # This broker uses BROKER, and CONTROLLER as listener names 
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: 'BROKER:PLAINTEXT,CONTROLLER:PLAINTEXT'
      KAFKA_CONTROLLER_LISTENER_NAMES: 'CONTROLLER'
      KAFKA_INTER_BROKER_LISTENER_NAME: 'BROKER'
      # Listener configuration
      KAFKA_LISTENERS: 'CONTROLLER://0.0.0.0:9093'
      KAFKA_CONTROLLER_QUORUM_VOTERS: '1@controller-1:9093,2@controller-2:9093,3@controller-3:9093'
      KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
    networks:
      - my-network

  controller-2:
    image: docker.io/tinsaetadesse/data-bus:latest
    container_name: controller-2
    hostname: controller-2
    environment:
      KAFKA_NODE_ID: 2
      # This instance acts as a controller
      KAFKA_PROCESS_ROLES: 'controller'
      # This broker uses BROKER, and CONTROLLER as listener names 
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: 'BROKER:PLAINTEXT,CONTROLLER:PLAINTEXT'
      KAFKA_CONTROLLER_LISTENER_NAMES: 'CONTROLLER'
      KAFKA_INTER_BROKER_LISTENER_NAME: 'BROKER'
      # Listener configuration
      KAFKA_LISTENERS: 'CONTROLLER://0.0.0.0:9093'
      KAFKA_CONTROLLER_QUORUM_VOTERS: '1@controller-1:9093,2@controller-2:9093,3@controller-3:9093'
      KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
    networks:
      - my-network

  controller-3:
    image: docker.io/tinsaetadesse/data-bus:latest
    container_name: controller-3
    hostname: controller-3
    environment:
      KAFKA_NODE_ID: 3
      # This instance acts as a controller
      KAFKA_PROCESS_ROLES: 'controller'
      # This broker uses BROKER, and CONTROLLER as listener names 
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: 'BROKER:PLAINTEXT,CONTROLLER:PLAINTEXT'
      KAFKA_CONTROLLER_LISTENER_NAMES: 'CONTROLLER'
      KAFKA_INTER_BROKER_LISTENER_NAME: 'BROKER'
      # Listener configuration
      KAFKA_LISTENERS: 'CONTROLLER://0.0.0.0:9093'
      KAFKA_CONTROLLER_QUORUM_VOTERS: '1@controller-1:9093,2@controller-2:9093,3@controller-3:9093'
      KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
    networks:
      - my-network

  broker-1:
    image: docker.io/tinsaetadesse/data-bus:latest
    container_name: broker-1
    hostname: broker-1
    ports:
      # Listner ports (port 19092 on the host is mapped to 9192 of the container port)
      - '19092:9192'
    environment:
      KAFKA_NODE_ID: 4
      # This instance acts as a broker
      KAFKA_PROCESS_ROLES: 'broker'
      # This broker uses BROKER, CONTROLLER, and CLIENT as listener names 
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: 'BROKER:PLAINTEXT,CONTROLLER:PLAINTEXT,CLIENT:SSL'
      KAFKA_CONTROLLER_LISTENER_NAMES: 'CONTROLLER'
      KAFKA_INTER_BROKER_LISTENER_NAME: 'BROKER'
      # Listener configuration
      KAFKA_LISTENERS: 'BROKER://0.0.0.0:9092,CLIENT://0.0.0.0:9192'
      # Internal and external advertised listeners
      KAFKA_ADVERTISED_LISTENERS: 'BROKER://broker-1:9092,CLIENT://localhost:19092'
      KAFKA_CONTROLLER_QUORUM_VOTERS: '1@controller-1:9093,2@controller-2:9093,3@controller-3:9093'
      KAFKA_SSL_KEYSTORE_LOCATION: '/etc/t-soft/data-bus/secrets/server.keystore.jks'
      KAFKA_SSL_KEYSTORE_CREDENTIALS: 'ssl.keystore.credential'
      KAFKA_SSL_KEY_CREDENTIALS: 'ssl.key.credential'
      KAFKA_SSL_TRUSTSTORE_LOCATION: '/etc/t-soft/data-bus/secrets/server.truststore.jks'
      KAFKA_SSL_TRUSTSTORE_CREDENTIALS: 'ssl.truststore.credential'
      KAFKA_SSL_CLIENT_AUTH: 'required'
      KAFKA_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM: 'HTTPS'
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
      KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
      KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
    depends_on:
      - controller-1
      - controller-2
      - controller-3
    networks:
      - my-network

  broker-2:
    image: docker.io/tinsaetadesse/data-bus:latest
    container_name: broker-2
    hostname: broker-2
    ports:
      # Listner ports (port 29092 on the host is mapped to 9192 of the container port)
      - '29092:9192'
    environment:
      KAFKA_NODE_ID: 5
      # This instance acts as a broker
      KAFKA_PROCESS_ROLES: 'broker'
      # This broker uses BROKER, CONTROLLER, and CLIENT as listener names 
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: 'BROKER:PLAINTEXT,CONTROLLER:PLAINTEXT,CLIENT:SSL'
      KAFKA_CONTROLLER_LISTENER_NAMES: 'CONTROLLER'
      KAFKA_INTER_BROKER_LISTENER_NAME: 'BROKER'
      # Listener configuration
      KAFKA_LISTENERS: 'BROKER://0.0.0.0:9092,CLIENT://0.0.0.0:9192'
      # Internal and external advertised listeners
      KAFKA_ADVERTISED_LISTENERS: 'BROKER://broker-2:9092,CLIENT://localhost:29092'
      KAFKA_CONTROLLER_QUORUM_VOTERS: '1@controller-1:9093,2@controller-2:9093,3@controller-3:9093'
      KAFKA_SSL_KEYSTORE_LOCATION: '/etc/t-soft/data-bus/secrets/server.keystore.jks'
      KAFKA_SSL_KEYSTORE_CREDENTIALS: 'ssl.keystore.credential'
      KAFKA_SSL_KEY_CREDENTIALS: 'ssl.key.credential'
      KAFKA_SSL_TRUSTSTORE_LOCATION: '/etc/t-soft/data-bus/secrets/server.truststore.jks'
      KAFKA_SSL_TRUSTSTORE_CREDENTIALS: 'ssl.truststore.credential'
      KAFKA_SSL_CLIENT_AUTH: 'required'
      KAFKA_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM: 'HTTPS'
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
      KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
      KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
    depends_on:
      - controller-1
      - controller-2
      - controller-3
    networks:
      - my-network

  broker-3:
    image: docker.io/tinsaetadesse/data-bus:latest
    container_name: broker-3
    hostname: broker-3
    ports:
      # Listner ports (port 39092 on the host is mapped to 9192 of the container port)
      - '39092:9192'
    environment:
      KAFKA_NODE_ID: 6
      # This instance acts as a broker
      KAFKA_PROCESS_ROLES: 'broker'
      # This broker uses BROKER, CONTROLLER, and CLIENT as listener names 
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: 'BROKER:PLAINTEXT,CONTROLLER:PLAINTEXT,CLIENT:SSL'
      KAFKA_CONTROLLER_LISTENER_NAMES: 'CONTROLLER'
      KAFKA_INTER_BROKER_LISTENER_NAME: 'BROKER'
      # Listener configuration
      KAFKA_LISTENERS: 'BROKER://0.0.0.0:9092,CLIENT://0.0.0.0:9192'
      # Internal and external advertised listeners
      KAFKA_ADVERTISED_LISTENERS: 'BROKER://broker-3:9092,CLIENT://localhost:39092'
      KAFKA_CONTROLLER_QUORUM_VOTERS: '1@controller-1:9093,2@controller-2:9093,3@controller-3:9093'
      KAFKA_SSL_KEYSTORE_LOCATION: '/etc/t-soft/data-bus/secrets/server.keystore.jks'
      KAFKA_SSL_KEYSTORE_CREDENTIALS: 'ssl.keystore.credential'
      KAFKA_SSL_KEY_CREDENTIALS: 'ssl.key.credential'
      KAFKA_SSL_TRUSTSTORE_LOCATION: '/etc/t-soft/data-bus/secrets/server.truststore.jks'
      KAFKA_SSL_TRUSTSTORE_CREDENTIALS: 'ssl.truststore.credential'
      KAFKA_SSL_CLIENT_AUTH: 'required'
      KAFKA_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM: 'HTTPS'
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
      KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
      KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
    depends_on:
      - controller-1
      - controller-2
      - controller-3
    networks:
      - my-network

⁠Example SSL Client Configuration

The properties file below is an example template you can use to quickly configure Producers, Consumers, and AdminClients.

security.protocol=SSL

ssl.truststore.location=/path/client.truststore.jks
ssl.truststore.password=<<your-password>>

ssl.keystore.location=/path/client.keystore.jks
ssl.keystore.password=<<your-password>>

ssl.key.password=<<your-password>>

⁠Production Recommendations

For production deployments:

Enable SSL for all client connections.
Require mutual TLS authentication.
Store certificates and passwords as container secrets.
Rotate certificates regularly.
Monitor certificate expiration.

⁠Troubleshooting

⁠Listener configuration

If clients can reach the bootstrap server but fail to obtain metadata, verify that advertised.listeners contains addresses reachable from the client environment.

When building the container image, temporary listener overrides are used to allow Kafka CLI utilities to communicate with the broker while generating the Java Shared Archive (JSA).

⁠SSL handshake failures

Verify:

Truststore location
Keystore location
Password credential files
Client certificates
Listener configuration

Tag summary

Recent tags

Content type

Image

Digest

sha256:688f54b83…

Size

321.9 MB

Last updated

7 days ago

docker pull tinsaetadesse/data-bus