bitnamicharts/fluentd repository overview

⁠Bitnami Secure Images Helm chart for Fluentd

Fluentd collects events from various data sources and writes them to files, RDBMS, NoSQL, IaaS, SaaS, Hadoop and so on.

Overview of Fluentd⁠

Trademarks: This software listing is packaged by Bitnami. The respective trademarks mentioned in the offering are owned by the respective companies, and use of them does not imply any affiliation or endorsement.

⁠ TL;DR

helm install my-release oci://REGISTRY_NAME/REPOSITORY_NAME/fluentd

⁠ Introduction

This chart bootstraps a Fluentd⁠ deployment on a Kubernetes⁠ cluster using the Helm⁠ package manager.

⁠ Before you begin

• Kubernetes 1.23+
• Helm 3.8.0+
• PV provisioner support in the underlying infrastructure

⁠ Installing the Chart

To install the chart with the release name my-release:

helm install my-release oci://REGISTRY_NAME/REPOSITORY_NAME/fluentd

These commands deploy Fluentd on the Kubernetes cluster in the default configuration. The Parameters⁠ section lists the parameters that can be configured during installation.

⁠ Configuration and installation details

This section describes credentials, configuration, and other installation options.

⁠ Resource requests and limits

Bitnami charts allow setting resource requests and limits for all containers inside the chart deployment. These are inside the resources value (check parameter table). Setting requests is essential for production workloads and these should be adapted to your specific use case.

To make this process easier, the chart contains the resourcesPreset values, which automatically sets the resources section according to different presets. Check these presets in the bitnami/common chart⁠. However, in production workloads using resourcesPreset is discouraged as it may not fully adapt to your specific needs. Find more information on container resource management in the official Kubernetes documentation⁠.

⁠ Rolling VS Immutable tags⁠

It is strongly recommended to use immutable tags in a production environment. This ensures your deployment does not change automatically if the same tag is updated with a different image.

Bitnami will release a new chart updating its containers if a new version of the main container, significant changes, or critical vulnerabilities exist.

⁠ Forwarding the logs to another service

By default, the aggregators in this chart will send the processed logs to the standard output. However, a common practice is to send them to another service, like Elasticsearch, instead. This can be achieved with this Helm Chart by mounting your own configuration files. For example:

apiVersion: v1
kind: ConfigMap
metadata:
  name: elasticsearch-output
data:
  fluentd.conf: |
    # Prometheus Exporter Plugin
    # input plugin that exports metrics
    <source>
      @type prometheus
      port 24231
    </source>

    # input plugin that collects metrics from MonitorAgent
    <source>
      @type prometheus_monitor
      <labels>
        host ${hostname}
      </labels>
    </source>

    # input plugin that collects metrics for output plugin
    <source>
      @type prometheus_output_monitor
      <labels>
        host ${hostname}
      </labels>
    </source>

    # Ignore fluentd own events
    <match fluent.**>
      @type null
    </match>

    # TCP input to receive logs from the forwarders
    <source>
      @type forward
      bind 0.0.0.0
      port 24224
    </source>

    # HTTP input for the liveness and readiness probes
    <source>
      @type http
      bind 0.0.0.0
      port 9880
    </source>

    # Throw the healthcheck to the standard output instead of forwarding it
    <match fluentd.healthcheck>
      @type stdout
    </match>

    # Send the logs to the standard output
    <match **>
      @type elasticsearch
      include_tag_key true
      host "#{ENV['ELASTICSEARCH_HOST']}"
      port "#{ENV['ELASTICSEARCH_PORT']}"
      logstash_format true

      <buffer>
        @type file
        path /opt/bitnami/fluentd/logs/buffers/logs.buffer
        flush_thread_count 2
        flush_interval 5s
      </buffer>
    </match>

As an example, using the above configmap, you should specify the required parameters when upgrading or installing the chart:

aggregator.configMap=elasticsearch-output
aggregator.extraEnvVars[0].name=ELASTICSEARCH_HOST
aggregator.extraEnvVars[0].value=your-ip-here
aggregator.extraEnvVars[1].name=ELASTICSEARCH_PORT
aggregator.extraEnvVars[1].value=your-port-here

⁠ Prometheus metrics

This chart can be integrated with Prometheus by setting metrics.enabled to true. This will expose Fluentd native Prometheus endpoint in a metrics service that can be configured under the metrics.service section. It will have the necessary annotations to be automatically scraped by Prometheus.

⁠Prometheus requirements

It is necessary to have a working installation of Prometheus or Prometheus Operator for the integration to work. Install the Bitnami Prometheus helm chart⁠ or the Bitnami Kube Prometheus helm chart⁠ to easily have a working Prometheus in your cluster.

⁠Integration with Prometheus Operator

The chart can deploy ServiceMonitor objects for integration with Prometheus Operator installations. To do so, set the value metrics.serviceMonitor.enabled=true. Ensure that the Prometheus Operator CustomResourceDefinitions are installed in the cluster or it will fail with the following error:

no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"

Install the Bitnami Kube Prometheus helm chart⁠ for having the necessary CRDs and the Prometheus Operator.

⁠ Gateway API

This chart provides support for exposing Fluentd using the Gateway API⁠ and its HTTPRoute resource. If you have a Gateway controller installed on your cluster, such as APISIX, Contour, Envoy Gateway, NGINX Gateway Fabric or Kong Ingress Controller you can utilize the Gateway controller to serve your application. To enable Gateway API integration, set aggregator.httpRoute.enabled to true. The Gateway to be used can be customized by setting the aggregator.httpRoute.parentRefs parameter. By default, it will reference a Gateway named gateway in the same namespace as the release.

You can specify the list of hostnames to be mapped to the deployment using the aggregator.httpRoute.hostnames parameter. Additionally, you can customize the rules used to route the traffic to the service by modifying the aggregator.httpRoute.matches and aggregator.httpRoute.filters parameters or adding new rules using the aggregator.httpRoute.extraRules parameter.

⁠ Ingress

This chart provides support for Ingress resources. If you have an ingress controller installed on your cluster, such as NGINX Ingress Controller or Contour you can utilize the ingress controller to serve your application. To enable Ingress integration, set aggregator.ingress.enabled to true.

The most common scenario is to have one host name mapped to the deployment. In this case, the aggregator.ingress.hostname property can be used to set the host name. The aggregator.ingress.tls parameter can be used to add the TLS configuration for this host.

However, it is also possible to have more than one host. To facilitate this, the aggregator.ingress.extraHosts parameter (if available) can be set with the host names specified as an array. The aggregator.ingress.extraTLS parameter (if available) can also be used to add the TLS configuration for extra hosts.

Adding the TLS parameter (where available) will cause the chart to generate HTTPS URLs, and the application will be available on port 443. The actual TLS secrets do not have to be generated by this chart. However, if TLS is enabled, the Ingress record will not work until the TLS secret exists.

Learn more about Ingress controllers⁠.

⁠ Backup and restore

To back up and restore Helm chart deployments on Kubernetes, you need to back up the persistent volumes from the source deployment and attach them to a new deployment using Velero⁠, a Kubernetes backup/restore tool. Find the instructions for using Velero in this guide⁠.

⁠ Using custom init scripts

For advanced operations, the Bitnami Fluentd charts allows using custom init scripts that will be mounted inside /docker-entrypoint.init-db. You can include the file directly in your values.yaml, depending on where you are going to initialize your scripts with aggregator.initScripts (or forwarder.initScripts), or use a ConfigMap or a Secret (in case of sensitive data) for mounting these extra scripts. In this case you use the aggregator.initScriptsCM and aggregator.initScriptsSecret values (the same for forwarder).

initScriptsCM=special-scripts
initScriptsSecret=special-scripts-sensitive

⁠ Forwarder Security Context & Policy

By default, the forwarder DaemonSet from this chart runs as the root user, within the root group, assigning root file system permissions. This is different to the default behaviour of most Bitnami Helm charts where we prefer to work with non-root containers⁠.

The default behaviour is to run as root because:

• the forwarder needs to mount hostPath volumes from the underlying node to read Docker container (& potentially other) logs
• in many Kubernetes node distributions, these log files are not readable by anyone other than root
• fsGroup doesn't work with hostPath volumes to allow the process to run non-root with alternate file system permissions

Since we would like the chart to work out-of-the-box for as many users as possible, the forwarder thus runs as root by default. You can read more about the motivation for this at #1905⁠ and #2323⁠, however you should be aware of this, and the risks of running root containers in general.

If you enable the forwarder's bundled PodSecurityPolicy⁠ with forwarder.rbac.pspEnabled=true it will allow the pod to run as root by default, while ensuring as many other privileges as possible are dropped.

⁠Running as non-root

You can run as the fluentd user/group (non-root) with the below overrides if:

• you have control of the hostPath filesystem permissions on your nodes sufficient to allow the fluentd user to read from them
• don't need to write to the hostPaths

Note that if you have enabled the bundled PodSecurityPolicy⁠, it will adapt to the Chart values overrides.

forwarder:
  daemonUser: fluentd
  daemonGroup: fluentd

  securityContext:
    runAsUser: 1001
    runAsGroup: 1001
    fsGroup: 1001

⁠Pod Security Policy & Custom hostPaths

Mounting additional hostPaths is sometimes required to deal with /var/lib being symlinked on some Kubernetes environments. If you need to do so, the bundled PodSecurityPolicy⁠ will likely not meet your needs, as it whitelists only the standard hostPaths.

⁠ Setting Pod's affinity

This chart allows you to set your custom affinity using the XXX.affinity parameter(s). Find more information about Pod's affinity in the kubernetes documentation⁠.

As an alternative, you can use of the preset configurations for pod affinity, pod anti-affinity, and node affinity available at the bitnami/common⁠ chart. To do so, set the XXX.podAffinityPreset, XXX.podAntiAffinityPreset, or XXX.nodeAffinityPreset parameters.

⁠ Securing traffic using TLS

TLS for the Fluentd can be enabled by setting tls.enabled=true. The chart allows two configuration options:

• Provide your own secrets for the forwarder and aggregator using the tls.aggregator.existingSecret and tls.forwarder.existingSecret values.
• Have the chart auto-generate the certificates using tls.autoGenerated=true.

⁠ FIPS parameters

The FIPS parameters only have effect if you are using images from the Bitnami Secure Images catalog⁠.

For more information on this new support, please refer to the FIPS Compliance section⁠.

⁠ Parameters

The following subsections list global, common, and component-specific parameters.

⁠ Global parameters

⁠ Common parameters

⁠ Fluentd parameters

Note: the README for this chart is longer than the DockerHub length limit of 25000, so it has been trimmed. The full README can be found at https://techdocs.broadcom.com/us/en/vmware-tanzu/bitnami-secure-images/bitnami-secure-images/services/bsi-app-doc/apps-charts-fluentd-index.html⁠