goprobe/sensor repository overview

⁠Introduction

Today, targeted analyses of network traffic patterns have become increasingly difficult due to the sheer amount of traffic encountered. To enable them, traffic needs to be captured and examined and broken down to key descriptors which yield a condensed explanation of the underlying data.

The NetFlow standard was introduced to address this reduction. It uses the concept of flows, which combine packets based on a set of shared packet attributes. NetFlow information is usually captured on one device and collected in a central database on another device. Several software probes are available, implementing NetFlow exporters and collectors.

goProbe deviates from traditional NetFlow as flow capturing and collection is run on the same device and the flow fields reduced. It was designed as a lightweight, standalone system, providing both optimized low-footprint packet capture and a storage backend tailored to the flow data in order to provide lightning-fast analysis queries.

⁠Scope of this image

This Docker image provides the sensor component (i.e. goProbe) and the local query CLI (i.e. goQuery) to perform requests locally (see below for an example).

The image can be run in non-privileged mode, the only requirement (by definition) is access to the host network in order to capture on system network interfaces.

⁠Usage (Docker)

In order to spin up a running instance of a goProbe sensor, first create an empty directory for the database and set appropriate permissions:

mkdir /tmp/db
chown 1000:1000 /tmp/db

Then, simply run the sensor image:

docker run --rm --name goprobe_sensor --network host -e CONFIG_PATH=/app/goprobe.yaml \
           -v ./examples/docker/goprobe.example.yaml:/app/goprobe.yaml:ro \
           -v /tmp/db:/usr/local/goProbe/db goprobe/sensor:latest

In order to perform a local query against the database, simply execute goquery inside the running container, e.g.

docker exec goprobe_sensor goquery -i any -f -1h sip,dip,dport,proto

⁠Usage (Compose)

A full docker-compose.yaml (including build parameters) can be found in the GitHub repository⁠. A minimal example could look like this:

services:
  sensor:
    image: goprobe/sensor:latest
    restart: unless-stopped
    container_name: goprobe_sensor
    network_mode: host
    environment:
      - DB_PATH=/usr/local/goProbe/db
      - LOCAL__BUFFERS_SIZE__LIMIT=33554432
      - LOCAL__BUFFERS_NUM__BUFFERS=1
      - AUTODETECTION_ENABLED=1
      - API_ADDR=127.0.0.1:3005
      - API_METRICS=1
      - API_PROFILING=0
      - API_KEYS=48fd01fa8f8b12a31fdc614c27c6c273459fe7755a141e9e8deed9f65123f97d
    volumes:
      - db:/usr/local/goProbe/db

volumes:
  db:
    driver: local
    driver_opts:
      type: 'none'
      o: 'bind'
      device: '/path/to/goprobe/db'

For an example configuration file, take a look here⁠.