pathwaycommons/factoid-docker-config repository overview

⁠factoid-docker-config

This guide is intended for developers and system administrators who need to deploy or maintain Biofactoid using Docker on DigitalOcean.

⁠Table of Contents

• Introduction⁠
• Getting Started⁠
  • Requirements⁠
  • Workflows⁠
• Data Archives⁠
• Database Access⁠
• SSH Access⁠
• Configuration⁠
• Dropbox API⁠
• Scripts⁠
• References⁠

⁠Introduction

Using Docker Compose⁠, multiple containers are coordinated that provide:

• A Reverse-proxy and static file server via NGINX⁠
• HTTPS certificate retrieval and renewal via Certbot⁠ & Let's Encrypt⁠
• An instance of the factoid⁠ application and its related dependencies
  • grounding-search⁠
  • Elasticsearch⁠
  • RethinkDB⁠
  • factoid-converters⁠
  • rethinkdb-backup⁠
• A cloud storage manager rclone⁠ for storing database dumps (e.g. Dropbox)
  • See here⁠ for cron scheduling format

The installation will also update the instance crontab to periodically check if the HTTPS certificates need renewal and if so, renew them.

Each branch of this repository (e.g. production, beta) has its own configuration, which is used to deploy the application to a remote host. Configuration is accomplished largely through environment variables, which are set in the respective .env files, e.g. for the beta branch:

# webapp
BASE_URL=https://beta.biofactoid.org
FACTOID_IMAGE_TAG=production
EMAIL_ENABLED=true

⁠Getting Started

A set of workflows for GitHub Actions have been created to formalize aspects of creating a host, configuring the host and installing the application. Several of the workflows for creating a host and adding a domain are tailored specifically for Digital Ocean⁠: A Digital Ocean Team named biofactoid has been created and you should be added as a member. These workflows represent an explicit description of 'how-to' and so we refer the reader to them for precise details.

⁠Requirements

• GitHub actions for this remote (branch: production, beta) entail SSH login as root, so you'll be asked to add a public SSH key
  • On the Digital Ocean control panel⁠ go to Settings > Security (tab) and copy the fingerprint for the key named factoid-docker-config
  • The associated private key is used by GitHub actions via secret SSH_PRIVATE_KEY
• A registered domain
  • Edit the .env file for NGINX_HOST and BASE_URL to include your registered domain
  • (Optional) If handling domains with Digital Ocean, point your domain to the Digital Ocean nameservers: NS1.DIGITALOCEAN.COM, NS2.DIGITALOCEAN.COM and NS3.DIGITALOCEAN.COM. DNS propagation may take several minutes to hours.
• A Digital Ocean API access token⁠
  • DIGITALOCEAN_ACCESS_TOKEN is used by GitHub actions to access the API
  • Like the SSH private key, this is also a secret available to actions in factoid-docker-config

⁠Workflows

To create an instance of Biofactoid, one of the following workflows should be performed (in order).

⁠Workflow A: Digital Ocean already setup

In this case, the following three items are already configured:

• Digital Ocean Firewall
  • Applied to any Droplet tagged webapp
• Public SSH key
  • To enable GitHub actions to login as root (see Requirements)
• Reserved IPs
  • As of writing (9/2025), there is an unattached IP (209.38.0.162) that maps from beta.biofactoid.org

Navigate to the Actions⁠ tab:

1. Create a Droplet on Digital Ocean (droplet-do-workflow.yml)
   • Name the Droplet something like biofactoid-production (or biofactoid-beta)
   • The Droplet tag needs to be webapp so the DO firewall applies
   • The fingerprint for the SSH key named factoid-docker-config should be added (see Requirements)
2. Assign a reserved IP (see Note 1)
   • From Control panel > Settings > Networking > Reserved IPs
   • Assign a reserved IP to the droplet
3. Configure the remote host (configure-host-workflow.yml)
   • Adds a non-root user, etc
4. Install app on remote host (install-host-workflow.yml)
   • Gets the certs, spins up the app
   • GitHub username and password are required to clone this remote
   • The branch refers to this remote (e.g. beta or production). These will have different configurations (e.g. beta will back up to a separate Dropbox folder)

⁠Workflow B: From scratch

1. Create a cloud firewall (cloud-firewall-do-workflow.yml)
   • Assigned to tags webapp by default
2. Create a Digital Ocean Droplet
3. Create a reserved IP on Digital Ocean (floatingIP-do-workflow.yml)
   • Will assign to indicated Droplet
4. (Optional) Create a domain on Digital Ocean (domain-do-workflow.yml)
   • Will create Type A records for the naked (@) name
5. Configure the remote host
6. Install app on remote host

You can also uninstall:

• Uninstall app on remote host (uninstall-host-workflow.yml)

Notes:

1. If you delete a Droplet associated with a reserved IP, it will remove the reserved IP too :(. - Instead, use Unassign unless you intend on releasing the IP.
2. Rather than run via GitHub Actions, you can install a local workflow runner act⁠

⁠Archives

The RethinkDB service db can be accessed via the dbbackup service, which contains the necessary libraries for dumping and restoring⁠. The rethinkdb-backup system creates RethinkDB dumps with the naming convention rethinkdb_dump_<yyyy-MM-dd_HH-mm-ss-SSS>.tar.gz.

⁠Restoring data from archive

To restore an archive named rethinkdb_dump_<yyyy-MM-dd_HH-mm-ss-SSS>.tar.gz:

1. Copy the archive to a Droplet (e.g. /home/baderlab/rethinkdb_dump_<yyyy-MM-dd_HH-mm-ss-SSS>.tar.gz)

2. Copy the archive in the Droplet to the dbbackup container:

docker cp /home/baderlab/rethinkdb_dump_<yyyy-MM-dd_HH-mm-ss-SSS>.tar.gz dbbackup:/home/appuser/app

1. Restore the archive to a container named db from within the dbbackup container:

docker-compose exec dbbackup /bin/bash

rethinkdb restore --connect db:28015 --import factoid --force /home/appuser/app/rethinkdb_dump_<yyyy-MM-dd_HH-mm-ss-SSS>.tar.gz

• Notes
  • --force: import (and overwrite) data even if a table already exists

⁠Accessing the database admin UI

Use SSH tunneling to connect to the remote database.

Start port forwarding to localhost:8080 for remote <username>@<host> with container name db:

ssh -fNTL localhost:8080:$(ssh <username>@<host> "docker inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' db"):8080 <username>@<host>

Stop port forwarding

kill $(lsof -t -i @localhost:8080 -sTCP:listen)

⁠Accessing via SSH public key authentication

Using the workflow configure-host-workflow.yml, a non-root, non-sudo user (e.g. baderlab) is created for which password access is disabled. To enable SSH public key authentication, obtain your public key (usually in ~/.ssh/id_rsa.pub) then:

Login as root to the instance machine and then login as the non-root user:

sudo su - <username>

Create the required directory/files if they do not already exist:

mkdir ~/.ssh/
chmod 700 ~/.ssh
touch ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys

Paste in your public key to .ssh/authorized_keys, then logout. You should be able to login directly to the non-root account.

⁠Configuration files (conf/)

⁠NGINX (nginx/)

• templates/: As of Nginx 1.19, you can use environment variables⁠ to populate a configuration template (/etc/nginx/templates/*.template).
• dhparam/
  • Contains a Diffie-Hellman shared secret, used for Perfect Forward Secrecy [OpenSSL (1.1.1g 21 Apr 2020)]

    openssl dhparam -out ./conf/nginx/dhparam/dhparam-2048.pem 2048
    

    Copy

⁠RCLONE (rclone/)

The sync service image is rclone⁠. This service can access any dumps created by dbbackup. The sync service runs an HTTP service which can be used to remote control⁠ rclone using its API. For example, it can be instructed to copy/sync files to a configured remote (see rclone.conf below)

• rclone.conf: The remote configuration (i.e. Dropbox) telling rclone which service to use for storage (e.g. Dropbox)

⁠Using the Dropbox API

The rclone container is configured to use a Dropbox App, which can be manipulated via their API⁠. However, authorization is provided by ephemeral access_tokens so they need to be retrieved via the API using a long lived refresh_token.

• Requirements
  • App ID (aka client_id)
  • App secret (aka client_secret)
  • Refresh token

⁠Step 1. Retrieve a Dropbox API access_token

HTTP POST  https://api.dropboxapi.com/oauth2/token
Content-Type: application/x-www-form-urlencoded
Authorization: Basic <Paste your Base64 encoded "App ID:App Secret" here>

grant_type=refresh_token
refresh_token=<Paste your Refresh token here>

You'll get a HTTP 200 response with "access_token" in the JSON response which you'll use below.

⁠Step 2. Use the API (e.g. create folder)

POST https://api.dropboxapi.com/2/files/create_folder_v2
Content-Type: application/json
Authorization: Bearer <Paste your "access_token" here>

{
	"path": "/full/path/to/create",
	"autorename": false
}

On success, you should expect a status 200 with some JSON metadata

⁠Scripts (scripts/)

• cron.sh: Installs cron jobs on host (i.e. every 12 hours)
  • ssl_renew.sh which checks and possibly renews the Let's Encrypt HTTPS certificates via Certbot
• config.sh: Utility for configuring Docker / Compose and environment

⁠References

• Digital Ocean tutorial for How To Secure a Containerized Node.js Application with Nginx, Let's Encrypt, and Docker Compose⁠
• Digital Ocean tutorial for How To Create a Self-Signed SSL Certificate for Nginx on Debian 10⁠
• Tutorial Boilerplate for nginx with Let’s Encrypt on docker-compose⁠