softwaredam/fs-compare repository overview

⁠FS-Compare Integrity checker

FS-Compare is Softwaredam's⁠ fast tool to compare two sources of files and find out whether they match, or otherwise what the differences are.

The use cases vary. Some of the examples are:

• Saving the meta-data (file structire and hashes) of files in a separate location than the files themselves.
• Keeping a manifest of backups to detect tampering
• Compare source and target files during storage migrations.

It does the following:

• Compare the file and directory structure.
• Calculate the MD5 checksums of each file.
• Produce a source-manifest.json which contains all the information of the source folder.
• Produce a target-manifest.json which contains all the information of the target folder.
• Produce a diff manifest of source and target.

⁠Caution

Please use this tool, as is, at your own risk. Be aware that playing with your production files can harm your business, unless you know what are you doing. If you need any assistance, please contact. We provide no guarantee at all!

Other precautions:

• Despite that we have done our best, we advise that you do not give write access to the process which runs on your files.
• For this tool it suffices to have read and list rights only.
• This tool only requires write access to a totally separate PVC/folder for writing its manifest reports to it.

⁠License

Except for commercial purposes, this product is otherwise free to use. If the default built-in trail period is not enough, please contact [email protected]⁠ to get your license.

⁠Usage

• We support Docker (image) and Kubernetes (helm) at the moment.
• We support multiarch containers, amd and arm.

⁠Helm

Our helm chart creates a Kubernetes Job, which compares the source and target folder. It also creates a PVC for saving the source, target and final result manifests.

Prepare your values file first, see values in the chart, example my-values.yaml:

resources:
  limits:
    cpu: "2"
    memory: 4Gi
  requests:
    cpu: "2"
    memory: 4Gi

#set extra labels, if needed. 
extraLabels:
  customer: your-org
  egress: label 


pullerSecrets: [ "any pull secret if you pull from protected proxies" ]
persistence: 
  storage:
    class: <class>
    size: 2Gi
    accessMode: ReadWriteMany

sourcePvc: <your source pvc that the job pods is mounted on. Only read-only and list file permissions are required>

targetPvc: <your target pvc that the job pods is mounted on. Only read-only and list file permissions are required>

appConfig:
  SOURCE: '/data/source/' #this is the mount points of the source pvc. which means that the whole root will be used for comparison.
  TARGET: '/data/target/' #this is the mount points of the source pvc. which means that the whole root will be used for comparison.
  SKIP_HASH: 'false' #default: true. Set to false to only compare the file structure.

Install:

helm template <my-release> \
  oci://registry-1.docker.io/softwaredam/fs-compare \
  --version 0.3-main-chart-70bf3ee5 \
  --values my-values.yaml
  --set appConfig.LICENSE_KEY="LICENSE_KEY_PROVIDED_BY_SOFTWAREDAM"

⁠Diagram source

Dockerhub unfortunately does not support mermaid graphs. We still keep the source here.

graph TD
    source@{ shape: "cyl", label: "Source" }
    target@{ shape: "cyl", label: "Target" }
    fs-compare@{ label: "FS-Compare" }
    fs-compare --> source
    fs-compare --> target