mgibio/standalone-input-staging repository overview

The stager.py script copies all inputs to a cwl workflow into a single location in order to cut down on unnecessary file copying as Cromwell runs. To use, run python /usr/bin/stager.py <input yaml | input json> <staging_directory> <workflow.cwl> within the mgibio/standalone-input-staging docker image. It outputs a new yaml file, with "staged_" prepended to the name of the input yaml; this new file can then be used as normal to run the workflow.

Currently, files with the same name will be ignored. This will likely change once we have discussed how we want to handle this. The workflow.cwl file must be passed to the script to properly capture secondaryFiles; upcoming updates will make parsing this file safer, but won't change the usage or output of the script. Currently, this can only be used as a standalone script, but there is a workaround that may allow this to be run as part of a workflow, if needed. It may also allow for links to be passed to Cromwell and then used normally, without the need for copying.