Official image for JQM (Job Queue Manager), an open source batch job manager
10K+
This is the official image for JQM (Job Queue Manager). It has Docker images available for Windows Nano 1709+ and Linux under the Apache Public License, v2.
The aptly named Job Queue Manager, or JQM for short, is a queue manager. It has three goals:
The result is a small-footprint, easy to use grid execution system that takes care of everything which would be boilerplate code or missing otherwise: configuring logs, throttling processes, handling priorities between different classes of jobs, distributing the load over multiple servers, distributing the files created by the jobs, and much more...
It is able to run anything that can be run on the command line without modifications. It also has a very rich Java integration which make it an ideal "job application server" for Java users - with no modifications required, allowing to directly use code from plain Main to Spring Batch and other frameworks... Basically, it is a very lightweight application server specifically tailored for making it easier to run Shell and Java batch jobs.
Full product description and documentation is available at read the docs, code is on GitHub. Maintainer (Enioka company) website (in French) is at http://www.enioka.com.
All these tags are shared, i.e. multi-architecture - each tag named "mytag" actually hides multiple platform-specific tags: mytag-windows-1803, mytag-windows-1809, mytag-linux-alpine. There should be no reason to directly use the platform-specific tags instead of the shared tags.
The first release tag is 2.1.0, previous versions have no official images.
Windows versions stay supported as long as they are publicly supported by Microsoft (1709 is no more supported for example, so 2.2.1 is the last version to have a 1709 image). Note that beginning with Windows 1809 (also known as 2019) there is limited kernel backward compatibility - the 1809 image should work on a Windows host 1909 for example. We still provide version-specific images.
Run: docker run -it --rm -p 1789:1789 enioka/jqm
A single-node, fully functional server starts after a few seconds. The web UI is available on the local machine, port 1789.
Use CTRL+C to exit. All changes made to the server are lost on exit.
In this scenario, a developer uses his preferred IDE or build tool and the resulting product of his work is automatically made available to a local JQM server.
It uses a ready to use single-node JQM install configured with sensible options for a developer: web-UI is enabled, authentication is disabled, sample jobs are ready to launch. It relies on a self-contained HSQLDB database, so it has no dependencies whatsoever.
Run: docker run -it -p 1789:1789 -v /path/to/target/build/directory:/jqm/hotdeploy enioka/jqm
Adapt paths to your environments (and do not forget C:/ on Windows). The web UI is made available on the local machine, port 1789.
Configure your build system (Maven, Gradle, Ant...) to copy your job and deployment descriptor (jars and the XML) inside /path/to/target/build/directory/myjob (myjob is a name of your choosing. Do not copy files directly inside the mount directory). Files should disappear after a few seconds, and jobs should appear as ready to run inside the web UI.
Use CTRL+C to exit the node (obviously, running the node in the background then using docker stop also works).
In more details...
To deploy your batch jobs, copy both the jar and the XML deployment descriptor inside the /jqm/hotdeploy mount.
- A hotdeploy sub-directory is a
deployment unit. It contains the deployment descriptor and the jar files referenced by this deployment descriptor.
- The name of this sub-directory is important, as subsequent deployments with the same name will replace each other.
- Only first-level sub-directories are scanned.
- The deployment mechanism is triggered by the deployment descriptor.
- When copying files into a subdirectory it is therefore preferable to copy the descriptor last.
- To avoid using
inotifywhich has cross-platform issues, the directories are simply scanned every 30s.- There can only be a single deployment descriptor per sub-directory. Directories with multiple XML files are ignored.
- The jqm.jar.path inside the deployment descriptor is interpreted without its full path (JQM will use its own directory structure). The file name (base name in Unix terms) inside this path is however still very important, as there may be multiple jar files per deployment unit. This allows to reuse the "production" deployment descriptor directly.
- All files are moved regardless of their nature, respecting existing directory structure. (such as an eventual
libdirectory with libraries, see the packaging documentation)- The files are removed from hotdeploy on success. (sub-directory is emptied, but not removed)
This deployment mechanism is a Docker image specificity, and is not available on non-Docker deployments. It is also only available in single-node deployments.
Note: in this scenario, the database is inside the container. So deleting the container deletes the database and all configuration changes with it.
In this scenario, batch jobs are packaged inside a custom image. The server still uses an internal HSQLDB database. The result is a ready to run server already containing the jobs (which can in turn be triggered manually, or simply be scheduled).
The only thing needed is to copy the batch files (jar(s) and deployment descriptor(s)) inside the job repository, and import the job descriptor as in any deployment.
An example Dockerfile would be:
FROM enioka/jqm
# Copy jars and deployment descriptors inside the job definition repository, as in a normal deployment
COPY buildresult/* /jqm/jobs/
RUN java -jar jqm.jar -importjobdef ./jobs
The resulting image can then be run just as the base JQM image.
In this scenario, the goal is to have an easy to scale and deploy image with Swarm (or any orchestrator like Kubernetes) on a production or production-like environment.
The batch jobs are directly inside a custom image. The driver required to access the external central database is inside /jqm/ext. So the image builder just has to copy the jar and deployment descriptors inside /jqm/jobs[/subdirectory...].
An example Dockerfile would be:
FROM enioka/jqm
ENV JQM_POOL_DRIVER="oracle.jdbc.OracleDriver" \
JQM_POOL_VALIDATION_QUERY="SELECT SYSDATE FROM DUAL" \
JAVA_OPTS="-Xms1g -Xmx1g -XX:MaxMetaspaceSize=128m" \
JQM_NODE_NAME=_localhost_
JQM_INIT_MODE=CLUSTER
COPY ojdbc4.jar /jqm/ext/
COPY buildtarget/* /jqm/jobs/
Note the JQM_INIT_MODE variable: it tells (among other things) JQM to create the node (as named by JQM_NODE_NAME, here the container name - _localhost_ is a special value valid in this image) in the configuration if it does not exists, using a template (here the default one, changeable with JQM_CREATE_NODE_TEMPLATE). That way starting the container is enough to add the new node to the cluster. Also, database information should be provided on the command line or in a compose/K8s/... file. For example:
docker run -p 1789:1789 --env "JQM_POOL_USER=login" --env "JQM_POOL_PASSWORD=secret" --env "JQM_POOL_CONNSTR=jdbc:oracle:thin:@//oraclehost:1521/MYINSTANCE"
When using JQM_INIT_MODE=CLUSTER the database is not automatically upgraded, nor are jobs imported on startup: inside a cluster, the decision to modify the shared store structure should be manual and not the result of the deployment of a single node with a different version. To help with upgrades, a mode named UPDATER exists - a JQM container started in this mode will simply upgrade the database and import job definitions. It can be used in environment bootstraps and upgrades.
Also, an interesting environment variable is JQM_CREATE_NODE_TEMPLATE. It specifies the name of a template node (any existing node is OK as a template) to create new nodes. When the first node is created, two templates are automatically referenced, named TEMPLATE_WEB (default value - with full web interface) and TEMPLATE_DRONE (without the web interface enabled). These templates can be modified, removed, etc - and replaced with your own if required.
A sample swarm file (also working as a compose file) is provided in the JQM sources.
For production environments, the first thing to do should be to enable security on the web services.
Rather than building directly upon the enioka/jqm image, it can be useful to build upon the result of the "Redistributable batch test" scenario in many integration pipelines.
status.podIP) - new in version 2.2.7.org.hsqldb.jdbcDriver.SELECT 1 FROM INFORMATION_SCHEMA.SYSTEM_USERS.Content type
Image
Digest
sha256:a82681c76…
Size
256.2 MB
Last updated
about 1 month ago
docker pull enioka/jqm