System Overview

NodePit Runner is made up of four services that work together. Understanding how they fit together helps you plan your deployment, troubleshoot problems, and appreciate what happens behind the scenes when a scheduled workflow runs.

Components

Web serves the browser-based user interface. It talks exclusively to the API — it has no direct access to the database or executor infrastructure.

API is the central service. It stores all configuration and state in MongoDB, serves the REST API consumed by the Web frontend and by external clients, and coordinates executors. See API Reference for the available endpoints.

MongoDB is the persistence layer. All projects, environments, schedules, runs, logs, and user accounts are stored here.

Executor is the service that actually runs your workflows and scripts. It polls the API for pending runs, builds a Docker image for the run's environment (or pulls a cached one), and starts a short-lived worker container to execute the workflow or script in isolation. When the worker finishes, the executor collects logs and output files and reports the result back to the API. You can run multiple executor instances to handle concurrent runs.

Infrastructure requirements

Each executor needs two things on the host:

• Docker socket (/var/run/docker.sock) — required to build images and spawn worker containers. This gives the executor the same privileges as the Docker daemon on the host; treat executor host access accordingly.
• Shared data volume — mounted at /opt/data in the executor container and passed through to each worker. Used to transfer the workflow file and additional files into the worker and collect output files from it.

Both are pre-configured in the official docker-compose.yml and require no manual setup for a standard deployment.

External network access

Runner makes outbound connections to several external services. Deployments in restricted network environments need to allow these.

Host	Environments	Purpose
nodepit.com	All	Extension Search in the UI; update version check shown in Settings
registry-1.docker.io	All	Docker Hub — Ubuntu base image (KNIME) and Alpine base image (Shell)
download.nodepit.com	KNIME	NodePit Batch, used internally to execute KNIME workflows
archive.ubuntu.com, security.ubuntu.com	KNIME	Ubuntu apt — system dependencies
download.knime.org	KNIME	KNIME Analytics Platform
Update sites (as configured)	KNIME	Extension update sites; only required when extensions are installed
dl-cdn.alpinelinux.org	Shell	Alpine apk — system dependencies

All external connections are outbound-only downloads. No workflow data, run results, logs, or user information is transmitted to any of these services.

Scaling and multi-executor deployments

In the default configuration, Runner runs one executor on the same host as the API and MongoDB. To handle more concurrent runs, scale up the number of executor containers on the same host:

docker compose scale executor=2

Each executor instance registers independently with the API and can pick up runs in parallel. However, the Docker image build stage — pull, build, and push — is serialized across all executors on the same host using a host-wide lock. Only one executor builds at a time; others wait. This means multiple executors on one host increase run throughput but not build throughput.

The recommended approach for higher capacity is one executor per host. This gives each executor the full resources of its host, eliminates build lock contention, and provides independent failure domains — if one host goes down, the remaining executors continue processing runs.

When running executors on multiple hosts, a private Docker registry lets them share built images instead of each rebuilding from scratch on every host.