Chapter 29: Running Models with the DisSModel Platform¶

Part VI — Data & Infrastructure

Covers the dissmodel-platform and dissmodel-configs packages.

Watch out

This chapter is architectural, not hands-on — running the platform means bringing up several Docker containers, which this book's own sandbox can't do. Every command below is accurate to the project's own setup instructions, but none of it was executed while writing this chapter, unlike every code cell elsewhere in this book. Treat the code as a guide to follow on your own machine, not as output to trust blindly.

Learning Objectives¶

By the end of this chapter you will be able to:

Bring the DisSModel Platform up locally with Docker Compose
Describe the platform's service architecture and how a job moves through it
Register a new model on the platform via a dissmodel-configs TOML file
Trace a real client — a QGIS plugin — as it submits, polls, and renders a remote experiment

In [ ]:

Copied!

# Standard imports
import numpy as np
import pandas as pd
# Standard imports
import numpy as np
import pandas as pd

Every ModelExecutor in this book so far has run exactly where you typed the command — your own terminal, your own notebook kernel. That stops scaling the moment a model needs more compute than your laptop has, or the moment someone without a Python environment at all needs to run it. The DisSModel Platform takes the identical ModelExecutor contract from Chapter 21 and Chapter 28 and turns it into a shared, multi-user service — without changing a line of model code.

Status and Where It Runs¶

The project is explicit that this is an MVP: security hardening and raw performance aren't the current focus, and production deployments should proceed with that in mind. It's designed to run anywhere from a single laptop to a shared cluster:

a local machine, for development and small experiments
an on-premise cluster (the kind universities and research institutes like INPE already run)
a private cloud (OpenStack and similar)
a public cloud (AWS, GCP, Azure), as an optional target rather than a requirement

That range matters for a research tool specifically: a lab doesn't need cloud infrastructure to start using it, and isn't locked out of scaling up later.

Installation and Architecture¶

Bringing the whole stack up locally is one git clone and one docker compose command:

git clone https://github.com/DisSModel/dissmodel-platform.git
cd dissmodel-platform
cp .env.example .env
docker compose up --build

Three services become reachable once it's running: JupyterLab at localhost:8888, interactive API documentation at localhost:8000/docs, and the MinIO storage console at localhost:9001. Tearing it down is symmetric — docker compose down, or docker compose down -v to also discard the storage volumes.

Five containers make up the running system, and one data-flow path connects them: a researcher's browser talks to JupyterLab, which either runs a light model directly (a plain Python import of dissmodel, exactly like every earlier chapter) or hands a heavy job off to a REST API for the platform to run instead.

Service	Role
JupyterLab	interactive development; runs light models locally, submits heavy ones
API Gateway (FastAPI)	receives job requests, validates them, enqueues work
Workers	pull jobs off the queue, run the model via `ModelExecutor` (Chapter 21, 22), write results
MinIO	S3-compatible object storage for inputs and outputs
Redis	the job queue itself (with priority support), and a metadata cache

A submitted job's actual path: input data lands in MinIO, a worker picks the job off the Redis queue, runs it through the exact load → run → save cycle Chapter 28 already built and tested by hand, and writes the result back to MinIO — the same ExperimentRecord fields from Chapter 28 populated automatically, not typed in by hand this time. Workers scale horizontally (docker compose up --scale worker=5 runs five of them in parallel); a Kubernetes-and-Dask production path is explicitly marked as a later phase, not yet built, consistent with the MVP status above.

The Executor Registry: dissmodel-configs¶

An executor being installed locally (Chapter 22) is not the same as it being available on the platform — that second step goes through a separate repository, dissmodel-configs, which exists specifically to keep model registration decoupled from both the executor packages themselves and the platform's own code. One TOML file per registered model:

# dissmodel-configs/models/brmangue_raster.toml
[model]
executor_module = "brmangue.executors"
name            = "brmangue_raster"
class           = "brmangue_raster"
description     = "BR-MANGUE raster simulation (mangrove + flood dynamics)"
package         = "git+https://github.com/DisSModel/brmangue-dissmodel@refactoring"

[model.parameters]
end_time      = 88
taxa_elevacao = 0.5
altura_mare   = 6.0
acrecao_ativa = false
resolution    = 100.0
crs           = "EPSG:31983"

Everything under [model] (besides parameters) surfaces as record.resolved_spec["model"] once a job runs; [model.parameters] becomes record.parameters — the identical field Chapter 22's run() already read initial_fire_density and seed from. Put a value in the wrong section and it doesn't error — it just silently falls back to whatever default the executor itself defines, a sharp enough edge that Exercise 3 asks you to deliberately trigger it once.

Files follow a <model>_<substrate>.toml naming convention directly — brmangue_raster.toml next to brmangue running on a GeoDataFrame elsewhere, disslucc_raster.toml next to disslucc_vector.toml — making Chapter 24's vector/raster pairing visible in the registry itself, not just in code. A richer version of the same format also supports [model.bands] / [model.columns]: a canonical vocabulary mapping a generic name like elevation onto whatever a specific dataset actually calls that column or band, resolved through band_map/column_map (fields Chapter 28's ExperimentRecord table already listed) so one executor's code runs unmodified against differently-named real-world datasets.

The platform doesn't re-read dissmodel-configs on every single request — a scheduled job pulls the repository roughly every 15 minutes and invalidates a cache when something changed, so merging a pull request there is enough to make a new model available platform-wide within that window, no redeploy required. An admin endpoint can force an immediate sync, and a separate "inline" submission path lets a researcher bypass the registry entirely for quick, exploratory runs — at a real reproducibility cost: an inline job's model_commit (Chapter 28) gets recorded as a placeholder rather than tied to an actual dissmodel-configs commit.

Usage Example: Triggering Experiments from QGIS¶

brmangue-qgis is a real, concrete client of everything above — a QGIS plugin that lets a researcher pick input layers inside QGIS itself and submit a full experiment without ever opening a terminal. Tracing its flow end to end shows what "a thin client on top of the platform" actually looks like in practice:

Configure. A dialog collects the platform's URL, an API key, an input location (an s3:// URI pointing into the MinIO bucket from Installation and Architecture), model parameters, and which output bands to load back afterward.
Submit. The plugin assembles exactly the JSON payload the API's job-submission endpoint expects — model name, input reference, parameters, an optional band mapping — the same shape The Executor Registry section's TOML file resolves on the server side.
Poll, off the UI thread. Submitting returns a job ID immediately; the plugin then polls a status endpoint every few seconds, in a background task, until the job reports completed or failed, surfacing the platform's own last log line if it failed.
Load and style automatically. Once complete, the plugin opens each output band as its own QGIS raster layer and applies sensible default styling per band — a categorical palette for a land-use band, a continuous gray-scale stretch for an elevation band — so nobody hand-builds a .qml style file for output they've never seen before.
Inspect provenance, from inside QGIS. The same fields Chapter 28 built ExperimentRecord around — model_commit, code_version, output_sha256 — are captured by the plugin and made available for inspection, so a researcher never has to leave the mapping tool to answer "which exact code produced this layer."

Nothing about the science runs on the researcher's own machine at any point in that flow — no local dissmodel, no local brmangue-dissmodel install required at all. That's the real point of the platform: the API contract from Installation and Architecture, not any particular client sitting on top of it, is what other tools are actually built against. A QGIS plugin is one such client; nothing about the contract prevents a second one, in a completely different tool, from talking to the identical API.

Exercises¶

Compare two clients. Bring the platform up locally and submit brmangue_raster directly with curl against the API, following the JSON shape payload_builder-style code in Usage Example implies. If you also have QGIS and the plugin available, submit the same job through it and compare the two JSON payloads — they should be structurally identical, since both target the same submission endpoint.
A missing API key. Every platform request besides a basic health check requires a valid API key. Submit a request with a deliberately wrong key and confirm what HTTP status comes back. Where in .env would a second researcher's key be added without invalidating the first?
A misplaced TOML field. Following brmangue_raster.toml's structure, write a new registry file for a model from an earlier chapter, but deliberately place one parameter under [model] instead of [model.parameters]. What value does that parameter end up with when the job actually runs, and why?
Why model_commit matters here specifically. The inline submission path records model_commit as a placeholder rather than a real commit hash. Using Chapter 28's ExperimentRecord field table, explain in your own words exactly what becomes unverifiable about a result submitted that way.
Trace the failure path. Based on Usage Example's description of the polling loop, sketch — in words — what you'd expect to happen if the platform returned a transient network error partway through polling, rather than a clean "failed" status. Would you want the plugin to give up immediately, or retry? Why?

Summary¶

Key concepts introduced¶

The DisSModel Platform as a shared, multi-user wrapper around the exact ModelExecutor contract Chapters 21 and 22 already built and tested by hand — no model code changes required to go from local to shared
Five services (JupyterLab, API Gateway, Workers, MinIO, Redis) and the one data-flow path connecting them, from a researcher's browser to a completed, storable result
dissmodel-configs: one TOML file per <model>_<substrate> combination, decoupling which models are available on the platform from both the platform's own code and the executor packages themselves
Band/column mapping as a canonical vocabulary, letting one executor run unmodified against differently-named real datasets
brmangue-qgis as a concrete, complete example of a thin client: submit, poll, render, inspect provenance — never running the science locally at all

Chapter 30 stays in this same infrastructure territory but shifts from running models to managing the data they consume and produce — spatial data cubes, and the disscube package built around them.