· Tutorial: building a Compute-to-Data algorithm
Step-by-step tutorial

How to build a Compute-to-Data algorithm for the AgrospAI data space

A practical guide to packaging your algorithms and running them over federated data on Ocean Protocol / Pontus-X — without the data ever leaving its source.

Python + Docker ocean-runner framework Real example: clip-raster Sentinel-2 · vegetation indices

How to navigate
Scroll horizontally, use the arrow keys or the buttons below. There are 13 slides.

02 / 13
The environment contract

Everything revolves around the /data directory

When the algorithm starts, the orchestrator mounts this folder structure. It is the only point of contact with the outside world.

📥 /data/inputs

Execution parameters (algoCustomData.json) and the files of each dataset, in subfolders named after the dataset identifier (DID).

🗂️ /data/ddos

Metadata (DDOs) of the contracted datasets. They describe the files available for the computation.

🔐 /data/transformations

Secrets and credentials for the algorithm (e.g. username/password for an external service).

📤 /data/outputs

The only directory you really write to. Here you leave the result that will be delivered to the user.

📝 /data/logs

Log output for debugging the execution.

💡 Golden rule

Read from the input folders (read-only), work in memory or in /tmp, and write only to outputs.

Accessing the dataset files
The files to process are mounted under /data/inputs/<did>/ — one subfolder per dataset, named with its identifier (DID). No need to walk the disk by hand: job_details.files gives you the files for each DID and job_details.inputs() iterates (did, path) pairs ready to use. In clip-raster only input parameters are used —the data comes from external services— but this is how a generic algorithm would consume the contracted dataset.

03 / 13
Repository structure

Anatomy of an algorithm project

Keep a clear separation between the algorithm code, the packaging and the test data.

  • algorithm/src/ — your Python code, split into modules.
  • Dockerfile — how the image is built.
  • docker-compose.yaml — for local testing.
  • _data/ — simulated inputs/outputs for development.

Key idea
This same template works for any algorithm in your organization: only the contents of src/ change.

clip-raster/
clip-raster/
├── algorithm/
│   ├── src/
│   │   ├── algorithm.py   # orchestrator (run/save)
│   │   ├── data.py        # input model (Pydantic)
│   │   ├── auth.py        # external credentials
│   │   ├── download.py    # data retrieval
│   │   ├── raster.py      # domain logic
│   │   └── indices.py     # computations
│   ├── pyproject.toml     # dependencies
│   └── uv.lock
├── _data/                 # local test data
│   ├── inputs/algoCustomData.json
│   ├── transformations/algorithm
│   └── outputs/
├── Dockerfile
└── docker-compose.yaml
04 / 13
The framework

ocean-runner: a 4-phase pipeline

You don't write the execution loop by hand. You register functions per phase with decorators — FastAPI style — and the runner orchestrates them.

@validate
Validate
Checks inputs and datasets before spending compute.
@run
Run
The core: your business logic produces a result.
@save_results
Save
Serializes the result to /data/outputs.
@on_error
Errors
Catches failures in a controlled way.

Why it matters
Splitting into phases makes your algorithm readable, testable and predictable. The runner injects the job context (job_details) and manages the lifecycle for you.

05 / 13
The skeleton

The minimal skeleton of any algorithm

Every algorithm starts the same way: you create an Algorithm instance with its Config, and hang the phases on it.

  • Algorithm[Input, Result] — generic typing: input and result type.
  • Config(custom_input=...) — declares your parameter model.
  • The algorithm object must be accessible from src.algorithm — that is what ocean-execute looks for.

Convention
Always start here and fill in each phase. The default run raises an error until you register your own.

src/algorithm.py
from pathlib import Path
from ocean_runner import Algorithm, Config
from .data import InputParameters

# Result type your algorithm produces
type ResultsT = dict[str, ...]

algorithm = Algorithm[InputParameters, ResultsT].create(
    Config(custom_input=InputParameters)
)

@algorithm.run
def run(_) -> ResultsT:
    params = algorithm.job_details.input_parameters
    ...                     # your logic
    return result

@algorithm.save_results
def save(_, result: ResultsT, base: Path):
    ...                     # write to `base` (= /data/outputs)
06 / 13
Phase 0 · Define inputs

Declare and validate parameters with Pydantic

The user sends parameters as JSON. Turn them into a typed model: that gives you automatic validation and clear error messages before running anything.

  • Each field is an attribute with its type.
  • With @field_validator you add business rules.
  • An invalid input fails fast, without spending compute or downloads.

In clip-raster
The only parameter is refcat, a cadastral reference. The validator requires it to be alphanumeric.

src/data.py
from typing import Any
from pydantic import BaseModel, field_validator

class InputParameters(BaseModel):
    refcat: str

    @field_validator("refcat", mode="before")
    @classmethod
    def is_valid_refcat(cls, v: Any) -> str | None:
        if v is None:
            return v
        stripped = str(v).strip()
        if stripped and not stripped.isalnum():
            raise ValueError(f"'{stripped}' must be alpha-numeric")
        return stripped
_data/inputs/algoCustomData.json
{ "refcat": "25168A010000060000IZ" }
07 / 13
Phase 1 · Validation · Optional

Validation (optional): fail fast, fail cheap

Before running the heavy logic, the runner calls the validation phase. By default it checks that the dataset metadata and files exist.

You can override it with @algorithm.validate to add your own preconditions: are the credentials present? is there a connection to the external service? does the dataset have the expected format?

Good practice
Every validation you add here is a failure you save mid-computation — and a clear message for the user.

src/algorithm.py (opcional)
@algorithm.validate
def validate(_) -> None:
    jd = algorithm.job_details

    # Did the dataset metadata arrive?
    assert jd.metadata, "Missing the dataset DDOs"

    # Are there files to process?
    assert jd.files, "The dataset contains no files"

    # Your own business preconditions...
    algorithm.logger.info("Input validated successfully")

If you don't register this phase, the runner applies a reasonable default validation. In clip-raster the data comes from external services, so the key validation lives inside run itself.

08 / 13
Phase 2 · Run

run orchestrates; the modules do the work

The run function should read like a summary of the algorithm: a few lines that tell the story. The detail lives in specialized modules.

  • Read parameters from job_details.input_parameters.
  • Delegate each step to a single-purpose function.
  • Return an in-memory result; don't write files here.

The 6 stages of clip-raster
① resolve geometry → ② find product → ③ download bands → ④ clip → ⑤ compute indices → ⑥ return result.

src/algorithm.py
import geopandas as gpd
from .download import download_product

@algorithm.run
def run(_) -> ResultsT:
    params = algorithm.job_details.input_parameters
    refcat = params.refcat

    # ① parcel geometry (Cadastre service)
    url = f"{CATASTRO_URL_PREFIX}{refcat}"
    gdf = gpd.read_file(url)
    geometry = gdf.geometry.item()

    # ②③ search and download the Sentinel-2 product
    download_product(geometry)

    # ④⑤⑥ clip bands and compute indices
    return compute_indices(geometry)
09 / 13
Credentials and errors

Secrets outside the code; errors under control

🔐 Secrets

Never embed credentials in the code or the image. The runner exposes them in /data/transformations/algorithm as key/value pairs you read at runtime.

In clip-raster
The Copernicus username and password are read from that file to request a token and temporary S3 credentials.

🧯 Errors

Raise Algorithm.Error for expected failures. The runner catches it in the on_error phase and ends the job cleanly, with a useful message in the logs.

src/auth.py
from dotenv import get_key
from ocean_runner import Algorithm

def get_copernicus_token() -> str:
    user = get_key('/data/transformations/algorithm', 'username')
    pwd  = get_key('/data/transformations/algorithm', 'password')
    if not user or not pwd:
        raise Algorithm.Error('Missing credentials')
    return request_copernicus_token(user, pwd)
10 / 13
Phase 3 · Save results

The result is materialized in /data/outputs

The save_results phase receives the result from run and the output base path. It is the only place where you write files the user will receive.

  • Receives (_, result, base)base already points to /data/outputs.
  • Choose a clear format: image, CSV, JSON, GeoTIFF…
  • Keep the separation: run computes, save persists.

In clip-raster
The four indices are drawn in a 2×2 grid and saved as indices.png.

src/algorithm.py
from pathlib import Path
from .raster import save_as_img

@algorithm.save_results
def save(_, result: ResultsT, base: Path):
    out_path = base / 'indices.png'   # /data/outputs/indices.png
    save_as_img(result, out_path)

Why separate run and save? Because it lets you change the output format without touching the logic, and test the computation without writing to disk.

11 / 13
Packaging

Dockerfile: the image that will run

The algorithm is delivered as a Docker image. Use a multistage build: one stage installs dependencies and another, lightweight, runs.

  • Install dependencies with uv and the uv.lock (reproducible).
  • Add any system libraries you need (here, GDAL for geodata).
  • The entrypoint is always ocean-execute --base-dir /data.

Important
The ocean-execute command locates your algorithm object in src.algorithm and triggers the pipeline. You don't need a main().

Dockerfile
# Stage 1 — build the environment with dependencies
FROM python:3.12 AS builder
COPY --from=ghcr.io/astral-sh/uv:0.5.11 /uv /uvx /bin/
ENV PATH="/algorithm/.venv/bin:$PATH"
WORKDIR /algorithm/
RUN --mount=type=bind,source=algorithm/uv.lock,target=uv.lock \
    --mount=type=bind,source=algorithm/pyproject.toml,target=pyproject.toml \
    uv sync --frozen --no-install-project

# Stage 2 — final, lightweight image
FROM python:3.12-slim
RUN apt-get update && apt-get install -y \
    libgdal-dev gdal-bin libspatialindex-dev \
    && rm -rf /var/lib/apt/lists/*
WORKDIR /algorithm
COPY --from=builder /algorithm/.venv /algorithm/.venv
COPY algorithm/src /algorithm/src
ENV PATH="/algorithm/.venv/bin:$PATH"
ENV PYTHONPATH="/algorithm"

CMD ["ocean-execute", "--base-dir", "/data"]
12 / 13
Test locally

Simulate the Ocean environment with docker-compose

Before publishing, test the image exactly as it will run on the portal. The trick is to mount your _data/ folder at the paths the algorithm expects.

  • Inputs are mounted read-only (:ro).
  • outputs and logs read/write (:rw).
  • Mounting src/ lets you iterate without rebuilding the image.

Result
After docker compose up, the file appears in _data/outputs/ — just as the end user would receive it.

docker-compose.yaml
services:
  algorithm:
    build: .
    entrypoint: 'ocean-execute --base-dir /data'
    volumes:
      - ./_data/inputs:/data/inputs:ro
      - ./_data/ddos:/data/ddos:ro
      - ./_data/transformations:/data/transformations:ro
      - ./_data/outputs:/data/outputs:rw
      - ./_data/logs:/data/logs:rw
      - ./algorithm/src:/algorithm/src/   # iterate without rebuild
terminal
docker compose up --build   # first time
docker compose up           # changes only in src/
Summary

Checklist to build your algorithm

  • Define the structure algorithm/src + Dockerfile + _data.
  • Model and validate inputs with Pydantic.
  • Register the phases: validate → run → save_results.
  • Keep run as a readable orchestrator.
  • Separate domain (pure) from plumbing (network/disk).
  • Read secrets from /data/transformations.
  • Raise Algorithm.Error on expected failures.
  • Write the result only to /data/outputs.
  • Package with multistage Docker + ocean-execute.
  • Test the image locally with docker-compose.

Resources
Reference code clip-raster: github.com/AgrospAI/ocean-algo
Questions about the framework or publishing? The AgrospAI team supports you through the integration.

AgrospAI · Universitat de Lleida · Compute-to-Data data space

to navigate