Building a Compute-to-Data Algorithm

A practical guide to packaging your algorithms and running them on federated data in Ocean Protocol / Pontus-X, without the data ever leaving its origin. We use a real-world example as a common thread: clip-raster, an algorithm that clips Sentinel-2 images over a cadastral parcel and calculates vegetation indices.

Technologies we will use:

• Python + Docker
• The ocean-runner framework
• Reference example: clip-raster

Prefer presentation format?

This same content is available as a slide presentation (navigable with arrows ← →).

The Environment Contract: Everything Revolves Around /data

When the algorithm starts, the orchestrator mounts a folder structure under /data. It is the only point of contact with the outside world.

Directory	Purpose
/data/inputs	Execution parameters (algoCustomData.json) and the files of each dataset, in subfolders with the dataset's identifier (DID).
/data/ddos	Metadata (DDOs) of the contracted datasets. They describe the files available for computation.
/data/transformations	Algorithm secrets and credentials (e.g., username/password for an external service).
/data/outputs	The only actual write directory. Here you leave the result that will be delivered to the user.
/data/logs	Log output for debugging execution.

Golden Rule

Read from input folders (read-only), work in memory or in /tmp, and write only to outputs.

Accessing dataset files

The files to be processed are mounted in /data/inputs/<did>/ — one subfolder per dataset, named with its identifier (DID). There's no need to manually traverse the disk: job_details.files gives you the files for each DID, and job_details.inputs() iterates (did, path) pairs ready for use. In clip-raster, only input parameters are used — data is obtained from external services — but this is how a generic algorithm would consume the contracted dataset.

Anatomy of an Algorithm Project

Maintain a clear separation between the algorithm code, packaging, and test data.

• algorithm/src/ — your Python code, divided into modules.
• Dockerfile — how the image is built.
• docker-compose.yaml — for local testing.
• _data/ — simulated inputs/outputs for development.

clip-raster/

clip-raster/
├── algorithm/
│   ├── src/
│   │   ├── algorithm.py   # orchestrator (run/save)
│   │   ├── data.py        # input model (Pydantic)
│   │   ├── auth.py        # external credentials
│   │   ├── download.py    # data retrieval
│   │   ├── raster.py      # domain logic
│   │   └── indices.py     # calculations
│   ├── pyproject.toml     # dependencies
│   └── uv.lock
├── _data/                 # local test data
│   ├── inputs/algoCustomData.json
│   ├── transformations/algorithm
│   └── outputs/
├── Dockerfile
└── docker-compose.yaml

Key Idea

This same template works for any algorithm in your organization: only the content of src/ changes.

The Framework: ocean-runner, a 4-Phase Pipeline

You don't write the execution loop manually. You register functions by phases with decorators — in the style of FastAPI — and the runner orchestrates them.

• @validate — checks inputs and datasets before spending computation.
• @run — the core: your business logic produces a result.
• @save_results — serializes the result to /data/outputs.
• @on_error — captures failures in a controlled manner.

Why it matters

Separating into phases makes your algorithm readable, testable, and predictable. The runner injects the job context (job_details) and manages the lifecycle for you.

The Minimum Skeleton of Any Algorithm

Every algorithm starts the same way: an Algorithm instance is created with its Config, and the phases are attached.

• Algorithm[Input, Result] — generic typing: input and result type.
• Config(custom_input=...) — declares your parameter model.
• The algorithm object must be accessible from src.algorithm — this is what ocean-execute looks for.

src/algorithm.py

from pathlib import Path
from ocean_runner import Algorithm, Config
from .data import InputParameters

# Type of result your algorithm produces
type ResultsT = dict[str, ...]

algorithm = Algorithm[InputParameters, ResultsT].create(
    Config(custom_input=InputParameters)
)

@algorithm.run
def run(_) -> ResultsT:
    params = algorithm.job_details.input_parameters
    ...                     # your logic
    return result

@algorithm.save_results
def save(_, result: ResultsT, base: Path):
    ...                     # writes to `base` (= /data/outputs)

Convention

Always start here and fill in each phase. The default run raises an error until you register your own.

Phase 0 · Declare and Validate Parameters with Pydantic

The user sends parameters as JSON. Convert them into a typed model: this way you get automatic validation and clear error messages before executing anything.

• Each field is an attribute with its type.
• With @field_validator you add business rules.
• An invalid input fails early, without spending computation or downloads.

src/data.py

from typing import Any
from pydantic import BaseModel, field_validator

class InputParameters(BaseModel):
    refcat: str

    @field_validator("refcat", mode="before")
    @classmethod
    def is_valid_refcat(cls, v: Any) -> str | None:
        if v is None:
            return v
        stripped = str(v).strip()
        if stripped and not stripped.isalnum():
            raise ValueError(f"'{stripped}' must be alpha-numeric")
        return stripped

_data/inputs/algoCustomData.json

{ "refcat": "25168A010000060000IZ" }

In clip-raster

The only parameter is refcat, a cadastral reference. The validator requires it to be alphanumeric.

Phase 1 · Validation (optional): Fail Early, Fail Cheap

Before executing the heavy logic, the runner calls the validation phase. By default, it checks that the dataset's metadata and files exist.

You can override it with @algorithm.validate to add your own preconditions: are the credentials present?, is there a connection to the external service?, does the dataset have the expected format?

src/algorithm.py (optional)

@algorithm.validate
def validate(_) -> None:
    jd = algorithm.job_details

    # Did the dataset metadata arrive?
    assert jd.metadata, "Missing dataset DDOs"

    # Are there files to process?
    assert jd.files, "Dataset contains no files"

    # Your own business preconditions...
    algorithm.logger.info("Input validated successfully")

Best practice

Every validation you add here is a failure you save yourself halfway through the computation —and a clear message for the user—. If you don't register this phase, the runner applies a reasonable default validation. In clip-raster, data is obtained from external services, so the key validation lives within the run itself.

Phase 2 · The run Orchestrates; Modules Do the Work

The run function should be read as an algorithm summary: a few lines that tell the story. The details live in specialized modules.

• Reads parameters from job_details.input_parameters.
• Delegates each step to a single-purpose function.
• Returns an in-memory result; does not write files here.

src/algorithm.py

import geopandas as gpd
from .download import download_product

@algorithm.run
def run(_) -> ResultsT:
    params = algorithm.job_details.input_parameters
    refcat = params.refcat

    # ① parcel geometry (Cadastre service)
    url = f"{CATASTRO_URL_PREFIX}{refcat}"
    gdf = gpd.read_file(url)
    geometry = gdf.geometry.item()

    # ②③ search and download Sentinel-2 product
    download_product(geometry)

    # ④⑤⑥ clip bands and compute indices
    return compute_indices(geometry)

The 6 stages of clip-raster

① resolve geometry → ② search product → ③ download bands → ④ clip → ⑤ calculate indices → ⑥ return result.

Credentials and Errors: Secrets Outside Code, Errors Under Control

Secrets. Never embed credentials in the code or image. The runner exposes them in /data/transformations/algorithm as key/value pairs that you read at runtime. In clip-raster, the Copernicus username and password are read from this file to request a token and temporary S3 credentials.

Errors. Raise Algorithm.Error for expected failures. The runner captures it in the on_error phase and terminates the job cleanly, with a useful message in the logs.

src/auth.py

from dotenv import get_key
from ocean_runner import Algorithm

def get_copernicus_token() -> str:
    user = get_key('/data/transformations/algorithm', 'username')
    pwd  = get_key('/data/transformations/algorithm', 'password')
    if not user or not pwd:
        raise Algorithm.Error('Missing credentials')
    return request_copernicus_token(user, pwd)

Phase 3 · The Result Materializes in /data/outputs

The save_results phase receives the result from run and the base output path. It is the only place where you write files that the user will receive.

• Receives (_, result, base) — base already points to /data/outputs.
• Choose a clear format: image, CSV, JSON, GeoTIFF…
• Maintain separation: run computes, save persists.

src/algorithm.py

from pathlib import Path
from .raster import save_as_img

@algorithm.save_results
def save(_, result: ResultsT, base: Path):
    out_path = base / 'indices.png'   # /data/outputs/indices.png
    save_as_img(result, out_path)

In clip-raster

The four indices are plotted on a 2×2 grid and saved as indices.png. Why separate run and save? Because it allows you to change the output format without touching the logic, and test the computation without writing to disk.

Packaging · Dockerfile: The Image That Will Be Executed

The algorithm is delivered as a Docker image. Use a multistage build: one stage installs dependencies and another, lightweight one, executes.

• Installs dependencies with uv and uv.lock (reproducible).
• Adds system libraries you need (here, GDAL for geospatial data).
• The entrypoint is always ocean-execute --base-dir /data.

Dockerfile

# Stage 1 — build the environment with dependencies
FROM python:3.12 AS builder
COPY --from=ghcr.io/astral-sh/uv:0.5.11 /uv /uvx /bin/
ENV PATH="/algorithm/.venv/bin:$PATH"
WORKDIR /algorithm/
RUN --mount=type=bind,source=algorithm/uv.lock,target=uv.lock \
    --mount=type=bind,source=algorithm/pyproject.toml,target=pyproject.toml \
    uv sync --frozen --no-install-project

# Stage 2 — final, lightweight image
FROM python:3.12-slim
RUN apt-get update && apt-get install -y \
    libgdal-dev gdal-bin libspatialindex-dev \
    && rm -rf /var/lib/apt/lists/*
WORKDIR /algorithm
COPY --from=builder /algorithm/.venv /algorithm/.venv
COPY algorithm/src /algorithm/src
ENV PATH="/algorithm/.venv/bin:$PATH"
ENV PYTHONPATH="/algorithm"

CMD ["ocean-execute", "--base-dir", "/data"]

Important

The ocean-execute command locates your algorithm object in src.algorithm and triggers the pipeline. You don't need a main().

Local Testing: Simulate the Ocean Environment with docker-compose

Before publishing, test the image exactly as it will run on the portal. The trick is to mount your _data/ folder to the paths the algorithm expects.

• Inputs are mounted read-only (:ro).
• outputs and logs in read/write (:rw).
• Mounting src/ allows iteration without rebuilding the image.

docker-compose.yaml

services:
  algorithm:
    build: .
    entrypoint: 'ocean-execute --base-dir /data'
    volumes:
      - ./_data/inputs:/data/inputs:ro
      - ./_data/ddos:/data/ddos:ro
      - ./_data/transformations:/data/transformations:ro
      - ./_data/outputs:/data/outputs:rw
      - ./_data/logs:/data/logs:rw
      - ./algorithm/src:/algorithm/src/   # iterate without rebuild

terminal

docker compose up --build   # first time
docker compose up           # changes only in src/

Result

After docker compose up, the file appears in _data/outputs/ —just as the end user would receive it—.

In Summary

• Define the structure algorithm/src + Dockerfile + _data.
• Model and validate inputs with Pydantic.
• Register the phases: validate → run → save_results.
• Keep run as a readable orchestrator.
• Separate domain (pure) from plumbing (network/disk).
• Read secrets from /data/transformations.
• Raise Algorithm.Error on expected failures.
• Write the result only to /data/outputs.
• Package with Docker multistage + ocean-execute.
• Test the image locally with docker-compose.

Next step and resources

Once your algorithm is built and tested locally, continue with Publishing a Compute-to-Data Algorithm to bring it to the portal.

Reference code clip-raster: github.com/AgrospAI/ocean-algo. Questions about the framework or publishing? The AgrospAI team will support you with the integration.