Semantic Validation of Data Schemas with the Semantic Schema Validator

Before processing a dataset, it is essential to check that its content truly conforms to the structure it declares. The Semantic Schema Validator is an algorithm designed to verify the structural and semantic integrity of datasets, applying formal schema validation and returning a JSON report.

How it works

1. Format detection: the algorithm inspects the content directly to determine the true format (text/csv, application/json, application/ld+json, text/turtle, application/n-triples, etc.).
2. Metadata extraction: it automatically retrieves the validation schema (dct:conformsTo) and the format (dct:format) from the dataset's own DDO metadata, also allowing the user to explicitly override them via input parameters if these are incorrect or unspecified.
3. Execution: the rdflib engine evaluates the data against the schema rules, applying SHACL for RDF graphs (JSON-LD, Turtle, N-Triples), JSON Schema for JSON data, XSD for XML, or CSVW for tabular data.
4. Report generation: a detailed JSON report (<did>_report.json) is generated and saved to the compute job's output folder.

Input Parameters

When triggering a compute job, users can configure the following parameters. If omitted, the algorithm safely falls back to the dataset's metadata.

Parameter	Required	Description
format	No	Expected data format. Can be left as - so the algorithm relies on its internal detection or the dataset's DDO metadata.
conformsTo	Yes, if the dataset does not include it in its metadata	Publicly accessible URL to the validation schema (.json, .ttl, or .xsd).

Supported Formats and Validation Engines

Data Format	Validation Engine
CSV	CSVW
JSON	JSON Schema
JSON-LD	SHACL
Turtle	SHACL
N-Triples	SHACL
XML	XSD

Error Handling

Rather than interrupting the compute job, the algorithm intercepts missing schemas or format mismatches and generates standardized JSON error reports.

Example Result

When the data conforms to the provided schema, the algorithm returns a COMPLETED status with the validation engine's results:

{
  "dataset": "did:op:12345...",
  "status": "COMPLETED",
  "detected_format": "application/json",
  "schema_url": "https://raw.githubusercontent.com/.../schema.json",
  "validation_results": {
    "conforms": true,
    "details": "Validation passed successfully."
  }
}

Demonstration Datasets

Seven test datasets are currently available on the portal to run against the Semantic Schema Validator algorithm, covering the text/csv, application/json, application/ld+json, text/turtle, and application/n-triples formats:

• CSV
• JSON
• JSON-LD
• Turtle
• XML
• N-Triples
• JSON without metadata

Six of the seven datasets already include the validation schema in their metadata, so they can be run directly without any input parameters. The JSON without metadata dataset was specifically designed to test the algorithm when the dct:conformsTo field is absent: in this case, the schema URL must be provided manually as the conformsTo parameter:

https://raw.githubusercontent.com/AgrospAI/ocean-algo/refs/heads/v2-ocean-runner/data-schema-validator/_data/test_inputs/schemas/shapes.jsonld

There is no need to specify the format parameter: the algorithm detects the file type from its content even without an extension, and ignores any format value that does not match the one detected.