Skip to main content

3. Publishing assets in AgrospAI

Guide on how to publish data and services in AgrospAI.

Publishing a dataset​

Adding a dataset to the data space means registering its metadata so that they appear in the marketplace so that they can be acquired and consumed. Only metadata is registered in the infrastructure of the data space, the data remains under the control of the data holder. The publication can be carried out interactively from the data space market by selecting the "Publish" menu option above, as shown in Fig. 5. The publication form first collects the necessary metadata for the dataset, including the title, description, labels... Then, the publisher must also provide how the dataset is made accessible, via "download" or "compute".

In both cases, a "data provider" will mediate to enforce the access rules. The default provider can be used, although the publisher is free to define a custom one. The additional information for access includes where the data is available (usually the URL from where it can be downloaded and which will be kept private by the "Data Provider"), the timeout for consumption or a link to a sample file that helps potential consumers get an idea about the data. The next step in the form is to set the price of the dataset, which can be offered for free or for a price in EUROe, a trial version of a digital currency linked to the euro. This is the recommended currency so that prices are more transparent for consumers.

Finally, it is possible to review how the asset will be presented in the marketplace before confirming the publication by signing the corresponding transactions that will keep an immutable record of the publication and link the dataset to the account that signs transactions.

Fig. 5. Dataset publication form (source: https://portal.agrospai.udl.cat)

Alternatively, it is also possible to publish a dataset from code using the TypeScript library Nautilus or from the command line using the "publish" command of the Pontus-X CLI command line tool.

Publishing an algorithm​

The AgrospAI data space supports the publication of datasets and algorithms. The latter are data processing services that are packaged as Docker containers and scripts that will run in the context of those containers. Publication is activated interactively in the same way that datasets are published, via the "Publish" menu bar option. In the associated form, it is necessary to select "Algorithm" as the asset type instead of "Dataset".

In addition, in addition to the metadata such as the title, description or labels, it is also necessary to define the Docker image to be used for the algorithm. As for the access details, a pointer to the location of the script that triggers the execution of the algorithm in the context of the container must be provided. More details on how to package and how to run the algorithm in the data space are provided in "Writing Algorithms" of Ocean Protocol. In addition, there are examples in the ocean-algo repository.

Algorithms can be available for "Download" or only for "Compute" processing. In the first case, the script used by the algorithm will be available for download once acquired. In both cases, the algorithm can be consumed together with a dataset configured for process-to-data consumption. As with datasets, it is also possible to define the consumption deadline after acquisition or the price of the algorithm. Finally, after reviewing the algorithm metadata, the publication is confirmed by signing the corresponding transactions.

Alternatively, it is also possible to publish a dataset from code using the library in TypeScript language Nautilus or from the command line using the "publish" command of the Pontus-X CLI.

Access and Use Policies​

The AgrospAI data space implements the set of access and use policies implemented in the Pontus-X ecosystem through smart contracts and distributed ledger technology. These policies can be defined through the market user interface during the publication of resources. In addition, they can be configured programmatically using Nautilus or Pontus-X CLI.

For example, as shown in Fig. 6, it is possible to define the timeout that anyone who acquires an asset (dataset or algorithm) has before it needs to be reacquired. It ranges from 1 day to 1 year. In addition, the right of consumption cannot expire and can be forever after the acquisition.

Fig. 6. Definition of asset consumption timeout during publication (source: https://portal.agrospai.udl.cat)

Additional directive definition instruments during the publication of assets are the lists of allowed and denied accounts, as shown in Fig. 7. The allowed list defines which data space participants can explicitly consume the resource (dataset or algorithm). If the list is empty, anyone can download or calculate this resource. On the other hand, if an address is on the denial list, the download or calculation of this resource will be denied for the corresponding data set participant.

Fig. 7. Definition of allowed and denied accounts for asset consumption during publication (source: https://portal.agrospai.udl.cat)

Finally, it is also possible to edit an existing dataset configured for "compute" consumption to define the individual algorithms that can be executed on the dataset, as shown in Fig. 8. In addition, the algorithms allowed for a dataset can also be customized by the dataset owner using the "edit-trusted-algos" command from the Pontus-X CLI tool.

Fig. 8. Definition of algorithms allowed for a dataset for compute consumption (source: https://portal.agrospai.udl.cat)