Yoda Metadata

Owned by Berend Dijk, van
Last updated: 30/03/2023
2 min read

Overview

Yoda supports adding metadata to collections. This facilitates archiving and (especially) publishing a collection of data as a data package. Each community or group has its own metadata schema, so different departments or faculties can optionally adapt the metadata format to their field.

This part of the documentation is about metadata in the context of Yoda data packages. iRODS metadata (AVUs) have a role in Yoda metadata, but can also be used for other purposes that are unrelated to Yoda data packages. Data request module metadata is also not in the scope of this page.

Yoda metadata is primarily stored in data objects named yoda-metadata.json.

Metadata schemas

In Yoda, a metadata schema is used to validate the metadata of a data package. The schema also contains information regarding how the metadata should be displayed in the web portal.

A Yoda metadata schema has two parts:

  1. A JSON schema file is used to validate the metadata of a data package.

  2. A UI schema file that contains information for the web portal regarding how to display a metadata form for this schema, optionally including placeholders and help text for each field.

Yoda environments have a default metadata schema. Additionally, specific metadata schemas can be configured for communities (also known as categories). As of Yoda 1.9, it is also possible to configure a metadata schema on a group level.

For more information, see:

Schema names and identifiers

Each schema has a globally unique identifier (URL) that is based on its name. For example, the default-2 schema is identified as https://leiden-yoda.irods.surfsara.nl/schemas/default-2/metadata.json.

Exporting of metadata during publication

When a data package is published, its metadata will be exported in three ways:

Landing page

The landing page HTML code is generated using a Jinja2 template. This template may need to be adapted in case of metadata schema changes. The template can be found in /etc/irods/yoda-ruleset/templates/landingpage.html.j2. The Ansible play book activates this template by uploading it to the /zone/yoda/templates/ collection.

JSON file for DataCite

A JSON file conforming to the DataCite v4 Schema is generated. The implementation can be found in /etc/irods/yoda-ruleset/json_datacite41.py.

OAI-PMH data

The metadata will be copied to the OAI-PMH server (public server). Once there, it will be processed by MOAI so that it can be harvested using the OAI-PMH protocol by external systems. Only a subset of fields that map to Dublin core and DataCite are processed by MOAI. Any changes to these fields will result in the metadata being omitted from the stream.

Metadata transformation

As a general rule, metadata will have to comply with the latest version of a schema. For example, if a new version of Yoda has a new default-2 schema, all data packages using the default-1 schema will need to be updated to default-2..

Vault metadata is upgraded as a part of the upgrade process, as described in the release notes. Research metadata is updated when a user opens it.

The transformation logic for updating to a newer version of a schema is defined in /etc/irods/yoda-ruleset/schema_transformations.py.