Plugins

Each research center has a different IT infrastructure and works with different frameworks. The aim of the model data explorer is to integrate these different systems into a user-friendly web-application. In order to be able to deploy the model data explorer at the different research centers, we aim for a modular structure where each component can be added or removed depending on the setup at the hosting institution.

This modular structure is implemented via plugins. Each plugin is a python package containing a django app that can be added or removed from the configuration. We distinguish between three plugin categories: core, framework and service plugins.

Creating plugins

To facilitate the creation of MDE-Apps, we have a cookiecutter-template that can be used to generate the basic structure of an MDE-App. The template is accessible at https://codebase.helmholtz.cloud/model-data-explorer/mde-app-template.

Core plugins

These plugins are essential to run the model data explorer as they define central models that each plugin depend on. Currently we only aim for one core plugin: the mde-core plugin (see Models).

Framework plugins

These plugins define optionally but commonly used features for other plugins. These platforms should be independent of the IT infrastructure and run everywhere. We currently aim for three different framework plugins that can be used by other plugins:

mde-config

The configuration plugin defines a viewset-based structure (analog to the django.admin package) where service plugins can provide configuration interfaces for individual datasets. mde-config additionally provides user configuration interfaces for the mde-core models.

mde-federation

The federation plugin to combine multiple MDE instances at different research centers. Admins can configure a federation between two instances to synchronize data groups and datasets. Furthermore, users can connect from one instance to the other using the mde-oauth plugin to synchronize the permissions between the two instances.

mde-notifications

The notification plugin implements the functionalities to send out notifications to the users. Notifications are usually send out when we need confirmations by dataset or datagroup owners. The mde-notifications app additionally provides an interface to the user for configuring notifications.

This plugin can also be used by service plugins to send out notifications.

mde-oauth

The OAuth plugin implements the possibility for the model data explorer to serve as a OAuth or SAML Provider. This gives service plugins the possibility to authenticate against the model data explorer and get the groups for the user.

Service plugins

Service plugins connect a dataset of the mde-core to an external service. This way, the model data explorer is configuring this external service. Service plugins should be independent of other service plugins (in possible) but they can depend on the above mentioned Framework plugins.

Service plugins can be based on a specific IT infrastructure. Currently we aim for the following setup at the Helmholtz-Zentrum Hereon.

Embedded

Image

These plugins can, however, be used at other research centers. The service plugins that we are developping are:

mde-viewer

A map-based viewer for datasets. This viewer provides a map-interface using ArcGIS4Javascript and static, angular-based frontend to display all metadata and services for an individual dataset. Services that are displayed in this viewer are configured through the mde-esri, mde-thredds, mde-analysis and mde-download plugins.

The layout for this viewer is designed in the FIGMA prototype accessible at https://www.figma.com/proto/FoCucLiWsUTfEQ6Fnob19s/Model-Data-Explorer-Portal.

mde-thredds

A configuration interface for a THREDDS-Server. This app expects that the configuration files of a THREDDS-Server are mounted on the server that runs the model data explorer web frontend. The mde-thredds app provides a convenient web interface to write the XML-files for a THREDDS catalog and, and it can restart the THREDDS server when the configuration changed via a call to the TOMCAT manager.

mde-csw

The metadata publisher for the model data explorer.The mde-csw plugin can combine multiple ISO-files (e.g. generated via THREDDS) and provides an OGC-compliant CSW endpoint via pycsw

Todo

Think more about mde-csw

mde-mams

The data storage for the model data explorer. At Hereon, we host an instance of the Media Assets Management System, also called MAMS. MAMS is also deployed at other research centers, namely Geomar and AWI. Through mde-mams, users can request workspaces in MAMS to upload netCDF files. These workspaces are additionally mounted on the THREDDS-Server.

mde-analysis

The Analysis framework for the model data explorer. Based upon the Data Analytics Software Framework (DASF) this plugin provides the framework to compute aggregated analysis on datasets without the need to download the raw data.

mde-download

The download framework for the model data explorer. As mde-analysis based upon the DASF. This plugin provides the framework to download subsets of the data.