SoilPulse – A software package for semi-automated metadata management and publication

Every model calibration/validation task is as good as the range of data available for the task and using other team’s data can greatly enhance the calibration outcome for wider range of conditions. Lot of data was gathered during the long research history on soil erosion, but the interoperability of this data is in many cases hindered by inhomogeneous data structure of the single datasets, if these are at least available digitally. The analysis and aggregation of existing digital data sets is a complicated task due to vastly heterogeneous field situations, various spatio-temporal scales involved, different experimental setups and equipment, and numerous repository types and structures. Resources often lack sufficient description in metadata making it hard for humans and impossible for computers to fully understand the structure and contents of the data set. The missing common data management and data structure format in soil erosion research can be seen as major drawback, which hampers data reusability and scientific exchange and progress. However, expecting all the research teams to adopt a common data management approach is naïve.

Within the NFDI4Earth pilot SoilPulse (soilpulse.github.io) we aim to develop a software library responsible for handling metadata from existing data sets of various types. The package will contain tools for metadata extraction (if already existing), creation (by parsing the data set and recognizing metadata elements), representing by a common general metadata scheme, storing the resource’s metadata image, and providing tools to query the storage to reach all available data sets fitting particular conditions.

The poster presents a SoilPulse package structure, intended process-flow of interactive dataset registration and recognition, and metadata mining tools overview. As SoilPulse is in active development we highly appreciate comments, hints and impulses to further improve the tool!