EGU2020-22052
https://doi.org/10.5194/egusphere-egu2020-22052
EGU General Assembly 2020
© Author(s) 2020. This work is distributed under
the Creative Commons Attribution 4.0 License.

Towards Seamless Planetary-Scale Services

Peter Baumann1 and the Peter Baumann*
Peter Baumann and the Peter Baumann
  • 1Jacobs University, Bremen, Germany (p.baumann@jacobs-university.de)
  • *A full list of authors appears at the end of the abstract

Collaboration requires some minimum of common understanding, in the case of Earth data in particular common principles making data interchangeable, comparable, and combinable. Open standards help here; in case of Big Earth Data specifically the OGC/ISO Coverages standard. This unifying framework establishes  a common framework for regular and irregular grids, point clouds, and meshes., in particular: for spatio-temporal datacubes. Services grounding on such common understanding can be more uniform to access and handle, thereby implementing a principle of "minimal surprise" for users visiting different portals. Further, data combination and fusion benefits from canonical metadata allowing alignmen, e.g, between 2D DEMs, 3D satellite image timeseries, 4D atmospheric data.

The EarthServer federation is an open data center network offering dozens of Petabytes of a critical variety, such as radar and optical Copernicus data, atmospheric data, elevation data, and thematic cubes like global sea ice. Data centers like DIASs and CODE-DE, research organizations, companies, and agencies have teamed up in EarthServer. Strictly based on OGC standards, an ecosystem of data has been established that is available to users as a single pool, in particular for efficient distributed data fusion irrespective of data location.

The underlying datacube engine, rasdaman, enables location-transparent federation: clients can submit queries to any node, regardless of where data sit. Query evaluation is optimized automatically, including multi-data fusion of data residing on different nodes. Hence, users perceive one single, common information space. Thanks to the open standards, a broad spectrum of open-source and proprietary clients can utilize this federation, such ranging from OpenLayers and NASA WorldWind over QGIS and ArcGIS to python and R.

In our talk we present technology, services, and governance of this unique intercontinental line-up of data centers. A demo will show distributed datacube fusion live.

Peter Baumann:

the rasdaman team

How to cite: Baumann, P. and the Peter Baumann: Towards Seamless Planetary-Scale Services , EGU General Assembly 2020, Online, 4–8 May 2020, EGU2020-22052, https://doi.org/10.5194/egusphere-egu2020-22052, 2020

Display materials

Display file

Comments on the display material

AC: Author Comment | CC: Community Comment | Report abuse

Display material version 2 – uploaded on 01 May 2020
  • AC1: Comment on EGU2020-22052, Peter Baumann, 06 May 2020

    continuing from sessionchat:

    Q: @peter, Have you looked at challenges around data integrity and provenance?
    A: integrity: that is given by the requirement of offering coverages as per OGC Coverage Implementation Schema (CIS). However, this is on data level, so it does not (yet) affect metadata.  Here we currently accept what is there, not the least because the federation members have different policies and traditions. It isongoing work to get metadata unified, just as reported by many other groups.
     
     
  • AC2: Comment on EGU2020-22052, Peter Baumann, 06 May 2020
    Q: If you manipulate or transform the data how can you trace it back to the original source?
    A: for each incoming file injected into a datacube we maintain individual metadata records, which can include a "contributing footprint". With our rasdaman system we deliver, in a request, only those metadata which apply to the region selected, so users can trace back the origin.
  • AC3: funny author list, Peter Baumann, 06 May 2020

    "Peter Baumann and the Peter Baumann" is a funny twist the EGU service has made - as shown at the bottom, it should read "Peter Baumann and the rasdaman team" instead. Just for the records.

Display material version 1 – uploaded on 01 May 2020, no comments