Published

OGC Best Practice

OGC Best Practice for Earth Observation Application Package
Pedro Gonçalves Editor
Version: 1.0
Additional Formats: XML PDF DOC
OGC Best Practice

Published

Document number:20-089r1
Document type:OGC Best Practice
Document subtype:General
Document stage:Published
Document language:English

License Agreement

Permission is hereby granted by the Open Geospatial Consortium, (“Licensor”), free of charge and subject to the terms set forth below, to any person obtaining a copy of this Intellectual Property and any associated documentation, to deal in the Intellectual Property without restriction (except as set forth below), including without limitation the rights to implement, use, copy, modify, merge, publish, distribute, and/or sublicense copies of the Intellectual Property, and to permit persons to whom the Intellectual Property is furnished to do so, provided that all copyright notices on the intellectual property are retained intact and that each person to whom the Intellectual Property is furnished agrees to the terms of this Agreement.

If you modify the Intellectual Property, all copies of the modified Intellectual Property must include, in addition to the above copyright notice, a notice that the Intellectual Property includes modifications that have not been approved or adopted by LICENSOR.

THIS LICENSE IS A COPYRIGHT LICENSE ONLY, AND DOES NOT CONVEY ANY RIGHTS UNDER ANY PATENTS THAT MAY BE IN FORCE ANYWHERE IN THE WORLD. THE INTELLECTUAL PROPERTY IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NONINFRINGEMENT OF THIRD PARTY RIGHTS. THE COPYRIGHT HOLDER OR HOLDERS INCLUDED IN THIS NOTICE DO NOT WARRANT THAT THE FUNCTIONS CONTAINED IN THE INTELLECTUAL PROPERTY WILL MEET YOUR REQUIREMENTS OR THAT THE OPERATION OF THE INTELLECTUAL PROPERTY WILL BE UNINTERRUPTED OR ERROR FREE. ANY USE OF THE INTELLECTUAL PROPERTY SHALL BE MADE ENTIRELY AT THE USER’S OWN RISK. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR ANY CONTRIBUTOR OF INTELLECTUAL PROPERTY RIGHTS TO THE INTELLECTUAL PROPERTY BE LIABLE FOR ANY CLAIM, OR ANY DIRECT, SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES, OR ANY DAMAGES WHATSOEVER RESULTING FROM ANY ALLEGED INFRINGEMENT OR ANY LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR UNDER ANY OTHER LEGAL THEORY, ARISING OUT OF OR IN CONNECTION WITH THE IMPLEMENTATION, USE, COMMERCIALIZATION OR PERFORMANCE OF THIS INTELLECTUAL PROPERTY.

This license is effective until terminated. You may terminate it at any time by destroying the Intellectual Property together with all copies in any form. The license will also terminate if you fail to comply with any term or condition of this Agreement. Except as provided in the following sentence, no such termination of this license shall require the termination of any third party end-user sublicense to the Intellectual Property which is in force as of the date of notice of such termination. In addition, should the Intellectual Property, or the operation of the Intellectual Property, infringe, or in LICENSOR’s sole opinion be likely to infringe, any patent, copyright, trademark or other right of a third party, you agree that LICENSOR, in its sole discretion, may terminate this license without any compensation or liability to you, your licensees or any other party. You agree upon termination of any kind to destroy or cause to be destroyed the Intellectual Property together with all copies in any form, whether held by you or by any third party.

Except as contained in this notice, the name of LICENSOR or of any other holder of a copyright in all or part of the Intellectual Property shall not be used in advertising or otherwise to promote the sale, use or other dealings in this Intellectual Property without prior written authorization of LICENSOR or such copyright holder. LICENSOR is and shall at all times be the sole entity that may authorize you or any third party to use certification marks, trademarks or other special designations to indicate compliance with any LICENSOR standards or specifications. This Agreement is governed by the laws of the Commonwealth of Massachusetts. The application to this Agreement of the United Nations Convention on Contracts for the International Sale of Goods is hereby expressly excluded. In the event any provision of this Agreement shall be deemed unenforceable, void or invalid, such provision shall be modified so as to make it valid and enforceable, and as so modified the entire Agreement shall remain in full force and effect. No decision, action or inaction by LICENSOR shall be construed to be a waiver of any rights or remedies available to it.

None of the Intellectual Property or underlying information or technology may be downloaded or otherwise exported or reexported in violation of U.S. export laws and regulations. In addition, you are responsible for complying with any local laws in your jurisdiction which may impact your right to import, export or use the Intellectual Property, and you represent that you have complied with any regulations or registration procedures required by applicable law to make this license enforceable.



I.  Abstract

Platforms for the exploitation of Earth Observation (EO) data have been developed by public and private companies in order to foster the usage of EO data and expand the market of Earth Observation-derived information. A fundamental principle of the platform operations concept is to move the EO data processing service’s user to the data and tools, as opposed to downloading, replicating, and exploiting data ‘at home’. In this scope, previous OGC activities initiated the development of an architecture to allow the ad-hoc deployment and execution of applications close to the physical location of the source data with the goal to minimize data transfer between data repositories and application processes.

This document defines the Best Practice to package and deploy Earth Observation Applications in an Exploitation Platform. The document is targeting the implementation, packaging and deployment of EO Applications in support of collaborative work processes between developers and platform owners.

The Best Practice includes recommendations for the application design patterns, package encoding, container and data interfaces for data stage-in and stage-out strategies focusing on three main viewpoints: Application, Package and Platform.

II.  Keywords

The following are keywords to be used by search engines and document catalogues.

ogcdoc, OGC document, EO, Application Package, Container, CWL, STAC


III.  Preface

Attention is drawn to the possibility that some of the elements of this document may be the subject of patent rights. The Open Geospatial Consortium shall not be held responsible for identifying any or all such patent rights.

Recipients of this document are requested to submit, with their comments, notification of any relevant patent claims or other intellectual property rights of which they may be aware that might be infringed by any implementation of the standard set forth in this document, and to provide supporting documentation.

IV.  Security Considerations

This document defines Best Practice to package and deploy an Application on a Platform that implies a trust relationship between the Application developer, the Application integrator, the Platform and the consumer.

No security considerations have been made for this Best Practice.

V.  Submitting Organizations

The following organizations submitted this Document to the Open Geospatial Consortium (OGC):

VI.  Submitters

All questions regarding this submission should be directed to the editor or the submitters:

Name Representing
Pedro Gonçalves (editor) Terradue
Fabrice Brito Terradue
Tom Landry CRIM
Francis Charette-Migneault CRIM
Richard Conway Telespazio VEGA UK
Adrian Luna European Union Satellite Centre
Omar Barrilero European Union Satellite Centre
Panagiotis (Peter) A. Vretanos CubeWerx Inc.
Cristiano Lopes European Space Agency (ESA)
Antonio Romeo RHEA Group
Paulo Sacramento Solenix
Samantha Lavender Pixalytics
Marian Neagul West University of Timisoara

OGC Best Practice for Earth Observation Application Package

1.  Scope

This document defines the Best Practice to package and deploy Earth Observation Applications in an Exploitation Platform.

The document is targeting the implementation, packaging and deployment of EO Applications in support of collaborative work processes between developers and platform owners. It supports developers that want to adapt and package their existing algorithms written in a specific language to be deployed in Earth Observation Exploitation Platforms and exposed through a Web Service endpoint, OGC API — Processes.

The Best Practice includes application design patterns, package encoding, container and data interfaces for data stage-in and stage-out strategies.

Section 6 introduces the information material about Earth Observation (EO) Platform architecture targeting the deployment and execution of EO Applications in distributed Cloud Platforms. The section provides an overview of EO applications design patterns, package and data interfaces.

Sections 7, 8 and 9 present the normative material defining the best practices to implement an EO Application, to package an EO Application and to deploy the packaged EO Application.

2.  Conformance

This document defines the Best Practice for Earth Observation Application Packages targeting three standardization targets:

For the three standardization targets listed above, this Best Practice focuses on Earth Observation Applications that require the staging, input, and/or output of EO Products. The Best Practice also discusses the implications for the Application Package and hosting Platform.

In order to conform to this OGC Best Practice, an application developer shall choose to implement the following conformance classes:

In order to conform to this OGC Best Practice, and according to the Application Conformance Class, the application integrator shall implement the following conformance classes:

In order to conform to this OGC Best Practice, and according to the Application Package Conformance Class, a platform shall choose to implement the following conformance classes:

Conformance with this Best Practice shall be checked using all the relevant tests specified in Annex A (normative) of this document. The framework, concepts, and methodology for testing, and the criteria to be achieved to claim conformance are specified in the OGC Compliance Testing Policies and Procedures and the OGC Compliance Testing web site.

All requirements-classes and conformance-classes described in this document are owned by the documents(s) identified.

3.  Normative references

The following documents are referred to in the text in such a way that some or all of their content constitutes requirements of this document. For dated references, only the edition cited applies. For undated references, the latest edition of the referenced document (including any amendments) applies.

Arliss Whiteside Jim Greenwood : OGC 06-121r9, OGC Web Service Common Implementation Specification. Open Geospatial Consortium (2010). https://portal.ogc.org/files/?artifact_id=38867

OGC API — Processes — Part 1: Core Standard, 2021. https://docs.ogc.org/is/18-062r2/18-062r2.html

Commonwl.org: Common Workflow Language Specifications, https://w3id.org/cwl/

Radiant Earth Foundation: SpatioTemporal Asset Catalog specification, https://stacspec.org

4.  Terms and definitions

This document uses the terms defined in OGC Policy Directive 49, which is based on the ISO/IEC Directives, Part 2, Rules for the structure and drafting of International Standards. In particular, the word “shall” (not “must”) is the verb form used to indicate a requirement to be strictly followed to conform to this document and OGC documents do not use the equivalent phrases in the ISO/IEC Directives, Part 2.

This document also uses terms defined in the OGC Standard for Modular specifications (OGC 08-131r3), also known as the ‘ModSpec’. The definitions of terms such as standard, specification, requirement, and conformance test are provided in the ModSpec.

For the purposes of this document, the following additional terms and definitions apply.

This document uses the terms defined in Sub-clause 5.3 of OGC 06-121r9, which is based on the ISO/IEC Directives, Part 2, Rules for the structure and drafting of International Standards. In particular, the word “shall” (not “must”) is the verb form used to indicate a requirement to be strictly followed to conform to this standard.

For the purposes of this document, the following additional terms and definitions apply.

4.1. Application

A self-contained set of operations to be performed, typically to achieve a desired data manipulation, written in a specific language (e.g. Python, R, Java, C++, C#, IDL).

4.2. Application Package

A platform independent and self-contained representation of an Application, providing executables, metadata and dependencies such that it can be deployed to and executed within an Exploitation Platform.

4.3. Compute Platform

The Platform providing the computational resources for the execution of the Application.

4.4. Container

A container is a standard unit of software that packages up code and all its dependencies so that includes everything needed to run an application: code, runtime, system tools, system libraries and settings.

4.5. Exploitation Platform

An on-line system made of products, services and tools for exploitation of data.

4.6. Spatiotemporal Asset

Any file that represents information about the earth captured in a certain space and time.

4.7. GeoTIFF

A public domain metadata standard that allows georeferencing information to be embedded within a TIFF file. The potential additional information includes map projection, coordinate systems, ellipsoids, datums, and everything else necessary to establish the exact spatial reference for the file.

4.8. HDF5

The Hierarchical Data Format version 5 (HDF5), is an open source file format that supports large, complex, heterogeneous data. HDF5 uses a “file directory” like structure that allows you to organize data within the file in many different structured ways, as you might do with files on your computer

4.9. JPEG2000

An image compression standard and coding system

4.10. SAFE

SAFE, the Standard Archive Format for Europe, is designed to act as a standard format for archiving and conveying Earth observation data within the European Space Agency (ESA) archiving facilities and, potentially, within the archiving facilities of cooperating agencies.

4.11. Processing Result

The Products produced as output of a Processing Service execution.

4.12. Processing Service

A non-interactive data processing provided as a service by a platform that has a well defined set of input data types, input parameterization, producing Processing Results with a well defined output data type.

4.13. Processing Software

A set of predefined functions that interact to achieve a result. For the exploitation platform, it comprises interfaces to derive data products from input data, conducted by a hosted processing service execution.

4.14. Products

Earth Observation data (commercial and non-commercial) and value-added data. It is assumed that the Exploitation Platform provides the data access mechanisms for an existing supply of Earth Observation Products.

5.  Conventions

This section provides details and examples for any conventions used in the document. Examples of conventions are symbols, abbreviations, use of XML schema, or special notes regarding how to read the document.

5.1.  Identifiers

The normative provisions in this document are denoted by the URI

http://www.opengis.net/spec/eoap-bp/1.0

All requirements and conformance tests that appear in this document are denoted by partial URIs which are relative to this base.

5.2.  Abbreviated terms

  • API Application Programming Interface

  • COG Cloud Optimized GeoTIFF

  • CWL Common Workflow Language

  • EO Earth Observation

  • EP Exploitation Platform

  • GDAL Geospatial Data Abstraction Library

  • IW Interferometric Wide

  • JSON JavaScript Object Notation

  • OGC Open Geospatial Consortium

  • OS Operating System

  • OWS OGC Web Services

  • REST Representational State Transfer

  • SAFE Standard Archive Format for Europe

  • SLC Single Look Complex

  • SNAP Sentinel Application Platform toolbox

  • STAC SpatioTemporal Asset Catalog

  • TBD To Be Determined

  • TIFF Tagged Image File Format

  • URL Uniform Resource Locator

  • YAML YAML Ain’t Markup Language

6.  Components Overview

6.1.  Introduction

In recent years, Platforms for the Exploitation of Earth Observation (EO) data have been developed by public and private companies in order to foster the usage of EO data and expand the market of Earth Observation-derived information. The domain is composed of platform providers, service providers who use the platform to deliver a service to their users, and data providers. The availability of free and open data (e.g. Copernicus Sentinel), together with the availability of affordable computing resources, creates an opportunity for the wide adoption and use of EO data in a growing number of fields in our society.

An EO exploitation platform is a collaborative virtual work environment providing the mechanisms to deliver EO data and the tools, processors, and ICT resources required to work with them, through one coherent set of interfaces. A fundamental principle of the platform operations concept is to move the EO data processing service’s user to the data and tools, as opposed to downloading, replicating, and exploiting data ‘at home’. Furthermore, platforms offer an environment which takes care of all data processing orchestration tasks and the availability of scalable computational resources offered by the Cloud shortens the time to market of applications.

In this scope, OGC Testbeds 13-16 initiated the drafting of an architecture to allow the deployment and execution of externally developed applications on Earth Observation (EO) data and processing platforms. During the OGC Innovation Program initiative OGC Earth Observation Applications Pilot, conducted between December 2019 and July 2020 (OGC 20-042, OGC 20-073), the participants explored and assessed the existent draft specifications, which addressed both application description and discovery, APIs for deployment, execution, and result access, as well as specifications for service chaining and workflow building. This document summarizes their findings and defines a Best Practice to package and deploy Earth Observation Applications in an Exploitation Platform.

In summary, the architecture as defined by the OGC Earth Observation Applications Pilot targets the following requirements:

  • Decouple application developers from exploitation platform operators and from application consumers

  • Allow application developers to make their applications available on any number of platforms with minimal modifications

  • Allow application developers to focus on application development by minimizing platform specific particularities

  • Enable exploitation platforms to virtually support any type of packaged EO application

The figure below provides a high-level view of the interactions between the actors and the architecture.