Towards a Federated Marine SDI: Connecting Land and Sea to Protect the Arctic Environment Engineering Report

An Application Programming Interface (API) is a standard set of documented and supported functions and procedures that expose the capabilities or data of an operating system, application, or service to other applications [adapted from ISO/IEC TR 13066-2:2016].

DDIL (Denied, Disrupted, Intermittent, and Limited Bandwidth environments) to describe scenarios where the connectivity is not ideal and actions need to be taken to guarantee a normal or minimum operation of software applications.

Interoperability is the ability to communicate, execute programs, or transfer data among various functional units in a manner that requires the user to have little or no knowledge of the unique characteristics of those units

MSDI is a specific type of Spatial Data Infrastructure (SDI) with a focus on the marine environment.

The FMSDI pilot built on the works of prior initiatives, such as the Marine Spatial Data Infrastructure Concept Development Study, the Maritime Limits and Boundaries Pilot, and the Arctic Spatial Data Infrastructure Pilot. Currently, FMSDI initiative include the following three phases which will be extended in future.

The IHO is a high level domain specific international intergovernmental standards organization, often represented by their national hydrographic offices. The IHO initially developed unique stand-alone standards, such as the IHO Transfer Standard for Digital Hydrographic Data S-57, but is in the process of replacing these standards with standards based on the ISO Geographic information/Geomatics standards (i.e., ISO/TC 211). The transition to the new IHO Universal Hydrographic Data Model (S-100) is in progress, and much of the hydrographic data currently in use is built to S-57 and is therefore only partially suitable for use with many of the Web Services standards and APIs available from OGC.

IHO S-100, the framework standard, has been under development since the publication of its predecessor, IHO S-57 in 2001. IHO S-57, the current standard for the encoding of Electronic Navigational Charts under the SOLAS convention is a vector based standard developed specifically for the purpose of encoding charts for the purpose of safe navigation but S-100, conceived shortly afterwards represented a much bigger step forward.

S-100 aimed to overcome many of the perceived shortcomings of the newly released S-57 standard and was defined as a much broader standard where a framework of ISO-like structures was defined leaving the details of content and encoding to the authors of specific product specifications which would then sit alongside the main framework. S-100 therefore had the following goals:

S-100 is currently at edition 4.0.0 with edition 5.0.0 in preparation. As part of the development of S-100 two encodings for vector feature data were defined, one was ISO8211, a compact binary format used predominantly in the encoding of Vector ENC charts, and the other was the S-100 GML profile, Part 10b. The S-100 GML (Geography Markup Language) profile is a subset of GML ISO19136 designed to support the encoding of simple vector datasets. The S-100 GML profile was developed by the UK around 2013 and incorporated into S-100 as Part 10b shortly thereafter. The profile has been used by various project teams within the IHOs Nautical Publications working group as well as other S-100 based product specification developments. S-100 does not define data content but only provides a toolkit for its definition. Data content is defined within S-100 product specifications. These product specifications detail how data is defined, aggregated, and exchanged along with metadata such as coverage, CRS, geometry and encoding details. The detailed definitions of feature, attribute and associations for an individual product specification are contained within its feature catalog and also lodged in the IHOs geospatial registry; an ISO compliant registry where all S-100 products' definitions, concepts and details are kept and reconciled by a dedicated team.

The following S-100 product specifications were used during this pilot:

It is important to note, then, that the development of IHO S-100 has continued during the Pilot’s progress and this is reflected in updates to the existing S-100 standard. This Pilot remains the most up to date implementation of the S-100 framework to date and certainly one of the few with concrete reference implementations in software from multiple participating vendors.

OGC’s main interaction with the marine community is mainly through its established Marine Domain Working Group (MDWG), formed within the IHO/OGC memorandum of understanding (MOU). This group was formed to address the gaps in the OGC framework within the marine domain. For many years the IHO has run an MSDI community through its MSDIWG and the OGC MDWG works closely with the MSDIWG to cross-fertilize ideas and outline where opportunities exist to improve the ecosystem for the benefit of stakeholders.

The OGC MDWG has a focus on the S-100 framework and its broader integration into the OGC community. This process is likely to take some time and the MLB Pilot is a significant step in exploring such interfaces. IHO and OGC standards have many common elements and both derive largely from overarching ISO standards. The practicalities of their use alongside each other are not always well defined however and the project has sought to explore these practical steps as much as possible.

For several years, the OGC members have worked on developing a family of Web API standards for the various geospatial resource types. These APIs are defined using OpenAPI. As the OGC API standards keep evolving, are approved by the OGC and are implemented by the community, the aviation industry can subsequently experiment and implement them.

The following OGC API Standards and Draft Specifications were used for the development of APIs during this Pilot.

OGC API – Features: a multi-part standard that defines the capability to create, modify, and query vector feature data on the Web and specifies requirements and recommendations for APIs that want to follow a standard way of accessing and sharing feature data. It currently consists of the following four parts.

OGC API - Maps: Maps offers a modern approach to the OGC Web Map Service (WMS) standard for provision map and raster content.

OGC API – Common: Common provides those elements shared by most or all of the OGC API standards to ensure consistency across the family.

OGC API - Processes: Processes allow for processing tools to be called and combined from many sources and applied to data in other OGC API resources though a simple API.

OGC API - EDR: The Environmental Data Retrieval (EDR) Application Programming Interface (API) provides a family of lightweight query interfaces to access spatio-temporal data resources by requesting data at a Position, within an Area, along a Trajectory or through a Corridor. A spatio-temporal data resource is a collection of spatio-temporal data that can be sampled using the EDR query pattern geometries. These patterns are described in the section describing the Core Requirements Class.

OGC API – Tiles: This API defines how to discover which resources offered by the Web API can be retrieved as tiles, retrieve metadata about the tile set (including the supported tile matrix sets, the limits of the tiled set inside the tile matrix set) and how to request a tile.

OGC SensorThings API: Provides an open, geospatial-enabled and unified way to interconnect the Internet of Things (IoT) devices, data, and applications over the Web.

Draft OGC API - DGGS: This draft API enables applications to organize and access data arranged according to a Discrete Global Grid System (DGGS).

Draft OGC API Coverages: Coverages allows discovery, visualization and query of complex raster stacks and data cubes.

Draft OGC API – Styles: This draft API specifies building blocks for OGC Web APIs that enables map servers and clients as well as visual style editors to manage and fetch styles.

The development and adoption of these APIs is likely to have a significant impact on how organizations like IHO engineer systems which seek to be interoperable within OGC standards. The move to an API based interconnecting system of standards clarifies the dividing line between content, its expression within a technical encapsulation (its encoding) and the transport of that data to its ultimate destination.

The IHO, similarly, is instigating a major drive towards implementation of IHO S-100. The standard, many years in the making, is already key to many activities and initiatives in the marine domain. Of particular note is the IMO’s eNavigation initiative, for which S-100 forms the common maritime data structure (CMDS). The IHO is embarking on a push to get S-100 accepted as equivalent for carriage of charts and publications under the global SOLAS convention, a large undertaking and one which will embed its use in live vessel navigation for many years. As a dynamic standard which relies on the creation of product specifications S-100 is wholly dependent on its ability to be interoperable with other standards frameworks and to remain current with those frameworks.

This project is intended to contribute positively to both the OGCs APIs interface with the IHO and to assist the productionisation of S-100 as part of the implementation roadmap.

Where possible, participants incorporate the Arctic Regional Marine Spatial Data Infrastructures Working Group’s (ARMSDIWG) Arctic Voyage Planning Guide (AVPG). The AVPG is intended as a strategic planning tool and a compilation of data and services for national and international vessels traveling in the Arctic. This guide is currently in its development stage. An example of a Canadian implementation is shown in Appendix B. Participants attempted to utilize multiple data from the below list of themes and content while identifying gaps, and making recommendations to improve the federation of data services required for usability of voyage planning in the Arctic.

The Arctic Voyage Planning Guide contains the following themes and content.

The following table summarizes the TIEs (Technology Integration Experiments) performed during FMSDI Phase 3.

In previous FMSDI pilots, IIC made use of open source components for the deployment of OGC API - Features services covering Marine Protected Areas which implement the IHO S-100’s General Feature Model (GFM) content. This blending of IHO content with OGC web services models has been extended in this pilot to a more diverse set of data, meeting the use cases defined in the project. As an in-kind contribution use has been made of IIC’s own featureBuilder S-100 authoring tools and an automated pipeline for deployment to the web services API endpoint. Additionally, a fully configured cloud server has been deployed which is used for serving data.

Of particular interest within this pilot is the use made of a mapping between the S-100 GFM and GeoJSON. This is reproduced at Appendix C: The S-100 GFM (Geo) JSON format in this document and it is intended to release the mapping as an OGC document. This provides a way of taking data which meets the GFM requirements, described by S-100 feature catalog and encoding it in a form natively understood by most OGC API implementations. The GeoJSON encoding, whilst limited in some respects, has a ubiquitous appeal to developers of web-based systems and the combination of OGC transport of IHO services is attractive to many members of the IHO community who wish to deploy marine geospatial data alongside navigational use cases.

The technology used for deployment was pygeoapi. It is a python server implementation of the OGC API suite of standards and also allows for metadata to be defined for services. This has allowed IIC to develop mappings, not just for content, but for its metadata too. Names, aggregations of features, services and many “delivery” and “discovery” aspects have been developed, all of which contributes to the goal of using S-100 in situations alongside, and outside direct navigational use cases, on ongoing theme for the IHO community in general.

The pilot described in these sections is a good opportunity to broaden the data scope of IHO-OGC interoperability as it encompasses navigational datasets and use cases, together with logistical/temporal use cases (water level against time, route exchange for moving vessels) and land/sea “interfaces” (use of land based logistics to effect search and rescue, use of land resources in exceptional circumstances)

The sub-scenario documented in this section is the detail of the grounding of a cruise vessel in the region of Kotzebue, Alaska. Much thought was given to the construction of the sub-scenario, balancing the following:

The location of the pilot is the region of Kotzebue in Alaska, approx 66° North and 166° West, illustrated in the following chart (NOAA).

The sub-scenario has been chosen and constructed to highlight the following:

The sub-scenario concerns a single vessel, “Discovery”. This was modeled on a real-world vessel and dimensions set accordingly. The vessel is a passenger vessel with 182 passengers on board, engaged on a coastal trip, entering Kotzebue harbor via a pilot boarding point, the entirety of Kotzebue harbor being subject to compulsory pilotage conditions. En route to the pilot rendezvous point (the pilots meet at the Pilot Boarding Place for transfer to the vessel) the vessel suffers a loss of power and propulsion and drifts towards the shore, eventually grounding approx 4.5 Nautical Miles NW of the Espenberg Light. At the point of engine failure the coastguard is notified and an emergency declared due to the number of passengers involved and the risk to environment and property.

The sub-scenario was designed to enable the use of many different types of data and so various aspects of the proposed chain of events are in place to exercise these aspects. These are described in more detail in the Demonstration subsection.

The architecture of the deployed server is similar in concept to that deployed in the previous pilots, using JSON encoded data within a pygeoapi / apache2 web server to render OGC API - Features conformant data to end users. The apache mod_wsgi plugin is used to enable the pygeoapi plugin which renders the GeoJSON according to requests made.

Data is initially either converted (in the case of ENCs where only S-57 data is available) or compiled from scratch (most of the data in the pilot has been compiled due to the sparsity of data in the region). GeoJSON is used as the encoding but this is moderated using the S-100 GFM encoding which maps all properties from S-100 attributes/subattributes and defines feature names and codes. In this way the S-100GFM is used as a universal model for the data promulgated by the pygeoapi server.

Other aspects of the technical architecture are as follows.

The sequence of events, broadly, is:

The route, shown in red, follows the image below.

The actions, actors, and technologies involved are described in the following table, according to each stage of the incident and its response.

The pilot has demonstrated how S-100’s GFM can represent multiple different datasets for different purposes in a search/rescue context. The ability to integrate such APIs together and form a common endpoint for users is a high priority as is the ability for end users to ingest OGC API endpoints. There is, obviously, already a well defined ecosystem for search/rescue using multiple spatial data sources and the extent to which OGC API is used in those systems is unknown. The aim of the pilot is to show how S-100 data can be reused, encoded as OGC APIs and its potential in a broader use case scenario.

Results of the interoperability and implementation were very mixed. For some data types a lot of coverage exists and standards are in place for them to be expressed using S-100. For example charts, S-421 routes, spot depths, aids to navigation and regulatory information all exist, even though some are still in publication form. This is to be expected as navigation in the bay is the primary usage and safety of navigation in the region is assured by the various authorities with jurisdiction.

However, there was a lack of any systematic S-100 data approach for the region, not unusual given that S-100 implementation is in its infancy. So, for this Pilot, a number of different datasets were created and converted and exposed via OGC API. This process worked very well and showed that, for test regions, a multi-product test bed can be established reasonably quickly to test/exercise scenarios.

The utility of the S-100 data and API endpoints created was effective with clients able to quickly use data and understand its source and intended usage. This allowed a number of aspects of the scenario to be explored and identified areas where data was deficient (e.g., it’s difficult to model whether the tide will refloat a vessel if only point predictions are available with no S-100 model to transport them), it also shows the value of certain data types (e.g., the skin of the earth from ENC as a ubiquitous backdrop and the value of route exchange during operational scenarios where multiple vessels are interacting).

The Land/Sea integration, from a basic maritime perspective, was not a major factor. Post-incident enquiries/modeling and broader considerations (e.g., effects of climate change / sea level rise on siltation etc etc) may require more detailed land sea integration but at a basic “vector features” level (we looked at coastlines, runways, amenities and landmarks) the data found for maritime purposes is interoperable with the land based sources - however, this should be countered with the fact that the land data was not rich nor dense so a meaningful assessment of the integration with the maritime data available is difficult.

No concrete methodology has been published for resolving these three different aspects of land/sea interoperability but one could be constructed using existing best practices. This would at least establish where the biggest gaps are, and identify technologies for their resolution. Certainly, the scenario considered for search and rescue in a remote region with sparse coverage is a very good example. In our scenario land data must be repurposed (for delivery of spares to the vessel, evacuation via helicopter or plane, contact details on shore) and where innovative methods must be used for safety reasons.

Compusult, incorporated in 1985, is a diversified information technology company with over 37 years of experience in software development and is a leading provider of geospatial software solutions. Compusult is a long-standing member of the Open Geospatial Consortium (OGC) that assisted in pioneering specifications like the Web Map Service (WMS) Catalog Service for the Web (CSW), GeoPackage, etc.

Our commercial geospatial software solution, Web Enterprise Suite (WES), manages geospatial data by delivering a complete, end-to-end system providing geospatial interoperability, publishing, data discovery, data management, data access/collection, data analysis/synchronization, user management and collaboration. WES components are based on OGC / IHO / ISO and other interoperability specifications.

GO Mobile is an iOS, Android and Windows-based mobile application that uses OGC standards to provide first responders, surveyors and in-field data collectors with the ability to collect, consume and share information in connected or disconnected environments.

Compusult has created server instance D104 and instantiated a copy of our Web Enterprise Suite software as an in-kind contribution to provide OGC Web and OGC API services to access the datasets that are outlined in the sections below. This allowed clients to be able to directly consume and compare the performance and applicability of the OGC interfaces with a common dataset.

WES product, DBWMS, utilizes RDBMS data stores with accompanying OGC Styled Layer Descriptor (SLD) documents to produce Web Map Service endpoints to display the RDBMS data using the OGC WMS standard. Compusult also provided an OGC API component that allows the data from RDBMS data stores to be provisioned as OGC API - Features and coverages.

The data sources described in the sections below were made available to all clients participating in the pilot via OGC standards to be used to support emergency operations and potential threat to the surrounding marine ecosystems. This data, and associated APIs, can also be useful in helping craft climate change and/or search and rescue scenarios. The availability of the services in both the OGC Web Service and OGC API service flavors allowed for the direct comparison of the performance and flexibility of these standards.

Rescuing / transferring passengers and crew (immediate transfer of any injured) and potential oil spill detection.

The Discovery cruise ship is being monitored as it travels from Nome, Alaska to Kangerlussuaq, Greenland. A distress message has been sent from the Discovery cruise ship via ship board sensors stating that the ship has lost power. This is confirmed with the captain that informs CG-SAR that the ship is adrift. The US Coast Guard Search and Rescue (CG-SAR) has been alerted and they have entered the incident into the Incident Command System (ICS).

An analyst with CG-SAR begins the process to create a situational awareness portfolio to support the rescue operations. The analyst begins the process by gathering relevant information and content to provide situational awareness and provide input into the decision-making process.

The following sources were made available to all participants as OGC APIs - Features and OGC APIs - Coverages and, where applicable, OGC WMS services. These data sources can be compared and contrasted historically to be used for comparison with other sources of weather-based data.

The analyst then incorporates the logistics to support the rescue operation. Logistics services include the following from Compusult as OGC APIs and OGC WMS services as well as other readily available sources from the internet such as hospitals.

The analyst then creates a situational view using the information above and dispatches the nearest aircraft to provide up to date reconnaissance conditions of the ship. This was simulated using the Compusult GO Mobile application to take pictures and gather metadata about the ship and its surroundings. This data was available as an OGC WMS and OGC API - Features for other participants to consume.

Reports from the reconnaissance mission indicate there is a potential fuel leakage. An analysis conducted using satellite imagery to determine the impact. The analysis concludes there are no serious impacts from the potential spillage and rescue operations are instigated with constant updates and briefings. The rescue operation includes the dispatch of the helicopter bringing in personnel to assist the captain and the dispatch of a nearby US Coast Guard cruiser to rescue all personnel. The analyst provides the cleanup crew with relevant information to support vessel recovery and disposal. The following data sources were used to determine potential fuel leakage.

Output of the fuel leakage analysis was made available as Esri Shapefile and OGC API - Features.

Actor(s), end-user(s), and/or stakeholder(s):

Interoperable Technologies:

Data and Platforms

Arctic Voyage Planning Guide Themes (additional info)

The figure below shows the technical architecture of the D104 server made available by Compusult to all OGC Participants. This diagram shows the flow of data from the various sources through OGC APIs and Services. For this pilot the open source application PyGeoAPI was used to create the OGC API services. OGC SensorThings and OGC WMS services were made available from Compusult’s Web Enterprise Suite application.

NOAA Global Forecast system (GFS), GRIB weather files are downloaded and processed to generate particle layers depicting the wind direction and speed. Additionally, the GRIB files are processed to create CoverageJSON that is fed to the client using OGC API - Coverages.

Text based Daily Polar Sea Ice Concentrations, METAR weather reports, FAA Flight Information, Automatic Identification System (AIS) Maritime Ship Traffic and National Data Buoy Center (NDBC) Buoys and Ships Observation Data are parsed, stored in a postgres database, and served in the client using OGC API - Features.

IoT Sensor Data is gathered and stored in SensorHub and made available externally using OGC SensorThings and OGC WMS standards.

Satellite data such as Sentinel and Radarsat are processed to detect potential oil spills. This process results in a shapefile which is rendered in the client using OGC API - Features.

Using GO Mobile, field data is collected in disconnected operations stored in OGC GeoPackages and rendered in the client using OGC WMS.

In order to provide context to the data that was rendered on the screen, Compusult extended the OGC API - Features spec to also include styling rules. These styling rules provided a default style for each layer with an URL to the associated SLD. In order to make use of the SLDs Compusult incorporated an open source application to extend the capabilities of the Leaflet to read and render complicated SLD documents. The SLD extension for Leaflet is located on github at https://github.com/orfon/Leaflet.SLD.

The sample JSON below shows the default style for the Metar Weather Report Observations layer.

The following storyboard D104 server demonstration storyboard can help illustrate how users will interact with the server component developed in this deliverable. This storyboard helps to clarify the product objectives and functionalities. This storyboard helps to clarify the product objectives and functionalities. The actions, actors and technologies involved are described in the following table, according to each stage of the demonstration.

Compusult was able to provide services in both OGC API and OGC WMS to the various clients. These services were successfully integrated with all clients.

These clients each provided different technologies including Esri’s ArcGIS Pro and Helyx with a web based leaflet JS client showing that the integration of OGC Standards for Features and WMS can be accomplished using various clients. There were no issues with interoperability between the servers and clients. By providing the root endpoint to the OGC API - Features server the clients were able to interact with the services and data without requiring additional information from Compusult.

From a client perspective Compusult successfully published the OGC API endpoint into our OGC CSW catalog to allow for easy search and discovery at a later point. The same OGC API was then used in a portfolio to enhance the data content for the scenario. However with the lack of defined styling rules and legend it made it difficult to determine visually what the intent of the data was when first viewed. Users are required to interact with the data to understand the content they are analyzing visually.

As described in the Challenges and Lessons Learned below, Compuslt feels that OGC API - Features can provide a great deal of information to the end user in a standard format that is easy to discover and navigate. However the lack of styling rules and legends takes away from the end user experience.

The results of this pilot showed the lack of data in the subject area. The Arctic Voyage Planning Guide provided a significant amount of data through much of the Arctic region but only a small subset of the layers contained data in the subject area. This provided great challenges in developing a scenario and hence simulated data had to be used.

Besides the lack of data, two of the main challenges Compusult faced when rendering the data in the client was the lack of styling rules and legends. This left us with rendering polygon and point features from various sources that all looked and felt the same. Without styling and legends it was difficult to put the data in the right context for the end user.

As described in the technical architecture section previously, Compusult overcame the styling problem with its own API and client by extending the OGC API - Features to induce references to SLDs that were utilized in Leaflet through the open source addition of Leaflet.sld.js.

With Coverage JSON, a legend to our client was added by mapping the data content in JSON to a color palette representing the legend for the data. The color profile is selected by the client based on the type of GRIB data, i.e., wind, temp, rain. This provides the end user with a legend representative of the data being viewed. This however comes with limitations as the client has to be updated to accommodate new data and it does not provide a solution to external clients.

Compusult was also able to compare the use of OGC WMS and OGC API - Features using the same data sources through this pilot. OGC WMS provided the benefit of uniform styling and legends across all clients as the images provided were rendered and served from the same server. However the images can be quite large and thus cause performance issues in bandwidth constrained environments. It was determined that the smaller size of the JSON responses from OGC APIs will provide a much faster response time. The added ability to filter the results from OGC API also provided much more control to the end user, allowing them to select features only relevant to the area in which they were interested. With the ability to do more fine grained filtering through the use of CQL queries, clients can make requests to the servers that would return only those features pertinent to the situation. For example, the scenarios shown in the resulting videos could be enhanced from the end user if they were provided the ability to find features contained in an AOI showing a water depth no greater than N meters. Such a query would have allowed the user to determine if and when the ship would run aground.

Our team is undertaking research into the use of DGGS within the University of Calgary, School of Engineering, Geomatics Department. Through successfully participation in Phase Two of the FMSDI Pilot Project, our Fusion server provided data agnostic features and coverage data fusion services discovered through OGC EDR API Collections (See Video Here). Participation in the original Arctic Spatial Data Pilot (See Video Here) showed where the particular scenario involved scientific modeling and analysis, based on a DGGS system architecture that fused data coming from OGC Feature and Coverage services to forecast change in permafrost distribution due to climate change. Slope stability, wildlife ranges and human socio-economic effects were assessed.

Our contribution to the Federated Marine SDI Pilot Phase 3 is a Discrete global grid system (DGGS) server deployed as the D103 Fusion Server. The D103 DGGS Fusion Server acts as a data integration engine, harvesting key data values from multiple sources of heterogeneous data and delivering data packets of analysis ready data through the OGC API - DGGS. As DGGSs are hierarchical equal area data structures, the data can reside at a native resolution while aligned with other spatial resolutions/scales, and the data in a collection of cells can be summarized using statistics and lists.

The D103 DGGS Fusion Server interacted with the DGGS client provided by Ecere corporation serving as a practical Technology Integration Experiment (TIE) for the current OGC API - DGGS candidate draft standard.

For Phase 3 of the pilot, the University of Calgary developed three sub-scenarios as follows.

The datasets described above come from different types of sources and services: coverage services, feature services etc. The Discrete global grid system (DGGS) server is able to act as a data integration engine and acquire this data via OGC standards. During the ingestion process, the data is quantized and sampled into DGGS cells. Using DGGS operations, the erosion model is implemented as a series of calculations over the cells. The results are exposed using the OGC API - DGGS, as data packets of analysis ready data that can be read by any DGGS-enabled client (Analysis of Coastal Erosion due to climate change scenario architecture.).

This section presents the results of the implementation of the sub-scenarios described in the sub-scenario and data section.

The D103 DGGS Fusion Server interacts successfully with the DGGS client provided by Ecere corporation using the current OGC API - DGGS candidate draft standard. It used the ISEA3H as the underlying DGGS, the Rhombus indexing as indexing method and tiling system, and the PYX0 format as a data encoding. Figure 36 shows the results of the integration of the coastal erosion susceptibility map displayed on the Ecere’s GNOSIS Cartographer 3D client. For more information on the integration results and interactions, please go to the Client 3 (D102) - Ecere client section.

Since the OGC draft standard API was implemented, any other client that is able to understand the DGGS and tiling system, and is able to decode the PYX0 format, would be able to read data tiles from the UCalgary DGGS server.

One significant outcome of the pilot project was the establishment of the DGGS server as a repository for connecting and using heterogeneous data relevant to arctic and marine spatial analysis that will be useful for testbed, pilots and hackathon type sandbox activity in the future.

Since during the development of the project it was discovered that a rhombus tile system conforms to the OGC Tiles API standard, future work should focus on working on an implementation of the Inverse Snyder Projection in proj4 library, and to propose a standardized identifier for this projection. This would allow other non-DGGS clients to access the data tiles and to utilize DGGS at its full potential. Also more types of integrated data applications that make use of machine learning and data mining algorithms, jupyter notebooks, and common GIS applications could start benefiting from DGGS capabilities.

We recommend that future work focus on prototyping a DGGS Tile Generator based on these recommendations to interoperate and transmit DGGS data via OGC Tile Matrix standard and have a client that can read these and operate on them through emerging OGC API standards. Of particular interest are the OGC API – Processes - Part 3: Workflows and Chaining for encoding processes and pipelines, and the DGGS API Zone query endpoint combined with the Common Query Language to refine by Search and Filter. Developing a more scientific oriented case study, such as an improved coastal erosion scenario, would demonstrate the value of all of these APIs and provide valuable feedback for the DGGS community and those writing the DGGS API Standard.

Additional challenges and lessons learned during the integration experiment and interactions are also presented in the Client 3 (D102) - Ecere client section.

Tanzle’s platform is a modular data integration platform that may be configured to source and supply any describable data including but not limited to data standards used within the FMSDI Pilot Phases. Tanzle is creating a new foundation for dynamic data that is refreshed and reconciled in time and place. The most unique aspect of our platform is how it can index, dynamically process and present data- both static and raw global sensor data- together into a collaborative environment. Through our data plane, it can organize, dynamically summarize, and present the data, allowing the user to explore raw sensor data or pass it through data processing pipelines with intermediate through full curation.

Tanzle participated in this phase of the FMSDI Pilot as an in-kind contribution by tasking our platform to demonstrate a cross-cutting server/client solution leveraging cloud infrastructure. It can be expected that further development/refinement, inside of Tanzle, will be rewired to effectively contribute to the Pilot Technology Integration Experiments (TIEs), and offer an alpha-level client component to demonstrate server-client without impeding the other Pilot participants.

A call center receives a call about an emergency incident. The location of the incident is electronically forwarded to emergency response professionals. A continuously updated “location report” is created and shared, allowing responders to view the incident location on a map, and also see their own position on the map. The initial report could include any relevant information from the past or present, or from a model. Once created, the report will be continuously updated to inform all users of dynamic conditions that might affect them.

Dynamic conditions not characterized by foundation data require rapid data integration of historic, new and time-dynamic data to ensure safe response and recovery. Past and present activity streamed across authorized clients, dynamically contributing to the report for in situ and post-emergency investigation. Several stakeholders may interact with the report. The stakeholders in this perspective include real-time actors such as vessel crew and first responders, law enforcement, analysts and engineers whereas the post-emergency activities expand the group to include reinsurance agents, media, corporate and government agents. Many of the stakeholders are high-level actors requiring simple tools and interfaces to the following basic data sources.

The Tanzle platform organizes data by time and space in a when-where-what order that enables both a real-time data plane and efficient visualization of data over time. Managing geospatial time-series data at scale requires a flexible, configurable combination of cloud infrastructure and reactive data management. Our creation is Tanzle Advanced Backend Services (TABS). TABS is uniquely derived from time-critical financial technology, adapted to support any definable data, with native support for geospatial. The figure below shows the core features of TABS. Particular attention should be paid to the multiple options (arrows) at major pipeline junctions. Elements of both real-time and curation workflows may be blended to suit a variety of scenarios, and this is done in a way that takes care to not overwhelm a frontend client.

TABS is configured here to index [geo]spatiotemporal data and algorithms, manage streaming data using reactive data processing and reduce latency and load on downstream clients and front ends. The relationship between TABS and any frontend is definable, and, in this case, are utilizing CesiumJS as a starting frontend for its advanced support for web-based and time-dynamic data and models. Due to time constraints of developing this new method, there was limited time to evaluate other clients such as QGIS. The implementation of QGIS as a frontend client was not successful - this will be addressed later and below. TABS and CesiumJS provide a flexible and efficient means of integrating native data (i.e., sensor streams) into OGC standards using a core TABS feature- ‘Algorithmic Map Tiles’. Algorithmic map tiles result when time-series data are input to a predefined algorithm, the result of which is stored in a time-dynamic web mapping service (WMS-T). What is novel about this approach is that:

Emergency responders building execution plans needed to access and collaborate on multiple sets of data. This includes the aggregation of Foundation and sensor data such as themed maps, shipping and weather patterns, and real-time sensor streams. TABS is the core technology that reacts to both changes in data and changes coming from users, providing a real-time, reactive backend to dynamic data. While TABS is the core technology, it is the client that ultimately connects the user to the data. The demonstration used an Edge-friendly web-based client. More specifically, we chose to use VueJS and CesiumJS as the frontend client, and tested QGIS client for interoperability.

There was insufficient real-time data to thoroughly demonstrate capabilities and therefore it required the demonstration to be moved away from the primary area of interest to successfully present the most fundamental components:

Interoperable Technologies

A significant aspect of this sub-scenario is the implementation of algorithmic map tiles - a reactive and dynamic data reduction and visualization approach. Algorithmic map tiles are similar to conventional map tiles; however, the pixel values are determined in real time, allowing for new data to automatically update the tile set. This allows consistent management of metadata across raw and simplified data and it allows tile specifications to be manipulated as the need arises, surfacing opportunities for improvement. The figure below illustrates two representations of a fleet of buoys obtained through Xerox PARC and DARPAs Ocean of Things program: 1) as raw entities, and 2) as a dynamically rendered heatmap. The fleet of drift buoy data are obtained in CSV or JSON format, ingressed, indexed and optimized such that they may then be dynamically processed and rendered at all supported levels of detail.

Dynamically rendered heatmaps are pursued here to demonstrate how real-time, streaming data may utilize algorithms and steps usually found in curation, only these steps are localized in time and space. These are not client-side graphics because the volume of data even in small spatiotemporal boundaries can be overwhelming. The figure below shows two examples, one being the long-term and all-inclusive heatmaps for vessel traffic and buoy environmental data and the other being the short-term heatmaps filtered to represent only one entity from either data set.

Each time-step of the dynamically rendered heatmaps is served to clients as either a low-level PNG tile or through the Web Mapping Service for Time-Dynamic data (WMS-T). It was determined that, while building support for self-service that a fully supported external, time-dynamic service would be a significant task and could only offer support for a subset during this phase of the Pilot. This includes the {time}/{tileset}/{row}/{col}.JPEG format of WMS, and also support for dynamic rendering of GRB2 files.

Data integration beyond algorithmic map tiles was significantly easier in TABS. Unstructured data coming from telemetered and broadcast messages (AIS and drift buoy) may be ingressed and fed through the appropriate data processing pipelines to produce either entity-mapping or heatmap rendering. Additionally, our heatmap renderings allow for the application of additional algorithms such as: 1) autocorrelation, and 2) cross correlation of the fields within or between data sets. More advanced heatmap results were provided to clients as the following.

We were able to serve interactive CAD models of the drift buoys, vessels and the International Space Station (ISS). Live video feeds were successfully streamed into the scene and could even be associated with CAD models. Video and snapshot media were streamed from NASA’s High Definition Earth Viewing (HDEV) experiment hosted on Ustream for the ISS, and from Trafficland for the Department of Transportation cameras. Unfortunately, access to the most optimum data and used surrogates was not available. For example, a single buoy model is used to represent all buoys, and the same is true for the AIS vessels, one model for all vessels.

Other data layers were used from various sources. Weather data in WMS and GRB2 formats was pulled from NOAA to test sideloading versus streaming and found both to be successful. Much of the weather data or data coming from other Pilot participants was not leveraged because the area of interest was changed to pursue a broader variety of data sources. This does not mean that it is not compatible with the Pilot-hosted layers - they are supported per OGC specifications. Topography and bathymetry from USGS and BOEM was also pulled and fused them together to produce a high-fidelity topobathymetric surface (a composite DEM) for the region. DEM creation did follow a curation workflow where conventional, static map tiles were produced using GDAL and sideloaded as a local tileset. However, the source elevation data (were it available) may also be assigned to streaming data processing pipelines in the future, and continuously updated elevation data could automatically update the tileset.

A few caveats must be known that there are cases where CesiumJS, OGC or other APIs, and data providers may not be well-supported, that these limitations are not limitations of TABS, and present opportunities for improvement in the services and data being integrated. And, while Tanzle hopes to improve upon TABS in service to overcoming limitations of interoperability like these through extended engineering, noting where such effort is required is likely all there is time for during this Pilot. Such notes provide beneficial topics to address in later Phases of FMSDI as well as in working groups focusing on standards and emerging strategies. A deeper dive on these and related topics is anticipated.

The following figures illustrate how the Tanzle platform successfully integrated all of the data sources and feeds that were available. However, managing real-time data required data that were at the time unavailable, forcing us to use a surrogate area - the Gulf of Mexico (GoM). The GoM provides abundant real-time and historic data in a variety of formats and projections. The figures below illustrate our successes in adapting parts of CesiumJS source code to accommodate a variety of features. One such feature is our introduction of location histograms to the native time scrubber as seen in the figure below. Like the algorithmic map tiles, the location histogram is also dynamically updated based on the viewport boundaries as well as time- and entity-filtered data results.

The location histogram works with the map layers and the layer manager tree to visually inform a user about data trends in space and in time, and the reactive nature of TABS allows a user to switch between entity-level data and the reduced, curated forms without the need to recreate entire layers when new data arrive. The figure below shows a scenario where a user filters data for a sub region allowing for entity-level analysis of AIS vessel traffic, and then uses the filtered heatmap to identify a dark vessel before sending the user to find satellite data to further explore the scenario. Unfortunately, the best data were unavailable and surrogate data was used from the International Space Station in the form of live video.

Streaming media need not come from satellites alone. It was determined that there is an airport and a lighthouse nearby to the Alaska emergency site in the sub-scenario, and it was decided to find surrogates for data that may stream from these sources as well. The possibility for Department of Transportation traffic cameras to image and stream onsite information about vessels nearing bridges was identified. It is shown, in the figure below, just how abundant these cameras are and how they could be useful. The cameras have known positions but the imagery they provide is not georeferenced. Therefore, the platform supplied the image/video feeds to a thumbnail in the lower left of the interface that opens when the associated maker is clicked on the map. This is a great example of integrating geospatial data with non geospatial data.

A significant challenge was the lack of data in the AOI. For example, historic AIS shipping data declined from late 2020 to nearly no data in the AOI during 2022. Collectively, Pilot participants struggled to find data in the AOI that supported their sub-scenarios, warranting either simulation of data or demonstrating a capability in another area. Tanzle’s preference was to focus on capability instead of simulation given that the collective effort aimed to identify opportunities for technology improvement and not to actually characterize the AOI. The following two figures speculate that Alaska AIS data are coming primarily from privately maintained ground-based receivers.

There are likely two related causes for the loss of AIS data in Alaska. First, satellite data coverage in northern latitudes does not match that of lower latitudes. The figure below illustrates various low Earth orbit constellations. It is clear that the cutoff coincides with the Aleutian Island chain, north of which there is little coverage. This would demand more of the ground-based AIS receivers, and those appear to be privately operated and maintained by only the Marine Exchange of Alaska (MXAK). Why MXAK AIS data does not trickle into “MarineCadastre.gov” in compliance with the Freedom of Information Act guidelines referred to by the US Coast Guard is a question that cannot be answered here.

Cesium and QGIS are both great candidates for Tanzle’s sub-scenario. However, both presented challenges and limitations. In Cesium’s case, utilizing and augmenting CesiumJS source code was required to leverage low-level PNG graphics as illustrated below.

A tiling algorithm was created based on theirs in order to be both time-dynamic and reactive. Augmentation of other components of CesiumJS source code such as the time scrubber were required. It was determined that the default time scrubber would be a great start but lacked a lot of the features needed to be expressed such as location histograms, time-optimized data filtering, and other backend or statistical features. Experimentation with various workstations and browsers before setting data volume caps derived from empirical data was also performed. The user was informed of “too much data” by coloring the entity in the layer manager red and reporting the number of vertices requested. This informs the user that switching to a reduced data view would be better than querying too much entity-level data. This provides an intuitive means of guiding users toward higher levels of detail or simplified data at lower levels of detail. There are more elegant solutions underway but this was a good starting point.

QGIS presented other challenges. First, QGIS does offer tools for low-level “hacking” and did not have time to explore solutions as as done with CesiumJS. It was decided to leverage QGIS to test the WMS-T egress from TABS as a result. This surfaced an immediate limitation in QGIS that it did not offer web-based authentication using a Bearer Token which is how we configured authentication to TABS. The long-term release of QGIS at the time of this writing was 3.22. It was found, in a bleeding-edge release (3.28), that there was an option to supply a HTTP header in the WMS layer manager. This could work because a WMS-T map tile with curl by passing a HTTP header, was able to successfully retrieved. The figure below illustrates the same layers used in CesiumJS above, and shows the Advanced option in QGIS where the HTTP header for authentication was supplied. This successfully authenticated QGIS to TABS. However, it required scraping the Bearer Token out of the web-developer tools of the web browser because Bearer Tokens expire daily in our configuration. Not elegant, but proves the concept and surfaces the limitation. Unfortunately, authenticating was not the only issue with QGIS. It was next discovered that QGIS was not properly parsing the capabilities that TABS was providing in our WMS-T implementation. The X-Y locations were not valid and the result is “Failed to parse capabilities.” This is likely due to the fact that we did not have time to fully implement the WMS-T spec, or perhaps that the version implemented, was no longer aligned with the version of QGIS we used. This is certainly something to further explore.

The final lesson that should be learned here is that any form of situational awareness will demand data coming from diverse sources provided through a variety of economic models, and these characteristics challenge the application of a data standards approach alone. We found that it is easier to integrate data through platform disaggregation, meaning that every aspect of the data workflow strives to maintain orthogonal relationships to every other aspect. The use cases driving the overriding products and applications determine where flexibility must yield to absolute standards. This even extends to the decisions governing whether a server or client is assigned a firmly defined set of roles.

Esri Canada appreciates the opportunity to contribute to the Open Geospatial Consortium (OGC) “Federated Marine Spatial Data Infrastructure” (FMSDI) Pilot Phase 3 and to this Engineering Report. Esri Canada are pleased to see that the international hydrographic community is continuing to investigate the use of Spatial Data Infrastructure (SDI) best practices for increasing the interoperability of systems and streamlining the exchange of geospatial data assets for important marine applications such as coastal erosion, climate change, coastal flooding, maritime safety, and marine traffic. Once fully implemented, the FMSDI will help improve the efficiency and quality of marine decision-making and help support governments, agencies, NGOs, private companies, and citizens unlock their valuable investments in spatial and observation data at national, regional, community and local levels.

National and international SDIs are a collection of publicly (and sometimes privately) available geospatial data, applications, policies, and standards that assist users find, use, and share geospatial information. An SDI is a distributed ‘system of systems’ based on a high level of interoperability, usually implemented, operated, and maintained by a particular community to support a specific application. The MSDI is an IHO initiative to develop the marine component of an SDI. The MSDI is a framework of suggested best practices and guidance for the management of marine geospatial data, underpinned by key principles such as standards, supporting interoperability, integration, institutional collaboration, and coordination. The FMSDI adds the important dimension of a “federated” model as part of that overarching MSDI framework.

Many governments and organizations have recognized that spatial data and related information collected to produce navigational charts, which help support the safety of navigation, are also useful for many other applications such as the study and management of the ocean, marine and coastal environments. Additional and sometimes duplicate thematic maritime, coastal, marine and oceans data is independently being collected and used. This means that the FMSDI must support a wider and less traditional base of marine data and data users, far beyond just the typical marine navigation community.

Esri Canada has been a leader in supporting SDI efforts in Canada and internationally, including the development of the Canadian Geospatial Data Infrastructure (CGDI) and Canada’s contribution to the international Marine Spatial Data Infrastructure (MSDI) initiative. Esri Canada is an OGC Technical Member and previously successfully participated in the OGC Arctic Spatial Data Pilot conducted in 2017 and the OGC Maritime Limits and Boundaries Pilot conducted in 2019.

In the FMSDI Phase 3 Pilot Project, the Esri Canada team developed a client application (Client 1 (D100)) that supports accessing and organizing FMSDI data to efficiently visualize, manage, analyze, and store the data and Web Service data for a hypothetical ship grounding scenario.

In this scenario, the expedition cruise ship, Discovery, is conducting a 22-day trip from Nome, Alaska, to Kangerlussuaq, Greenland. Discovery is carrying 142 passengers and 104 crew and departs Nome on July 22 at around 1700 hours. She passes through the Bering Strait overnight to her first stop, Kotzebue Bay (pronounced kaat·suh·byoo). She is scheduled to arrive in the area around 0800 hours and will remain there for most of the day, providing an opportunity for passengers to view flora and fauna via shore excursions.

However, early in the morning, the ship had a major power failure that, due to a cascading generator shutdown, caused the vessel to lose propulsion. One of the generators could be restarted within an hour to supply partial electricity to the ship but not before the relatively high winds at the time pushed the ship dangerously close to the shore (Cape Espenberg). Under partial power, the ship tried to navigate the shallow area indicated on the nautical chart, but the ship ran aground. With only partial power to run on-board systems and a low probability of becoming seaworthy, Discovery declares an emergency.

For this Client, Esri Canada used ArcGIS Pro (latest version) software to consume and test the performance and compatibility of the datasets provided by the servers (server 1 and 2) to develop an Arctic Voyage Planning Guide to help the ship captain in search and rescue scenarios.

The Esri Canada Client application provided current information to the ship’s captain and crew based on the data layers included in the Arctic Voyage Planning Guide (AVPG) and other authoritative sources. The client supports the hypothetical scenario of the cruise ship, “Discovery”, which is conducting a 22-day cruise when a power failure occurs and the ship drifts aground near Cape Espenberg area of Kotzebue Bay, Alaska.

Actor(s), end-user(s) and/or stakeholder(s)

Interoperable Technologies

Data and Platforms

Arctic Voyage Planning Guide Themes

Esri Canada incorporated the arctic voyage planning guide using the themes provided by the OGC to be used by the ship captain and the crew for search and rescue. In addition to the AVPG data layers, an additional layer was developed for data specifically related to the ship grounding incident. In addition, an “other” data layer was included for relevant data not directly related to the incident and not included as an AVPG layer.

The technical architecture of the D-100 Client is based on the ArcGIS Pro architecture (https://pro.arcgis.com/en/pro-app/latest/get-started/get-started.htm).

SDIs are evolving and developing. From local and regional data cooperatives to National Spatial Data Infrastructures to application specific SDIs like the FMSDI, the internet and cloud computing are transforming the way organizations manage data and collaborate in a global system of systems. (https://www.esri.com/en-us/arcgis/integrated-geospatial-infrastructure/overview).

The process for developing the Client 1 first used various internet and geospatial catalog searches looking for data or web services related to this application. When applicable data or services were found, they were included in the ArcGIS Pro demonstration client. For support of the servers developed in this pilot project, the web services were integrated into the client simply by adding the web address and indicating the type of web service that was being added.

The interoperable technologies used by the D100 client included the following.

The platform by the D100 client included the following.

The data access by the D100 client included the following.

The architecture wiring diagram of the technical architecture for the D100 Client is provided below.

The following storyboard can help illustrate how users will interact with the client component developed in this deliverable. This storyboard helps to clarify the product objectives and functionalities. This storyboard helps to clarify the product objectives and functionalities. The actions, actors and technologies involved are described in the following table, according to each stage of the demonstration.

It was easy to integrate the OGC compliant data from other servers into ArcGIS Pro. There were no issues in terms of data interoperability and interactions.

The FMSDI D-100 Client demonstration showed how easy it is to implement standards-based web services. The demonstration showed how web service geospatial data can be ingested from various sources via the internet and then how this data can be used in important client applications. Multiple web service standards were demonstrated including WMS, WFS, WMTS, GeoServices REST, and the new OGC API - Features. In addition, multiple IHO S-100 hydrographic and marine data model standards were also shown.

Web services allow implementers to easily employ geospatial functionality over the Internet. Common standards for web services and data models helps make their use straightforward and easy to integrate. Web services underpin the findable, accessible, interoperable, and reusable data principles and D-100 Client highlighting the hypothetical Emergency Management application using standards-based interoperability shows how and why the use of geospatial web service data in geospatial data application is now a necessity.

To provide situational awareness information in the hypothetical scenario to the ship’s captain, crew and passengers, geospatial web services were used to deliver geographic information simultaneously from numerous internet servers to the client map display program.

The D100 web service client was focused on selecting, ingesting and displaying user requested data. The geospatial data was layered and displayed in an organized fashion according to the Arctic Voyage Planning Guide themes with additional themes for information about this specific emergency incident plus a theme for other data layers not defined in the guide. The client functionality includes the following.

The D100 client developed for this project was:

The data layers presented in the D100 client demonstration include the following web services.

In addition, the FMSDI demonstration showed the following layers, which were from FMSDI project published Web Services from the IIC Technologies – Server 1 (D103) at http://18.130.213.16/ogcfmsdi/collections.

The specific point/location of the vessel grounding is also shown with a special yellow, red, and black ship icon symbol.

Helyx Secure Information Systems Ltd is a UK professional services company that covers a wide range of relevant skills and capabilities, specializing in the provision of information management, exploitation, assurance, and GIS solutions and services. Helyx have been a member of the OGC since 2016 and have been involved in previous OGC Testbeds and Pilots. Helyx were also involved in Phase 2 of the FMSDI Pilot, investigating how S-122 Marine Protected Area (MPA) datasets could be used in low-connected environments, developing a web-based client and contributing to the Summary Engineering Report. Several members of Helyx have extensive geographical and environmental backgrounds, and combined with technical knowledge and hands-on experience, recognize the increasing importance of the terrestrial and maritime interface in relation to climate change.

For the FMSDI Pilot Phase 3, which focuses on the maritime and terrestrial domains in the Arctic, Helyx was interested in how users, applying various interoperable technologies, can access, process, and visualize datasets that span these two domains. Additionally, these two domains have traditionally been segregated as the end users generally have different requirements and use cases, as well as varying processes and standards for data capture, processing, storage, and dissemination. Helyx views this as an opportunity to demonstrate how users could use and process data, and a chance to increase the exposure of OGC APIs, IHO standards, and the terrestrial / maritime interface to a growing user base.

As described in the Chapter 7, the overall scenario is based on the expedition cruise ship ‘Discovery’ losing engine power and subsequently running aground off the coast of Kotzebue, Western Alaska. The sub-scenario for the D101 client is centered on an oil spill from the grounded vessel, and determining what anthropogenic and environmental areas may be affected when the vessel is recovered and towed to a suitable port. The sub-scenario therefore occurs some hours-days after the grounding of the vessel has occurred, once the Search and Rescue (SAR) operation is underway or has been completed.

The emphasis is on allowing an end user, who may not be equipped with the appropriate software or processing capabilities, to analyze an event in a web client using data from several disparate sources. Whilst it can be favorable to have all the necessary data stored or hosted on one server, whereby powerful server-side processing and analytics can occur, this is not always possible. For example, an event of interest occurs in a new or unusual Area of Interest (AOI) and there is not the existing datasets or infrastructure in place.

In this sub-scenario the end user utilizes client-side processing to conduct the analysis using available datasets in the region. This end user could be an analyst from a government or non-profit environmental organization, or an analyst from the vessel’s company (or insurance company) who needs to monitor the nature and extent of the oil spill and then identify the potentially disrupted areas when the vessel is recovered.

Actor(s), end-user(s), and/or stakeholder(s)

Platform, Datasets, and Technical Architecture

The web client is built using Leaflet JS, a lightweight and openly available JavaScript library for creating interactive maps. This was chosen for its accessibility, ease of use and the ability to extend the core capabilities using a vast number of plugins. A number of datasets from two participant servers (D103 & D104) were utilized, alongside several other external data providers and web services that were sourced.

The types of technologies, services, and datasets accessed include the following.

Arctic Voyage Planning Guide

Where possible, datasets from across seven common themes were ingested into the web-client (APVG theme and corresponding dataset for the demonstration sub-scenario).

The D101 client utilizes connections to multiple data services, including those from other FMSDI Pilot participants, requesting data by submitting queries. Some datasets are loaded into the client by default, whereas others are only loaded once the user has provided input e.g., drawn an AOI. Various types of services are used, including OGC APIs, other standard geospatial web services, and locally held data (Figure X).

The D101 client demonstrates client-side geoprocessing of datasets from different sources and ingest routes. The table below (D101 client demonstration storyboard) summarizes the sub-scenario in the video demonstration.This storyboard helps to clarify the product objectives and functionalities. The actions, actors and technologies involved are described in the following table, according to each stage of the demonstration.

In general, there were no issues in accessing and integrating the various data services from either the participant servers or external services. Most were served via OGC compliant APIs or as standard geospatial formats which Leaflet can easily ingest. It needs to be noted though that having a disparate number of data sources, with some being held locally, is not the most efficient method for getting data into a client or displaying the relevant metadata associated with each dataset. Additionally, the level of metadata with each dataset varied; therefore, the usefulness of some datasets to an end user may be a challenge if they are unfamiliar with them.

Leaflet is a versatile library; however, many plugins are required to make the most out of the datasets, i.e., geoprocessing capabilities, WMS enhancements, Esri interoperability, etc.

DDIL / Low Network Connectivity

Whilst not the primary focus during this phase of the FMSDI Pilot, the D101 client recognizes the need to account for the fact that network access and connectivity can be difficult, particularly with maritime users or terrestrial users in remote areas. With that in mind, several methods have been employed in the client to minimize the volume of data requested from providers such as the following.

There are various mechanisms relating to spatial data infrastructures, network connectivity, and as covered in Phase 2 of the FMSDI Pilot, the data itself, that could be implemented to reduce the volume of data if this was of primary concern to the end users.

Data fidelity

One noticeable challenge was the limited processing capability of the client when analyzing high fidelity data, as these were typically large datasets with complex shapes. The oil spill dataset in particular:

This therefore took a long time to process in the web client and would occasionally crash. There are some possible workarounds including the following.

Land and Maritime Domains

The area that connects the land maritime domains is crucial for datasets covering them to meet and integrate effectively. However, datasets from these two domains don’t always achieve this, particularly when data has been captured at different times, to different standards, or to varying levels of precision and accuracy. This was almost certainly exacerbated during this Pilot as data was obtained from a wide range of data providers.

Oil Spill Modeling

The processing for the oil spill was deliberately rudimentary as it is a very complex topic with a lot of variables, and the focus for the Pilot was on the data and client-side processing. There is scope for future efforts to work on using a more sophisticated model (such as NOAA’s GNOME) or creating one in-house to better reflect the reality in an oil spill scenario. To do this would require the correct datasets, some of which were not readily available over the scenario AOI. Finally, the buffer method used within the client is generated in all directions as there is not the functionality to determine or emphasize a particular direction. This would again be improved using a more sophisticated model.

Data Availability

Finally, as noted by the other Pilot participants, whilst there were many different data sources available, the actual amount of data situated within the scenario AOI was minimal. This made it difficult to test some of the datasets within the AVPG. A future scenario could occur in an AOI where there are more datasets available, either over a larger geographic extent or several smaller AOIs tied to a master scenario.

Ecere Corporation is a Canadian company developing geospatial software solutions. Ecere implements several standards part of the OGC API family in its GNOSIS Map Server, GNOSIS Software Development Kit, and GNOSIS Cartographer (3D visualization client) software products. Ecere also actively contributes to the OGC API standards themselves, including through a co-editor role for OGC API - Discrete Global Grid Systems (DGGS).

As a participant in the Federated Marine SDI Pilot Phase 3, Ecere focused on the implementation of a DGGS client interacting with the DGGS server provided by the University of Calgary. This close collaboration allowed to validate the current draft OGC API - DGGS candidate standard in the context of practical Technology Integration Experiments (TIEs).

The sub-scenario for the Ecere client consisted in demonstrating the use of DGGS in performing analytics related to coastal flooding and erosion in combination with the DGGS server provided by the University of Calgary, focused on the integration of both terrestrial and marine spatial information. The client enabled visualizing the results in 3D perspective together with additional sources of information provided by other services.

Due to the amount of effort involved in researching and developing support for the DGGS and Icosahedral Snyder Equal Area projection necessary to achieve successful TIEs with the University of Calgary DGGS server, the demonstration of the pilot results for Ecere’s contribution consists mainly in the above detailed description of a proven technical architecture for client/server DGGS interaction, as well as screenshots and a video of Ecere’s GNOSIS Cartographer client visualizing data retrieved from the University of Calgary’s DGGS server in its 3D virtual globe. These screenshots are featured in the following section.

In a real-world scenario, a client could allow a user to independently discover data sources and processes available from one or more OGC API servers, define a workflow integrating these data sources and processes in a meaningful way, request data for its current area and resolution of interest (using e.g., OGC API - DGGS) based on its current viewport configuration which would trigger on-the-fly processing, and quickly display results visually to the user. Demonstrating such interaction involving multiple end-points where a workflow is built starting from data and process discovery to final visualization in the context of a practical scenario would be interesting and insightful future work for a follow-on phase of the Federated Marine SDI Pilot, or for another OGC Innovation Program initiative.

In this section, several additional research and development experiments related to an ISEA3H DGGS based on hexagons (and 12 pentagons) are described on which (very) considerable time and effort was spent, but in the end did not contribute directly to the final results, which relied solely on rhombus grids. However, this work led to valuable and interesting insights as well as familiarity with the ISEA3H Discrete Global Grid System. Details on the challenges leading to taking on this extra work are mentioned in the following section on challenges and lessons learned.

The PYXIS indexing successfully achieves a similar aim to the indexing Ecere attempted to develop (described above), but relying on the very useful property that every child zone centered on a parent zone’s vertex (Vertex Child) always has exactly one parent that is a Centroid Child. This property is not particularly obvious, since three levels of the hierarchy need to be considered, but can be used to easily select the primary parent for Vertex Child zones. The descendants of a root pentagon or hexagon zone form an interesting fractal pattern resembling a snowflake.

Ecere implemented support for this indexing as a simpler and more reliable approach for uniquely indexing zones, as well as a mechanism to encode zone indices in 64 bit identifiers, down to level 24 (with 10 additional levels within the data tiles themselves, resulting in roughly ~6.2 cm / pixel resolution), made of a 5 bits {Level} component, a 5 bits (0-31) {Root} component corresponding to Level 1 (truncated icosahedron) root zones, and a 54 bits {Index} component computed as follows.

The goal of achieving one of the first OGC API - Discrete Global Grid Systems TIEs between different software solutions, each implementing very different Discrete Global Grid Systems prior to the start of this short pilot, presented several challenges.

The most important of these challenges was that an initial server API with which to attempt TIEs, and associated documentation, was only available to Ecere late in the pilot phase. As a result, it was very difficult for Ecere to plan for the best approach in developing its DGGS client component. Significant efforts were spent in different directions, exploring various possibilities that provided valuable insights and familiarity with the research topic, but had no direct impact on the final results demonstration.

A large portion of this extraneous work related to the complexities of dealing with the hexagons (and 12 pentagons) of the ISEA3H DGGS, which can be avoided completely when benefiting from the contrasting simplicity of rhombus grids. The results of these additional experiments are presented in the previous section.

A lesson to take away from this is that although hexagons offer unique interesting sampling characteristics (the hexagon being the regular polygon tiling the plane closest to a circle), they do add a significant amount of complexity compared to square or rhombus tiles. This is precisely why using the ISEA9R DGGS as a simpler transport mechanism for ISEA3H is an appealing proposition.

Another challenge was that the Icosahedral Snyder Equal Area (ISEA) projection is not currently widely supported by popular geospatial software tools. One of these most widely used tools is the PROJ library, which currently only supports the forward projection (from a geodetic latitude, longitude on the WGS84 ellipsoid to the unfolded icosahedron plane), but not the inverse projection (from the unfolded icosahedron plane to a geodetic latitude, longitude on the WGS84 ellipsoid). As a result, the ability for other tools relying on PROJ such as GDAL and QGIS to integrate data from DGGS based on the ISEA projection is very limited. Several operations, such as QGIS visualization and gdalwarp to re-project raster data, require the projection transformation to be implemented in both directions.

A lesson learned is that it should not be very difficult to implement the missing reverse transformation in the PROJ library. An implementation of both the forward and reverse projection in the Java programming language under an MIT license, which was used as starting point for Ecere’s implementation, is available from:

https://github.com/mocnik-science/geogrid/blob/master/src/main/java/org/giscience/utils/geogrid/projections/ISEAProjection.java

A related lesson learned the hard way is that the sphereToIcosahedron() and icosahedronToSphere() functions in this module appear to deal with geocentric latitudes, rather than geodetic latitudes.

In addition to the lack of support in PROJ, there does not seem to be a well-defined standardized identifier (e.g., EPSG code) to refer to the ISEA projection. This in turn makes it difficult to properly geo-reference ISEA-projected data in a GeoTIFF file. The (currently invalid) http://www.opengis.net/def/crs/OGC/0/1534 identifier was used in a provisional manner for the pilot, but referred to the ISEA projection after applying an affine transformation to define a 2D Tile Matrix Set.

Effective communication with the University of Calgary participants on the complex topics of Discrete Global Grid Systems initially also proved to be difficult. Different terminology is sometimes used by different researchers and by the OGC DGGS working groups. As an example, what were previously called “DGGS cells” are now called “DGGS zones” by the DGGS working groups to avoid confusion with the coverage concept of cells. The available documentation on Discrete Global Grid Systems is limited and mostly consists of academic papers, often very dense in content and difficult to understand, sometimes ambiguous to the reader.

After several regular DGGS task-specific meetings and a learning experience by both DGGS participants, by the end of the pilot, a valuable common understanding had emerged. A lesson learned is that more accessible documentation, focused on practical implementation but with an in-depth understanding of a particular discrete global grid system, would be highly valuable, and something towards which the pilot participants would be in a unique position to contribute based on the experience gained.

Another lesson learned is that the current draft candidate OGC API - DGGS Standard is on solid ground to answer typical DGGS questions of the type “What is here?” and “Where is it?”, and that the candidate OGC API - Processes - Part 3: Workflows and Chaining Standard nicely complements these capabilities by allowing to describe workflows integrating multiple OGC API data sources and processes, regardless of where they originate from.

Finally, the key take-away lesson from this pilot’s DGGS activity is that an axis-aligned and equal-area ISEA9R DGGS can be defined and communicated very simply based on a 2D Tile Matrix Set using a CRS constructed by applying an affine transformation to the Icosahedral Snyder Equal Area (unfolded icosahedron plane) projection. This ISEA9R DGGS can serve as a simple transport mechanism for values corresponding to zones of the ISEA3H DGGS’s even hierarchy levels. It should be easy for several existing geospatial tools to enable support for this transport mechanism once the reverse ISEA projection is implemented in the PROJ library, since the mechanism can be described strictly in terms of the widely supported OGC 2D Tile Matrix Set Standard, allowing data to be accessed through OGC API - Tiles as well as OGC API - DGGS.

API extracts data from the source: The most attractive element of the API model for data producers, particularly of marine data, is the retrieval from the authoritative source of the data. This poses questions in the navigational context, obviously, most notably: How can they be assured that data is being used in the correct context, and if any attempts should be made to standardize data import to the bridge in such API formats.

OGC API - Features does not contain any style information. This might not be as simple as having the source service provide the style because different countries and organizations could have different standards or priorities when visualizing the data. Some possible solutions are listed below.

The goal of achieving one of the first OGC API - Discrete Global Grid Systems TIEs between different software solutions, each implementing very different Discrete Global Grid Systems prior to the start of this short pilot, presented several challenges.

The most important of these challenges was that an initial server API with which to attempt TIEs, and associated documentation, was only available to Ecere late in the pilot phase. As a result, it was very difficult for Ecere to plan for the best approach in developing its DGGS client component. Significant efforts were spent exploring various possibilities that provided valuable insights and familiarity with the DGGS, but had no direct impact on the final results demonstration.

A lesson to take away from this is that although hexagons offer unique interesting sampling characteristics (the hexagon being the regular polygon tiling the plane closest to a circle), they do add a significant amount of complexity compared to square or rhombus tiles. This is precisely why using the ISEA9R DGGS as a simpler transport mechanism for ISEA3H is an appealing proposition.

Early in the pilot it was decided to encode DGGS data into a GeoTIFF format. During the process, it was determined that the rhombus tiling schema can be constructed to conform with the OGC API - Tiles standard, and can be described in a mechanism suitable for GDAL and popular GIS tools, so that other non-DGGS clients can access these data tiles.

Another challenge was that the Icosahedral Snyder Equal Area (ISEA) projection is not currently widely supported by popular geospatial software tools. One of these most widely used tools is the PROJ library, which currently only supports the forward projection (from a geodetic latitude, longitude on the WGS84 ellipsoid to the unfolded icosahedron plane), but not the inverse projection (from the unfolded icosahedron plane to a geodetic latitude, longitude on the WGS84 ellipsoid).

A lesson learned is that it should not be very difficult to implement the missing reverse transformation in the PROJ library. An implementation of both the forward and reverse projection in the Java programming language under an MIT license, which was used as starting point for Ecere’s implementation, is available from:

Related to this is that the sphereToIcosahedron() and icosahedronToSphere() functions in this module appear to deal with geocentric latitudes, rather than geodetic latitudes.

A lesson learned is that more accessible documentation, focused on practical implementation but with an in-depth understanding of a particular discrete global grid system, would be highly valuable, and something towards which the pilot participants would be in a unique position to contribute based on the experience gained.

There were DGGS API Server limitations. While a process-based analysis is desirable, the current structure of the DGGS Server has focused on the discovery-based aspects of DGGS capabilities. The implemented OGC API - DGGS does expose some of this functionality, but it assumes the client has detailed knowledge of the DGGS geometry, including tessellations, refinement, indexing / encoding, etc. The DGGS API implemented only allows access to DGGS data through the PYX0 format and static PNGs.

Another challenge while implementing the OGC API - DGGS was the lack of clear documentation in the OGC website. If there are still two approaches being discussed for the OGC API - DGGS, it should be clearly stated in the documentation. If a decision has not yet been made, it should not be published on the website, or it should include a warning.

A key take-away lesson from this pilot’s DGGS activity is that an axis-aligned and equal-area ISEA9R DGGS can be defined and communicated very simply based on a 2D Tile Matrix Set using a CRS constructed by applying an affine transformation to the Icosahedral Snyder Equal Area (unfolded icosahedron plane) projection. This ISEA9R DGGS can serve as a simple transport mechanism for values corresponding to zones of the ISEA3H DGGS’s even hierarchy levels. It should be easy for several existing geospatial tools to enable support for this transport mechanism once the reverse ISEA projection is implemented in the PROJ library, since the mechanism can be described strictly in terms of the widely supported OGC 2D Tile Matrix Set Standard, allowing data to be accessed through OGC API - Tiles as well as OGC API - DGGS.

This phase of the pilot has shown that the draft candidate OGC API - DGGS standard is on solid ground with regards to answering typical DGGS questions of “What is here?” and “Where is it?”, and that the candidate OGC API - Processes - Part 3: Workflows and Chaining Standard nicely complements these capabilities by allowing to describe workflows integrating multiple OGC API data sources and processes, regardless of where they originate.

Public Web Services for the project area of interest in Kotzebue Bay, Alaska that were discovered for the FMSDI project are as follows:

From NWS Integrated Dissemination Program (IDP) GIS Services Showcase - https://noaa.maps.arcgis.com/apps/MapSeries/index.html?appid=cf93575e5535467199da7358ee6c825c

From NOAA NowCOAST Web Mapping Portal - https://nowcoast.noaa.gov/

From State of Alaska Open Data Portal https://gis.data.alaska.gov/pages/imagery

From Maritime Cadastre National Viewer https://marinecadastre.gov/nationalviewer/

From FEMA Web Services - https://gis.fema.gov/arcgis/rest/services/FEMA

From NOAA Charttools - https://gis.charttools.noaa.gov/arcgis/rest/services

From Marineregioins.org Portal - https://www.marineregions.org/downloads.php

From Esri ArcGIS Maritime Server - https://enterprise.arcgis.com/en/server/10.8/publish-services/windows/what-is-a-maritime-chart-service.htm

From OceanWise - https://www.oceanwise.eu/data/enc-wms/

From Lifewatch Metadata Catalog - https://metadatacatalogue.lifewatch.eu/srv/api/records/5ec6b072-9e83-433b-b293-e0f75a714e34

Ecere visualized in its GNOSIS Cartographer 3D client the results of a coastal erosion workflow from the University of Calgary’s DGGS server which integrated four datasets:

In addition, Ecere also used the following datasets for its visualization demonstration:

The collection object uses the GeoJSON standard of “Feature Collection” and consists of no other properties or attributes, merely a (possibly empty) array of feature objects. These feature objects (“feature” in this sense is the JSON definition of “feature”, not the S-100 GFM definition of feature. Where the difference is not obvious in context it has been defined accordingly.)represent either S-100 GFM feature types or information types. Example:

Currently no dataset metadata is included in the FeatureCollection – this could be implemented and should be modeled on the S-10 Part 10b encoding for simplicity. The FeatureCollection only acts as an aggregation operator for S-100 features and information types so there is no requirement for it where only a single feature (or information type) is being encoded.

An example of an individual feature object is shown below:

The feature object contains the following sub-objects.

As yet, no formal mapping from S-100 Feature Catalogs to JSON Schemas is defined. This is an ongoing work in progress but, once complete, should provide a normative method for deriving a schema against which S-100 JSON data can be validated.

The properties sub-object contains the thematic attribution for each feature. This is a direct mapping from the names and values defined by the S-100 feature catalog. Only defined attributes are included and they are referenced by the Feature Catalog “code” value, according to the feature catalog defined for the product specification (and defined by the S-100 XML Schemas for feature catalogs). This is similar in structure to the S-100 Part 10b GML encoding.

No ordering is imposed on the JSON data, unlike GML Schemas defined under Part 10b

Sub-Attributes are defined as sub-objects, for example:

Here the subattribute endDate (defined by the feature catalog) is implemented as a sub-object f the fixedDateRange attribute (also defined by the feature catalog schema)

Where the feature catalog allows for multiplicity > 1 the individual values are arranged in a JSON array, for example:

Here communicationChannel has multiple values (as the feature catalog has a multiplicity >1) and so values are implemented as an array. There is a possible ambiguity here as a single value for an attribute with multiplicity>1 could be implemented as an array with one value or as a single value. The implementation uses a single value instead of a single-valued array in order to increase interoperability with GIS and spatial tools but this probably requires further experimentation and discussion.

JSON supports a restricted set of types, and therefore only the following explicit mappings are defined.

This means that the JSON encoding is, with the exclusion of boolean and integer types, effectively untyped. This is not necessarily a problem and has not hindered takeup of JSON forms in the broader geospatial community. It will remain to be seen whether this causes a problem . It is not dissimilar to the S-100 PArt 10a ISO8211 encoding where attribute and sub-attribute values are encoded as binary strings, regardless of their S-100 GFM type - these types can be enforced by S-100 data editors, rather than at the encoding level.

Geometry can be directly implemented as GeoJSON geometry objects. Point, LineString and Polygon using coordinates defined in an appropriate CRS can be directly embedded in GeoJSON objects. The suggestion is to use inline coordinates using the simple geometry types allowable in GeoJSON. The big gap is, of course, that S-100 topology is not supported under GeoJSON as geometric primitives can not be indirectly referenced. This means all geometry must be inline and shared geometry (and, by implication, attributes on spatial features) is not supported. These features are not universally used - it is suggested that more complex spatial types can be supported by other encodings (GML or ISO8211 under Part 10b and Part 10a of S-100).

The hardest element of the S-100 GFM to implement is the relationships between features and information types. JSON does not currently implement a standardized way of referencing between objects in a datasets - currently there is a $ref protocol that can be used for such things, and the encoding uses unique identifiers for all features. The S-100 GFM names both relationships, and the individual roles using unique codes and so it seems a referencing mechanism could be finalized reasonably simply. This is an ongoing area, the pygeoapi implementation uses $ref to implement this as a first cut.