Publication Date: 2019-01-20

Approval Date: 2018-11-12

Submission Date: 2018-09-21

Reference number of this document: OGC 18-084

Reference URL for this document: http://docs.opengeospatial.org/per/18-084

Category: Public Engineering Report

Editor: Luis Bermudez

Title: OGC Geospatial to the Edge Plugfest Engineering Report


OGC Engineering Report

COPYRIGHT

Copyright © 2019 Open Geospatial Consortium. To obtain additional rights of use, visit http://www.opengeospatial.org/

WARNING

This document is not an OGC Standard. This document is an OGC Public Engineering Report created as a deliverable in an OGC Interoperability Initiative and is not an official position of the OGC membership. It is distributed for review and comment. It is subject to change without notice and may not be referred to as an OGC Standard. Further, any OGC Engineering Report should not be referenced as required or mandatory technology in procurements. However, the discussions in this document could very well lead to the definition of an OGC Standard.

LICENSE AGREEMENT

Permission is hereby granted by the Open Geospatial Consortium, ("Licensor"), free of charge and subject to the terms set forth below, to any person obtaining a copy of this Intellectual Property and any associated documentation, to deal in the Intellectual Property without restriction (except as set forth below), including without limitation the rights to implement, use, copy, modify, merge, publish, distribute, and/or sublicense copies of the Intellectual Property, and to permit persons to whom the Intellectual Property is furnished to do so, provided that all copyright notices on the intellectual property are retained intact and that each person to whom the Intellectual Property is furnished agrees to the terms of this Agreement.

If you modify the Intellectual Property, all copies of the modified Intellectual Property must include, in addition to the above copyright notice, a notice that the Intellectual Property includes modifications that have not been approved or adopted by LICENSOR.

THIS LICENSE IS A COPYRIGHT LICENSE ONLY, AND DOES NOT CONVEY ANY RIGHTS UNDER ANY PATENTS THAT MAY BE IN FORCE ANYWHERE IN THE WORLD. THE INTELLECTUAL PROPERTY IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NONINFRINGEMENT OF THIRD PARTY RIGHTS. THE COPYRIGHT HOLDER OR HOLDERS INCLUDED IN THIS NOTICE DO NOT WARRANT THAT THE FUNCTIONS CONTAINED IN THE INTELLECTUAL PROPERTY WILL MEET YOUR REQUIREMENTS OR THAT THE OPERATION OF THE INTELLECTUAL PROPERTY WILL BE UNINTERRUPTED OR ERROR FREE. ANY USE OF THE INTELLECTUAL PROPERTY SHALL BE MADE ENTIRELY AT THE USER’S OWN RISK. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR ANY CONTRIBUTOR OF INTELLECTUAL PROPERTY RIGHTS TO THE INTELLECTUAL PROPERTY BE LIABLE FOR ANY CLAIM, OR ANY DIRECT, SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES, OR ANY DAMAGES WHATSOEVER RESULTING FROM ANY ALLEGED INFRINGEMENT OR ANY LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR UNDER ANY OTHER LEGAL THEORY, ARISING OUT OF OR IN CONNECTION WITH THE IMPLEMENTATION, USE, COMMERCIALIZATION OR PERFORMANCE OF THIS INTELLECTUAL PROPERTY.

This license is effective until terminated. You may terminate it at any time by destroying the Intellectual Property together with all copies in any form. The license will also terminate if you fail to comply with any term or condition of this Agreement. Except as provided in the following sentence, no such termination of this license shall require the termination of any third party end-user sublicense to the Intellectual Property which is in force as of the date of notice of such termination. In addition, should the Intellectual Property, or the operation of the Intellectual Property, infringe, or in LICENSOR’s sole opinion be likely to infringe, any patent, copyright, trademark or other right of a third party, you agree that LICENSOR, in its sole discretion, may terminate this license without any compensation or liability to you, your licensees or any other party. You agree upon termination of any kind to destroy or cause to be destroyed the Intellectual Property together with all copies in any form, whether held by you or by any third party.

Except as contained in this notice, the name of LICENSOR or of any other holder of a copyright in all or part of the Intellectual Property shall not be used in advertising or otherwise to promote the sale, use or other dealings in this Intellectual Property without prior written authorization of LICENSOR or such copyright holder. LICENSOR is and shall at all times be the sole entity that may authorize you or any third party to use certification marks, trademarks or other special designations to indicate compliance with any LICENSOR standards or specifications.

This Agreement is governed by the laws of the Commonwealth of Massachusetts. The application to this Agreement of the United Nations Convention on Contracts for the International Sale of Goods is hereby expressly excluded. In the event any provision of this Agreement shall be deemed unenforceable, void or invalid, such provision shall be modified so as to make it valid and enforceable, and as so modified the entire Agreement shall remain in full force and effect. No decision, action or inaction by LICENSOR shall be construed to be a waiver of any rights or remedies available to it.

None of the Intellectual Property or underlying information or technology may be downloaded or otherwise exported or reexported in violation of U.S. export laws and regulations. In addition, you are responsible for complying with any local laws in your jurisdiction which may impact your right to import, export or use the Intellectual Property, and you represent that you have complied with any regulations or registration procedures required by applicable law to make this license enforceable.

Table of Contents

1. Summary

The Geospatial to the Edge Interoperability Plugfest, co-sponsored by the Army Geospatial Center and the National Geospatial-Intelligence Agency (NGA/CIO&T), brought together technology implementers and data providers to advance the interoperability of geospatial products and services based on profiles of OGC standards. Specifically, servers and data available via GeoPackage, Web Feature Service (WFS), Web Map Service (WMS), and Web Map Tile Service (WMTS), all following National System for Geospatial Intelligence (NSG) profiles, were exercised and improved in various clients. Compliance Tests were executed and advanced based on feedback from the participants.

1.1. Motivation

The geospatial communities supporting defense, emergency response, and intelligence rely on geospatial data and open standards to accomplish their missions. Profiles of standards and data models were used to ensure sharing of data meet their specific needs. Profiles provide strict implementation guidance to ensure interoperability of geospatial systems and data in these highly specialized and demanding environments. Implementations following profiles compliant with open standards support mission critical operations for enhancing effective and efficient execution.

A Plugfest is an initiative of the OGC Innovation Program. Plugfests provides the right venue for sponsors and technology implementers to come together in a collaborative agile process to solve geospatial challenges. The Plugfest assisted tool enhancement and provided guidance to improve the delivery of enterprise geospatial data to end users. In this initiative, a Plugfest was used to bring more than thirteen data/service producers and clients of data following NSG profiles. The plugfest help discover implementation issues and advance executable test suites.

1.2. Prior-after comparison

Before the Plugfest, very few implementations were able to interact with NSG profiles of OGC standards. This is commonly the case when communities want to restrict a rule from the base standard or want to extend what the base standard offers. Support for NSG profiles is not commonly a feature that comes packaged in software products. After the Plugfest more implementations became available that implement the NSG profiles for GeoPackage, WMS, WFS, and WMTS.

The profiles implemented in the Plugfest had corresponding executable test suites. These profiles test suites were in beta by the end of the initiative, ready to be move forward for public release by the OGC Technical Committee. Feedback related to the executable test suites was provided by the participants. In particular, the GeoPackage test was improved during the Plugfest.

1.3. Recommendations for future work

Plugfests should be performed for any new profile of an existing OGC standard or any new candidate OGC standard including revisions. Allowing participants to come together to solve interoperability issues has high value in maturing and stress testing implementations of profiles or candidate standards. The result, of high value to the geospatial community, is improvement of the standards and advancement of test suites.

Recommendations for implementing the profiles in regards to the specific profiles are detailed in the Results and Recommendations Section.

1.4. Document contributor contact points

All questions regarding this document should be directed to the editor or the contributors:

Contacts

Name Organization

Luis Bermudez

OGC

1.5. Foreword

Attention is drawn to the possibility that some of the elements of this document may be the subject of patent rights. The Open Geospatial Consortium shall not be held responsible for identifying any or all such patent rights.

Recipients of this document are requested to submit, with their comments, notification of any relevant patent claims or other intellectual property rights of which they may be aware that might be infringed by any implementation of the standard set forth in this document, and to provide supporting documentation.

2. References and abbreviated terms

2.1. References

The following normative documents are referenced in this document.

2.2. Abbreviated terms

  • CSV Comma-separated values

  • DoD United States Department of Defense

  • ESA European Space Agency

  • GEOINT Geospatial Intelligence

  • GGDM Ground-Warfighter Geospatial Data Model

  • GMES Global Monitoring for Environment and Security

  • GML Geography Markup Language

  • IC Intelligence Community

  • JSON JavaScript Object Notation

  • NAS NSG Application Schema

  • NSG National System for Geospatial Intelligence

  • NSGM National System for Geospatial-Intelligence Manual

  • OGC Open Geospatial Consortium

  • WFS Web Feature Service

  • WMS Web Map Service

  • WMTS Web Map Tile Service

3. Overview

This OGC Plugfest, co-sponsored by Army Geospatial Center and the National Geospatial-Intelligence Agency (NGA/CIO&T), brought together technology implementers and data providers to advance the interoperability of geospatial products and services based on community profiles of existing OGC standards . The Plugfest assisted in tool enhancement and provided guidance to improve the delivery of enterprise geospatial data to end users.

Examples of how end user communities will benefit from this work are:

  • First responders, relief workers and fire fighters preparing for and operating in austere network environments;

  • Emergency planners and managers supporting hurricane, wildfire, and earthquake preparedness, relief/response activities and damage assessment; and

  • Soldiers/warfighters during planning and executing operations specifically in disconnected, intermittent, and limited network environments.

The geospatial communities supporting defense, emergency response, and intelligence rely on geospatial data and open standards to accomplish their mission. To make sharing of data meet their specific needs, they require the definition, testing, and community approval of profiles of existing OGC standards. Profiles provide strict implementation guidance to ensure interoperability of geospatial systems in these highly specialized and demanding environments. Non-compliance to open standards profiles prohibits mission critical operations from executing effectively and efficiently.

Note
The OGC membership recently approved an official definition of the concept ”profile” that includes the case in which a profile extends the base standard. NSG Profiles are of the latter type of profile and in the future these types of profiles should be referred as Profiles with Extension. Profiles restrict (remove) options (e.g., file formats or coordinate reference systems) that may be specified in the base standard. GML Simple Features is an example of a profile that restricts the number of requirements to be implemented. Extensions add options and/or capabilities that are not included in the base standard.

Additionally, members of the Intelligence Community (IC), United States Department of Defense (DoD), non-DoD/IC Federal agency members of the NSG, international partners, state/local municipalities, and Native American tribal organizations that are responsible for the operation, acquisition and/or development of systems and applications which collect, procure, produce, serve, exchange, or use Geospatial Intelligence (GEOINT) data are mandated to comply with NSG implementation standards (NSGM 3202). Support of these standards, which may be profiles of OGC standards, affect government acquisition decisions to ensure that all systems within the government can communicate appropriately. [From GEOINT Functional Manager Standards Assessment (GFMSA) Program Manual, NSGM 3202, June 2016]

A Plugfest, an initiative of the OGC Innovation Program, provides a collaborative venue for sponsors and technology implementers to come together in a agile process to solve geospatial challenges. A plugfest provides the scenario and testing environment to advance implementation of profiles in commercial and open source software products. A Plugfest allows organizations to test and validate that their software products can interoperate with other products implementing the same standard or profiles of a standard.

4. Plugfest description

4.1. Goal

The goal of this OGC Plugfest was to increase interoperability of information systems using community defined profiles. The Plugfest assisted in tool enhancement and provided guidance to improve the delivery of enterprise geospatial data to end users.

4.2. Profiles

The profiles used in the initiative were:

4.3. Plugfest participation roles

The participants in the Plugfest took the following roles:

  • Data Providers were organizations providing data sources either serving it via OGC services or providing file.

  • Service Providers were organizations that produced services for clients to access.

  • Client Providers were organizations that provided clients that consumed GeoPackage data and services to be used for a specific purpose.

4.4. Sprints

Sprints happened virtually allowing participants around the world to participate and to minimize costs related to travel. Each sprint lasted for 1 week. During the sprint week participants were available to respond to inquiries posted by other participants. In the sprint time clients performed a set of operations following a scenario and documented their success and failures.

Two sprints were executed in this Plugfest.

Sprint 1 tested existing commercial and/or open source products against the scenarios. A limited (not for public dissemination) report on Sprint 1 findings, not revealing vendor information, was shared with the Sponsors and relevant OGC Standards Working Groups. The findings that include the need for improvement in data structure, servers and clients were documented.

Sprint 2 Occurred 3 months after Sprint 1 was completed and documented. Sprint 2 repeated the tests performed in Sprint 1. Participants had three months to improve their software to better create the GeoPackage files and improve their services and clients based on the feedback from Sprint 1.

4.5. Schedule

  • March 6 - Participants kickoff (Pre-sprint clarification open to registered participants)

  • March 30 - Update scenarios

  • April 16 - Information about sources (data and servers) was provided to the OGC

  • April 20 - Data and Servers ready for Sprint 1

  • April 23-27 - Sprint 1 clients testing

  • April 30-May 11 - Compilation of results by OGC staff

  • April 30-July 16 - Discussions in GitHub issue tracker

  • July 16 - Information about data and servers was provided to the OGC

  • July 20 - Data and Servers ready for Sprint 2 (Improved services based on feedback from Sprint 1)

  • July 23-Aug 2 - Sprint 2 clients testing

  • September 9 - Draft engineering report

  • September 28 - Report submission to the OGC TC for public release approval

4.6. Data

Two types of geospatial data, based on NSG Profiles, were used in the plugfest:

  • Vector data used to create, publish and ingest GeoPackage files as well as to publish to and ingest data from WFS servers.

  • Raster data used to exercise WMS and WMTS servers.

4.6.1. Vector data

Vector data used in the Plugfest was based on the Ground-Warfighter Geospatial Data Model (GGDM 3.0), which is based on the NSG Application Schema (NAS) version 7. Various files were created using the GGDM 3.0 schema, which were used by participants to create GeoPackages or vector files that were served via Web Feature Service (WFS) service instances.

The NAS Model Entity Catalog provides a set of features, attributes and enumeration values to be used when encoding vector information. The catalog was provided as an Excel File. It was used by participants to better understand the semantics of the feature types (semantics) of the data.

The was area if interest for the data was based on Puerto Rico. The datasets contained the following layers.

  • USGS Puerto Rico data for roads and trails, airports, rivers, selected buildings, built-up areas, and some surface areas including runways, counties, a forest, cemeteries and conservation areas.

  • US Transportation Data from which Heliport and Water Aerodrome points extracted from airport data and ports and anchorages extracted from existing port data.

  • US Maritime data for maritime limits restricted to the Puerto Rico area.

  • US Census Bureau coastlines restricted to land/water boundaries for the Puerto Rico area.

The Data was available for download in the following formats:

The list of features types contained in the source files was as follows:

  • ADMINISTRATIVE_SUBDIVISION_S

  • BUILDING_P

  • BUILT_UP_AREA_P

  • CEMETERY_S

  • CONSERVATION_AREA_S

  • DAM_C

  • DAM_S

  • DATASET_S

  • DOLPHIN_P

  • ENTITY_COLLECTION_METADATA_S

  • FORESHORE_S

  • FOREST_S

  • GAUGING_STATION_P

  • HELIPORT_P

  • INLAND_WATERBODY_S

  • LAND_AERODROME_P

  • LAND_WATER_BOUNDARY_C

  • MARITIME_LIMIT_C

  • MILITARY_INSTALLATION_S

  • NAVIGABLE_CANAL_S

  • PARK_S

  • PIPELINE_C

  • PORT_P

  • REEF_C

  • RIVER_C

  • RIVER_S

  • ROAD_C

  • ROCK_FORMATION_P

  • RUNWAY_S

  • SOIL_SURFACE_REGION_S

  • TRAIL_C

  • TUNNEL_C

  • WATER_AERODROME_P

  • WATER_WELL_P

4.6.2. Raster data

The raster data for the plugfest was based on the Sentinel 2A Multispectral Instrument (MSI), which was made available by the European Space Agency (ESA) within the Global Monitoring for Environment and Security (GMES) programme. The true color composites (red, green, blue) from the orthorectified Level-lC products were used to generate map data for WMS, WMTS, and GeoPackage files.

The Sentinel data are freely available through the Copernicus Open Access Hub. The data used in the Plugfest corresponded to the region of Puerto Rico and the True Color Image (TCI).

The Table below lists the subset Product ID’s from the Sentinal 2A mission that were used by the data providers. From each image set, the *TCI.jp2 image was used to create the output products. Participants used the Copernicus Hub recommend API script to download each dataset.

Table 1. Sentinel 2 Product IDs
Product ID Unique ID (API access)

S2B_MSIL1C_20171209T150709_N0206_R082_T19QFA_20171209T195400

a6a9d67d-fbd5-47be-b5c7-92d680b5028b

S2B_MSIL1C_20171209T150709_N0206_R082_T19QGA_20171209T195400

2c6a75a4-7327-45b0-b493-ea9a40982b13

S2B_MSIL1C_20171209T150709_N0206_R082_T19QGV_20171209T195400

2590351c-a1ae-4592-9b3d-83358d8b13f1

S2B_MSIL1C_20171209T150709_N0206_R082_T19QHA_20171209T195400

87f334c4-1993-409a-bd46-79a58a8ba243

S2B_MSIL1C_20171209T150709_N0206_R082_T19QHV_20171209T195400

96c5aee0-68d9-4c11-8182-e78b8adca7c1

S2A_MSIL1C_20161219T150712_N0204_R082_T19QFV_20161219T150714

31d6900f-3164-4243-84f8-84d39982a4fe

After setting up an account, the unique id can be plugged in the URL string to form the link for download. For example:

https://scihub.copernicus.eu/dhus/odata/v1/Products('a6a9d67d-fbd5-47be-b5c7-92d680b5028b')/$value

After downloading the data the participants were responsible for the merge and tiling of this imagery data as per the NSG specifications for raster data.

4.7. Data consumer testing reports during sprints

4.7.1. Organizations acting as clients/users

The following organizations acted as clients/users of the Plugfest.

  • AGC-Nett Warrior

  • AGC-Sitaware

  • Distributed Common Ground System–Army (DCGSA)

  • Compusult

  • Envitia

  • Esri

  • Image Matters

  • U.S. Naval Research Laboratory (NRL)

  • Visual Awareness Technology and Consulting (VATC)

4.7.2. Data, services and templates

The Data and Services were provided by:

  • AGC-ENFIRE

  • Aviation and Missile Research Development and Engineering Center (AMRDEC)

  • Compusult

  • Distributed Common Ground System–Army (DCGSA)

  • Esri

  • GeoSolutions

  • U.S. Naval Research Laboratory (NRL)

  • Visual Awareness Technology and Consulting (VATC)

The name of the sources were anonymized. The links and templates used in Sprint 1 are detailed in the table below.

Source type Source short name (with link) Template used to report

GeoPackage_Vector

GeoPackage_Vector_Apollo

GeoEdgePlugfest-S1-GeoPackageVector-sourceName-orgName

GeoPackage_Raster

GeoPackage-Raster_Jupiter

GeoEdgePlugfest-S1-GeoPackageRaster-sourceName-orgName

WMS

WMS_Mercury

GeoEdgePlugfest-S1-WMS-sourceName-orgName

WMS

WMS_Mars

GeoEdgePlugfest-S1-WMS-sourceName-orgName

WMTS

WMTS_Pluto

GeoEdgePlugfest-S1-WMTS-sourceName-orgName

WMTS

WMTS_Oberon

GeoEdgePlugfest-S1-WMTS-sourceName-orgName

The links and templates used in Sprint 2 are detailed in the table below.

Source type & Source short name Template to provide feedback

GeoPackage_Vector_Apollo

GeoEdgePlugfest-S2-GeoPackageVector-sourceName-orgName.doc

GeoPackage_Vector_Jupiter

GeoEdgePlugfest-S2-GeoPackageVector-sourceName-orgName.doc

[GeoPackage_Vector_Rigel]

GeoEdgePlugfest-S2-GeoPackageVector-sourceName-orgName.doc

GeoPackage_Vector_Orion

GeoEdgePlugfest-S2-GeoPackageVector-sourceName-orgName.doc

GeoPackage-Raster_Apollo

GeoEdgePlugfest-S2-GeoPackageRaster-sourceName-orgName.doc

GeoPackage-Raster_Jupiter

GeoEdgePlugfest-S2-GeoPackageRaster-sourceName-orgName.doc

GeoPackage-Raster_Orion

GeoEdgePlugfest-S2-GeoPackageRaster-sourceName-orgName.doc

WMS_Mercury

GeoEdgePlugfest-S2-WMS-sourceName-orgName.doc

WMS_Mars

GeoEdgePlugfest-S2-WMS-sourceName-orgName.doc

WMTS_Pluto

GeoEdgePlugfest-S2-WMTS-sourceName-orgName.doc

[WMTS_Calypso]

GeoEdgePlugfest-S2-WMTS-sourceName-orgName.doc

WMTS_Oberon

GeoEdgePlugfest-S2-WMTS-sourceName-orgName.doc

WFS_Janus

GeoEdgePlugfest-S2-WFS-sourceName-orgName.doc

WFS_Neptune

GeoEdgePlugfest-S2-WFS-sourceName-orgName.doc

4.7.3. Communications

If there were any issues with a test, the questions were logged to the issue tracker. Participants were encouraged to watch the repository during the sprint so they would be notified and be able to provide comments if a question from another participant came up.

4.8. Templates

Each user (client) had to test a data or server and then answer a set of questions. The questions for the vector and raster sources are summarized in this section.

4.8.1. Vector questions

Inland water body query

Find all inland water bodies where the full name starts with 'Lago’ and the highest elevation is greater than 70.

Reservoir query

Find all inland water bodies where the inland water type is reservoir and the area is greater than 0.046.

Trail Number query

Provide the full names of all trails that have specified domain value attribute containing the string subset: TraillNumber:T300.

Linear Rivers query

Find all linear rivers where the full name ends with 'de la Plata'.

Guaynabo query

Find all built up areas where the height is less than 35, the memorandum is ‘San Juan’, and the World Port Index Identifier is 'Guaynabo'.

Firefighting Carretera query

Find all buildings where the address does not contain Carretera, the feature function is firefighting, and the specified domain value starts with '(Zipcode:006'.

S1200 query

Find all roads where the geography name is 'Pr- 20', and where the feature unique identifier is S1200.

Conservation area query

Find the largest conservation area based on area and report the full name of that conservation area. What is the full name? What is the Area? Provide a screenshot.

ICAO query

Find the ICAO Location Indicator for the heliport located at the Bayamon Rgnl Hospital. What is the location indicator? Provide a screenshot.

Subdivision query

Find the administrative subdivision that contains the building “Cuerpo de Bomberos de Orocovis”. What is the name? Provide a screenshot?

4.8.2. Raster questions

Provide screenshot for the zooms (and scales) specified.

Note: In the below requests, the center point of a designated area is identified. The participant should go to the center point and then zoom to the designated scale and take a screen capture of the resulting image. The screen capture should be bigger (contain) than the image returned to ensure that we will be able to compare images returned by different clients.

Zoom to full extent of the layer
Scale: 1:500,000

The extent of the image returned should be centered on the centroid of Puerto Rico, which is approximately this location: EPSG 4326: - 66.66, 18.20 and then zoom to the 1:500,000 scale. If the designated scale is not available, zoom to the closest scale that is available and include that information along with the image.

Scale: 1:20,000

The extent of the image returned should be centered on the centroid of Puerto Rico, which is approximately this location: EPSG 4326: - 66.66, 18.20 and then zoom to the 1:20,000 scale. If the designated scale is not available, zoom to the closest scale that is available and include that information along with the image.

5. Results and recommendations

5.1. Summary of experiments

The results of Sprint 1 were summarized in the following table.

7 results afe2b
Figure 1. Results Sprint 1

The first row provides the label used to identify each client in an anonymized way. For example: Client E, Client L, etc. In total, ten clients were used in the Plugfest.

The first column identifies the sources of data and servers. The contributions were also anonymized. They were sorted in the table by the type of contribution. For example, the Plugfest had 3 WMTS servers: Pluto, Calypso, and Oberon.

The red marks indicate that the client was not able to interact with the server provider. Overall most servers were able to communicate with the clients.

The results for Sprint 2 are shown in the Sprint 2 Results table. Similar to the previous table, it shows if a client was able to interact with the server provider. In addition, it was also captured by shading the cells green if the client was able to successfully perform all the tests with a particular source.

7 results 503ea
Figure 2. Sprint 2 Results

5.2. Selected screenshots

In the following figures are some examples provided by the participants.

ClientA gpk apollo raster Q1
Figure 3. Full extent raster data from GeoPackage with Client A and GeoPackage Vector Apollo
ClientA wms mercury Q1
Figure 4. Full extent raster data from WMS with Client A and WMS Mercury
ClientL wms mercury Q1
Figure 5. Full extent raster data from WMS with Client L and WMS Mercury
ClientH wms mercury Q4
Figure 6. 1:25000 extent raster data from WMS with Client H and WMS Mercury
ClientG wms mercury Q4
Figure 7. ~1:25000 extent raster data from WMS with Client G and WMS Mercury
ClientG wms mars vector Q4
Figure 8. Vector and Raster data from WMS Mars by Client G
raster vector gpk
Figure 9. Client showing both GeoPackage vector and raster data

5.3. WMS

5.3.1. Use both name and title for naming layers

A server didn’t use the <Name> element to name the layer. Clients were not able to do GetMap request. The WMS 1.3 Specification states the role of <Title> and <Name>:

7.2.4.2 Names and titles
A number of elements have both a <Name> and a <Title>. The Name is a text string used for machine-to-machine communication while the Title is for the benefit of humans. For example, a dataset might have the descriptive Title “Maximum Atmospheric Temperature” and be requested using the abbreviated Name “ATMAX”.

Related issues:

5.4. WFS

5.4.1. Axis order

Data providers should treat properly urn:ogc:def:crs:epsg::4326. The axis order is Latitude Longitude. This can happen in the following situations:

  1. When performing an HTTP request

  2. When returning data about a feature

The snippet code bellow shows an incorrect axis order for Puerto Rico.

Example Incorrect XML Return
<gml:MultiSurface     gml:id="INLAND_WATERBODY_S.5.pl" srsName="urn:ogc:def:crs:EPSG::4326" srsDimension="3"> <gml:surfaceMember> <gml:Polygon gml:id="INLAND_WATERBODY_S.5.pl.0" srsName="urn:ogc:def:crs:EPSG::4326" srsDimension="3"> <gml:exterior> <gml:LinearRing> <gml:posList>
  -66.57471640699998 18.36690177100007 0
  -66.57475487399995 18.36692017100006 0
  -66.57479807399994 18.36692930400005 0
  -66.57483167399994 18.36693850400007 0
  -66.57488447399999 18.36694770400004 0
  -66.57494700699993 18.36696597100007 0

5.4.2. Query issues

Some queries seem difficult to execute. Several clients reported that Query 10 can’t be executed:

Query 10: Find the administrative subdivision that contains the building “Cuerpo de Bomberos de Orocovis”

The process might involve a 2 step process:

  1. Select the layer

  2. Run a query inside that layer: select * from BUILDING_P WHERE ZI005_FNA="Cuerpo de Bomberos de Orocovis";

q10 2steps
Figure 10. Query 10 Client View

Similarly it was reported that it was only possible to query one layer per request (Issue 52).

A thin client didn’t support attribute querying (Issue 85).

Related issues:

5.4.3. Interacting with different versions WFS

Some servers support multiple version of WFS. This server capability enables clients to get the data using the preferred version of a WFS.

5.5. WMTS recommendations

5.5.1. Validate GetCapabilities document

Service providers should verify that the GetCapabilities document validates against the XML Schema.

5.6. GeoPackage results

5.6.1. Raster and vector data in one file

The GeoPackage standard and NSG profile permit a user to include both raster and vector data in the same file. One provider combined the GGDM vector data with a patch of imagery at a higher resolution than the supplied Sentinel 2A data and also added an elevation raster dataset. Although their demo exceeded the scope of the prescribed tests, it highlighted the potential for doing useful raster/vector mashups within NSG profile guidelines. This screenshot shows the edge of the imagery patch overlaid on a public basemap.

geop raster vector cut
Figure 11. GeoPackage with both vector and raster data

5.6.2. Sort attributes in the SQLite schema

When users are interacting with clients and users are not experts in the data, it might be useful that the schema is presented in a more useful way. Sorting attributes alphabetically in the SQLite schema will allow to easier find attributes in user interfaces when selecting features to filter.

Related issues:

If GeoPackage files contain links to data producer local file system, some data (e.g styles) might not be accessible. In the testing, at least one client saw significant slowing in the opening and displaying of the data because of these links.

A client reported:

Selecting all layers caused QGIS 2.18 to show pinwheel of death for 15 minutes. Data loaded when selected only the 9 feature tables included in the test, excluding other feature tables and all style and attribute tables.

Since test does not require styled features it might be useful to omit style tables from schema, as they account for most of its size and complexity.

Related issues:

5.6.5. Investigate further GeoPackage performance

Some files >600 MB were slower to load. Need to investigate further the raw causes of such behavior.

Two files in Sprint 1 with raster data were 1 GB and 5 GB. Raster queries were easy to perform.

5.6.6. Investigate further transparency

Some clients reported apparent transparency in raster layers.

5.6.7. Mashups containing raster elevation data

All the clients were able to open the file containing both raster and vector data. The GeoPackage contained high resolution images and elevation data in the 2D gridded coverage extension schema. Tile-based, pyramidal, floating-point raster data is a distinguishing feature of GeoPackage.

gpk hr elev
Figure 12. GeoPackage with high resolution and elevation data

6. Test suites issues and releases

The test results of the Plugfest identified several bugs or shortcomings in the test suites. This section provides a summary of the releases of updated test suites and issues reported as part of this initiative.

  • GeoPackage 1.2 NSG test suite

    • Release 0.5 (2018-08-28)

      • #21: Test NSG_filenameExtension is a duplicate of the test * filenameExtension

      • #36: Lack of metadata results in SQLITE ERROR hard failure

      • #38: Fortify scan reports issues

      • #37: Geopackage having no Tile data results in SQLITE ERROR

      • #27: Improve exception message of test * dataValidity_gpkg_spatial_ref_sys

      • #30: Test metadataSchemaValidation fails if table gpkg_metadata contains multiple values with at least one not NMIS valid entry

      • #42: Introduce Dockerfile and Maven Docker plugin

    • Release 0.4 (2018-07-13)

      • #33: Set GeoPackage 1.2 ETS dependency to version 0.7

      • #28: Remove duplicated test inherited from ets-gpkg12

      • #10: No Such Function: ST_MinX

      • #22: Test “dataValidity_gpkg_tile_matrix” fails if gpkg_tile_matrix contains zoom levels which are not present in data

      • #12: N S G_CRSdefinitions Test - java.lang.NoClassDefFoundError: org/geotools/util/UnsupportedImplementationException

      • #18: Clean up ETS

  • GeoPackage 1.2 test suite

    • Release 0.7 (2018-07-13)

      • Fix #76: Several tests are executed multiple times

      • Fix #64: Failure due to space in filename

      • Merge #73: R146 147

      • Fix #51: Review test requiredSRSReferences

      • Fix #60: The spatial issue revisited

      • Merge #69: Adding two samples

      • Merge #65: Adding a test case with a file with a space in it

6.2. Issues and Pull Requests of Executable Test Suites and Specifications

7. Applications and strategies from implementers

This section provides direct feedback and recommendations from implementers about how to use their tools to interact with NSG profiles data and services.

7.1. FME

Vector queries in the FME Client can be performed two ways.

1 - Interactively using FME Data Inspector as the client alone.

client fme data insp
Figure 13. FME Client

2 - Using the FME Workbench with a workspace script to automate the process.

client fme ws
Figure 14. FME Workbench

When FME is used to read a GeoPackage raster tile dataset, the Data Inspector client optimizes the display by balancing the displayed resolution with the zoom level. Unless a specific zoom level is chosen, FME automatically chooses the highest resolution zoom level that can be displayed at the extents chosen, and then resamples as needed.

7.2. Esri

ArcGIS Desktop was used to Create Mosaic Dataset (Data Management Tools) and to add the images into the dataset. When adding the images the default parameters were kept including the calculation of raster statistics. With the calculation of the statistics, the mosaics remain interactive and available to further analysis.

ArcGIS Desktop was used for publishing the Map Services with allows for the creation of 1-22 zoom levels. The default values were kept for all services.

For the creation of the GeoPackages, Esri turn to the Data Interoperability Tool as opposed to the Add Raster To GeoPackage (Conversion Tool). Work is in progress to make the creation of GeoPackage files more straight forward, in particular, in ArcGIS Pro.

Esri built the queries into the JavaScript and .NET apps, which is easy to use by non-experts. In ArcGIS Pro, the SQL statements were copied and referred back to them for each data source. Nothing special was done to speed up the return of the requests.

Esri stated that setting up the raster and vector GeoPackages, the WMS, WMTS, and WFS was fairly straight forward. Feedback was provided related to test engine irregularities. Esri achieved the goal to reduce the number of errors found in the NSG profiles.

7.3. GeoSolutions

During this experiment two services where provided: WFS and WMTS, both based on the correspondent NSG profiles. The two services were made available with a single GeoServer instance and the necessary GeoServer NSG extensions and plugins, providing a different end-point for each service, i.e. WFS and WMTS.

The provided vector and raster data was also configured in the server. Vector data was stored in a PostgreSQL database in the server. The database schema was adapted to support the NSG versioning needs. Auxiliary world files (.wld) where created for the raster data and directly stored on the file system and served through image mosaic GeoServer extension.

Clients tests feedback and the follow up was done with the support of GitHub issues. The provided WFS and WMTS services where respectively tagged as WFS_NEPTUNE and WMTS_CALYPSO. A total of six issues where reported for the WFS service and three issues for the WMTS service (in both sprints).

The raster data was published using GeoServer image mosaic extension, which allows the user to publish a mosaic from a number of georeferenced rasters. An auxiliary world file (.wld) was created for each granule, and then an image mosaic datastore pointing to the granules directory was created in GeoServer.

The already available image overviews were used as-is, image mosaic takes care of matching the correct overview with the requested zoom level.

Tool ​ogr2ogr w​as used to insert the provided vector data into the PostgreSQL database and gdalinfo was used to get the necessary information to complete the auxiliary world files (.wld) content for each granule.

The raster files were already optimized, e.g. tiled, compressed and with overviews (zoom levels). For vector data, an index was created for each primary key column of each dataset.

When configuring the tile matrix sets for a certain layer, special care should be taken to select only tile matrix sets that make sense for the layer. By default all the tile matrix sets defined by the WMTS NSG profile were available.

8. Initiative Feedback

8.1. GeoSolutions

In a distributed initiative like this one, the ability to provide the necessary feedback in a concise and straightforward way and encourage discussion happen between all interested parts is fundamental. It is also important to be able to keep track of what happened and be able to get a quick status overview, e.g., show all of the issues related to WFS.

GitHub issues was a good choice for tracking issues. The simple User Interface (UI) makes GitHub issues easy to use by both technical and non-technical people. The labels mechanism provide a good way of managing the issues and the discussion mechanism (with the associated notifications mechanism) is very efficient to use. When creating an issue and assigning it to the interested persons, GitHub will take care of notifying those persons, making them aware of that issue.

In an ideal world, everyone involved with an issue should be able to reproduce that issue in their own environment (debuggable environment), unfortunately this is usually not the case. People work on different environments, they don’t have access to the same clients or servers, etc. This means that special care should be taken when describing an issue.

Improving the information provided in an issue will help all the interested parties better understand the issue and solve the problem. For example, when describing an issue related with an UI, a simple scree capture visually showing the problem is usually easier to interpret than a verbal description. When describing an issue involving a client invoking a server, the actual request sent by the client to the server is a fundamental piece of information. An alternative means to review the issue is to reproduce the issue with a client that is commonly available, such as QGIS.

Appendix A: Revision history

Date Editor Release Primary clauses modified Descriptions

Aug 10 2018

L. Bermudez

.1

all

Initial version

Sep 6 2018

L. Bermudez

.2

all

Added section test issues, applications and initiative feedback and Plugfest description. Formatted document with better numbering.

Sep 8 2018

L. Bermudez

.3

all

Reviewed all the document, cleaned name of organizations acting as servers or clients, and added summary matrices of the sprints.

Sep 18 2018

L. Bermudez

.4

all

Incorporated comments and edits from Melissa Pham, Amy Youmans and Pull request 97-102.

Sep 20 2018

L. Bermudez

.5

all

Incorporated comments from Matt Sorenson and Annette Filer.

Sep 21 2018

L. Bermudez

.6

er.doc

Added doc number and logo.