OGC Testbed-14 Next Generation APIs: Complex Feature Handling Engineering Report

The dataset used in this section is about cadastral parcels and related information. It uses the AFIS-ALKIS-ATKIS application schema of the Surveying Authorities in Germany. The GML application schema is the so-called "Standards-based Data Exchange Interface" (NAS).

The application schema is optimized for maintaining the data by the Surveying Authorities and reflects the legal requirements. As a result, the application schema contains many relationships between feature types as well as structured data types. The application schema includes information related to portrayal of the data at map scales that are traditionally used for base maps, too. For example, information is included about text placement in specific maps.

Since cadastral law permits arcs in a boundary of a cadastral parcel, Simple Feature geometries are not sufficient for storing the authoritative feature data.

Outside of the Surveying Authorities the data is often simplified / flattened to simplify the use of the data, but a number of workflows require the full use of the complex data structures included in the dataset.

As the application schema is used in Germany, it uses German terms. For better readability we show a simplified schema using English translation for each query described in this section.

This is a relatively simple use case where features are selected based on properties of related features, i.e. features that are the value of an association role property.

The following aspects are covered by this use case:

This use case is similar to the previous one, but uses value references along multiple associations.

The following aspects are covered by this use case:

Often, the history of a dataset is important. The example that we are using here is a cadastral parcel dataset, where it can be important to know the state of the parcels at a point in the past.

There are two options for how this is typically handled in application schemas.

One approach is that the features are in fact feature versions. That is, different versions of the same feature / real-world entity are each represented as separate features. This is the approach we are considering in this use case. To avoid confusion we use the terms "version" and "real-world entity" in the description of this use case instead of "feature" which could mean the feature or a specific version of the feature.

The advantage of this approach is that no specific temporal support is required in clients processing the data. This pattern is therefore frequently used with data that is used in map-based GIS clients, for example, with datasets provided by mapping or cadastral agencies.

The other approach is to model the feature properties as timestamped sequences of values. GML supports this approach with the Dynamic Features pattern. The downside of this approach is that clients and servers must support this specific pattern, which typically requires customized software. A domain that is using this approach is the aviation domain.

The following aspects are covered by this use case:

A common requirement is to present features in a dataset on a map (or in a 3D scene). In this use case we look at rendering feature data on a 2D map, for display in a web browser.

This may be implemented using a WFS 2.0 as the backend, i.e. the rendering engine is a WFS client and then renders the data, either directly in the browser or in a server, for example, a WMS 1.3.

For server-side rendering, the data will typically be rendered closer to the database and not via a WFS 2.0 interface - for performance reasons. For client-side rendering, the data will typically not use GML, but a format that is optimized for the rendering purpose. Nevertheless, the use case is still relevant in the context of complex feature handling, for at least two reasons:

The following aspects are covered by this use case:

This use case has been added to include a query that - while not overly complex - cannot be executed with most WFS implementations as it would require support for spatial joins.

Including this use case does not imply that WFS 3.0 should include an extension that supports such a query. However, as spatial relations are an important aspect of spatial data, we should at least include it in our considerations, even if we recommend to include explicit spatial relations in the feature representations, consistent with the recommendations of the W3C/OGC Spatial Data on the Web Best Practice document.

This section is about using 3D city models to simulate a building’s heating demand and visualize the results in a web-based 3D scene. Why is this relevant? The building sector has large potential for energy efficiency gains and CO₂-reductions and is thus a priority area for achieving the ambitious climate and energy targets for 2020 and 2050 in the European Union [2]. In order to reach the 2% energy refurbishment rate promoted by the European Union and to realize long-term climate neutral communities, information about the current and future demand is necessary in order to develop strategies for policy making as well as to raise awareness of the citizens. Some pioneering work has already been done worldwide [3], [4], [5], [6]. As an input for the simulation, a building model in CityGML is usually used. The use case is split into several parts:

The following aspects are covered:

This has been showcased and implemented in Testbed-13 [7]. The implementation is available and could be further used and extended in Testbed-14 with the New York data set. The experiment evaluated the complete flow of data from its originating CityGML format to a web-enabled visualization with Cesium via OGC’s 3D Portrayal Service (3DPS). This data flow included the conversion from the CityGML data format served by GeoRocket, to 3D Tiles dataset, and the import of the 3D Tiles dataset to the 3DPS Framework.

A public demonstration is available at: http://tb13.igd.fraunhofer.de:8080/Apps/Sandcastle/index.html?src=3DPS_r.html&label=Showcases.

The following aspects are covered by this use case:

In Testbed-13, the 3DPS getFeatureInfo request has been used to query heating demand (simulation result) per building from a server at HFT Stuttgart. See video on the Testbed-13 S3D Performance Experiment #2.

It would make sense to use the building id to retrieve additional attributes from a WFS. However, the building geometry is no longer required because as it is already available.

The following aspects are covered by this use case:

Most important in this use case is that solids are supported as feature geometries.

In addition, it needs to be discussed how to deal with the Level of Detail (LoD) concept of CityGML. The CityGML model that is retrieved will typically be used by another process - in our example the simulation of the heating demand of that building.

In this example, all buildings in a rectangular region are requested and the selected building features are returned in a feature collection.

These examples have been provided by Claus Nagel, virtualcitySYSTEMS. They cover the following query categories:

In an application schema, the property has a maximum multiplicity greater than "1".

WFS 3.0 Core and the GeoJSON and GML Simple Feature Profile Level 2 conformance classes already support this requirement.

GeoJSON does not place any restrictions on the contents of properties, i.e., the natural approach to encoding such properties in GeoJSON would be a JSON array.

GML Simple Feature Level 2 is similar as it only constrains the spatial geometries used, but has no constraints on the other feature properties. In GML, multiple values are represented by repeated property elements.

In an application schema, the property has value type that is neither a geometry, another feature nor a literal value (like Integer or CharacterString).

Again, WFS 3.0 Core and the GeoJSON / GML Simple Feature Profile Level 2 conformance classes already support this requirement.

In GeoJSON, the natural approach is to represent such values as a JSON object.

In GML, the value of the property is an object element representing an instance of the data type.

For example, the following XML shows a GML feature with a property property that has two values, each a data type with two properties (sub-property1 and sub-property2). The other properties are suppressed:

The GeoJSON representation would be:

Even though GeoJSON and GML support such complex data structures, software tools are sometimes limited to simple values only. In that case, the data structures would have to be flattened. The following GeoJSON snippet is a flattened version of the GeoJSON feature above.

If application schemas according to ISO 19109 are used, the ShapeChange [8] can help to create flattened implementation schemas, i.e. where all feature properties have a literal value. ShapeChange is a Java tool that takes application schemas constructed according to ISO 19109 from a UML model and derives implementation representations.

In an application schema, the property is typically a relationship role - or in some cases an application-schema-specific link data type (External references in the German cadastre application schema or in CityGML that have a URI as a value are such data types).

The GML encoding in a WFS 3.0 Core supports such properties already using Xlink references (again, only in Simple Feature Profile Level 2, not in Level 0).

GeoJSON has no concept of links, but specifies extension points and no constraints on the content of "properties". The links could thus be implemented in multiple ways:

Spatial relationships between features are a somewhat special case as they can be derived by spatial analysis (see the discussion in the use cases and the W3C/OGC Spatial Data on the Web Best Practice document), but they can use the same pattern in an encoding.

The W3C/OGC Spatial Data on the Web Working Group discussed whether the most commonly used spatial relation types should be registered with Internet Assigned Numbers Authority (IANA), but eventually deferred this. Note that schema.org now has a pending change to support them.

WFS 3.0 supports 3D coordinates but not 3D solids. WFS 3.0 Core is therefore currently limited to 2D geometries in a 2D or 3D coordinate reference system (elevation values may be included for each position). All curves and surface boundaries are restricted to linear interpolation (plus circular arc interpolation in the GML Simple Feature conformance classes). This is not sufficient for the use cases identified in the previous chapter.

An example are geometries at different map scales / levels of details.

GML, including the Simple Feature Profiles Levels 0 and 2, supports this, but GeoJSON does not.

That is, in cases where this capability is important, another JSON encoding has to be used or the features have to be split so that there is one feature per spatial geometry property.

Another option could be to include additional geometries inside the "properties" JSON array, but GeoJSON-aware software would not identify these values as geometries. Such an option should, therefore, not be considered.

This is fully supported by GML, but not the GML Simple Feature Profiles or GeoJSON as both encodings do not support solids. That is, none of the conformance classes in WFS 3.0 Core is able to support this requirement.

Again, this is supported by GML, but neither by the GML Simple Feature Profiles nor by GeoJSON.

Again, this is supported by GML, but neither by the GML Simple Feature Profiles (with the exception of arcs) nor by GeoJSON.

Typical examples are validation against an XML Schema or a JSON Schema.

The current draft of WFS 3.0 Core includes - as a stopgap - a recommendation to include relationships to the schema documents. Another option would be /schema paths.

The OpenAPI Initiative is discussing more robust support ("alternative schemas") for this requirement in the OpenAPI specification. OGC should wait for the resolution of this development before looking for other solutions.

As the use cases have shown, support for richer data structures is not only required for representing the features in JSON or XML, but these data structures have to be accessible in queries, too.

In FES 2.0 this is supported by the matchAction parameter.

In WFS 2.0 and FES 2.0 this is supported by the use of XPath expressions in value references, including for cases that require the traversal of a link to a related feature.

If the data includes explicit spatial relationships, these may be used for filtering, too.

This requires support for grouping as well as the standard logical operators specified in FES 2.0.

WFS 3.0 Core only requires support for spatial queries using 2D bounding boxes.

Intersects is the most important relation, but in general the full set of standard operators should be supported.

DWithin and Beyond are the standard operators in FES 2.0.

Using the time parameter, WFS 3.0 Core already supports access to features at a certain moment or period in time, but currently the default is always "no temporal filter".

For datasets where the features are versions of a real-world entity, valid for a given time period, the default response would return all versions. It would be more helpful, to change the default behavior for such datasets.

This has two aspects: being able to specify a different default value for time ("now" instead of "no filter") and to be able to explicitly state values for the indefinite past and future. The latter issue is already under discussion by the WFS 3.0 community.

If a time period is used in time the response for such datasets could include multiple versions of the same real-world entity.

This is a requirement that would need to be supported in the Core.

Some of the use cases benefit from post-processing the feature or feature collection that has been selected.

In some use cases this will avoid repeated, additional requests to the server to access the related features as their information is needed by the client application.

The related feature may either be embedded inside the property representing the relationship or - in particular where that relationship is always represented by a hyperlink - elsewhere in the document, referenced by a local link.

WFS 2.0 supports this capability using a set of resolve parameters.

This could be any combination, for example (links go to existing discussion in the WFS 3.0 development):

WFS 2.0 supports this capability using so-called projection clauses as well as using the GetPropertyValue operation.

WFS 3.0 Core only supports queries on a single feature collection (feature type in WFS 2.0, layers in WMS/WMTS). In practice, it is often useful to query features from multiple collections at once, even if the query is restricted to simple filtering, e.g. bbox or time.

This GitHub issue has additional discussion that should be considered.

During the discussions at the WFS 3.0 hackathon in March 2018 there was agreement that the path /search/{endpoint} should be reserved for offering richer queries on a dataset. Different search implementations could be published as separate search resources. i.e., a server could support multiple types of queries in parallel. For example, /search/fes could be an endpoint for WFS 2.0 ad-hoc queries with Filter Encoding 2.0 and /search/graphql could be an endpoint for GraphQL queries on the same data.

In cases where only a query language is specified (e.g. WFS 2.0 ad-hoc queries with Filter Encoding 2.0, CQL, XQuery, etc.), but no rules exist how to execute queries via HTTP, there are three general options for the API design:

The third option is quite heavy for queries that are not used repeatedly. For queries that are executed more than once, however, this approach has benefits, e.g. for caching. The approach would also support parameterized queries where the parameters could be passed as query parameters when accessing the query resource with a HTTP GET request (like stored queries in WFS 2.0).

The OGC Filter Encoding Standard 2.0 [10] is the standard filter language for ad-hoc queries in WFS 2.0 [11]. Together they support the query requirements identified in the previous chapter.

However, the specifications are closely tied to XML including the use of XPath, which makes them an unnatural match for servers that want to return JSON.

Implementations exist, but the requirements identified in the previous chapter are supported only to a limited extent in freely available libraries. Implementations typically support relational database backends.

Conclusion: A candidate query language for WFS 3.0 implementations that also support WFS 2.0 and GML. Complex feature handling in WFS 3.0 will, of course, only be supported, if the WFS 2.0 implementation already supports the identified use cases.

CQL is short for "OGC Common Query Language" and is defined in the OGC Catalogue Services Standard 2.0 [12] in section 6.2.2. It is a text-based syntax similar to the SQL "Where" clause, i.e. on a similar level as Filter Encoding, but easier to write for most developers than the XML syntax of Filter Encoding.

Implementations typically support relational database backends, CQL is less suited for document stores.

As a text-based query language it is not strongly tied to XML, JSON or any other encoding.

CQL does not support all identified query requirements. CQL does not have well-defined semantics for querying properties with multiple values, does not support link traversal across relationships (query predicates on related features) or 3D geometries.

Implementations exist, but the requirements identified in the previous chapter are only supported to a limited extent in freely available libraries. An example is limited support for "compound attributes", i.e. path expressions that might be used to query nested data structures.

Beside the fact that implementations support only a subset of the language, CQL has a major limitation with respect to the requirements identified in the previous chapter: Attribute values are assumed to be literal. That is, CQL does not support values that are collections (maximum multiplicity > 1) or objects / data types.

That is, either the use of CQL is restricted to feature data that meets these constraints or CQL would need to be extended to support feature data that have the following data structure requirements:

For nested structures and relationships, probably the dot-notation the compound attribute names could be used, but for each encoding the mapping between the compound attribute names and the feature encoding would need to be specified.

For predicates on attributes with multiple values, there are basically two options:

In addition, the specification for time period expressions in CQL should be amended to cover the cases discussed in the related WFS 3.0 issue.

In order to support clients to construct queries, the feature properties that may be queried should be enumerated for each feature type. This could be included in the feature collection metadata or, which is probably preferable, it could be made available in an additional resource listing all queryable properties. For example at /collections/{collectionId}/queryables. The result could be a JSON object with a member for each property of the feature. The value of the member could be used to identify the data type. If the property value is a related object in the dataset, the queryables resource of that collection could be referenced. For nested objects, the compound attribute values could be used explicitly.

It should also be allowed to declare queryables that do not have to be, for example, a direct member of the properties object in a GeoJSON feature.

CQL is currently "buried" in the Catalogue Service specification and the specification of the language is largely restricted to the definition of the grammar (with a number of inconsistencies and ambiguities in the definition). If CQL would be supported by a WFS extension, it should be extracted from the Catalogue Service standard and become a standard on its own, with a clear and unambiguous specification of the language and requirements for implementations.

Conclusion: A candidate query language for an intermediate WFS 3.0 query capability that goes beyond the limited support that WFS 3.0 Core offers, but that may not address all requirements identified. The lack of implementation support for document stores is an issue that needs a broader discussion, too.

Falcor [13] is a data platform that powers the Netflix user interfaces.

The starting point of Falcor is to assume that all data is a single (virtual) JSON object. This allows clients to work with the data using standard operations on JSON objects and support for Path expressions. In the words of Netflix: "If you know your data, you know your API" [14].

In a way this would be comparable to using XQuery as a query language in WFS 2.0 where the dataset is basically a large GML feature collection.

Falcor has additional conventions to allow that the virtual JSON object can be used as a graph with shared resources and not just the that a JSON object is. This avoids multiple copies of the same object in different parts of the virtual JSON object. An enhanced path notation is used to reference nodes within the virtual JSON object.

Falcor has no schema of the data and assumes that the developer knows the data (see the quote above).

It is mainly designed for use in JavaScript and has no support for geometries or spatial predicates.

Conclusion: Falcor may be a candidate for a WFS 3.0 implementation that only supports JSON and that is mainly accessed from JavaScript. However, support for spatial aspects would need to be specified and implemented first. It is thus not considered in more detail in this report.

GraphQL [15] is a declarative, string-based query language created by Facebook to support fetching data for use in a user application from a server.

One of the main drivers for GraphQL was the goal to provide an interface that allows mobile app developers to retrieve exactly the data that they need in a single query from a single endpoint. This is based on the observation that in REST APIs one usually needs multiple requests to fetch the information and/or that the response often contains unnecessary information ("overfetching").

That is, support for GraphQL would basically be complementary to the current WFS 3.0 Web API. The blogpost "GraphQL: Everything You Need to Know" [16] includes a comparison of strengths and weaknesses of both approaches.

Unlike Falcor, where the client has to know the data, GraphQL requires a schema of the data. GraphQL is strongly typed and supports nesting, multiplicities, etc.

Typically, GraphQL schemas are tailored for the specific application needs. That is, GraphQL queries are in practice in a way closer to the stored queries of WFS 2.0 than the generic ad-hoc queries of WFS / FES 2.0 - although with a much richer mechanism to specify parameters and projection clauses.

A significant plus for GraphQL is that it has a lively, and growing, ecosystem with good tools, support, etc.

However, currently there is no support for geometries or spatial queries in GraphQL.

GraphQL is not tied to JSON, but JSON seems to be by far the most commonly used encoding.

Conclusion: GraphQL is a promizing candidate because of its popularity and its characteristics, in particular for usages that are close to end user applications. Spatial support may be an issue and needs to be explored in more detail, including the use of GeoJSON or CityJSON.

The SpatioTemporal Asset Catalog (STAC) specification [9] intends to standardize the way geospatial assets are exposed online and queried. The specification defines a spatiotemporal asset as "any file that represents information about the earth captured in a certain space and time". Right now, the focus is on remotely-sensed imagery.

Querying STAC is very similar to general feature querying and since the principles and technologies used are very similar, the WFS 3.0 hackathon in March 2018 [17] was co-located with a STAC sprint [18]. During these meetings the API building blocks were aligned so that STAC implementations will conform to WFS 3.0 Core.

STAC extends the Core with a /search/stac endpoint, which for now is restricted to bounding box and time interval searches like WFS 3.0 Core.

Several ideas are discussed or explored for supporting more advanced queries:

Conclusion: Before any decision is made for WFS 3.0 query extensions, the plans should be discussed with the STAC community to check for additional opportunities to align the specifications.

The following table summarizes in how far the candidates support the identified requirements.

For 2D data, a commonly used approach is to organize the feature data in tiles, in particular for visualization in map-based client applications in a web browser. Tiles are provided for different zoom levels (scales) and how the features that are located in the bounding box of a tile are included in the tile will depend on the zoom level (e.g. no buildings at a scale of 1:1.000.000).

In parallel to OGC Testbed 14 another OGC Innovation Program initiative, the Vector Tiles Pilot, is investigating how tiled vector data should be provided via a NextGen service Web API as Mapbox Vector Tiles (using Google Protocol Buffers) and as GeoJSON. Of course, the tiles could also be rendered as bitmap images, too, if the server has styling information.

In general, as tiles are different resources, they would be made available under new resource paths. For example:

These paths (in addition to paths for the tiling scheme information) represent the capabilities of an OGC WMTS in a NextGen architecture.

In addition, Google Protocol Buffers following the Mapbox Vector Tile format could also be served from the /collections/{collectionId}/items path as an additional encoding. A pre-requisite is a media type for the encoding to support content negotiation.

A common approach to provide optimized access to 3D feature data for visualization in a browser are "scenes". A scene provides 3D geometries with texture data and attribute information, organized as a scene graph and/or spatial index. Each node in the graph represents a spatial partition and data for display at a certain level of detail, depending on the distance from the viewpoint, etc.

The OGC 3DPS standard [19] provides access to scenes, usually using the OGC community specifications i3s [20] and 3D Tiles [21].

A difference to the 2D tiles case described in the previous section is that it is the client that requests tiles for display based on the knowledge of the tiling scheme. In the 3DPS case, the nodes in the scene graph are secondary resources, linked from the scene graph. That is, the client accesses a scene graph and then access the nodes linked from the graph.

A possible implementation for fetching scenes of a dataset in a NextGen architecture could be

The scene paths should support the usual WFS 3.0 query parameters:

The last two parameters would be up for discussion, but in general it should be helpful for clients, if feature selection is done consistently across the different resources in the API.

In addition, the other parameters of the GetScene request (beside request, version and format which are no longer needed as this is handled differently) would be supported, too. For example, lods or styles.

Steinbeis Transfer Center at HFT Stuttgart (Steinbeis) has implemented an experimental 3D Portrayal Service scene resource following the approach based on the Testbed 13 showcase using the 3D CityGML model of New York described here. The implementations is based on OpenAPI and the Web API can be tested at http://steinbeis-3dps.eu:8080/tb14/API/index.html.

Alternatively, to use the Web API call in an application, make the following request by defining the correct bbox coordinates in the request:

The aforementioned call will deliver a 3D building model (as one layer of the 3D city model (3D-DLM)) of Manhattan using 3D Tiles. It can be rendered in the Cesium globe (See Figure 11.). A prototype implemetation is available at http://81.169.187.7:8081/3DPS_App/.

The current OpenAPI implementation supports two resources Data Layer and 3D Scene (See Figure 12.).

The data Layer resource is used to retrieve the name of the available data layers. The current implementation supports one layer only: “3D_CityModel_manhattan” . However, in case the service supports more then one data layer it will show a list of supported data layers.

To test the Web API, first click on the Try it out button and then the Execute button.

Since this call does not take any input you will see the output as shown in Figure 13. In addition, it also shows the curl request with all the necessary query parameters.

The 3D Scene function on the other hand takes three input parameters. The first is a path parameter (layer) and the remaining two are query parameters (format,bbox) see Figure 14.

A successful execution will provide the tileset link (see example below).

This tilesetLink can be rendered in the Cesium globe. To show the feasibility of our approach we have developed an integrated prototype see Figure 15. Please visit the following http://81.169.187.7:8081/3DPS_App/.

To learn more and get used to the platform please visit the you tube link https://www.youtube.com/watch?v=6EwlwAhtK7Q.

To include multiple layers (for example, buildings and vegetation), the request would be:

The information that is currently included in 3DPS 1.0 capabilities would be included in the OpenAPI document and in extensions to the feature collection metadata resources like in WFS 3.0.

The selection of an encoding would follow the same approach as in WFS 3.0. That is, every server will support content negotiation using media types to negotiate the format of the response. In addition, servers should support a mechanism to include the format information in the path to support hyperlinks.

Media types for i3s and 3D Tiles are an open issue.

For i3s application/vnd.esri.i3s.json+gzip is specified, but not registered with IANA yet.

For 3D Tiles no specific media type has been specified yet and general media types are used (in particular application/json). This should be changed as the use of the general media types is ambiguous.

The Hochschule für Technik Stuttgart has implemented a prototype of such an API as a facade on top of the 3DPS implementation from Testbed 13, which uses the New York dataset.

In addition to scenes, the 3DPS 1.0 standard [19] specifies additional requests that would need to be mapped, too. A detailed analysis is out-of-scope for this report, but from looking at the feature info requests we can see the benefits of the NextGen architecture that supports the different resources derived from a dataset in a single API. Where 3DPS 1.0 has to define its own requests to access feature data, in the NextGen architecture these resources are often already provided by the API building blocks specified by WFS 3.0.

GetFeatureInfoByObjectId in 3DPS 1.0, for example, is an operation that allows a client to retrieve information about features that are selected based on object identifiers. In the NextGen architecture this is simply the paths /collections/{collectionId}/items/{featureId} specified by WFS 3.0 Core.

On the other hand GetFeatureInfoByRay introduces a new way of spatially selecting features based on a virtual ray. In the NextGen architecture this would be implemented in an extension to the WFS 3.0 /collections/{collectionId}/items resource where additional query parameters specify the ray.