Document type: OGC® White Paper
Document stage: Approved for Public Release
Document language: English

Preface

OGC defines interfaces that enable interoperability of geospatial applications. API’s are a popular method to implement interfaces for accessing spatial data. This White Paper provides a discussion of Application Programming Interfaces (APIs) to support discussion of possible actions in the Open Geospatial Consortium (OGC).

Annexes are provided on these topics:

The main themes of paper are:

Keywords

ogcdoc, OGC documents, api, protocol, interoperabiity, standards

Submitters of this document

All questions regarding this white paper should be directed to the editor or the submitters:

Table 1. Document Contributors
Name Affiliation
Chris Holmes Planet Labs
Dave Wesloh NGA
Chuck Heazel WiSC Enterprises
Gary Gale What 3 Words
Arnulf Christl Metaspatial
Josh Lieberman Tumbling Walls
Carl Reed Carl Reed and Associates
John Herring Oracle
Martin Desruisseaux Geomatys
David Blodgett USGS
Scott Simmons OGC
Bart de Lathower OGC
George Percivall OGC

1. What is an API?

1.1. API defined

This chapter discusses software engineering definitions for APIs.

API (Application Programming Interface) - An interface that is defined in terms of a set of functions and procedures, and enables a program to gain access to facilities within an application. (Definition from Dictionary of Computer Science - Oxford Quick Reference, 2016)

Interface - a shared boundary between two functional units, defined by various characteristics pertaining to the functions, physical interconnections, signal exchanges, and other characteristics, as appropriate (Definition from ISO/IEC 2382:2015 Information technology - Vocabulary)

The value of APIs is to provide well-known interfaces between components of a software system. APIs are useful in software development as they support modularity. Well defined interfaces allow separation of functionality into independent, interchangeable modules. APIs are often materialized as web-services, but are also commonly targeted at a specific programming language, typically Javascript.

APIs are popular today as they allow an organization to publize their APIs for reuse. Public APIs enable developers from other organizations to access the functionality provided behind the APIs. APIs are closely related to role of "platforms". For example the Facebook API provides functionality used by multiple organizations thereby advancing the market value of the Facebook platform. APIs may promote “stickiness” and "lock-in" to a platform.

Public APIs may or may not be Open APIs. Being "public" means that the API is visible and accessible outside of the organization that owns the API. The organization that controls and promotes the API owns the API. Open APIs are similar to open standards in that Open APIs are managed as a public good and managed in an open process. Open APIs adhere to open principles such as those defined by OpenStand:
1. Cooperation,
2. Adherence to Principles (including broad consensus),
3. Collective Empowerment,
4. Availability, and
5. Voluntary Adoption.
OGC standards go beyond OpenStand principles as all OGC standards are free to download and are royalty-free. (See Chapter 5 for more on the meaning and values of "Open".)

1.2. APIs in context

APIs are tools used by software engineers and developers to design modular and interoperable systems. From an OGC open standards perspective, APIs are understood in the context of requirement specifications and implementing code.

APIs in Context
Figure: : APIs in Context

Requirements Specifications are documents that define aspects of a protocol or data model to solve an interoperability problem. A protocol is the set of requirements that end points in a distributed system use when they communicate. The requirements may be written in plain text or in a technology neutral modeling language like UML. Web Protocol specifications include http structures that can be directly used in APIs. OGC standards specify requirements for interfaces and encodings using a modular approach (OGC 08-131r3). Implementations of an OGC standard must satisfy requirements as demonstrated by passing conformance tests. The requirements in a specification can apply across multiple API implementations.

APIs are interfaces that when backed by implementing code satisfy an interoperability need that may be documented in a Requirements Specification. The API package is a set of structure, function and procedure protocol definitions written in declarative statements that enables a program to access an application package containing implementations for these functions and procedures using these protocols. For bindings in most programming languages (C++, JAVA and other Object Oriented languages) the declarative code can be initially generated from UML defining the abstract API, using Model Drive Architecture (MDA) to produce the appropriate protocols. Implementations may use any procedures they deem appropriate as long at the functional requirements of the original specification are upheld.

Applications are software programs accessible by an API that meet user needs. Applications are distinct from system software including the operating system and system utilities. Applications implement algorithms to provide functionality behind APIs. The particulars of the algorithm are distinct from the protocols. A development strategy is to implement methods in the simplest manner possible and then do system testing and measurement to identify performance issues to optimize code, which is usually an algorithmic issue and independent of the protocol definitions.

1.3. Diversity of APIs

The application programming interface (API) entry on Wikipedia defines API as an interface that supports modularity and separation of components. The components can be implemented independently with the API serving as the common point of exchange. It is the notion of "API as an Interface" that makes it relevant to the activities of the OGC.

As defined in OGC® Testbed 11 REST Interface Engineering Report, OGC Document 15-052r1, an API is an interface definition that permits invoking services from application programs without knowing details of their internal implementation.

The OGC APIs listed next show the diversity of the API concept while still retaining the core concept that "APIs are interfaces."

1.3.1. OGC GeoAPI Standard

The OGC GeoAPI standard provides a set of Java language programming interfaces for geospatial applications. The GeoAPI interfaces closely follow the abstract model and concrete specifications published collaboratively by the International Organization for Standardization (ISO) in its 19100 series of documents and the Open Geospatial Consortium (OGC) in its abstract and implementation standards.

The GeoAPI interfaces provide a layer which separates client code from application code, which implements the API. This follows a similar pattern to the well-known JDBC API which provides a standardized interface to databases. Clients can use the JDBC API without concern for the particular implementation which they will use. Various groups have implemented different subsets of GeoAPI, see the list of known implementations for details.

1.3.2. OGC SensorThings API

The OGC SensorThings API provides an open, geospatial-enabled and unified way to interconnect the Internet of Things (IoT) devices, data, and applications over the Web. At a high level the OGC SensorThings API provides two main functionalities and each function is handled by a part. The two parts are the Sensing part and the Tasking part. The Sensing part provides a standard way to manage and retrieve observations and metadata from heterogeneous IoT sensor systems. The Tasking part is planned as a future work activity and will be defined in a separate document as the Part II of the SensorThings API.

The SensorThings API part 1 identifies resources as entities that are relevant to IoT Sensing, e.g., Thing, Observation, Location, Sensor. SensorThings API then grounds entities into entity sets and identifies URI components for requesting CRUD actions on those resources. The CRUD actions are specified using the HTTP protocol as well as MQTT protocol. MQTT is used to enhance the SensorThings service publish and subscribe capabilities. SensorThings API follows OData’s specification for requesting entities. However, SensorThings API does not follow the OData Common Schema Definition Language and should not be seen as an OData compliant API. SensorThings API’s future work will explore possible harmonization between SensorThings API and OData.

1.3.3. OGC NetCDF Standard

The OGC netCDF Standard is a set of data formats that support the creation, access, and sharing of array-oriented scientific data. The netCDF data model and encodings are particularly well suited to providing data in forms familiar to atmospheric and oceanic scientists.

Related to the data model and encodings listed in the netCDF standard are a set of APIs. Considering the popularity of the netCDF APIs they may be included in future extensions of the OGC netCDF standard. Bindings for C, Fortran, C++, Java are described in the CF-netCDF Core and Extensions Primer.

1.3.4. APIs and the Web - WebAPIs

The Web follows the Internet approach of Protocol-Based Interoperability where the protocols specify the syntax, semantics, and sequencing constraints of the messages interchanged. Protocols designed to be resilient in the face of widely varying environments have helped the Web scale and have facilitated communication across multiple trust boundaries. Traditional APIs do not always take these constraints into account. One effect of protocol-based design is that the technology shared among components often lasts longer than the components themselves. To support Web development the concept of a WebAPI was defined. A WebAPI is an API for either a web server or a web browser.

Web Programming API.svg
Figure: : WebAPIs

OGC Protocols for WebAPIs

OGC Web Service (OWS) Standards define Protocols that can be implemented as WebAPIs. The OWS standards are Requirements Specifications that define a protocol and in some cases a data model. The OWS standards include WMS, WMTS, WFS, WCS, WPS, WCPS, SOS, SPS, CSW, and OpenLS.

The OWS standards define requirements to be satisfied by both client-side and server-side WebAPIs. The OWS standards as currently written do not specify the declarative statements needed for an API package. The OWS standards do not specify API design rather they specify protocols that provide requirements for APIs. Multiple APIs with differing designs can satisfy the OWS requirements (See Section 2 Web Mapping examples for further discussion).

The OWS standards are the basis for OGC Conformance Tests that are used to confirm that functional requirements are met by implementations. OGC Compliance Tests have predominantly focused on server-side testing, while more recently client-side compliance tests have been defined for WMS.

REST Architecture for OGC Web Services

REST (Representational State Transfer) is a term coined by Roy Fielding in his doctoral dissertation to describe an architectural style for “distributed hypermedia systems” such as the Worldwide Web that are to have desirable characteristics including separation of concerns, scalability, resiliency, visibility, and reliability. The REST architectural style rests on five constraints on how computing systems are configured and component interactions are carried out (see for example A RESTful Tutorial). Examples of non-open standard RESTful web mapping APIs are provided in Appendix A.

OGC Web Services (OWS) were defined before the publication of REST principles. OGC has evaluated the implications of REST vis-a-vis OWS finding that the existing OWS Standards have aspects of REST but also that OGC services should be migrated to, or at least make available as more RESTful bindings. For example, The OGC Testbed 11 REST Interface Engineering Report defines possible uniform practices for developing bindings or interaction styles for OGC Web services that appropriately leverage REST principles. OGC Testbed 12 continues these developments and will conclude late in 2016.

2. Need for Interoperability across APIs

2.1. The Proliferation of Public APIs

This Chapter considers the variation in existing geospatial APIs as the motivation for proposing API design based on open standards. The objective is to promote wide spread interoperability based on open API design while maintaining competitive opportunities.

There are many public APIs. As of November 2016, The Programmable Web listed over 16,000 public APIs. (See Chapter 5 for distinction between public and open.) Richardson identified challenges with the proliferation of APIs including

The recent proliferation of APIs for geospatial applications has degraded the interoperability previously established by open standards. This degradation is do both to the variability of API practices across the IT industry as well as variability in geospatial APIs specifically.

Advancements in API practices is needed across the software development profession. "APIs are often difficult to use, and programmers at all levels, from novices to experts, repeatedly spend significant time learning new APIs. APIs are also often used incorrectly, resulting in bugs and sometimes significant security problems." ("Improving API Usability" in the Communications of the ACM)

2.2. Example variations in Web Mapping APIs

Web mapping APIs define interfaces and bindings to access maps over the web. This section provides examples of variation across existing geospatial APIs to demonstrate the need for improved API design practices. While web mapping APIs are defined in multiple programming languages, the examples in this chapter focus on JavaScript.

APIs for web mapping were listed in The Programmable Web with the "top ten" being: Google Maps, Microsoft Bing Maps, OpenLayers, Foursquare, OpenStreetMap, MapQuest, Mapbox, CartoDB, Esri, and Yahoo. Some of these APIs implement the WMS Protocol either explicitly or as an extension - others do not support WMS, defining their own semantics and syntax for web mapping.

Annex A lists details and code snippets for these Web Mapping API examples: Google Maps, OpenLayers, MapQuest API for OpenStreetMap, Leaflet. Additionally two more RESTful examples are listed in Annex A: Esri ArcGIS REST and Mapbox.

The Web Mapping APIs listed in the Annex A use similar concepts sometimes in attributes differing names. The Table below lists two such attributes - "Center" and "Zoom" - that are in the four example APIs. "Center" is the center point of the web map to be displayed. "Zoom" is related to the scale of the web map to be displayed. The concepts for these attributes are similar in each API but the precise specification is different in each case.

The variability of similar concepts across different APIs as seen in Table 1 suggests an opportunity to quickly converge on a small set of "essential" attributes usable across otherwise disparate APIs. This simple example motivates the discussion about geospatial domain mapping and OGC Essentials for geospatial APIs based on open standards.

Table. Comparison of Map-related attributes from several APIs

test

2.3. Interoperability across Multiple Map Servers

Interoperability is a primary objective of the OGC. Consistent semantics across multiple servers independent of underlying API leads to interoperability and the ability to merge results from the multiple servers. The figure below was created by making individual requests to several map services (the text in each “square” indicates the name of the server). The figure illustrates what can be done if services use the same indexing systems. The figure uses the “indexing system” defined in the WMTS Simple.

MultipleMaps
Figure: : Multiple Maps with common semantics - Interoperablity (Source: Joan Maso)

3. Geospatial API design based on open standards

To address the challenges of API proliferation addressed in Chapter 2, this Chapter identifies several methods based on open standards for the design of Geospatial APIs based on open Requirement Specifications.
These topics are addressed in the following sections for this Chapter:

The Figure below provides a summary of the topics discussed above Geospatial API Design using open standards.

chapter 02 2cb28

The following sections includes Recommendations from "Web APIs for Environmental Data", a report from CSIRO (See Annex B). The CSIRO report provides Recommendations as a basis to design consistent, cross-organization APIs for access to a range of environmental data products. Observations from the CSIRO report and listed in this Chapter provide examples of opportunities for improving geospatial API design.

3.1. Modeling Geospatial Domain Resources

Designing for a connected set of concepts will lead to more robust interactions with domain resources, long term evolvability and fitness for use (Spec Driven Development). Design should consider an interconnected set of API resources with clear ideas and support of why and how API consumers will follow certain paths. This supports the objective that APIs may not just compete by features, but also by how easily and robustly they can be used ( API Maps: Hypermedia API Design)

The CSIRO API Report (Annex B) makes the following observations about modeling the environmental data domain.

Multiple map APIs designed with a consistent domain model will improve development using multiple APIs, e.g., a developer can use an open API to draw the map, open map data from multiple sources in combination with commercial APIs say for advanced features such as traffic or routing. Capturing the geospatial domain in open Requirements Specifications provides for achieving such domain interoperability.

One aspect of this analysis is the use of Landing Pages to document APIs. The advent of distributed Hypermedia, in particular the World Wide Web, has introduced a new pattern of distributed processing. In this pattern, users follow links from one resource to the next. Each resource provides the information needed to proceed to the next set of resources. This pattern assumes a simple protocol (URLs over HTTP). Landing Pages are an emerging approach to document an API using this pattern. A landing page is a collection of web pages which document the API. The web pages also include URLs which can be accessed using the HTTP protocol to exercise the advertised capabilities. Two examples of landing page APIs are: John Deere and Twitter

3.2. API design based on geospatial requirements

As discussed in Chapter 1, Requirements Specifications define protocols and schemas for interoperability across distributed heterogeneous distributed information systems. APIs that implement the Requirement Specifications support such interoperability.

The OGC Web Service (OWS) Standards are requirements specifications that define geospatial interoperability. The OWS Standards define data structures, behaviors and semantics for protocols based on http. The OWS Standards were initially defined based on a Service Oriented Architecture (SOA). The OGC Web Map Service was defined with a service interface including multiple operations, e.g., GetMap. OGC is currently reviewing the OWS Standards in a discussion will lead to addition of more RESTful protocols based on a Resource-Oriented Architecture (ROA). (See OGC Testbed 11 REST Interface Engineering Report)

Consistent with ROA and supportive of RESTful APIs, geospatial resources need to be identified in OGC standards. Geospatial resources include: feature collection, feature, coverage, map, layer, trajectory, etc. Relationships are also important to be modeled, e.g. topological associations between features, parent-child associations, etc.

Based on identification of geospatial resources RESTful protocol signatures are being defined for OWS standards. For example:

http://ows.example.org/dataset/ows/featurecollection/featureID http://ows.example.org/dataset/ows/coverage/AvgLandTemp

An example defining a relation between a requirements specification and implementing APIs is available in these two standards from the Intelligent Transport Systems (ITS) Domain:

Corresponding to services defined in ISO 13185-2, the UVIP Java Client API (ISO 13185-3) contains services that can be called by any application to execute a service. A UVIP Client Application communicates with a UVIP Server Application.

Resources, protocols, semantics and behaviors defined in Geospatial Requirement Specifications become the basis for API Specification and Management.

3.3. API Specification and Management

API specification and management are crucial software engineering functions for any organization using information technology. As the popularity of APIs has grown in the past few years, so too have the tools, best practices and consulting services that support organizations in API specification and management. While these tools were initially developed for organizations to manage their organization-specific APIs, the tools can also be considered for use in API design in an open standards context. Here we consider one such tool: the Open API Specification formerly known as the Swagger specification.

Documenting APIs can be aided by using the approach of the OpenAPI Initiative (OAI). OAI is focused on defining a vendor neutral API Description Format based on the Open API Specification (OAS). The approach will allow specification of REST APIs using modular sub-elements. Sub-elements can then live on their own and be shared by multiple APIs. More information about OAI and OAS is available in Annex C.

Providing complete documentation of your API using OpenAPI is a W3C Data on the Web Best Practice. This best practice for APIs is being discussed in the OGC/W3C Spatial Data on the Web Working Group.

Swagger was used in the OGC WaterML2.0 part 2 Interoperability Experiment to document the CSIRO RESTful API. The CSIRO API Report (Annex B) makes the following observations:

An advantage of documenting APIs with OAS is the industry-wide tools and practices that are based on OAS. The first Figure below shows how OAS enables robust management of software development lifecycle activities. The second figure below shows the automated generation of client libraries based on code generation tools. Use of OAS also helps organizations with API response types, including error handling, link updates, moved and re-aggregated resources, etc.