Open Geospatial Consortium
OGC Document 16-019r4
External identifier of this OGC® document: http://www.opengis.net/doc/WP/api-whitepaper
URL for this OGC® document: http://docs.opengeospatial.org/wp/16-019r4/16-019r4.html
Internal reference number of this OGC® document: OGC 16-019r4
Category: OGC® White Paper
Editor: George Percivall
Previous Version: https://portal.opengeospatial.org/files/?artifact_id=71743
Publication Date: 2017-02-23
Approval Date: 2017-02-13
Submission Date: 2016-11-16
OGC® Open Geospatial APIs - White Paper
Copyright © 2016-2017 Open Geospatial Consortium.
To obtain additional rights of use, visit http://www.opengeospatial.org/legal/
This document is not an OGC Standard. This document is an OGC White Paper and is therefore not an official position of the OGC membership. It is distributed for review and comment. It is subject to change without notice and may not be referred to as an OGC Standard. Further, an OGC White Paper should not be referenced as required or mandatory technology in procurements
Document type: OGC® White Paper
Document stage: Approved for Public Release
Document language: English
OGC defines interfaces that enable interoperability of geospatial applications. API’s are a popular method to implement interfaces for accessing spatial data. This White Paper provides a discussion of Application Programming Interfaces (APIs) to support discussion of possible actions in the Open Geospatial Consortium (OGC).
Annexes are provided on these topics:
The main themes of paper are:
APIs are popular because they are an effective method for rapid software development in particular for distributed systems. Examples of successful geospatial mapping APIs are listed in Annex A: Google Maps, OpenLayers, MapQuest API for OpenStreetMap, Leaflet, Esri ArcGIS REST and Mapbox
The proliferation of API variations degrade interoperability defined by web protocols. The examples in Annex A show such variations in several parameters including "Zoom" level. A summary of the differences of Zoom is provided in Chapter 2, Table 1 - Comparison of Map-related attributes from several APIs.
Protocols provide interoperability when implemented consistently across APIs. The OGC WMTS protocol when implemented across multiple APIs enables interoperability as demonstrated in the Figure at the end of Chapter 2. By using consistently using the WMTS TileMatrixSet, graphics returned from several APIs can provide a consistent, composite map. Similarly consistent use of OGC Essentials will improve interoperability (See Chapter 4).
API implementation experience should be used to improve the OGC Web Service Protocols. Several RESTful approaches for defining OWS protocols based on resource types are being considered in OGC. Further, this white paper recommends that OGC consider activities that provide for consistent implementation of OWS protocols across multiple APIs (See in particular Chapters 3 and 4).
ogcdoc, OGC documents, api, protocol, interoperabiity, standards
Submitters of this document
All questions regarding this white paper should be directed to the editor or the submitters:
|Chris Holmes||Planet Labs|
|Chuck Heazel||WiSC Enterprises|
|Gary Gale||What 3 Words|
|Josh Lieberman||Tumbling Walls|
|Carl Reed||Carl Reed and Associates|
|Bart de Lathower||OGC|
This chapter discusses software engineering definitions for APIs.
API (Application Programming Interface) - An interface that is defined in terms of a set of functions and procedures, and enables a program to gain access to facilities within an application. (Definition from Dictionary of Computer Science - Oxford Quick Reference, 2016)
Interface - a shared boundary between two functional units, defined by various characteristics pertaining to the functions, physical interconnections, signal exchanges, and other characteristics, as appropriate (Definition from ISO/IEC 2382:2015 Information technology - Vocabulary)
APIs are popular today as they allow an organization to publize their APIs for reuse. Public APIs enable developers from other organizations to access the functionality provided behind the APIs. APIs are closely related to role of "platforms". For example the Facebook API provides functionality used by multiple organizations thereby advancing the market value of the Facebook platform. APIs may promote “stickiness” and "lock-in" to a platform.
Public APIs may or may not be Open APIs. Being "public" means that the API is visible and accessible outside of the organization that owns the API. The organization that controls and promotes the API owns the API. Open APIs are similar to open standards in that Open APIs are managed as a public good and managed in an open process. Open APIs adhere to open principles such as those defined by OpenStand:
2. Adherence to Principles (including broad consensus),
3. Collective Empowerment,
4. Availability, and
5. Voluntary Adoption.
OGC standards go beyond OpenStand principles as all OGC standards are free to download and are royalty-free. (See Chapter 5 for more on the meaning and values of "Open".)
APIs are tools used by software engineers and developers to design modular and interoperable systems. From an OGC open standards perspective, APIs are understood in the context of requirement specifications and implementing code.
Requirements Specifications are documents that define aspects of a protocol or data model to solve an interoperability problem. A protocol is the set of requirements that end points in a distributed system use when they communicate. The requirements may be written in plain text or in a technology neutral modeling language like UML. Web Protocol specifications include http structures that can be directly used in APIs. OGC standards specify requirements for interfaces and encodings using a modular approach (OGC 08-131r3). Implementations of an OGC standard must satisfy requirements as demonstrated by passing conformance tests. The requirements in a specification can apply across multiple API implementations.
APIs are interfaces that when backed by implementing code satisfy an interoperability need that may be documented in a Requirements Specification. The API package is a set of structure, function and procedure protocol definitions written in declarative statements that enables a program to access an application package containing implementations for these functions and procedures using these protocols. For bindings in most programming languages (C++, JAVA and other Object Oriented languages) the declarative code can be initially generated from UML defining the abstract API, using Model Drive Architecture (MDA) to produce the appropriate protocols. Implementations may use any procedures they deem appropriate as long at the functional requirements of the original specification are upheld.
Applications are software programs accessible by an API that meet user needs. Applications are distinct from system software including the operating system and system utilities. Applications implement algorithms to provide functionality behind APIs. The particulars of the algorithm are distinct from the protocols. A development strategy is to implement methods in the simplest manner possible and then do system testing and measurement to identify performance issues to optimize code, which is usually an algorithmic issue and independent of the protocol definitions.
The application programming interface (API) entry on Wikipedia defines API as an interface that supports modularity and separation of components. The components can be implemented independently with the API serving as the common point of exchange. It is the notion of "API as an Interface" that makes it relevant to the activities of the OGC.
As defined in OGC® Testbed 11 REST Interface Engineering Report, OGC Document 15-052r1, an API is an interface definition that permits invoking services from application programs without knowing details of their internal implementation.
The OGC APIs listed next show the diversity of the API concept while still retaining the core concept that "APIs are interfaces."
The OGC GeoAPI standard provides a set of Java language programming interfaces for geospatial applications. The GeoAPI interfaces closely follow the abstract model and concrete specifications published collaboratively by the International Organization for Standardization (ISO) in its 19100 series of documents and the Open Geospatial Consortium (OGC) in its abstract and implementation standards.
The GeoAPI interfaces provide a layer which separates client code from application code, which implements the API. This follows a similar pattern to the well-known JDBC API which provides a standardized interface to databases. Clients can use the JDBC API without concern for the particular implementation which they will use. Various groups have implemented different subsets of GeoAPI, see the list of known implementations for details.
The OGC SensorThings API provides an open, geospatial-enabled and unified way to interconnect the Internet of Things (IoT) devices, data, and applications over the Web. At a high level the OGC SensorThings API provides two main functionalities and each function is handled by a part. The two parts are the Sensing part and the Tasking part. The Sensing part provides a standard way to manage and retrieve observations and metadata from heterogeneous IoT sensor systems. The Tasking part is planned as a future work activity and will be defined in a separate document as the Part II of the SensorThings API.
The SensorThings API part 1 identifies resources as entities that are relevant to IoT Sensing, e.g., Thing, Observation, Location, Sensor. SensorThings API then grounds entities into entity sets and identifies URI components for requesting CRUD actions on those resources. The CRUD actions are specified using the HTTP protocol as well as MQTT protocol. MQTT is used to enhance the SensorThings service publish and subscribe capabilities. SensorThings API follows OData’s specification for requesting entities. However, SensorThings API does not follow the OData Common Schema Definition Language and should not be seen as an OData compliant API. SensorThings API’s future work will explore possible harmonization between SensorThings API and OData.
The OGC netCDF Standard is a set of data formats that support the creation, access, and sharing of array-oriented scientific data. The netCDF data model and encodings are particularly well suited to providing data in forms familiar to atmospheric and oceanic scientists.
Related to the data model and encodings listed in the netCDF standard are a set of APIs. Considering the popularity of the netCDF APIs they may be included in future extensions of the OGC netCDF standard. Bindings for C, Fortran, C++, Java are described in the CF-netCDF Core and Extensions Primer.
The Web follows the Internet approach of Protocol-Based Interoperability where the protocols specify the syntax, semantics, and sequencing constraints of the messages interchanged. Protocols designed to be resilient in the face of widely varying environments have helped the Web scale and have facilitated communication across multiple trust boundaries. Traditional APIs do not always take these constraints into account. One effect of protocol-based design is that the technology shared among components often lasts longer than the components themselves. To support Web development the concept of a WebAPI was defined. A WebAPI is an API for either a web server or a web browser.
OGC Web Service (OWS) Standards define Protocols that can be implemented as WebAPIs. The OWS standards are Requirements Specifications that define a protocol and in some cases a data model. The OWS standards include WMS, WMTS, WFS, WCS, WPS, WCPS, SOS, SPS, CSW, and OpenLS.
The OWS standards define requirements to be satisfied by both client-side and server-side WebAPIs. The OWS standards as currently written do not specify the declarative statements needed for an API package. The OWS standards do not specify API design rather they specify protocols that provide requirements for APIs. Multiple APIs with differing designs can satisfy the OWS requirements (See Section 2 Web Mapping examples for further discussion).
The OWS standards are the basis for OGC Conformance Tests that are used to confirm that functional requirements are met by implementations. OGC Compliance Tests have predominantly focused on server-side testing, while more recently client-side compliance tests have been defined for WMS.
REST (Representational State Transfer) is a term coined by Roy Fielding in his doctoral dissertation to describe an architectural style for “distributed hypermedia systems” such as the Worldwide Web that are to have desirable characteristics including separation of concerns, scalability, resiliency, visibility, and reliability. The REST architectural style rests on five constraints on how computing systems are configured and component interactions are carried out (see for example A RESTful Tutorial). Examples of non-open standard RESTful web mapping APIs are provided in Appendix A.
OGC Web Services (OWS) were defined before the publication of REST principles. OGC has evaluated the implications of REST vis-a-vis OWS finding that the existing OWS Standards have aspects of REST but also that OGC services should be migrated to, or at least make available as more RESTful bindings. For example, The OGC Testbed 11 REST Interface Engineering Report defines possible uniform practices for developing bindings or interaction styles for OGC Web services that appropriately leverage REST principles. OGC Testbed 12 continues these developments and will conclude late in 2016.
This Chapter considers the variation in existing geospatial APIs as the motivation for proposing API design based on open standards. The objective is to promote wide spread interoperability based on open API design while maintaining competitive opportunities.
There are many public APIs. As of November 2016, The Programmable Web listed over 16,000 public APIs. (See Chapter 5 for distinction between public and open.) Richardson identified challenges with the proliferation of APIs including
"Thousands of APIs are superficially similar but mutually incompatible
Everyone has a slightly different view of basic real-world concepts like “person” and “event”
This lack of agreement makes it impossible to create client-side software that can be reused between APIs
It’s not hard to find two APIs that do exactly the same thing but have nothing in common except the application/json media type."
The recent proliferation of APIs for geospatial applications has degraded the interoperability previously established by open standards. This degradation is do both to the variability of API practices across the IT industry as well as variability in geospatial APIs specifically.
Advancements in API practices is needed across the software development profession. "APIs are often difficult to use, and programmers at all levels, from novices to experts, repeatedly spend significant time learning new APIs. APIs are also often used incorrectly, resulting in bugs and sometimes significant security problems." ("Improving API Usability" in the Communications of the ACM)
APIs for web mapping were listed in The Programmable Web with the "top ten" being: Google Maps, Microsoft Bing Maps, OpenLayers, Foursquare, OpenStreetMap, MapQuest, Mapbox, CartoDB, Esri, and Yahoo. Some of these APIs implement the WMS Protocol either explicitly or as an extension - others do not support WMS, defining their own semantics and syntax for web mapping.
Annex A lists details and code snippets for these Web Mapping API examples: Google Maps, OpenLayers, MapQuest API for OpenStreetMap, Leaflet. Additionally two more RESTful examples are listed in Annex A: Esri ArcGIS REST and Mapbox.
The Web Mapping APIs listed in the Annex A use similar concepts sometimes in attributes differing names. The Table below lists two such attributes - "Center" and "Zoom" - that are in the four example APIs. "Center" is the center point of the web map to be displayed. "Zoom" is related to the scale of the web map to be displayed. The concepts for these attributes are similar in each API but the precise specification is different in each case.
The variability of similar concepts across different APIs as seen in Table 1 suggests an opportunity to quickly converge on a small set of "essential" attributes usable across otherwise disparate APIs. This simple example motivates the discussion about geospatial domain mapping and OGC Essentials for geospatial APIs based on open standards.
Table. Comparison of Map-related attributes from several APIs
Interoperability is a primary objective of the OGC. Consistent semantics across multiple servers independent of underlying API leads to interoperability and the ability to merge results from the multiple servers. The figure below was created by making individual requests to several map services (the text in each “square” indicates the name of the server). The figure illustrates what can be done if services use the same indexing systems. The figure uses the “indexing system” defined in the WMTS Simple.
To address the challenges of API proliferation addressed in Chapter 2, this Chapter identifies several methods based on open standards for the design of Geospatial APIs based on open Requirement Specifications.
These topics are addressed in the following sections for this Chapter:
Modeling the geospatial domain
API design based on geospatial requirements
API Specification and Management
OGC Essentials for API elements
Managing multiple API Bindings
Layered API design
API Security Patterns
The Figure below provides a summary of the topics discussed above Geospatial API Design using open standards.
The following sections includes Recommendations from "Web APIs for Environmental Data", a report from CSIRO (See Annex B). The CSIRO report provides Recommendations as a basis to design consistent, cross-organization APIs for access to a range of environmental data products. Observations from the CSIRO report and listed in this Chapter provide examples of opportunities for improving geospatial API design.
Designing for a connected set of concepts will lead to more robust interactions with domain resources, long term evolvability and fitness for use (Spec Driven Development). Design should consider an interconnected set of API resources with clear ideas and support of why and how API consumers will follow certain paths. This supports the objective that APIs may not just compete by features, but also by how easily and robustly they can be used ( API Maps: Hypermedia API Design)
The CSIRO API Report (Annex B) makes the following observations about modeling the environmental data domain.
Observation 18: Involve domain specialists in resource design. In organizations as complex as multi-disciplinary environmental agencies, there won’t be a single information model or Web API to rule them all. Rather there exists multiple domain contexts and words with different meaning, e.g. what is a monitoring station? Any domain-level simplifications should be reviewed within these contexts to ensure they make sense.
Observation 19: Consider point, grid and trajectory abstractions. This has been used effectively in existing APIs as a way of delineating different data types.
Multiple map APIs designed with a consistent domain model will improve development using multiple APIs, e.g., a developer can use an open API to draw the map, open map data from multiple sources in combination with commercial APIs say for advanced features such as traffic or routing. Capturing the geospatial domain in open Requirements Specifications provides for achieving such domain interoperability.
One aspect of this analysis is the use of Landing Pages to document APIs. The advent of distributed Hypermedia, in particular the World Wide Web, has introduced a new pattern of distributed processing. In this pattern, users follow links from one resource to the next. Each resource provides the information needed to proceed to the next set of resources. This pattern assumes a simple protocol (URLs over HTTP). Landing Pages are an emerging approach to document an API using this pattern. A landing page is a collection of web pages which document the API. The web pages also include URLs which can be accessed using the HTTP protocol to exercise the advertised capabilities. Two examples of landing page APIs are: John Deere and Twitter
As discussed in Chapter 1, Requirements Specifications define protocols and schemas for interoperability across distributed heterogeneous distributed information systems. APIs that implement the Requirement Specifications support such interoperability.
The OGC Web Service (OWS) Standards are requirements specifications that define geospatial interoperability. The OWS Standards define data structures, behaviors and semantics for protocols based on http. The OWS Standards were initially defined based on a Service Oriented Architecture (SOA). The OGC Web Map Service was defined with a service interface including multiple operations, e.g., GetMap. OGC is currently reviewing the OWS Standards in a discussion will lead to addition of more RESTful protocols based on a Resource-Oriented Architecture (ROA). (See OGC Testbed 11 REST Interface Engineering Report)
Consistent with ROA and supportive of RESTful APIs, geospatial resources need to be identified in OGC standards. Geospatial resources include: feature collection, feature, coverage, map, layer, trajectory, etc. Relationships are also important to be modeled, e.g. topological associations between features, parent-child associations, etc.
Based on identification of geospatial resources RESTful protocol signatures are being defined for OWS standards. For example:
An example defining a relation between a requirements specification and implementing APIs is available in these two standards from the Intelligent Transport Systems (ITS) Domain:
Unified gateway protocol requirements and specification for vehicle-ITS-station gateway (V-ITS- SG) interface (ISO 13185-2)
Unified vehicle interface protocol (UVIP) server and client API specification (ISO 13185-3)
Corresponding to services defined in ISO 13185-2, the UVIP Java Client API (ISO 13185-3) contains services that can be called by any application to execute a service. A UVIP Client Application communicates with a UVIP Server Application.
Resources, protocols, semantics and behaviors defined in Geospatial Requirement Specifications become the basis for API Specification and Management.
API specification and management are crucial software engineering functions for any organization using information technology. As the popularity of APIs has grown in the past few years, so too have the tools, best practices and consulting services that support organizations in API specification and management. While these tools were initially developed for organizations to manage their organization-specific APIs, the tools can also be considered for use in API design in an open standards context. Here we consider one such tool: the Open API Specification formerly known as the Swagger specification.
Documenting APIs can be aided by using the approach of the OpenAPI Initiative (OAI). OAI is focused on defining a vendor neutral API Description Format based on the Open API Specification (OAS). The approach will allow specification of REST APIs using modular sub-elements. Sub-elements can then live on their own and be shared by multiple APIs. More information about OAI and OAS is available in Annex C.
Providing complete documentation of your API using OpenAPI is a W3C Data on the Web Best Practice. This best practice for APIs is being discussed in the OGC/W3C Spatial Data on the Web Working Group.
Swagger was used in the OGC WaterML2.0 part 2 Interoperability Experiment to document the CSIRO RESTful API. The CSIRO API Report (Annex B) makes the following observations:
Observation 9: Make use of automated API documentation where possible. These can often be synched directly with an implementation version, which helps to minimize divergence. Some also provide interactive (e.g. Swagger) documentation that allows inline requests to be made. This helps to lower the barrier of entry for developers and quickly builds understanding.
Observation 10: Avoid bleeding edge API description languages and response patterns. There has been an explosion in the number of these. Picking a winner is difficult. Using an overly complex, non-supported service description framework and/or response structure can be an impediment to developers.
An advantage of documenting APIs with OAS is the industry-wide tools and practices that are based on OAS. The first Figure below shows how OAS enables robust management of software development lifecycle activities. The second figure below shows the automated generation of client libraries based on code generation tools. Use of OAS also helps organizations with API response types, including error handling, link updates, moved and re-aggregated resources, etc.