Publication Date: 2018-01-26

Approval Date: 2018-01-22

Posted Date: 2017-11-15

Reference number of this document: OGC 17-032r2

Reference URL for this document: http://www.opengis.net/doc/PER/t13-FA001

Category: Public Engineering Report

Editor: Anneley McMillan, Sam Meek

Title: OGC Testbed-13: Aviation Abstract Quality Model Engineering Report


OGC Engineering Report

COPYRIGHT

Copyright © 2018 Open Geospatial Consortium. To obtain additional rights of use, visit http://www.opengeospatial.org/

WARNING

This document is not an OGC Standard. This document is an OGC Public Engineering Report created as a deliverable in an OGC Interoperability Initiative and is not an official position of the OGC membership. It is distributed for review and comment. It is subject to change without notice and may not be referred to as an OGC Standard. Further, any OGC Engineering Report should not be referenced as required or mandatory technology in procurements. However, the discussions in this document could very well lead to the definition of an OGC Standard.

LICENSE AGREEMENT

Permission is hereby granted by the Open Geospatial Consortium, ("Licensor"), free of charge and subject to the terms set forth below, to any person obtaining a copy of this Intellectual Property and any associated documentation, to deal in the Intellectual Property without restriction (except as set forth below), including without limitation the rights to implement, use, copy, modify, merge, publish, distribute, and/or sublicense copies of the Intellectual Property, and to permit persons to whom the Intellectual Property is furnished to do so, provided that all copyright notices on the intellectual property are retained intact and that each person to whom the Intellectual Property is furnished agrees to the terms of this Agreement.

If you modify the Intellectual Property, all copies of the modified Intellectual Property must include, in addition to the above copyright notice, a notice that the Intellectual Property includes modifications that have not been approved or adopted by LICENSOR.

THIS LICENSE IS A COPYRIGHT LICENSE ONLY, AND DOES NOT CONVEY ANY RIGHTS UNDER ANY PATENTS THAT MAY BE IN FORCE ANYWHERE IN THE WORLD. THE INTELLECTUAL PROPERTY IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NONINFRINGEMENT OF THIRD PARTY RIGHTS. THE COPYRIGHT HOLDER OR HOLDERS INCLUDED IN THIS NOTICE DO NOT WARRANT THAT THE FUNCTIONS CONTAINED IN THE INTELLECTUAL PROPERTY WILL MEET YOUR REQUIREMENTS OR THAT THE OPERATION OF THE INTELLECTUAL PROPERTY WILL BE UNINTERRUPTED OR ERROR FREE. ANY USE OF THE INTELLECTUAL PROPERTY SHALL BE MADE ENTIRELY AT THE USER’S OWN RISK. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR ANY CONTRIBUTOR OF INTELLECTUAL PROPERTY RIGHTS TO THE INTELLECTUAL PROPERTY BE LIABLE FOR ANY CLAIM, OR ANY DIRECT, SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES, OR ANY DAMAGES WHATSOEVER RESULTING FROM ANY ALLEGED INFRINGEMENT OR ANY LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR UNDER ANY OTHER LEGAL THEORY, ARISING OUT OF OR IN CONNECTION WITH THE IMPLEMENTATION, USE, COMMERCIALIZATION OR PERFORMANCE OF THIS INTELLECTUAL PROPERTY.

This license is effective until terminated. You may terminate it at any time by destroying the Intellectual Property together with all copies in any form. The license will also terminate if you fail to comply with any term or condition of this Agreement. Except as provided in the following sentence, no such termination of this license shall require the termination of any third party end-user sublicense to the Intellectual Property which is in force as of the date of notice of such termination. In addition, should the Intellectual Property, or the operation of the Intellectual Property, infringe, or in LICENSOR’s sole opinion be likely to infringe, any patent, copyright, trademark or other right of a third party, you agree that LICENSOR, in its sole discretion, may terminate this license without any compensation or liability to you, your licensees or any other party. You agree upon termination of any kind to destroy or cause to be destroyed the Intellectual Property together with all copies in any form, whether held by you or by any third party.

Except as contained in this notice, the name of LICENSOR or of any other holder of a copyright in all or part of the Intellectual Property shall not be used in advertising or otherwise to promote the sale, use or other dealings in this Intellectual Property without prior written authorization of LICENSOR or such copyright holder. LICENSOR is and shall at all times be the sole entity that may authorize you or any third party to use certification marks, trademarks or other special designations to indicate compliance with any LICENSOR standards or specifications.

This Agreement is governed by the laws of the Commonwealth of Massachusetts. The application to this Agreement of the United Nations Convention on Contracts for the International Sale of Goods is hereby expressly excluded. In the event any provision of this Agreement shall be deemed unenforceable, void or invalid, such provision shall be modified so as to make it valid and enforceable, and as so modified the entire Agreement shall remain in full force and effect. No decision, action or inaction by LICENSOR shall be construed to be a waiver of any rights or remedies available to it.

None of the Intellectual Property or underlying information or technology may be downloaded or otherwise exported or reexported in violation of U.S. export laws and regulations. In addition, you are responsible for complying with any local laws in your jurisdiction which may impact your right to import, export or use the Intellectual Property, and you represent that you have complied with any regulations or registration procedures required by applicable law to make this license enforceable.

Table of Contents

1. Summary

This OGC® Engineering Report (ER) describes an Abstract Quality Model (AQM) for data in the aviation domain. Requirements for data quality in aviation are stringent, as the data is often used for safety critical purposes. The services considered are those that serve aeronautical information, flight information and weather forecasting. The model is built upon recognized standards of the International Organization for Standardization (ISO) with extensions and additions made according to the requirements of the domain. These requirements include an ability for the model to record information about the precision of measurements and an understanding of the timeliness of a piece of data, as information utility degrades with time. The result is an ISO compliant data quality model with the required extensions included.

1.1. Requirements

This document answers the requirement for an Abstract Data Quality Model to be designed to enable service advertising, service validation and assurance within the Aeronautical Domain.

In service advertising, a service makes known to a potential consumer the quality of the data provided by the service. Based on this information, the consumer can determine whether or not the service meets their needs.

In service validation, assurance is given that the quality of the data provided by a service is consistent with the quality that is explicitly defined in a service contract or any kind of agreement that may exist between a service provider and service consumer.

Both use cases share two common preconditions:

  1. an unambiguous definition of the concept of data quality exists

  2. a set of measurable parameters that allow specifying data quality is defined. This Abstract Data Quality model provides the basis for these components.

1.2. Key Findings and Prior-After Comparison

The Abstract Quality Model forms a basis for implementing data quality services specifically for the aviation domain. Prior to this ER, data quality services were reliant on the standards inherited into the three aviation exchange models (Aeronautical Information Exchange Model (AIXM), Flight Information Exchange Model (FIXM) and Weather Information Exchange Model (WXXM)), these vary from not having any capability to record data quality to the data quality metrics for ISO 19157. Although the ISO standard is recognized and provides some of the required data quality functionality, it is missing the key concepts of timeliness and precision. The abstract quality model described in this document introduces these concepts to the AQM in a standards-compliant way.

1.3. What does this ER mean for the Working Group and OGC in general

The ER contributes to the Data Quality DWG by providing a re-interpretation of the ISO elements and extensions to the ISO models to record mission critical information in the aviation domain. Additionally, the Aviation DWG should be consulted as to the utility of this ER.

1.4. Document contributor contact points

All questions regarding this document should be directed to the editor or the contributors:

Table 1. Contacts
Name Organization

Sam Meek

Helyx SIS Ltd

Anneley McMillan

Helyx SIS Ltd

Joan Masó

UAB-CREAF

1.5. Future Work

An emerging requirement mentioned throughout this ER is data quality metrics in heterogeneous data created through either informal or formal conflation processes. To address this emerging requirement, future work will need to consider:

  • Efficient ways to store feature level and attribute level metadata

  • Optimal ways for presenting feature level and attribute level metadata to end users

1.6. Foreword

Attention is drawn to the possibility that some of the elements of this document may be the subject of patent rights. The Open Geospatial Consortium shall not be held responsible for identifying any or all such patent rights.

Recipients of this document are requested to submit, with their comments, notification of any relevant patent claims or other intellectual property rights of which they may be aware that might be infringed by any implementation of the standard set forth in this document, and to provide supporting documentation.

2. References

The following normative documents are referenced in this document.

  • OGC 06-121r9, OGC® Web Services Common Standard

  • OGC 15-097r1, Geospatial User Feedback Standard. Conceptual model

  • OGC 15-098r1, Geospatial User Feedback Standard. XML Encoding extension

  • OGC 16-050, OGC® Imagery Quality and Accuracy ER

  • ISO 19115-1:2014, Geographic information - Metadata - Part 1: Fundamentals

  • ISO 19115-2:2009, Geographic information - Metadata - Part 2: Extensions for imagery and gridded data

  • ISO 19115-3:2016, Geographic information - Metadata - Part 3: XML schema implementation for fundamental concepts

  • ISO 19119:2016, Geographic information - Services

  • ISO 19138:2006, Geographic information - Data quality measures

  • ISO 19156:2011, Geographic information - Observations and Measurements

  • ISO 19157:2013, Geographic information - Data quality

  • AIXM, Air Information Exchange Model

  • FIXM, Flight Information Exchange Model

  • WXXM, Weather Information Exchange Model

3. Terms and definitions

For the purposes of this report, the definitions specified in Clause 4 of the OWS Common Implementation Standard OGC 06-121r9 shall apply. In addition, the following terms and definitions apply.

3.1. accuracy

closeness of agreement between a test result or measurement result and the true value.
[SOURCE: ISO 3534-2:2006, 3.3.1]
a degree of conformance between the estimated or measured value and true value.
[SOURCE: ICAO Annex 15]

3.2. conformance

fulfillment of speicified requirements.
[SOURCE: ISO 19105:2000, 3.8]

3.3. data quality basic measure

generic data quality (4.21) measure used as a basis for the creation of specific data quality measures.
[SOURCE: ISO 19157:2013, 4.7]

3.4. quality

degree to which a set of inherent characteristics fulfills requirements.
[SOURCE: ISO 9000:2005, 3.1.1]
a degree or level of confidence that the data provided meets the requirements of the data user in terms of accuracy, resolution and integrity.
[SOURCE: ICAO Annex 15]

3.5. integrity

a degree of assurance that an aeronautical data and its value has not been lost or altered since the data origination or authorized amendment.
[SOURCE: ICAO Annex 15]

3.6. Precision

the smallest difference that can be reliably distinguished by a measurement process.
[Source: ICAO Annex 15]

3.7. Timeliness

speed of dissemination of the data - i.e., the lapse of time between the end of a reference period (or a reference date) and dissemination of the data.
[SOURCE: OECD Glossary of Statistical Terms, 3090, https:stats.oecd.org/glossary/detail.asp?ID3090]

4. Abbreviated terms

  • ADQR Aggregated Data Quality Results

  • ADS-B Automated Dependent Surveillance - Broadcast

  • AGL Above Ground Level

  • AIM Aeronautical Information Management

  • AIS Aeronautical Information Services

  • AIRAC Aeronautical Information Regulation and Control

  • AIXM Aeronautical Information Exchange Model

  • ANSPs Air Navigation Service Providers

  • AQM Abstract Quality Model

  • ANSPs Air Navigation Service Providers

  • API Application Program Interface

  • ASDI Aircraft Situation Display to Industry

  • ATM Air Traffic Management

  • BARR Block Aircraft Registration Request

  • COM Component Object Model

  • CORBA Common Object Request Broker Architecture

  • COTS Commercial Off The Shelf

  • DCE Distributed Computing Environment

  • DCOM Distributed Component Object Model

  • ETMN European Air Traffic Management Network

  • FIXM Flight Information Exchange Model

  • GUF Geospatial User Feedback

  • ICAO International Civil Aviation Organization

  • IDL Interface Definition Language

  • ISO International Standards Organization

  • LNAV Lateral Navigation

  • MSL Mean Sea Level

  • NOAA National Oceanographic Atmospheric Association

  • NOTAM Notification to Airman

  • O&M Objects and Measurements

  • QoS Quality of Service

  • QualityML Quality Indicators Dictionary and Markup Language

  • SDCM Service Description Conceptual Model

  • SES Single European Sky

  • SESAR Single European Sky ATM Research

  • SLA Service Level Agreement

  • SWIM System Wide Information Management

  • UoM Unit of Measurement

  • VNAV Vertical Navigation

  • WSQM Web Service Quality Model

  • WXXM Weather Information Exchange Model

5. Overview

Abstract
Aviation is a domain that has stringent quality requirements for data due to the safety critical nature of operations. Data quality metrics are also utilized as a secondary measure assessing decision-making capability as the fitness for purpose of a dataset will be governed largely by regulation.

This document describes the Abstract Quality Model (AQM) for the Aviation domain. There are three types of data that the AQM is designed to complement that are supported by information exchange models, these are: AIXM, FIXM and WXXM. The model is built upon several different standards including the Accuracy, Currency, Completeness and Consistency (A3C) framework and ISO standards 19115-1, 19115-2 and 19157. The aviation domain has specific data quality requirements that are not covered sufficiently by the standards, these are Timeliness and Precision. Timeliness is linked to the Temporal concepts in the ISO standards, but is fulfilled by update frequency measures in the AQM. Precision refers to the number of decimal places that the data contain, not the repeatability of the measurement as described in ISO 19157, the concept of precision is addressed via extensions to the ISO standards for the relevant data quality elements. Extended elements include measurements of accuracy, temporal measurements and quantitative attributes. The AQM consists of overarching metaclasses taken from the A3C model with descriptive elements lifted from the ISO standards.

Business value
This ER describes the Abstract Quality Model in the Aviation domain, it is part of a larger set of work items that include:

  • FA002 - Data Quality Specification ER

  • FA003 - Quality Assessment Service ER

  • FA004 - Geospatial Taxonomies ER

Together these work items provide a thorough investigation into the state and direction of data quality in the Aviation Domain.

Keywords
OGCdocs, Aviation, Testbed-13, Data quality, Abstract quality model

Proposed OGC Working Group for Review and Approval
Data Quality DWG

6. Background

Standardized quality fields are required for aviation data services as they allow users to assess whether a particular service is fit for purpose. Some services offer safety critical data, others offer context or situational awareness. Data quality use cases are varied, in some instances accuracy is the most important measure, for others it maybe completeness. A likely scenario is that a combination of quality measures are required to provide an overall description of fitness for purpose. A manager responsible for service creation and maintenance may not know the application of the data offered is ahead of time, therefore benchmarking, assessing and displaying a number of common measures of quality is preferable to bespoke metrics designed at point of service inception. Additionally, a standardized set of metrics allows for coherent comparison of datasets and services and lends itself to automated processing and decision-making.

A set of conformance metrics are required to certify a service as suitable for a use case such as navigation or situational awareness. In addition to choosing services, quality related metadata assists across the workflow from data ingest, analysis, visualization and dissemination.

This ER is consistent with the scope set out by Testbed 12 (e.g. Imagery Quality and Accuracy ER), covering the following aspects of data quality: completeness, logical consistency, positional accuracy, temporal accuracy and thematic accuracy. In particular the ER addresses the following issues:

  • Quality parameters offer a way to compare and determine the right data that is fit for purpose. Standard vocabularies and taxonomies are paramount to describing data quality and providing the ability for service comparison. We consider the A3C model quality framework and its fitness for purpose for describing aviation data quality. We then consider other pertinent standards such as ISO 19157, 19115, 19119 and the QualityML encoding standard. Comparing aviation requirements with these standards allows the development of an abstract quality model (AQM) for aviation, including extensions and implementation guidance where required.

  • Certain data quality measures are set at the service or dataset level (for instance a data service may have been authored by one authoritative source). This level of data quality is discussed in Dataset level quality. This is in contrast to the feature or attribute level metrics that are allowed via the scope code in many of the aforementioned ISO standards. The aviation domain currently works at the dataset level, and therefore this AQM is designed to describe assets at the dataset level whilst considering future work at the feature level and applicability at the service level.

  • Other data quality measures are likely to require metrics below the dataset level, for instance the currency of a Notice To Airman (NOTAM) and other airspace information will be set at the airspace feature level. Therefore, a general way of expressing quality at levels below dataset is needed. These topics are discussed in the feature level quality section in terms of a way forward for future iterations of the abstract quality model.

  • Data quality in instances of conflation and fusion is a topic of interest in the Data Quality DWG and is discussed in the aviation data aggregation section.

  • Spatial data quality is governed by standards including the ISO 9000 series, the AQM should be in line with these recognized standards.

7. Types of aviation services

There are three types of aviation data services that fall within the scope of the AQM, these are; aeronautical information, which includes information about structures, routes and procedures. The second is flight information including information about aircraft such as routes, flows and coordination. The third type of information is meteorological, which contains information primarily about the weather and aircraft operating conditions. These three data types have different requirements with respect to data quality that are discussed in the following sections. Briefly, meteorological information is likely to have a strong temporal and currency element to it. From an operational perspective, knowing the weather at a given location at any time is important for flight operations, however forecasting is also important and includes measures of uncertainty. Flight information has an even more stringent requirement for currency from a safety perspective, it is important to know where flights are to a high degree of accuracy at any given time. In contrast to these services, elements of aeronautical information are required to be up-to-date, but high temporal frequency of updates are not likely to be required as aircraft infrastructure such as aerodromes do not often change and when changes do occur, they are planned and communicated to the relevant authorities.

7.1. Aeronautical information

Aeronautical information services provide data concerning the physical infrastructure in the aeronautical domain. This data is encoded in AIXM 5.1.1, which is openly documented. The model is maintained by EUROCONTROL and the FAA and is under active development. AIXM is built on recognized, open standards including ISO. From a data quality perspective, AIXM inherits the data quality metrics of ISO 19157 directly into the model, therefore any abstract quality model should also conform to similar standards to ensure simple interoperability.

7.2. Flight information

FIXM 3.0.0 is the flight and aircraft flow information model that is standardized across the sponsoring flight authorities. The flight information is a model that contains data about supporting aerodromes, flights and addresses. Many of the features are inherited from ISO standards, however, the model does not appear to contain any objects from the ISO 19115-1/19157 models, therefore the quality of the flight information is not considered as part of the model.

7.3. Meteorological information

Meteorological data is required in the aviation domain for flight planning, monitoring and traffic management use cases. Weather data can be complex, therefore it has been standardized as WXXM 2.0. The standard is built on ISO 19156 Observations and Measurements (O&M) and includes references to the AIXM model.

8. Aeronautical information services

As Aeronautical information transitions from publications and proprietary systems to digital and Aeronautical Information Management (AIM) services, an array of aeronautical information is being made available. High quality, timely and on-demand air information is already being produced by agencies and Air Navigation Service Providers (ANSPs) across the world. Development of digital services will increasingly expose a variety of official Aeronautical Information Services (AIS), commercial aviation services, military services and open source services to professional industry users with limited access to public users. As more services are made available, methods of quickly ascertaining the quality of services, the data they offer and their fitness for purpose become increasingly important. In order to provide a consistent, authoritative method of comparison, standardized data quality elements are required. This involves understanding standard data quality elements, but also the specifics of data quality in the aviation domain. This document aims to introduce some of these issues and provide an AQM to act as a basis for data quality system and service development.

As stated in the International Civil Aviation Organization (ICAO) Annex 15[1], one of the least known and most vital roles in support of international civil aviation is filled by the aeronautical information service (AIS). The objective of the AIS is to ensure the flow of information necessary for the safety, regularity and efficiency of international air navigation. Each nation state must provide an AIS for the collection and distribution of aeronautical information for use by all types of aircraft operations. It also specifies that the state concerned remains responsible for the aeronautical information post publication. When the aeronautical information is published on behalf of a State, it must be clearly indicated that it is published under the authority of that State, thus providing a party responsible for maintaining the consistency of the service.

Although Aeronautical Information Publications (AIPs) have traditionally been paper based products followed by digital documents, data providing services are becoming more prevalent. A notable example is EUROCONTROL’s Network Manager Business-to-business (NM B2B) Web Services[2] that gives a set of application program interfaces (APIs) to eligible operational stakeholders enabling the development of applications, using web services, for establishing direct interfaces with the EUROCONTROL Network Manager’s operational systems and data. For interoperability reasons, NM B2B Web Services is based on open web technologies that do not require the installation of proprietary software on the client side, and follows the architecture standards recommended by the System Wide Information Management (SWIM) concept. This transition from paper publications to digital services provides new opportunities, and responsibilities, to enhance data quality information available to service users.

8.1. Military services

Certain products and services are particularly important to military use, for instance military charting, and Digital Vertical Obstruction File data for low flying. Although not a focus of this ER, it is recognized that military users have their own data quality requirements compared to civilian flight requirements that should be addressed if the model is to be used in military services. The military operates under a different rule set compared to the compliance requirements for civil aviation bodies. However, this moratorium on regulation is due to shortly expire and compliance will be required. Additionally, the military have a right to fly over the nation to which they belong, however they often have a duty of care to civil air users that is not reciprocated. Flight planning for military aircraft is often short notice, does not have a solid, defined flight plan due to the response to threats. Therefore, the military, as users of air information, are likely to require access to timely flight data and other air information at all times with the added constraint of the duty of care. An additional requirement is information classification and the military reliance on disconnected networks that must be considered by an implementing system.

8.2. Commercial services

Commercial providers range from joint public and private cooperatives to fully private companies and even individuals providing aviation services. As many organizations shift from AIS to AIM, commercial vendors will provide a range of enabling services such as aeronautical charting, flight procedures and airspace services to customers. Although ideally, these services would comply with new ICAO standards on the management, handling and aggregation of data for publication as aeronautical information using automated AIM systems, in reality, service quality is variable. For certain purposes, data quality is less critical, whereas in others, high data quality is mandated. Although data services within a particular commercial system may display internal integrity and compliance with International Civil Aviation Organization (ICAO) standards, where they may be exported, aggregated and disseminated further, there is the potential for loss of integrity, or a loss of an auditable data trail. Data quality metadata assists in maintaining high data quality standards through individual measures, but also through the concept of lineage, or a record of the data originator and processing history.

8.3. Public / open services

Commercial, government and open source communities also provide free-to-use services to the public for interest or information. These tend to be flight trackers, aggregated from Automated Dependent Surveillance - Broadcast (ADS-B) and RADAR data, or airspace exploration. Data here can be provided as is without detailed metadata and quality information as it is not designed to be used beyond being a public information service. This type of service is not considered as part of the abstract data quality model due to its lack of accreditation possibilities and public target audience.

9. Quality measures in the aviation domain

This section outlines quality measures as they are understood from supporting, regulatory, documentation in the Aviation Domain.

9.1. Convention on civil aviation

The ICAO Convention on International Civil Aviation, also known as the Chicago Convention (Doc 7300)[3] is a major defining endeavor within the domain. It is useful to consider the requirements for data quality from official aeronautical regulatory framework for data quality measures that may be suitable to display in data discovery operations.

9.1.1. ICAO Annex 3 Meteorological service for international air navigation

This documentation[4] released in 2007 outlines the international standards and recommended practices for meteorological services for air navigation. Section 2.2 makes recommendations for the supply, quality assurance and use of meteorological information. Briefly, these include:

  • Maintenance of a close liaison between the provider and the user of the information.

  • Recognition that the contracting state is responsible for creating and maintaining a suitable quality procedure to validate meteorological data feeds.

  • State quality systems are likely to be unique, but should conform to ISO 9000 series of standards.

  • Data quality should be measured in terms of its:

    • Spatial coverage.

    • Format.

    • Content.

    • Timeliness.

    • Frequency of update.

    • Period of validity.

  • Timeliness of message exchange is of particular importance and should be considered as part of any implementing system.

  • Data quality compliance of services should be determined by an auditing process.

  • Human factors should be considered in a data quality system to ensure system usability.

The data quality measures determined above, by the numbered bullets, are largely covered by the ISO standards, ISO 19157 in particular. However, frequency of update and timeliness are not covered explicitly in the mentioned standards, extensions are required to form the AQM defined in this document. The quality measures for weather differ according to the currency of the forecast, current weather can be measured and its accuracy determined by observation, whereas forecasts are based upon prediction that should be assessed separately although recorded in the AQM implementation.

9.2. ICAO Annex 15 Aeronautical information services

The object of the AIS is to ensure the flow of information/data necessary for the safety, regularity and efficiency of international air navigation. Annex 15 states the Standards and Recommended Practices for Aeronautical Information Services. Within the document are a several definitions that are useful for aeronautical service quality elements:

Data quality: A degree or level of confidence that the data provided meets the requirements of the data user in terms of accuracy, resolution and integrity.

Accuracy: Accuracy is defined as the degree of conformance between the estimated or measured value and the true value. With measured positional data the accuracy is normally expressed in terms of a distance from a stated position within which there is a defined confidence of the true position.

Precision: Precision is defined by ICAO Aeronautical Information Services Annex 15 as: “The smallest difference that can be reliably distinguished by a measurement process.” This refers to the reproducibility of a measurement rather than the number of decimal places that are reported. For example, measuring the dispersion of a spatial reference from the Global Positioning System (GPS) many times to understand the likelihood of successful repeatability. In this case if the values are close together then it has a high degree of precision or repeatability.

Resolution: A number of units or digits to which a measured or calculated value is expressed and used, note that this definition is akin to many users' view of precision.

Traceability: Ability to trace the history, application or location that is under consideration. When considering product, traceability can relate to:

  • the origin of materials and parts;

  • the processing history; and

  • the distribution and location of the product after delivery.

Timeliness: Annex 15 of the Convention on Civil Aviation refers to the timeliness of aeronautical information/data services:

"8.2.5 Where automated pre-flight information systems are used to provide the harmonized, common point of access by operations personnel, including flight crew members and other aeronautical personnel concerned, to aeronautical information/data and meteorological information, the civil aviation authority or the agency to which the authority to provide service has been delegated in accordance with 3.1.1 c) shall remain responsible for the quality and timeliness of the aeronautical information/data provided by means of such a system."

Assemble: The annex also talks about assembling: A process of merging data from multiple sources into a database and establishing a baseline for subsequent processing. The assemble phase includes checking the data and ensuring that detected errors and omissions are rectified. This concept has parallels with conflation, a set of methodologies for combining data from different sources and the data aggregation concepts discussed in this document.

9.3. European Commission Aeronautical Data Quality Implementing Rule

The European Commission mandated Data Quality using Regulation 73/2010[5] that was adopted on 26 January 2010. Aeronautical Data Quality (ADQ) regulation 73/100 states requirements on the quality of aeronautical data and aeronautical information for the Single European Sky (SES). The overall objective of this rule is to achieve aeronautical information of sufficient quality, accuracy, timeliness and granularity as a key enabler of the European Air Traffic Management (ATM) Network. In terms of scope, the aeronautical information data process chain extends from the original data sources (e.g. surveyors, procedure designers, etc.) through AIS and publication to the end users of the data for aeronautical applications. It is noted in this regulation that quality requirements relating to aeronautical data and aeronautical information are not always met within the European Air Traffic Management Network (EATMN), in particular the accuracy and integrity requirements. The shift to digital data products and services aimed to address these requirements, among others. As in Annex 15, data quality is defined as a degree or level of confidence that the data provided meets the requirements of the data user in terms of accuracy, resolution and integrity, with the same definitions adopted. Step P-02 - Data integrity monitoring states: “Data integrity requirements introduced by safety objectives must be measurable and adequate.”

9.4. Aviation domain specification for data quality requirements

9.4.1. European Aviation Safety Agency - Technical requirements and operational procedures for AIS and AIM

Rulemaking Task 0593, 0594[6] and associated Notices of Proposed Amendments 2014-20 and 2016-02 extend the definition of Data Quality to: "a degree or level of confidence that the data provided meets the requirements of the data user in terms of accuracy, resolution and integrity (or equivalent assurance level), traceability, timeliness, completeness, and format'."

9.4.2. Service Description Conceptual Model (SDCM)

The Service Description Conceptual Model 1.0 (SDCM) that is developed by the U.S. Federal Aviation Administration (FAA) SWIM and Single European Sky ATM Research (SESAR) Joint Undertaking (SJU) provides a graphical and lexical representation of the properties, structure and interrelationships of all service metadata elements, collectively known as a Service Description.

The service description contains information needed in order to assess and use a service. This includes titles, descriptions, contact details, categories and dates. This service description contains a Service Profile class that provides Quality of Service (QoS) elements, including parameter names, values, definition, calculation method and units of measure. The elements only present QoS elements that are not associated with any specific service consumer. Specifics are usually governed in a Service Level Agreement (SLA). The type of parameters dealt with by the QoS component includes capacity, response time and others.

Although the SDCM contains QoS elements, it does not make provisions for the quality of the data measures within the services. For instance, it does not consider the comparison of Accuracy, Precision, Completeness or Levels of Detail within and between services to guide the user to the most appropriate service. The AQM should be linked to the SDCM as a method of recording data quality that is accessible through the SDCM service quality model.

9.4.3. Other methods of measuring service quality

This section contains a brief overview of the quality of service metrics from two well-known schemas, OASIS Web Service Quality Model (WSQM) and Quality of Service for Wireless Networks (QoS WN). QoS measurement has obvious crossovers with the abstract quality model for aviation. Note that the QoS models are generic and therefore not aviation specific. The measures can be implemented for any end point, however unlike data quality which is usually a snapshot measurement of characteristics of a dataset (it is noted that data quality can also vary over time), QoS should be measured over time, as measurements are only useful in comparison of service provision over time. Both quality of service frameworks contain the following concepts, however they differ in implementation:

  • Response Time

  • Throughput

  • Availability

  • Reliability

  • Accessibility

9.4.4. Response time

Framework

Description

WSQM

Time taken to send a request and to receive the response. The Response Time is measured at Web service call and is calculated by applying the following formula: Response Time = Response Completion Time - User Request Time. The Response Completion Time is the time that all the data for response arrives at a user, while the User Request Time is the time when the user sends a request. In general, the Response Time is calculated by the mean value during a certain time.

QoS WN

This allows the user to measure the performance of a Web Service by timing the a series of requests and responses to a service endpoint. As with the WQSM framework, the response time metric is calculated using an average of the series of requests and responses.

9.4.5. Throughput

Framework

Description

WSQM

The maximum number of services that a platform providing Web services can process for a unit time. Throughput can be used as a performance index to evaluate a Web services provider. Maximum Throughput can be calculated with the following formula: Maximum Throughput = Maximum Complete Requests/Unit Time.

QoS WN

Throughput is derived as the total number of invocations for the given period of measurement.

9.4.6. Availability

Framework

Description

WSQM

Availability is defined as the ratio of time period that a Web service exists or it is ready for use, that is, the Web service is maintained. Assuming that the time when a system is not available is Down Time and the time when a system is available is Up Time, the Availability is the average Up Time. To get Availability, instead of monitoring Up Time continuously, we suggest using the Down Time. Down Time is obtained by monitoring system down events occurred in operation. The following formula calculates the Availability while unit time is a time to measure the time: Availability = 1-(Down Time/ Unit Time).

QoS WN

This test allows the user to measure the availability of a known Web Service. User sets a period of measurement and the frequency of invocation resulting in a percentage. It is derived by using the successful invocations divided by total invocations for the given period of measurement. Total uptime – downtime / Total uptime X 100 = number of successful invocation X frequency of invocation)/period of measurement X 100% = (number of successful invocations/total invocations) X 100

9.4.7. Reliability

Framework

Description

WSQM

The approximation of Reliability in WSQM framework is Successability, which is defined as the extent to which Web services yield successful results over request messages, i.e. the degree to which a service is fulfilled in a given time according to an agreed contract. Successability can be calculated as the number of successful response messages over the number of request messages. That is, it represents the ratio of successfully returned messages after requested tasks are performed without errors: Successability = number of response messages / number of request messages

QoS WN

This test allows the user to measure the reliability of a known Web Service. User sets a period of measurement. The number of failures over a period of time is the measure of Reliability. It is derived as the unsuccessful invocations for the given period of measurement.

9.4.8. Accessibility

Framework

Description

WSQM

Accessibility represents the degree that a system is normatively operated to counteract request messages without delay. In some cases, a Web service is accessible for external users to try accessing its resources even if its services are not available. We can know whether a Web service system is accessible by just inspecting that the system can return an acknowledgement normally for a request message. Thus, Accessibility can be calculated as the ratio of number of acknowledgements received to the number of request messages. Accessibility = number of packets received / number of request messages.

QoS WN

This is a measure denoting the success rate or chance of a successful service instantiation at a point of time. User sets the number of times of invocations. User invokes the known web service at the number of times set by the user at one go. The result of this measure is in percentage. It is derived by using the successful invocations divided by total invocations for the given period of measurement. Accessibility = (Successful invocations / Total invocations) * 100

9.5. FAA Block Aircraft Registration Request (BARR)

Regulations stipulating data quality may also be used to deliberately block data, or to add lag. An example is the FAA Block Aircraft Aviation Request (BARR) program to limit aircraft data displayed via Aircraft Situation Display to Industry (ASDI). This is due to security concerns, and many commercial flight trackers have at least a 5 minute delay. From a data quality perspective, delays in data service have ramifications for different measures of quality such as completeness, but also provide challenges for products produced using aggregation or conflation techniques as different parts of a resultant product will likely produce data with heterogeneous fitness for purpose. Measurement of varying quality across an image is discussed in the Testbed 12 Engineering Report OGC 16-050 Imagery Quality and Accuracy.

9.6. Definition of data quality

There are definitions of data quality throughout the reviewed documentation, the two that are considered most relevant are as follows:

  • Degree to which a set of inherent characteristics fulfills requirements.

  • A degree or level of confidence that the data provided meets the requirements of the data user in terms of accuracy, resolution and integrity.

9.7. Requirements for data quality in the aviation domain

Throughout the review of aviation data quality documentation, mandates from different regulatory bodies, a review of the existing standards and engagement with stakeholders, the following concepts are required for an AQM:

  • Positional Accuracy

  • Completeness

  • Logical Consistency

  • Thematic Accuracy

  • Temporal Accuracy

  • Timeliness

  • Precision

10. Applicability of quality standards in the aviation domain

This section reviews the different quality standards considered in part, or in full as candidates to fulfill the requirements of the AQM in the aviation domain.

10.1. The A3C quality framework

The Accuracy, Currency, Completeness and Consistency (A3C) model quality framework was introduced by DigitalGlobe[7] as an instrument to compare imagery from multiple sources and determine the best fitness for purpose. This model was initially suggested in the Request for Participation, Annex B of Testbed 12 and was elaborated upon in the Imagery Quality and Accuracy ER. A first consideration of this work is whether the A3C quality framework is suitable to describe aviation data services. Aviation services are wider in scope than purely imagery data, they contain a mixture of features (vector) and coverages (raster) data that may represent flight data, aviation infrastructure, sensor data or meteorological data. However, the main classes of the A3C quality framework are generic and are still appropriate. The following sections discuss these classes and compare with the aviation data case.

10.1.1. Accuracy

The A3C definition of accuracy refers to the positional accuracy of a location derived from the pixel in X, Y and Z dimensions. In order to make this definition appropriate it needs to be expanded from purely pixel locations to the location of any geographically enabled feature associated with aviation data. Further than this, there are a number of aspects related to the geographic accuracy of aviation data. For instance, referencing ICAO Annex 15 of the Convention on Civil Aviation, the core accuracy requirements for horizontal and vertical reference system involve the accuracy of coordinates, the projection used, and coordinate transforms (WGS84, ITRS 2000, MSL (EGM-96)): "At those geographical positions where the accuracy of EGM-96 does not meet the accuracy requirements for elevation and geoid undulation specified in ICAO Annex 14[8], Volumes I and II, on the basis of EGM-96 data, regional, national or local geoid models containing high resolution (short wavelength) gravity field data must be developed and used."

The Z dimension is also particularly important for aviation data. Height can be referred to in a number of ways; indicated altitude, true altitude (known as altitude above Mean Sea Level (MSL)), absolute altitude (known as altitude Above Ground Level (AGL)), height above a specified datum (usually an airfield elevation) or pressure altitude known by flight levels (e.g. FL180). Some of these are more likely to be used for aviation data services (e.g. altitude above MSL) than others e.g. indicated altitude. No matter the concept of altitude used, it should be describable by some measure of accuracy, the only changeable aspect is the base reference point.

10.1.2. Currency

In A3C, currency refers to providing temporally relevant content to match customer requirements for timeliness and persistence. In imagery in particular, this includes temporal extent of imagery and products used to cover the associated area, since multiple dates of collections are typically required to cover a large area. In aviation, currency has a wider remit, and can refer to multiple factors, such as the time the data was collected, the time the data it is valid for or the time that data needs to be reviewed. As well as time periods, ICAO Annex 15 states that the calendar (Usually the Gregorian Calendar), time zone (Usually UTC) and any daylight savings are crucial. ADQ 73/100 defines an official term, the period of validity as the period between the date and time on which aeronautical information is published and the date and time on which the information ceases to be effective. Currency has an obvious relationship to timeliness. In this context, currency refers to the absolute vintage or age of the data asset, where as timeliness adds in the fitness for purpose aspect.

10.1.3. Completeness

In A3C, completeness of imagery and products relates to imagery and information with the required resolution, coverage and spatial richness. In aviation completeness may relate to the completeness of coverage of flight data, aeronautical facilities, infrastructure or airspace information. Deficiencies in completeness may happen due to a lack of spatial coverage, temporal coverage, granularity or type of data. The reasons for these deficiencies may be multifarious, such as lack of data collection, commercial license restrictions, storage or transmission limitations, and commercial or military sensitivity.

10.1.4. Consistency

In A3C, the Consistency metric describes the consistency of colors, relative accuracy over time and over different sensors, spectral and spatial error propagation from collection to production. All of these aspects are important in terms of consistency of data in the aviation domain as well as symbology, relative accuracy of data sources and error propagation. In addition, consistency when applied to the aviation domain closely aligns to the concept of Integrity. Integrity of aviation data is often of crucial importance, as data providers, managers and users alike need to have high confidence that the data they provide, manage or use has not been unduly or erroneously manipulated or corrupted. This manipulation may occur in a deliberate or accidental manner, and may be human or machine related. Thus, this aspect of data quality may need to be extended to fully encapsulate the importance of data integrity to aviation stakeholders. Note that integrity has connections with the security aspect of aviation services, however it is considered out of scope for the AQM and should be recorded in a service quality model.

10.1.5. A3C lessons learned

The A3C framework represents a useful starting point to develop an Aviation AQM. The overall organizing metaclasses are suitable for grouping quality metrics of aspects of data quality. However, A3C is missing the key concepts of Timeliness and Precision that are required for the Aviation Domain. A3C remains and influence for the AQM at a conceptual level, but requires granular elements to enable standards compliance and low level descriptive metrics for aviation data.

10.2. ISO Quality framework

The ISO standards coming from TC 211[9] form a collection of documents dealing with different aspects of Geographic Information and Geomatics. Specific ISO standards considered are:

  • ISO 19115-1-1 Metadata - This ISO standard forms the basis of ISO 19157 in terms of the data quality elements and is used as a fuller description of a data record.

  • ISO 19115-1-2 Metadata for imagery - This standard specifies some extensions to 19115. In particular introduces the idea of a coverage to specify pixel level quality. ISO 19115-1-2 also specifies how to encode quantitative/conformance measures in ISO metadata.

  • ISO 19115-1-3 XML schema implementation for Metadata - The schema specifies XML encoding rules for ISO 19xxx standards that partially replaces ISO 19139.

  • ISO 19138 Data quality measures - An outdated document that specifies a quantitative/conformance measures for data quality that has recently been superseded by ISO 19157.

  • ISO 19157 Data quality – This document establishes the principles (elements, measures and procedures for evaluation and reporting) for describing the quality of geographic data.

ISO 19115-1 defines general-purpose metadata for geographic information. The metadata model enables definition of domain-specific user extensions based on a common pattern to facilitate implementation of software and services using those extensions. The purpose of metadata is to describe resources, whether they are services or datasets. In the case of data, generally this description remains with the data and does not change. It can be used both to describe the data for interpretation and for discovery, the focus of this section is to discuss the ISO 19157 elements, as they are most relevant to the aviation domain.

10.2.1. ISO 19157 elements

Figure 1 shows the core elements of ISO 19157 Data Quality Model:

19157 elements
Figure 1. High level Data Quality elements of ISO19157

These are:

  • Completeness: presence and absence of features, their attributes and relationships;

  • Logical consistency: degree of adherence to logical rules of data structure, attribution and relationships (data structure can be conceptual, logical or physical);

  • Positional accuracy: accuracy of the position of features within a spatial reference system;

  • Thematic accuracy: accuracy of quantitative attributes and the correctness of non-quantitative attributes and of the classifications of features and their relationships;

  • Temporal quality: accuracy of the temporal attributes and temporal relationships of features;

  • Usability: Usability is based on user requirements. All quality elements may be used to evaluate usability. Figure 2 describes the construction of a metaquality element:

metaquality
Figure 2. Metaquality
  • Metaquality: a set of quantitative and qualitative statements about an evaluation and its result. The knowledge about the quality and the suitability of the evaluation method, the measure applied and the given result may be of the same importance as the result itself in some circumstances. These quality classes (or elements) can be elaborated in several quality indicators (or sub-elements). When describing the quality of geographic data, different quality elements and different subsets of the data may be considered. In the aviation domain there are mandated data quality measures that are imposed to produce a result. It is also possible that metadata from data production or distribution services contain metadata that describes the capabilities of a sensor, which then translate into the quality measures. In this case, the metaquality information would simply be a recognition that a quality metric is a sensor capability.

These together form the seven quality elements (or classes) describing certain aspects of the quality of a geographic dataset.

A data quality unit is the combination data quality elements and scope. The scope of the data quality unit(s) specifies the extent, spatial and/or temporal, and/or common characteristic(s) that identify the data on which data quality is to be evaluated. One data quality scope shall be specified for each data quality unit. Moreover, an evaluation of a data quality element is described by the following:

  • Measure – the type of evaluation

  • Evaluation method – the procedure used to evaluate the measure

  • Result – the output of the evaluation (includes value type, unit and date)

10.2.2. Discussion about ISO 19115-1 and 19157 elements with respect to the aviation domain

Accuracy

Accuracy in the aviation context is concerned with the recorded position of each feature compared to its actual position. Positions maybe recorded as absolute, relative, internal or external positioning depending on the requirement for each. This element has an obvious requirement for a base dataset considered as truth for any particular use case (also called the universe of discourse).

Positional accuracy

The Positional Accuracy class contains three main elements:

  1. Absolute External Positional Accuracy - closeness of reported coordinate values to values accepted as or being true;

  2. Relative Internal Positional Accuracy - closeness of the relative positions of features in a dataset to their respective relative positions accepted as or being true;

  3. Gridded Data Positional Accuracy - closeness of gridded data spatial position values to values accepted as or being true;

These are detailed in the following sections.

Absolute external positional accuracy

Absolute external positional accuracy records the position of objects in reference to an overarching coordinate system or datum (such as WGS84). If considering some common aviation services, examples of absolute external positional accuracy may include the positional accuracy of flight tracker information, the positional accuracy of aerodrome features or obstacle data. For instance, positional accuracy of a flight on a flight tracker service is likely to vary based on a number of aspects such as:

  • Whether the aircraft is using an ADS-B transponder and GPS to show its location or whether the position is calibrated on the ground and then current position is estimated using time, distance and speed.

  • If estimation from time, distance and speed is being used, how far the aircraft has traveled, and its trajectory. Estimations are likely to get worse over time. This emphasizes the importance of variability in absolute external positional accuracy from different datasets within a service, and over time.

Relative internal positional accuracy

There may be cases where aviation services consider relative internal position accuracy as a greater concern than absolute external positional accuracy. This tends to be encountered for Lateral Navigation (LNAV) and Vertical Navigation (VNAV) where aircraft position is measured relative to a desired track and deviation allowance. An example is a detailed survey of an aerodrome, that is not referenced to a global coordinate system, but is drawn relative to a local datum. Another example is the absolute position of an aircraft, the relative position of an aircraft is recorded in relation to other aircraft in the area, particularly for the vertical axis. Although not necessarily required from a services perspective, relative position may be given to the pilot of an aircraft to describe the position of other aircraft in the vicinity on approach or take off.

Gridded data positional accuracy

Gridded data positional accuracy is likely encountered in terrain and meteorological datasets for aviation as well as the usual imagery datasets. The measurements for gridded data are similar to other types of data with a similar requirement for the universe of discourse.

Extending positional accuracy for the aviation domain

It may be the case that the exact positional accuracy is not known as there is no ground truth to compare against. However, what may be useful in service comparison is to know immediately whether particular datasets within a service have come from sensor positions or from estimations. For this reason we recommend two additions to the standard:

  • Add an unknown value to the positional accuracy fields

  • To add a Positional Precision element with coded values such as ADS-B, ADS-BLostCoverageEstimation or TimeSpeedDistanceEstimation, to provide user’s information regarding the relative accuracy of each. With ADS-B errors may be in the meters, with estimation due to loss of coverage or time and distance estimation for older aircraft they may be 100km or more in error.

Thematic accuracy

Thematic accuracy is defined as the accuracy of quantitative attributes, the correctness of non-quantitative attributes, and of the classifications of features and their relationships. It consists of three data quality elements:

  • classification correctness; comparison of the classes assigned to features or their attributes to the universe of discourse;

  • non-quantitative attribute correctness; measure of whether a non-quantitative attribute is correct or incorrect;

  • quantitative attribute accuracy; closeness of the value of a quantitative attribute to a value accepted as or known to be true.

Classification correctness

Classification correctness is measured as the number of incorrectly classified features, and the misclassification rate. In terms of aviation services, terrain and meteorological datasets are more likely to have classified content than other aviation services such as flight information. This might include classes of height information on a terrain base map behind a moving map, or classes of rainfall intensity on a radar image. Ground truth may be altitude data from aircraft, or weather station data. The quality of thematic classification may therefore have relevance within an aviation service registry. A question to clarify is whether classes of airspace (A, B, C etc.), and Unit of Measurement (UoM) codelists (e.g. NM, KM, M etc.) fall under classification correctness, or whether they fall under non-quantitative attribute correctness.

Non-quantitative attribute correctness

Non quantitative attribute correctness within aviation services may refer to attribute information attached to a range of aviation services. Flight data is likely to contain non-quantitative attributes related to the aircraft category (e.g. landplane, seaplane, amphibious, helicopter), engine base (turboprop, jet, piston), airline, capabilities, as well as en-route information such as airspace classification. Aerodrome information may have many attributes related to the services that the aerodrome provides, contact details, components of the aerodrome.

Quantitative accuracy

Quantitative accuracy is crucial in many areas of aviation data and refers to flight information such as speed, distance, pressure and time. It may also encompass the quantitative attribute information attached to meteorological reports such as rainfall and pressure.

Within The European Commission Rule 73/100 (commonly known as ADQ IR), accuracy is defined as the degree of conformance between the estimated or measured value and the true value. For many services, the ground truth of speed or distance may be available through aircraft sensors, ground or space based sensors. For some services, no ground truth will be available. In which case, a true accuracy measurement cannot be calculated, only precision, and the authority of the source. As with other elements, an unknown option may be useful for this element.

The temporal aspect of accuracy is also important, a service is measured at a certain accuracy over a certain time, however this may be a one off measure for a temporally changing accuracy.

Extending thematic accuracy for aviation services
  1. Add unknown value to accuracy elements.

  2. Add unclassified rate as different to misclassification rate.

Temporal accuracy

ISO 19157 contains an element for Temporal Accuracy. The temporal accuracy of a dataset is divided into three elements:

  • Accuracy of a time measurement: considers the correctness of the temporal references of an item.

  • Temporal Consistency: considers whether the order of events or sequences is correct if reported.

  • Temporal Validity: validity of date specified by the scope with respect to time.

Applying the Temporal Accuracy element to aviation data is a rigorous test case, as there are a number of aspects of aviation data that deal specifically with temporal accuracy and validity. The engineering report will consider the appropriateness of these core elements for aviation services, and then we will consider other aspects of time measurement that are applicable.

Accuracy of a time measurement

Accuracy of a time measurement considers the correctness of the temporal references of an item. Time accuracy is measured at a particular significance level, which dictates the range to which a time measurement can deviate from its true value and still be valid. This aligns with the Temporal Uncertainty measure described within the AIXM data, and so is a useful measure for aviation data.

What is not specifically addressed in this measure is the resolution or precision of the time measurement. In AIXM, the TimeBaseType class has resolution of 1 minute. If time is truncated or artificially precise, then it is not conformant with the AIXM class, and this should be described in a record on precision.

Temporal consistency

Temporal consistency is known to be an important aspect of codifying aviation data. Many aviation components have start and end dates (e.g. AIXM, NOTAMS, AIPs) under which they are valid. Making sure that the end date is after the start date should be a business rule check for logical consistency. - Dictated by Business rules (end date must start after start date). - Useful for data such as the AIXM Timesheet class (startDate and endDate, startTime and endTime).

Temporal validity

Again, temporal validity is known to be an important aspect of codifying aviation data. As stated many aviation features and messages have periods under which they are valid, and a key business rule should be to check for temporal validity.

  • Dictated by Business rules (Is feature within valid time period or not).

  • Useful for data such as that controlled by Aeronautical Information Regulation and Control (AIRAC) cycles.

Extending temporal accuracy for the aviation domain.

Related concepts of temporality, currency and timeliness should be incorporated into the AQM, this can be done with the following:

  • Coded Values Relating To Time: Aviation data contains a number of coded values relating to time (for instance CodeTimeEvents such as Sunrise, Sunset, Earliest, Latest, CodeTimeReference relating to a timezone, such as UTC+1, UTC+2 and UoM such as HR, MIN, SEC). This could be considered a case that can be covered under non-quantitative attribute correctness, or possibly classification accuracy. This can be managed through a recognized code list to manage the possible entries for this field.

  • Temporal Precision: It is necessary to add a further element to specifically deal with temporal resolution or precision separately from accuracy, the two are very different concepts.

  • Time format and Calendar Appropriateness: The appropriateness of the time format and calendar used for measuring time is important (as dictated by EUROCONTROL regulations for instance). If a non-standard time format or calendar is used this should be recorded and users notified. This does not affect the structure of the abstract model, but needs to be applied at a business rule level as implementation guidance, checking against the format and calendar fields of the data. If however, the calendar used or time format is not stated at all, this leads to uncertainty about the quality of any time value within the dataset.

  • Data currency: The currency of data is somewhat related to validity, however it focuses more on whether the data is sufficiently up to date. Data could still be valid even if it is old, and may not be valid even if it is up to date. Currency is also somewhat related to the purpose that it is used for. A service might be deemed to have sufficient currency for some tasks and not others.

  • The use of Time Slices: Much like AIXM, Each UML class within ISO 19157 is required to be understood as describing a real-world entity whose properties may vary over time, and must therefore give rise to representations which permit the expression of such variation. This is achieved by equipping each UML class in question with a collection of time slices, which indicate the temporal scope of the collection of properties.

Resolution
Spatial resolution

ISO 19115-1 contains an overarching Spatial Resolution element factor, which provides a general understanding of the density of spatial data in the resource or describes the range of resolutions in which a digital resource may be used. NOTE This element should be repeated when describing upper and lower range.

These can be defined under MD_Metadata.identificationInfo > MD_Identification.spatialResolution > MD_Resolution. This then contains specific sub-elements (equivalentScale, distance, vertical, angularDistance or levelOfDetail) that will be described below.

Describing measurements of spatial resolution

Resolution is described in ISO 19115-1 via the MD_Resolution class. Resolution is a requirement for the aviation domain and the class is a suitable representation of imagery resolution. Therefore, it is suggested that this class be used to describe the resolution for aviation data via an organizing metaclass.

For reference, MD_Resolution contains:

  1. equivalentScale: MD_RepresentativeFraction

  2. distance: Distance

  3. vertical: Distance - This element was added to allow specification of vertical resolution.

  4. angularDistance: Angle - This element was added to allow for specification of angular sampling distance.

  5. levelofDetail: - This element was added to allow for the textual description of the spatial resolution of the resource.

Describing measurement of temporal resolution
TM_Duration

ISO 19115-1 contains a temporal resolution element under MD_Identification. TemporalResolution is the TM_Duration domain which defines the smallest resolvable temporal period in a resource. In addition to spatial resolution, the TM_Duration class is suitable for recording temporal metadata and, like MD_Resolution, can be used in an overarching organizing class to describe the two aspects of resolution required by the aviation domain.

Traceability

The traceability of aviation data within a service describes the ability to trace the history, application or location of that which is under consideration (ISO 9000). The closest match for this element within the considered ISO standards is the LI_Lineage element.

LI_Lineage

The ISO Lineage element exists to describe the history of a dataset and, in as much as is known, recount the life cycle of a dataset from collection and acquisition through compilation and derivation to its current form. This lineage element can be used to describe the traceability of data, giving the user assurance of its authoritativeness.

Integrity

In the AQM, integrity refers to the consistency of the contained data, not the security aspects of the service offering the data, which is recorded at the service level as part of service metrics. Additionally, there is likely to be regulatory restrictions on data security, especially if the data are used in operational decision-making.

Logical consistency

Logical consistency is defined as the degree of adherence to logical rules of data structure, attribution and relationships (data structure can be conceptual, logical or physical). If these logical rules are documented elsewhere (for example in a data product specification) then the source should be referenced (for example in the data quality evaluation). It consists of four data quality elements:

  1. Conceptual consistency – adherence to rules of the conceptual schema;

  2. Domain consistency – adherence of values to the value domains;

  3. Format consistency – degree to which data is stored in accordance with the physical structure of the dataset;

  4. Topological consistency – correctness of the explicitly encoded topological characteristics of a dataset.

Applying this to aviation services, this type of data quality measure might express whether the dataset is conformant to the structure of the AIXM standard, for instance, or whether the codelists within AIXM are being adhered to. Other structured datasets with specifications include FIXM and WXXM, NOTAMS and e-NOTAMS, and general aeronautical messaging formats.

Completeness

This element refers to the presence or absence of features, their attributes and relationships. Within an aviation context, this is very pertinent to data quality within a service. In terms of ISO 19157, completeness is defined by two elements: Errors of Commission, and errors of Omission. The elements covered under DQ_Completeness in ISO 19157 do not go as far as to make reference to the completeness of information surrounding the data, for instance whether Units of Measurement are populated, or whether metadata is complete. This may instead be covered under the metaquality element. Completeness of data could have a geographic component, a temporal component, or both. Of peripheral interest to understanding metadata quality is the GeoViQua FP7 funded EU project that sought to produce a dashboard like interface to describe metadata population and thus indicative quality. This section is concerned with completeness in terms of a dataset’s omission or commission, rather than the completeness of the metadata.

In aviation terms, the circumstances surrounding incomplete / excess data are multifarious and could relate to:

  • Partial data due to incomplete coverage of the sensor or flight database.

  • Partial data due to license constraints of the service.

  • Data integrity or corruption issues at any stage of the workflow.

  • Deliberate blocking by providers.

  • Filtering by users.

Within a service registry users may need to be able to compare the completeness of the data within two or more similar services. As completeness is variable depending on the type of feature, geographic location or temporal coverage, they may be interested in data completeness at the overall service level, or specific data coverage for a particular scale or level of detail. To facilitate this, different completeness criteria could be populated under different scopes within the service metadata.

Completeness commission

Overall, commission is a measurement of data completeness and is concerned with over representation of data compared to the universe of discourse. This can be measured via counting the number of items within a feature dataset or has been interpreted to look at aspects such as resolution for imagery (i.e. a resolution higher than the universe of discourse would in essence constitute too much data and therefore data commission). Duplication is potentially the measurement most suited to the aeronautical domain. Errors of commission are described in a number of ways in ISO 19157:

  • Excess Item: Boolean True or False. More items than the universe of discourse.

  • Amount of Items (e.g. 2).

  • Rate of items (e.g. 10%).

  • Number of duplicates (exact geographical coordinates and attributes).

Completeness omission

The measurement of omission is in contrast to the commission measurement as it records missing data in datasets when compared to the universe of discourse.

Errors of omission are described in a number of ways in ISO 19157:

  • Missing item: Boolean True or False. Fewer items than in universe of discourse.

  • Number of missing items.

  • Rate of missing items.

Data omission may commonly occur in aviation services such as flight tracking services due to patchy sensor coverage, or flight blocking if flights are related to military or national security.

Although these provide a useful start to describe and compare the completeness of data within a service, there are a number of aspects that are not discussed which might be common occurrences with aviation data services. These are presented as recommendations to extend the standard below.

Extending Completeness for aviation services.

There are no extensions to completeness required for aviation services.

Temporality aspects of completeness

Aviation datasets operate across a range of temporal bounds, from near real time, to static. Datasets and resultant services may change their completeness over time as conditions change. Temporality or timeliness maybe extended in its own right, but also has an impact on the completeness of a dataset. For example if a dataset is created from a set of images, potentially from different platforms and different temporal resolutions, then a temporally constrained measurement of commission will fluctuate as updates to underlying datasets are provided. A measurement of completeness is updated according to the status of the dataset updates at any given time. This obviously creates a challenge for maintaining lineage as the product is essentially live and changing at all times.

When the completeness of the dataset is unknown

As well as known errors of commission or omission, which might be identified through manual or automatic quality control, there are likely to be some services for which the status of data completeness is partially or fully unknown. This could be dealt with in a number of ways:

  1. An estimate of commission and omission is populated, and a metaquality element is filled out to specify the uncertainty / confidence attached to these estimates.

  2. An extra element within the completeness elements is added, to specify whether the completeness of the service is known or unknown.

When there is incomplete metadata

A second case that is not covered within these elements is the completeness of the metadata of the resource. This could be:

  1. That certain mandated fields are missing, which should be picked up by a validator

  2. That certain mandated fields are empty, which again should be picked up by a validator

  3. That the quality of the metadata is not enough to be able to provide useful information for decision-making. In this instance a subjective judgment needs to be made around the quality of the entry of the metadata. This may be a candidate for the 19157 'usability' measure.

As well as quantitative elements, there are several non-quantitative elements on the metadata that are related to data quality:

  1. Purpose: describes the rationale for creating a dataset and contain information about its intended use.

  2. Usage: describes the application(s) for which a dataset has been used or uses of the dataset by the data producer or by other, distinct, data users.

ISO 19165 Fixity

Although not explicitly reviewed in this document, ISO 19165[11] is a Draft International Standard for preservation of data and metadata. The fixity element refers to "An element’s state of being unchanged or permanent". Many data repositories will have their own systems for verification and signing of data to prove authenticity of a dataset or element, however this data needs to be recorded as part of integrity to indicate the seal of approval in the metadata.

Utilizing fixity in the AQM

Fixity should be used in the AQM as a method of recording data authenticity and signatures through the GP_Fixity class. Throughout the ISO documentation, there is no other suitable class for recording this information.

10.2.4. Extension to the ISO models

Precision
Spatial precision

Neither ISO 19115-1-1, 19115-2 nor 19157 contain an element for precision, defined as; The smallest difference that can be reliably distinguished by a measurement process. It defines the spread, or the spatial error around a series of spatial measurement. The average of the measurements and the difference between it and the true value is the accuracy.

It is recommended that an element be created to describe spatial precision:

  1. level of Detail - This element could be added to allow for the textual description of the spatial precision of the resource. For instance the number of measurements the range was calculated over

  2. horizontal: Precision - This element could be added to allow specification of horizontal precision. This could be measured using a mean +/- range, a mean +/- absolute deviation or a mean +/- standard deviation for instance

  3. vertical: Precision - This element could be added to allow specification of vertical precision. This could be measured using a mean +/- range, a mean +/- absolute deviation or a mean +/- standard deviation for instance

Temporal precision

Neither ISO 19115-1-1, 19115-2 nor 19157 contain a separate element for temporal precision, defined as The smallest difference that can be reliably distinguished by a measurement process. It defines the spread, or the temporal error around a series of time measurements. The average of the measurements and the difference between it and the true value is the accuracy.

It is recommended that an element be created to describe temporal precision that includes:

  1. level of detail - A textual description of the temporal precision of the resource

  2. precision of a time measurement - An element to describe the precision of a time measurement for example hours, minutes, seconds, milliseconds

Quantitative measurement precision

Quantitative measurement of precision may refer to an attribute or the position of features within a dataset. Classes with a quantitative measurement or element should contain a separate measurement of precision to record the number of decimal places that can be considered valid on a quantitative measurement.

Recommendation to extend MD_Identification with an element to describe quantitative measure precision. This acts as an element that can be invoked for quantitative data that is not temporal or spatial and is therefore expected to be reused liberally.

  1. level of detail - A textual description of the measurement of precision. Population of this element will likely be mandatory as tests for quantitative will vary depending on the element/dataset

  2. quantitative precision - A measurement of a quantitative measurement precision.

Usage

In FP7 GeoViQua (http://www.geoviqua.org) the usage concept was extended to geospatial user feedback (GUF). GUF is a form of metadata derived from the experience that users gain by using the data. The GUF Standards Working Group (SWG) developed a standard in two separate documents:

  • OGC 15-097r1, Geospatial User Feedback Standard. Conceptual model

  • OGC 15-098r1, Geospatial User Feedback Standard. XML Encoding extension Recently, the OGC TC approved these two documents as international standards.

Enhancing other aspects of data quality for aviation services
  • In terms of lineage, it is suggested that the element should have the capacity to specify the data originator, an official term stated in ADQ 73/100, defined as an entity responsible for data origination.

  • Metadata Quality - The completeness of metadata is not specifically addressed in ISO 19157, or in ISO 19115-1. However, some organizations have built on this basis to provide tools to improve the quality and completeness of metadata held for data and services. One example is the National Oceanographic Atmospheric Association (NOAA) Completeness Rubric[10].

Usability

Usability is used to record information about fitness for purpose of a particular dataset to a defined use case. ISO 19157 offers the example of a data producer describing suitable data usage with bounding terms, it can also act as a catch all for data quality elements that the other elements do not cover.

Usability for aviation services

From an aviation perspective, the usability could be used to describe the suitability of the data within the service to a particular use case, for example, navigation, monitoring of air traffic, route planning due to weather plus others. From an implementation perspective, the usability element may have to be updated regularly to take into account the timeliness element described as extensions to the ISO standard as a dataset or service’s appropriate use has a temporal element that goes beyond those described in ISO 19157.

Metaquality

Metaquality elements are a set of quantitative and qualitative statements about a quality evaluation and its result. The knowledge about the quality and the suitability of the evaluation method, the measure applied and the given result may be of the same importance as the result itself. Quality measures as described by metadata quality information also refer back to the appropriate use of the data (described by usability) as the appropriate use will change as the data ages.

Confidence

Concepts of metadata quality in the aviation domain are likely to determine or at least influence the appropriate use of the data. As mentioned in the introduction, appropriate use of the data is at least partially determined by the quality, the confidence measure enables users to have a threshold associated with the quality measures. Practically, this is likely to be stringent as data considered usable in the aviation domain will likely have a high level of confidence associated with its collection or capture.

Representativity

Datasets are often sampled to produce data quality metrics, this metaquality metric attempts to identify how well the sample selected from the dataset represents the whole dataset (within the scope). The aviation domain maybe more stringent and require that every feature is included in this metaquality calculation.

Homogeneity

This measure records how the dataset varies over geography and time. For example, it is common for imagery mosaics to be compiled from different sources/platforms capturing imagery at different times. Software processes are then used to stitch together the constituent pieces into a single view. Data quality measures do not take this into account directly, therefore this is recorded as part of the metaquality element.

Metaquality for aviation services

Aviation requires metaquality measures to understand the metrics detailed in the previous section. All of the metrics are important, however understanding data homogeneity is likely to be most relevant to the user as this operation is often completed to produce a dataset, but the implications are not considered. It also feeds into the notion of timeliness or differences in data vintage across any given dataset. Timeliness in an aviation perspective will affect the overall given quality of any dataset as it brings together concepts of currency and fitness for purpose.

11. Levels of detail

Levels of detail in ISO standards are governed by the Scope code, as it defines the scope of the data quality metrics. As with many domains, the scope for data quality in aviation is at the dataset level, with aggregation possible at the service level. In addition to feature and dataset, the ISO standards also support attribute level quality metrics, although the advantages of this level of detail remain to be seen.

11.1. Service level quality

The quality of a service can be determined by metrics in the SDCM or WSQM QoS metrics, however in this ER, service level quality refers to the contained data rather than the quality of the service provide by the endpoint. Measuring data at a service level is concerned with describing a group of datasets provided by a service. This differs from aggregation of datasets, which is where features from datasets are combined to produce a single, new dataset with quality that varies across a single dataset. Service level quality in this instance seeks to describe how the quality of the datasets offered by the service vary and may include an aggregation of the quality metrics from the datasets.

11.2. Dataset level quality

Dataset quality is the main level of quality used in the aviation domain, however there is often a debate as to what constitutes a dataset. Taking into account the concepts described in the Metaquality section, the homogeneity of a dataset has an effect on the dataset scope or grouping of features. If a dataset is homogeneous, i.e. contains features or tiles of identical origin and vintage, then the concept of a dataset is quite simple as it is a group of common features. However, if the dataset is a mosaic of features that have different origins, then quality metrics are likely to be affected. Additionally in modern systems, a single feature held in a database may belong to many different datasets, for example, a single flight over the UK will belong to a dataset of all flights and a dataset of flight across the UK. A metadata record (along with data quality metrics) should exist for each of these datasets for discoverability and qualification purposes.

11.3. Feature level quality

Measuring the quality of individual features or elements is useful in a variety of use cases, notwithstanding the complications of grouping heterogeneous features or tiles, feature level quality is useful in processes that involve conflation of datasets. This is largely done to visualize differences in source and quality across a conflated dataset, additionally it provides a metric of usability at the feature level, which defines how a conflated dataset maybe used operationally. In the aviation domain, it is likely that datasets are more homogeneous in terms of their timeliness and source due to the constraints regarding air safety. The introduced concept of precision has obvious descriptive power to features or even attribute level metadata.

11.4. Aggregated data quality measures

According to ISO 19157 The quality of a dataset may be represented by one or more aggregated data quality results (ADQR). The ADQR combines quality results from data quality evaluations based on different data quality elements or different data quality scopes. Interestingly, a dataset may be deemed to be of an acceptable aggregate quality even though one or more individual data quality results fails acceptance (i.e. a weighted sum over a particular threshold). Whether this is acceptable in an aviation dataset is a key question. There are a number of considerations for creating implementation rules in the aviation domain:

  1. Whether there are already recognized measures of aggregation within the aviation domain.

  2. Whether a certain amount of poor quality data is acceptable within an aggregated service and if so, what the limits are.

  3. If poor quality data is not acceptable in the domain or for certain use cases, does the entire dataset fail or are parts of the dataset deemed trustworthy and fit for purpose?

  4. How is the user notified?

Aggregation should only be used when compelling reasons exist. The meaning of the aggregate data quality result should always be made clear.

11.4.1. Aviation data aggregation use case

Data aggregation can prove problematic for calculating and disseminating data quality metrics, especially when metadata assets are restricted to describing a dataset level resource. In cases of an imagery mosaic, it implies that parts of the mosaic are suitable for a particular use case and some parts are not. In ISO standards, the Lineage element is implemented in both the context of the resource and in the context of the describing metadata. A potential solution to aggregation is to treat the tiles as separate pieces of data with their own metadata and then to have a separate metadata document for the mosaic with the metadata lineage element describing the constituent metadata documents that describe the resources that make up the mosaic. By using the metadata lineage element, this allows for mosaics of mosaics whilst reserving the resource lineage field to describe changes to the individual imagery tiles.

Vector data has many of the same issues with aggregation that imagery does, however the problem can be exacerbated due to the number of features that can be conflated in any given process. In this instance, the resource identifier is used to maintain the lineage of each feature which provides a constant record of where features have come from.

A more complicated aggregation use case in the conflation/aggregation space is when new features are generated or the geometry of old features are altered as part of a data fusion process. Lineage is particularly important in these cases as it maybe the only method of tracking individual features back to their original positions before conflation takes place. It is appreciated that this is currently a niche possibility as conflated data used in decision-making may prove regulatory prohibitive.

11.5. QualityML

QualityML (Quality Indicators Dictionary and Markup Language) is a hierarchical dictionary of data quality terms that operate across the board of standards. Essentially, QualityML offers the user the ability to maintain the semantic descriptions of quality (and meta-quality) measures no-matter what profile of the standards they are encoded in. QualityML is important to consider in the abstract data quality model as it is likely how the eventual AQM will be expressed in an implementation. However, the implications of implementing the AQM in QualityML is discussed other ERs in the Testbed 13 aviation thread.

12. Abstract Quality Model

Based on the above models and discussions, the high level architecture of the AQM aligns to a great extent with the elements of ISO 19157. Figure 3 provides a diagram with proposed elements, with adaptations for Aviation Data Quality in yellow. This model forms the basis of the full aviation abstract quality model described in Figure 4.

AviationDataQualityTrunc
Figure 3. Proposed elements to extend ISO 19157 for aviation data

The AQM data quality elements are grouped according to metadata classes, or super classes of different measures of quality. Overall, it contains all of the data quality fields required by ISO 19157 and is therefore compliant, however it also contains a Timeliness metaclass that utilizes the metadata maintenance information found in the ISO standard. Overall, the model contains the following themes, expressed as metaclasses:

  • Accuracy - A measurement of the positional quality of the dataset, note that this also includes a measurement of precision, a separate measure for the aviation domain.

  • Traceability - A measurement including the dataset lineage.

  • Resolution - Contains a measure of spatial resolution, expressed using the MD_Resolution class. It also contains a measurement of temporal resolution.

  • Completeness - Describes the amount of data in a dataset according to a comparison with the expected data, it described using a measure of commission and omission.

  • Timeliness - A measurement of currency or vintage of the data, this measure can be used to describe the variation in vintage over a conflated or dataset mosaic.

  • Integrity - Describes the degree of assurance that can be given that the dataset has not been altered or lost since creation or change from the required body.

These six overarching themes provide a comprehensive, standards compliant model for implementation with the added concepts of Timeliness and Precision included to ensure relevance to the aviation domain. These six metaclasses comprehensively cover the requirements outlined in the Aviation quality requirements section. The detailed AQM is in Figure 4, explanations of the metaclasses are documented below.

AviationDataQuality
Figure 4. The Aviation Abstract Quality Model

12.1. Accuracy

The Accuracy metaclass has the ability to describe the quality of positional accuracy, thematic accuracy, quantitative accuracy and accuracy of a time measurement. This class is not simply concerned with positional accuracy as might be expected. The ISO 19157 elements included are:

  • DQ_PositionalAccuracy

  • DQ_ThematicAccuracy

  • DQ_QuantitativeAttributeAccuracy

  • DQ_NonQuantiativeAttributeAccuracy

  • DQ_ThematicClassClassificationCorrectness

Each of these elements describe the closeness of each of the aspects of the data to the universe of discourse. There are extensions proposed to the model that are concerned with describing precision of different aspects of the dataset.

  • DQEx_PositionalPrecision

  • DQEx_QuantitativeAttributePrecision

  • DQEx_PrecisionOfATimeMeasurement

These added elements address the requirements of the aviation domain to understand the precision of different aspects of the dataset.

12.2. Traceability

The Traceability of a dataset is used to understand the lineage of the resource including the originator/person responsible and the processing steps taken on the dataset to produce its current form. Note that an EX_Extent class is included in the responsible party information to account for heterogeneity within the dataset, that is, processing may have been completed on part of the dataset, for example and update and other parts of the dataset may not have been updated. Traceability includes:

  • Citation and responsible party information

  • Lineage information

    • Process step

    • Source

12.3. Resolution

The Resolution class contains information about the resolution of a dataset including its spatial and temporal resolution. These aspects are recorded using standard ISO elements:

  • Spatial resolution through the MD_Resolution class

  • Temporal resolution through the TM_Duration class

12.4. Completeness

The completeness of an aviation dataset aligns with the definition outlined in the ISO standards. The Completeness class includes the following:

  • DQ_CompletenessCommission

  • DQ_CompletenessOmission

12.5. Timeliness

As mentioned previously, timeliness is a representation of the concepts of currency and fitness for purpose and is described as part of the metaclass Timeliness. The Timeliness element uses the MDMaintenanceInformation class to describe the update frequency, date of last update and other information about the update. The class also includes a TemporalExtent to describe the time period. The metadata update frequency value is restricted by the MD_MaintenanceFrequecyCode. The class also includes an extent element to restrict the timeliness measure by a geographic region, this provides the AQM with the ability to record timeliness metadata in mosiaced datasets. Timeliness is described using the following ISO compliant classes:

  • DQ_TemporalValidity

  • Temporal extent using a time primitive

  • Metadata information including

    • The metadata update frequency governed by a codelist

    • A user defined maintenance frequency for ad-hoc updates and granular recording.

12.6. Integrity

Integrity is a metaclass that is concerned with the internal consistency of data according to its defined concept, schema, format and topology. The ISO standards contain elements and implementation guidance that is suited to recording this metadata, the classes included are:

  • DQ_LogicalConsistency

  • DQ_ConceptualConsistency

  • DQ_DomainConsistency

  • DQ_FormatConsistency

  • DQ_TopologicalConsistency

The packaging information for the data is recorded in the GP_Fixity class. As mentioned in ISO standards review, GP_Fixity is a class to record information about a dataset’s authenticity through signing. The class simply records the information, it does not provide a methodology for implementing authenticity checks as domains are likely to have their own restrictions and implementing systems.

12.7. AQM reference to the SDCM

A founding motivation for undertaking this work is the SDCM does not contain a reference to data quality, only to service quality. Therefore, the AQM is related back to the SDCM according to the Data Entity class with the data quality of an asset offered by a service described by the Quality of data class. Figure 5 outlines the relationship between the AQM and the SDCM. A full explanation of the relationship between the AQM and the SDCM is explained in Testbed 13 FA002 - Data Quality ER.

SDCMv2DataQualityModelDiagram
Figure 5. The Aviation Abstract Quality Model related to the SCDM

13. Applicability of the AQM to aviation services

This section provides an overview of how the AQM can be applied to the three different aviation services, AIXM, FIXM and WXXM. This section is designed to give overall guidance for considerations or use of the AQM, the formal use cases can be found in FA002 and FA003.

Aerodrome structural information, flight information and weather information can be considered in terms of their propensity to change over time. Aerodrome information is generally static with any changes to physical infrastructure planned with a long lead time. Therefore, data quality for these structures is likely to be static and simple to verify as there is a lot of corroborating information (referred to as the universe of discourse) such as imagery. Flight information is changeable over relatively short periods of time, but has the advantage, in data quality terms, of being planned. Therefore, data quality can be measured using the flight plan as one universe of discourse, although flight information changes over time, the changes are predictive but arguably predictable. Weather information is very changeable and unlike aerodrome information, it is based on predictive modeling that becomes closer to the truth, the nearer the modeling gets to present time.

The three different services are utilized at different times and by different stakeholders during a flight. Therefore, if the data quality elements are based on fitness for purpose then they will have to be interpreted by the different stakeholders to understand suitability.

13.1. Applications in aerodrome structure information

AIXM is concerned with the information exchange of physical infrastructure in the aeronautical domain and as mentioned previously, inherits some of ISO19157 fields. The AQM is easily implemented within AIXM due to the interoperability with ISO, however the extensions to the ISO model also have to be considered. Additionally, of interest to AIXM users is the scale that the data are captured at, as this will have an impact on many other aspects of data quality.

Aerodrome information is largely static with only aspects of the features likely to be of interest in many use cases. Requirements for airport information for flight planning and monitoring maybe restricted to aspects such as runway length, capacity, flying height and local flight restrictions rather than the precise geometry measurements for the terminal buildings or the location of amenities. Considering these requirements, the abstract quality model is less concerned with timeliness in this use case as the data provided is static, it is concerned with precision as the geometry of obstructions is paramount to safety. The crucial aspect of data quality in this use case may be the authority who is serving or publishing the data, as there is likely to be authoritative sources who have the mandate to produce this data for use in a safety critical context.

13.2. Applications in flight information

Flight information has aspects in common with the air information use case and also with the weather use case. Unlike weather, flights are planned with a specific route, therefore, planned flight routes will generally reflect the routes actually taken by the aircraft. However, flights are not static and aircraft position changes over time, therefore there is a predictive element to understanding the data quality of flight information. A major consideration for data quality in flight information is the sensor that is used to monitor the flights in transit. There are several methods for monitoring flight information and include humans spotting planes and report their location to technological means of monitoring aircraft such as RADAR and imagery. In these cases, the AQM could be used to record the properties of the sensor that made the observation. These parameters could be used to discern positional accuracy and precision as well as the all important timeliness parameter.

13.3. Applications in weather information

Weather information is changeable and based upon predictive modeling. Understanding the quality of weather information has a fitness-for-purpose element to it, as it is incorporated into flight planning at different stages. Weather information is utilized at all stages of flight planning, an example usage is as follows:

  • A week before the flight; long range forecasting used to understand whether there is a threat or possible barrier to flight. The forecast is likely to cover a wide area and does not afford decision-making.

  • A day before the flight; the forecast is checked for any likely events to prevent or severely disrupt flight. The geographic region is also considerably more localized. Mitigation planning measures may be taken, but more likely planned for at this stage.

  • The day of the flight; further planning and mitigation may be put into place should weather forecasting suggest disruption likely. There is also low level planning required for non-disruptive events such as rain or snow.

  • An hour before the flight; take-off plan and order of aircraft leaving now understood (TAF).

  • Upon take-off; very localized weather information is required for low level take-off plans, airspace exit planned (METAR).

  • During the flight; localized planning of route that takes into account existing weather system, however prediction is still required for possible weather system movement during flight.

These different stages have different data quality requirements, the abstract data quality model is suited to providing information at each of these times, however the requirements for some of the quality measures are likely to differ. For example, long range flight planning elements are by their nature, not reliable, however they may still be timely as the user can be working with the most up-to-date predictions. Additionally, long range weather predictions are unlikely to be localized down to a single aerodrome operating space, as long range forecasts tend to be regional. Highlighting these aspects in the abstract data quality model provides users with information they require to make a decision on the best data depending on what stage of planning they are at.

14. Conclusion

This document describes an abstract data quality model for the aviation domain. It was found that the majority of the data quality requirements can be fulfilled by organizing the ISO elements, influenced by the A3C framework to produce an interoperable AQM suited to the aviation domain. However, it has been acknowledged that there are requirements in the aviation domain that are not fulfilled by the existing ISO standards, these include;

  • Timeliness - how quality may change according to the age of the data. This is tied to the use case of the data quality metrics and the type of data being qualified and is expressed through the Timeliness metaclass in the AQM.

  • Precision - how precise a quantitative measurement is. There are three main areas that require a measurement of precision, these are quantitative attribute measurements, accuracy of a time measurement and positional accuracy.

The extra requirements in aviation are a recognition of the safety critical aspects of flight planning, but also the sliding scale of data changeability. Generally, aerodrome information does not change, and any changes made are planned in advance, documented and surveyed to an accurate and precise measurement. Flight information is changeable and reported in separate NOTAMS, but routes are planned and flight location reported using a variety of methods.

15. Emerging requirements

An emerging requirement mentioned throughout this ER is data quality metrics in heterogeneous data created through either informal or formal conflation processes. ISO 19157 has the capability to record metrics at the feature level through the scope code. This ability to go beyond dataset level metrics extends beyond feature level to attribute level if required. Generation of metrics at the feature level can be accomplished through implementation guidance provided in the ISO documents. Challenges with feature level metadata are not necessarily the generation of the metrics, but the discoverability of the features and the usability of the metrics. The accepted method of storing and presenting metadata is through a catalogue service where records of dataset are stored. It is difficult to see how users will discover and utilize metadata records about individual features and aggregation will be required. Also, storage of feature level metadata cannot be done in the same way as at the dataset level because the number of records will quickly become unmanageable, therefore the metadata would potentially be tightly coupled to the features that would in turn increase the volume of overall data.

Feature-level metadata could be utilized without exposing it directly to the user. For example, an aviation service could be offered that generates datasets on-the-fly according to a set of data quality rules. A dataset metadata record would also have to be generated on-the-fly to describe the bespoke dataset. Implementation of feature level metadata requires that the operation procedures of the aviation domain allow for feature level metadata to be considered but this has yet to come to fruition.

Appendix A: UML model

SDCM Quality of Data Diagram
Figure 6. The Aviation Abstract Quality Model related to the SCDM
AviationDataQuality
Figure 7. The Aviation Abstract Quality Model

Appendix B: Revision History

Table 2. Revision History
Date Release Editor Primary clauses modified Descriptions

September 4, 2017

S. Meek

.1

all

Initial version

Appendix C: Bibliography

[0] OGC: OGC Testbed 13 Annex B: Architecture. (2017).
[1] ICAO: Annex 15. http://cockpitdata.com/Software/ICAO%20Annex%2015 (2014).
[2] EUROCONTROL: NM B2B Web Services Fact Sheet. https://www.eurocontrol.int/sites/default/files/publication/files/nm-b2b-factsheet-2016.pdf (2017).
[3] ICAO: Chicago Convention Document 7300. https://www.icao.int/Pages/default.aspx (2017).
[4] ICAO: Annex 3. https://www.icao.int/APAC/Meetings/2016%20MET_SG20/IP05_AI.4%20ICAO%20-%20Review%20AM%2077%20-%20Annex%203.pdf (2017).
[5] European Commission: Regulation 73/2010. http://www.eurocontrol.int/articles/aeronautical-data-and-information-quality-adq-mandate (2017).
[6] EUROCONTROL: Specifications for data quality. http://www.eurocontrol.int/sites/default/files/publication/files/20160225-dqr-spec-v1.2.pdf (2016).
[7] DigitalGlobe: The A3C quality framework. http://blog.digitalglobe.com/industry/introducing-the-digitalglobe-a3c-quality-framework/ (2017).
[8] ICAO: Annex 14. http://cockpitdata.com/Software/ICAO%20Annex%2014%20Volume%201 (2014).
[9] TC211: Geographic Information/Geomatics. https://committee.iso.org/home/tc211 (2017).
[10] NOAA: Completeness Ruberic. https://geo-ide.noaa.gov/wiki/index.php?title=Completeness_Rubric 2017).
[11] ISO 19165, Geographic information - Preservation of Digital Data and Metadata. Draft International Standard. (2017)