Data Quality Evaluation Framework at DQWG15 Conference
Explore the framework of data quality concepts discussed at the DQWG15 conference in Monaco. Learn about the ISO-19157 standards for evaluating data quality, including aspects like data format consistency, logical consistency, completeness, and accuracy. Discover production validation checks for S-1xx exchange sets and classification abbreviations for ENC data quality assessment.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
How to evaluate an S-1xx Exchange Set DQWG15-04.4A DQWG15, Monaco 4-7 February 2020
FRAMEWORK OF DATA QUALITY CONCEPTS courtesy of ISO-19157 DQWG15, Monaco 4-7 February 2020
ISO 19157 ORDERING IN DATA QUALITY EVALUATION actual dataset format consistency evaluation (1) no not readable part readable? yes readable part of actual dataset other logical consistency evaluation (2) no conformant with rules? data items violating rules yes data suitable for further assessment ref ISO-19157 figure I.1 DQWG15, Monaco 4-7 February 2020
ISO 19157 ORDERING IN DATA QUALITY EVALUATION data suitable for further assessment completeness evaluation (3) items present in actual data and ground truth? no items present in either actual data or ground truth yes features present both in actual and ground truth data accuracy evaluation (4) Data Quality Result ref ISO-19157 figure I.1 DQWG15, Monaco 4-7 February 2020
S-1xx EXCHANGE SET S-1xx Exchange Set S-111 S-123 S-101 S-121 S-127 S-129 S-102 S-122 series dataset subset feature/attribute type feature/attribute instance DQWG15, Monaco 4-7 February 2020
PRODUCTION VALIDATION CHECKS FOR A S-1XX PS Check Classification Abbreviation Type Description C Critical An error which would make an ENC unusable in ECDIS through not loading or causing an ECDIS to crash or presenting data which is unsafe for navigation. E Error An error which may degrade the quality of the ENC through appearance or usability but which will not pose a significant danger when used to support navigation. W Warning An error which may be duplication or an inconsistency which will not noticeably degrade the usability of an ENC in ECDIS. DQWG15, Monaco 4-7 February 2020
PRODUCTION VALIDATION CHECKS FOR A S-1XX PS Check Application Abbreviation Type Description B Base Apply check to new dataset, new edition, and post- update dataset (after updates have been applied to the base). U Update Apply check to update datasets in isolation. S Post- Update Apply check only to a post-update dataset, i.e., subsequent to application of all available updates. DQWG15, Monaco 4-7 February 2020
EVALUATION AGAINST A SINGLE PS (E.G. S-101) level of detail SERIES DATASET SUBSET FEATURE / ATTRIBUTE TYPE FEATURE / ATTRIBUTE INSTANCE CONFORMITY TO S-101 PS S-101 Exchange Set FULLY PARTIAL NOT TESTED FORMAT CONSISTENCY LOGICAL CONSISTENCY CONCEPTUAL CONSISTENCY classes, attributes, basic data types, primitive types, complex types, predefined derived types, enumerated types, codelist types, relationships / associations, composition / aggregation, stereotypes, optional, conditional and mandatory attributes and associations, naming and name spaces, notes, packages DOMAIN CONSISTENCY feature catalogue (use XSD tools) TOPOLOGICAL CONSISTENCY within, crosses, touches, disjoint, overlaps, contains, equal, intersects, covered (by), coincident type of check COMPLETENESS commission / omission ACCURACY absolute (external) / relative (internal) DQWG15, Monaco 4-7 February 2020
EVALUATION OF S-1XX EXCHANGE SET (user needs) S-129 (FULLY / PARTIAL / NOT TESTED) S-127 (FULLY / PARTIAL / NOT TESTED) S-123 (FULLY / PARTIAL / NOT TESTED) WARNING BASE S-122 (FULLY / PARTIAL / NOT TESTED) ERROR S-121 (FULLY / PARTIAL / NOT TESTED) UPDATE POST-UPDATE CRITICAL S-111 (FULLY / PARTIAL / NOT TESTED) S-102 (FULLY / PARTIAL / NOT TESTED) S-101 (FULLY / PARTIAL / NOT TESTED) DQWG15, Monaco 4-7 February 2020
IHO HSSC Data Quality Working Group DQWG15, Monaco 4-7 February 2020
