New Techniques & Technologies for Official Statistics at NTTS 2021
The European Platform for Trusted Smart Surveys is being developed to enhance smart survey methodology by focusing on incentive schemes, privacy-preserving techniques, technical infrastructure, and process auditability. Main goals in Work Package 3 (WP3) include defining a reference architecture for trusted smart surveys and developing proofs-of-concept for essential aspects of the platform's architecture. Smart survey methodology addresses sensor integration, machine learning, survey pipelines, and staff profiles for efficient data collection and analysis.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Bruxelles, Thursday 11 March New Techniques & Technologies for Official Statistics (NTTS 2021) The Conceptual Framework for the European Platform for Trusted Smart Surveys Claudia De Vitiis, Mauro Bruno (ISTAT) Franck Cotton (INSEE) Matjaz Jug (CBS) Jacek Maslankowski (Statistic Poland) Nils Meise (Destatis)
Outline o Main goals and activities in WP3 o Smart Survey Methodology o Incentive schemes o Privacy preserving techniques o Technical infrastructure o Process auditability and Metadata o Future works in 2021: Proofs-of-Concepts 2 THE CONCEPTUAL FRAMEWORK FOR THE EUROPEAN PLATFORM FOR TRUSTED SMART SURVEYS
Main goals and activities in WP3 1/2 The ESSNet on Smart Surveys is carrying out preparatory work to create a European wide platform to share und re-use smart survey solutions and components. In this context, the Work Package 3 (WP3) is working on the characterization of TSS platform through two main tasks: Definition of a new reference architecture for trusted smart surveys: o methodological framework for smart surveys o conceptualization and development of a general platform for Trusted Smart Surveys, following a top-down design approach; the platform will be flexible and implementing a set of common functions and configurable services that can be used to build instances of trusted smart surveys for specific application domains and/or target areas. Developmentof proofs-of-concept in the form of modular prototype elements for essential aspects of the architecture. 3 THE CONCEPTUAL FRAMEWORK FOR THE EUROPEAN PLATFORM FOR TRUSTED SMART SURVEYS
Smart survey methodology Smart surveys could be designed for any device that has access to passive data collection features. Usually, these devices have one or multiple sensors that collect data for their own functionality or provide access to their sensor data to other applications. Design choices are based on informed choices of the integration of big data and preceding methodological research within the ESS for the use of mobile devices and sensors. The main topics addressed are error treatment new forms of data also pose new forms of problems and applied reviews of machine learning techniques. The task on smart survey methodology delivers guidelines, infrastructure and staff profile requirements for the implementation of smart data in smart surveys and ways to deal with hybrid forms of data already in the design process 4 THE CONCEPTUAL FRAMEWORK FOR THE EUROPEAN PLATFORM FOR TRUSTED SMART SURVEYS
Smart survey methodology Methodological design principles for smart surveys o sensor integration o machine learning o survey pipelines o staff profiles Smart survey methodology deals with o hybrid forms of data collection active data collection (e.g. questionnaires) passive data collection (e.g. sensors) o machine learning error recognition Analysis o GSBPM phases: design, collect and analysis o survey and analysis automation Sensor data categories Motion: Movement of a device (e.g. accelerometer, gyroscope) Environment: Surroundings of a device (e.g. temperature, pressure, humidity, illumination) Position: Physical position of a device (e.g. GPS, magnometer) and indirect measurement via radios (e.g. WiFi, Bluetooth, cellular network) Machine learning o Types of errors: new sources of sensor data and a general broader access to diverse (big) data requires specialized methods o Improved quantity and quality of data analysis o New opportunities: trained models can be used in ongoing or future surveys. 5 THE CONCEPTUAL FRAMEWORK FOR THE EUROPEAN PLATFORM FOR TRUSTED SMART SURVEYS
Privacy preserving techniques To maintain full respondent s trust and be fully compliant with the EU and national privacy regulation ESS smart surveys need to apply established principles, develop guidelines and introduce new technologies and techniques in area of privacy and transparency. This task explores best practices, identifies concrete challenges related to smart surveys, and proposes guidelines and methodological and technical solutions. Processing the smart surveys personal data also requires new architectural, methodological and technological techniques and approaches: Guidelines are based on established privacy-by- design principles : o Proactive not reactive; preventive not remedial; o Privacy as the default setting; o Privacy embedded into design; o Full functionality positive-sum, not zero-sum; o End-to-end security full lifecycle protection; o Visibility and transparency keep it open; o Respect for user privacy keep it user-centric: o Data minimization (for example pre- processing on survey participant devices) o Advanced privacy-preserving methods (for example Secure Multi-Party Computation); o Solutions for full auditability and complete transparency of the processing methods applied to the data 6 THE CONCEPTUAL FRAMEWORK FOR THE EUROPEAN PLATFORM FOR TRUSTED SMART SURVEYS
Incentive schemes Trusted Smart Surveys should be engaging and not addictive in nature, which could violate the trust respondents put into it. Trusted Smart Surveys offer the potential to include gamified parts or full gamification. To gamify a survey means to apply game mechanics to motivate and engage respondents in the activity. Especially mobile devices as a gateway for smart data and questionnaires are prone to that implementation Incentives should increase the benefit and decrease the burden of a survey A smart incentive uses the data from a survey to create such a benefit Gamification is a strong incentive for active participation or use of an app and game mechanics can be based on survey data. Gamification can provide personalized feedbackbased on respondent s engagement Implementing gamification requires changes to the user interface and/or back-end systems and more time in development and testing 7 THE CONCEPTUAL FRAMEWORK FOR THE EUROPEAN PLATFORM FOR TRUSTED SMART SURVEYS
Technical infrastructure One of goals of the architectural task is to define the logical components of the TSS platform. In this context BREAL, developed in ESSnet Big Data II grant by work package F, is used as a reference architecture. BREAL divides business processes into different phases, i.e., acquisition and recording, data wrangling, data representation, modelling and interpretation with shape output as the last phase. Main outcome of the task: o Prepare a list of requirements for the technical infrastructure o Technical infrastructure is based on HBS / MOTUS smart surveys applications o Present specific use cases (i.e., HBS / MOTUS) and provide generic model based on them 8 THE CONCEPTUAL FRAMEWORK FOR THE EUROPEAN PLATFORM FOR TRUSTED SMART SURVEYS
Technical infrastructure - Requirements # 1 Id 1.1. Requirement Data storage Description All data gathered should be stored in structured format in database Stakeholder NSI or machine-readable files. Application should gather most of the data from sensors available in 1.2. Sensors Eurostat 2 devices. The application should transmit the data with standards, i.e. pass 1.3. API Eurostat 3 through JSON, SDMX etc. The smart survey should use the most common devices, such as 1.4. Devices Eurostat 4 mobile phones rather than computers. The application should be available to all popular platforms of 1.5. Platforms Eurostat 5 devices, e.g. Google Android, Apple iOS. Respondents who are in the sample frame and does not have a 1.6. Alternative Eurostat 6 versions device needed by smart surveys should have a possibility to use alternative format, e.g. paperback questionnaire. 9 THE CONCEPTUAL FRAMEWORK FOR THE EUROPEAN PLATFORM FOR TRUSTED SMART SURVEYS
Process auditability and metadata 1/2 As part of the definition of the TSS platform, a specific task is dedicated to the specification of the metadata that are useful in the context of smart surveys. All the metadata traditionally associated to statistical operations are relevant, including definitions of concepts, code lists, descriptive metadata, information and process models, etc. but for the TSS environment, special attention will be given to process execution and quality indicators, provenance and privacy. The use of sensors represents a relatively new field for official statistics therefore their modelling and processing will receive a specific focus. 10 THE CONCEPTUAL FRAMEWORK FOR THE EUROPEAN PLATFORM FOR TRUSTED SMART SURVEYS
Process auditability and metadata 2/2 Referential metadata Structural metadata Information Classifications Process Process specification Methods Tools Trust Quality Provenance Traceability Privacy 11 THE CONCEPTUAL FRAMEWORK FOR THE EUROPEAN PLATFORM FOR TRUSTED SMART SURVEYS
Future work in 2021: Proofs-of-Concepts The preliminary framework is described in a deliverable for the ESSNet In 2021, some aspects of the architecture will be experimentally developed in Proofs-of-Concept. Through the identification and implementation of use cases the Proofs-of-Concept will focus on: Machine learning Incentive schemes Privacy preservation Metadata IT components of the platform 12 THE CONCEPTUAL FRAMEWORK FOR THE EUROPEAN PLATFORM FOR TRUSTED SMART SURVEYS
