Fingerprinting Smart Devices Through Acoustic Components
Smartphones can be uniquely identified through their embedded acoustic components, allowing for targeted advertising and secondary authentication. This method leverages sensors like microphones to create device fingerprints. The goal is to extract these fingerprints through scenarios involving audio recordings or covert app activities. The article also discusses top Android permissions related to device tracking and control.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Do You Hear What I Hear? Fingerprinting Smart Devices Through Embedded Acoustic Components Anupam Das (UIUC), Nikita Borisov (UIUC), Matthew Caesar (UIUC) CCS 2014 October 5, 2024 1
Smartphone Usage How many people use smartphones? Source: Business Insider Source: Gartner Shipment of smartphone is increasing every year. PC(Desktop & Notebook) Tablet Mobile Phone 2,250,000 2,000,000 Thousands of Units 1,750,000 1,500,000 75% of the mobile phones are smartphones and highly- featured phones 1,250,000 1,000,000 750,000 500,000 250,000 0 2012 2013 2014 2015 Year 2 October 5, 2024
A Closer Look at Smartphones Today, smartphones come with a wide range of sensors. All of which are useful for a variety of tasks. Motion detection Gesture detection Audio Genre detection Location detection Interaction with nearby devices Compass However, sensors could also be potential source for unique fingerprints. 3 October 5, 2024
Why Fingerprint Smartphones? Smartphones can be fingerprinted for: Targeted Advertisement Secondary Authentication factor The main drawback for software based approaches is that the source of the fingerprints are not too stable. Software based approaches: Browser based features, Cookies Different Firmware and Device drivers Moreover, device IDs are not always usable. UDID for Apple devices has been removed since May 1, 2013 and IMEI for an Android Device requires explicit permission. Hardware based approaches: Clock Skew rate Radio Transmitter Accelerometer 4 October 5, 2024
Our Goal We focus on fingerprinting smart devices through their embedded speakers and microphones. Requires: Deployed microphones Scenario 1: External attacker locally present 1. Attacker records audio signal from distance 2. Create a fingerprint of the recorded audio and link the fingerprint to a unique smartphone Scenario 2: Stealthy App 2. Audio signal App Need access to only: 1. Microphone 2. Internet connection 4. Extract Fingerprint 5 October 5, 2024
Top Android Permissions Top 17 Android permissions out of 173 total permissions 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 Storage : modify/delete USB storage and SD card contents Network communication : full Internet access Network communication : view network state System tools : prevent device from sleeping Phone calls : read phone state and identity Hardware controls : control vibrator System tools : automatically start at boot Network communication : view Wi Fi state Your location : fine (GPS) location Your location : coarse (network based) location System tools : retrieve running applications Your personal information : read contact data Your messages : read SMS or MMS Your messages : receive SMS Hardware controls : take pictures and videos Hardware controls : record audio System tools : modify global system settings Source: Mario Frank, Ben Dong, Adrienne Porter Felt, Dawn Song. Mining Permission Request Patterns from Android and Facebook Applications, ICDM, 2012 6 October 5, 2024
Real World Apps There are popular apps that people use that require similar permissions. For example: My Talking Tom 4,405,876 7 October 5, 2024
Closer Look at Microphones MEMS microphone: Sound Wave Mechanical Energy Capacitive Change Voltage Change The sensitivity of the microphone depends on how well the diaphragm deflects to acoustic pressure Imperfections can arise due to: 1. Slight variations in the chemical composition of components from one batch to the next 2. Wear and tear in the manufacturing machines 3. Changes in temperature and humidity Source: STMicroelectronics 8 October 5, 2024
Closer Look at Speakers Basic functionality of a speaker : Electrical current around voice coil Magnetic field Mechanical Energy Sound Wave The frequency of the sound wave produced is dictated by the rate at which the voice coil moves. Source: Center Point Audio 9 October 5, 2024
Experimental Setup Maker Model # Apple iPhone 5 1 HTC Nexus One 14 Nexus S 8 Samsung Galaxy S3 3 Galaxy S4 10 Motorola Droid A855 15 Sony Ericsson W518 1 Total 52 To emulate an attacker we use a laptop Audio Type Description Variations Instrumental Musical instruments playing together 4 Human Speech Small segments of human speech 4 Song Combination of human voice & instrumental sound 3 10 October 5, 2024
Fingerprinting Acoustic Components 1. Fingerprinting Speakers This scenario is suited for the case where the attacker has deployed nearby microphones to capture audio signal (e.g., ringtone) from user device. 11 October 5, 2024
Fingerprinting Acoustic Components 2. Fingerprinting Microphones 3. Fingerprinting Both Speakers and Microphones In these scenarios the attacker has stealthy convinced the user to provide access to microphone. 12 October 5, 2024
Acoustic Features We investigate a total of 15 acoustics features # Feature Dimension 1 Root Mean Square 1 2 Zero Crossing Rate 1 3 Low-Energy-Rate 1 4 Spectral Centroid 1 5 Spectral Entropy 1 6 Spectral Irregularity 1 7 Spectral Spread 1 8 Spectral Skewness 1 9 Spectral Kurtosis 1 10 Spectral Rolloff 1 11 Spectral Brightness 1 12 Spectral Flatness 1 13 Mel-Frequency-Cepstral-Coefficients 13 14 Chromagram 12 15 Tonal Centroid 6 13 October 5, 2024
Feature Selection Are there any dominant features? Identifying the dominant feature set benefits us in two ways: 1. Less computation (potentially shifting the feature extraction component into mobile devices) 2. Improve accuracy as there might be conflicting features Feature Reduction (Dimensionality reduction) Feature Extraction Feature Selection [subset of old features] [new features=function(old features)] e.g., PCA, LDA Feature selection is preferable to feature extraction when dimensionality and numerical transformations of the features are inappropriate. 14 October 5, 2024
Sequential Feature Selection We adopt a well-known machine learning technique called sequential forward selection (SFS). It is a greedy approach where features are added only if it increases accuracy. We found Mel-Frequency-Cepstral-Coefficients (MFCCs) as the dominating features. MFCCs: Compactly represent the spectrum along the nonlinear mel-scale of frequency. Distinguish the low and fast varying spectral envelopes of the signal. 15 October 5, 2024
Evaluation Algorithms & Metrics We evaluate using two classification algorithm: k-NN: k-nearest neighbors GMM: Gaussian Mixture Model Evaluation metrics: ?? TP: True Positive FP: False Positive FN: False Negative ????????? = ?? + ?? ?? ?? + ?? ?????? = ?1 ????? =2 ????????? ?????? ????????? + ?????? We compute the average across all classes. We use a 1:1 ratio for training and testing the classifiers. 16 October 5, 2024
Different Make & Model Sets We consider one set from each make & model, giving us a total of 7 different sets. Maker Model # Apple iPhone 5 1 Speaker Microphone Speaker & Microphone HTC Nexus One 14 10 Nexus S 8 9 8 Samsung Galaxy S3 3 1 - F1_score (%) 7 6 Galaxy S4 10 5 Motorola Droid A855 15 4 2.6 3 Sony Ericsson W518 1 2 1 0 0 0 0 0 0 0 0 0 Total 52 Instrument Human Speech Song Genre So we can accurately distinguish smartphones of different make and model. 17 October 5, 2024
Same Make & Model Sets We consider 15 Motorola Droid A855 handsets. Maker Model # Apple iPhone 5 1 Speaker Microphone Speaker & Microphone HTC Nexus One 14 10 9 Nexus S 8 8 1 - F1_score (%) Samsung Galaxy S3 3 7 5.5 6 Galaxy S4 10 4.7 5 3.9 Motorola Droid A855 15 4 3 Sony Ericsson W518 1 1.7 1.7 2 1 0 0 0 0 0 Total 52 Instrument Human Speech Song Genre Fingerprinting both the speaker and microphone seems to provide better results. 18 October 5, 2024
Heterogeneous Smartphones We develop an Android App to collect audio samples from 50 Android sets. In this case the audio clip is played and recorded on the phone set. Maker Model # 10 Apple iPhone 5 1 9 HTC Nexus One 14 8 1- F1_score (%) 7 Nexus S 8 6 Samsung Galaxy S3 3 5 4 Galaxy S4 10 3 1.7 Motorola Droid A855 15 2 0.7 1 0 Sony Ericsson W518 1 0 Instrument Human Speech Genre Song Total 52 We were able to correctly classify ~98% of the recorded audio clips. 19 October 5, 2024
Sensitivity Analysis We analyze how different factors impact our fingerprinting accuracy to better understand the feasibility of our approach. We analysis the following factors: Sampling rate Training set size Distance between speaker and microphone Ambient background noise We only consider same make and model handsets for the following set of experiments (as this is a harder problem) 20 October 5, 2024
Impact of Distance We vary the distance between the speaker and microphone. k-NN GMM 100 90 80 We used an Audio- Technica ATR-6550 shotgun microphone (~$45) for this experiment 70 F1 Score 60 50 40 30 20 10 0 0.1 1 2 3 4 5 Distance (in meters) After a distance of 2 meters the accuracy tend to go down at a faster rate. 21 October 5, 2024
Impact of Background Noise We emulate four types of ambient background noise using external speakers. Played audio from the instrument category Environment SNR (dB) GMM 2 Meters Shopping Mall 15.84 (16%) 94.2 Restaurant 17.77 (13%) 91.6 City Park 15.43 (17%) 94.6 Airport Gate 14.92 (18%) 93.9 Even with relative amount of background noise we can successfully fingerprint smartphones. 22 October 5, 2024
Discussions Providing a counter measure against such a side channel attack is still a open research question. One could attempt to shift the frequencies of the audio signal in a way so as to not deter the quality of the audio stream too much. But a thorough analysis of the impact of such a counter measure is required to fully understand its feasibility as audio streams are used for legitimate purposes too. 23 October 5, 2024
Conclusion We see that it is possible to fingerprint smartphones through embedded microphones and speakers. We were able to accurately attribute ~98% of all recorded audio excerpts from 50 different Android devices. So, the next time you talk to Tom keep in mind whether you are giving Tom a fingerprint of your device. More details of the project is available at the following link- http://web.engr.illinois.edu/~das17/SmartphoneFingerprint.html 24 October 5, 2024
The End 25 October 5, 2024
