Advancements in Wearable Cognitive Assistance Technology

This presentation explores the potential of wearable technology in aiding cognitive decline, such as mild cognitive impairment and survivors of stroke. It delves into how wearable devices can continuously capture, interpret, and give guidance to users, potentially saving millions of dollars in healthcare costs. The discussion covers challenges in architecture related to interactive response and parallelism. The comparison between standalone and offloading to cloud or cloudlets shows significant improvements in latency and energy efficiency. Overall, the focus is on leveraging wearable devices to provide effective cognitive assistance.

• Wearable Technology
• Cognitive Assistance
• Interactive Response
• Architecture Challenges
• Health Tech

brave Follow

Uploaded on Oct 08, 2024 | 0 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

Presentation Transcript

1. Gabriel: Towards Wearable Cognitive Assistance Kiryong Ha, Zhuo Chen, Wenlu Hu, Wolfgang Richter, Padmanabhan Pillai, and Mahadev Satyanarayanan

2. Cognitive Decline mild cognitive impairment survivors of stroke One-month delay in nursing home 20,000,000+ Americans affected admissions saves $1,000,000,000+/year Alzheimer s disease traumatic brain injury Ron Faces Daily Routine Text 10/8/2024 2

3. Can Wearable Technology Help? Continuously capture, interpret, and give guidance System Architecture 10/8/2024 3

4. Barack is saying hello to you Please stop and check traffic Your dog wants to go out for a walk 10/8/2024 4

5. Why Today? Advances in 3 Independent Arenas Algorithms Wearable Hardware Offloading Technology 10/8/2024 5

6. Challenges Architecture 1. Crisp Interactive Response 2. Graceful Degradation of Services 3. Coarse-grain Parallelism 10/8/2024 6

7. C1. Crisp Interactive Response Humans are fast and sensitive Face exists? <700ms Voice Phrases? ~400ms Whose face? ~500ms Satellite Phone? ANNOYING! Goal: Latency of infrastructure = tens of millisecond 10/8/2024 7

8. S1. Crisp Interactive Response Choice 1: standalone apps Choice 2: offload to cloud Choice 3: offload to cloudlets ~74ms Distant Resources in Cloud WAN Glass Device Wi-Fi LAN <10ms Nearby Cloudlet 10/8/2024 8

9. Exp. Cloudlet Shortens Latency Offloading vs. Standalone (OCR) Offloading saves latency and energy Metric Standalone With Offload Per-image speed (s) 10.49 1.28 Per-image energy(J) 12.84 1.14 Cloudlet vs. Cloud (Augmented Reality) 1 0.75 Cloudlet Cloud Cloudlet shortens response time 0.5 0.25 0 0 200 Response Time (ms) 400 600 10/8/2024 9

10. C2. Graceful Degradation of Services What if offloading impossible? Situation 1: No cloudlet Situation 2: No network Wi-Fi ? Distant Resources in Cloud WAN Glass Device ? LAN Nearby Cloudlet Goal: still work during failures with performance drop 10/8/2024 10

11. S2. Graceful Degradation of Services Use fallback resources No cloudlet offload to cloud No network offload to carry-on device ? Wi-Fi or Bluetooth Distant Resources in Cloud WAN Glass Device Wi-Fi LAN? Nearby Cloudlet On-body Offload Device Application-specific fidelity vs. Crispness & battery life 10/8/2024 11

12. C3. Coarse-grain Parallelism Don t reinvent the wheel Face recognition Object detection Activity inference OCR Goal: reuse existing work, but Programming languages are different Runtime systems are different (different OSes, closed- source, etc.) 10/8/2024 12

13. S3. Coarse-grain Parallelism VM Ensemble and PubSub Backbone Virtual Machine Core Algorithm Logic Proxy Dataset Config 10/8/2024 13

14. S3. Coarse-grain Parallelism VM Ensemble and PubSub Backbone Cognitive VMs Cognitive Engine 1 e.g. Face recognition Control VM Wi-Fi Glass Device Cognitive Engine 2 Video/Acc/ sensor streams Device Comm PubSub Cognitive Engine 3 UPnP ... User Guidance VM Sensor flows Cognitive Engine n Cognitive flows Cloudlet VM boundary 10/8/2024 14

15. S3. Coarse-grain Parallelism VM Ensemble and PubSub Backbone Cognitive VMs Wi-Fi Cognitive Engine 1 e.g. Face recognition Control VM Wi-Fi Context Inference Glass Device Cognitive Engine 2 Video/Acc/ sensor streams Device Comm PubSub Cognitive Engine 3 UPnP ... User Guidance VM Sensor flows Cognitive Engine n Cognitive flows Cloudlet VM boundary 10/8/2024 15

16. Exp. Gabriel Overhead Gabriel Latency: ~50ms Gabriel Overhead: ~3ms Echos every image Dummy Cognitive Engine 1 0.75 No VM, echos from host 0.5 Gabriel Ideal 0.25 0 0 20 40 60 80 100 Response Time (ms) 10/8/2024 16

17. Prototype Back-end Server GDK Preview TCP Connection Speech Guidance Ice pack to cool down Glass 10/8/2024 17

18. Prototype Back-end Server Cloudlet: 4 advanced desktop machines Running OpenStack Virtualized Cloud Computing Platform 10/8/2024 18

19. Prototype Cognitive Engines Face Recognition Commercial Product Object Recognition (1. MOPED 2. STF ) OCR (1. Tesseract 2. VeryPDF ) Motion Classifier Based on Accelerometer Augmented Reality Activity Detection 10/8/2024 19

20. Exp. Full System Performance Cognitive Engines are slower Response time (ms) Cognitive Engine FPS Glass Life 1% 10% 50% 90% 99% Face Recognition 4.4 196 389 659 929 1175 Object (MOPED) 1.6 877 962 1207 1647 2118 Object (STF) 0.4 4202 4371 4609 5055 5684 OCR (Open) 14.4 29 41 87 147 511 ~1 hour OCR (Comm) 2.3 394 435 522 653 1021 Motion Classifier 14.0 126 152 199 260 649 Augmented Reality 14.1 48 72 126 192 498 Gabriel architecture allows easy upgrade. 10/8/2024 20

21. Exp. Full System Performance Cognitive Engines require different FPS Response time (ms) Cognitive Engine FPS Glass Life 1% 10% 50% 90% 99% Face Recognition 4.4 196 389 659 929 1175 Object (MOPED) 1.6 877 962 1207 1647 2118 Object (STF) 0.4 4202 4371 4609 5055 5684 OCR (Open) 14.4 29 41 87 147 511 ~1 hour OCR (Comm) 2.3 394 435 522 653 1021 Motion Classifier 14.0 126 152 199 260 649 Augmented Reality 14.1 48 72 126 192 498 Gabriel uses two-level token-based flow control 10/8/2024 21

22. Control VM Processed Items Count M A X Glass Device Cognitive VM sensor drop drop tokens Cognitive VM a b Drop Count a > b? drop yes no sum sensor data counts 10/8/2024 22

23. More in the Paper 1. Token-based flow control improves response time a lot 2. Gabriel supports multi-VM parallelism 3. Tradeoff between fidelity reduction and crisp user interaction 10/8/2024 23

24. Conclusion & Future Work Gabriel: low-latency, flexible architecture Speed improvement needed Algorithms Wearable Hardware Offloading Technology Longer battery, better thermal dissipation Cloudlets are helpful, need good biz. model 10/8/2024 24

25. Gabriel: Towards Wearable Cognitive Assistance Contact: Kiryong Ha (krha@cmu.edu) Zhuo Chen(zhuoc@cs.cmu.edu) 10/8/2024 25