Voice Programming Advancements at VL/HCC 2004
Explore the advancements in programming by voice technology showcased at VL/HCC 2004, including discussions on disambiguating file references and multiple valid interpretations. Witness how programming tasks were tackled using voice commands on September 28, 2004.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Programming By Voice Andrew Begel Advisor: Prof. Susan L. Graham University of California, Berkeley VL/HCC 2004 September 28, 2004
Programming by Voice for int i equals zero i less than ten i plus plus September 28, 2004 VL/HCC 2004 Graduate Student Consortium 1
Programming by Voice for int i equals zero i less than ten i plus plus for (int i = 0; i < 10; i++ ) { } September 28, 2004 VL/HCC 2004 Graduate Student Consortium 1
Ambiguities Spelling of ID? KW or ID? KW or #? 4 int eye equals 0 aye less then ten i plus plus for (int i = 0; i < 10; i++ ) { } 2 September 28, 2004 VL/HCC 2004 Graduate Student Consortium
Sometimes its hard! for times ate equals file two load equals one September 28, 2004 VL/HCC 2004 Graduate Student Consortium 3
Many Valid Interpretations! for times ate equals file two load equals one for (times; ate == file; to().load = 1) { } September 28, 2004 VL/HCC 2004 Graduate Student Consortium 3
Many Valid Interpretations! for times ate equals file two load equals one for (times; ate == file; to().load = 1) { } 4 * 8 = file; toload = won September 28, 2004 VL/HCC 2004 Graduate Student Consortium 3
Many Valid Interpretations! for times ate equals file two load equals one for (times; ate == file; to().load = 1) { } 4 * 8 = file; toload = won fore.times(8).equalsFile(2, load == 1) September 28, 2004 VL/HCC 2004 Graduate Student Consortium 3
Disambiguating filetoload file to load file 2 load file toload filetoload September 28, 2004 VL/HCC 2004 Graduate Student Consortium 4
Disambiguating filetoload file to load file 2 load file toload filetoload 1. file to load 9. file 2 load 12. file toload 15. filetoload() 2. file(to, load) 10. file(2, load) 13. file(toload) 16. filetoload 3. file(to.load) 11. (file, 2, load) 14. file.toload 4. file(to(load)) 5. file.to(load) 6. file.to load 7. file to.load 8. file.to.load September 28, 2004 VL/HCC 2004 Graduate Student Consortium 4
Disambiguating filetoload file to load file 2 load file toload filetoload 15. filetoload() 1. file to load 9. file 2 load 12. file toload 2. file(to, load) 10. file(2, load) 13. file(toload) 16. filetoload 3. file(to.load) 11. (file, 2, load) 14. file.toload 4. file(to(load)) 1. No immediately adjacent identifiers 5. file.to(load) 6. file.to load 7. file to.load 8. file.to.load September 28, 2004 VL/HCC 2004 Graduate Student Consortium 4
Disambiguating filetoload file to load file 2 load file toload filetoload 15. filetoload() 2. file(to, load) 10. file(2, load) 13. file(toload) 16. filetoload 3. file(to.load) 11. (file, 2, load) 14. file.toload 4. file(to(load)) 1. No immediately adjacent identifiers 5. file.to(load) 2. No variables named file 8. file.to.load September 28, 2004 VL/HCC 2004 Graduate Student Consortium 4
Disambiguating filetoload file to load file 2 load file toload filetoload 15. filetoload() 2. file(to, load) 10. file(2, load) 13. file(toload) 16. filetoload 3. file(to.load) 4. file(to(load)) 1. No immediately adjacent identifiers 2. No variables named file 3. No methods named file September 28, 2004 VL/HCC 2004 Graduate Student Consortium 4
Disambiguating filetoload file to load file 2 load file toload filetoload 15. filetoload() 16. filetoload 1. No immediately adjacent identifiers 2. No variables named file 3. No methods named file 4. No method named filetoload September 28, 2004 VL/HCC 2004 Graduate Student Consortium 4
Disambiguating filetoload file to load file 2 load file toload filetoload filetoload 1. No immediately adjacent identifiers 2. No variables named file 3. No methods named file 4. No method named filetoload September 28, 2004 VL/HCC 2004 Graduate Student Consortium 4
The Vision Naturally Verbalized Programs Spoken Java Language Navigation and Editing Command Language Analyses that Resolve Ambiguities Prototype: SPED: SPeech EDitor September 28, 2004 VL/HCC 2004 Graduate Student Consortium 5
Implementation Speech Recognition: IBM ViaVoice Eclipse IDE Harmonia program analysis toolkit Generalized LR parsing with input stream ambiguities Persistent, incremental semantics September 28, 2004 VL/HCC 2004 Graduate Student Consortium 6
Evaluation Hypothesis Programmers can learn to use SPED efficiently for many programming tasks User Study 1. Train users on Spoken Java and command language 2. Edit an existing program 3. Create some new code Metrics: Speed, vocabulary mistakes, grammatical mistakes, system understanding errors, subjective impressions September 28, 2004 VL/HCC 2004 Graduate Student Consortium 7
Contributions Enabling programming by voice New methods for handling input ambiguities Exploiting syntax and semantics of programming domain Analyses for mixed command and programming languages Interface to commercial speech recognition tools Andrew Begel: abegel@cs.berkeley.edu September 28, 2004 VL/HCC 2004 Graduate Student Consortium 8
