Evaluation of Audience Response Systems in College Classrooms

This research project evaluated two audience response systems, TurningPoint and Poll Everywhere, in the college classroom to determine their adequacy. Major differences between the clicker systems were discussed along with the research methodology used including observation points, courses involved, and timeline of the study. Reasons for using audience response systems in the classroom were explored, emphasizing their benefits such as immediate feedback, formative assessment, and enabling students to compare responses.

• Audience Response Systems
• College Classroom
• TurningPoint
• Poll Everywhere
• Research Project

cjag Follow

Uploaded on Sep 29, 2024 | 0 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

Presentation Transcript

1. Evaluation of two audience response systems in the college classroom Stephen L. Firsing III, PhD, MPA, MA John Delport, PhD Fredanna A.D. M Cormack McGough, PhD, RD John Yannessa, PhD Mariel Celina Po, BS Kaitlyn Brown, Student of Public Health

2. Lets try out the TurningPoint Audience Response System!

3. What is your favorite Super Hero? Superman Batman Iron Man Captain America Wonder Woman 40% 40% A. B. C. 20% D. 0% 0% E. Superman Iron Man Batman Wonder Woman Captain America

4. If you had to be some kind of animal, what kind of animal would you be? A. Eagle B. Shark C. Cheetah D. Dolphin Turtle 80% 20% E. 0% 0% 0% Shark Eagle Dolphin Cheetah Turtle

5. Now lets try out Poll Everywhere!

6. Why use audience response systems in the classroom? 1. Immediate Feedback 2. Formative Assessment 3. Students Compare Responses

7. What was the Purpose of this Research Project? The purpose of this research project was to evaluate the adequacy of two popular clicker systems in the Coastal Carolina University classroom: TurningPoint and Poll Everywhere.

8. What are the major differences between the clicker systems? TurningPoint ($, RF clicker) VS. Poll Everywhere (40 free, online, text)

9. What was the Research Methodology? = 13 Public Health Courses (3P) 4 Special Education Courses (1P) = TurningPoint = Poll Everywhere = Observation Points R Xa Xb 0 R Xa O1 Mid- Semester Xb O2O3 End- Semester Start- Fall 2014 Spring 2015

10. What were the Outcomes for Observation #1 (TurningPoint)? Quantitative = Audience Response Technology Questionnaire or ART-Q (25 items) Appraisal/Learning Negative Grade Impact Attendance Enjoyment Preparation/Motivation Ease of Use Instructor Performance Qualitative = Overall Likes & Dislikes (2 items)

11. What were the Outcomes for Observation #2 (Poll Everywhere)? Quantitative = Audience Response Technology Questionnaire or ART-Q (25 items) Appraisal/Learning Negative Grade Impact Attendance Enjoyment Preparation/Motivation Ease of Use Instructor Performance Qualitative = Overall Likes & Dislikes (2 items)

12. What were the Outcomes for Observation #3 (Comparison)? Quantitative = 5 ARS comparison items Was easier for you to use? (TP, PE, Neither) Engaged you more in the course material? Was easier for your instructor to use? Was more reliable or had fewer technical problems? Did you prefer using overall? Qualitative = General comments about the ARS systems and about the use of ARS (of any kind) in the classroom (2 items)

13. Study Participants = Undergraduate Students TurningPoint (O1) N = 306 Poll Everywhere (O2) N = 317 Comparison Survey (O3) N = 323

14. Appraisal/Learning Results (01 vs. 02) TurningPoint Mean Score SA=5, SD=1 Poll Everywhere Mean Score SA=5, SD=1 Question on ART-Q Questionnaire 1. Because of the ARS system, I had a better idea of what to expect on exams, quizzes, or assignments. 4.070 (p = .057) 3.934 2. I believe that I knew more about what would be emphasized on exams, quizzes, or assignments because of the ARS system. 4.073 (p = .244) 3.993 3. Using the ARS system gave me a preview of what I needed to know for exams, quizzes, or assignments. 4.137 (p = .130) 4.037 Note: P values calculated by paired samples t-test; significantly different values = p < .05

15. Negative Grade Impact Results (01 vs. 02) TurningPoint Mean Score SA=5, SD=1 Poll Everywhere Mean Score SA=5, SD=1 Question on ART-Q Questionnaire 4. The ARS system interfered with my getting a good grade. 1.967 (p = .259) 2.077 5. Because we used the ARS system, I expect to get a lower grade than I would have otherwise. 1.641 (p = .026) 1.809 6. Using the ARS system negatively impacted my grade. 1.585 (p = .083) 1.709

16. Attendance Results (01 vs. 02) TurningPoint Mean Score SA=5, SD=1 Poll Everywhere Mean Score SA=5, SD=1 Question on ART-Q Questionnaire 7. Because the ARS system was used, I attended class more regularly than I would have otherwise. 2.924 (p = .561) 2.877 8. Using the ARS system increased my likelihood of attending class on time. 2.947 (p = .969) 2.950 9. The ARS system motivated me to attend class on time. 2.997 (p = .750) 2.970

17. Enjoyment Results (01 vs. 02) TurningPoint Mean Score SA=5, SD=1 Poll Everywhere Mean Score SA=5, SD=1 Question on ART-Q Questionnaire 4.183 (p = .000) 10. Using the ARS system was fun. 3.940 11. It was exciting to answer questions using the ARS system. 4.150 (p = .000) 3.841 4.275 (p = .000) 12. I enjoyed using the ARS system. 3.933 1.622 (p = .001) 13. I did not like using the ARS system. 1.811 14. I had a good experience with the ARS system. 4.220 (p = .001) 3.977

18. Preparation/Motivation Results (01 vs. 02) TurningPoint Mean Score SA=5, SD=1 Poll Everywhere Mean Score SA=5, SD=1 Question on ART-Q Questionnaire 15. Using the ARS system made me more likely to review my notes prior to class. 3.372 (p = .177) 3.260 16. Using the ARS system encouraged me to do readings prior to class. 3.250 (p = .967) 3.253 17. Because we used the ARS system, I prepared for class more than I would have otherwise. 3.268 (p = .252) 3.174 18. The ARS system boosted my enthusiasm for studying the material we learned in the course. 3.483 (p = .259) 3.392

19. Ease of Use Results (01 vs. 02) TurningPoint Mean Score SA=5, SD=1 Poll Everywhere Mean Score SA=5, SD=1 Question on ART-Q Questionnaire 4.317 (p = .000) 19. Using the ARS system was easy. 4.042 20. I had no problems using the ARS system. 4.168 (p = .000) 3.874 21. Using the ARS system was pretty hard. 1.484 (p = .000) 1.707

20. Instructor Performance Results (01 vs. 02) TurningPoint Mean Score SA=5, SD=1 Poll Everywhere Mean Score SA=5, SD=1 Question on ART-Q Questionnaire 22. The ARS questions used by the instructor were easy. 3.808 (p = .543) 3.766 23. The ARS questions used by the instructor were clear and understandable. 24. The ARS questions used by the instructor were asked at the appropriate times during class. 4.220 (p = .015) 4.077 4.271 (p = .018) 4.142 25. Overall, I feel the ARS system improved the instructor s ability to teach. 3.961 (p = .120) 3.846

21. Overall Mean Scores Results (01 vs. 02) TurningPoint (O1) ART-Q Mean Score = 3.3788 Poll Everywhere (O2) ART-Q Mean Score = 3.2925 Mean Scores Significantly Different (p = .021)

22. Comparison Results (03) Of the two audience response systems used in this class, which system Turning Point Poll Neither Everywhere 28. Was easier for you to use? 57.4% 39.8% 2.8% 29. Engaged you more in the course material? 53.0% 39.8% 7.2% 30. Was easier for your instructor to use? 31. Was more reliable or had fewer technical problems? 45.7% 46.3% 8.0% 43.8% 44.1% 12.2% 32. Did you prefer using overall? 54.3% 41.6% 4.0%

23. Qualitative Results: Dislikes about Poll Everywhere Did not like texting long codes (28) Too many numbers to text, shorten numbers The codes to enter were too long. Maybe if it were like one letter it would have been better It confused some students about which numbers to text I have a flip phone too challenging Too much to keep texting numbers over and over instead of one click Not as fast as using the clickers

24. Qualitative Results: Dislikes about Poll Everywhere Did not like the technical issues/problems (24) Does not work on my phone; message block My texts never go through Sometimes my answers wouldn t submit Technical difficulties Tech problems!!! Sometimes the texts don t go through Sometimes technology is slow, doesn t work

25. Qualitative Results: Dislikes about Poll Everywhere Did not like using cell phone/texting in-class (19) Having to use my phone for it If your cell phone is dead/has no service then you can t use it Incorporating cell phones into the classroom seems inherently distracting Hard to stay focused with my phone in front of me Having to use my cell phone. It was so tedious Might not always have cell phone or data service to use it

26. Qualitative Results: Dislikes about Poll Everywhere Did not like that responses were real time (7) Everyone went with the majority answer No challenge because it shows the answers selected by others as they answer You could see everyone s response as they were going could influence vote You could see the most popular answers before you vote

27. Qualitative Results: Dislikes about TurningPoint Did not like the technical issues/problems (41) It did not work any time we used it It wasn t 100% reliable It has glitches a lot of the time Sometimes it worked and sometimes it didn t A lot of technical problems, wish we were able to use it more

28. Qualitative Results: Likes about Poll Everywhere Students liked that Poll Everywhere was: Engaging/interactive (37) Simple to use (36) A good way to review material (35) Anonymous (17) Using cell phones (11)

29. Qualitative Results: Likes about TurningPoint Students liked that TurningPoint was: Engaging/interactive (67) Good to review course material (57) Simple to use (42) Anonymous (26)

30. Conclusion Findings indicate, both quantitatively and qualitatively, that students at Coastal Carolina University favor using the TurningPoint system over the Poll Everywhere system in the classroom.