Assessment Instrument for Blended Language Learning Module

An Assessment Instrument for Blended Language learning Module The Revised Community of Inquiry Survey Xinying Zhang Shenzhen University, China 18/06/2019

Contents: Revised Community of Inquiry (CoI) and previous research Blended language learning module in this study Research questions and the survey Findings and discussion Conclusion and future study

Framework

Some researchers have sought to expand and refine the CoI framework: Most prominently, Peter Shea et al. (2012) propose a fourth factor, learner presence . Cleveland-InnesandCampbell (2012) propose emotional presence as a fourth factor. Garrison (2017) argues against these additions, claiming they unnecessarily complicate and distort the core structure of the framework.

Figure 2 The RCoI framework applied in this study (Kim et al. 2014, 39)

Type of Presence Definition Example Cognitive Presence Knowledge building involving critical and creative thinking Challenging tasks, the cyclical process of practical inquiry, and a multivariate measure of critical and creative thinking Social Presence Encouraging collegial settings Discourses among students and instructor that promote positive affect, interaction, and cohesion Teaching Presence Instructional orchestration appropriate to the learning environments Task sets such as organization, design, discourse facilitation, and direct instruction Learner Presence Self- and co-regulation of learning Self-and co-regulatory strategies that marshal thoughts, emotions, motivations, behaviors and strategies Table 1 Revised community of inquiry framework.

Researchers proposed a valid, reliable, and efficient instrument to measure the dimensions of social, cognitive and teaching presence, thereby providing additional support for the validity of the CoI as a framework for constructing effective online learning environments (Arbaugh et al. 2008). An empirical research to investigate the effectiveness of online courses in a domestic normal university in China applied this assessment instrument and proved its reliability and validity.

Blended learning Blended learning is the organic integration of thoughtfully selected and complementary face- to-face and online approaches and technologies. (Garrison & Vaughan, 2008) The strength of integrating face-to-face synchronous communication and text-based online asynchronous communication is powerfully complementary for higher educational purposes (Vaughan, Cleveland-Innes & Garrison, 2013).

Learner cohort: top 10% of undergraduate freshmen 36 weeks of teaching period (two semesters) 10 credits of compulsory course Level equivalent to B1 of CEFR SPOC embedded blended language learning module Face-to-face teaching 80 mins/week Class size of 28-35

Research questions: The reliability and validity of the RCoI survey. Pinpointed by this assessment instrument, what are the students perception of their learning experiences, including the technology use? what are the strength and weakness of this learning module? what will be the further revision of this module?

Survey (Kim et al. 2014): The survey was given out in July, 2017,before the end of the term. Participants: -7 out of 25 classes were invited. -180 valid results were received. (response rate: 73.4%) -Age group of 18-20, and the gender ratio: M to F 3:5

In English and Chinese Survey composition: RCoI survey (32 items of likert scale 1-4) RCoI open ended questions (5 questions) Technology use perception (5 items of likert scale 1-4)

Results N Alpha Teaching presence 180 0.92 Social presence 180 0.88 Cognitive presence 180 0.91 Learner presence 180 0.86 Technology use 180 0.71 Table 1 The Cronbach alpha coefficient for each presence and technology use

Figure 3 Correlations among presences

N Mean SD Min Max Teaching presence 180 3.62 0.53 1 4 Social presence 180 3.34 0.76 1 4 Cognitive presence 180 3.42 0.58 1 4 Learner presence 180 2.86 0.87 1 4 Table 2 Descriptive statistics of presences

Open ended question: Describe how you would change the blended learning activities to be more valuable to you.

pieces (%) examples Teaching presence 69 I enjoyed the time in the class and it is a new window in my life. I hope my teacher could give more support to students about the discussions in class and students need more chances to express themselves. Interactions with my group mate helps me understand other people s opinions better. I hope I could improve my communication with my classmates and learn more from them. The content on SPOC is quite disorganized and most of the time I have to watch it all by myself. When we talked about one topic, it is all the time our Chinese people s ideas and perceptions. I usually preview the content before I go to class and I will repeat watching if necessary. I like using the extra materials to help me understand the topic better. I always find a good environment to study SPOC, e.g. library or computer lab, but not dormitory. Table 3 Open ended questions responses (48.3%) Social presence 23 16.1% Cognitive presence 14 10% Learner presence 37 25.9%

Numbers Classification of Teaching presence responses Examples The design of SPOC (content organization, quantity, quality, scope, time spending and tech problems, etc.) Use of teaching materials (topics, knowledge scope, video and audio materials) Tasks design (forms, quantity and quality of tasks) Face-to-face teaching design (teachers guidance and management of class) 19 In SPOC, quizzes should be put before the reading analysis. 26.8% 15 More themes- related topics should be added into the materials. More revision tasks should be added. In class, there should be more chances for students to speak in public. 21.1% 23 32.4% 14 19.7% Table 4 Classification of teaching presence responses

Survey questions on technology use are about students -confidence of using technologies in learning -benefits of using technologies in learning Tech1 Tech2 Tech3 Tech4 Tech5 SD 0.74 0.69 0.63 0.86 0.78 Mean 3.03 3.19 3.33 2.66 3.04 Table 5 Descriptive statistics for technology use

Discussions: The RCoI survey is a reliable and valid measurement for blended language learning module. The SPOC embedded blended language learning module is a successful community of inquiry. Students have a positive learning experience. Students feel rewarding in using technology in language learning, but anxiety exists.

Teaching presence needs to be more focused in future learning design, as expected by students. Social presence of online learning part needs to be improved. Learner presence shows the lack of self- regulatory among students.

Limitations and future study A post-survey interview is needed to address: What are the detailed challenges and difficulties causing their anxiety when using technology in language learning? Students have greater accessibility to handheld devices and what learning design will bridge the mobile learning with formal learning?

Thank you! Questions?