Understanding Students' Perception of Grade Boundaries in Higher Education

Research presented at the CLT Pedagogic Research Conference explores students' expectations and understanding of grade boundaries in higher education. The study reveals a disconnect between effort, time spent on assignments, and actual grades received. Many students strive for high grades without fully grasping the assessment criteria beyond memorization. This lack of understanding can impact their ability to demonstrate essential skills valued by employers. The findings suggest the need for better alignment between student expectations and assessment practices in HE.

• Higher Education
• Student Expectations
• Assessment Practices
• Grade Boundaries

kenl_388 Follow

Uploaded on Sep 15, 2024 | 0 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

Presentation Transcript

1. CLT Pedagogic Research Conference Friday 05 February 2016 Theme: Transition into and trajectories through HE How well do students understand the distinction between grade boundaries and what can they realistically expect in terms of steering to hop between boundaries? Dr Dipak K Sarker Pharmacology & Therapeutics Division School of Pharmacy & Biomolecular Sciences University of Brighton Email: d.k.sarker@brighton.ac.uk

2. CLT Pedagogic Research Conference Friday 05 February 2016 Abstract Abstract It is common to hear students say I worked really hard and spent a lot of time on this assignment, but I still only got a really low mark for it. The disappointment of this experience leads many to question the purpose of all this assessment . Often their concern to achieve high grades has been shaped by the students experience of secondary education, and especially by a widespread belief that high grades are required to get a job . However, recent evidence suggests this is not universally true (1) and potential employers look for key skills (2), such as problem-solving, numeracy and higher level communication skills. Assessment in HE aims to benchmark a student s development of these and diverse other complex attributes (3, 4). But if students fail to understand this believing instead that effort and time are the key criteria - it becomes difficult for them to demonstrate the expected attributes. The starting point for this research, therefore, is that students frequently want (or expect) to achieve a high grade (5) without being able to pinpoint what merits this (anecdotal). The research, which is work-in-progress, involves data based on surveys conducted with students at levels 6 (BSc final year) and 7 (MSc). (The surveys were carried out by level 7 professional degree students.) The data indicate that while many students were able to demonstrate they were familiar with and could recall a comprehensive knowledge base , they did not always have the ability to engage with higher-level thinking and scenario problem-solving (3, 6, 7). The session will provide opportunities to discuss the implications of this innovative work for assessment practices and student expectations in HE (4).

3. CLT Pedagogic Research Conference Friday 05 February 2016 Talk Outline Introductions Student scoring Skills Attitudes Relation to learning, teaching & assessment Context Research process & methodology Questionnaire Findings Implications and transferability General conclusions Collaborators References

4. CLT Pedagogic Research Conference Friday 05 February 2016 Introduction Introduction The session will cover the context for the study and the management of expectations, which do not always fit with the HE environment. These are typified by statements from students such as I worked really hard and spent a lot of time but all I got was a . Grade. The hypothesis for the study is: the students want higher grades (1: McClure et al. 2011) without being able to pinpoint what merits this (anecdotal). It is common to hear students say I worked really hard and spent a lot of time on this assignment, but I still only got a really low mark for it. The disappointment of this experience leads many to question the purpose of all this assessment .

5. CLT Pedagogic Research Conference Friday 05 February 2016 What students score 25% score 70%+ 50% score 60%+ 25% score 50%+ Virtually no students score <50% MPharm degree students percentages n = 119 16 60 50 40 30 Students expect to score at least 50% and view this as a poor mark 50% is good according to University marking criteria 20 10 0 2010-11 2011-12 2012-13 2013-14 2014-15 PaBS undergraduate degree students, n~400 UoB undergraduate degree students, n~6000 60 60 50 50 40 40 30 30 20 20 10 10 0 0 2010-11 2011-12 2012-13 2013-14 2014-15 2010-11 2011-12 2012-13 2013-14 2014-15

6. CLT Pedagogic Research Conference Friday 05 February 2016 Introduction: skills Knowledge, skills and attitudes Miller s Pyramid of (Clinical) Competence Knows: Fact gathering (Recall?) Knows How: Interpretation, estimation (Tool use) Does Shows: Via simulations, structured assessment (Opinion) Shows Integrated thinking Does: Workplace based assessment (Application) Knows How Integrated and adaptive thinking Knows + Knows How: tests cognition Knows Shows + Does: tests behaviour Miller, G. (1990) The Assessment of Clinical Skills/Competence/Performance. Academic Medicine, 65: s63-s67.

7. CLT Pedagogic Research Conference Friday 05 February 2016 Introduction: skills Integrated and adaptive thinking is often missing or less prominent in work from 75% of students Recall and a jigsaw-puzzle-piece approach often pre-dominate students thinking Miller, G. (1990) The Assessment of Clinical Skills/Competence/Performance. Academic Medicine, 65: s63-s67. Sarker, D.K. (2012) Comparative studies based on the form of viva voce examination, which accompanies presentation of a formatted manuscript-type thesis and its impact on M-level (level-7) and level-6 pharmacy and chemistry students, respectively. In: Transitions: Quality Adaptability and Sustainability in Times of Change, Centre for Learning and Teaching Publications, University of Brighton, UK, pp63-76

8. CLT Pedagogic Research Conference Friday 05 February 2016 Introduction: skills Look at it this way: Would you jump on a flight for a cross-channel journey if the pilot said they had merely read how to fly a plane or exclusively sat in a flight simulator rather than flown daily over a 20-year period, with a flawless safety record? Answer: no!!! Dent and Harden (2013) also talk about mastery of a subject that rests above does on the pyramid. Mastery lies at the apex of the linear gradation. Expectations need management? Dent, J.A. & Harden, R.M. (2013) A Practical Guide for Medical Teachers. Fourth Edition, Elsevier, Edinburgh, pp.

9. CLT Pedagogic Research Conference Friday 05 February 2016 Attitudes Attitudes Over the past decade the National Student (Satisfaction) Survey has revealed that students are dissatisfied with quality of feedback they receive (Higher Education Funding Council For England [HEFCE] 2009,32; Krause et al.2005), often highlighting procedural elements (such timeliness and legibility), the level of guidance offered and lack of understanding of comments as issues of particular concern. However this was not a view shared by teaching staff (Carless 2006) who generally believe that feedback is of a high quality and more comprehensive.

10. CLT Pedagogic Research Conference Friday 05 February 2016 Attitudes Hounsell (2003,1) defined feedback as any information, process or activity which affords or accelerates learning, whether by enabling students to achieve higher-quality learning outcomes than they might have otherwise attained, or by enabling them to attain these outcomes sooner or rapidly. Steering tends to be written comments on assignment front-sheets notes in the margins or end of the text and an opportunity to have a one to-one verbal feedback at a specified time shortly after the work has been assessed

11. CLT Pedagogic Research Conference Friday 05 February 2016 Relation to learning, teaching & assessment Relation to learning, teaching, assessment First part of the study As teachers and trainers, we find ourselves, as students hopefully do asking what is the point of all this grading and assessment? The process hopefully aims to give a benchmark to a valuable and much deeper process (2: Miller 1990 , 3: Chinn & Hertz 2002) than simple exam sitting, and is more appropriately about knowledge accrued, skill-set and opinion forming capability instilled in students. Second part of the study To explore what MPharm students think assessment feedback and feedforward mean in real terms and what academics feel this information is. The study also seek to find if there is a mismatch between what MPharm students think feedback and feed forward mean in real terms and what academics feel this information is.

12. CLT Pedagogic Research Conference Friday 05 February 2016 Context Table 1. Integrated Masters in Pharmacy: student achievement M-level 1st M-level 2i M-level 2ii M-level 3rd 3rd BSc (hons) Unclassified degree DipHE (yr2) CertHE (yr1) 2010-11 21.6 49.3 20.9 0.0 0.7 1.5 0.7 5.2 2011-12 26.1 55.4 14.1 0.0 1.1 0.0 0.0 3.3 2012-13 17.3 55.5 19.1 0.0 0.0 1.8 0.0 6.4 2013-14 22.8 45.7 20.5 0.0 2.4 1.6 0.0 7.1 2014-15 23.3 53.4 21.1 0.0 0.0 0.8 0.0 1.5 Requirement to proceed to GPhC professional exams

13. CLT Pedagogic Research Conference Friday 05 February 2016 Context 99.2% of students passed the MPharm at 2ii grade or above 2014-15 All Brighton MPharm candidates taking the GPhC registration exam 2014-15 76.0% passed the professional exam at the first attempt* The nearest competitor university averages were 70-73% so Brighton was better than its main competitors University degree does not seem to equate fully to professional competency * This figure is in line with the sector and averages across all modern large Schools of Pharmacy. - as judged by the GPhC professional examinations

14. CLT Pedagogic Research Conference Friday 05 February 2016 Context Context A grade-driven perspective inherited from secondary education and FE, with a perceived absolute 'need' for higher grades to get a job. Recent evidence suggests this is not universally true (4: Asonitou 2015) but that potential employers look for key skills (5: Lapi a & eulovs 2014), such as problem-solving, numeracy and higher level communication skills. Often their concern to achieve high grades has been shaped by the students experience of secondary education, and especially by a widespread belief that high grades are required to get a job . However, recent evidence suggests this is not universally true (1: Asonitou 2015 ) and potential employers look for key skills (2: Lapi a & eulovs 2014 ), such as problem- solving, numeracy and higher level communication skills. Assessment in HE Asonitou, S. (2015) to benchmark a student s development of these and diverse other complex attributes (3: Miller 1990, 4: Chinn & Hertz 2002). But if students fail to understand this believing instead that effort and time are the key criteria - it becomes difficult for them to demonstrate the expected attributes. The starting point for this research, therefore, is that students frequently want (or expect) to achieve a high grade (5:McClure et al. 2011) without being able to pinpoint what merits this (anecdotal).

15. CLT Pedagogic Research Conference Friday 05 February 2016 Research process & methodology Research process / methodology This research is on-going and anonymous, involving continuous and discrete data based on student surveys. Questionnaires were undertaken after Ethics approval on students at level 7 (Masters' level) and level 6 (BSc 3 level) by level 7 professional degree students. The research, which is work-in-progress, involves data based on surveys conducted with students at levels 4-5, then 6 (BSc final year) and 7 (MPharm/MSc). (The surveys were carried out by level 7 professional degree students.) The methodology is to apply a mix-method approach of both quantitative and qualitative analysis for the study as this offers robustness that comes from drawing on strengths of both methods The main participants are MPharm students from year 1 to 4. Explanation of the survey will be done online. 4. Participants: will be MPharm undergraduate students from year 1 to year 4. All participants are over 18 and on questionnaires will carry a statement by completing the questionnaire you are giving permission to participate in the survey. The recruitment strategy via questionnaire and to a clearly labelled drop box. 5. Confidentiality and anonymity: assured, password controlled University PC. No partial or full details/data is passed on to a third party.

16. CLT Pedagogic Research Conference Friday 05 February 2016 Questionnaire 20 questions SBA type questions with A scaled question of two types One correct answer Between Yes and no or Multiple answer options Nil and all or Like and dislike Discrete value between x and y

17. CLT Pedagogic Research Conference Friday 05 February 2016 Questionnaire examples SBA type question. Multiple answer options: How much feedback do you consider it reasonable to expect from the assessor/tutor? A few "coded" (illustrative) words A. A paragraph of text B. A detailed synopsis C. D. Multiple opportunities for discussion A scaled question. How much help in preparing an assessed piece of coursework do you expect from a university tutor? Minimal. ____________________________ Anything asked for. (Make a mark on the line where your opinion sits)

18. CLT Pedagogic Research Conference Friday 05 February 2016 Findings (The Uni) At the University % credits in higher band, award=75% 75% % credits in higher band, award=50% 25% A quarter of student know the rules How many get a 2:1, 75-85% 25% 50% Overestimation, getting a good degree is easy? How many get a 2:1, 55-65% 25% How many get a 2:1, 45-55% 12.5% First and third %, 70/40 87.5% YR3/4 students know the system First and third %, 80/50 12.5% No. firsts, 27% 25% 25% Overestimation No. firsts, 17% 37.5% 37.5% No. firsts, 7% One third underestimate What's the mean PaBS mark 65-69% 50% 50% Half overestimate What's the mean PaBS mark 58-62% 139 students No. MPharm students = 140 0 20 40 60 80 140 Percentage or value

19. CLT Pedagogic Research Conference Friday 05 February 2016 Findings (Tutors) University lecturers - what students think Underestimated, having been told a standard working week is 37.5h 39 h How many hours do academics work? 9.6 h 25% of time marking? How many hours do academics spend marking Academics and teachers are synonymous 12.5% Disappointing? Academics dislike teaching 12.5% Surprising not 100% Academics think lecturer is a researcher 75% Academics think lecturer is a teacher 37.5% Good but surprising Students think lecturer is a researcher 87.5% Students think lecturer is a teacher 75% Unsurprising 0 20 40 60 80 100 Percentage or value

20. CLT Pedagogic Research Conference Friday 05 February 2016 Findings (Feedback) Feedback Indifferent 15 days for feedback more time? 25% 15 days for feedback is sufficient 75% Unsurprising Amount of tutor feedback: a few words 0% No student! Amount of tutor feedback: many meetings 62.5% Time constraints n=140 Time constraints n=140 37.5% Amount of tutor feedback: detailed synopsis Do you get enough feedback: want more 87.5% Unsurprising Do you get enough feedback: no 12.5% Feedback expected: %/reference to more info 12.5% Feedback expected: detailed critique 37.5% Time constraints n=140 50% Feedback expected: percentage/highlights Feedback expected: grade/comment 12.5% Disappointing 25% The point of FEEDBACK is ... To pass The point of FEEDBACK is ... Boost skill 87.5% Unsurprising The point of FEEDBACK is ... Job prep 50% Insightful Unsurprising 12.5% The point of FEEDBACK is ... Uni stats, etc Cynical The point of FEEDBACK is ... Mark boosting 75% 82.3% Untrue: I'm a paying customer, therefore EXPECT 0 20 40 60 80 100 120 Surprising? Only 18% believe they are customers Percentage

21. CLT Pedagogic Research Conference Friday 05 February 2016 Findings (Expectations) The mindset and expectations Ambivalent Two thirds were positive about University learning 62.4% Enjoyment of the university "journey" About a third believe grubbing style will not hinder them in later life 31.2% Asking for help on any assessed work is good (in the long-term) More than a half think Uni means self-reliance 42.3% A student can expect to ask for any amount of help About a half think this ethically dubious! 52.7% Pre-marking is perceived as unfair 0 10 20 30 40 50 60 70 80 90 100 Percentage

22. CLT Pedagogic Research Conference Friday 05 February 2016 Implications & transferability Implications and transferability The implications of this innovative work extend to standard practices and expectations throughout HE and in the shape and form of elements of assessment (3: Chinn & Hertz 2002). The session will provide opportunities to discuss the implications of this innovative work for assessment practices and student expectations in HE (4: Chinn & Hertz 2002).

23. CLT Pedagogic Research Conference Friday 05 February 2016 General conclusions Xxx

24. CLT Pedagogic Research Conference Friday 05 February 2016 Collaborators ACKNOWLEDGEMENTS 1. Kennedy Mashoko (MPharm project student: Feedback/forward) 2. Ridwan Mebude (MPharm project student: Grade boundaries) 3. Drs George Olivier, Fiona Ponikwer, Ralitza Valtcheva-Sarker, Alison Lansley 4. Respondents to questionnaire (MPharm Yr4, Yr3; n = 8 participants)

25. CLT Pedagogic Research Conference Friday 05 February 2016 References Background references 1 Asonitou, S. (2015) Employability Skills in Higher Education and the Case of Greece. Procedia - Social & Behavioural Sciences, 175: 283-290 2 Lapi a, I. & eulovs, D. (2014) Employability and skills anticipation: competences and market demands. Procedia - Social & Behavioural Sciences, 156: 404-408. 3 Miller, G. (1990) The Assessment of Clinical Skills/Competence/Performance. Academic Medicine, 65: s63-s67. 4 Chinn, R. N. & Hertz, N.R. (2002) Alternative Approaches to Standard Setting for Licensing and Certification Examinations. Applied Measurement Education, 15: 1-14. 5 McClure, J., Meyer, L.H., Garish, J., Fischer, R., Weir, K.F. & Walkey, F.H. (2011) Students attributions for their best and worst marks: Do they relate to achievement? Contemporary Educational Psychology, 36(2): 71-81. 6 Dent, J.A. & Harden, R.M. (2013) A Practical Guide for Medical Teachers. Fourth Edition, Elsevier, Edinburgh, pp. 7 Wass, V., Van der Vieuten, C., Shatzer, J. & Jones, R. (2001) Assessment of Clinical Competence. The Lancet, 357: 945-949. Background references McClure, J., Meyer, L.H., Garish, J., Fischer, R., Weir, K.F. & Walkey, F.H. (2011) Students attributions for their best and worst marks: Do they relate to achievement? Contemporary Educational Psychology, 36(2): 71-81. Miller, G. (1990) The Assessment of Clinical Skills/Competence/Performance. Academic Medicine, 65: s63-s67. Chinn, R. N. & Hertz, N.R. (2002) Alternative Approaches to Standard Setting for Licensing and Certification Examinations. Applied Measurement Education, 15: 1-14. Asonitou, S. (2015) Employability Skills in Higher Education and the Case of Greece. Procedia - Social & Behavioural Sciences, 175: 283-290. Lapi a, I. & eulovs, D. (2014) Employability and skills anticipation: competences and market demands. Procedia - Social & Behavioural Sciences, 156: 404-408. Dent, J.A. & Harden, R.M. (2013) A Practical Guide for Medical Teachers. Fourth Edition, Elsevier, Edinburgh, pp. Wass, V., Van der Vieuten, C., Shatzer, J. & Jones, R. (2001) Assessment of Clinical Competence. The Lancet, 357: 945-949. REFERENCES: DUNCAN.N. (JUNE 2007). 'FEED-FORWARD':IMPROVING STUDENTS USE OF TUTORS' COMMENTS. ASSESSMENT &EVALUATION IN HIGHER EDUCATION. 32 (3), 271-283. KRITIKOS.V.S,WOULFE.J,SUKKAR.M.B,SAINI.B. (MAY 2011). INTERGROUP PEER ASSESSMENT IN PROBLEM-BASED LEARNING TUTORIALS FOR UNDERGRADUATE PHARMACY STUDENTS. AMERICAN JOURNAL OF PHARMACEUTICAL EDUCATION. 75 (4), 73. LI.J & DE LUCA.R. (SEPTEMBER 2012). REVIEW OF ASSESSMENT FEEDBACK. STUDIES IN HIGHER EDUCATION. 39 (2), 378-393. ROBINSON.S, POPE.D AND HOLYOAK.L. (NOVEMBER 2013). CAN WE MEET THEIR EXPECTATIONS? EXPERIENCES AND PERCEPTIONS OF FEEDBACK IN FIRST YEAR UNDERGRADUATE STUDENTS. ASSESSMENT &EVALUATION IN HIGHER EDUCATION. 38 (3), 260-272. SCOLES.J,HUXHAM.M & McARTHUR. (APRIL 2012). NO LONGER EXEMPT FROM GOOD PARCTICE: USING EXEMPLARS TO CLOSE FEEDBACK GAP FOR EXAMS. ASSESSMENT &EVALUATION IN HIGHER EDUCATION. 38 (6), 631-645. TANG.J AND HARRISON.C. (AUGUST 2011). INVESTIGATING UNIVESIRTY TUTOR PERCEPTION OF FEEDBACK: THREE TYPES OF TUTOR BELIEFS. ASSESSMENT &EVALUATION IN HIGHER EDUCATION. 36 (5), 583-604.

26. CLT Pedagogic Research Conference Friday 05 February 2016 References Anderson, J. (2004) Multiple-Choice Questions Revisited. Medical Teacher, 26 (2): 110-113. Chinn, R. N. & Hertz, N.R. (2002) Alternative Approaches to Standard Setting for Licensing and Certification Examinations. Applied Measurement Education, 15: 1-14. Dent, J.A. & Harden, R.M. (2013) A Practical Guide for Medical Teachers. Fourth Edition, Elsevier, Edinburgh, pp. Downing, S.M. (2002) Threats to the Validity of Locally Developed Multiple-Choice Tests in Medical Education: Construct irrelevant Variance and Construct Under-representation. Advances Health Science Education, 7: 235-241. Miller, G. (1990) The Assessment of Clinical Skills/Competence/Performance. Academic Medicine, 65: s63-s67. Paxton, M. (2000) A Linguistic Perspective on Multiple Choice Questioning. Assessment Evaluation Higher Education, 25(2): 109-119. Wass, V., Van der Vieuten, C., Shatzer, J. & Jones, R. (2001) Assessment of Clinical Competence. The Lancet, 357: 945- 949 Sarker, D.K. (2012) Comparative studies based on the form of viva voce examination, which accompanies presentation of a formatted manuscript-type thesis and its impact on M-level (level-7) and level-6 pharmacy and chemistry students, respectively. In: Transitions: Quality Adaptability and Sustainability in Times of Change, Centre for Learning and Teaching Publications, University of Brighton, UK, pp63-76

27. CLT Pedagogic Research Conference Friday 05 February 2016 Thanks and questions