Briefing on the November 2022 National Senior Certificate Examination
This briefing covers the Umalusi mandate, quality assurance processes, standardization, problematic questions in the Mathematics paper, recommendations, and conclusions regarding the November 2022 National Senior Certificate Examination administered by the Department of Basic Education. It discusses the regulatory framework, quality assurance framework, and areas of improvement identified during the assessment. The presentation outlines the processes undertaken in 2022 and the actions taken for compliance and improvement.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Portfolio Committee on Basic Education Briefing on the November 2022 National Senior Certificate Examination 24 January 2023
Presentation Outline 1. Umalusi mandate and regulatory framework 2. Framework for Quality Assurance of Assessment 3. The quality assurance processes undertaken in 2022 3.1 Scope of the 2022 quality assurance of assessment 3.2 Areas of improvement 3.3 Directives for compliance and improvement 3.4 Recurring areas of non-compliance 4. Standardisation and Resulting. 5. The problematic question in DBE Mathematics Paper 2 6. Recommendations 7. Conclusion 7 2
UMALUSI MANDATE AND REGULATORY FRAMEWORK 3
Umalusi mandate and regulatory framework Umalusi derives its mandate for quality assurance of assessment from the: National Qualifications Framework (NQF) Act No. 67 of 2008 Sections 27 (h) and 27 (i) and The General and Further Education and Training Quality Assurance Act as amended in 2008. Section 17 (A) which state that: (3) The Council must perform the external moderation of assessment of all assessment bodies and education institutions (4) The Council may adjust raw marks during the standardisation process and (5) The Council must, with the concurrence of the Director-General and after consultation with the relevant assessment body or education institution, approve the publication of the results of learners if the Council is satisfied that the assessment body or education institution has satisfied the conditions that warrant such an approval. 4
FRAMEWORK FOR QUALITY ASSURANCE OF ASSESSMENT 5
Framework for Quality Assurance of Assessment Quality assurance of assessment is conducted to ensure that assessment leading to the award of certificates in schools, adult education centres and technical and vocational education and training colleges is of the required standard. This is in order to ensure that the certificates issued by Umalusi are credible. This is achieved through: Moderation of examination question papers, Practical Assessment Tasks (PAT) and Common Assessment Tasks in the case of Life Orientation Monitoring and moderation of School Based Assessment (SBA) Audit of the State of Readiness to conduct examinations; Audit of appointed markers; Monitoring of the conduct, administration and management of assessment and examination processes Standardisation of marking guidelines Monitoring and verification of marking Management of concessions and examination irregularities Standardisation of assessment outcomes Approval of release of results 6
National Senior Certificate The National Senior Certificate examinations are administered by three Assessment bodies (Department of Basic Education (DBE), Independent Examinations Board Comprehensive Assessment Institute (SACAI). All three assessment bodies are quality assured and certificated by Umalusi: (IEB) and South African Overview of candidates that wrote the NSC examinations Assessment Body DBE FT DBE PT TOTAL IEB FT IEB PT TOTAL SACAI TOTAL 2020 2021 2022 610 484 115 851 726 335 12 024 1 139 13 163 2 927 2 927 733 198 163 965 897 163 12 857 752 265 168 635 920 900 12 581 968 955 13 825 3 998 3 998 13 536 4 951 4 951 7
QUALITY ASSURANCE PROCESSESS UNDERTAKEN in 2022 8
Overview of the Quality Assurance of Assessment and Examination Processes: DBE Umalusi moderated and approved 162 NSC question papers for this examination. This included back up question papers for Life Orientation, Information Technology paper 1 and Computer Applications Technology paper 1. Umalusi sampled ten NSC subjects for school-based assessment (SBA) moderation, across the departments (PED). The evidence of Practical Assessment Tasks (PAT) was moderated as part of SBA for subjects with a practical component. The DBE sstate of readiness to conduct the November 2022 NSC examinations was reported on 14 October 2022. nine provincial education Umalusi monitored a sample of 400 examination centres and 10 marking centres. 9
Overview of the Quality Assurance of Assessment and Examination Processes Umalusi participated in the marking guideline standardisation meetings of 59 subjects comprising 139 question papers for the November 2022 examinations. Umalusi approved and signed-off marking guidelines of all these question papers. Umalusi verified the marking of 37 subjects across the nine PED. DBE presented a total of 66 subjects for the standardisation of the November 2022 National Senior Certificate (NSC) examinations. All irregularities that were identified were managed in accordance with the Regulations Pertaining to the Conduct, Administration and Management of the National Senior Certificate Examinations by the Provincial Examination Irregularities Committees (PEIC) and ratified by the National Examinations Irregularities Committee (NEIC). A detailed draft report, covering all the quality assurance of assessment processes, was shared with the Department of Basic Education. 10
MODERATION OF QUESTION PAPERS 11
Moderation of Question Papers Areas of Improvement The DBE is commended for: a. Forty-two question papers being approved at first moderation. However, this is a huge decline compared to the numbers approved in the previous two years. That some remain constant positive features in this list for several years in succession is an indication of stability in those question papers. The same approaches that the successful examining panels use should be adapted for those examining panels that struggle to get approvals at first moderation; and b. The fact that not a single question paper was rejected in the first moderation. This symbolises some form of stability in the subjects that were affected in the previous two years of the examination cycle. 12
Moderation of Question Papers Directives for Compliance and Improvement The DBE is urged to: a. Design differentiated training sessions for groups of personnel involved in the development of question papers; b. Management plans and evidence of such training must be presented to Umalusi; and c. Ensure that questions are not taken verbatim from the past three years question papers. 13
Moderation of Question Papers Recurrent Non-compliance The following areas with recurrent non-compliance were noted: a. Low performance levels in the two criteria: Text selection, types, and quality of questions; and Accuracy and reliability of marking guidelines 14
MODERATION OF SBA, PAT AND ORAL ASSESSMENT 15
Scope of Moderation of SBA LIST OF SUBJECTS Business Studies Economics Geography Mathematical Literacy Mathematics Physical Sciences South African Sign Language Home Language Technical Sciences Maritime Economics Marine Sciences 16
Scope of Moderation of SBA P SCIENCES MAR ECON B STUDIES MARINE TECH SCI SCIENCE SASL HL GEOG ECON MATH M LIT PED Eastern Cape X Free State X Gauteng X X X Kwa-Zulu Natal X X X Limpopo X Mpumalanga X X X Northern Cape X North West X Western Cape X X Total 2 1 2 2 1 1 4 1 1 1 17
Moderation of SBA, PAT and oral assessment Areas of Improvement The technical aspects of the SBA have improved, as most subjects and, in some cases, the moderated schools in a whole province, submitted well organised files that contained the correct documentation. The following areas of improvement were also noted: a. The Free State, Gauteng, KwaZulu-Natal, Mpumalanga, Northern Cape and Western Cape PED provided assessment tasks that were compatible with the prescribed CAPS programme of assessment in the verified subjects, an indication that learners had been well prepared for the format of exit examinations; and b. Another improvement to note was in the presentation of marks and mark sheets of learners performance scores. Very few subjects had not provided this information. 18
Moderation of SBA, PAT and oral assessment Directives for Compliance and Improvement The DBE must ensure that: a. The PED capacitate and support the teachers teaching SASL Home Language (Eastern Cape) in subject policy prescripts; b. The PED continue to upskill the teachers in question setting and ability to measure levels of question difficulty in Mathematical Literacy (Gauteng); c. The PED strengthen internal moderation processes and upskill teachers on moderation practices to improve the implementation of the SBA process, such as in Geography (Mpumalanga) and the Marine Sciences (Western Cape); and d. The PED capacitate teachers on assessment task presentation (technical layout of assessment tasks). 19
Moderation of School-Based Assessment Recurrent Non-compliance The following areas with recurrent non-compliance were noted: a. Poor quality of internal moderation in general; b. Lack of constructive feedback and developmental plans for learners; and c. Non-adherence to CAPS and other policy requirements in SASL HL. 20
Monitoring the State of Readiness to conduct examination Areas of Improvement Umalusi noted that progress was made in addressing most of the previously issued directives for compliance and improvement. Directives for Compliance and Improvement The DBE must ensure that: a. A long-term solution to address the staff shortage at PED level is implemented; b. Major criteria as determined for storage and nodal points, is adhered to; 22
Monitoring of the State of Readiness to Conduct Examinations Directives for Compliance and Improvement The DBE must ensure that: c. Systems are in place to promote, support, and monitor the effective functionality of the School Assessment Irregularities Committees (SAICs) and District Assessment Irregularities Committees (DAICs), and to ensure that these structures execute their regulatory responsibility as required d. A national strategy to advocate examination concessions/accommodations is in place and reports on granted examination concessions/accommodations are submitted as required; and e. Concessions requirements across unaccredited centres are upheld. 23
Monitoring the State of Readiness to conduct examination Recurrent Non-compliance The following areas with recurrent non-compliance were noted: a. Persisting staff shortage at Provincial and district levels; b. Storage points which do not security norms and standards as prescribed by the DBE ; and c. Failure to submit concession applications for unaccredited centres as required by Umalusi. 24
AUDIT OF APPOINTED MARKERS 25
Audit of Appointed Markers Areas of Improvement Umalusi noted the following improvements: a. The selection and appointment of marking personnel with the required subject qualification and subject specialisation, as well as the required teaching experience at Grade 12 level across PED. b. The submission of documents for auditing purposes (Mpumalanga and Limpopo) as per Umalusi requirements. This was in comparison with the areas of non-compliance regarding lack of information regarding markers appointed to mark non language subjects in Afrikaans as indicated in 2021, qualifications and subject specialisation information captured as other, without details and Non indication of the level of subject specialisation (Limpopo and Mpumalanga). c. The implementation of enhancements, as adopted in 2021, were being carried forward by the PED, to strengthen their marking process. 26
Audit of Appointed Markers Directives for Compliance and Improvement The DBE must ensure that: a. The PED adhere to the stipulated ratio of 1:5 senior marker to markers and deputy chief marker to senior markers. b. The PED obtain authorisation from the Director-General and/or HoD for deviation from policy on appointment of markers. 27
Audit of Appointed Markers Recurrent Non-compliance The following area with recurrent non-compliance was noted: a. Non-adherence to the ratio 1:5 , senior markers to markers and deputy chief markers to senior markers. 28
MONITORING OF THE WRITING OF THE EXAMINATIONS 29
Monitoring of the Writing of the Examination Areas of Improvement The following areas of improvement were noted: a. Chief invigilators and invigilators were trained adequately in the execution of their invigilation roles and responsibilities. This addressed a directive issued in 2021 that required the DBE to mitigate inconsistencies in the execution of roles and responsibilities by invigilators; and b. Adherence by examination centres to the prescribed requirements for storage and safekeeping of question papers was noted. 30
Quality Assurance of Assessment Processes Monitoring of the Writing of the Examination Directives for Compliance and Improvement The DBE must ensure that: a. Disciplinary measures are imposed on invigilators who fail to uphold their roles and responsibilities; and b. Innovative ways are implemented to minimise examination irregularities. 31
Monitoring of the Writing of the Examinations Recurrent Non-compliance The following areas with recurrent non-compliance were noted: a. Non-adherence to general examinations procedures as provide for in regulations pertaining to the conduct, administration and management of the NSC examinations. b. Increasing cases of Behavioural Offences and Acts of Dishonesty (BOADS)that occur during writing: Possession of crib notes and cell phones in the examination rooms of which their use in the examination get to be discovered during the marking of scripts. Possible assistance by invigilators and/or fellow candidates while examination is in progress. 32
MARKING GUIDELINES STANDARDISATION 33
Marking Guidelines Standardisation Areas of Improvement The following area of improvement was noted: a. The Northern Cape (which in 2021 did not fully comply) and North West complied with the requirement to provide chief markers and internal moderators with the required 20 scripts for pre-marking for a significant number of question papers. Directives for Compliance and Improvement The DBE is required to: a. Ensure that all provincial internal moderators and chief markers meet the pre-marking requirements. 34
Marking Guidelines Standardisation Recurrent Non-compliance The following area with recurrent non-compliance was noted: a. Non-adherence to the 20 scripts pre-marking requirement in preparation for marking guideline standardisation; 35
MONITORING OF THE MARKING 36
Monitoring of the Marking of Examinations Areas of Improvement The following areas of improvement were noted: a. All marking centres were ready when marking started, with suitable resources for the conduciveness of the marking centres provided. This responded to a directive issued in 2021 that required marking centres be equipped with adequate facilities; b. Lists of all marking personnel were available and could be verified at all centres monitored, a provision made to address a directive issued in 2021 on the non-availability of evidence to verify appointed marking personnel; 37
Monitoring of Marking of Examinations Areas of Improvement The following areas of improvement were noted: c. All centres could produce Occupational Health and Safety (OHS) certificates and, although one was outdated, this was an improvement that resulted from a directive issued in 2021 for centres to provide valid OHS evidence; d. Timeous arrival of scripts and marking guidelines at all centres, except one. This improvement addressed directives issued in 2021 as there was evidence of late distribution of marking guidelines by the DBE to marking centres; and e. A vast improvement in the consistency and standard of security measures, thus addressing a directive Umalusi issued in 2019 and that was partially improved upon in 2020, where there was evidence of inconsistencies in the manner the service provider contracted to provide security at marking centres. 38
Monitoring of the Marking of Examinations Directives for Compliance and Improvement The DBE must ensure that: a. Reserve lists of appointed markers are available at all centres; b. All marking guidelines are made available before the arrival of markers; and c. Occupational Health and Safety (OHS) certificates at all centres are valid. 39
Monitoring of the Marking of the Examinations Recurrent Non-compliance The following areas with recurrent non-compliance were noted: a. While there is evidence of improvement in the timeous supply of marking guidelines to the marking centres, there are isolated incidents of late delivery of marking guidelines for the subjects whose marking guideline discussions are held towards the start of marking or coincides with the start of the marking sessions . 40
VERIFICATION OF MARKING 41
Verification of Marking Areas of Improvement The area that showed improvement in the marking process was noted as follows: a. There was marked improvement in internal moderation across various levels of moderation, as it was highlighted in 2020 as an area of non- compliance. Only sporadic incidents of shadow marking were reported. Directives for Compliance and Improvement The DBE must ensure that: a. The internal moderation of the marking of South African Sign Language Home Language is intensified. b. The PED comply with the 1:5 ratio for the appointment of senior markers to markers, deputy chief markers to senior markers, respectively, is adhered to. 42
Verification of Marking Recurrent Non-compliance The following areas with recurrent non-compliance were noted: a. Non adherence to the ratio 1:5, senior markers to markers; and deputy chief markers to senior markers; and b. Inconsistent marking and internal moderation especially in SASL HL paper 2. 43
STANDARDISATION AND RESULTING 44
What is Standardisation? A process used to eliminate the effect of factors other than the learners knowledge, abilities and aptitude on their performance. 45
Why Standardise Results? To ensure that learners are not advantaged disadvantaged by factors other than their knowledge of the subject, abilities and aptitude. To achieve and consistency of the results from one year to the next. or comparability 46
Standardisation Principles In the case of an individual candidate, the adjustment effected should not exceed half of the raw mark obtained by the candidate No If the After considering qualitative and quantitative reports, Umalusi formulates positions on each subject adjustment should exceed 10% of the historical average in either direction (upward or downward) distribution of the raw marks is below or above the historical average, the marks may be adjusted either way subject to limitations 47
Results Standardised by Umalusi per Qualification National Certificate (Vocational) General Education and Training Certificate N2-N3 Engineering Studies Department of Higher Education and Training (DHET) National Senior Certificate Senior Certificate (amended) Department of Basic Education (DBE) National Senior Certificate General Education and Training Certificate Independent Examinations Board (IEB) National Senior Certificate General Education and Training Certificate South African Comprehensive Assessment Institute (SACAI) 48
Standardisation Decisions DBE NSC Description 2018 2019 2020 2021 2022 Number of 67 67 65 67 66 instructional offerings standardised Raw marks 39 (58%) 47 (70%) 48 (74%) 35 (52%) 47 (71,2%) Adjusted (mainly 17 (26%) 13 (19%) 9 (14%) 27 (40%) 16 (24,2%) upwards) Adjusted (mainly 11 (16%) 7 (11%) 8 (12%) 5 (8%) 3 (4,5%) downwards) 49
DEALING WITH QUESTION 5.1 IN DBE MATHEMATICS PAPER 2 50
