State of Readiness of Assessment Bodies for November 2023 National Senior Certificate Examination
Umalusi conducts an audit to assess the readiness of assessment bodies to manage the November 2023 National Senior Certificate Examination. The preliminary report outlines the approach, findings, and upcoming activities. The audit ensures compliance with quality assurance requirements and identifies potential risks that could impact the examination's credibility. The report highlights the readiness of Department of Basic Education, Independent Examinations Board, and South African Comprehensive Assessment Institute, with a comprehensive report to be shared in October. Umalusi's mandate and regulatory framework, as per the National Qualifications Framework Act, guide the quality assurance processes.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
Presentation Transcript
STATE OF READINESS OF ASSESSMENT BODIES TO CONDUCT ADMINISTER AND MANAGE THE NOVEMBER 2023 NATIONAL SENIOR CERTIFICATE EXAMINATION AND ASSESSMENT PRELIMINARY REPORT AS AT 31 AUGUST 2023 19 SEPTEMBER 2023
Presentation Outline 1. Background 2. Umalusi mandate and regulatory framework 3. Framework for Quality Assurance of Assessment 4. Umalusi State of Readiness approach 5. Preliminary findings on the State of Readiness to conduct, administer and manage the November 2023 National Senior Certificate Examinations as at 31 August 2023 DBE IEB SACAI 6. Conclusion 2
Background Umalusi conduct an audit of the state of readiness of the assessment bodies to gauge their level of preparedness to conduct, administer and manage examinations. This undertaking is a process and not an event, which commence with the tracking of directives for compliance and improvement issued to the assessment bodies after every examination. The adopted risk management-based approach is an independent, objective, and value-add approach aiming at identifying potential risks that may compromise the credibility and integrity of the examination. Annually, Umalusi communicates its quality assurance requirements on the process to assessment bodies. To this end, the assessment bodies met all Umalusi requirements by submitting their self evaluation reports. Subsequently, a desktop analysis was conducted. This report provides the scope of work covered as at 31 August 2023 as well as Umalusi findings on the audit of the Department of Basic Education (DBE), the Independent Examinations Board (IEB) and the South African Comprehensive Assessment Institute (SACAI). Considering the management plans of the assessment bodies, it is worth-noting that there are processes to be finalised later in September and October but prior to the start of the writing of the examination. Umalusi identified all outstanding activities and will follow up and conduct the verification in September 2023. A comprehensive report on the state of readiness of the assessment bodies to conduct, administer, and manage the November 2023 NSC examination will be shared with the public during the media brief scheduled to take place on 13 October 2023. 3
Umalusi mandate and regulatory framework According to the National Qualifications Framework (NQF) Act No. 67 of 2008, as amended Umalusi must: Develop and implement policy and criteria for assessment for the qualifications on its sub-framework; with regard to quality assurance within its sub-framework: (i) develop and implement policy for quality assurance; (ii) (iii) ensure that such quality assurance as is necessary for the sub-framework is undertaken; Section 17 A of the GENFETQA Act 58 of 2001, as amended (3) The Council must, perform the external moderation of assessment of all assessment bodies and education institutions. (4) The Council may adjust raw marks during the standardisation process. ensure the integrity and credibility of quality assurance; 4
Umalusi mandate GENFETQA Act 58 of 2001, as amended, Section 17 The Council must, with the concurrence of the Director-General and after consultation with the relevant assessment body or education institution, approve the publication of the results of learners if the Council is satisfied that the assessment body or education institution has: (i) conducted the assessment free from any irregularity that may jeopardise the integrity of the assessment or its outcomes; (5) (ii) complied with the requirements prescribed by the Council for conducting assessments; (iii) applied the standards prescribed by the Council, with which a learner is required to comply with in order to obtain a certificate; and (iv) complied with every other condition determined by the Council. 5
Framework for Quality Assurance of Assessment Quality assurance of assessment is conducted to ensure that assessment leading to the award of certificates in schools, adult education centres and technical and vocational education and training colleges is of the required standard. This is in order to ensure that the certificates issued by Umalusi are credible. This is achieved through: Moderation of examination question papers, Practical Assessment Tasks (PAT) and Common Assessment Tasks in the case of Life Orientation; Monitoring and moderation of School Based Assessment (SBA); Audit of the state of readiness to conduct, administer and manage the examinations; Monitoring of the conduct, administration and management of assessment and examination processes; Monitoring and moderation of marking; Management of concessions and examination irregularities; Standardisation of assessment outcomes; Approval of the publication of the results of learners if the Council is satisfied that the assessment body has conducted the assessment free from any irregularities that might jeopardise the integrity of the assessment; Certification. 6
State of Readiness approach Desktop evaluation The assessment bodies conduct their self-evaluation and submit the reports to Umalusi for a desktop analysis. Risk analysis Documents submitted were analysed to determine the risk profile of the assessment bodies to conduct, administer and manage the NSC examination. Evidence-based audits This process was used to evaluate the submitted evidence and/or any other reports received from the assessment bodies. In addition, Umalusi may conduct independent on-site audits of the assessment bodies. 7
Summary of Findings on the State of Readiness of the Department of Basic Education (DBE) to conduct, administer and manage the November 2023 NSC examination 8
Brief key findings on level of readiness to conduct, administer and manage examination as at 31 August 2023 Status on key examination processes Hundred percent (100%) of the November 2023 NSC question papers have been moderated and approved, and all met Umalusi approval requirements. The subject structures for 2023 November NSC examination were finalised and received by Umalusi to implement for resulting. The subject entries are finalised for the full-time candidates with those of the part-time candidates at an advance stage of finalisation. The November 2023 NSC examination time-table is already available on the DBE website for all interested stakeholders to access. The candidates' registrations are in the final stage of verification and preliminary schedules are sent to schools to make final corrections. The uploads are underway after the prelim examination in September 2023 onto the mainframe with accurate candidates' registration data approved. The national examination was scheduled for Life Orientation Common Assessment Task (LO CAT) examination on 4 September 2023, and question papers were printed, packaged and stored safely in audited storage facilities before distribution to the distribution points in September. 9
Brief key findings on level of readiness to conduct, administer and manage examination as at 31August 2023 Status on key examination processes The marker selection and their appointment are underway in September 2023 in the Provincial Education Departments (PED). Umalusi will conduct markers audits in September as communicated to all PEDs in August. The PED plans to conduct the September prelim examinations are finalised with the Prelim time-table already issued to schools. Dummy examination admission letters are prepared for use as dry runs during the prelim examinations scheduled for September 2023 which aim to confirm the accuracy in the registration of full-time candidates. 10
Monitoring the State of Readiness Focus Areas FOCUS AREA Focus area 1: Management as at 31 August 2023 shortages. 2023 Finding An audit was conducted by the DBE through a survey on all PED. To- date, PED prioritised the filling of key positions and there is evidence of posts being advertised across PED. This effort is addressing the directive issued in 2022. As observed during the June 2023 NSC examinations, all PED were able to deliver successfully a credible examination based on the contingency measures put in place by the PED. The same strategy is said to be adopted again for the November 2023 NSC examination, whilst the PED are putting together a long term solution. FINDINGS Staff capacity: Directive issued in 2022: In 2022, Umalusi issued a directive for the DBE to mitigate staff 11
Monitoring the State of Readiness Focus Areas FOCUS AREA FINDINGS Focus area 2 (a): Registration of candidates as at 31August 2023 The DBE has demonstrated the effectiveness of its examination registration systems during the registration of the part-time June 2023 Senior Certificate Amended and National Senior Certificate examination candidates. The DBE was able to account for all the candidates who were registered and wrote the June 2023 SC(a) and NSC. In preparation for the conduct, administration, and management of the November 2023 NSC examination, the PED were able to use the national registration database, the South African School Administration Management System (SA-SAMS) for the capturing of the 2023 NSC candidates. The DBE promotes, encourage all PED to use the available SA-SAMS learner data. recommends and 12
Monitoring the State of Readiness Focus Areas FOCUS AREA Focus area 2 (a): Registration of candidates as at 31August 2023 FINDINGS Based on the analytical evidence gathered, eight PED used the SA-SAMS except for the Western Cape Education Department (WCED) who used the Centralised Educational Management Information System (CEMIS) to extract the learner registration and Integrated Examination Computer System (IECS). Overall, the cohort registered to write the November 2023, National Senior Certificate examinations have been captured successfully on the IECS. All candidates registered subjects were also entered successfully. The requirement for registration of candidates were in line with the prescripts of the policies and guidelines. Outstanding is the finalisation of the registration of the part- time candidates who wish to write in November 2023. The delay in finalisation of the registration was due to the extension of the closing date granted to candidates who sat for the June 2023 SC(a)examination. upload onto the 13
Monitoring the State of Readiness Focus Areas FOCUS AREA Focus area 2 (a): Registration of candidates as at 31 August 2023 REGISTRATI ON PROFILE 2023 NSC EXAMINATION CANDIDATES Full Time Part Time Grand Total FINDINGS The total registered candidate number will be finalised and confirmed by end of September 2023. The preliminary registration for the full-time candidates as at 31 August 2023 as received from the DBE is as follows: REGISTERED NOVEMBER REGISTERED NOVEMBER 2022 NSC EXAMINATION CANDIDATES 753 964 167 915 921 879 723 877 131 730 855 607 Nocandidatesfrom outside the borders of South Africa in Eswatini will be registered for the NSC examination. 14
Monitoring the State of Readiness Focus Areas FOCUS AREA Focus area 2(a) Registration of candidates as at 31 August 2023 FINDINGS Registration and processing of immigrant candidates: The adjudication of received applications is in process and will be finalised by September. PED are in constant communication with schools which could not submit all supporting documentation. All applications are managed within policy prescripts. Registration of learners with learning barriers on examination system: The Inclusive Education and Curriculum follow the policy on Screening, Identification, Assessment and Support (SIAS) in the management of learners with learning barriers. The two directorates across PED work in close collaboration with examination and assessment directorate in the registration of qualifying candidates. To this end, system administrators in the PED and National has finalised the registration of Special Needs Education (SNE) candidates. The finalisation of applications received for accommodations /concessions from schools is in progress. The final stats on granted accommodations/ concessions will be available by end of September 2023. 15
Monitoring the State of Readiness Focus Areas FOCUS AREA Focus area 2(b) Registration of examination centres as at 31 August 2023 As observed during the June 2023 SC(a)/NSC examination, PED established designated examination centres and these were well-managed through effective systems the PED put in place. It is expected that the same rigor observed in the management of designated centres will apply during the November 2023 in cases where part timers will be accommodated. FINDINGS Schools registered to conduct examinations are granted full examination centre status. There is also an arrangement to accommodate the NSC-part time candidate in these examination centres, ONLY in compelling circumstances, PED will establish designated examination centres. The DBE has systems in place to assess the suitability of examination centres for the conduct of examinations. 16
Monitoring the State of Readiness Focus Areas FOCUS AREA Focus area 2(b): Registration of examination centres as at 31 August 2023 audit of centres offering the South African Sign Language Home Language. Management of Independent Schools: PED effectively implement the Umalusi policy in the management of centres which are undergoing accreditation process. To-date, approval letters were issued to the PED which applied for examination status concession on behalf of the independent schools. FINDINGS The PED audits the examination centres through desktop evaluation. The audit of examination centres is in the process of being finalised in the Districts. Post the audit the DBE and examination directorates will verify the information received. In addition, the DBE has put in place systems for the 17
Monitoring the State of Readiness Focus Areas FOCUS AREA Focus area 2(b) : Registration of examination centres as at 31 August 2023 FINDINGS Directives issued in 2022: In 2022, Umalusi issued a directive which required the DBE to ensure that concession unaccredited centres are upheld. Umalusi successfully hosted the accreditation briefing session with assessment bodies on 18 August 2023 as a drive to standardise the implementation of the requirements. Systems and plans for quality assurance of internal assessment and related components (e.g. Orals and Practical Assessment Task) are implemented across PED. requirements across Focus area 3: School-Based Assessment Audit and verification conducted for SBA as at 31 August 2023 The DBE issued Circular E06 of 2022 for the quality assurance of school-based assessment in grade 10-12 to PED and it is currently implemented. Umalusi conducted the 1st phase of the SBA moderation between 30 June and 20 August 2023 across the nine PED. The 2nd phase of SBA moderation will be conducted in October. 18
Monitoring the State of Readiness Focus Areas FOCUS AREA Focus area 4: Printing, Packaging, and Distribution of question papers/materials as at 31 August 2023 and requirements. FINDINGS In eight PED, the printing will take place in-house, one PED will print at the Government Printing Works (GPW). Seven PED have done the in-house printing in the previous examination. The North-West will print its maiden question papers for the November 2023 NSC examination in the newly acquired precinct. Five of the printing precincts were audited through the completed self-evaluation reports of the PED and Umalusi found the said precincts compliant to the DBE set standard operating procedure In addition, Umalusi conducted an independent on-site audit of the Western Cape Education Department (WCED) and Northern Cape Education Department (NCED) printing sites and found the two PED well resourced. The WCED and NCED were compliant to the requirements set out by the DBE. It is worth noting that the measures on security are regarded as classified information and cannot be published, except to highlight that Umalusi is satisfied with the vetting of the printing sites by the South African Secret Service in addition to the security measures set out by the DBE/PED. 19
Monitoring the State of Readiness Focus Areas FOCUS AREA Focus area 4: Printing, Packaging, and Distribution of question papers/materials as at 31 August 2023 In line with the DBE standard security practice, declaration forms and conflict of interest forms have been completed by staff who are involved in the Printing, Packaging, Storage and Distribution (PPSD). FINDINGS PED entered into Service Level Agreements with service providers who provide infrastructure and technical support. These SLAs are in place and in possession of the PED. These are confidential documents in the custody of the PED. Umalusi was able to verify the signed SLAs of the WCED and NCED during the recent on-site audit. All security measures are classified as Top Secret and are enshrined in the SLA. The development of management plans for PPSD are mandatory and all PED have already submitted those to the DBE. The submission of PPSD plans is a requirement for collection of the question papers consignments. This requirement has been complied with by all nine PED. 20
Monitoring the State of Readiness Focus Areas FOCUS AREA Focus area 4: Printing, Packaging, and Distribution of question papers/materials as at 31 August 2023 examinations including the latest, the June 2023 SC(a)/NSC examination. Risk assessment was conducted on the PPSD by each of the PED and Umalusi was able to verify the identified risks reports in the WCED and NCED during its on-site audit of the PPSD in August. On the remaining PED, based on the analysis of the self-evaluation reports, it was noted that all PED met the DBE requirement to conduct the risk assessments and submit risk assessment reports to the DBE. All PED fully complied and DBE was able to compile a national examination risk register using the submitted risk assessment reports received from the PED. FINDINGS Based on the observation from the administration of the June 2023 SC(a)/NSC examination, Umalusi remains confident that the same security measures will be applied as standing requirements even for the November 2023 NSC examination. All PED complied with the submission of the PPSD plans within the August 2023 timeline set by the DBE. Umalusi observed that the same requirements were fully complied with in the previous 21
Monitoring the State of Readiness Focus Areas FOCUS AREA FINDINGS Focus area 4: Printing, Packaging, and Distribution of question papers/materials as at 31 August 2023 Umalusi noted the identified risks and all mitigating strategies and will monitor the implementation of the strategies closely during the conduct of the Oct/November examination. The national question paper collection plan has been clearly outlined by the DBE. To this end, the Life Orientation Common Assessment Task (LO CAT) were successfully collected from the DBE. Umalusi note with appreciation the level of efficiency exercised by the PED in the preparation for the LO CAT examination. As at August, all PED managed the printing, packaging and storage of LO CAT question papers well. These question papers were stored in audited storage facilities before distributed to the distribution points and examination centres in September. 22
Monitoring the State of Readiness Focus Areas FOCUS AREA Focus area 4: Printing, Packaging, and Distribution of question. papers/materials as at 31 August 2023 Available strong room Strong room space Security guards Strong room to have a double locking system Level of manager keeping the storage keys FINDINGS Status of the storage facilities: The DBE criteria for the audit of the storage points is clearly articulated in terms of conditions for approval, partial approval and non- compliance requirements. The approval of storage points is the competency of the DBE based on the standardised key indicator specifications as indicated below: Major criteria (non-negotiable) Additional criteria Location Lighting Armed response Alarm system CCTV Monitoring 23
Monitoring the State of Readiness Focus Areas FOCUS AREA Focus area 4: Printing, Packaging, and Distribution of question papers/materials as at 31 August 2023 The directive issued in 2022 required that the major criteria as determined for storage and nodal points, be adhere to. 2023 Findings In 2023 DBE approved a good number of storage points after being fully compliant to the five-point major criteria requirements when compared to the same time in 2022 as evidenced in the audit report. Based on this positive report from the DBE, Umalusi is confident that its independent verification audit on the remaining seven PED will provide the same outcome as that of the DBE. Umalusi verification audit will be conducted in September. FINDINGS In line with the DBE norms and standard on security of question papers, all storage points are required to meet the major criteria (non-negotiable) in order to receive the DBE approval status. In addition to the safety requirements, the key holder policy is mandatory. All PED implemented the policy as evidenced during the 2023 June examination. Directive issued in 2022 24
Monitoring the State of Readiness Focus Areas FOCUS AREA Focus area 5: Conduct of Examinations as at 31 August 2023 As observed from the June 2023 SC(a)/NSC examination, the PED were able to categorise the designated centres under risk levels. The centres classified as high-risk were managed by resident monitors. The DBE adopted monitoring hierarchy model as a national monitoring approach. This model promotes monitoring to be conducted across different levels of management to include national, provincial and district. Notably, PED monitoring plans intends to reflect the model. The DBE advocate a monitoring target of 70% to 100% coverage across districts in the provinces, particularly when subjects with high enrolments are written. However, Umalusi observed that the coverage monitoring during the June 2023 was far below the desired targets of 70-100% by most of the PED. FINDINGS The DBE has systems in place to ensure that the PED conduct examination in line with the prescripts of the regulation that governs the conduct, administration, and management of the NSC examination. This include the profiling of examination centres according to the level of risks which PED are required to categorise. 25
Monitoring the State of Readiness Focus Areas FOCUS AREA Focus area 5: Conduct of Examinations as at 31 August 2023 FINDINGS In addition, the integrated monitoring model which is intended to strengthen the monitoring across level of the examination system is been adhered to since its inception in 2020. Draft monitoring plans are developed and will be finalised in September. Based on the analysis conducted on the self evaluation reports, PED are in the process of finalising the monitoring plans which will address the envisaged monitoring approach. Umalusi intends to monitor the implementation model closely during the November 2023 NSC examination. Invigilator training strategy and training plans are being finalised. In all full-time centres, the educators are deemed invigilators in line with the condition of service of educators. However, the WCED adopted 50:50 invigilation approach where 50% of the invigilator cadre will comprise of community members who the PED will remunerate, and the other 50% will comprise of educators. All PED determined the training of invigilators strategies for 2023. The training plans are in place, and have been approved in the examination and assessment directorates across PED. 26
Monitoring the State of Readiness Focus Areas FOCUS AREA Focus area 5: Conduct of Examinations as at 31 August 2023 FINDINGS Strategies adopted by PED include the pre-recorded videos to be used as a structured tool with the aim to communicate the same standards in the district's trainings. The approach is based on pyramid level training where the PED will train chief invigilators appointed across districts in the province, and PED trained chief invigilators to train the invigilators in established clusters. The model is underway for implementation in September. Cluster monitoring approach is in place, this is where necessary, the provincial training teams will conduct the monitoring of invigilator trainings in line with the drawn monitoring plans. The intervention is meant to enhance training of invigilators through co-training teams as envisaged by the DBE in the improvement plan which was submitted to Umalusi. The training will be reported on as and when it happens. The training are set to commenced in September and October 2023 prior to the start of the writing of examination. 27
Monitoring the State of Readiness Focus Areas FOCUS AREA Focus area 6: Marker selection and appointment of marking personnel as at 31 August 2023 September 2023. FINDINGS Northern Cape PED: The PED finalised the selection of markers process. The recommended markers will be appointed on approval of the Head of Department. Umalusi s verification on appointed markers is underway in Western Cape PED: The PED finalised the nomination of markers. A new strategy to mitigate the shortage of markers was piloted during the June 2023 marker appointments for some subjects. This entail the appointment of marker not per paper but per subject. The strategy was implemented in the appointment of Home Language level papers and History. The process to finalise the marker appointments is in an advance stage. Umalusi s verification of appointed markers is underway in September 2023. 28
Monitoring the State of Readiness Focus Areas FOCUS AREA Focus area 6: Marker selection and appointment of marking personnel as at 31 August 2023 Mpumalanga PED: The PED finalised the selection of markers process on 13 August 2023. Mpumalanga PED reported possible marker shortages in English Home Language, and markers who are proficient to Afrikaans scripts in the following subjects: Agriculture Sciences, Economics, History and Religion Studies. To mitigate these shortages, Mpumalanga PED has planned to mark English HL Paper 1 prior to the December marking session and outsource the marking of Afrikaans scripts for Agriculture Sciences, Economics, History and Religion Studies to Gauteng. FINDINGS Eastern Cape PED: The PED finalised the selection of markers process on 25 August 2023. Umalusi s verification on appointed markers will be conducted from 15 to 22 September 2023. Free State PED: The PED will finalise the selection of markers process on 19 September 2023. Umalusi s verification on appointed markers will be conducted from 22 to 24 September 2023. Umalusi s verification on appointed markers will be conducted from 15 to 22 September 2023. 29
Monitoring the State of Readiness Focus Areas FOCUS AREA Focus area 6: Marker selection and appointment of marking personnel as at 31 August 2023 Umalusi s verification on appointed markers will be conducted from 15 to 22 September 2023. FINDINGS Gauteng PED: The PED finalised the selection of markers process on 31 August 2023. The PED has planned to re-open electronic marker application system in papers with shortages. Umalusi s verification on appointed markers will be conducted from 15 to 22 September 2023. North West PED: The PED has not as yet finalised its management plan for the recruitment, selection, appointment, and training of marking personnel. KwaZulu Natal PED: The PED conducted its marker selection process from 15 May 2023 and concluded on 26 May 2023. Umalusi has verified the PED s selection and appointment of markers on 28 July and 01 August 2023. According the marker audit conducted, there were no marker shortages. Limpopo PED: Marker selection is scheduled to be finalised by 1 September 2023. Umalusi s verification on appointed markers will be conducted from 15 to 22 September 2023. 30
Monitoring the State of Readiness Focus Areas FOCUS AREA Focus area 7: Systems for mark capturing as at 31 August 2023 FINDINGS Based on the June 2023 conduct, administration and management of the examination, Umalusi was satisfied with the level of readiness demonstrated by the DBE on the capturing of marks. The PED plans to use the same mark capturing venues since they are well resourced and compatible to the process. However, audits of sites/venues will be conducted in October. Data capturers will be sourced from the pool which was appointed during the previous two examinations. Umalusi will conduct its independent verification processes in November/December prior to the commencement of marking. 31
Monitoring the State of Readiness Focus Areas FOCUS AREA Focus area 8: Management of Examination Irregularities as at 31 August 2023 place across PEDs, annually. In addition, each of the PEDs have put in place stringent mechanisms to combat irregularities. FINDINGS The DBE has functional structures at national and PED level which deals with the examination and assessment irregularities. The national strategy to combat examination irregularities has been advocated through the pledge ceremonies which takes Based on the observation of the June examination, a significant reduction of irregularities during the June 2023 examinations suggests the effectiveness of the strategies used by the DBE/PED. The DBE complied with Umalusi reporting requirements during the June 2023 SC(a)/ NSC examination. To curb the recurring possession of cell phone irregularities, the NCED planned to conduct the pre readiness for the writing session with the candidates who arrive late for the examination, as it was discovered that most of the cell phone irregularities emanates from the late-comers. 32
Summary of Findings on the State of Readiness of the Independent Examinations Board (IEB) to conduct, administer and manage the November 2023 NSC examination 33
IEB: Brief on key findings on the level of readiness to conduct, administer, and manage examinations Status on key examination processes The IEB was able to comply with Umalusi question paper requirements. To this end as at 31 August 2023, 61% of the November 2023 NSC question papers have been externally moderated and approved. The final November 2023 NSC examination time-table is published and all stakeholder including the candidates have access to it. The registration of candidates is finalised and printing of the question papers has commenced. The training of invigilators was conducted prior to the conduct, administration and management of the June examination with a follow up training scheduled for September 2023. This is aimed at accommodating invigilators who were not able to attend the training sessions held for the June examinations. The November 2023 NSC marking centres are identified. These are the same centres the IEB used for the marking of the June 2023 and the November 2022 examinations. The appointment of markers is in progress and Umalusi audit on appointed markers will be conducted in September. 34
Monitoring the State of Readiness Focus Areas FOCUS AREA Focus area 1: Management as at 31 August 2023 office and therefore will be able to deliver all examination processes. FINDINGS The IEB has outlined its resources (both human and financial) to Umalusi. In the report to Umalusi, the IEB was able to demonstrate its stability in the running of national examinations through adequate staff compliment in the administrative and management of examinations positions at head Focus area 2: Registration of candidates and examination centres as at 31 August 2023 An overview on the number of registered candidates for the November 2023 as at 31 August 2023 in comparison to 2022 is indicated in the table below. REGISTRATION PROFILE REGISTERED NOVEMBER 2023 NSC EXAMINATION CANDIDATES REGISTERED NOVEMBER 2022 NSC EXAMINATION CANDIDATES Full-Time 14 127 12 581 Part-Time 1 060 955 Grand Total 15 187 13 536 The IEB examination centres are classified as low-risk due to them having satisfied the pre-registration criteria as an IEB school. All examination centres which will be administering the NSC examination for the first time with the IEB will be monitored closely through face-to-face and online. 35
Monitoring the State of Readiness Focus Areas FOCUS AREA Focus area 3: School-Based Assessment (SBA) as at 31 August 2023 with the schools. The IEB appointed moderators for the regional moderation of SBA, PAT and oral assessment. FINDINGS The IEB has systems in place to moderate the School-Based Assessment (SBA). Umalusi will moderate the SBA in October. The IEB has the SBA policy and guidelines in place and have shared it Focus area 4: Printing, Packaging, and Distribution of question papers/materials as at 31 August 2023 Based on the analysis conducted from the self-evaluation report, the following were noted: the printing of question papers is outsourced to an external service provider and contract has been signed to that effect. a detailed service level agreement (SLA) between the IEB and service provider is renewed annually. the IEB has in place a management plan which incorporated printing, packaging, storage and distribution of examination materials. 36
Monitoring the State of Readiness Focus Areas FOCUS AREA Focus area 4: Printing, Packaging, and Distribution of question papers/materials as at 31 August 2023 requirements. IEB has invested highly in putting in place stringent security measures at all levels of the printing of question papers, packaging, storage and the distribution. FINDINGS In line with the SLA provision, the IEB officials monitor the printing facility when printing takes place. All the processes post printing are monitored. Substantive evidence supporting the focus area was received from the IEB, and this was in line with Umalusi audit IEB complied fully with Umalusi SOR requirements by providing the required evidence in line with the key indicators. It is worth noting that the measures on security are regarded as classified information and cannot be published so as not to compromise the security of the question paper at different levels of the value chain. 37
Monitoring the State of Readiness Focus Areas FOCUS AREA Focus area 5: Conduct of Examinations as at 31 August 2023 FINDINGS Training of invigilators will be conducted in September. The IEB will randomly monitor schools invigilator training sessions. Monitoring plans will be finalised after receiving the final training plans from the schools. Appointment of monitors will be finalised in September. The training manual for the training of monitors has been developed and submitted to Umalusi. The IEB makes extensive use of audio-visual monitoring of examinations and the IEB staff monitor the conduct of examinations via video streaming. The monitors have been trained and related monitoring plans and necessary guidelines examinations were part of the training content. The risks associated with the conduct of the writing of examination were outlined and mitigating strategies were highlighted. on the administration of 38
Monitoring the State of Readiness Focus Areas FOCUS AREA Focus area 6: Marker selection and appointment of marking personnel as at 31 August 2023 FINDINGS Appointment of markers is at an advanced stage and still within the IEB management plan. Umalusi has put processes in place to audit the selected and appointed markers processes. The audit of markers is underway in September/October 2023. The IEB has the policy and criteria for the selection and appointment of markers in place. The systems for mark capturing are in place. The IEB was able to provide accurate marks for the June 2023 NSC examination standardisation. Umalusi dry run testing in preparation for the standardisation of the examination is underway in December 2023. Focus area 7: Systems for mark capturing as at 31 August 2023 to Umalusi for November 2023 NSC 39
Monitoring the State of Readiness Focus Areas FOCUS AREA FINDINGS Focus area 8: Management of Examination Irregularities as at 31 August 2023 The IEB has a system in place to manage the examinations irregularities. The policy and regulations pertaining to the conduct, administration and management of the NSC informs the IEB process and procedures in dealing with examinations irregularities. An Examination Irregularity Committee (EIC) is a structure established to deal with the irregularities and is fully functional. All the registered candidates receive an examination instruction booklet wherein the penalty clause clearly articulates the examination irregularities and the sanctions imposed in cases of said contraventions. In addition, the IEB has incorporated within the invigilator and monitor training manual strategies to manage irregularities and protocol to be followed in reporting examination and assessment irregularities. Umalusi observed that the IEB has effective systems in place to combat irregularities. The use of SMART-LOCK key to seal the question papers and scripts together with the real-time monitoring through the online system are some of the measures the IEB has put in place to safe-guard the credibility and integrity of the examination. The June 2023 NSC examination was conducted free from irregularities. 40
Summary of Findings on the State of Readiness of the South African Comprehensive Assessment Institute (SACAI) to conduct, administer and manage the November 2023 NSC examinations 41
SACAI: Brief key findings on level of readiness to conduct, administer, and manage examinations Status on key examination processes Sixty-five percent (65%) of the November 2023 NSC question papers have been externally moderated and approved, as all met Umalusi approval requirements. The 2023 subject structures were finalised and submitted to Umalusi to quality assure and upload on the resulting mainframe. The year-end November 2023 NSC examination time-table is published and SACAI has distributed to its clients. The 2023 registration of candidates is closed, and final candidate registration data is processed to ensure that accurate candidates registration is captured. SACAI staff conducted on-site audits of newly registered examination centres during July and August 2023. List of examination centres were finalised and submitted to Umalusi for approval. SACAI management plan for the appointment of marking personnel is in place. Umalusi audit on appointed markers is underway in September. 42
Monitoring the State of Readiness Focus Areas FOCUS AREA Focus area 1: Management as at 31 August 2023 including the establishment is currently in line with their organogram and all key positions are filled. Focus area 2: Registration of candidates and examination centres as at 31 August 2023 REGISTRATION PROFILE FINDINGS SACAI has the capacity to manage the examinations and assessment, in that it has a budget that can manage to finance all examination and assessment deliverables staff salaries. The head office staff SACAI candidates are registered as part-time. Candidates registration has been finalised and below is the number of registered candidates for the November 2023 compared to the November 2022. NOVEMBER 2023 NSC EXAMINATION DATA 5 660 5 660 NOVEMBER 2022 NSC EXAMINATION DATA 4 951 4 951 Part-Time Grand Total 43
Monitoring the State of Readiness Focus Areas FOCUS AREA Focus area 2: Registration of candidates as at 31 August 2023 FINDINGS The policy on Screening, Identification, Assessment and Support (SIAS) is a regulatory framework used in the management of examination accommodations and concessions in recognition of candidates with learning barriers. The process of approval of the 2023 application for candidates seeking examination concessions and accommodations is at an advance stage. SACAI established 78 examination centres for the November 2023 NSC examination. An audit report for the June examinations was conducted and the report was submitted to Umalusi. Registration of examination centres as at 31 August 2023 44
Monitoring the State of Readiness Focus Areas FOCUS AREA FINDINGS Focus area 3: School-Based Assessment (SBA) as at 31 August 2023 SACAI SBA policy and guidelines are in place. The SACAI conducts information sharing sessions to ensure good implementation of school-based practical assessment tasks (PAT) and oral assessment across SACAI centres and schools. assessment (SBA), SACAI shared with its centres/schools the 2022 external moderation reports as part of feedback for improvement and developmental purposes. The 2023 SACAI moderation of SBA, oral assessment and PAT will be concluded in September 2023. Umalusi will moderate SACAI SBA in October 2023. 45
Monitoring the State of Readiness Focus Areas FOCUS AREA Focus area 4: Printing, Packaging, and Distribution of question papers/materials as at 31 August 2023 SACAI and the service provider. Based on the printing plan for the November 2023, preparations are at an advance stage with the printing of answer books. Outstanding to be finalised as per the printing plan are the following: Upload and printing of the Information Technology (IT) and Computer Applications Technology (CAT) question papers and full scale printing of all examination papers will be done in September with the last printing in October 2023. Umalusi will conduct verifications on the outstanding printing process in September 2023 prior to the commencement of the writing in October 2023. FINDINGS Printing is outsourced to an external service provider, who supplied the printing infrastructure competencies on the operation of the printers. The printing work is done on-site at SACAI premises. All printing work is done in a controlled environment that complies fully to the service level agreement (SLA) between and technical 46
Monitoring the State of Readiness Focus Areas FOCUS AREA FINDINGS Focus area 4: Printing, Packaging and Distribution of question papers/materials as at 31 August 2023 As observed from the preparation and delivery of the June 2023 NSC examination, the printing and packaging of question papers were successfully managed. SACAI has invested highly in the security of question papers, and these measures cut across at all levels of the value chain. PACKAGING AND DISTRIBUTION A SLA with another service provider for distribution and collection of question papers and answer scripts is in place. An integrated management plan for the packaging and distribution of question papers is in place. 47
Monitoring the State of Readiness Focus Areas FOCUS AREA Focus area 4: Printing, Packaging and Distribution of question papers/materials as at 31 August 2023 FINDINGS SACAI has in place a Standard Operation Procedure (SOP) for Printing, Packaging, and Distribution of question papers. The assessment body has clearly defined and stipulated measures for security in the distribution of examination question papers. It is worth noting that the measures on security are regarded as classified information and cannot be published. 48
Monitoring the State of Readiness Focus Areas FOCUS AREA Focus area 5: Conduct of Examinations as at 31 August 2023 Training of chief invigilators and invigilators was conducted. SACAI examination centres are categorised according to their risk profile and all newly registered examination centres are profiled as high-risk examination centres. Resident monitors will be deployed to high-risk centres. Risk management plan and mitigation strategies are compiled. FINDINGS Management plans for the monitoring of the conduct of examination are finalised. Monitors are appointed but not yet trained. 49
Monitoring the State of Readiness Focus Areas FOCUS AREA Focus area 6: Marker selection and appointment of marking personnel as at 31 August 2023 FINDINGS SACAI has the policy and criteria for the selection and appointment of markers in place. The selection and appointment of markers process for the marking of November 2023 NSC examinations is in progress. The systems for mark capturing are in place and provided accurate marks as observed during the June 2023 NSC examination. Umalusi dry run testing on SACAI systems is underway in December 2023. Focus area 7: Systems for mark capturing as at 31 August 2023 50
