Technology Mediated Assessments

 
Technology Mediated Assessments:
Lesson from Open University of Tanzania
By
Prof Elifas T. Bisanda
Vice Chancellor, Open University of Tanzania
 
 
 
 
INTRODUCTION
Traditional Assessment of learners, is labour
intensive, time consuming and costly
It usually consumes lots of paper, requires
involvement of many personnel,  from setting
questions, printing, Distribution, invigilation,
collection answer scripts to the centre, organizing
marking, compiling results, and final approval of
results.
There is insecurity and prone to cheating and
examimimation leakages.
 
OUT Established an Examinations syndicate
 in 2007,
became operational in 2008 with core mandate of centrally
coordinating and overseeing the management of:
U
n
i
v
e
r
s
i
t
y
 
E
x
a
m
i
n
a
t
i
o
n
s
 
f
o
r
 
a
l
l
 
l
e
v
e
l
s
:
 
N
D
,
 
U
G
 
&
 
P
G
S
t
u
d
e
n
t
s
 
R
e
s
u
l
t
s
,
 
Q
u
a
l
i
f
i
c
a
t
i
o
n
s
 
a
n
d
 
A
w
a
r
d
s
 
T
h
e
 
O
U
T
 
E
x
a
m
i
n
a
t
i
o
n
s
 
S
y
n
d
i
c
a
t
e
 
P
h
y
s
i
c
a
l
 
S
e
c
u
r
i
t
y
 
Locked Room equipped with
:
Grilled Doors & Windows
Finger Print Device in a main door
C
C
T
V
 
C
a
m
e
r
a
s
F
i
r
e
 
E
x
t
i
n
g
u
i
s
h
e
r
s
 
O
p
e
r
a
t
e
 
u
n
d
e
r
 
o
u
t
-
o
f
-
b
o
u
n
d
s
 
s
t
a
t
u
s
 
S
t
o
r
a
g
e
 
S
e
c
u
r
i
t
y
 
&
 
I
s
o
l
a
t
i
o
n
s
 
Use of Locked Iron Safe Cabinets
Isolations
External Disconnected LAN & Use of
Standalone PCs
Limited Staff Access (Currently Only 2)
Controlled Printing Facilities
 
L
o
g
i
c
a
l
 
S
e
c
u
r
i
t
y
 
Course tutors have no control of examination
papers of their own courses. The course tutor
cannot leak exam to students as he/she 
does not
know the contents of the exam paper for his/her
course.
Systematic Random Selection of Questions
controlled by in-built Software Algorithm
Increased number of possible sets of question
papers that can be printed, hence reducing the
chance of guessing which questions may be picked
and appear in examination room.
 
 
S
e
c
u
r
i
t
y
 
i
n
 
Q
u
e
s
t
i
o
n
s
 
B
a
n
k
 
Centralized DB with Annual Updates
Each ODEX course has 180 questions & 6 KAs
Distinct question papers can be generated:
180
C
6 
(= 43,424,719,800) ODEX-AE and 
90
C
3
 (=117,480) for ODEX-
MTT with repetitions.
30 MTT Sets 
and 
60 AE Sets 
without repetitions (100% distinct sets)
 
Reuse Policy is 5 years/Change of Curriculum
 
S
e
c
u
r
i
t
y
 
i
n
 
Q
u
e
s
t
i
o
n
 
P
a
p
e
r
 
Watermarked with 
Time of Print (ToP)
PDF Security using Alphanumeric
Passwords
Online File Transfer
Password through Mobile Phones (30
Mins)
Sealed Non-Reusable 
Security
Envelopes
 
Comprehensive 
Master List
Well defined 
Sitting Plans
Students Identification:
Examination Hall Ticket (
EHT
) & Other IDs
 
Involvement of (
Half-Armed) 
Intelligence
Officers
Vetted 
Invigilators
Centrally 
Coordinated 
from DES
 
I
n
v
i
g
i
l
a
t
i
o
n
 
S
e
c
u
r
i
t
y
 
Use of System generated Packing lists (Center,
Course, Exam Category & Registered Candidates)
Use of Non-Reusable Worked Scripts Security
Envelopes
With serial recorded numbers
Signed by Chief Invigilator & 3 Other
Invigilators
 
P
a
c
k
i
n
g
 
P
r
o
c
e
s
s
 
o
f
W
o
r
k
e
d
 
A
n
s
w
e
r
 
b
o
o
k
s
A
u
t
o
m
a
t
i
o
n
 
P
r
o
c
e
s
s
 
M
I
S
s
ERIS-Examination Registration System
EDBMIS-Exam Databank Management System
ODEX-On Demand Examination System
SARIS-Student Academic Register Information System
OREX-Oral Examination System
OREX
OREX
Software
Software
 
C
u
s
t
o
m
e
r
 
C
a
r
e
 
&
 
S
u
p
p
o
r
t
 
Students with special needs
Examination Room for 
disabled students
Font Size, Type, etc
 
R
E
P
O
R
T
 
A
B
U
S
E
 
Transcript Issuance: 
Within 24 Hours
Email Response: 
Max of 2 days
User-friendly 
Students Clearance 
& 
Certificate Collection
C
A
S
H
 
F
R
E
E
 
Z
O
N
E
 
R
e
a
l
i
z
e
d
 
D
E
S
 
A
c
h
i
e
v
e
m
e
n
t
s
 
O
t
h
e
r
 
U
n
q
u
a
n
t
i
f
i
a
b
l
e
 
A
c
h
i
e
v
e
m
e
n
t
s
 
Improved Quality of exam papers including Uniformity in Layout - 
ODEX
Question Paper Design (Blueprint) for MTT (2 Hours) & AE (3 Hours)
 
 
Improved quality of invigilation and rapid response from invigilators
Provision of a master list of all expected candidates for each exam
session
Increased discipline by staff handling examinations
Increased confidentiality of examinations by both staff and students
Increased Exam Sessions (ODEX)
 
U
n
q
u
a
n
t
i
f
i
a
b
l
e
 
A
c
h
i
e
v
e
m
e
n
t
s
 
Improved work environment to academic staff
One Information Center (DES) for exam regulations by students and
staff
Improved quality customer service to students and staff.
Increased tuition fee payments by students due to controlled entry
in exams.
Early feedback to students through early return of TT scripts.
Automation of DES operations
 
U
n
q
u
a
n
t
i
f
i
a
b
l
e
 
A
c
h
i
e
v
e
m
e
n
t
s
 
I
n
c
r
e
a
s
e
d
 
G
r
a
d
u
a
t
e
s
 
R
a
t
e
 
Source
: OUT Facts &
Figures & Graduation
Book 
2009-2020
 
In 2020, Most education institutions were closed due to
Covid-19. OUT did not close as its course delivery was
already online.
However, OUT students could not assemble to write
exams. Instead, the University established an online oral
examination system – OREX.
 
O
R
A
L
 
E
X
A
M
I
N
A
T
I
O
N
 
S
Y
S
T
E
M
 
(
O
R
E
X
)
 
O
R
A
L
 
E
X
A
M
I
N
A
T
I
O
N
 
S
Y
S
T
E
M
 
(
O
R
E
X
)
 
V
i
r
t
u
a
l
 
E
x
a
m
 
H
a
l
l
 
(
i
~
R
o
o
m
)
 
R
e
a
l
i
z
e
d
 
S
t
r
e
n
g
t
h
s
 
o
f
 
O
R
E
X
 
More objective than Written Exams.
Highly Flexible, Examiners & Candidates can take OREX from anywhere
A very Good Measure of someone's Understanding of a Subject.
Purely Open & Online Assessment Model.
Comply well with Covid-19 Social Distancing Order.
Save Time and Resources.
 
Random Allocation of Examiners to maintain high level of Anonymity
Examination Timetable is UNIQUE for Examiner & Student
Opportunity for Examiner to get Instant Feedback from Students
Automated Questions & Marking Scheme.
Question Paper is generated on Exam Date & Time.
Questions are Comprehensive & Integrated.
 
R
e
a
l
i
z
e
d
 
S
t
r
e
n
g
t
h
s
 
o
f
 
O
R
E
X
 
Follow-ups Questions probe student to provide elaborative answers.
On the Spot Marking, Question Paper with inbuilt Marking Scheme.
Help student to build Confidence, Communication Skills.
Assess Creativity Aspect of the Subject, Application, Problem Solving,
Knowledge, and Understanding of the Learning Outcomes.
Improve ICT Skills <<Smart Devices & Online Collaboration Tools>>
 
R
e
a
l
i
z
e
d
 
S
t
r
e
n
g
t
h
s
 
o
f
 
O
R
E
X
 
O
R
E
X
 
S
y
s
t
e
m
 
F
u
n
c
t
i
o
n
a
l
i
t
i
e
s
 
Facilitate 
Registration
 & 
Examination
Zoom Link 
is shared through Software
Students generate 
OREX Visa Card
Pairing of CE/SE/EO and Candidate is automated and randomly determined by
Software
Question Paper 
is randomly generated from the Software
Visible on Exam Date & Time of Print (ToP)
With 2 questions: Q1 from Part1, Q2 from Part2
 
O
R
E
X
 
S
y
s
t
e
m
 
F
u
n
c
t
i
o
n
a
l
i
t
i
e
s
 
Marking Scheme 
is randomly generated from the software in
form of 
Model Answer
OREX Marking in 
instant 
instant 
before close of session
Examination Timetable 
is unique to each student and
Examiner.
Provides efficient Session allocation to both Examiner &
Candidate based on the user workload.
 
R
o
l
e
 
o
f
 
E
x
a
m
i
n
e
r
s
-
C
E
 
&
 
S
E
 
To create ZOOM Link and share to Candidate
                 (
45 Min + Unlimited Time
)
CE will ask Question 1 & SE will ask Question 2
CE & SE will mark both questions.
To record OREX session (MP4 Files) and send to HoD on daily basis.
Coursecode_RegNo_Date eg. OCP100_U18-545-1111_040620 (
Exam Appeal
)
HoD will compile & send all OREX files to DES on weekly basis. (
One Copy of MP4
Files should be kept at Dept
)
EO will observe session and write observer’s report
 
H
o
w
 
t
o
 
A
s
k
 
O
r
a
l
 
Q
u
e
s
t
i
o
n
s
 
Probe student to provide elaborative answers.
Useful sample phrases for Examiner:
Could you elaborate?
Is there anything else published about this topic?
Would like to expand your answer?
Are there any other different lines of thought on this topic?
Could you be more specific about “X”?
You previously mentioned “X”, could you elaborate your thoughts on this?
Would you like us to repeat the question?
 
P
o
s
i
t
i
v
e
 
E
x
p
e
r
i
e
n
c
e
s
 
New Experience &  Unique 
innovation
OUT can innovate and implement ideas
Applied “
Construction by Design
Helped to get staff to know each other (HQ/RCs)
 
Staff trained on core issues:
Technical or Technology Training
 by improving staff
skills on the use of web system (OREX) in real-time
environment.
 
Soft Skills Training
 through use of
audiovisual conferencing tool
(Zoom facility).
Team Training
 due to OREX
structure of CE/SE/EO.
Oral Assessment skills
.
 
P
o
s
i
t
i
v
e
 
E
x
p
e
r
i
e
n
c
e
s
 
Realtime test for IT Infrastructure Strength 
(Staff & Students login
concurrently)
A special opportunity for course tutors to 
brush up
 with their courses
Participatory Exercise 
(Almost all academic staff were engaged in
invigilation, marking.
Quick recovery from fear, panic and technophobia
Student confidence for oral interview increased
 
C
o
n
c
l
u
s
i
o
n
s
 
Technology improves the security of examinations and reduces
cheating
Technology reduces considerably, the number of people in the
entire examination process
Technology allows examinations to be availed any time (On-
demand), thereby increasing graduation rates.
Oral examinations are the future mode of assessment when
network speeds will be increased and latency reduced through
5G.
 
Thank You
For Your
Attention
Slide Note
Embed
Share

Discover how the Open University of Tanzania revolutionized traditional assessments using technology, reducing costs, enhancing security, and increasing efficiency. From the implementation of technology-mediated assessments to the secure management of exam processes, explore their innovative approach to ensuring integrity in examinations while improving the overall assessment experience.


Uploaded on Mar 13, 2024 | 0 Views


Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

E N D

Presentation Transcript


  1. Technology Mediated Assessments: Lesson from Open University of Tanzania By By Prof Prof Elifas Elifas T. Vice Chancellor, Open University of Tanzania Vice Chancellor, Open University of Tanzania T. Bisanda Bisanda

  2. INTRODUCTION Traditional Assessment of learners, is labour intensive, time consuming and costly It usually consumes lots of paper, requires involvement of many personnel, from setting questions, printing, Distribution, invigilation, collection answer scripts to the centre, organizing marking, compiling results, and final approval of results. There is insecurity and prone to cheating and examimimation leakages.

  3. The OUT Examinations Syndicate The OUT Examinations Syndicate OUT Established an Examinations syndicate in 2007, became operational in 2008 with core mandate of centrally coordinating and overseeing the management of: University Examinations for all levels: ND, UG & PG Students Results, Qualifications and Awards

  4. Physical Security Physical Security Operate under out-of- bounds status Locked Room equipped with: Grilled Doors & Windows Finger Print Device in a main door CCTV Cameras Fire Extinguishers

  5. Storage Security & Isolations Storage Security & Isolations Use of Locked Iron Safe Cabinets Isolations External Disconnected LAN & Use of Standalone PCs Limited Staff Access (Currently Only 2) Controlled Printing Facilities

  6. Logical Security Logical Security Course tutors have no control of examination papers of their own courses. The course tutor cannot leak exam to students as he/she does not know the contents of the exam paper for his/her course. Systematic Random Selection of Questions controlled by in-built Software Algorithm Increased number of possible sets of question papers that can be printed, hence reducing the chance of guessing which questions may be picked and appear in examination room.

  7. Security in Questions Bank Security in Questions Bank Centralized DB with Annual Updates Each ODEX course has 180 questions & 6 KAs Distinct question papers can be generated: 180C6 (= 43,424,719,800) ODEX-AE and 90C3 (=117,480) for ODEX- MTT with repetitions. 30 MTT Sets and 60 AE Sets without repetitions (100% distinct sets) 90 ! 90 89 x 88 x 87 x ! x 180 ! 180 179 x 178 x 177 x x 176 x 175 x 174 x ! x x ! = = = n 90 117 , 480 C = = = 180 43 , 424 , 719 800 , C = n Cr 3 ( ! 3 90 3 )! 3 2 ( 1 x 87 ) ! 6 ( ! 6 180 6 )! 6 5 4 3 2 ( 1 x 174 ) ! x ( ! r )! n r Reuse Policy is 5 years/Change of Curriculum

  8. Security in Question Paper Security in Question Paper Watermarked with Time of Print (ToP) PDF Security using Alphanumeric Passwords Online File Transfer Password through Mobile Phones (30 Mins) Sealed Non-Reusable Security Envelopes

  9. Invigilation Security Invigilation Security Comprehensive Master List Well defined Sitting Plans Students Identification: Examination Hall Ticket (EHT) & Other IDs Involvement of (Half-Armed) Intelligence Officers Vetted Invigilators Centrally Coordinated from DES

  10. Packing Process of Packing Process of Worked Answer books Worked Answer books Use of System generated Packing lists (Center, Course, Exam Category & Registered Candidates) Use of Non-Reusable Worked Scripts Security Envelopes With serial recorded numbers Signed by Chief Invigilator & 3 Other Invigilators

  11. Automation Process MISs Automation Process MISs ERIS-Examination Registration System EDBMIS-Exam Databank Management System ODEX-On Demand Examination System SARIS-Student Academic Register Information System OREX-Oral Examination System OREX Software

  12. Customer Care & Support Customer Care & Support Students with special needs Examination Room for disabled students Font Size, Type, etc REPORT ABUSE REPORT ABUSE CASH FREE ZONE CASH FREE ZONE Transcript Issuance: Within 24 Hours Email Response: Max of 2 days User-friendly Students Clearance & Certificate Collection

  13. Realized DES Achievements Realized DES Achievements

  14. Other Unquantifiable Achievements Other Unquantifiable Achievements Improved Quality of exam papers including Uniformity in Layout - ODEX Question Paper Design (Blueprint) for MTT (2 Hours) & AE (3 Hours) Option I - Higher Level Questions Only Option II - Lower Level Questions Only Description 3 Questions each from KA I, II and III. Student answer 2 questions only with each carry 50%. Description 4 Questions each from Knowledge Area I, II and III. Student answers ALL questions with each carries 25%. Description 3 Questions each from Knowledge Area I, II and III. 2 questions are lower level and 1 is higher level. Student answers ALL questions with each carries lower level carries 25% and higher level carries 50% carry 25%. Description Section A consists of 4 lower level questions each question from different knowledge area (I to VI). Each question carries 10% totaling to 40% of section A. Student answers ALL questions from this section. Section B consists of 5 higher level questions each from different knowledge area (I to VI). Student selects 3 questions out of 5 in section B. Each question carries 20% totaling to 60% of section B. Option III Both Lower Level and Higher Level Questions Description 6 Questions each from KA I, II, III, IV, V, and VI. Student answer 4 questions only with each Option I - Higher Level Questions Only Option II - Lower and Higher Level Questions with Sections A and B.

  15. Unquantifiable Achievements Unquantifiable Achievements Improved quality of invigilation and rapid response from invigilators Provision of a master list of all expected candidates for each exam session Increased discipline by staff handling examinations Increased confidentiality of examinations by both staff and students Increased Exam Sessions (ODEX)

  16. Unquantifiable Achievements Unquantifiable Achievements Improved work environment to academic staff One Information Center (DES) for exam regulations by students and staff Improved quality customer service to students and staff. Increased tuition fee payments by students due to controlled entry in exams. Early feedback to students through early return of TT scripts. Automation of DES operations

  17. Increased Graduates Rate Increased Graduates Rate Source: OUT Facts & Figures & Graduation Book 2009-2020

  18. ORAL EXAMINATION SYSTEM (OREX) ORAL EXAMINATION SYSTEM (OREX) In 2020, Most education institutions were closed due to Covid-19. OUT did not close as its course delivery was already online. However, OUT students could not assemble to write exams. Instead, the University established an online oral examination system OREX.

  19. ORAL EXAMINATION SYSTEM (OREX) ORAL EXAMINATION SYSTEM (OREX) Virtual Exam Hall (i~Room i~Room)

  20. Realized Strengths of OREX Realized Strengths of OREX More objective than Written Exams. Highly Flexible, Examiners & Candidates can take OREX from anywhere A very Good Measure of someone's Understanding of a Subject. Purely Open & Online Assessment Model. Comply well with Covid-19 Social Distancing Order. Save Time and Resources.

  21. Realized Strengths of OREX Realized Strengths of OREX Random Allocation of Examiners to maintain high level of Anonymity Examination Timetable is UNIQUE for Examiner & Student Opportunity for Examiner to get Instant Feedback from Students Automated Questions & Marking Scheme. Question Paper is generated on Exam Date & Time. Questions are Comprehensive & Integrated.

  22. Realized Strengths of OREX Realized Strengths of OREX Follow-ups Questions probe student to provide elaborative answers. On the Spot Marking, Question Paper with inbuilt Marking Scheme. Help student to build Confidence, Communication Skills. Assess Creativity Aspect of the Subject, Application, Problem Solving, Knowledge, and Understanding of the Learning Outcomes. Improve ICT Skills <<Smart Devices & Online Collaboration Tools>>

  23. OREX System Functionalities OREX System Functionalities Facilitate Registration & Examination Zoom Link is shared through Software Students generate OREX Visa Card Pairing of CE/SE/EO and Candidate is automated and randomly determined by Software Question Paper is randomly generated from the Software Visible on Exam Date & Time of Print (ToP) With 2 questions: Q1 from Part1, Q2 from Part2

  24. OREX System Functionalities OREX System Functionalities Marking Scheme is randomly generated from the software in form of Model Answer OREX Marking in instant before close of session Examination Timetable is unique to each student and Examiner. Provides efficient Session allocation to both Examiner & Candidate based on the user workload.

  25. Role of Examiners Role of Examiners- -CE & SE CE & SE To create ZOOM Link and share to Candidate (45 Min + Unlimited Time) CE will ask Question 1 & SE will ask Question 2 CE & SE will mark both questions. To record OREX session (MP4 Files) and send to HoD on daily basis. Coursecode_RegNo_Date eg. OCP100_U18-545-1111_040620 (Exam Appeal) HoD will compile & send all OREX files to DES on weekly basis. (One Copy of MP4 Files should be kept at Dept) EO will observe session and write observer s report

  26. How to Ask Oral Questions How to Ask Oral Questions Probe student to provide elaborative answers. Useful sample phrases for Examiner: Could you elaborate? Is there anything else published about this topic? Would like to expand your answer? Are there any other different lines of thought on this topic? Could you be more specific about X ? You previously mentioned X , could you elaborate your thoughts on this? Would you like us to repeat the question?

  27. Positive Experiences Positive Experiences New Experience & Unique innovation OUT can innovate and implement ideas Applied Construction by Design Helped to get staff to know each other (HQ/RCs)

  28. Positive Experiences Positive Experiences Staff trained on core issues: Technical or Technology Training by improving staff skills on the use of web system (OREX) in real-time environment. Soft Skills Training through use of audiovisual conferencing tool (Zoom facility). Team Training due to OREX structure of CE/SE/EO. Oral Assessment skills.

  29. Realtime test for IT Infrastructure Strength (Staff & Students login concurrently) A special opportunity for course tutors to brush up with their courses Participatory Exercise (Almost all academic staff were engaged in invigilation, marking. Quick recovery from fear, panic and technophobia Student confidence for oral interview increased

  30. Conclusions Conclusions Technology improves the security of examinations and reduces cheating Technology reduces considerably, the number of people in the entire examination process Technology allows examinations to be availed any time (On- demand), thereby increasing graduation rates. Oral examinations are the future mode of assessment when network speeds will be increased and latency reduced through 5G.

  31. Thank You For Your Attention

Related


More Related Content

giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#