Intelligent Tutoring Systems: Enhancing Learning through Technology
Intelligent Tutoring Systems
Jim Warren
Professor of Health Informatics
Outline
•
Dimensions of intelligent tutoring systems
(ITS)
•
Examples
–
AutoTutor
–
LISP tutor
•
Implications for future of learning and MOOCs
Some basics from Brusilovskiy
•
Student model types
–
What does it hold?
•
Scalar
estimate of expertise (‘B+ for COMPSCI 101’)
•
Overlay
of the domain topics
–
0/1 (or more granular) for each domain concept (‘loop structure,
tick; recursion, cross’)
•
Error (or ‘buggy’) model
–
Include common ways that student might misconceive (‘always
borrowing from leftmost digit in 3 or 4 digit subtraction’)
–
Is it executable?
•
Can I ‘run’ the model to estimate what a student at a certain
level might say/do?
How to acquire/update the
student model?
•
Implicit
–
Watch how they act and infer their knowledge (‘ah, he didn’t
initialise the iterator variable; he’s not very familiar with writing
loops’)
•
Explicit
–
Ask them a question that tests a concept and rate the
correctness of the answer
(note that’s not the same kind of explicit approach as asking the
student whether they think they know something)
•
Inferred
–
From domain structure: ‘he doesn’t know about variables so it’s
safe to say he won’t know about arrays’
–
From background: ‘well, she passed COMPSCI 101, so she must
know about variables’ or ‘he’s solving these problems quickly, I’ll
skip ahead’
AutoTutor
•
Design with a theory of how you will influence
the user
–
“AutoTutor adopts the educational philosophy that
students learn by actively constructing explanations
and elaborations of the material”
•
The dialog tactics are aimed at implementing the
theory
–
E.g.
pumps
(“What else?”)
–
Hints, prompts, assertions
–
Backchannel feedback (nodding at important nouns)
AutoTutor
•
https://www.youtube.com/watch?v=aPcoZPjL2G8
and Sidney D'mello and Art Graesser.
2013. AutoTutor and affective autotutor: Learning by talking with cognitively and emotionally
intelligent computers that talk back.
ACM Trans. Interact. Intell. Syst.
2, 4, Article 23 (January
2013), 39 pages.
Solving further problems
•
Latent Semantic Analysis (LSA) allows
assessment of student response
–
Good/bad and which of a set of expected
concepts are involved
•
Other refinements are addressing limitations
of the established approach
–
ATLAS:
knowledge construction dialogs
to elicit
fundamental principles
–
WHY2: engaging in qualitative reasoning on
physics concepts
LISP tutor
•
Developed way back in the early 1980s at Carnegie-
Mellon
–
A tutor for the (then at least) popular AI programming
language, LISP
–
Had an underlying rule-based system for the rules of
programming as well as how to tutor
•
Interface had 3 windows
–
User code
–
Tutor feedback
–
Goal hierarchy (as reminder of task)
LISP tutor
Example tutor feedback
Example coding rule
Thoughts on the LISP tutor:
First, the system itself needs
to be an expert
•
Productions for use of LISP language
–
Simple rules for how to use particular functions
–
Higher level rules about how to tackle programming
tasks
•
Unclear
–
to what extent it could author its own solutions
–
how the problem is expressed
•
In practice, for the tutorials the general structure
of the solution appears to be pretty well spelled
out for the system
Second, know how students go wrong
•
LISP Tutor is a great example of a system with
a strong ‘buggy rule’ component
–
A lot of the system development effort in buggy
rules: “325 production rules about planning and
writing LISP programs and 475 buggy versions of
those rules”
Third, have a model for learning
(which
drives the model for interaction)
•
Hard to find a deep educational theory here, but
dedicated to rapid feedback
•
So the system is a ‘critic’
–
Offers support only when it detects that it’s needed
(or when asked)
•
This requires synchronization to be closely following the
student’s intentions
•
Aided by the template of the intelligent editor
–
Could call it ‘mixed-initiative’
•
Student can request explanation
•
Also, system is constantly prompting student to select
options (generally only one of which is ‘correct’)
Fourth, have a lesson plan
•
A curriculum of
8 progressively
more
challenging
topics
Fifth, evaluate
•
Have a meaningful control
–
Human tutor and ‘usual’ (on your own, or in lecture)
•
Dependent measures
–
E.g. time to learn
•
They found
–
40 hours for lecture (according to poll of students)
–
26.5 hours (extrapolating drop-outs) for learning on-your-own
–
15 hours with LISP tutor
–
11.4 hours with experience human tutors
–
Performance after completing recursion model: about equal
Lastly, utterly fail to realise
your potential
•
The last sentence leave us to expect results
that are ‘nothing short of revolutionary’ once
access to computers with 1MB RAM is
commonplace
•
Where are the artificially intelligent tutors?
MOOCs (Massive Open Online Courses):
A good thing
•
Not quite sure why it didn’t all happen
immediately after LISP tutor, but… here we are
•
Reasons that this might be a
good level of global free
education service
–
E.g. it scales so well that it
can be a ‘public good’
function of a coalition of
universities
An ITS research agenda for MOOCs:
1. Agents in MOOC design
•
“Intelligent” or otherwise
–
Could offer ‘critic’ functionality on design
•
Like Fischer’s critics, have rules for good educational
material design and point out violations
•
Could apply to individual exercises or to larger schema
(e.g. too much focus on one type of presentation or
favouring one learning style)
2. Agents on MOOC delivery/analytics
•
Lots of users provides strong indication of
usage patterns
–
Potential to communicate patterns to the course
manager, tutor or design (for next offering) about
•
Little-used segments
•
Areas with poor performance (e.g. requiring many
attempts and being a point of drop-out)
–
Maybe be able to offer diagnostic critique on the
likely problem
3. Agents for delivery:
modelling the user
•
Learn who the users (i.e. student users) are
–
Background knowledge, learning style, aims
•
Individual starting point for attaining specific
competencies
•
Individualise presentation style
•
Don’t necessarily need to provide the same
syllabus to everyone if aims differ
–
Depending on what it means in the end to have
‘completed’ the MOOC
•
All the usual techniques apply
–
Ask, directly assess, infer
4. MOOCs for assessment
•
As per user model of aims
–
Maybe not the same test for everyone
•
And can we help with the ‘cheating’ problem?
–
Easier if we use speech, and maybe video
•
E.g. detect individual prosody of speech
–
Mismatch of user-modelled performance and
actual performance could be at least a cue for
human attention
5. Oh, and, actual intelligent tutors
•
Y’know, Lisp Tutor etc.
•
Maybe it could be better when adding the
social MOOC aspects to the individual tutoring
Conclusion
•
ITS model the user
–
Long-term: to guide instruction across a
curriculum
–
Medium-term: to assess learning achievement
and interests
–
Short-term: to structure dialog and provide
feedback (including back-channels)
•
They really should be able to revolutionize
learning with enough focused effort
Explore the world of Intelligent Tutoring Systems (ITS) with Jim Warren, a Professor of Health Informatics. Learn about the dimensions of ITS, examples like AutoTutor and LISP tutor, and implications for the future of learning and MOOCs. Delve into student modeling basics, methods to acquire/update student models, and AutoTutor's design philosophy focusing on active learning. Discover how AutoTutor influences users through educational dialogue tactics and its impact on learning. Dive into the realm of cognitive and emotionally intelligent computers in education with AutoTutor and Affective AutoTutor.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Intelligent Tutoring Systems Jim Warren Professor of Health Informatics
Outline Dimensions of intelligent tutoring systems (ITS) Examples AutoTutor LISP tutor Implications for future of learning and MOOCs
Some basics from Brusilovskiy Student model types What does it hold? Scalar estimate of expertise ( B+ for COMPSCI 101 ) Overlay of the domain topics 0/1 (or more granular) for each domain concept ( loop structure, tick; recursion, cross ) Error (or buggy ) model Include common ways that student might misconceive ( always borrowing from leftmost digit in 3 or 4 digit subtraction ) Is it executable? Can I run the model to estimate what a student at a certain level might say/do?
How to acquire/update the student model? Implicit Watch how they act and infer their knowledge ( ah, he didn t initialise the iterator variable; he s not very familiar with writing loops ) Explicit Ask them a question that tests a concept and rate the correctness of the answer (note that s not the same kind of explicit approach as asking the student whether they think they know something) Inferred From domain structure: he doesn t know about variables so it s safe to say he won t know about arrays From background: well, she passed COMPSCI 101, so she must know about variables or he s solving these problems quickly, I ll skip ahead
AutoTutor Design with a theory of how you will influence the user AutoTutor adopts the educational philosophy that students learn by actively constructing explanations and elaborations of the material The dialog tactics are aimed at implementing the theory E.g. pumps ( What else? ) Hints, prompts, assertions Backchannel feedback (nodding at important nouns)
AutoTutor https://www.youtube.com/watch?v=aPcoZPjL2G8 and Sidney D'mello and Art Graesser. 2013. AutoTutor and affective autotutor: Learning by talking with cognitively and emotionally intelligent computers that talk back. ACM Trans. Interact. Intell. Syst. 2, 4, Article 23 (January 2013), 39 pages.
Solving further problems Latent Semantic Analysis (LSA) allows assessment of student response Good/bad and which of a set of expected concepts are involved Other refinements are addressing limitations of the established approach ATLAS: knowledge construction dialogs to elicit fundamental principles WHY2: engaging in qualitative reasoning on physics concepts
LISP tutor Developed way back in the early 1980s at Carnegie- Mellon A tutor for the (then at least) popular AI programming language, LISP Had an underlying rule-based system for the rules of programming as well as how to tutor Interface had 3 windows User code Tutor feedback Goal hierarchy (as reminder of task)
LISP tutor Example coding rule Example tutor feedback
Thoughts on the LISP tutor: First, the system itself needs to be an expert Productions for use of LISP language Simple rules for how to use particular functions Higher level rules about how to tackle programming tasks Unclear to what extent it could author its own solutions how the problem is expressed In practice, for the tutorials the general structure of the solution appears to be pretty well spelled out for the system
Second, know how students go wrong LISP Tutor is a great example of a system with a strong buggy rule component A lot of the system development effort in buggy rules: 325 production rules about planning and writing LISP programs and 475 buggy versions of those rules
Third, have a model for learning (which drives the model for interaction) Hard to find a deep educational theory here, but dedicated to rapid feedback So the system is a critic Offers support only when it detects that it s needed (or when asked) This requires synchronization to be closely following the student s intentions Aided by the template of the intelligent editor Could call it mixed-initiative Student can request explanation Also, system is constantly prompting student to select options (generally only one of which is correct )
Fourth, have a lesson plan A curriculum of 8 progressively more challenging topics
Fifth, evaluate Have a meaningful control Human tutor and usual (on your own, or in lecture) Dependent measures E.g. time to learn They found 40 hours for lecture (according to poll of students) 26.5 hours (extrapolating drop-outs) for learning on-your-own 15 hours with LISP tutor 11.4 hours with experience human tutors Performance after completing recursion model: about equal
Lastly, utterly fail to realise your potential The last sentence leave us to expect results that are nothing short of revolutionary once access to computers with 1MB RAM is commonplace Where are the artificially intelligent tutors?
MOOCs (Massive Open Online Courses): A good thing Not quite sure why it didn t all happen immediately after LISP tutor, but here we are Reasons that this might be a good level of global free education service E.g. it scales so well that it can be a public good function of a coalition of universities
An ITS research agenda for MOOCs: 1. Agents in MOOC design Intelligent or otherwise Could offer critic functionality on design Like Fischer s critics, have rules for good educational material design and point out violations Could apply to individual exercises or to larger schema (e.g. too much focus on one type of presentation or favouring one learning style)
2. Agents on MOOC delivery/analytics Lots of users provides strong indication of usage patterns Potential to communicate patterns to the course manager, tutor or design (for next offering) about Little-used segments Areas with poor performance (e.g. requiring many attempts and being a point of drop-out) Maybe be able to offer diagnostic critique on the likely problem
3. Agents for delivery: modelling the user Learn who the users (i.e. student users) are Background knowledge, learning style, aims Individual starting point for attaining specific competencies Individualise presentation style Don t necessarily need to provide the same syllabus to everyone if aims differ Depending on what it means in the end to have completed the MOOC All the usual techniques apply Ask, directly assess, infer
4. MOOCs for assessment As per user model of aims Maybe not the same test for everyone And can we help with the cheating problem? Easier if we use speech, and maybe video E.g. detect individual prosody of speech Mismatch of user-modelled performance and actual performance could be at least a cue for human attention
5. Oh, and, actual intelligent tutors Y know, Lisp Tutor etc. Maybe it could be better when adding the social MOOC aspects to the individual tutoring
Conclusion ITS model the user Long-term: to guide instruction across a curriculum Medium-term: to assess learning achievement and interests Short-term: to structure dialog and provide feedback (including back-channels) They really should be able to revolutionize learning with enough focused effort
