Understanding Item Analysis for Exam Questions Study

bb jmp exam item analysis bb jmp exam item n.w
1 / 12
Embed
Share

Explore the process of item analysis for studying exam questions, focusing on difficulty and discrimination measurements to enhance training programs and evaluate trainee abilities. Discover the insights beyond exam scores through item analysis.

  • Item Analysis
  • Exam Questions
  • Training Program
  • Difficulty
  • Discrimination

Uploaded on | 0 Views


Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.

E N D

Presentation Transcript


  1. BB JMP Exam Item Analysis BB JMP Exam Item Analysis JMP Discovery Summit2023 JMP Discovery Summit2023- -US US- -30MP 30MP- -1421 1421 Kemp Wu, Yuchen Lai

  2. Objective Due to current GB/BB exams have no questions based on JMP, to use Item Analysis to study the BB JMP exam questions to develop this GB/BB exam questions. The BB JMP training program is a new training program starting in 2022. To study 3 chapters of BB JMP training program BB JMP STIPS-01 (Intermediate Statistics) ,02 (Graphical Analytics) and 03 (MSA, PCA and SPC ) Identify each question s difficulty and discrimination Validate if opportunity to modify the training materials/exam questions based on analysis result Validate the capability of the exam questions to judge the ability of trainees Item Analysis Continuous Improvement Opportunity

  3. What does Exam Scores Tell Us Through scores, we only can tell difficulty or not. Through Item Analysis, we can know more than that such as Difficulty, Discrimination etc.

  4. Item Analysis - Characteristic Curves P( ) is the probability of answering the item correctly for an ability level. The a parameter defines the steepness of the curve at its inflection point. It provides an estimate of the discriminatory power of the item. The b parameter defines the location of the inflection point on the Ability axis. It provides an estimate of the difficulty of an item. The b Parameter: Item Difficulty The b parameter, or the location of the inflection point with respect to ability, provides a measure of item difficulty. The a Parameter: Item Discrimination the a parameter, or the steepness of the curve at its inflection point, provides a measure of the discriminatory power of an item. 4 Example Example

  5. Item Analysis - Parameter Estimates Parameter Estimates Difficulty A histogram of the difficulty parameters is shown beside the difficulty estimates. Discrimination A histogram of the discrimination parameters is shown beside the discrimination estimates. N discrimination < 3 Y discrimination 3 Easy < 0 Hard 0

  6. Analysis Summary - Exam STIPS-01

  7. Item Analysis - Dual Plot Hard Dual Plot The Dual Plot report contains a plot that shows question difficulty and student ability in one plot. Difficulty and ability use a common standardized scale shown on the vertical axis. The right 3rdHistogram chart indicated the groups of students. Easy

  8. Capability Analysis of Exam STIPS- 01/02/03 STIPS-01 (Intermediate Statistics) STIPS-02 (Graphical Analytics) STIPS-03 (MSA, PCA, SPC) Hard Difficulty of questions Discrimination of question Student of ability Characteristic of exam Easy Is exam achieving expectation?

  9. Easy with Different Discrimination Questions Group 1 Easy with no discrimination Question Group 2 Easy with discrimination Question STIPS-02 Q12: What key(s) should be used to highlight the portion of Weight of 100-120 and Age= 13 ? STIPS-02 Q24: can we conclude which following statement can describe the parallel plot provided? A. B. C. D. E. Positive Correlation Negative Correlation Non-Linear Correlation Outliers detected Random Correlation (White Noise) A. B. C. D. E. Shift Key only Ctrl Key only Alt key only Shift +Alt keys Ctrl + Alt keys Easy, but if not well understand parallel plot don t know how to read the graph Easy and can find in the training material

  10. Hard with Different discrimination Questions Group 4 Hard with discrimination Question Group 3 Hard with no discrimination Question STIPS-03 Q16: which following method is NOT a Long Form method? STIPS-03 Q22: which following is the main concern when using the P/TV criteria? A. B. C. D. E. ANOVA with Interaction ANOVA without Interaction Range Method Xbar-R Method Xbar-S Method A. B. C. D. Tolerance information is NOT available How to preparing the Gauge Samples Sample size is larger P/TV can not be used to judge GRR noise against SPC control chart Involved in Box Plot Marginal Outliers E. Hard, also in training material have guide to choose the Xbar-R Method Hard, student need to concentrate in the class

  11. Summary Item Analysis Continuous Improvement Opportunity Proposal Mindset: Depends on the exam requirement to adjust the percentage of 4 groups (Customized)

  12. Take Away Learnings Item analyze help to optimize GB/BB exam questions based on JMP Item analyze helps to detect difficulty and discrimination of exam questions for instructor identify any gaps in teaching effectiveness Modify GB/BB training material based on JMP item analysis model Proliferate to internal training program such as 8D, Lean

Related


More Related Content