Methods of Training Evaluation: Overview and Importance
Training and development are fast-growing fields globally, including in India. Evaluation plays a crucial role in understanding the effectiveness and efficiency of training programs for human resource development. This presentation covers the concept of training, evaluation strategies, methods/models of evaluation, and the importance of an evaluation framework in assessing the impact of training programs.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
WELCOME WELCOME
III SEMINAR ON METHODS OF TRAINING EVALUATION Presented by Mr. NARESH NT ID NO.PALB 3016 III PhD
Flow of Presentation Introduction Concept of Training Concept of Evaluation Training Process Evaluation Strategies Methods of Evaluation Conclusion
Training and development today are among the fastest growing professionals all the world over. India is no exception Evaluation is a robust, difficult and important arena of activity directed at collecting, analysing and interpreting information on the need for, implementation of and effectiveness and efficiency of training for human resource development. Evaluation takes place against training s wider objectives and expectations Evaluation model is of prime importance in the effort to identify an evaluation framework. The crux of an evaluation framework lies in determining, as far as it is feasible, the respective relevance of these models to the assessment of the impact of the system.
Objectives To know the concept of training and evaluation To understand the training process and evaluation strategies To know the different methods/ models of training evaluation
TRAINING It is a planned communication process for manpower development in bringing a desired changes in knowledge, skills, attitude, behaviour etc. in an individual in accordance with the job requirement and for better job performance in the interest of the organization. literature, interaction and other communication media where as relevant skills can only be acquired through practicing and professionals. The knowledge can be gained through working with the
Training consists of largely well organized opportunities for the person to acquire necessary knowledge and skill to perform better in an organization. - (Lynton and Pareek,1967) In simple words, training is the process whereby an individual s efficiency and effectiveness in the given context of a job can be maximized .
Components of Training process A. PLANNING Job Analysis Trainee Analysis Training Need Assessment Determine Training Objective B. PREPARATION Select and organise content Select training techniques, methods and aids Prepare lesson plan Plan evaluation
C. PRESENTATION Conduct training Evaluate Review and revise training for future programme D. TRAINING EVALUATION Job improvement plan Review and revise the training activities based on the suggestions received from the participants Post training contact / Follow up
Training Need individual or organisation and desired level of competency is perceived gap between the work performance of an Training need can be assessed by: i. An individual himself ii. Assessed by supervisors iii.Assessment questionnaires through structured set of
Types of Training Need Analysis Informal interviews Observations Survey method Formal interviews Group conference Reports from superiors Examination of records Advisory committees Check list Questionnaires Management requests Formal research
Simple Approach to Training Needs Assessment (SATNA) It is a simple and pragmatic approach for performing training need assessment. It also produces operative results which will create a dynamic and compelling process suitable for all problems oriented training programme
Steps of SATNA i. Problem identification ii. Job description (Proposed solution) iii. People identification iv. KAS identification v. Analysis of trainees KAS vi. Disparity identification vii. Prioritizing of discrepancies of suit o available trainers o available time o available training resources (money, material and manpower etc.)
Training Objective It is a statement of desired end product to be achieved through training programme. It should clearly spell out the type of changes or desired end product, when they will be attained and finally how will be measured to determine the level of success.
Training objective must be: well defined based on the training needs of the trainees directly related to ones job specific and precise able to define a change that is measurable able to identify and enlist criteria against which success can be measured or judged clearly specifying end results achievable in its stipulated time
Training objective must be SMART Simple Measurable Attainable Realistic Time bound Purpose of a training objective create awareness generate interest motivate learning impart knowledge change attitude generate skills
ABCD in determining the training objective Audience who are to be trained Behaviour i.e., type of change you expect in the trainees behaviour Condition i.e., under which conditions you expect this change to occur in Degree i.e., how much change you expect
Evaluation Evaluation : Latin word - valere means to be strong, valiant or determination of value . Evaluation in its broadest sense means judging the value of something . means the assessment of value or worth Assess the worth of, value of given activity.
Evaluation is defined as method for determining how far an activity has progressed and how much further it should be objectives . carried to accomplish -(Kelsey and Hearne, 1967) conscious use of communication of information to help people from sound opinions and make good decisions. (Van den Ban and Hawkins, 1996)
Training Evaluation Training evaluation not only the assessment of value , but also the collection and analysis of the information on the basis of which the assessment is made Evaluation should be done before training begins, during the course of training and after the training is over Evaluation is about the principles and methods used in assessing the effectiveness of various aspects of training
PURPOSEOFEVALUATION Assessing the aims and results Supporting the extension workers Ensuring contacts with other institutions at the same or a higher level - (Bollinger et. al, 1995)
Evaluation strategies 1. Pre training evaluation 2. Evaluating Trainee s Attitudes 3. Evaluating Participants Learning 4. Evaluation at the job level 5. Evaluation at the organizational level
1.Pre training evaluation Methods: Informal meetings and discussions with supervisors, division heads, training officers, ex participants, colleagues etc Pilot presentations Advantages: An informal approach to evaluation prior to its delivery can bring about many on the spot improvements It can help focus the training s objectives Limitations : Time consuming Generalisation may be difficult Wrong early impressions may either create feelings of false security or undermine trainer confidence
2.Evaluating Trainees Attitudes Methods: Group review Small group review Advantages: Simple and direct No preparations required Possibilities for clarification Limitations : Some individuals may not be good at articulating their views More time consuming Discussion may lack focus or may be dominated by a few
3.Evaluating Participants Learning Methods: Written tests or quiz Advantages: Cheap and easy to administer Readily scored Quickly processed Increased participants attention Limitations : Tests are time specific Performance in a test is not often a good indicator of their performance in real job situation May be threatening to participants
4.Evaluation at the job level Methods: Discussion Interview Observation Advantages: Element of personal contact Opportunity for clarification Flexible Limitations : Can be costly Labour intensive May need trained interviewers
5.Evaluation at Organizational level Methods: Comparing organizational performance before and after training on areas like results, quality standards, organizational relations and monitory valuation Advantages: Establishes links between training and organizational goals Useful in highlighting additional training needs Limitations : Difficult to evaluate indirect and intangible benefits(eg: improved interpersonal communication ) Difficult to isolate factors attributable to training only May require elaborate, time consuming and expensive studies. Danger of no tangible results
Importance of Evaluation Essential maintenance and performance of organization and Decision making component in the development, To identify needs for concentrated efforts and builds confidence among public Helps in judging the value of methods or devices used Helps the extension teaching by compelling the extension workers in deciding the objectives Helps in choosing the best tools which required for achieving a goal
Types of Training Evaluation Planning Evaluation it consists of two phases Process evaluation Terminal evaluation Impact ( Misra, 1990)
Planning evaluation In the first phase training needs of the participants are assessed through one of the following four o Performance analysis o Task analysis o Competency study o Training needs survey In the second phase , an assessment of instructional Methods is done and the most appropriate training methodology and instructional strategy is worked out.
Process Evaluation It is also known as formative evaluation. It is done when the training is going on and helps in solving the problems implementation of programme arise while According to Raabetal (1987) as reported by Misra (1990), Process evaluation is performed to detect or predict defects , the procedural design of a training activity during the implementation phase.
Terminal evaluation It is done at the end of the programme to find out to what extent the objectives and desired benefits of the programme have been achieved. The strengths and weakness of the programme as perceived by the participants are also analyzed, the results of which will be useful to make future improvements.
Impact evaluation Refers to assessing the programme impact on the job performance of the participants. It attempts to find answer to the following questions: a. What are the improvements in the performance of job as the result of training? b. What are the benefits accrued to the participants and the organisations as the result of improved job perfromance? c. What are the problems in applying the skills learned in real work situation? d. What can be done to enhance the application for newly learnt knowledge and skill in the work place?
Tools and techniques for evaluation 1. Questionnaire 2. Interview - Structured - Unstructured - Group interview 3. Projective techniques 4. Observation - Participant - Non participant 5.Critical incident techniques 6. Case study 7. Analysis of report 8. Ratings by self, peer and supervisors 9. Use of scales and checklist 10.Document analysis
Steps in Evaluation of training 1.Identify the objectives of the training programme to define the parameters of impact in terms of knowledge, skill, attitude etc. 2. Develop test and measure the identical parameters. 3. Develop necessary tools and techniques to collect data at various stages as per the objectives of evaluation. 4. Test for reliability and validity of the data collection devices. 5. Collect necessary data from the participants using the developed instrument. 6. Analyze the data and prepare the reports.
Methods of Evaluation Goal-based evaluation Goal-free evaluation Responsive evaluation Systems evaluation Professional review Quasi-legal
Goal-based evaluation begins with goals in mind and seeks to determine if those goals were achieved Goal-free evaluation does not seek to confirm or deny a pre-determined outcome or goal. Rather, it seeks to discover any benefits that result from the intervention Responsive evaluation is based on client requirements. This can present unique challenges for the evaluator The systems evaluation focuses on whether the intervention was efficient and effective Professional review evaluation uses external expert appraisal to evaluate The quasi-legal is not much frequently practiced, but is uses an actual court-of inquiry format to present evidence, take testimonials, and evaluate an intervention or product.
i. Objective /goal based a)Donald Kirkpatrick s 4 levels Reaction a measure of satisfaction (what the trainees/fellows thought and felt about the training); evaluation here focuses on the reaction of individuals to the training or other improvement intervention Learning a measure of learning (the resulting increase in knowledge or capability); evaluation here assesses what has been learned as measured with end of course tests;
Behaviour a measure of behaviour change (extent of behaviour and capability implementation/application); improvement and Results a measure of results (the effects on the institutional environment resulting from the fellows performance); evaluation here measures (at least tries to) the impact of the training on overall organizational results (in the private sector on business results).
42 Jayadeva de Silva
The Four Levels represent a sequence of ways to evaluate (training) programs. As you move from one level to the next, the process becomes more difficult and time- consuming, but it also provides more valuable information. (Kirkpatrick, 1994,) The four conditions Kirkpatrick identifies for changes to occur: Desire to change Knowledge of what to do and how to do it Work in the right climate Reward for (positive) change
Reasons for Evaluating Kirkpatrick gives three reasons why there is a need to evaluate training: 1. To justify the existence of the training department by showing how it contributes to the organizations objectives and goals. 2. To decide whether to continue or discontinue training programs. 3. To gain information on how to improve future training programs. (Kirkpatrick, 1994 )
Kirkpatricks model is useful to evaluate a) whether learners liked their instruction, b) whether they learned something from it, and c)whether it had some positive effect for the organization, its weakness is that it cannot be used to determine the cost-benefit ratio of training
i. b.) Hamblin s 5 levels Objective/goal based Level 1: Reactions Level 2: Learning Level 3: Job behaviour Level 4: Organization the effects on the organization, from participant s job to performance changes Level 5: Ultimate value the financial effects, both on the organization and the economy.
ii. Systems-based Among these the CIPP, IPO and TVS models are the best known a) CIPP (Context, Input, Process, Product) The CIPP model was developed by Daniel Stufflebeam Context evaluation which helps in planning and developing objectives Input evaluation which helps to determine the design by examining capability, resources and different strategies Process evaluation which helps to control the operations by providing on-going feedback Product evaluation which helps to judge and react to the programme attainments in terms of outputs and outcomes
1.Context evaluation Objectives : To define the training environment To identify trainer and other needs To delineate problems To formulate goals Methods : By describing the content through data gathering By specifying problems and goals
2.Input evaluation Objectives: To identify and assess alternative strategies for achieving goals or overcoming problems which are identified in content evaluation Methods : By analysing existing resources identifying possible determining their comparisons solutions feasibility to and problems making
3.Process evaluation Objectives: o To observe program operation o attempting to ascertain procedural events and activities o to identify problems in design and implementaion Method : By observing and otherwise monitoring program operation To determine if happenings occur as they were planned to occur
