Foundations of Evaluation - Conference Experience and Approaches
The Foundations of Evaluation course covers diverse topics such as improvement- and accountability-oriented evaluation approaches, social agenda evaluation, and eclectic approaches. The course material includes insights on decision- and accountability-oriented studies, methods for program improvement, and informing decision-making processes. Participants are encouraged to reflect on their experiences at the American Evaluation Association conference and share their learning highlights and challenges for further discussion.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
EVAL 6000: Foundations of Evaluation Dr. Chris L. S. Coryn Nick Saxton Fall 2014
Agenda American Evaluation Association (AEA) conference experience(s) Improvement- and accountability- oriented evaluation approaches Social agenda and advocacy evaluation approaches Eclectic approaches
NOTE: If we do not get through all of this material tonight we will continue next week
What was your most profound experience at the conference? What did you learn (or not learn that you expected to)? What did you like most? What did you like least?
Improvement- and accountability-oriented approaches
(Ideally) are expansive and cover all aspects related to merit, worth, and significance of a program or other type of evaluand Intended to inform program improvement and/or decision making (i.e., accountability) Extremely difficult to implement in practice Stufflebeam Scriven
Approach 15: Decision- and accountability-oriented studies Advance organizers Relevant decision makers, decisions to be made, accountability requirements, etc.. Purposes Decision making whether formative or summative Sources of questions Stakeholders (typically limited) Questions How can the evaluand be improved? Does the evaluand meet accountability requirements?
Methods Any relevant method for determining how to improve a program or provide information regarding accountability Pioneers Lee Cronbach, Daniel Stufflebeam Use considerations Generally directed toward program management and staff Strengths Scope Weaknesses Emphasis on program management and staff
Approach 16: Consumer- oriented studies Advance organizers Complex conceptions of values (i.e., criteria) Purposes Typically, but not always, summative Extent to which consumers needs are met Sources of questions Relevant sources of values (see Scriven s Key Evaluation Checklist [KEC]) Questions How well does the evaluand meet consumers needs?
Methods Any relevant method for determining how well consumers needs have been met Pioneers Michael Scriven Use considerations Emphasis is on instrumental use Strengths Independence Weaknesses Independence
Approach 17: Accreditation and certification Advance organizers Accreditation or certification requirements Purposes Whether accreditation or certification requirements are met Sources of questions Accreditation or certification requirements Questions Have accreditation or certification requirements been met?
Methods Any relevant method for determining how well consumers needs have been met Pioneers Various Use considerations Accordance to accepted standards Strengths For making informed judgments Weaknesses Emphasis on inputs and processes
Social agenda and advocacy approaches
Predominately aimed at increasing social justice Giving power to marginalized/disenfranchised groups Tend to favor qualitative methods and a constructivist/constructionist perspective Stress cultural pluralism, relativism, and multiple realities Only a small emphasis on determining merit, worth, or significance
Approach 18: Responsive or stakeholder-centered evaluation Advance organizers Stakeholders concerns Purposes Varies widely, depending on stakeholders needs Sources of questions Stakeholders Questions What has the program achieved? How well has the program been implemented? What do experts think about the program?
Methods Most often qualitative Document analysis Observations Interviews Pioneers Robert Stake Use considerations Primary stakeholders Strengths Intended to address stakeholders concerns Weaknesses External credibility
Approach 19: Constructivist evaluation Advance organizers Rejects the notion that merit, worth, and significance can be objectively determined Purposes To construct stakeholders experiences Sources of questions Pluralistic values of stakeholders Questions Emergent
Methods Predominately qualitative Pioneers Evon Guba, Yvonne Lincoln Use considerations Often conflicting accounts and no definitive judgments Strengths Stakeholder involvement Weaknesses Time, costs, and most importantly relativist results
Approach 20: Deliberative democratic evaluation Advance organizers Democratic participation Dialogue Deliberation Purposes Democratic participation Sources of questions Stakeholders/evaluator(s) Questions What is the merit and/or worth of the program?
Methods Any relevant methods for gathering evidence Pioneers Ernie House Use considerations Should represent interests of all relevant stakeholders Strengths Democratic participation Weaknesses Ambitious demands required to execute the approach
Approach 21: Transformative evaluation Advance organizers Social justice and equity Purposes Recognizing situational nature of knowledge claims Sources of questions Least advantaged groups of stakeholders Questions None that can be identified in advance
Methods Generally mixed-method Pioneers Donna Mertens Use considerations Inclusion of marginalized stakeholders Strengths Emphasis on social justice Weaknesses External credibility
Eclectic evaluation approaches
No particular philosophical or methodological orientation Draw from various evaluation concepts and methods to secure useful evaluation findings Tend to invoke the Program Evaluation Standards
Approach 22: Utilization- focused evaluation Advance organizers Potential users and uses Purposes Information necessary for intended uses by intended users Sources of questions Intended users, often determined through situational analysis Questions Specific questions articulated by intended users
Methods Any relevant method for addressing intended users questions Pioneers Michael Patton Use considerations Situational, and tailored to intended users intended uses Strengths Maximizes evaluation impact Weaknesses Too much user input and involvement
Approach 23: Participatory evaluation Advance organizers Promoting buy-in and use Purposes Tends to be directed toward program improvement Sources of questions Intended users and other stakeholders Questions Specific questions articulated by intended users and other stakeholders
Methods Any relevant method for addressing intended user and stakeholder questions Pioneers Brad Cousins Use considerations Situational, and tailored to intended users intended uses Strengths User-friendly Weaknesses Sometimes, poor technical quality
