Foundations of Evaluation - Conference Experience and Approaches

Slide Note
Embed
Share

The Foundations of Evaluation course covers diverse topics such as improvement- and accountability-oriented evaluation approaches, social agenda evaluation, and eclectic approaches. The course material includes insights on decision- and accountability-oriented studies, methods for program improvement, and informing decision-making processes. Participants are encouraged to reflect on their experiences at the American Evaluation Association conference and share their learning highlights and challenges for further discussion.


Uploaded on Sep 18, 2024 | 0 Views


Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

E N D

Presentation Transcript


  1. EVAL 6000: Foundations of Evaluation Dr. Chris L. S. Coryn Nick Saxton Fall 2014

  2. Agenda American Evaluation Association (AEA) conference experience(s) Improvement- and accountability- oriented evaluation approaches Social agenda and advocacy evaluation approaches Eclectic approaches

  3. NOTE: If we do not get through all of this material tonight we will continue next week

  4. AEA experience

  5. What was your most profound experience at the conference? What did you learn (or not learn that you expected to)? What did you like most? What did you like least?

  6. Improvement- and accountability-oriented approaches

  7. (Ideally) are expansive and cover all aspects related to merit, worth, and significance of a program or other type of evaluand Intended to inform program improvement and/or decision making (i.e., accountability) Extremely difficult to implement in practice Stufflebeam Scriven

  8. Approach 15: Decision- and accountability-oriented studies Advance organizers Relevant decision makers, decisions to be made, accountability requirements, etc.. Purposes Decision making whether formative or summative Sources of questions Stakeholders (typically limited) Questions How can the evaluand be improved? Does the evaluand meet accountability requirements?

  9. Methods Any relevant method for determining how to improve a program or provide information regarding accountability Pioneers Lee Cronbach, Daniel Stufflebeam Use considerations Generally directed toward program management and staff Strengths Scope Weaknesses Emphasis on program management and staff

  10. Approach 16: Consumer- oriented studies Advance organizers Complex conceptions of values (i.e., criteria) Purposes Typically, but not always, summative Extent to which consumers needs are met Sources of questions Relevant sources of values (see Scriven s Key Evaluation Checklist [KEC]) Questions How well does the evaluand meet consumers needs?

  11. Methods Any relevant method for determining how well consumers needs have been met Pioneers Michael Scriven Use considerations Emphasis is on instrumental use Strengths Independence Weaknesses Independence

  12. Approach 17: Accreditation and certification Advance organizers Accreditation or certification requirements Purposes Whether accreditation or certification requirements are met Sources of questions Accreditation or certification requirements Questions Have accreditation or certification requirements been met?

  13. Methods Any relevant method for determining how well consumers needs have been met Pioneers Various Use considerations Accordance to accepted standards Strengths For making informed judgments Weaknesses Emphasis on inputs and processes

  14. Social agenda and advocacy approaches

  15. Predominately aimed at increasing social justice Giving power to marginalized/disenfranchised groups Tend to favor qualitative methods and a constructivist/constructionist perspective Stress cultural pluralism, relativism, and multiple realities Only a small emphasis on determining merit, worth, or significance

  16. Approach 18: Responsive or stakeholder-centered evaluation Advance organizers Stakeholders concerns Purposes Varies widely, depending on stakeholders needs Sources of questions Stakeholders Questions What has the program achieved? How well has the program been implemented? What do experts think about the program?

  17. Methods Most often qualitative Document analysis Observations Interviews Pioneers Robert Stake Use considerations Primary stakeholders Strengths Intended to address stakeholders concerns Weaknesses External credibility

  18. Approach 19: Constructivist evaluation Advance organizers Rejects the notion that merit, worth, and significance can be objectively determined Purposes To construct stakeholders experiences Sources of questions Pluralistic values of stakeholders Questions Emergent

  19. Methods Predominately qualitative Pioneers Evon Guba, Yvonne Lincoln Use considerations Often conflicting accounts and no definitive judgments Strengths Stakeholder involvement Weaknesses Time, costs, and most importantly relativist results

  20. Approach 20: Deliberative democratic evaluation Advance organizers Democratic participation Dialogue Deliberation Purposes Democratic participation Sources of questions Stakeholders/evaluator(s) Questions What is the merit and/or worth of the program?

  21. Methods Any relevant methods for gathering evidence Pioneers Ernie House Use considerations Should represent interests of all relevant stakeholders Strengths Democratic participation Weaknesses Ambitious demands required to execute the approach

  22. Approach 21: Transformative evaluation Advance organizers Social justice and equity Purposes Recognizing situational nature of knowledge claims Sources of questions Least advantaged groups of stakeholders Questions None that can be identified in advance

  23. Methods Generally mixed-method Pioneers Donna Mertens Use considerations Inclusion of marginalized stakeholders Strengths Emphasis on social justice Weaknesses External credibility

  24. Eclectic evaluation approaches

  25. No particular philosophical or methodological orientation Draw from various evaluation concepts and methods to secure useful evaluation findings Tend to invoke the Program Evaluation Standards

  26. Approach 22: Utilization- focused evaluation Advance organizers Potential users and uses Purposes Information necessary for intended uses by intended users Sources of questions Intended users, often determined through situational analysis Questions Specific questions articulated by intended users

  27. Methods Any relevant method for addressing intended users questions Pioneers Michael Patton Use considerations Situational, and tailored to intended users intended uses Strengths Maximizes evaluation impact Weaknesses Too much user input and involvement

  28. Approach 23: Participatory evaluation Advance organizers Promoting buy-in and use Purposes Tends to be directed toward program improvement Sources of questions Intended users and other stakeholders Questions Specific questions articulated by intended users and other stakeholders

  29. Methods Any relevant method for addressing intended user and stakeholder questions Pioneers Brad Cousins Use considerations Situational, and tailored to intended users intended uses Strengths User-friendly Weaknesses Sometimes, poor technical quality

Related


More Related Content