The Evolution of Educational Testing in the United States
This article explores the history and progression of educational testing in the United States, starting from the scientific study of educational tests in the 1890s to the mass commercialization of school tests in the 1910s. It discusses the International Test Commission's virtual conferences in July 2021, highlighting key developments and challenges faced in the field. The diminishing research on testing effects in US schools is also examined, shedding light on the impact of tests on students and schools.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
The Successful Degradation of Evidence The Successful Degradation of Evidence on Educational Testing in the United States on Educational Testing in the United States Richard P. Phelps International Test Commission Virtual 12th Conference, July 2021
1890s+ Scientific study of educational tests 1890s+ Scientific study of educational tests Two Strands: Aptitude: A. Binet & T. Simon, France Achievement: J.M. Rice, E.L. Thorndike, USA International Test Commission, 12th (virtual) Conference, 2 July 2021
1910s+ Mass commercialization of school tests 1910s+ Mass commercialization of school tests Examples: Courtis Series of Tests in Arithmetic (1909) Hillegas-Thorndike Composition Scale (1912) Ayres Spelling List and Scale (1915) International Test Commission, 12th (virtual) Conference, 3 July 2021
International Test Commission, 12th (virtual) Conference, 4 July 2021
International Test Commission, 12th (virtual) Conference, 5 July 2021
International Test Commission, 12th (virtual) Conference, 6 July 2021
Dismissing testing effects literature - 1971 Since these issues affect the lives of so many, and since so much has been written about tests, one would expect to find a great deal of empirical research in this area. However, a review of the literature revealed only a few small-scale and somewhat peripheral empirical studies. -- The Effects of Tests on Students and Schools, Review of Educational Research International Test Commission, 12th (virtual) Conference, 7 July 2021
The disappearing research on testing in US schools, 1971 700 600 500 400 300 200 100 0 1942 Ross 1971 Review of Ed Research International Test Commission, 12th (virtual) Conference, 8 July 2021
Dismissing testing effects literature - 1980 Almost ten years ago, Anonymous reviewed the literature on test impact on students and schools and found that while much had been written about tests, few empirical studies were evident. Today there remains a dearth of empirical support on actual test use practices. -- Review of the Research Literature on Test Use, University research center International Test Commission, 12th (virtual) Conference, 9 July 2021
The disappearing research on testing in US schools, 1980 700 600 500 400 300 200 100 0 1942 Ross 1971 Review of Ed Research 1980 University center International Test Commission, 12th (virtual) Conference, 10 July 2021
Dismissing the testing effects literature - 1991 Gaslighting We are not opposed to testing, standards, or accountability. We just don t know enough yet to endorse this [current proposal]. Little research has been conducted on this [current proposal]. So, we need to more research first (we can do it; pay us). International Test Commission, 12th (virtual) Conference, 11 July 2021
The disappearing research on testing in US schools, 1991 700 600 500 400 300 200 100 0 1942 Ross 1971 Review of Ed Research 1980 University center 1991 National center International Test Commission, 12th (virtual) Conference, 12 July 2021
Dismissing the testing effects literature - 2003 "It is important to keep in mind the limited body of data on the subject . . . We are just getting started in terms of solid research on standards, testing and accountability. -- Brookings Institution scholar, early 2003 International Test Commission, 12th (virtual) Conference, 13 July 2021
The disappearing research on testing in US schools, 2001 700 600 500 400 300 200 100 0 1942 Ross 1971 Review of Ed Research 1980 University center 1991 National center 2001 Political advisors International Test Commission, 12th (virtual) Conference, 14 July 2021
Study 1: The Effect of Testing on Student Achievement, 1910--2010 analyzed: over 800 separate studies, and more than 1,700 separate effects International Test Commission, 12th (virtual) Conference, 15 July 2021
Study 2: Dismissive Reviews in Education Policy Research: A List The top, most influential education policy researchers in the US. Over a thousand dismissive reviews recorded Majority concern testing, evaluation, accountability International Test Commission, 12th (virtual) Conference, 16 July 2021
Professor Smith literature review: 19972003 Several types of effects of educational tests: A. Curricular and instructional reform: Good, bad, or nonexistent? B. Motivation/morale/stress/ethical behavior of teachers: Increase of decrease? C. Motivation and self-concepts of students: Up or down? D. True improvement in student learning, or just higher test scores? E. Restore public confidence or provide data for critics? International Test Commission, 12th (virtual) Conference, 17 July 2021
Number of testing effect studies, 2002 and earlier, by category Education journal Psychology journal Research/advocacy org Education association Dissertation or thesis Book Governmental agency Education magazine Higher education journal/publication Non-education association Language testing Testing in the professions journal Program evaluation journal 0 20 40 60 80 100 120 Phelps meta-analysis Smith article International Test Commission, 12th (virtual) Conference, 18 July 2021
Literature search blind spots Literature reviews are easy and quick If the research existed, I would have heard of it We can trust others dismissive reviews All research has an equal chance of being known Confirmation bias Ideological or self-interest bias International Test Commission, 12th (virtual) Conference, 19 July 2021
International Test Commission, 12th (virtual) Conference, 20 July 2021
[The Center] is a stellar center, in my view, and is doing very important work on testing and assessment. It has the leading experts in this area, and they are pursuing not only scientific problems, but problems that lead to solutions. -- 1st Director, Institute for Education Sciences, 2003 International Test Commission, 12th (virtual) Conference, 21 July 2021
Prior to IESs leadership, RCTs were rarely conducted in education and not valued among many researchers. -- Chair, Institute for Education Sciences (IES) & Harvard U economist, 2013 RCT = randomized controlled trial International Test Commission, 12th (virtual) Conference, 22 July 2021
education has not, until recently, benefitted from rigorous, scientific research. IES has made remarkable progress. The Institute is only able to convene top experts because researchers believe [IES] now promotes scientifically-grounded research. -- Stanford U economist, 2011 International Test Commission, 12th (virtual) Conference, 23 July 2021
examples of things weve learned from recent education research that are very important to improving America s schools and student achievement. [Spacing effect] [Testing effect] I could provide many more pages of example of things we know now about education that we did not know 15 years ago. -- 1st Director, Institute of Education Sciences, 2011 International Test Commission, 12th (virtual) Conference, 24 July 2021
See Yana Weinstein, Learning Scientists Blog https://www.learningscientists.org/blog/2018/8/31-1 International Test Commission, 12th (virtual) Conference, 25 July 2021
The act of testing alone, irrespective of other factors, tends to improve achievement. -- C. C. Ross, Measurement in Today s Schools, 1941 International Test Commission, 12th (virtual) Conference, 25 July 2021
Comparing US National Testing to Others Comparing US National Testing to Others USA: annual testing in 7 grades, language & math, no stakes for students Others: multi-level, multi-target grade span, full battery, medium- or high-stakes testing International Test Commission, 12th (virtual) Conference, 27 July 2021
Annual Grade Span Single level (1 test for all) Single target (i.e., scale, cut score) 2.5 subject areas every student, teacher, every year no stakes for students more student-hours more educator-hours greater facility requirements lax security almost guaranteed Multi-level Multi-target 5+ subject areas tested only every fourth year stakes for students possible fewer student-hours fewer educator-hours lower facility requirements tight security possible makes VAM possible good for monitoring & intervention VAM not possible bad for monitoring & intervention International Test Commission, 12th (virtual) Conference, 28 July 2021
Capsule Summary: History of Federal Educational Testing Policy in the USA (1890 1950) Psychologists develop and study educational testing (1920 1980) Education schools diverge from origins in Psychology (1970+) Ed school professors suppress positive research on testing (2000 2003) Ed testing becomes high profile issue; Economists interested (2001+) Economists believe Ed school professors; ignore Psych research International Test Commission, 12th (virtual) Conference, 29 July 2021
