Full Description
This open access book describes and reviews the development of the quality control mechanisms and methodologies associated with IEA's extensive program of educational research. A group of renowned international researchers, directly involved in the design and execution of IEA's international large-scale assessments (ILSAs), describe the operational and quality control procedures that are employed to address the challenges associated with providing high-quality, comparable data.
Throughout the now considerable history of IEA's international large-scale assessments, establishing the quality of the data has been paramount. Research in the complex multinational context in which IEA studies operate imposes significant burdens and challenges in terms of the methodologies and technologies that have been developed to achieve the stated study goals. The demands of the twin imperatives of validity and reliability must be satisfied in the context of multiple and diverse cultures, languages, orthographies, educational structures, educational histories, and traditions. Readers will learn about IEA's approach to such challenges, and the methods used to ensure that the quality of the data provided to policymakers and researchers can be trusted. An often neglected area of investigation, namely the consequential validity of ILSAs, is also explored, examining issues related to reporting, dissemination, and impact, including discussion of the limits of interpretation.
The final chapters address the question of the influence of ILSAs on policy and reform in education, including a case study from Singapore, a country known for its outstanding levels of achievement, but which nevertheless seeks the means of continual improvement, illustrating best practice use of ILSA data.
Contents
Foreword; Dirk Hastedt.- 1 Introduction to Reliability and Validity of International Large-Scale Assessment; Hans Wagemaker.- 2 Study Design and Evolution, and the Imperatives of Reliability and Validity; Hans Wagemaker.- 3 Framework Development in International Large-Scale Assessment Studies; John Ainley and Wolfram Schulz.- 4 Assessment Content Development; Liz Twist and Julian Fraillon.- 5 Questionnaire Development in International Large-Scale Assessment Studies; Wolfram Schulz and Ralph Carstens.- 6 Translation: The Preparation of National Language Versions of Assessment Instruments; Paula Korsnakova, Steve Dept, and David Ebbs.- 7 Sampling, Weighting, and Variance Estimation; Sabine Meinck.- 8 Quality Control during Data Collection: Refining for Rigor; Lauren Musu, Sandra Dohr, and Andrea Netten.- 9 Post-collection Data Capture, Processing, and Scoring; Alena Becker.- 10 Technology and Assessment; Heiko Sibberns.- 11 Ensuring Validity in International Comparisons Using State-of-the-art Psychometric Methodologies; Matthias von Davier, Eugene Gonzalez, and Wolfram Schulz.- 12 Publishing and Dissemination; Seamus Hegarty and Sive Finlay.- 13 Consequential Validity: Data Access, Data Use, Analytical Support, and Training; Sabine Meinck, Eugene Gonzalez, and Hans Wagemaker.- 14 Using IEA Studies to Inform Policymaking and Program Development: The Case of Singapore; Hui Leng Ng, Chew Leng Poon, and Elizabeth Pang.- 15 Understanding the Policy Influence of International Large-Scale Assessments in Education; David Rutkowski, Greg Thompson, and Leslie Rutkowski.-
