Full Description
Which curricular and cocurricular practices promote student learning and persistence? While most research and assessment on college student outcomes offers limited insight into causal effects, this volume provides strong evidence of the impact of college on students. The first section discusses statistical analyses that offer more accurate estimates of the causal effect of a particular student experience, such as receiving a need-based scholarship or using academic support services. Providing an overview of the analytical framework, it also includes real-world examples to illustrate implementation for institutional researchers. The second section includes original research to enhance the value of student surveys, including: aspects of questionnaire design and techniques to cope with item nonresponse, variation in respondent effort, interpretation of student self-reported gains, and practical insights to improve survey-based research. This is the 161st volume of this Jossey-Bass quarterly report series.Timely and comprehensive, New Directions for Institutional Research provides planners and administrators in all types of academic institutions with guidelines in such areas as resource coordination, information analysis, program evaluation, and institutional management.
Contents
EDITORS NOTES 1 Nicholas A. Bowman, Serge Herzog 1. Applying Regression Discontinuity Design in Institutional Research 3 Allyson Flaster, Stephen L. DesJardins The goal of this chapter is to provide a brief introduction to one of the most rigorous nonexperimental analytical methods currently employed by education researchers: regression discontinuity. 2. The Propensity Score Analytical Framework: An Overview and Institutional Research Example 21 Serge Herzog Estimating the effect of campus math tutoring support, this study demonstrates the use of propensity score weighted and matched-data analysis and examines the correspondence with results from parametric regression analysis. 3. Item Response Theory: Overview, Applications, and Promise for Institutional Research 41 Jessica Sharkness Item Response Theory (IRT) is a measurement theory that is ideal for scale and test development in institutional research, but it is not without its drawbacks. This chapter provides an overview of IRT, describes an example of its use, and highlights the pros and cons of using IRT in applied settings. 4. The Meaning and Interpretation of College Student Self-Reported Gains 59 Nicholas A. Bowman This chapter examines the relationship between student self-reported gains and college satisfaction, and it considers whether self-reported gains constitute a form of college satisfaction. 5. Identifying Low-Effort Examinees on Student Learning Outcomes Assessment: A Comparison of Two Approaches 69 Joseph A. Rios, Ou Lydia Liu, Brent Bridgeman This chapter describes a study that compares two approaches (selfreported effort [SRE] and response time effort [RTE]) for identifying low-effort examinees in student learning outcomes assessment. Although both approaches equally discriminated from measures of ability (e.g., SAT scores), RTE was found to have a stronger relationship with test performance, identified more low-effort examinees, and led to higher mean performance upon filtering than the SRE approach. 6. Survey Page Length and Progress Indicators: What Are Their Relationships to Item Nonresponse? 83 Shimon Sarraf, Malika Tukibayeva The popularity of online student surveys has been associated with greater item nonresponse. This chapter presents research aimed at exploring what factors might help minimize item nonresponse, such as altering online survey page length and using progress indicators. 7. Effects of Item Order and Response Options in College Student Surveys 99 Nicholas A. Bowman, Jonathon P. Schuldt This chapter discusses an experimental study that shows that the order of items on a questionnaire and the response options for those items both affect the results of college student surveys. INDEX 111
