TITLE: Evaluating the Impact of CUNY Start through a Researcher (MDRC) – Local Education Agency (City University of New York) Partnership CENTER: NCER YEAR: 2014 PRINCIPAL INVESTIGATOR: Weiss, Michael GRANTEE: MDRC PROGRAM: Evaluation of State and Local Education Programs and Policies [Program Details] AWARD PERIOD: 5 years (7/1/2014-6/30/2019) AWARD AMOUNT: $4,796,087 AWARD NUMBER: R305H140065 DESCRIPTION: Co-Principal Investigators: Alexandra Logue and Julia Wrigley (The City University of New York) Purpose: Many students enter postsecondary education underprepared. At open access colleges (e.g., community colleges), many incoming students are referred to developmental coursework. These colleges have typically responded to the needs of underprepared students by designing multi-level, non-credit course sequences in reading, English, and math. To help address the needs of incoming students, the City University of New York (CUNY) developed CUNY Start (CS), a multifaceted pre-matriculation program that provides intensive instruction in reading, writing, and math through a carefully prescribed curriculum and instructional delivery. CS represents one of the most ambitious initiatives in the country designed to help students with low basic skills levels succeed in college, but there is no experimental evidence of CS’s success. In this study, researchers will provide this initial evidence of the estimated effect of CS on students’ academic outcomes, how these effects vary by population and context, how the program is implemented, and its associated costs. These findings will help CUNY refine its institutional strategy and contribute to the national discussion on how to increase the success rate of students who enter community college with substantial remedial needs. Project Activities: The research team will carry out a random-assignment study to estimate the impact of CS on student academic outcomes. Over the course of this project, the research partners (MDRC and CUNY) will also co-develop a joint research agenda and build CUNY’s research capacity. Researchers will examine the extent to which CS is more effective for particular subgroups of students and whether effects vary across the four participating colleges. Researchers will define confirmatory subgroups based on students’ intent to enroll in the full- or part-time CS program, number of developmental needs, and college. Exploratory subgroups include gender, age, race, and high school diploma/GED. All subgroups are defined based on pre-random assignment data. The team will also collect information about the implementation of CS across campuses in the CUNY system (led by project collaborator, Community College Research Center) and on the costs associated with CS. Products: The team will produce several public documents such as policy briefs, a practice guide, and reports for the CUNY partners. They will also prepare publications for peer-review. STRUCTURED ABSTRACT Setting: The research will take place at four of the six colleges implementing CUNY Start (CS) in the City University of New York (CUNY) network. Sample: The sample for this project is students with significant remediation needs at the participating CUNY community colleges. The sample will include nearly 4,500 of these students. Intervention: CUNY Start (CS) is a multifaceted pre-matriculation program that provides intensive instruction in reading, writing, and math through a carefully prescribed curriculum and instructional delivery method to incoming postsecondary students with significant remedial needs. CS condenses the time students spend preparing for college-level English and math into a single semester. Full-time students take classes in reading/writing and math for 24 hours per week; part-time students take either reading/writing or math for 12 hours per week. In addition, CS delivers enhanced academic and nonacademic supports in the form of advisors, tutors, and a weekly seminar that builds college success skills. CS’s instructional intensity, integrated supports for students and faculty, and position in continuing education distinguish it from the traditional developmental college experience. The underlying theory of change posits that students with developmental needs are best served through a cohort-based integrated academic and nonacademic support model designed to build academic preparedness, confidence, and college know-how prior to matriculating in college. Costing only $75 for the semester, CS allows students to preserve their financial aid for future college-credit courses. Research Design and Methods: The research plan for the evaluation of CS is composed of three complementary components: (1) a randomized trial to estimate the effects of CS on student academic progress, (2) implementation research that focuses on how the program is put into effect, fidelity, and the treatment contrast, and (3) cost analyses. The random assignment design will estimate the causal effects of CS. Over three semesters, approximately 4,500 students will be assigned to either a control group or a program group during the class registration process. The target population for the study includes students who fail the writing and math placement assessments for the full-time program and students who fail one or more of these tests for the part-time program. The evaluation will examine the extent that CS is more effective for particular subgroups of students and whether effects vary across the four study colleges. Confirmatory subgroups will be defined based on students’ intent to enroll in the full- or part-time CS program, number of developmental needs, and college. Exploratory subgroups include gender, age, race, and high school diploma/GED. All subgroups are defined based on pre-random assignment data. Control Condition: Control group members have the opportunity to matriculate in college and enroll in traditionally-structured courses. Most students will likely enroll in one or two developmental (remedial) courses offered for 3 to 6 hours per week that use a lecture-based pedagogy. They may also enroll in some college-level courses (e.g., psychology). Key Measures: Using college transcript records, the researchers will collect data on the confirmatory outcomes: reduction and/or elimination of remedial needs and college-level credit accumulation (a proxy for overall academic progress toward a degree). The researchers will also explore the program’s effects on students’ knowledge and skills through college-administered standardized exams (e.g., the CUNY Assessment Test for reading and writing). In addition, they will also measure implementation fidelity and the treatment contrast through surveys (student and instructor), interviews and focus groups (instructors, advisors/counselors, academic support staff, and administrators), classroom observations, and transcript data. Data Analytic Strategy: Researchers will use standard statistical tests to compute intent-to-treat estimates by comparing average outcomes of the program group and control group members (controlling for college and a small set of covariates, such as baseline placement test scores). The research team will also conduct subgroup analyses based on students’ intent to enroll in the full or part-time program, level of remedial need, and college. Researchers will estimate the program’s effects for each subgroup and conduct statistical tests for evidence of differences in subgroup effects. Sensitivity analyses will be conducted using cluster-robust standard errors.
Posted on: Wed, 06 Aug 2014 19:40:42 +0000