Impact on Students
Portland State University
In the redesign, did students learn more, less or the same compared to the traditional format?
Improved Student Learning
Oral Exam Scores. The redesign of the first-year Spanish sequence focused on de-emphasizing rote grammar and focusing on oral proficiency. End-of-year oral exam scores, the primary focus of the First-Year Spanish redesign, showed improvement, although not a statistically significant difference: redesign 87.3%, traditional course 85.8%. Even with the initial year implementation issues, the ability of students to use the language showed marked improvement.
End of Year Grades. The data showed no significant difference between student learning in the traditional course and the redesigned course with regard to final exam scores or course grades at the end of the three-course sequence. This may mask important findings that the team will not know until next year because 1) the traditional course was taught by a different professor than the redesigned course although the book was the same; 2) the professor in the redesigned course had greater ability to standardize and monitor the graduate teaching assistant delivery and content in their sections of the course than the previous professor.
WebCAPE Scores (BYU's Computerized Adaptive Placement Exam). The mean CAPE score, which primarily measures reading achievement, for the traditional course (spring 2002) was 308.1, the mean for spring 2003 (redesigned course) was 311.7. These differences were not significantly different.
In 2002-2003 students could move between partially redesigned sections and sections fully redesigned with reduced seat time. The mean CAPE score for students completing the entire first-year sequence in the fully redesigned, reduced seat-time course was 15 points higher than that for other students, but this difference was not statistically significant.
ACTFL OPI Scores (Oral Proficiency Interview). The mean OPI scores for the traditional course (data collected spring 2002) and the fully redesigned course (spring 2003) were not significantly different.
The goal was to have students on average meet the ACTFL Intermediate level of proficiency. The level for the redesigned course (spring 2003) was at the Intermediate level (Intermediate Low). For years the literature has stated that Intermediate-Mid is about the highest a true beginner can get in a first-year course, and that it is infrequent. A median of Novice-High is typical of first-year courses. The redesign achieved the learning outcome goal of ACTFL Intermediate Spanish proficiency, with an average rating of Intermediate Low. The rating is not significantly different from the traditional course.
Aggregate attrition remained relatively constant across the year, with an enrollment rate of 81.3% in the traditional course and a rate of 82.3% in the redesigned courses combined. DFW rates for the final course in the sequence were 5.5% for the traditional course in spring 2002, and 10.6% for the fully redesigned sections in spring 2003. These differences are a bit difficult to interpret. Based on results of a survey given at the end of this school year, lower continuance rates for 2002-03 may reflect dissatisfaction with a less than smooth transition into the redesign. Students were also able to shift between fully-redesigned and not-fully-redesigned sections.
Program in Course Redesign Quick Links: