The Learning MarketSpace, October 2006
A quarterly electronic newsletter of the National Center for Academic Transformation highlighting ongoing examples of redesigned learning environments using technology and examining issues related to their development and implementation.
TABLE OF CONTENTS:
Offering perspectives on issues and developments at the nexus of higher education and information technology
Final R2R Results
Regular readers of The Learning MarketSpace know that NCAT has just completed a three-year, FIPSE-funded program, The Roadmap to Redesign (R2R). Its goal was to simplify the course redesign process and make it affordable within existing institutional resources. FIPSE funding was used to develop and provide a wide variety of resources that supported new institutions as they redesigned large-enrollment, introductory courses.
R2R identified 20 institutions that wanted to redesign courses in four disciplines—precalculus mathematics, psychology, Spanish and statistics—in order to test a new model. The new model partnered experienced, successful institutions with new institutions and took advantage of best practices, virtual repositories of research-based learning materials and a streamlined redesign methodology.
At the start, we said we could guarantee that if new institutions followed our advice—derived from the successes achieved in the Program in Course Redesign—they would improve student learning, increase retention and reduce instructional costs.
And that is exactly what happened.
Not all institutions followed our advice. Five of the original 20 institutions did not complete the program. (More about this in the January issue of The Learning MarketSpace.)
Here are the results for the 15 institutions that completed the program.
Eight of the 15 R2R projects report improved learning outcomes; four report learning equivalent to the traditional format, and three have not yet submitted final data. Among the findings are the following:
At Georgia State University, students in the redesigned College Algebra pilot sections performed significantly better than students in traditional sections on five common examination questions (a mean score of 3.04 vs. 2.66 for traditional students.)
In the First-Year Spanish full implementation at Texas Tech, redesign students performed significantly better than traditional students in vocabulary (a mean score of 81.04 vs. 74.18 for traditional students.) In the Second-Year Spanish pilot, redesign students performed significantly better than traditional students on assessments of vocabulary, grammar, reading composition and culture (74.04 vs. 60.52) and on an assessment of speaking skills (78.56 vs. 72.91).
At the University of Alabama , a comparison of common final exam scores between the traditional Spanish courses and the redesigned courses indicated that mean final exam scores improved in all three of the redesigned courses: 66.81 vs. 65.54 in Spanish 101; 72.32 vs. 67.86 in Spanish 103; and 79.72 vs. 71.42 in Spanish 102. There was a significant increase in the number of students receiving a final grade of A in the redesigned Spanish 101 (40% vs. less than 12%) and in Spanish 102 (41% vs. 19%.)
At the University of Missouri-Saint Louis, students learned more in the redesigned College Algebra course than in the traditional course as indicated by improved performance on common comprehensive final exams. There were more As and Bs and fewer Ds and Fs. Combined results for two academic years indicate that 56.6% of students in the redesigned sections achieved in the top two of six score ranges, while only 31.5% of students in the traditional sections did so.
The fall 2005 pilot of Precalculus at the University of North Carolina-Chapel Hill compared student performance across three different formats: the traditional course format, a hybrid format and the redesigned format. Student gains in test performance were slightly greater with the redesigned and hybrid sections (average gain of 50.4% ± 1.4 vs. 47.6% ± 2.0, p=0.28.)
At the University of North Carolina-Greensboro, the average score on common, comprehensive final examinations in Precalculus I increased significantly from 58.2 for the traditional course to 75.5. In Precalculus II, the final exam average increased from 65 for the traditional course to 69.6. The average score on common, comprehensive final examinations in Statistics increased from 63.5 for the traditional course to 68.5 for full implementation of the redesigned course.
The Wayne State University team compared eight semesters of data from the traditional Beginning Algebra course to four semesters of data from the redesign using uniformly graded final examinations. The average score increased from 64.6% to 68.6%. The team also looked at success in the follow-on math course. The overall pass rate in the next course was slightly higher for those coming out of the redesigned course (44.0% vs. 42.9%). However, comparable final exam scores seem to produce better results for students coming from the redesign. For example, of those students who scored 90 -100% on the final exam, 86.7% of the redesign students passed the next course, while only 78.4% of the traditional students did so.
Nine of the 15 R2R projects show improvement in course completion rates; two report no change; and four have not yet submitted final data. Among the findings are the following:
In fall 2005, the Business Statistics course at Calhoun Community College enjoyed an 82% completion rate compared with a 65% rate in the traditional course. The overall course grade average was B (83%), and no D or F grades were given.
The drop-failure-withdrawal (DFW) rate for the redesigned Introductory Psychology course at Eastern Washington University was 23.0% compared to 26.1% in the traditional sections.
The DFW rate for the redesigned College Algebra course at Georgia State University was 12.2% in the fall 2004 pilot compared to 27.6% in the fall 2003 traditional sections.
At Texas Tech, the number of Ws in the Spanish redesign full implementation dropped compared to a two-semester average of the traditional format. The number of Ds and Fs combined dropped despite more stringent grading standards in the redesign.
The percentage of students who successfully completed ( obtaining a final grade of C or better) two of the three redesigned Spanish courses at the University of Alabama increased compared to those in the traditional formats (in Spanish 101, 82% in the redesign vs. 75% in the traditional; in Spanish 102, 92% in the redesigned sections vs. 90% in the traditional sections.)
At the University of Missouri-Saint Louis, retention rates improved in the redesigned College Algebra course relative to the traditional course. The successful completion rate (grade of C or better) for students increased from 63.3% in the traditional course to 78.4% in the redesign.
At the University of North Carolina-Greensboro, student retention improved substantially in all redesigned courses. In College Algebra, the DFW rate dropped from 62% in the traditional course to 49% in the redesigned course, in Precalculus I from 77% to 38% , and in Precalculus II from 60% to 41%. In Statistics, the DFW rate dropped from 70% in the traditional course to 60% in the redesigned course.
At Wayne State University, the student success rate in Beginning Algebra increased from 35.5% in the traditional course to 42.8% in the redesigned course. Approximately two-thirds of those who took the final exam passed it compared to only about half in the traditional course.
With regard to cost savings, the R2R results are an unqualified success. All 15 of the R2R projects reduced their costs. Eleven of the 15 carried out their original plan to produce cost savings. Some saved more; others saved less; but all produced cost savings.
Three universities saved more than they had planned. Georgia State University originally estimated that the cost-per-student in College Algebra would decrease from $96 to $87, a 9% savings. The actual cost-per-student was reduced to $80, producing a savings of about $24,000, a reduction of 17%. Louisiana State University originally estimated that the cost-per-student in College Algebra would decrease from $121 to $95, a 21% savings. Changes made in the full implementation further reduced the cost-per-student to about $78, an overall 36% savings. The University of North Carolina-Greensboro carried out its planned cost reduction strategy in Statistics and reduced the cost-per-student from $102 to $82. In addition, overall faculty presentation and test-proctoring hours were reduced by an additional 350 hours above what was initially planned, thus further reducing the workload of instructors.
Two institutions saved less than they had planned. Calhoun Community College planned to increase the number of students in Statistics from 281 to 480 and the number of sections from 14 to 20. Final enrollment at the time of full implementation was 391, and the number of sections increased to 17. Calhoun also added an unanticipated half-time lab assistant with a background in statistics to the final redesign. With these changes, the cost- per-student declined from $170 to $144, a 15% decrease rather than the planned 35% decrease. Eastern Washington University’s original goal was to reduce the number of sections in Introductory Psychology from 10 to four, a savings of 60%. This goal could not be met because departmental obligations to individual faculty members made it necessary to offer two traditional sections. Approximately 100 more students than projected enrolled in the redesigned course. After adjusting for the increase in enrollment, the final cost-per-student was reduced from $100 in the traditional course to $53 in the redesigned course, a 47% reduction.
We asked the R2R institutions, will the redesign be sustained now that the R2R program has concluded? Here are their answers:
At Calhoun Community College, the faculty members who teach the redesigned course have seen its benefits and are committed to making further improvements to their initial effort. Additionally, the second-semester Business Statistics course is scheduled to begin the redesign process in the 2006-07 school year.
The team at Chattanooga says, “We will be continuing our redesigned course and look forward to adding two new enthusiastic faculty members to our team.”
Although East Carolina’s redesign project was negatively impacted by technology and infrastructure challenges as well as instructor family and health issues, the team plans to conduct a new redesign pilot in fall 2006 that will profit from the lessons learned during the past two years and then move to full implementation.
At Eastern Washington, the psychology department voted unanimously to make the redesigned course the standard model. One or two traditional sections may be taught each year as needed, but more than 80 percent of students will take the redesigned version.
At Georgia State, the sustainability of the redesign is not in jeopardy because of its success in increasing the number of As, Bs and Cs and reducing the DWF rate.
LSU says, “There is absolutely no doubt that the redesign will be sustained. The university has already invested substantial resources in the existing 116-seat learning lab, and a second 122-seat learning lab will be operational in fall 2006.”
At Seton Hall, the redesign will be continued since it has improved student learning and reduced the costs of the program. The team plans to do some fine-tuning to allow for further customization of the material.
At Texas Tech, the course redesign is intended to be long-term. As long as the department retains its present chair, there is no likelihood that the redesign will be abandoned or substantially altered.
The University of Alabama team is confident of the sustainability of the program. Several minor changes are planned for next year, which are intended to reduce even further the amount of time instructors spend on grading and improve student attitudes toward the new format.
At UM-SL, the campus and the faculty are dedicated to continuing the successful redesign and expanding the model to other courses.
The UNC-Chapel Hill math department is satisfied with the overall results of the initial implementation and is likely to continue it with some modifications.
UNC-Greensboro will continue to explore the best format for the redesigned courses. In addition to offering the original replacement model, UNCG will also offer a fully online version and a supplementary version. During the 2006-2007 academic year, all three models will be compared.
At Wayne State, the redesign has produced both improved learning and cost savings. Consequently, the university has made a commitment to build a new 150-station computer lab that will allow the team to expand the redesign to additional courses and to accommodate increased enrollments.
We also asked the R2R teams, Will you apply the redesign methodology to other courses and programs on campus? Here again are their answers:
Some are exploring options. Based on the encouraging results from the full implementation of the redesign in Spanish, the University of Alabama is currently studying the possibility of redesigning introductory French courses as well.
Others report definite plans in addition to exploring other options. At East Carolina, a required health course redesign has been piloted and will be fully implemented in fall 2006. A second psychology course has been redesigned, and another department is exploring the possibility of redesigning its introductory courses. At Chattanooga, elements of the redesign have been included in Psychology of Personal Adjustment and Abnormal Psychology. Sociology is considering redesigning Introduction to Sociology
Others have definite plans for expanding the redesign methodology to other courses. At Georgia State, the redesign will expand to the Calculus course sequence. At LSU, Precalculus will also be taught using the redesigned format, and plans are being made to redesign Trigonometry as well. At Seton Hall, the redesign will be expanded to the Pre-Algebra portion of the Developmental Math program. The redesign of Developmental Math is part of an overall redesign of Precalculus courses. The Texas Tech team has already piloted the redesign in second-year Spanish. In fall 2006 they will do a full implementation followed by other Spanish courses in spring 2007. During summer 2006, the German department converted their workbooks to online and adopted a supplemental model version for the fall semester. Texas Tech is encouraging other languages to follow suit.
Based on the initial success of UM-SL’s redesign of College Algebra, the supplemental model was introduced in the Trigonometry course. Various faculty members are offering pilot sections for Basic Calculus, the Calculus series, and Statistics. If the results are positive, the changes will be made to all the sections of these courses. Based on the positive results achieved thus far, there is no reason not to continue this winning formula. By fall 2006, Wayne State will have redesigned three courses: the original Beginning Algebra, Intermediate Algebra, which was added in fall 2005, and Finite Math for the Social and Management Sciences. The team will also add a homework and quiz component to some other math courses that will not be fully redesigned.
Finally, other institutions plan an even more elaborate expansion. Calhoun Community College has adopted NCAT’s course redesign process in its initiative to redesign 38 of its large enrollment courses over a five-year period. Redesign work is now underway in ten other courses. The Eastern Washington Teaching and Learning Center is sponsoring efforts to develop redesign models for the departments of business, mathematics and biology. Pilot courses are underway and internally funded grants are being awarded to faculty for developing redesign variants for their own courses.
The Roadmap to Redesign has achieved its goals. The pioneering institutions in the Program in Course Redesign established replicable models for those institutions that want to use technology to improve student learning while reducing instructional costs. Building upon that valuable experience, the R2R projects have adopted those models and achieved similar and, in some cases, even stronger results.
In the January issue of The Learning MarketSpace, we will discuss what contributed to these results and what lessons have been learned that can be applied as other colleges and universities embark upon their own course redesigns. Congratulations to our R2R colleagues for their extraordinary accomplishments!
For more information about any of the R2R projects, please go to http://www.thencat.org/R2R/R2R_ProjDiscipline.htm and follow the links to the individual project descriptions and their final reports.
--Carol A. Twigg
Featuring updates and announcements from the Center
NCAT Launches New Organization: The Redesign Alliance
On October 9, 2006, the Redesign Alliance Founding Members held a planning meeting in Dallas, TX. Representatives from 34 organizations discussed a draft version of the Call to Participate, which will be issued to the higher education community in early November, and the activities that will be conducted during the Alliance’s first year. The Redesign Alliance will hold its first annual conference on March 18-20, 2007 in Orlando, Fl. To learn more about becoming a member of the Redesign Alliance, contact Carolyn Jarmon at cjarmon@theNCAT.org or visit http://www.thencat.org/RA.htm.
The National Center for Academic Transformation has been awarded a new three-year grant from the Fund for the Improvement of Post-Secondary Education (FIPSE.) The purpose of the grant is the further adoption of NCAT’s methods of course redesign throughout the higher education community. The grant will support and extend the work of the Redesign Alliance, including an annual national conference to disseminate successful redesign strategies and to bring together the growing body of work in course redesign. Specifically, the project will 1) establish a Redesign Scholars Program to enable faculty and administrators experienced in course redesign to serve as a resource for inexperienced colleges and universities; 2) select 60 institutional teams committed to engaging in course redesign to participate in the project; 3) hold 12 disciplinary-based institutes over the three-year period for experienced institutions to share proven course redesign strategies and techniques with the 60 new participating institutions; and, 4) support collaboration among NCAT staff, Redesign Scholars and institutional teams to help them apply what was learned at the institutes on campus. Please see http://www.thencat.org/RedesignAlliance/DissemProposalNarr.htm to read the full FIPSE proposal. Application guidelines for participating in the new program will be issued in early November.
The Commission on the Future of Higher Education convened by Secretary of Education Margaret Spellings issued its report in September. Its three areas of focus are affordability, access and success. In the words of Secretary Spellings, "Over the years, we've invested tens of billions of dollars in taxpayer money and just hoped for the best—we deserve better. To remain competitive in the 21st century global economy, we must act now and continue the national dialogue and work together to find the right solutions." A diverse panel of commissioners reviewed a range of data and heard testimony from numerous experts in these three areas, including NCAT’s Carol Twigg. Apparently, NCAT’s track record made a positive impression on the commissioners. The report makes the following comments and recommendations: “With too few exceptions, higher education has yet to address the fundamental issues of how academic programs and institutions must be transformed to serve the changing needs of a knowledge economy. We recommend that America’s colleges and universities embrace a culture of continuous innovation and quality improvement by developing new pedagogies, curricula, and technologies to improve learning, particularly in the area of science and mathematical literacy. . . . Effective use of information technology can improve student learning, reduce instructional costs, and meet critical workforce needs. We urge states and institutions to establish course redesign programs using technology- based, learner-centered principles drawing upon the innovative work already being done by organizations such as the National Center for Academic Transformation." To access the full report, see http://www.ed.gov/about/bdscomm/list/hiedfuture/reports/pre-pub-report.pdf.
Featuring initiatives to scale course redesign through state- and system-wide redesign programs
On October 17, 2006, more than 100 faculty members, professional staff and administrators from the University System of Maryland (USM) participated in an orientation workshop as part of the system-wide Maryland Course Redesign Initiative. The initiative is a partnership between USM and NCAT to develop at least one successful redesign project at each of the system’s 11 institutions. The goal of the workshop was to stimulate the thinking of all about the many possible ways to accomplish a redesign. Participants learned more about what is involved in implementing a large-scale redesign, what models have proven successful and how to measure both student learning outcomes and instructional costs. The group also exchanged ideas about course redesign during a case study exercise. The next step for those interested in participating in the redesign initiative is to respond to a group of readiness criteria designed to help institutional teams select the appropriate course and analyze their institutional circumstances to be sure they are prepared to launch a successful project. A second workshop will be held on January 22, 2007 to work more specifically on each institution’s redesign plan. To learn more about the USM program, see http://www.usmd.edu/usm/academicaffairs/courseredesign/ or contact Nancy Shapiro at email@example.com or Don Spicer at firstname.lastname@example.org.
In the second year of a redesign initiative at the University of Hawaii System (UH), the team at UH-Manoa has begun to plan a redesign of its Hawaiian Studies course. This course, the highest enrolled on campus with more than 1,100 students each term, experiences greater demand than the department can meet. Taught primarily by graduate teaching assistants, the course includes a wide variety of academic areas such as culture, language, music, art and geography, which provides a challenge for many GTAs. The team anticipates using the replacement model and will reduce the number of lecturers needed to staff the course while increasing the number of students who can enroll each term. Another goal of the redesign is to increase the consistency of course content by preserving a number of interviews with experts captured on both video and audio tapes that can be easily shared widely among students. For more information, contact Hae Okimoto at email@example.com.
The Tennessee Board of Regents (TBR) and the Education Commission of the States have been awarded a three-year FIPSE grant to partner with NCAT in a statewide redesign initiative. The grant will focus on reform of the remedial and developmental curriculum in the state of Tennessee, including teaching and learning methods and assessment strategies. In 2005, more than 60% of students entering TBR institutions were required to take at least one developmental course. In addition, the time needed to complete remedial and developmental sequences has increased as students enter with fewer skills and require more developmental courses. With large numbers of college students in Tennessee needing some level of remedial or developmental help, the goals of the grant include increasing success rates in these courses and reducing the time that students need to complete the sequences. NCAT will work with TBR institutions to meet these goals. The grant will support multiple pilots in English and math to increase student learning and reduce the cost of delivery. For more information, please contact Houston Davis at Houston.Davis@tbr.edu.
In September, 2006, the Arizona Board of Regents (ABOR) issued a Request for Proposals to establish a Grant Program for Improved Learning and Cost Effective Delivery in High Enrollment Undergraduate Courses. The ABOR institutions, which include Arizona State University, Northern Arizona University and the University of Arizona, enroll nearly 120,000 undergraduates. This two-to-three-year initiative will focus on learner-centered education at these three institutions using “an established and proven model to (a) increase the effectiveness of instruction and learning in high-enrollment undergraduate courses; (b) provide improved measurable learning outcomes and retention for students enrolled in those courses; and (c) reduce the per-student cost of instruction in these courses.” NCAT has submitted a proposal in response and hopes to add Arizona to a growing list of states and systems working with us on large-scale course redesign.
Representing all public higher education institutions in the state of Texas, the Texas Higher Education Coordinating Board (THECB) is formulating a redesign initiative. Texas House Bill #1, passed earlier this year, mandates the establishment of a course redesign project that will improve student learning and reduce the cost of delivery in lower division, academic courses. On August 30, 2006, Carolyn Jarmon met with an advisory group representing multiple Texas higher education systems as well as staff from the THECB. She shared results of the Program in Course Redesign and the Roadmap to Redesign and discussed with committee members steps they might take to establish the program. On October 26 and 27, 2006, Carol Twigg spoke to the Commissioners of the THECB to provide insight about NCAT’s methodology and to describe the successes that institutions using it have enjoyed. She also met with the advisory group, campus representatives and THECB staff to advise them on next steps they might consider in meeting the goals of House Bill #1. A number of Texas institutions are already quite interested in redesign. Texas Tech University successfully completed a redesign of First-Year Spanish as a participant in the Roadmap to Redesign, and four Texas institutions are founding members of The Redesign Alliance: Austin Community College, Dallas County Community College District, the University of North Texas and the University of Texas System. To learn more about House Bill #1 and the Texas initiative, contact Carol Raney at Carol.Raney@THECB.state.tx.us.
On October 19, 2006, Carol Twigg met with the presidents and senior system staff at the State University of New York (SUNY) to learn more about NCAT’s programs and to consider how NCAT’s redesign methodology can benefit students at SUNY’s 64 diverse campuses. SUNY Chancellor, John Ryan, asked the group to discuss the feasibility of establishing a system-wide course redesign program. The State University of New York at Buffalo was one of the participating institutions in the Program in Course Redesign and completed a successful redesign of its introductory Computer Science course. SUNY’s new provost, Risa Palm, has just moved from Louisiana State University, a successful participant in the Roadmap to Redesign. Carol will next meet with the SUNY provosts in early November. At the invitation of Chancellor David Carter, Carol traveled to Connecticut on October 24, 2006, to give a presentation to a leadership group from the Connecticut State University System including members of the Board of Trustees; system senior staff; campus presidents, provosts and deans; faculty leaders; and state government officials on ways colleges and universities across the nation are improving learning while reducing costs through course redesign. Both systems are now considering next steps to leverage what has been accomplished nationally in their own states.
Linking content and software providers with leading edge institutions
Over 50 faculty from 41 institutions and representatives from Pearson Education gathered in San Diego, CA on October 20-21, 2006 for a seminar on course redesign. During the first session, Carolyn Jarmon presented the results of the various NCAT-sponsored course redesign projects and discussed some of the approaches used in them. To provide examples of individual applications, faculty from various institutions described their redesigns. Joe Benson from the University of Alabama, Kirk Trigsted from the University of Idaho, Adreana Grimaldo from Quinsigamond Community College and Steve Dorfman from DeVry University described their initiatives in math. Tina Irvine from Johnston Community College and Pat Baldwin of Pitt Community College described their redesigns in English. On the second afternoon, attendees had a choice of small sessions including getting started with course redesign at their campuses, viewing redesigned courses and demonstrations of MyMathLab and MyWritingLab. The response from those who attended was extremely positive, and the majority expressed an interest in further exploring the possibility of course redesign at their home campuses. For more information about Pearson Education content, contact Karen Silverio at Karen.Silverio@pearsoned.com.
Reporting on initiatives that share the Center's goals and objectives
The University of Mississippi Improves Learning with Hawkes Learning Systems
At North Carolina State University, students in the SCALE-UP program show a significant improvement in learning over traditional formats. SCALE-UP means Student-Centered Activities for Large Enrollment Undergraduate Programs. SCALE-UP creates a highly interactive, collaborative, computer-rich learning environment. Rather than lecturing, faculty work with three-person groups of students who are engaged in challenging problem-solving activities, simulations and other hands-on applications. Course materials compliment the text book. Nearly 16,000 students have participated, and results show that SCALE-UP students consistently have a higher percentage of correct problems on common final exams when compared to those who have not. SCALE-UP students have also achieved a better understanding of physics concepts. Overall, students’ ability to solve problems has improved, conceptual understanding has increased, attitudes toward physics have improved and failure rates have been reduced, especially for women and underserved students. For more information about SCALE-UP, contact Bob Beichner at firstname.lastname@example.org or see www.ncsu.edu/PER/scaleup.html.
In the April 2005 issue of The Learning MarketSpace, we reported on the efforts of Virginia Union University (VUU), a small (1,700 undergraduate and graduate students) historically black institution to redesign courses in three disciplines: English, psychology, and drama. Funded by the Lilly and W. M. Keck Foundations, the VUU project used a formal, tightly structured Instructional Systems Design (ISD) process. Assessment results for the first pilot year show significant learning increases in the redesigned General Psychology course vs. the traditional format (a mean increase from pre-test to post-test of 26.1% vs. 4.6%) and in a number of redesigned Humanities courses (an overall mean increase of 26.2% vs. 11.75%.) The redesigns seemed to be particularly effective with weak students. Overall, students were very satisfied with the new methods of learning, and class attendance has increased. Faculty found developing new learning approaches and implementing first-year pilots both challenging and rewarding. For more information about VUU’s active learning approach, contact James Armstrong at James_R_Armstrong@Dom.com or Jeff Clark at email@example.com.
The National Center for Academic Transformation serves as a source of expertise and support for those in higher education who wish to take advantage of the capabilities of information technology to transform their academic practices.
Copyright 2006, The National Center for Academic Transformation