Changing the Equation:
Scaling a Proven Innovationby
Carol A. Twigg
With support from the Bill & Melinda Gates Foundation, the National Center for Academic Transformation (NCAT) is conducting a major program, Changing the Equation, to engage the nation’s community colleges in a successful redesign of their remedial/developmental math sequences (i.e., all mathematics courses offered at the institution prior to the first college-level math course.) The goal of this new redesign program is to improve student learning outcomes in remedial/developmental math while reducing costs for both students and institutions using NCAT’s proven redesign methodology. Thirty-eight institutions have been selected to participate in the program through a competitive application process. Those institutions will pilot their redesign plans in spring 2011 and fully implement their plans in fall 2011.
Background
A major obstacle for students who are pursuing degrees or credentials in community colleges is successfully completing the college mathematics requirement. Unfortunately, that frequently means completing both remedial and/or developmental math courses as well as college-level math courses. A 2004 study by the U.S. Department of Education found that over 60% of community-college students needed remediation. Students lacking in the competencies and skills required to enroll in college-level courses face significant challenges persisting to a degree.
Unfortunately, there has been very little change in how institutions design their academic programs and create support systems to meet the needs of their students who enter college without the necessary skills to perform college-level work. Successful completion rates in community colleges for remedial and developmental math courses rarely move beyond 50% and are often less than that. Completing a series of non-credit courses to overcome deficiencies involves significant time and money for students, slowing academic progress and sometimes derailing the momentum that comes with initial enrollment in postsecondary education.
Course redesign is a proven, data-driven innovation in institutional practice that makes it possible to get more students to and through credential-granting programs, to accelerate the rate of academic catch-up for poorly prepared students and to improve the first-year experience.
NCAT has ten years of experience in conducting large-scale course redesign programs that improve learning while reducing costs. Developmental math redesigns at NCAT partner institutions have
- increased the percentage of students successfully completing a developmental math course by 51% on average (ranging from 10% to 135%),
- reduced the cost of instruction by 30% on average (ranging from 12% to 52%), and
- impacted ~10,000 students annually.
For example, at Cleveland State Community College the number of students passing a developmental math course increased by 29% while the cost of offering developmental math was reduced by 20%. At Jackson State Community College, the number of students passing a developmental math course increased by 44% while the cost of offering developmental math was reduced by 20%. Changing the Equation will scale such successes to additional institutions. (The NCAT web site includes Math Learning and Math Cost summary charts that provide supporting data.)
In addition to measuring course completion rates and cost reduction, all NCAT redesign projects compare student learning outcomes in the traditional format with those achieved in the redesigned format. This is done by 1) running parallel sections of the course in the two formats or 2) comparing baseline data from a traditional course to a later offering of the redesigned course, looking at differences in outcomes in the "before and after." Techniques used to assess student learning include comparing the results of common final examinations, comparing common questions or items embedded in examinations or assignments, comparing pre/post-tests and comparing final grades when the same assignments, tests and final exams are used and graded using the same criteria. Student learning gains as expressed in increased percentage points have averaged 14 points per project (ranging from 4 to 38 points.)
Increases in course completion rates in developmental math may be more indicative of success than are increases in direct measures of learning, assuming similar grading standards are used. We know that students who “do the work” in developmental math courses will succeed in the course. The problem that most institutions face is the large number of students who simply do not do the work and subsequently fail to complete the course successfully. Thus, the 51% increase in the average rate of successful course completion in NCAT course redesigns may be more significant than the average gain in learning outcomes.
Where Does Changing the Equation Fit in the Panoply of Developmental Math Reform?
In “Technology Solutions for Developmental Math: An Overview of Current and Emerging Practices,” Rhonda Epper and Elaine Baker observe the following: “In the past five years, the critical role of developmental math in the retention and success of community college students has come under additional scrutiny, partially through the attention and resources of national initiatives, such as the Lumina Foundation’s Achieving the Dream project, the Ford Foundation’s Bridges to Opportunity project, the Joyce Foundation’s Shifting Gears project, the Charles Stewart Mott Foundation’s Breaking Through Initiative, a joint project of Jobs for the Future (JFF) and the National Council of Workforce Education (NCWE) and the William and Flora Hewlett Foundation and Carnegie Foundation for Teaching and Learning’s Strengthening Pre-Collegiate Education in California Community Colleges (SPECC) project. This attention has been accompanied by several research efforts in developmental education and a parallel policy focus on the implications of this research for higher education policy. The combination of interest from foundations, public policy groups and researchers who view math as the gatekeeper to college success have yielded a variety of new strategies and programmatic innovations, with a parallel focus on evaluating and assessing the promise of these strategies in terms of replication, scalability and sustainability.”
These reform efforts appear to be either completely open-ended, leaving the development of solutions up to individual institutions with few guiding parameters offered by the grantor, or heavily based on pre-determined solutions, with the grantor having already decided what will address the problem (e.g., open-source materials) and then looking for willing institutions to implement the solution. Few of these initiatives have produced definitive conclusions about how to increase student success in developmental math.
In contrast, NCAT’s approach has been first to establish a set of broad parameters (e.g., redesign the whole course, use instructional technology, reduce cost, modularize the curriculum) and then let experimentation bloom within those parameters. From this process, a number of solutions have emerged. In many cases, these solutions were anticipated, but in some cases they were not. NCAT has continually extracted lessons learned (models, principles, techniques) from these experiences and refined the parameters, iterating this process over the past 10 years.
From working with large numbers of students, faculty and institutions, NCAT has learned what works and what does not work in improving student achievement in developmental mathematics. The underlying principle is simple: Students learn math by doing math, not by listening to someone talk about doing math. Interactive computer software combined with personalized, on-demand assistance and mandatory student participation are the key elements of success. NCAT calls this model for success, the Emporium Model, named after what the model’s originator, Virginia Tech, called its initial course redesign.
The Emporium Model has been implemented in various ways. Some institutions have large computer labs; others have small computer labs. At some institutions, students spend a required number of hours in the lab at any time that the lab is open. At other institutions, instructors meet with students in the lab or in a classroom at scheduled hours. Each institution makes design decisions in the context of the constraints it faces. What is critical is the focus on using interactive computer software combined with personalized, on-demand assistance.
We believe that NCAT and its partner institutions are far ahead of other reform initiatives in that we have proven what works: that redesigning the developmental math sequence by modularizing the curriculum and using NCAT’s Emporium Model will result in dramatic increases in student success and reductions in instructional costs. Furthermore, we have done so with very large numbers of students. Our task is now to convince the nation’s community colleges that they can replicate that success by implementing the Emporium model and assist them in doing so.
How Does Changing the Equation Differ from Other Reform Initiatives?
The purpose of Changing the Equation is to scale a proven innovation to additional institutions. For those of you thinking about reforming your developmental math program, let’s consider more specifically the characteristics of NCAT redesigns and how they differ from other reform initiatives.
- Whole course redesign conducted by teams of faculty and administrators. In each NCAT redesign, the whole course rather than a single class or section is the target of redesign. In contrast to traditional courses where each instructor typically does his or her own thing, redesigned courses are consistent in content, in coverage, in assessment and in pedagogy across all sections of the course. A collective commitment is a key factor for the success and the sustainability of redesign projects. Innovations in higher education frequently fail because they are dependent upon a single “champion,” who might be a risk-taking, creative faculty member or an administrator trying to create change within the institution. If that champion leaves the institution or changes positions within it, there goes the innovation. In contrast, NCAT establishes course redesign teams that do not rely on an individual faculty member or a particular administrator but rather include multiple representatives of both types who follow a redesign plan that is fully supported by the entire department. Because we require the entire developmental math sequence to be redesigned, sustainability is an integral part of the outcome since it becomes the way developmental math is offered. The redesign becomes “institutionalized,” making the innovation relatively impervious to individual shifts in personnel.
Lesson: Individual experiments frequently produce good results but rarely lead to sustained change. “Random acts of progress,” as Bill Graves has called them, characterize most reform efforts. A collective commitment to redesign the whole course is key to success.
- Commercially available computer-based learning resources. Successful course redesign that improves student learning while reducing instructional costs is heavily dependent upon high-quality, interactive learning materials. Instructional software packages such as MyMathLab, ALEKS or Hawkes Learning Systems--which include interactive tutorials, computational exercises, videos, practice exercises and online quizzes--play a central role in engaging students with course content. Students spend more time on task than when they simply watch or listen to a lecture given by an instructor in a traditional format. Students find the software easy to use and achieve a comfort level with the technology in a short amount of time. Redesign teams can rely on commercial providers for training, support and software maintenance.
Lesson: Innovations in higher education that focus on materials creation rather than how the materials are used frequently fail. Faculty members who incorporate commercially available materials are able to focus on pedagogical and organizational issues rather on materials creation, adaptation and maintenance.
- Proven methods of integrating technology and learner-centered pedagogy. Most attempts to use technology in developmental math reform are simply “add-ons” to an otherwise unchanged instructional process. Students continue to meet in groups at fixed times and places, and technology is used as a “supplement.” This approach may bring marginal but not dramatic improvements. In contrast, NCAT redesigns integrate student time-on-task working with interactive software with on-demand, individualized assistance. Students work in lab settings where instructors, tutors and/or peers are available to provide help on an individualized basis when students encounter difficulties. Every student gets his or her questions answered every time.
Lesson: Innovations that continue to rely on students meeting in small groups in traditional classroom settings with “teacher-led” activities are not the answer. Individualizing each student’s learning experience enables instructors to concentrate on specific areas of weakness and move the student to successful completion.
- Modularization + mastery learning. NCAT has learned that the combination of a modularized curriculum (rather than a course-based curriculum) and a mastery-based learning strategy (rather than “you get it or you don’t and, if you don’t, you start over”) is critical to increasing success in developmental mathematics. Most community colleges offer a series of remedial/developmental courses taught primarily in traditional classroom settings in a semester or quarter format. Weaker students may be required to complete up to three or four full terms of coursework prior to advancing into regular college-level courses. Further, the course delivery strategy does not allow students to get up to an acceptable performance level in one stage so that they can quickly move onto the next stage. Students are required to take an entire course even though they may only be deficient in a portion of the topics. All students are required to learn at the same pace and with the same instructional strategies as the entire class. In contrast, NCAT redesigns allow students to start anywhere in the remedial/developmental course sequence based on their learning needs. Students can progress through content modules at a faster pace if possible or at a slower pace if necessary, spending the amount of time needed to master the module content.
Lesson: Innovations that maintain the lockstep pacing of traditional classroom formats may unnecessarily prolong the time students spend in developmental math. Modularizing the remedial/developmental math curriculum provides efficiencies in meeting requirements, which benefit both the student and the institution.
- Oh, yes – cost reduction. Unfortunately, many innovations in higher education rely on grant funding in order to exist rather than to support a transition to a sustainable model. Increased student success may be achieved due to extra resources provided by the grant. When the grant funding ends, there goes the innovation. In contrast, sustainability is a built-in component to every NCAT redesign because the cost of offering the course is reduced in every successful implementation. Institutions that have completed the redesign process have fully established a successful learning environment for students at a reduced cost. They have no reason or motivation to return to a less successful, more expensive approach. Each redesign includes sustainability in its plan from the outset, and no new resources are needed on a recurring basis to sustain the redesign.
Lesson: Reform efforts that are dependent on grant funding frequently fade away after the grant period is over. Institutions whose redesign plans reduce costs will inherently be more likely to succeed over the long run.
The Bottom Line
Most reform efforts currently underway in developmental mathematics are simply tinkering at the margins and have no clear vision of how to create significant and sustainable change. They are experimenting with theories of change, and the results of those experiments will not be known for years. The institutions participating in Changing the Equation are serious about increasing student success in developmental math and want to do it now. The support, structure and guidance offered by the program will short-cut the typical pace of change in higher education and let them see results in less than two years.
In a June 9, 2008 Inside Higher Education article, Vincent Tinto contends, “We must stop tinkering at the margins of institutional life, stop our tendency to take an ‘add-on’ approach to institutional innovation, and stop marginalizing our efforts and in turn our academically under-prepared students, and take seriously the task of restructuring what we do.”
In other words, it’s time to Change the Equation!
