Course Development Issues
University at Buffalo (SUNY)
To what extent have you been able to use previously developed materials in your re-design instead of creating new materials?
We had planned from the start to incorporate as many previously developed materials as possible into the redesigned course. While many of these materials have been effective from the point of view of student satisfaction and learning, they have been problematic in requiring significant faculty and technical staff time to operate. A number of unanticipated issues emerged in three areas of the course: the concepts portion of the course (learning basic computing concepts), the skills portion of the course (learning general computing skills), and the course management software. The following is a list of previously developed materials selected and issues relating to these materials:
Concepts book with CD-ROM: The book we selected is by far the most appropriate for the course; however, the software for the book is not the best among the texts. There are technical problems installing the software in the laboratory environment that our system staff is working on. The seemingly best software for teaching concepts in a lab setting turned out to have many technical problems. Publishers have developed these materials, primarily by outsourcing to software development companies. Even the publishers' technical staffs don't understand how their own software works: they make claims that are frankly incorrect. These problems don't surface until a full-scale laboratory installation is attempted. Because the materials are on CD-ROM, they cannot be dynamically altered. In addition to the problems at the publisher end, we lack highly experienced Windows NT staff people who might be more creative and proficient on our end in terms of installation.
Concepts Laboratory CD-ROM: The laboratory CD-ROM we selected does not require installation and allows a student to email results from laboratory exercises to an instructor. However, the selection of topics covered by the CD-ROM is not exactly what we had hoped. The publisher is in the process of updating this and adding more labs, and the update will be available in January 2001. This is not problematic because we have decided to de-emphasize this component of the redesigned course in favor of concentrating on using technology to deliver the skills training component. There are no technical problems using these materials in the laboratory.
Skills Training CD-ROM: All publishers have skills training and assessment software. We have elected initially to use the CBT Systems training courses for which our university has a site license. The files required for installation of this software are large, requiring nearly a gigabyte of space. This is slightly problematic in our current laboratory configuration, but we are increasing the size of the hard drives on the lab machines, so this should not be a problem after that. None of the training materials met our expectations: all of them were developed by CBT Systems; all have the same look and feel. Newer skills training materials operate "live in the application," that is, they start the appropriate program and proceed. There are several problems with these types of materials so we have decided to use the simulated environment materials at least for fall 2000. First, we don't know what sort of software problems we will have with Office 2000 in the lab, as it will not be installed until the end of summer 2000. Second, we are unsure as to which versions of MS Office students will have on their home computers: live in the application materials require that all students have the same version of the software.
Course Management Software: We anticipate few problems with the course management software, although a couple of issues have arisen. If the course is hosted on campus, it requires considerable resources: it is still undecided where the hosting would take place. For national hosting, students pay the cost of hosting the course, which is desirable, but there may be security issues about storing student grades and information at any off-campus site. In summer 2000, we used WebCT with national hosting. In fall 2000 we will switch to Blackboard with local hosting. This will allow us to compare the two and better determine which approach is most beneficial and cost efficient.
In summary, we have encountered more problems than anticipated using previously developed materials. These problems range from lack of technical knowledge on the part of publishers to lack of training of our technical staff. Most software is designed for a single user/single PC model: the academic laboratory setting is multi-user/single PC. Web-based software seemed to obviate some of the problems involved with running software over a LAN, but different problems arise when running off a web site including security and connectivity delays. We are working around problems as they come up and are meeting with success. While we are able to get things working, it is taking considerably more time and involvement than we had anticipated for this component of the course redesign. It would be utterly impossible without all of the faculty time we had budgeted.
What kinds of activities took up the most time in your course development effort?
The things we have spent significant time doing so far are:·
During the first-year, baseline phase of the project, steps were taken to identify the instructional goals relevant to the course. The purpose of this effort was to have a basis for evaluating the extent to which course materials (including texts), instructional procedures, and student assessment are consistent with the stated goals for the course.
First, potential goals from three sources were listed, including (1) the goals in each of the three categories (i.e., capabilities, concepts, and skills) specified in the National Research Council report, (2) goals identified in our original proposal, and (3) goals either specified in or inferred from the course syllabi for year 1 of the project. The four computer science faculty familiar with the course then independently rated the extent (high, medium, low) to which each of the 44 listed goals is relevant to the re-designed course. Agreement varied considerably. Several goals received unanimous high ratings from all 4 raters. Although none of the goals received unanimous low ratings, and several goals received mixed low to medium ratings. This effort has assisted us in selecting texts for and planning of the re-designed course to be implemented during the second phase of the project.
The most time-consuming activity to date has been testing software products. In addition to testing it out on a single-user PC, it is necessary to try it out in the laboratory. This requires one-on-one work with a system administrator, which is very time consuming. Many glitches have been discovered in software and problems with installation have been numerous. The time and problems encountered in this area has exceeded expectations by a considerable amount. The lack of robustness of the course management, on-line testing and publisher provided software caused the greatest unexpected increase in faculty time.
We spent more time than anticipated on building support for the project on campus, explaining the relevance of the project to several campus goals and in negotiating resources and other forms of support.
Have students been directly involved in the re-design process?
Some former students have been directly involved in the redesign. In particular, one undergraduate who took the course several years ago has continued on as an undergraduate learning assistant. During the past semester he pursued an independent study and developed some Web materials for students. His suggestions have been extremely helpful. Other former students have served as ULAs and have been involved directly in the redesign. Students who have taken the course have been extensively surveyed and have participated in focus groups. Their responses are helping guide us through the redesign in the most appropriate manner.
After the first offering of the course, some students were not pleased with the reduction of class lecture time from three hours to two hours a week. In the second offering, a third hour of lecture was reinstated, but it was used as a lab preparation period rather than as a regular lecture period.
What kinds of support for your project have you been receiving from your department or the institution more broadly?
The university has been very supportive of our project. Support in the form of upgraded space and the rehabilitation of that space has been provided. They will be moving the course lab from a small, narrow basement room to a large, centrally located, circular room that will be ideal for setting up a studio-type environment for the course lab sections. The University has added $30,000 of additional funding to improve the existing hardware so that we can service a larger number of students thus further reducing the cost per student over time. The success of this project has been made a priority within the University.
In addition, we have received technical support from many groups on campus. However, getting access to the individuals within the structure who can answer technology-based questions can sometimes be difficult. Our distributed support system has led to some initial confusion about where to go for which particular service.
Our largest disappointment has been the inability to use some of the cost savings to help support the computer labs necessary for the course. As the cost savings occurred at a time when the school was experiencing budget reductions, it was necessary to apply all of the cost savings to support other existing costs within the school.
Program in Course Redesign Quick Links: