Jump to content - Jump to section navigation
Course Management Systems (CMS) products are relatively new to higher education and many institutions are struggling to determine the best way to support faculty use of these technologies in teaching and learning. More importantly, higher education has a need to determine if these systems provide significant benefits to the teaching and learning process for faculty and students. Project cms@wbw.edu was undertaken to establish a uniform methodology to evaluate CMS use and effectiveness at three institutions: Wesleyan University, Brandeis University, and Williams College.
Project cms@wbw.edu engaged the senior partners from Eduworks to develop an evaluative framework in which to assess the benefits and costs of CMS products (including open source ones) after they have been purchased and implemented. In the larger context of the use of technology in teaching and learning, the objective was to determine which elements of a selection model hold up after purchase and implementation. The framework was intended to help institutions answer tactical questions: Is ease of use mainly a faculty concern or does it apply to students as well? Is a CMS product with basic functionality "good enough?" It also provides input to larger strategic questions: Is cost now the primary driver for selecting a CMS? Does faculty satisfaction outweigh student satisfaction in deciding the future of an established CMS?
In the ECAR Research Bulletin, Selecting Course Management Software to Meet the Requirements of Faculty and Students, Volume 2002, Issue 8, April 16, 2002, Halloran argues that most recent CMS studies have focused on checklists of what the products can do, not necessarily how well they perform. The Bulletin article outlined a methodology "to evaluate the usability and usefulness of course management software from both the faculty and student perspectives" that the author developed in a series of experiments involving faculty and students. We proposed to develop an assessment methodology following the selection-process work of Halloran to determine how our choice had stood the test of time, and to evaluate the benefits and corresponding costs of continuing with WebCT and Blackboard.
Wesleyan, Brandeis, and Williams (WBW) all implemented a CMS product, each through a different selection and implementation process. Wesleyan and Brandeis were in the process of re-evaluating their choices. Brandeis University had until September 2003 to resolve its commitment to WebCT, which meant a decision about which CMS to support going forward had to be made by May 2003 at the latest to be ready for the 2003 fall term. Williams was comfortable with where they were, and was looking at this project as an additional method for evaluating the impact of their choice and to look for ways improve their use of the product.
With the recent announced increases in the cost of CMS software, the requirement for an assessment methodology comes to the fore. The cms@wbw.edu project developed a framework that can be used to assess what is working well and what is not working well in the context of implemented CMS products and their associated features and costs. The varied WBW deployments of CMS provided different experiences on which to develop a methodology that can also be adapted by other institutions involved in CMS assessments.
To see the final report of the Project cms@wbw Pilot Study click here: cms@wbw report.pdf
Questions or comments about Project cms@wbw.edu? Contact Dave Wedaman - wedaman@brandeis.edu
