LMS Choice 2006
	
	
aaatimed quiztimed assignmentessay quizpowerpointfeedback on assignmenthidden filesingle filetest forumessay quiz 2Anonymous Surveycool siteuploaded web pageeasier to do
	
 
Moodle_Practice È LMS È Resources È LMS Report to ET - USE THIS
 
LMS Report to ET
INTRODUCTION
This report describes the evaluation process for selecting a Learning Management System (LMS) to support the development and delivery of online, hybrid, and web-enhanced courses at Lane. After discussing the definition of an LMS, it's history at LCC and the rationale for choosing a single system to support now, we will explain the selection process, present our findings (including input from the various stakeholders), and conclude with our final recommendations.
WHAT IS AN LMS?
A Learning Management System (LMS) refers to an integrated set of networked, computerized tools that support online learning (Virtual Learning Environment (VLE) or Course Management System (CMS) are other terms that are sometimes used). Typically, these systems include tools like discussion lists and chat rooms for online communication, automated quizzes and other assessment tools, tools that make it easier to prepare content for online presentation and some tools to monitor and manage student activities.
HISTORY
In 1997, the Strategic Learning Initiative (SLI) began the Instructional Technology Infrastructure (ITI) project, today known as Lane's faculty webmasters. ITI funded two science instructors, Stan Swank and Joe Russin, to evaluate a number of LMS systems available at that time. The two of them chose WebCT and that became Lane's default system for online courses. These two instructors should be commended for choosing a system which did not become obsolete during the next 8 years.

WebCTtraining has received continuous support at Lane, from the early years when SLI ITI Faculty Webmasters were responsible, until the last 4 or 5 years when ITC Coordinator has assumed those duties. After 8 years as the default system for online instruction, only 44 instructors have WebCT accounts, not all of which are active. Last Spring Term 2005, instructors in just over half of Lane's 29 online classes used WebCT as the LMS.

Just prior to Fall Term 2004 Moodle was installed and in just over a year, 75 instructors have requested accounts or have attended inservice and Faculty Webmaster training on this LMS.

I counted 22 WebCT courses, 8 Moodle, 2 BBs, 2 website and 1 slacker with an email, phone and, yes, FAX! On the plus side students weren't required to purchase an abacus or sliderule at the bookstore.

I think it'd be fine if we left this section alone, or we could compare new, total or active WebCT and Moodle accounts here (I guess Joe and Dave would be the ones with those numbers).

>>>mkw The above history accounting is not presented in a factual manner. You are already showing a bais for Moodle.

WHY CHOOSE ONE LMS TO SUPPORT NOW?
Lane needs to grow its online offerings to better meet student demand:
Lane's current online offerings are few, when compared to similar community colleges in the state. In March 2005, the faculty webmasters reported to Lane's Executive Team (ET) that while Chemeketa and PCC were offering 205 and 265 online classes respectively for the Spring term. ity within the first few days of registration.

>>>>mkw Did something get cut here? Is the point to compare us with other colleges or talk about the student demand?

LMS use is expected to increase through Hybrid scheduling.
A hybrid is a course which combines distance learning with traditional classroom activities. Hybrid courses have some in-class sessions, while other classroom hours are replaced with online interaction, assignments and projects. Surveys show that students prefer this blend of online and classroom instruction to purely online classes, and the administration has given the green light for faculty to begin offering such classes, which which will allow enrollment growth without increasing classroom allocation problems.

>>>>mkw This speaks more to student demand than the previous paragraph. Should we drop reference to DL?

Lane's offerings are few and the current cost of converting to another system are currently minimal:
If we are going to begin supporting another LMS, now is the time to do it, when our program is small, not when we have five or ten times the course offerings we currently have. Conversion of a program that size a few years down the line would be much more difficult and expensive.

The scope of LMS use at Lane is changing:
Currently, faculty webmasters are responsible for supplying faculty with training, as well as continuing to explore technological innovations in instruction and learning. What was not possible years ago is possible today. We expect Banner CRN and Student data to be used to create a "course website" for every LCC instructor, for every CRN in the class schedule, and to populate those LMS websites with student data. All three LMS vendors that we considered can accomplish this task. This direction will greatly aide in faculty training in technology use for their classes. The move to a single LMS is supported by the Information Technology (IT) department and will allow them to offer a greater level of support for faculty as well as technical help for students via the new Student Help Desk (SHeD).

>>>> mkw Good points. We may want to mention the pharse 'In the future..' before the sentence We expect Banner CRN and Student data to be used to create a "course website" for every LCC instructor. David mentioned this in NOT something that would happen in the immediate future.

Blackboard has recently purchased WebCT and major changes are on the way:
In the Fall of 2005, LMS vendor Blackboard bought out WebCT. Lane's WebCT instructors are currently using version 4.1 of the software, although the current version is 6.1. WebCT and Blackboard claim they will continue to sell and support each of their own products until a new product, combining the two systems, is developed in the next 2 or 3 years. Since whatever system we decide to support will mean major changes for the college in the coming years, now is the best time to evaluate whether to continue with WebCT.

Rising Licensing Costs:
Licensing of software from commercial vendors WebCT and Blackboard will cost, at minimum (based on current, January 2006 quotes) $35,000 and $63,000 a year. As an open-source product, Moodle has no licensing costs. While basic support for WebCT and Blackboard is included in their licensing, and basic Moodle support is supplied on the Moodle.org website free of charge, advanced technical support can be purchased for all three systems, ranging from an hourly to a yearly support contract that ranges in cost from $2000 to $15,000 a year. Lane has never needed to purchase extended technical support for either WebCT or Moodle over the last 8 years that Lane has used an LMS system. In the light of recent budget difficulties, it is important to consider whether we want to be locked into increasing fees from the WebCT or Blackboard. It is likely that WebCT acquisition by Blackboard will significantly reduce previous competitive pressure to keep licensing fees low.

HOW THE SYSTEMS TO BE EVALUATED WERE SELECTED

In coordination with the IT Department, we chose to evaluate three LMS packages: Our oldest current system, WebCT, our newest current system, Moodle, and Blackboard, mainly because the system is in use at the University of Oregon. In addition, both WebCT and Blackboard are the largest commercial vendors of LMS software, most community colleges in the state of Oregon are using one of the two systems. These two products are based on Java programming language technology. Moodle was chosen as the largest open-source LMS system which most closely matches WebCT and Blackboard, feature for feature. Moodle's use began at LCC in the Fall of 2004 as a Faculty Webmaster innovation. Moodle is based on "LAMP" technology. (Linux, Apache, Mysql, and PHP - opensource technologies that drive over 70% of the world's websites). As can be seen from the following chart, BlackBoard, WebCT and Moodle represent the vast majority of the higher education market.

JOE: The pie chart is a cool graphic, but misleading--implies that a the pie only contains these four vendors (i.e. no Angel, other LMS's). Need some good numbers on total market to use a pie chart, with numbers for Angel or simply "Other". Otherwise, we could use a bar graph comparing the numbers we know.
>>> mkw Agreed, I like the pie chart, show 'other'.

    * Moodle: 2,981/ 54%
          o 8,772 X (348 / 1024) 
    * Blackboard + WebCT: 2,500 / 45%
          o 3,700 - 1,200 K-12 
    * Sakai: 35 / .63%


                + moodle chart


We ruled out two system: Angel and Sakai
>>>mkw ...system(s)

Angel (http://www.angellearning.com)is a newer, up-and-coming LMS system which has received some favorable press. It is a commercial system with yearly licensing cost, but it is based on .asp technology (a declining technology, in our opinion), a Windows-only server platform and Windows-only SQL database server.

Sakai (http://www.sakaiproject.org) is another open-community project that has received some press in education cirlcles. It is based on Java and the collaborative effort of a number of prestigious 4-year universities with the help of a large donation from the Mellon Foundation. It began in January, 2004, but as of October, 2005, (faculty webmasters attended CIT conference in Dallas, Texas and spoke to Sakai partners) it was considered "not ready" for adoption or production by schools other than the 10 official Sakai partner schools. There is an annual charge to be a contributing Sakai partner school, and no charge to simply use the software.

THE PROCESS

WebCT, Blackboard and Moodle representatives were each invited to do a two-hour presentation for faculty and IT staff in the LCC Boardroom. The presentations were advertised in the Daily, via several campus and group-wide emails, and by word-of-mouth encouragement from faculty webmasters. Dean Middleton, Multimedia and Video Specialist from the Distance Learning Department, was recruited to tape each demonstration so it could be seen by faculty who were unable to attend. Announcements were made in The Daily about the availability of the tapes, and faculty who either attended the sessions or viewed the tapes were encouraged to provide feedback via an online survey form. Finally, a general call for feedback from faculty was given, regardless of whether they had seen any of the presentations.
FEEDBACK FROM FACULTY AND TECHNICAL SUPPORT STAFF

Faculty thought the least of Blackboard. Comments included that it was "not intuitive for someone who for someone who did not have a technical background", "slow and clunky", "cumbersome, hard to understand" and "awkward complex and difficult". One instructor summed up Blackboard's problems as having "too many buttons to push and no clear pathways from one function to the next."

There were also comments about the poor quality of the Blackboard presentation and that it may not bode well for quality of support we might expect, as the presentation was from the regional sales director for our area. Licensing cost and not being able to customize the look and feel of individual courses were also cited as concerns. One faculty member commented that if we were going to go with a corporate "cookie-cutter" solution that we would be better off staying with WebCT, since instructors at Lane are familiar with it.

Comments about WebCT were much more favorable. A repeated comment was that WebCT had improved or the past few years and that the new version seems even more user-friendly, that WebCT sites were easy to navigate with its icon-view, and that man "canned" courses were available for it. Several comments were made about WebCT's gradebook being especially good. Concerns included apprehension about it being bought out by Blackboard and likely increases in licensing fees with increasing budget problems at the college. There were also a few instructors who said they found WebCT difficult. Some reported hearing about problems with WebCT from other instructors in their department and as a consequences starting with Moodle. Others reported switching from WebCT to Moodle and reported finding Moodle easier to use.

>>>>mkw What is ...that man "canned" courses

Moodle garnered the most written comments, which were overwhelmingly positive:
"It seemed easy to use and dirt cheap."

"Moodle is beautifully flexible."

"WebCT and BlackBoard both make course monitoring so difficult that I gave up doing it. Moodle makes it simple..."

"I prefer Moodle because I think it is easier to use for both instrcutors and students."

One instructor who is using WebCT to teach a class at Linfield and is using Moodle at LCC commented, "The organization from the home page is clean and understandable. Once I turn editing on, everything is in front of me, easy to understand, providing a palette of options...I am a Moodle Convert!"

Another current Moodle user writes, "My students have had no difficultings in accessing the course, in downloading/uploading assignments, in completing the exams, in receiving feedback and so forth...To date everything is working well."

Yet another faculty member summed it up this way, "I have compared and contrasted all three systems head to head...in the online classroom, and Moodle is far superior to any other for sheer ease of use, richness of features and flexibility."

Faculty liked the option of a Weekly, Topical or Social (blog-style) layout, the variety of options in the Forums, including graded forums, the weighted grade feature in the gradebook, the Calendar and wikis (collaborative web documents with trackable version changes). The large online support community and the fact that it is open source software were also cited as pluses. Aside from the obvious cost-savings, it was suggested that implementation of new features would be done more quickly than with the product of a large corporation. It was also suggested that money otherwise spent on licensing fees could be used to fund in-house technical support.

From the technical standpoint, one IT staff member wrote the following:
"I feel that Moodle compares well with WebCT and Blackboard. As I'm not an instructor, I don't feel that I can speak to the pedagogical aspects of these systems. However, from a technical perspective, Moodle seems like a sound platform that LCC could grow into a fairly full featured LMS system for the entire campus."

The only substantial criticism of Moodle was its lack of a whiteboard to go with its web-based chat room. Recurring positive themes were ease of use for both faculty and students (which faculty rated as the single most important factor in selecting an LMS), richness of the feature set, customizability and cost savings. It should be noted that most users who had used both Moodle and the other systems preferred Moodle for reasons having little or nothing to do with cost savings, which was often regarded as a fringe benefit.

>>>>mkw You mentioned downsides to WebCt and BB but not any to Moodle. The main concern of faculty seems to be the conversion factor and learning to use a new system.

TECHNICAL QUESTIONS PROVIDED BY LANE'S IT DEPARTMENT

Technical questions for LMS vendors


1. What platforms can the server run on (Windows, Novell, Mac OS X, what varieties of Linux)? What platform is most commonly used for the server, and which do you recommend?

3. What back-end databases does your solution work with (e.g., Oracle, Microsoft SQL Server, BDB. PostGres, MySQL)? What versions are allowable or required (e.g., Oracle 9 or 10)? Can the database backend run on the application server, or does it require a separate machine?

2. What are the minimum and recommended hardware configurations for running your solution? How well does your product work in a clustered environment (for example, two- or three-node blade configurations with physically attached SAN storage)? What are the minimum disk requirements?

3. How well does your product integrate with standard Student Information Systems (e.g., SCT Banner)? What products are required for such integration? Is it possible to set up live links (with two-way integration), or is the integration a snap-shot system?

4. How well does your product integrate with portal productsÑeither open-source solutions like uPortal and Plone, or commercial solutions like SCT Luminis? Are any third-party products required to set up such integration? In your experience, what resources (staff time, training, expertise) are required for such a project?

5. How well does your product work with open standards like LDAP and Kerberos for authentication? Is it possible for an LDAP server to provide both authentication and course and registration data? If so, what schema modifications are typically required to do this?

6. Does your product allow for an independent database of users for test purposes? For example, if LDAP authentication is used to authenticate students and instructors against an external server, is it possible to enable additional, ÒlocalÓ accounts that are not in that server, for test purposes?

7. Does your solution allow for live backup of student and class data, or does the application have to be taken down for a complete backup?

8. What command-line interfaces are available for manipulating objects in this productÕs database (e.g., are there ways to script mass import of user and creation and deletion of classes, etc)? Can these tools be used while the application is live, or must it be taken off-line?

9. What provisions are there for moving user and course databases from one server to another, in case the primary server fails? Can we transparently back up the database on the primary server and restore it to a backup server?

10. If your product requires licensing or product activation codes, must we purchase a separate license if we wish to keep a near-line backup on a second machine that is running your application?

>>>>mkw I assume the above is the info coming in from David
FINAL RECOMMENDATIONS
(Mere and Jeffrey - Jim Salt has asked that we present this report as a recommendation to be accepted or rejected by ET, thus limiting their ability to micro-manage an issue on which they have no true expertise. I concur with Jim)

We ask the LCC Executive Team for your approval or rejection of the entire list of recommendations:

#1 - That Lane officially adopt Moodle as the LMS of choice, to begin Fall Term 2006.

#2 - That Moodle be supported by Lane's IT department on both the technical end and student support end (both of these which are already in place), and eventually, after a transition year, the staff training portion.


#3 - That LCC cease it's use of WebCT by the start of Fall Term 2006. At the end of this year, both Moodle and WebCT will have been used concurrently for 2 years. We see no advantages to continuing the transition for another year. In fact, we see only liabilities.

Continuing to support both would:

    * Cost the college $9000 for the WebCT license for 2006/7.
    * Double the work of the IT department technical support person to support both systems.
    * Require that the IT department incur hardware costs by having to purchase an addional server.
    * Hinder the faculty webmasters' ability to develop and implement a Moodle training program for the 2006-7 year.
    * Not result in a quicker conversion of courses and instructors from WebCT to Moodle.

This process began in Spring Term 2005. There is no reason why it cannot be implemented in Fall Term 2006.

#4 - If possible, utilize some of the $9000 in WebCT software license savings to aide in the transition.

On a related note, Lane will be encouraging the development of hybrid courses, which have many benefits, some being financial, such as the reduction of need for classroom, especially expensive smart-classroom space. In order to proceed with hybrid class promotion, Lane should have a campus-wide LMS implemented which can support ALL of Lane's course offerings. A move in this direction will also reduce staff training costs as an LMS system can do what perhaps 95% of faculty need to do with their courses.
>>>mkw Not sure what you mean here about reducing staff training costs.

The cheapest cost to accomplish this with a commercial LMS vendor is with WebCT at over $33,000 a year in on ongoing licensing costs. These costs have doubled in the past 8 years, when Lane first purchased a WebCT license. Considering Lane's troubled financial state, getting the same service from an opensource LMS with no licensing costs is a prudent decision.

We ask that this decision and official announcement happen as soon as possible to allow the faculty webmasters to begin training and supporting faculty who must make the transition from WebCT to Moodle. This can occur over the Spring and Summer Terms, 2006, just in time for a Fall Term 2006 start.






---------------------------------------------
SOME MISC COMMENTS BELOW

While Lane does not have any current Java-specific in-house expertise, it does have in-house LAMP expertise.


LCC is in a position where external forces (WebCT software changes, higher licensing costs, acquisition by Blackboard) require a major, unplanned change and a major reallocation of resources to support the change.

In contrast to the brief evaluation in 1997, in 2006 there was a much wider selection of sophisticated LMS products on the market and LCC had made a significant investment in the use of an LMS, both in monetary terms and in the time and commitment of academic and support staff in preparing online resources.

Three years of experience had exposed shortcomings of the system being used, as well as the ÔriskÕ attached to a commitment to the use of an Ôoff the shelfÕ LMS. However, there was still a growing demand for the benefits offered by an LMS in the development of online courses. This evaluation did not seek to make judgements about the effectiveness of an LMS as a means for improving the effectiveness of learning in higher education; it followed the approach of Dutton, Cheong and Park (2004), whose case study inquires into the Òvariety of technical, institutional, social and economic constraints on the innovation processÓ (p. 69) from one major enterprise LMS to another.

>>>mkw Lost me on this one. What is the main point?



*************************************************************

OTHER USEFUL (OR NOT) VERBAGE BELOW
>>> mkw Not. smile I did not wade through the info below

THE EVALUATION APPROACH AND ITS INTERPRETATION

METHOD

As described earlier, the University faced a version-change WebCT in the short-term. This would have involved some installation work and systems conversion work for technicians but little work in conversion of content. However, WebCT indicated that the life of the new version could be limited and that an upgrade to a more expensive enterprise system (Vista)
would be required. These prospective changes, the trial nature of the original commitment to WebCT, pressure from some Faculties to move to other LMSs, problems experienced with WebCT in the CQU context and the fact that staff had developed some experience with the operation of a LMS led to a feeling among current LMS users that the University should conduct a rigorous evaluation of LMSs including an assessment of the issues and costs involved in changing to another LMS.

TIMEFRAME

During the initial stages of the evaluation, project managers identified a shortlist of four LMSs for detailed evaluation based on lists of essential criteria developed by the four interest groups. Initial work with representatives from these groups established project milestones that would provide technical staff with time to install and test the new system,
provide time for a trial to develop and deliver a small number of courses to students and provide time to convert all existing online courses to the new LMS for delivery in 2004. Only two months could be allowed for the evaluation of shortlisted LMSs, with another month set aside for reporting and the final decision to be reached.

STAKEHOLDERS IN THE CHANGE PROCESS

While senior management of the University was informed early in 2002 that a decision had to be made about choosing an LMS for 2004, no priority for action was given until the issue was raised at a staff development forum in November of that year. Teaching staff indicated that they required at least 12 months notice of such a change to allow them to develop appropriate skills and resources for new courses. This illustrates the different perspectives
and priorities that can arise between what Luck et al. (2004) call the management subculture and the academic sub-culture.
Pressure for change came from the grass-roots users of the LMS. To harness the energy for change and to build commitment to the final decision, project managers decided to involve critical stakeholder user-groups in a participatory action evaluation process to conduct a summative evaluation of commercially available LMSs and to make a recommendation to senior management to guide the decision to choose the University LMS for 2004 and some time into the future. The project managerÕs task was to involve representatives of the four critical stakeholder groups (academic, technical, development support and management) in the evaluation and decision-making process at three levels:
1. In the plenary group with representatives of all four groups.
2. In each of the four groups as they conducted separate evaluations of their
facet of the four shortlisted LMSs.
3. In the final collaborative process in which the four groups would justify and develop a recommendation with active involvement of senior management
who carried final responsibility for the selection of the LMS.
To make best use of the available time, each interest group worked largely independently to develop its own evaluation criteria appropriate for its area of expertise, and implement suitable testing and evaluation processes for the four shortlisted LMSs.
A team of computer administrators looked at technical issues such as the reliability and robustness of the software, technical support and the LMSsÕ compatibility with other CQU computer systems and industry technical standards. CQUÕs multimedia development unit investigated the potential of each LMS for specialised development, IMS standards and implications for conversion of a large number of courses currently existing in WebCT. The
administration group investigated contract conditions, pricing, commercial viability of the vendor, planned version changes and related commercial matters. The fourth group consisted of teaching academics and focussed on a range of issues that related to teaching and learning. The timing of the evaluation made it unfeasible to include students as members of this working party, but the teaching staff undertook to represent this group.
While the management, technical and development stakeholder sub-cultures consisted of relatively small numbers of people, the academic issues group, representing the interests of a large group of disparate users, had first to consult academic staff to determine what issues were seen as important and to promote ownership of the outcome.
The effort required by teaching staff to learn to use a different LMS application was one concern which has since been identified as a factor in reducing academicsÕ motivation to teach online (Dutton et al., 2004). The academic working party evaluation team identified this as a major issue that was particularly of concern to those who had spent considerable time developing skills in WebCT. A number of institutional changes that had occurred at CQU in the preceding years had left the staff change-weary (Luck et al., 2004) and there were concerns about whether staff would support yet another change. Thus ease of use was one of the key criteria that emerged from this group.

DECISION-MAKING PROCESS

Evaluations involved a range of methods including where appropriate tests and trials of the LMSs, surveys, consultations with other users and vendor presentations. Each group prepared a written report indicating the evaluation process and criteria used, the justification of the criteria, the ranking of each LMS and the justification of the ranking. All parties then met with senior management to discuss the issues and to debate the advantages and disadvantages of various options, including the option to continue with the
new version of WebCT. The final decision, to change to a different LMS, Blackboard, was made late in March 2003.
Since the decision, installation and trials have proceeded successfully, with Blackboard in full operation for the March-June 2004 teaching period. Formative evaluation processes, including research and user forums, development teams and staff development for academic staff, are supporting current and future developments.

DISCUSSION OF REFLECTIONS ON THE PROJECT
Critical reflection on practice is an important element in the action evaluation process (Wadsworth, 1991). Organisational lessons learned by the evaluation team from this participatory evaluation process are outlined in the following paragraphs.

CONCLUSION

The CQU experience shows that changing an LMS has major consequences for academic, administration, technical and multimedia development staff. Evaluation to choose a new LMS should draw on the experience of all these groups and provides an opportunity for these groups to understand better the perspectives of the other groups. Substantial university resources are absorbed by such a change, yet the need to change may be beyond
the UniversityÕs control. Institutions should monitor the health and intentions of the vendor of their LMS software to obtain warning of change and manage risks associated with a sudden change of LMS. OwensÕ (1998) sociotechnical systems model highlights the need to involve the critical organisational units in evaluation and decision-making. It also draws attention to the impact that technology changes will continue to have on teaching tasks, university
structures, participants and the goals of the university as LMS software and the web continue to develop. In this dynamic technological environment, the choice of technology is never a ÔfinalÕ decision leading to a separate implementation phase. New problems can be expected to arise as the different interest groups learn to use new and more complex LMSs
and identify strengths and weaknesses that were not apparent in the initial evaluation or have been introduced in upgrades. At CQU, regular user forums involving all the participant groups are conducted in a participative, formative evaluation process to address ongoing software and organisational change and to improve the LMS.



What if I wanted to add a chart?

not real

And the report continues...



Last modified: Wednesday, 22 February 2006, 08:37 PM
You are logged in as Joe Escobar (Logout)
LMS