Well, we are still in the midst of this difficult sprint that focuses on the Exam Editor. I believe we are in the final couple of days of it, though. We are obviously not time-boxing our effort in this sprint.
Although it has taken a bunch of extra time, the next exam editing functionality is really coming together nicely. Our internal demo a few days ago went quite well. We got some great feedback from our product sponsor, and we are incorporating most of those changes.
We have also finished the user import feature for portals, and I’m very happy with it. This will immediately help populate one of our new portals.
I have also finished writing some new on-line help materials for the exam editor, which should help ease the transition into the new exam building processes. We don’t have very much help on the site yet, but something is better than nothing.
I’ve got a ton to do to get this release ready, so that’s all for now. As usual, please feel free to comment.
Thomas Klassen
When you pull in the reins, you slow down or stop something that has been a bit out of control. That is what the development team is doing right now. OK, things are not completely out of control, but the current sprint is not working out as well as others.
As you may have noted in my 8/7/09 entry, this sprint is all about rewriting the exam builder and student exam user interfaces. We took on a large list of enhancements and improvements, some of which included some pretty advanced requirements. After some amount of research and trial and error, we have determined that we are going to temporarily drop a few of the sprint tasks for now and come back to them later, in the interest of capping off this sprint. We need to move on to other high priority needs. The main items we are putting on the backburner for now are the support of question groups and the ability for authors to order their questions.
So, what did I learn from this? Well, several things. One, I think we jammed too much into this single sprint. I think we tried to incorporate everything anyone had ever suggested for exams. They are all great ideas, but probably don’t all need to be completed right now. I also think we didn’t hold ourselves accountable for making and meeting estimates. We allowed a number of larger tasks to go un-estimated. We could have also broken up some of our larger tasks and perhaps created initial R&D tasks with limited durations. This would have time-boxed our R&D work or put the go or no-go decision in front of our faces each morning during scrum. Finally, the team has been pretty fragmented during the last few weeks – time off, working remotely, other priorities, etc.
So, as usual, there is a lot to learn from this experience.
Thomas Klassen
 One hot summer's day a Fox was strolling through an orchard till he came to a bunch of Grapes just ripening on a vine which had been trained over a lofty branch. "Just the thing to quench my thirst," quoth he. Drawing back a few paces, he took a run and a jump, and just missed the bunch. Turning round again with a One, Two, Three, he jumped up, but with no greater success. Again and again he tried after the tempting morsel, but at last had to give it up, and walked away with his nose in the air, saying: "I am sure they are sour."
Like the fox in the fable, early failure or lack of confidence that success is possible in a training program might cause a student to drop out and label the training program “sour.” One of the great challenges of eLearning is to overcome the dropout factor. Albert Bandura, Stanford University, wrote that behavior corresponds closely to level of self-efficacy change, regardless of the method by which self-efficacy is enhanced. Therefore people that have a bit of success will continue down the path while others that experience failure can give up trying because they seriously doubt that they can do what is required (American Psychologist 1982, 37(2)). According to Rossert and Chan, an effective eLearning program must provide opportunities for success, not failure or uncertainty (Engaging in the New eLearning). Taking an online course that is difficult and time-consuming and downs’t reinforce that learning is occurring can be discouraging and cause the learner to drop out. If your confidence takes a hit then motivation decreases and involvement will stop. As you design a program, design in ways to nurture self-efficacy in participants. Rather than testing participants to reveal how much they do not know, remind them of related prior knowledge and past successes. LearningZen provides multiple ways in which this issue can be addressed. Here are some examples:  LearningZen was designed for the seminar-type course not the 40+ hour university course. By creating smaller courses, students will not get bogged down in a long and consuming course, but have the opportunity for quick success and validation. - LearningZen provides the educator the opportunity to create post-course exams that are valid with less than a perfect passing score. This allows the student to achieve some success without perfection.
- The educator can allow tests to be taken multiple times in order to allow the student to continue to learn even though the first time through may not have provided the desired results.
- LearningZen will provide pre-tests in order for the student and teacher to see progress from before the course is taken. The pre-test also provides the student with an idea or roadmap to what content is important.
- Tests can be taken open book to allow the student further learning opportunities by reviewing material that may not be remembered.
- LearningZen is ideal for reinforcing concepts taught in the classroom with additional practice and reinforcement of important concepts.
We have watched as educators have used the flexibility of LearningZen to design courses that are very creative in how material is presented. Practice exercises, homework reviews, self-test and review, downloading of sample work, and many other ideas have come out that enhance self-efficacy. These creative solutions will help reduce the dropout rate and improve the learning opportunity for all.
Dennis Phillips, Ph.D.
When do you say enough is enough and plot a new course?
One of our developers reached this point yesterday when he was developing a new user interface for managing course exams. He was using a third-party control and had spent a ton of time getting it to handle multiple different types of data – questions, answers, and question narratives. It became apparent that the control’s way of managing data was not going to support our needs.
So, he decided to abandon that approach and adopt a simpler, more streamlined approach, using proven resources and approaches.
Sometimes you just have to muddle through these hard things. You have to slog through the mud of unknowns, uncertainties and untested solutions. People put their necks on the line when doing this and I’m grateful when they do. It shows that they are able to take risks and in spite of adversity and apparent lack of success, learn and go on. Also, when it is done with quiet integrity, self assurance, and not a bunch of ego being involved, it is a beautify thing to see. Thank you, Chris.
This reminds me of the minimalist principle and its application to software development – do just enough to meet the requirements and omit needless functionality and code.
This is my moment of Zen for today.
NOTE: LearningZen Co-Founder Dennis Phillips is writing a series of blogs with regard to Getting the Most from e-Learning (GMEL) This is number one in the series. The introductory blog can be found in the archives.
One of the great challenges of eLearning is to overcome the dropout factor. In the first article of this series, I discussed the need to create e-Learning content that is perceived as useful. But if the value is not obvious, a vivid case must be made for the course (Engaging in the New eLearning).
The potential student will ask one question that must be answered by the educator: Is this for me? If the answer to that question is not obvious, you will lose your student and the opportunity for learning.
In his book What would Google Do, author Jeff Jarvis states that “one benefit of the distributed, connected university is that students may select teachers. Instructors won’t be able to rest on tenure…but rise on merit.” (p.215) LearningZen provides numerous tools to create fantastic content. However, in reality, a piece of e-learning could include the best graphics, lots of interaction, numerous tests and content written by the most learned subject matter experts, if the learner themselves is not really interested in the topic, that course will more than likely not be completed. (Learning Technologies)
LearningZen provides two methods for vividly stating the case of value for a course. The first is the internal rating system. Each and every student that comes to LearningZen and takes a course has the opportunity to rate and comment on both the course and the author. For example, this course and author have been rated.
This simple system of 5-stars allows the value of a course to be seen immediately by potential students. This is the same concept that has proven so valuable to trusted vendors on eBay and Amazon. Over time, the best courses and the best instructors will rise to the top and the value will be obvious.
The second feature that allows the user to immediately see value is that LearningZen provides a template for course creation that shows the student right away what is included in the course. You've probably heard the following three sentences before. Though simplistic, they offer great advice.
- Tell your audience what you're going to tell them.
- Tell them.
- Then tell them what you told them.
Here is an example of a course that states the value of the course in the introduction. 
This is really a streamlined version of Aristotle's ideas on giving speeches (Presentations), and LearningZen has built the course creator tool around this idea. Each course includes an introduction, the content or chapters and a review or summary. While an Author can certainly choose not to follow this template, the template provides a pattern for showing the value of the course to potential students. Following this template will keep you on track as you prepare your course. It will make your students comfortable and secure because they'll know where you're headed. And it'll help keep you focused as you make your. It will also provide you a means of vividly showing the value in your course.
Dennis Phillips, Ph.D.
NOTE: LearningZen Co-Founder Dennis Phillips is writing a series of blogs with regard to Getting the Most from e-Learning (GMEL) This is number one in the series. The introductory blog can be found in the archives.
 A farmer, being on the point of death, wished to be sure that his children would give the same attention to his farm as he himself had given it. He called them to his bedside and said, "My children, there is a great treasure hid in one of my fields." The children, after his death, took their spades and mattocks and carefully dug over every portion of their land. They found no treasure, but the fields repaid their labor by an extraordinary and superabundant crop.
Just like the farmer, we educators want our students to be anxiously engaged in looking for great treasure. In order to do that, we need to create interesting and valuable content. One of the key issues with e-learning is overcoming the high dropout rate. There are no national statistics, but a recent report in the Chronicle for Higher Education found that institutions report dropout rates ranging from 20 to 50 percent for distance learners. And administrators of online courses concur dropout rates are often 10 to 20 percentage points higher in distance offerings than in their face-to-face counterparts (Frankola).
According to a recent white paper (Engaging in the New eLearning) the most important factor in making e-learning effective is the students “must see the value in what is available and what is asked of them—and they must spot that importance swiftly. There is little patience for murky benefits to be revealed in the future.”
Our eLearning programs must have obvious and immediate benefit or we will lose the attention of those we serve with our training. A teacher in a seminar, or a classroom, has a captive audience. The value can come slowly and the educator has multiple opportunities to catch the attention of the student. Not so in e-learning.
 At LearningZen, we have addressed this important factor in two ways. The first is the ability to prove that learning has taken place. LearningZen provides a certification to each and every student that completes a course and passes the associated exam. This is one of the distinguishing features of LearningZen. Obviously a certificate is no more valuable than the certifying body. However, if your company or teacher recognizes the certification and places a duly earned certificate in your personnel folder or your report card, that is true value and will encourage you to continue with the course to completion.
 The second way that we have addressed this concern is by providing the educators with a plethora of tools to make your courses as rich as possible. Courses that have been published on LearningZen are not just plain HTML text that you might see on Wikipedia or other repositories of information. Rather the educators have been creative in using multiple fonts, text sizes and styles. In addition, courses contain images, audio, video, flash and many other documents. There is really no limitation to the quality and interest that a course can have outside of the educator’s ability and willingness to be creative.
Like the farmer in the fable at the beginning of the blog, I say “There is gold in them there hills!” And we hope our students will diligently search for it. Have fun designing great classes.
Dennis Phillips, Ph.D.
I am very excited about the new functionality we are building in our current development sprint. Here is a quick rundown of those enhancements.
Directly responding to our users’ requests, the major focus of this sprint is to rewrite the exam portion of courses, from both author and student perspectives. For course authors:
- We are adding the ability for authors to include pretests in their courses. This will enable authors to measure how much learning and comprehension is happening in their courses.
- One of the largest changes to exams relates to the entire experience of creating exam questions and exams. We are creating an enhanced authoring experience that is going to be easier to use and will function in more of a wizard-like manner. The exam builder will step authors through the process of creating questions and answers.
- We will change the definition of courses a bit, with regards to the requirement that all courses have exams. Authors will still be able to create courses with exams, but they will also have new options to either create courses without exams, or to create “courses” that contain only exams.
- Exam questions will now support HTML markup and images, using the same editor interface we use throughout the site.
- Exams will now support the grouping of questions, allowing authors to present a scenario or description before asking one or more questions.
- We are enhancing the way multiple choice questions are created. Specifically, we are adding built-in options for “all of the above” and “none of the above” options. This will facilitate the ease and speed of creating these types of questions.
- Exam questions will have an option for an explanation to students. This will allow authors to explain their question in detail and to provide supportive information relating to the correct answer.
Course and exam enhancements for students include:
- The display of which questions were incorrectly answered. This has been a frequent request and we think this will help students gain a greater understanding of what the author is presenting. The display of incorrect questions will be something that authors can control on a course-by-course basis.
- Students will see exam question explanations that were supplied by the author.
- Some “courses” may not contain an exam, and some may not contain any course material. This functionality makes sense when the course material is only available outside of LearningZen, but the mastery of that material needs to be proven by the student.
- Authors will have the ability to describe their exams’ relatively difficulty and expected time requirements for completion. This will help students make decisions about signing up for specific courses and decisions as to whether they want to purchase them.
- Finally, if the course is set up for a pretest, the student will be required to complete the pretest prior to entering the course material.
If all of that were not enough, we are also working on the following other enhancements in this sprint:
- We know that the ability to import Microsoft PowerPoint and Word files has been requested by many users. This is a high priority for us as well. We are researching the best ways to accomplish this important feature.
- We are working on a new feature that would allow a course author to share his or her course with others during the authoring process, in a collaborative manner. This feature will be implemented in a minimal way to start. I’m sure we will end up building out this feature more completely in future sprints. This feature is one in which I would like to get more feedback, so please share you thoughts.
- We are adding functionality to highlight new courses on the site. We will be showcasing them in a new list as well as with the use of RSS feeds. Users of LearningZen will be able to subscribe to this feed and several others. Stay tuned for even more feeds soon.
- Finally, we are adding the ability to do a mass import of users into a Premium Services portal, which should dramatically improve the process of authorizing users for portals.
Holy cow that is a lot a new stuff! I’m even more excited about this sprint than I was before starting this blog entry. Our development team is really working hard to enhance this great site. As usual, I am interested in your feedback. Please share your thoughts by commenting on this blog entry.
Thomas Klassen
 The fable is told of a crow that was perishing with thirst. As he flew over the countryside he saw a pitcher, and hoping to find water, flew to it with delight. When he reached it, he discovered to his grief that it contained so little water that he could not possibly get at it. He tried everything he could think of to reach the water, but all his efforts were in vain. At last he collected as many pebbles as he could carry and dropped them one by one with his beak into the pitcher, until he brought the water within his reach and thus saved his life.
For those that are responsible for training and development, we know that there are a plethora of e-learning tools available for use. We have social networking, wikis, twitter, blogs, learning management systems, virtual environments, and the list goes on and on. We literally can see the water at the bottom of the pitcher and know that it will quench our thirst if we could just get at it. But having access to the water and knowing how to drink it is a different story. We can deliver training and information at will but is it effective and will it “quench the thirst?” I recently read a white paper written by Allison Rossett, long-time professor of educational technology at San Diego State University and Antonia Chan, a Fulbright scholar from Panama. You can find the white paper at Engaging in the New eLearning. In the paper, the authors discuss what we can do to make our e-learning efforts effective. They outline 12 different factors to consider when creating courses or programs. I think these are the pebbles that if used, will allow your content to quench the thirst for effective e-learning. They are:
- The e-Learning must be perceived as useful by participants.
- If value is not obvious, a vivid case must be made.
- The program must provide opportunities for success, not failure or uncertainty.
- Make it real.
- Since the new e-Learning relies on involvement and generosity, reveal what that participation might look like.
- Make it active and thoughtful.
- Showcase people, emotions, and successes.
- Guide and track participants.
- Situate e-Learning within a blend.
- Make relationships, collaboration, and teaming a part of the effort.
- Make it WOW.
- Measure and continuously improve.
We have seen numerous courses published in LearningZen. At last count, we were over 90 courses with at least twice that in some stage of development. I have created many courses myself some better than others. In my efforts, I have found several tricks that can perhaps assist you in creation of effective content. Over the next few weeks, I will share with you examples of courses and techniques that address each of these 12 points listed above. We realize that creation of content is really the difficult and talent driven task associated with LearningZen and want to do our best to provide resources that will allow you to use this tool to create effective e-learning.
Dennis Phillips, Ph.D.
|
|
