Online Learning Works Teaching a Complex Subject using PBL

Distance Learning is as Effective as Classroom Learning

In general, many studies have shown that online learning is as effective as classroom learning.    The most common finding when comparing classroom to online learning is that they’re essentially equivalent if they’ve been carefully designed.  This has been shown to be true for many different kinds of learning experiences, but what about a highly interactive kinds of learning?  

A New Study Compares Online to Classroom Learning of Statistics

Does online learning work for learning statistical methods using a highly interactive methods like problem-based learning (PBL)?  That’s the question asked in a new paper published in Advances in health Sciences Education. (1)

The learners were enrolled in a Master’s degree program in public health and were learning multilevel and exploratory factor analysis.  

The classroom version of the course consisted of lectures, guided practice using statistical software, and PBL tutorials.  

Online learning used video lectures, individual practice with statistical software, and asynchronous PBL using discussion boards.  

Students’ experiences (positive or negative) were elicited after the course and their scores on the class final exam were compared.  

Measures indicated no significant differences between the two groups on examination scores.   

Some Advantages to Asynchronous PBL Discussions

Tutors and students noted some advantages of the asynchronous PBL discussions.  Tutors found that the online PBL took a little more time, to write feedback rather than delivering immediate verbal feedback.

Tutors like having the ability to monitor what students were doing and who was contributing the most to each discussion.  

Overall, the results of the study are consistent with many other studies showing that online learning can be as effective as classroom-based learning, even for a highly complex subject. 




de Jong, N., Versteen, D.M.L., Tan, F.E.S., and O'Connor, S.J., 2012, A comparison of classroom and online asynchronous problem-based learning for students undertaking statistics training as part of a Public Health masters degree. Advances in health Sciences Education (epublication).  

Practice is Important for Learning - How we Practice may Also Be Important

I miss your heartbeat.  Uploaded to Flickr by digitalnoise

Wired.com recently ran an article called “Everything You Thought You Knew About Learning is Wrong.”  The article was popular and was re-posted by many different people interested in learning and education.  The article focused on an interview with Robert Bjork, a UCLA psychology professor and the Director of the UCLA Learning and Forgetting Lab.  

One of Bjork’s main themes was the importance of “interleaving,”  that is working on several skills at once rather than focusing on a single skill.  

“First, he told me, think about how you attack a pile of study material.

“People tend to try to learn in blocks,” Bjork said. “Mastering one thing before moving on to the next.”

Instead of doing that Bjork recommends interleaving. The strategy suggest that instead of spending an hour working on your tennis serve, you mix in a range of skills like backhands, volleys, overhead smashes, and footwork.

“This creates a sense of difficulty,” Bjork said. “And people tend not to notice the immediate effects of learning.”

Instead of making an appreciable leap forward with your serving ability after a session of focused practice, interleaving forces you to make nearly imperceptible steps forward with many skills. But over time, the sum of these small steps is much greater than the sum of the leaps you would have taken if you’d spent the same amount of time mastering each skill in its turn.”

Numerous studies support the positive effects of practicing different skills, rather than simply practicing a single skills then moving on.  


Interleaving (mixed) practice also works for learning in medicine
A paper in Advances in Health Sciences Education looked at the effect of mixed practice (interleaving) in learning to electrocardiogram (ECG) diagnosis.  


To diagnose an ECG, students have to be able to correctly identify the the features on the ECG and to know the rules for each cardiac diagnosis (for instance left ventricular hypertrophy).  

Traditionally, students have learned and practiced these diagnoses one at a time.  Once they had mastered a diagnosis, they moved on to a new one, with a new learning and practice cycle.  


Study Design - groups differ in timing of practice
The study used a 2 hour learning session for both groups.  Each of the groups got the same instruction on five different ECG diagnoses, but they differed in the timing and presentation of the practice materials.  
 

Control Group	Contrastive Practice
Topic 1 Instruction Practice Topic 2 Instruction Practice Topic 3 Instruction Practice Topic 4 Instruction Practice Topic 5 Instruction Practice	Topic 1 Instruction Topic 2 Instruction Topic 3 Instruction Topic 4 Instruction Topic 5 Instruction Topics 1-5 Mixed Practice  Students encouraged to compare and contrast patterns  across diagnoses.

Study outcomes - mixed practice works
Students were tested on a six-item diagnostic test.  The group receiving contrastive practice had an over 50% increase in accurate diagnoses (1.8 for the control group v. 2.8 for the contrastive practice group).



Hatala, R.M., Brooks, L.R., and Norman, G.R. (2003).  Practice makes perfect: The critical role of mixed practice in the acquisition of ECG interpretation skills.  Advances in Health Science Education, 8:17, 17-26.  

Picture downloaded from Flickr (Christina Care)

Simulation training is widely used for surgical training and it seems to be very well suited for learning these kinds of procedural skills.  In a new paper in the Annals of Surgery (Zendejas, B, Cook DA, Bingener, J, Huebner, M, Dunn, WF, Sarr, MG, and Farley, DR. Simulation-based mastery learning improves patient outcomes in laparoscopic hernia repair,  Annals of Surgery, 254(3) 502-510). 

Their training resulted in improved patient outcomes including complications and the need for an overnight stay.  The control group  got "standard practice," which consisted of their regular clinical training. 

The training treatment group got additional training consisting of nine online learning modules and skills training on a simulator.  Each of these two components was built on a mastery learning model.  Students moved on from the online modules once they achieved 90% or better on multiple-choice knowledge test.

Skills training consisted of practice sessions, supervised by an expert, on a simulation task trainer.  Participants practiced until they were able to repair two simulated hernias in less than 2 minutes, on 2 consecutive attempts.  If they couldn't do this in 10 tries, or if they had worked for an hour, they stopped for the say and continued on a later day. 

This study shows us once again that sills practice can be an important part of successful training interventions.

Distance Education can be Effective for Rural Physicians

Rural doctors don't have the same access to learning resources as physicians in cities often do, they depend much more on distance learning methods than other physicians do. We know that distance learning methods can be just as effective as any other methods. Hepatitis C is a problem in rural New Mexico. The treatment is complex, and rural physicians often don't have access to effective continuing education to help them learn to treat it. Physicians at the University of New Mexico have developed an interesting system for using a telehealth network to teach Hepatitis C treatment, called Project ECHO.



In the Project ECHO model, physicians begin by spending a day or two at a Hepatitis C clinic, where they shadow a specialist.  Then, follow weekly, two-hour case presentations that include discussion of treatment complications and psychiatric, medical, and substance abuse issues.  Collaborators in these sessions might include various medical and mental health specialists. 

The cases are worked through collaboratively, with shared case management and decision making.  The rural physicians present their own cases, and occasionally, are asked to research and present didactic information to the group. 

Project ECHO has resulted in positive differences in the lives of patients.  The outcomes for the ECHO project, published in a June, 2011 article in the New England Journal of Medicine, showed that patients treated at the ECHO sites had similar rates of sustained viral response as patients seen at the specialty clinic.

CME Associated with Decreased Mortality in Patients with Coronary Heart Disease

Low density lipoprotein

Evidence showing real changes in patient outcomes are rare, even though that's the ultimate goal for most CME programs.  A very nice study, published in the Annals of Family Medicine, showed a change in practice, and a positive change in patient outcomes after a CME program (Kiessling, A.; Lewitt, M.; and Henriksson, P. (2011), Case-based training of evidence-based clinical practice in primary care and decreased mortality in patients with coronary heart diseases.  Annals of Family Medicine, 9:211-218.).   

CME on new Guidelines for Coronary Heart Disease
Practice guidelines in Sweden on the use of lipid lowering drugs were developed after the 1994 publication of the effectiveness of lipid lowering in coronary heart disease.  In order to help general practitioners follow the new guidelines, a series of CME programs were developed. The training was designed to be interactive and to "activate" learners.  They felt that a case-based method was the most effective way to activate the learners and help them improve the sills and attitudes needed for decision making. 

CME Design
All participants were mailed a copy of the new guidelines.  The instruction began with a lecture deliverd by a specialist.  This was followed by a series of seminars, delivered over the next two years.  Physicians participated in 1-2 cases each year, in groups of 4-7 people.  The cases were designed to be simple and well-organized, and presented a problem without an obvious correct answer.  The seminars were one hour each and consisted of the case presentation, followed by interactive problem-solving, with a thorough discussion of the pros and cons of various decisions.  Physicians in the control group only listened to the lecture. 

Results
This seemingly simple design resulted in changes in practice, that at two years, had significantly reduced patient's low-density lipoprotein cholesterol levels compared to a control group. 

Ten years later, the mortality rate of the control group was 44%, while mortality in the intervention group was 22%.  That's right, ten years later, patients whose doctors went through a few interactive cases with an specialist,  had half the mortality of patients whose doctors listened only to a lecture and read the guidelines. 

What Happened?
The results of this study are pretty amazing.  Long -term effects are very hard to find in education studies.  It's very rare to see a study looking at an effect over a few months, let alone years, but there are a few things that we see with other kinds of successful CME that may be giving us a hint about why this worked so well. 

1. Learning over a period of time.  It takes time to learn new skills and new habits.  Physicians in this study had two years of exposure to this material.  This repeated exposure may well have helped these doctors incorporate the new guidelines into their daily practice. 

Interactive and problem focused.  The cases were designed so that they didn't have clear or obvious answers, and participants were given the chance work their way through all of the nuances of these cases, including "defining and valuing the pros and cons, and feasibility of different decisions."  One of the problems with presentations by experts is that they don't let learners work through these important details themselves. 

New Review of Simulation Education

A new article in JAMA looks at the effectiveness of simulations in health education.  The meta-analysis, which looked at over 600 published research papers, found large effects for the use of technology-enhanced simulations for knowledge, skills, and behaviors. 

Many of the papers included in the analysis dealt with surgical procedures, but there were other studies too, including dentistry, communication skills, nursing skills, examination skills, and obstetrics. 

The big limitation with this analysis is that it only looked at studies comparing simulation to no-instruction controls.  It's not surprising that some kind of instructional intervention is better than nothing at all, but this paper can serve as a good resource for those interested in learning more about simulations in medical education. 

Interactive CME Improves Physician Communication Skills

A very nice study, published last year in Patient Education and Counseling, shows that interactive instruction, that's heavily based on practicing skills, can have a positive influence on physician practice.  The study uses several elements that have been shown to influence provider behavior: skills practice, interactivity, and a design that spreads over several different sections (spaced learning). 

The Practice Setting
Physicians in a university health service clinic where there is a strong need to counsel patients on risky behaviors. 

The Instructional Design
The instruction consisted of five 2-hour workshops at 4-6 week intervals, over a span of six months.  the workshops all emphasized communication skills, and each focused on a specific area of need including managing difficult patient encounters, screening and assessment for depression, screening and assessment for alcohol use, taking a sexual history, and behavior change counseling.

Each of the workshops began with a short video or demonstration, to trigger discussion, then there was a brief, highly interactive lecture.  After that, the majority of the time was spent practicing skills with standardized patients and receiving feedback.  After each session, participants were given pocket cards to uses as references, and readings for further information. 

The Measures
This very thorough study, measured four different outcomes - participant satisfaction, learning (measured prior to the intervention and one month afterward), performance (assessed by chart review both before and after the intervention), and the impact on patient satisfaction.

The Results
The results showed positive increases in many important outcomes including knowledge, and more importantly, physicians changed their practice.  The patient satisfaction scores were already very high, and did not increase significantly as a result of the training. 


What it Means
Providers can change their practice.  Often, change requires something more than just education and training, but the elements that can help physicians change practice behavior are not a mystery, we see the same elements over and over again when we find CME that that changes practice:

1. Skills practice with feedback - people need to practice new skills before they can use them
2. Interactive information delivery
3. Practice over time - time is an often overlooked factor, but one of the most powerful