How to write effective multiple choice questions

I never used to be a fan of the humble multiple choice question, and where possible I tried to avoid using them.    

Some of the difficulties with multiple choice questions are:

1. They can often be easy for the learner to guess.

2. They tend to focus on testing knowledge.

3. They rarely focus on developing a deep understanding of the topic at hand.

For me, an indicator of bad elearning is a dump of information (often a series of slides, perhaps with a voice-over) with a knowledge quiz at the end. This model of ‘teach and test’ comes from how schools and universities used to work (and many of us in L&D roles are guilty of merely replicating the model for our own use). But education has moved on, and now a significant amount of learning at universities is more experiential, and assessment concerns the application of knowledge rather than testing how well we remember it.

The tipping point – when I saw the potential of multiple choice questions

My attitude to multiple choice questions changed when one of our instructional designers wrote a multiple choice question that was impossible to guess. It was an activity type we now call an ‘interactive report’. The learner is presented with a partially-completed report and is asked to choose from a number of alternatives the paragraph that will correctly complete the report. But the choice involves more than just choosing a chunk of text – instead the learner must go through a series of other decisions in order to find the correct alternative.

At the core of a performance-focus approach to elearning is a focus on the decisions that a learner has to make in the workplace. Multiple choice questions are the perfect tool to enable the learner to rapidly and safely practise making those decisions.   

What holds back multiple choice questions from being performance focused?

1. PowerPoint-based authoring systems
   Often with PowerPoint-based systems like Storyline and Captivate you cannot get enough text on the screen to make rich, complex questions. One solution is to build the scenario up over a number of slides, but then if a learner needs to refer back to past slides it can get messy. With new cutting-edge authoring systems like Adapt and Glasshouse, which are based on scrolling pages, it’s easier to build more complex scenarios.  

1. A focus on content
   The focus on providing content – not on what the learner needs to do and the decisions they need to make – is one of the continuing problems with elearning. Just what performance-focus learning looks like is one of the things we’ll be exploring in our series of blog posts on the elearning Manifesto.     

Why face-to-face trainers struggle with multiple choice questions

We often work with face-to-face trainers as SMEs on projects who struggle with writing multi-choice questions. In the classroom they are used to asking open questions that don't have right or wrong answers, and tend to favour short-answer questions when designing activities. This is problematic because with short-answer questions it’s difficult to provide good feedback. These same SMEs see multiple choice questions as being closed and not particularly useful for complex learning.

There are a few ways to deal with this challenge.

1. Move away from the idea that multiple choice questions are testing knowledge, and think about them more as interactions.

2. They don’t just have to be about correct or incorrect answers. In Glasshouse we have the option for answers to be partially correct, and also for all the choices to be correct. This means we can give feedback on each choice, while doing so explaining the complexities or grey areas.

3. Many open-ended questions can be reworded for multiple choice by thinking about what could be a good, average and bad response to the question. Then in the feedback you can bring in some of what might have been discussed in a face-to-face situation, for example with a statement such as ‘You also need to think about …’

Alternative approaches

If the decisions are really too complex to fix into a multiple choice format, other options could be:

1. using a ‘choose your own adventure’ branching scenario

2. using social learning such as forums or online commenting systems.

Features of multiple choice questions

Label 1 - A multiple choice questions has a ‘stem’. At Sprout Labs we have re-framed these to be known as ‘sparks’.

Label 2 - Then there is a series of choices.   

Label 3 - [points the A mathematical equation]-  The incorrect answers are called distractors. Writing great multiple choice questions is all about the distractors – you don't want the correct answer to be obvious.  

A recipe for great multiple choice questions

Great multiple choice questions are based on authentic situations and the real decisions someone has to make in a job role. If the course has a broad audience this focus on actual decisions can be challenging, because it’s difficult to cater for multiple situations.     

If you're a subject matter expert, focus on the decisions you have to make when solving a challenge, and think about the concepts and models you rely upon. Many of these decisions have become automatic to you, so the challenge you face in designing effective elearning is to see the decisions from the learner’s perspective.

If you're an instructional designer it’s your role to uncover what these decisions are.  

A useful technique when writing a multiple choice question is to write the correct answer, then come up with the distractors. When writing good distractors it pays to consider what a novice might (incorrectly) choose.

Multiple choice question DOs and DON’Ts

Finally, feedback is critical to learning. Multiple choice questions used effectively can be a great way to provide learners with the feedback they need.