I feel obligated to share my thoughts after I’ve seen many guidelines about developing multiple choice questions to assess higher order thinking. Multiple-choice tests are best used to assess learners’ ability to recall information. Anything more than recall is a stretch for multiple-choice tests. I will explain how I’ve come to this conclusion.
Let’s define higher-order thinking skills as the cognitive skills which require greater cognitive resources to complete the learning assessment activity. In Bloom’s taxonomy, higher order thinking activities refer to analysis, synthesis, creation and evaluation. With this definition in mind, let us see why multiple-choice tests are inadequate for assessing analysis, synthesis, creation, and evaluation.
A good assessment must have the following four attributes. First, the assessment should be relevant to the course because it aligns with one or more course objectives. Second, the expectations for the students should be clear and reasonable. Third, the assessment should provide the students the opportunity to demonstrate their learning. Fourth, the assessment should allow the instructor to provide constructive feedback. Multiple choice assessments for higher-order thinking fail to have all these attributes.
- Multiple choice questions are designed to have one correct answer. Higher-order thinking skills, however, involve authentic, ill-structured problems which may have no solution or multiple solutions, depending on the context. Assessments promoting higher-order thinking skills should focus on ill-structured problems (Weiss, 2003) so students can demonstrate their higher order thinking process. A multiple-choice question is not open to additional solutions besides those provided among the existing answer choices, so a student cannot demonstrate an alternative yet equally plausible answer.
- Multiple choice tests do not allow students to demonstrate their higher order thinking process. Higher order thinking skills require a higher amount of cognitive resources and complex thinking. Therefore, the student needs to demonstrate their thought processes through observable performance or comprehensive written submission. The student can explain the rationale for the chosen solution. With multiple choice questions, a student may select the correct answer with a lucky guess, and this obviously does not demonstrate the advanced cognitive process.
- Presumably, a correct multiple choice response as the solution to a problem should indicate that the student had gone through the desired thought processes to come to that conclusion. However, we cannot know that for certain. Additionally, incorrect answers on multiple choice tests fail to provide enough information to identify where the students’ thought process failed. Thus, the instructor cannot provide constructive feedback to correct student misconceptions. The closest an instructor could come is to have a multiple-choice question with incorrect choices addressing each anticipated incorrect thought process. A wrong answer would signal where the misconception occurred. Even with such a well-designed multiple-choice test, the instructor is still making guesses about the students’ thought processes, and may not have accounted for all possible variations.
Considering the limitations of multiple-choice tests for assessing higher-order thinking, why are multiple-choice tests so popular? I have several hypotheses. First, multiple-choice tests are convenient for instructors because they are automatically graded in the learning management systems and online tests may have automatically randomized questions. The multiple-choice tests save significant time for grading. Second, multiple-choice tests look more “objective” since they do not involve unique responses from each student which instructors assess with a rubric. However, it is not objective since the instructor or textbook publishing company develops those questions and answer options based on their subjective judgment. Third, multiple-choice tests make an assessment in large classrooms more manageable. True, but why are we sacrificing quality learning for the sake of quantity of enrollment? Perhaps we should instead ask ourselves, “what is the proper class size for this subject matter?” Last, some instructors use multiple-choice exams to mimic licensing exams for graduate degree programs in health sciences or law degrees. However, courses have very different goals than licensing exams. In the classroom, instructors have a responsibility to provide feedback to every individual student in our class so each exam is a formative learning opportunity. Testing centers are not obligated to provide constructive feedback; they merely determine whether the individuals meet the minimum criteria set by profession.
In summary, I encourage the instructors to use the right tool for the right purpose. Multiple-choice exams are great assessing recall activities. If you want to measure higher-order thinking, please consider assessments which allow students to make their thought processes clear in addition to selecting the desired answers. Better choices for higher-order thinking can include methods such as essays, Socratic questioning, case discussions, project-based work, and simulations.
Weiss, R. E. (2003). Designing problems to promote higher-order thinking. New directions for teaching and learning, 2003(95), 25-31.