Four Typical Online Learning Assessment Mistakes

Category: Course Evaluation Methods

By Patti Shank, Ph.D, CPT http://www.learningpeaks.com
Adapted and reprinted with permission from "Faculty Focus", Magna Publications http://www.facultyfocus.com/

The goal of learning assessments should be to measure whether actual learning outcomes match desired learning outcomes. We measure whether instruction “works” by seeing if the instruction we build actually helps people achieve the learning objectives. I’d even argue that we cannot be considered competent builders of instruction if we can’t show that what we built helps learners learn.

People who build instruction make some typical but unfortunate mistakes when designing learning assessments, and these mistakes compromise both their competence as designers of instruction and the quality of the instruction they build. These mistakes include:

• Expecting a bell curve
• The wrong type of assessment
• Not valid (enough) assessments
• Poorly written multiple-choice tests

Expecting a Bell Curve

Benjamin Bloom (1968), a distinguished educational psychologist, proposed that a bell curve model, with most students performing in the middle and a small percentage performing very well and very poorly (e.g., a normal or bell curve) is the wrong model of expected outcomes from most instruction.

The bell curve model is what might be expected without instruction. Instruction should be specifically designed to provide the instruction, practice, feedback, and remediation needed to bring about achievement of the desired outcomes. His “mastery” model assumes that most students will be high achievers and that the instruction needs to be fixed if this does not occur.

The Wrong Type of Assessment

There are two primary learning assessment formats: performance assessments and “test” assessments. The former involves assessing performance in a more realistic way (in situations), and the second involves paper or computer-based forms with multiple choice, matching, fill-in-the- blank, and short- and long-answer type (i.e., essay) questions. Test assessments are by their nature a less authentic way of assessing learning but are very practical and are therefore commonly used.

The optimal assessment type depends primarily on whether the objective is declarative (facts: name, list, state, match, describe, explain…) or procedural (task: calculate, formulate, build, drive, assemble, determine…).

Research shows that there is a big difference between these two types—the difference between knowing about and knowing how (practical application to real-world tasks). Many higher education instructors don’t adequately infuse their courses with real-world implications and skills, and I believe this is a mistake.

Not Valid (Enough) Assessments

The gold standard for assessment quality is validity. A valid assessment measures what it claims to measure. It’s easier than you might think to design assessments that measure something other than what is intended.

Lack of validity impacts course quality and fairness. And if the results of assessments impacts passing the course (as they usually do), invalid assessments are not only unfair but potentially illegal.

The best way to establish validity is to carefully match objectives and assessments.

Poorly Written Multiple-Choice Tests

Many assessments, even if they are the right kind, are poorly written. Two of the most common mistakes for multiple-choice questions are confusing or ambiguous language and implausible distractors (wrong alternatives from which the learner selects the correct answer[s]). A poorly written multiple-choice question automatically lowers the validity of the assessment.

Final Thoughts

Inadequate learning assessments are at best frustrating. At worst, they can damage students and institutions. Adequate learning assessments are one of the hallmarks of competence in building good instruction and markedly improve the quality of instruction. In an online course, where students often require extra feedback and motivation, unintended frustrations and unfairness can cause many problems including complaints, reduced enrollments, and lack of persistence. Writing good performance assessments and test questions is a skill that takes training, time, and feedback.

American Educational Research Association, American Psychological Association (APA), & National Council on Measurement in Education (NCME). (1999). Standards for educational and psychological testing.

Bloom, B. (1968). Learning for mastery. Evaluation Comment, 1(2), 1–5.

Bond, L. A. (1995, August). Norm-referenced testing and criterion-referenced testing: The difference in purpose, content, and interpretation of results. Oak Brook, IL: North Central Regional Educational Laboratory. (ERIC Document Reproduction Service No. ED402327).

Shank, P. (2005). Developing learning assessments for classroom, online, and blended learning. Workshop Materials. Denver, CO: Learning Peaks.

Shrock, S., & Coscareli, W. (2000). Criterion-referenced test development. Silver Spring, MD: International Society for Performance Improvement.

Return to the Top of the Document...