Writing Good Items and Item Analysis (Difficulty, Discrimination, Distractors)

An MCQ has a stem (question or incomplete statement) and options – one key (correct answer) and distractors (plausible wrong answers).

Guidelines:

• The stem should present a single, clear problem; put most words in the stem.
• Avoid negative stems; if unavoidable, bold/underline NOT.
• All options should be plausible, homogeneous and of similar length.
• Avoid grammatical clues and repeating words from the stem in the key.
• Avoid "all of the above"; use "none of the above" sparingly.
• Place options in a logical order (e.g. numbers ascending); vary the position of the key.
• Each item should test an important objective, preferably beyond recall.

• Clearly define the task with action verbs (explain, compare, justify, design).
• Indicate marks and expected length/time.
• Prefer several shorter questions to one long question for better content sampling.
• Avoid optional questions (students then take different tests).
• Prepare a model answer and a rubric/marking scheme in advance.
• Score anonymously, one question at a time across all papers, to improve inter-rater and intra-rater reliability.

p = number of students who answered correctly ÷ total number of students (or using upper + lower groups: p = (RU + RL) ÷ (NU + NL)).

• Range 0 to 1; higher p = easier item.
• p = 0.50 gives maximum discrimination; 0.30–0.70 is generally acceptable for classroom tests.
• p > 0.80 → very easy; p < 0.20 → very difficult.

Example: 30 of 60 students answer correctly → p = 30 ÷ 60 = 0.50.

Shows how well an item separates high achievers from low achievers.

1. Rank papers by total score.
2. Take the upper 27% and lower 27% groups (or upper and lower halves for small classes).
3. D = (RU – RL) ÷ n, where RU and RL are correct responses in the upper and lower groups and n is the number of students in one group.

• Range –1 to +1.
• Interpretation (Ebel): ≥ 0.40 very good; 0.30–0.39 reasonably good; 0.20–0.29 marginal, needs improvement; < 0.20 poor – revise or discard.
• Negative D (more low achievers answer correctly) → the item is faulty (wrong key, ambiguity).

Example: upper group 20 students, 16 correct; lower group 20, 6 correct → D = (16 – 6) ÷ 20 = 0.50.

A good distractor attracts more students from the lower group than from the upper group. A distractor chosen by almost no one (commonly < 5%) is non-functional and should be replaced. A distractor chosen by more upper-group students than the key suggests ambiguity or a wrong key.