The influence of item-stem format and item-answer format on the results for selected items 83 in TIMSS 2003

Abstract

This paper considers the influence of the stem format and answer format of survey items on the responses (results) for those items. The items referred to in this analysis are from TIMSS 2003, conducted by the International Association for the Evaluation of Educational Achievement (IEA). The analysis focuses on the national test booklets completed by 806 Grade 8 students (from 150 schools) in Lithuania. In TIMSS 2003, Lithuania added two national booklets for mathematics, in which the TIMSS items were changed in a way that made it possible to examine the effect of the stem and answer format of an item on the results for that item. The analysis showed that Lithuanian Grade 8 students were significantly better at solving multiple-choice answer format items than open-ended answer format items. They also were less likely to omit multiple-choice than openended items. No difference was found in these regards between the genders. The analysis also showed that the distracters given in multiple-choice answer format items influenced students’ choice of answer. When students were presented with exactly the same items but without the answers, considerably fewer of them gave the answers given in the distracters. The hypothesis that formulating the stems of the TIMSS items more in “Lithuanian style” would help students solve the items was discounted. However, in general, the wording of the stem item did affect the answers to (the results of ) the item.