Abstract
This study illustrates how a diagnostic testing model can be used to make detailed comparisons between student populations participating in international assessments. The performance of Turkish students on the TIMSS-R mathematics test was reanalyzed with a diagnostic testing model called the Rule Space Model. First, mathematical and cognitive skills (‘attributes’) measured by the test were determined. One hundred sixty-two items were coded in terms of their attribute involvement, creating an incidence matrix—the Q-matrix. Using the Q-matrix and the student response data, each student’s attribute mastery profile was determined. Mean attribute mastery levels of Turkish students were computed and compared to those of their American peers. It was shown that Turkish students were weak in algebra and probability/statistics. They also demonstrated poor profiles in skills such as applying rules in algebra, approximation/estimation, solving open-ended problems, recognizing patterns and relationships, and quantitative reading.
Similar content being viewed by others
Notes
BUGSHELL classified each student according to their mastery on each attribute. However, composite attributes were created to reduce the number of knowledge states and hence produce more manageable and interpretable results.
References
Buck, G., & Tatsuoka, K. (1998). Application of the rule-space procedure to language testing: Examining attributes of a free response listening test. Language Testing, 15(2), 119–157.
Corter, J. E., & Tatsuoka, K. (2002). Cognitive and measurement foundations of diagnostic assessment in mathematics (Technical Report). New York, NY: College Board.
Guerrero, A. (2001). Cognitively diagnostic perspectives on English and Spanish versions of a test of mathematics aptitude. Dissertation Abstracts International, 62(08), 543B (UMI no. 3005725).
MEB-EARGED (2003). Üçüncü Uluslararası Fen ve Matematik Çalışması (TIMSS 1999) Ulusal Rapor [Third international mathematics and science study (TIMSS 1999) national report]. Ankara: Turkey.
Milewski, G. B., & Baron, P. A. (2002). Extending DIF methods to inform aggregate reports on cognitive skills. In 1998 National Council on Measurement in Education annual meeting, New Orleans, LA.
Mullis, V. S., Martin, M. O., Gonzales, E. J., Gregory, K. D., Garden, R. A., O’Conner, K. M., et al. (2000). TIMSS 1999 international mathematics report: Findings from IEA’s repeat of the third international mathematics and science study at the eighth grade. Chestnut Hill, MA: Boston College.
Scott, H. S. (1998). Cognitive diagnostic perspectives of a second language reading test. Dissertation Abstracts International, 59(11), 6113B (UMI no. 9912372).
Tatsuoka, K. K. (1993). Proficiency scaling based on conditional probability functions for attributes. (Technical rep. no. RR-93-50-ONR). Princeton, NJ: Educational Testing Service.
Tatsuoka, K. K., & Gallagher, A. (1998). Variables that are involved in the underlying cognitive processes and knowledge of GRE quantitative (Technical report). Princeton, NJ: Educational Testing Service.
Tatsuoka, K. K., & Boodoo, G. M. (2000). Subgroup differences on the GRE quantitative test based on the underlying cognitive processes and knowledge. In A. E. Kelly, & R. Lesh (Eds.), Handbook of research design in mathematics and science education (pp. 821–857). Mahwah, NJ: Erlbaum.
Tatsuoka, C., Varadi, F., & Tatsuoka, K. K. (1992). BUGSHELL computer software. Ewing, NJ: Tanar Software.
Tatsuoka, K. K., Corter, J., & Tatsuoka, C. (2004). Patterns of diagnosed mathematical content and process skills in TIMSS-R across a sample of 20 countries. American Educational Research Journal, 11(4), 901–926.
Van der Linden, W. J. (1998). A discussion of some methodological issues in international assessments. International Journal of Educational Research, 29, 569–577.
Xin, T., Xu, Z., & Tatsuoka, K. K. (2004). Linkage between teacher quality, student achievement, and cognitive skills: A rule-space model. Studies in Educational Evaluation, 30(3), 205–223.
Yepes-Baraya, M., Tatsuoka, K., Allen, N. L., O’Sullivan, C., Liang, J., & Hui, X. (1998, April). Application of rule space methodology to the 1996 NAEP science assessment: Grade 4 preliminary results. In 1998 National Council on Measurement in Education annual meeting, San Diego, CA.
Acknowledgments
This study has been supported by the National Science Foundation (REC NO. 0126064).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Dogan, E., Tatsuoka, K. An international comparison using a diagnostic testing model: Turkish students’ profile of mathematical skills on TIMSS-R. Educ Stud Math 68, 263–272 (2008). https://doi.org/10.1007/s10649-007-9099-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10649-007-9099-8