Analysis of the Cognitive Structure of the Area of Quantitative Skills of the EXHCOBA Using Fisher’s LLTM Model
DOI:
https://doi.org/10.37467/gka-revedu.v3.584Keywords:
Rasch Model, Cognitve Structure, Q-Matrix, Admission TestAbstract
The present study integrates aspects of cognitive psychology and psychometrics (Rupp & Mislevy, 2006) for the analysis of the sources of difficulty in the area of quantitative skills (HC, for its acronym in Spanish) of the Basic Skills and Knowledge Test (EXHCOBA, for its acronym in Spanish) (Backhoff, Tirado, & Larrazolo, 2001), using the Linear Logistic Test Model (LLTM) developed by Fischer (1973). To achieve this purpose, a group of experts proposed fourteen operations to solve items in the HC area, based on the results of retrospective and concurrent verbal report,s obtained from a group of students of secondary education. In addition, the cognitive structure determined by experts was analyzed, and the dimensionality and the adjustment of the items to Rasch model and LLTM was assessed. The results showed that the data fit moderately to the LLTM with a Q-matrix of fourteen attributes and, also that, a refinement of the operations to resolve the items could explain better the sources of difficulty and the performance of examinees in the test.
References
Abida, K., Azeem, M. & Bashir, M. (s.f.). Assessing Students’ Math Proficiency Using Multiple-choice and Short Constructed Response Item Formats. International Journal of Technology, Knowledge and Society, 7 , 135-150. DOI: https://doi.org/10.18848/1832-3669/CGP/v07i03/56206
Adams, R., Wilson, M., & Wang, W-C. (1997). The multidimensional random coefficients multinomial logit model. Applied Psychological Measurement, 21 , 1-24. DOI: https://doi.org/10.1177/0146621697211001
Backhoff, E. & Larrazolo, N. “Evaluación de competencias escolares”. Trabajo presentado en el I Foro Iberoamericano de Evaluación Educativa de la Red Iberoamericana de Medición y Evaluación de Sistemas Educativos. 7 de noviembre de 2012, baja California. Consultado en: http://www.ustream.tv/recorded/26785327
Backhoff, E., Tirado, F. & Larrazolo, N. (2001). Ponderación diferencial de reactivos para mejorar la validez de una prueba de ingreso a la universidad. Revista Electrónica de Investigación Educativa, 3 (1). Consultado el 20 de febrero de 2013 en: http://redie.ens.uabc.mx/vol3no1/contenido-tirado.html
Chi, M. (1994). Analyzing the content of verbal data to represent knowledge: A practical guide. Unpublished manuscript, Learning Research and Development Center, University of Pittsburgh.
Cortada de Kohan, N. (2000). Importancia de la investigación psicométrica. Revista Latinoamericana de Psicología, 34 (3), 229-240.
DiBello, L., Stout, W., & Roussos, L. (1995). Unified cognitive/psychometric diagnostic assessment likelihood-based classification techniques. En P. D. Nichols, S. F. Chipman & R. L. Brennan (Eds.), Cognitively diagnostic assessment, (pp. 361-389). Hillsdale, NJ, England: Lawrence Erlbaum Associates, Inc.
Embretson, S. (1984). A general latent trait model for response processes. Psychometrika, 49 (2), 175-186. DOI: https://doi.org/10.1007/BF02294171
Embretson, S. (1994). Applications of cognitive design systems to test development. En C.R. Reynolds (Ed.), Cognitive assessment. A multidisciplinary perspective (pp. 107-135). New York: Plenum. DOI: https://doi.org/10.1007/978-1-4757-9730-5_6
Embretson, S. E., & Wetzel, C. D. (1987). Component latent trait models for paragraph comprehension tests. Applied Psychological Measurement, 11 (2), 175-193. DOI: https://doi.org/10.1177/014662168701100207
Ericsson, K., & Simon, H. (1993). Protocol analysis: Verbal reports as data . Cambridge, MA: The MIT Press. DOI: https://doi.org/10.7551/mitpress/5657.001.0001
Fischer, G. (1973). The linear logistic test model as an instrument in educational research. Acta Psychologica, 37 , 359-374. DOI: https://doi.org/10.1016/0001-6918(73)90003-6
Fischer, G. (1995). The linear logistic test model. En G. H. Fisher, & I. W. Molenaar (Eds.), Rasch models: Foundations, recent developments, and applications . New York: Springer-Verlag.
Fischer, G. & Molenaar, I. (1995). Rasch Models. Foundations, Recent Developments and Applications. Ed: Springer-Verlag. New York. DOI: https://doi.org/10.1007/978-1-4612-4230-7
Fischer G. & Ponocny-Seliger, E. (1997). LPCM-WIN Program. IEE ProGAMMA. Groningen. Fraser, C. (1988). NOHARM: Computer software and manual. Australia: Author.
Gierl, M., Tan, X. & Wang, Ch. (2005). Identifying content and cognitive dimensions on the SAT . Research Report nº 2005-11. College Board.
Junker, B., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25 (3), 258-272. DOI: https://doi.org/10.1177/01466210122032064
Leighton, J., & Gierl, M. (2007). Verbal reports as data for cognitive diagnostic assessment. En J. P. Leighton y M. J. Gierl (Eds.), Cognitive diagnostic assessment for education. Theory and applications, (pp. 146–172). Cambridge, MA: Cambridge University Press. DOI: https://doi.org/10.1017/CBO9780511611186
Messick, S. (1989). Validity. En R. L. Linn (Ed.), Educational measurement (3rd ed.), (pp. 13- 103). New York, NY, England: Macmillan Publishing Co, Inc.
Mislevy, R. (1995). Probability-based inference in cognitive diagnosis. En P. D. Nichols, S. F. Chipman y R. L. Brennan (Eds.), Cognitively diagnostic assessment, (pp. 43-71). Hillsdale, NJ, England: Lawrence Erlbaum Associates, Inc.
Pérez, J.C. (2013). Análisis del aspecto sustantivo de la validez de constructo de una prueba de Habilidades Cuantitativas. Tesis doctoral. Ensenada, Instituto de Investigación y Desarrollo Educativo, UABC, 2013.
Prieto, G & Delgado, A. (1999). Medición cognitiva de las aptitudes. En J. Olea, V. Ponsoda & G. Prieto (Eds.), Test informatizados: Fundamentos y aplicaciones . Madrid: Pirámide.
Revuelta, J. & Ponsoda, V. (1998). Un test adaptativo informatizado de análisis lógico basado en la generación automática de ítems. Psicothema, 10 , 753-760.
Romero, S. Propiedades y aplicaciones del método de las distancias mínimo-cuadráticas (LSDM) para la validación y análisis de atributos cognitivos. Tesis, Universidad Europea de Madrid, 2010.
Romero, S.J., Ponsoda, V. & Ximénez, C. (2006). Validación de la estructura cognitiva del test de signos mediante modelos de ecuaciones estructurales. Psicothema, 18 , 835-840.
Rouse, S.V., Finger, M.S. & Butcher, J.N. (1999). Advances in clinical personality measurement: An item response theory analysis of the MMPI – PSY- 5 scales. Journal of Personality Assessment, 72 , 282-307. DOI: https://doi.org/10.1207/S15327752JP720212
Rupp, A. & Mislevy, R. (2006). Cognitive Foundations of Structured Item Response Models. En J.P. Leighton y M.J. Gierl (Eds.), Cognitive Diagnostic Assessment: Theories and Applications. Cambridge: Cambridge University Press. DOI: https://doi.org/10.1017/CBO9780511611186.008
Spada, H. (1977). Logistic models of learning and thought. En H. Spada, & W. F. Kempf (Eds.), Structural models of thinking and learning, (pp. 227-262).
Spada, H. & May, R. (1982). The linear logistic test model and its application in educational research. En D. Spearrit (Ed.), The improvement of measurement in Education and Psychology, (pp. 67-84). Hawthorne, Victoria: Australian council for educational research.
Tatsuoka, K. (1985). A probabilistic model for diagnosing misconceptions by the pattern classification approach. Journal of Educational Statistics, 10 (1), 55-73. DOI: https://doi.org/10.3102/10769986010001055
Van der Linden, W. & Hambleton, R. (1997). Handbook of modern item response theory . New York: Springer. DOI: https://doi.org/10.1007/978-1-4757-2691-6
Downloads
Published
How to Cite
Issue
Section
License
Those authors who publish in this journal accept the following terms:
- Authors will keep the moral right of the work and they will transfer the commercial rights.
- After 1 year from publication, the work shall thereafter be open access online on our website, but will retain copyright.
- In the event that the authors wish to assign an Creative Commons (CC) license, they may request it by writing to publishing@eagora.org
