An Early Numeracy Digital Brief Assessment: Parametric and Non-parametric Item Response Theory Models
Cecilia Marconi
,
Dinorah de León
,
Mario Luzardo
,
Alejandro Maiche
Developing efficient and reliable tools for assessing early mathematical skills remains a critical priority in educational research. This study aimed .
- Pub. date: May 15, 2025
- Online Pub. date: May 14, 2025
- Pages: 245-266
- 41 Downloads
- 296 Views
- 0 Citations
- #Early numeracy assessment
- # item response theory
- # kernel smoothing IRT
- # parametric/non-parametric IRT models
- # symbolic/non-symbolic mathematics skills.
Abstract:
Developing efficient and reliable tools for assessing early mathematical skills remains a critical priority in educational research. This study aimed to develop and validate a brief version of the Prueba Uruguaya de Matemática (Uruguayan Mathematics Test, PUMa), a digital tool to assess mathematical abilities in children aged 5 to 6. The original test included 144 items covering both symbolic (66%) and non-symbolic (34%) tasks, such as approximate number system, counting, numerical ordering (forward and backward), math fluency, composition and decomposition of numbers, and transcoding auditory-verbal stimuli into Arabic-visual symbols. Unlike most existing tools that require individual administration by trained professionals and lack cultural adaptation for Latin American contexts, PUMa is self-administered, culturally grounded, and suitable for large-scale assessments using tablets. Using a sample of 443 participants and applying parametric and non-parametric models within the framework of Item Response Theory (IRT), along with correlations with TEMA-3, preliminary evidence was generated showing that the brief version retained precision and validity. The resulting shortened tests included 69 and 73 items for the parametric and non-parametric versions, yielding a balanced representation of symbolic (56%) and non-symbolic (44%) tasks. Despite item reduction, ability scores remained highly correlated between original and brief versions (r > .90), and both brief versions demonstrated strong internal consistency (α = .94). PUMa improves upon existing assessments by combining cultural relevance, group-based digital administration, and real-time data collection, offering a scalable solution for early identification and intervention. These features support personalized educational strategies that foster cognitive and academic development from the earliest stages.
early numeracy assessment item response theory kernel smoothing irt parametricnon parametric irt models symbolicnon symbolic mathematics skills
Keywords: Early numeracy assessment, item response theory, kernel smoothing IRT, parametric/non-parametric IRT models, symbolic/non-symbolic mathematics skills.
0
References
Archbald, D., & Farley-Ripple, E. N. (2012). Predictors of placement in lower level versus higher level high school mathematics. The High School Journal, 96(1), 33-51. https://doi.org/10.1353/hsj.2012.0014
Aubrey, C., Godfrey, R., & Dahl, S. (2006). Early mathematics development and later achievement: Further evidence. Mathematics Education Research Journal, 18, 27-46. https://doi.org/10.1007/BF03217428
Aunio, P., & Niemivirta, M. (2010). Predicting children's mathematical performance in grade one by early numeracy. Learning and Individual Differences, 20(5), 427-435. https://doi.org/10.1016/j.lindif.2010.06.003
Baker, F. B. (1985). The basics of item response theory. Heinemann.
Baker, F. B., & Kim, S.-H. (2004). Item response theory: Parameter estimation techniques (2nd ed.). CRC Press. https://doi.org/10.1201/9781482276725
Baroody, A. J. (1987). Children's mathematical thinking: A developmental framework for preschool, primary, and special education teachers. Teachers College Press.
Benavides-Varela, S., Callegher, C. Z., Fagiolini, B., Leo, I., Altoè, G., & Lucangeli, D. (2020). Effectiveness of digital-based interventions for children with mathematical learning difficulties: A meta-analysis. Computers and Education, 157, Article 103953. https://doi.org/10.1016/j.compedu.2020.103953
Ceibal. (n.d.). Ceibal homepage. https://ceibal.edu.uy
Clements, D. H., & Sarama, J. (2015). Developing young children's mathematical thinking and understanding. In S. Robson, & S. Flannery Quinn (Eds.) The Routledge international handbook of young children's thinking and understanding (pp. 331-344). Routledge.
Collins, M. A., & Laski, E. V. (2015). Preschoolers’ strategies for solving visual pattern tasks. Early Childhood Research Quarterly, 32, 204-214. https://doi.org/10.1016/j.ecresq.2015.04.004
Davis‐Kean, P. E., Domina, T., Kuhfeld, M., Ellis, A., & Gershoff, E. T. (2022). It matters how you start: Early numeracy mastery predicts high school math course‐taking and college attendance. Infant and Child Development, 31(2), Article e2281. https://doi.org/10.1002/icd.2281
Dellatolas, G., Von Aster, M., Willadino-Braga, L., Meier, M., & Deloche, G. (2000). Number processing and mental calculation in school children aged 7 to 10 years: A transcultural comparison. European Child and Adolescent Psychiatry, 9(Suppl 2), S102-S110. https://doi.org/10.1007/s007870070003
Dette, H., Neumeyer, N., & Pilz, K. F. (2006). A simple nonparametric estimator of a strictly monotone regression function. Bernoulli, 12(3), 469-490. https://doi.org/10.3150/bj/1151525131
DeWind, N. K., & Brannon, E. M. (2016). Significant inter-test reliability across approximate number system assessments. Frontiers in Psychology, 7, Article 310. https://doi.org/10.3389/fpsyg.2016.00310
Douglas, J. (1997). Joint consistency of nonparametric item characteristic curve and ability estimation. Psychometrika, 62(1), 7-28. https://doi.org/10.1007/BF02294778
Douglas, J., & Cohen, A. (2001). Nonparametric item response function estimation for assessing parametric model fit. Applied Psychological Measurement, 25(3), 234-243. https://doi.org/10.1177/01466210122032046
Duncan, G. J., Dowsett, C. J., Claessens, A., Magnuson, K., Huston, A. C., Klebanov, P., Pagani, L. S., Feinstein, L., Engel, M., Brooks-Gunn, J., Sexton, H., Duckworth, K., & Japel, C. (2007). School readiness and later achievement. Developmental Psychology, 43(6), 1428-1446. https://doi.org/10.1037/0012-1649.43.6.1428
Fritz, A., Ehlert, A., Ricken, G., & Balzer, L. (2017). Mathematik- und Rechenkonzepte bei Kindern der ersten Klassenstufe–Diagnose [Mathematics and arithmetic concepts in first-grade children – Diagnosis]. Hogrefe. https://bit.ly/4iwAVZl
Gilmore, C., & Batchelor, S. (2021). Verbal count sequence knowledge underpins numeral order processing in children. Acta Psychologica, 216, Article 103294. https://doi.org/10.1016/j.actpsy.2021.103294
Ginsburg, H., & Baroody, A. J. (2007). Tema-3: Test de competencia matemática básica [Tema-3: Test of basic mathematical competence]. Tea & Hogrefe. https://bit.ly/3RYFCQp
Gliksman, Y., Berebbi, S., & Henik, A. (2022). Math fluency during primary school. Brain Sciences, 12(3), Article 371. https://doi.org/10.3390/brainsci12030371
Habermann, S., Donlan, C., Göbel, S. M., & Hulme, C. (2020). The critical role of Arabic numeral knowledge as a longitudinal predictor of arithmetic development. Journal of Experimental Child Psychology, 193, Article 104794. https://doi.org/10.1016/j.jecp.2019.104794
Halberda, J., & Feigenson, L. (2008). Developmental change in the acuity of the "Number Sense": The Approximate Number System in 3-, 4-, 5-, and 6-year-olds and adults. Developmental Psychology, 44(5), 1457-1465. https://doi.org/10.1037/a0012682
Hayfield, T., & Racine, J. S. (2008). Nonparametric econometrics: The np package. Journal of statistical software, 27(5), 1-32. https://doi.org/10.18637/jss.v027.i05
Hutchison, J. E., Ansari, D., Zheng, S., De Jesus, S., & Lyons, I. M. (2020). The relation between subitizable symbolic and non‐symbolic number processing over the kindergarten school year. Developmental Science, 23(2), Article e12884. https://doi.org/10.1111/desc.12884
Hyde, D. C., Khanum, S., & Spelke, E. S. (2014). Brief non-symbolic, approximate number practice enhances subsequent exact symbolic arithmetic in children. Cognition, 131(1), 92-107. https://doi.org/10.1016/j.cognition.2013.12.007
Koleszar, V., de León, D., Díaz-Simón, N., Fitipalde, D., Cervieri, I., & Maiche, A. (2020). Cognición numérica en Uruguay: De la clínica y los laboratorios al aula [Numerical cognition in Uruguay: From clinics and laboratories to the classroom]. Studies in Psychology, 41(2), 294-318. https://doi.org/10.1080/02109395.2020.1749000
Libertus, M. E., Feigenson, L., & Halberda, J. (2011). Preschool acuity of the approximate number system correlates with school math ability. Developmental Science, 14(6), 1292-1300. https://doi.org/10.1111/j.1467-7687.2011.01080.x
Lord, F. M. (1980). Applications of item response theory to practical testing problems. Routledge.
Luzardo, M. (2019). Item selection algorithms in computerized adaptive test comparison using items modeled with nonparametric isotonic model. In M. Wiberg, S. Culpepper, R. Janssen, J. González, & D. Molenaar (Eds.), Quantitative psychology (Vol. 265, pp. 95-105). Springer. https://doi.org/10.1007/978-3-030-01310-3_9
Luzardo, M., & Rodríguez, P. (2015). A nonparametric estimator of a monotone item characteristic curve. In L. van der Ark, D. Bolt, W. C. Wang, J. Douglas, & S. M. Chow (Eds.), Quantitative psychology research (Vol. 140, pp. 99-108). Springer. https://doi.org/10.1007/978-3-319-19977-1_8
Lyons, I. M., & Beilock, S. L. (2011). Numerical ordering ability mediates the relation between number-sense and arithmetic competence. Cognition, 121(2), 256-261. https://doi.org/10.1016/j.cognition.2011.07.009
Maiche, A., González, M., Puyol, L., López, F., & De León, D. (2020). Prueba Uruguaya de Matemática: PUMa (Versión digital) [Software]. Registro Ley N.º 9.739 – Libro 41/835. http://puma.cicea.uy
Maydeu-Olivares, A., & Joe, H. (2006). Limited information goodness-of-fit testing in multidimensional contingency tables. Psychometrika, 71(4), 713-732. https://doi.org/10.1007/s11336-005-1295-9
Miller, M. R., Rittle-Johnson, B., Loehr, A. M., & Fyfe, E. R. (2016). The influence of relational knowledge and executive function on preschoolers’ repeating pattern knowledge. Journal of Cognition and Development, 17(1), 85-104. https://doi.org/10.1080/15248372.2015.1023307
Morsanyi, K., Peters, J., Battaglia, E., Sasanguie, D., & Reynvoet, B. (2024). The causal role of numerical and non-numerical order processing abilities in the early development of mathematics skills: Evidence from an intervention study. Current Research in Behavioral Sciences, 6, Article 100144. https://doi.org/10.1016/j.crbeha.2023.100144
Moura, R., Wood, G., Pinheiro-Chagas, P., Lonnemann, J., Krinzinger, H., Willmes, K., & Haase, V. G. (2013). Transcoding abilities in typical and atypical mathematics achievers: The role of working memory and procedural and lexical competencies. Journal of Experimental Child Psychology, 116(3), 707-727. https://doi.org/10.1016/j.jecp.2013.07.008
Mulligan, J., Papic, M., Prescott, A. E., & Mitchelmore, M. (2006). Improving early numeracy through a pattern and structure mathematics awareness program (PASMAP). In Conference of the Mathematics Education Research Group of Australasia, (pp. 376-383). MERGA. https://bit.ly/41xIFU5
Nadaraya, E. A. (1964). On estimating regression. Theory of Probability & Its Applications, 9(1), 141-142. https://doi.org/10.1137/1109020
Odic, D., & Starr, A. (2018). An introduction to the approximate number system. Child Development Perspectives, 12(4), 223-229. https://doi.org/10.1111/cdep.12288
Olgan, R., Cengizoğlu, S., & Solmaz, G. (2017, April 20-23). Preschool children’s views on unsolvable math problems [Conference presentation]. 26th International Conference on Educational Sciences, Antalya, Turkey. https://open.metu.edu.tr/handle/11511/75178
Outhwaite, L. A., Aunio, P., Leung, J. K. Y., & Van Herwegen, J. (2024). Measuring mathematical skills in early childhood: A systematic review of the psychometric properties of early maths assessments and screeners. Educational Psychology Review, 36, Article 110. https://doi.org/10.1007/s10648-024-09950-6
Purpura, D. J. (2010). Informal number-related mathematics skills: An examination of the structure of and relations between these skills in preschool (Publication No. 3462344) [Doctoral dissertation, The Florida State University]. ProQuest Dissertations Publishing. http://bit.ly/4lNlizd
Purpura, D. J., Reid, E. E., Eiland, M. D., & Baroody, A. J. (2015). Using a brief preschool early numeracy skills screener to identify young children with mathematics difficulties. School Psychology Review, 44(1), 41-59. https://doi.org/10.17105/SPR44-1.41-59
Rajlic, G. (2020). Visualizing items and measures: An overview and demonstration of the kernel smoothing item response theory technique. The Quantitative Methods for Psychology, 16(4), 363-375. https://doi.org/10.20982/tqmp.16.4.p363
Ramsay, J. O. (1991). Kernel smoothing approaches to nonparametric item characteristic curve estimation. Psychometrika, 56(4), 611-630. https://doi.org/10.1007/BF02294494
Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Nielsen and Lydiche.
Reigosa-Crespo, V., & Estévez-Pérez, N. (2023). Conceptual foundations of early numeracy: Evidence from infant brain data. In M. Gomides, I. Starling-Alves, F. H. Santos (Eds.), Progress in brain research, (Vol.282, pp. 1-15). Elsevier. https://doi.org/10.1016/bs.pbr.2023.10.002
Reise, S. P., & Moore, T. M. (2023). Item response theory. In H. Cooper, M. N. Coutanche, L. M. McMullen, A. T. Panter, D. Rindskopf, & K. J. Sher (Eds.), APA handbook of research methods in psychology: Foundations, planning, measures, and psychometrics (2nd ed., pp. 809-835). American Psychological Association. https://doi.org/10.1037/0000318-037
Romano, E., Babchishin, L., Pagani, L. S., & Kohen, D. (2010). School readiness and later achievement: Replication and extension using a nationwide Canadian survey. Developmental Psychology, 46(5), 995-1007. https://doi.org/10.1037/a0018880
Seitz, M., & Weinert, S. (2022). Numeracy skills in young children as predictors of mathematical competence. British Journal of Developmental Psychology, 40(2), 224-241. https://doi.org/10.1111/bjdp.12408
Singer, V., & Cuadro, A. (2014). Psychometric properties of an experimental test for the assessment of basic arithmetic calculation efficiency. Studies in Psychology, 35(1), 183-192. https://doi.org/10.1080/02109395.2014.893657
Szkudlarek, E., & Brannon, E. M. (2017). Does the approximate number system serve as a foundation for symbolic mathematics? Language Learning and Development, 13(2), 171-190. https://doi.org/10.1080/15475441.2016.1263573
van der Linden, W. J., & Hambleton, R. K. (Eds.). (1997). Handbook of modern item response theory. Springer https://doi.org/10.1007/978-1-4757-2691-6
Watson, G. S. (1964). Smooth regression analysis. Sankhyā: The Indian Journal of Statistics, Series A, 26(4), 359-372. https://www.jstor.org/stable/25049340
Xu, X., & Douglas, J. (2006). Computerized adaptive testing under nonparametric IRT models. Psychometrika, 71(1), 121-137. https://doi.org/10.1007/s11336-003-1154-5