Equity Strategies in the Design of Student Competency Assessment

Introduction. The article presents the current situation of the problem under consideration in the works of Russian and foreign authors. The diffi  of assessing competencies as complex, deeply latent and interdisciplinary in nature learning outcomes are considered. The study is relevant due to the need to ensure the reliability of estimates. Its novelty lies in the presentation of an algorithm for the development of competence assessment based on the modelling of the construct and conditions for the manifestation of competencies in the process of students’ activities when performing a complex task.
Purpose. A systematic presentation of modern approaches to the authentic assessment of competencies based on the theory and practice of pedagogical measurements and experience in designing and using tools based on models of evidencebased argumentation that ensure the fairness of the evaluation process and the authenticity of the evaluation results.
Materials and Methods. As theoretical methods, work with sources, collection, generalization, structuring and analysis of information on existing approaches to assessing the students’ competencies in Russia and abroad were used. In the practical part, the experience of organizing and conducting various forms of evaluation of learning outcomes in the context of competence-based learning of students was used.
Results. Modern approaches and mechanisms for designing evaluation tools based on models of reasoned proof of evaluation results based on the theory and practice of pedagogical measurements are proposed. Extended pattern designs have been created for teachers, and developers of assessment tools and procedures, ensuring the authenticity of the results through the analysis and interpretation of data, considering contextual information about the activities of the subjects in the assessment situation.
Discussion and Conclusion. The interrelation of the problems of competence assessment with their complexity and interdisciplinarity is analyzed, and an appeal to the models of S. Tulmin and R. Mislevi is proposed in the process of developing evaluation tools. The trends in control development and evaluation activities for the measurement of training quality are shown.

1. Almond, R. G., Mislevy, R. J., Steinberg, L. et al. (2019). Bayesian networks in educational assessment. New York: Springer.

2. Avdeeva, S. M., Rudnev, M. G., Vasin, G. M. et al. (2017). Assessment of information and communication competence of students: approaches, tool, validity and reliability of results. Educational Studies, 4, 104−132. https://doi.org/10.17323/1814-9545-2017-4-104-132 (In Russ.).

3. Care, E., Griffin, P., Wilson, M. (2018). Assessment and Teaching of 21st Century Skills. Research and Applications. New York: Springer. http://dx.doi.org/10.1007/978-3-319-65368-6

4. Chandra, M. Yu., Baikina, E. A. (2018). Strategies for assessing students’ competencies in the process of mastering the educational program of the university. Bulletin of Volgograd State Pedagogical University, 7(130), 10–15. (In Russ.).

5. Efremova, N. F. (2018b). Arguments and Evidence of Reliability Assessment of Students’ Competencies. DSPU Journal, Psychological and pedagogical sciences, 12(2), 43−50. (In Russ.).

6. Efremova, N. F. (2019). Conceptual model for assessing students’ competencies. Modern science-intensive technologies, 7, 169−174. https://doi.org/10.17513/snt.37607 (In Russ.).

7. Griffin, P., Care, E., McGaw, B. (2012). The Changing Role of Education and Schools. In P. Griffin, B. McGaw, E. Care (eds): Assessment and Teaching of 21st Century Skills. Research in Science Teaching (pp. 1–15). New York: Springer. https://www.doi.org/10.1007/978-94-007-2324-5_1

8. Jumariati, Febriyanti, E. R., Rizki, M. (2021). Argumentation Skills: An Analysis on EFL Students’ Essays Based on Toulmin’s Model of Argument. In: Advances in Social Science, Education and Humanities Research, Proceedings of the 2nd International Conference on Education, Language, Literature, and Arts (ICELLA 2021), 587 (pp 86–95). https://www.doi.org/10.2991/assehr.k.211021.013

9. Messick, S. (1994). The interplay of evidence and consequences in the validation of performance assessments. Education Researcher, 23(2), 13–23. https://www.doi.org/10.3102/0013189x023002013

10. Mislevy, R. J., Almond, R. G., Lukas, J. F. (2003). A Brief Introduction to Evidence-centered Design. Educational Testing Service, 2003(1), i–29. https://www.doi.org/10.1002/J.2333-8504.2003.TB01908.X

11. Mislevy, R. J., Haertel, G., Riconscente, M. et al. (2017). Assessing Model-Based Reasoning using Evidence-Centered Design: A Suite of Research-Based Design Patterns. New York: Springer. https://www.doi.org/10.1007/978-3-319-52246-3

12. Mislevy, R. J., Levy, R. (2007). Bayesian psychometric modeling from an evidence-centered design perspective. Handbook of statistics, 26, 839–865. https://doi.org/10.1016/S0169-7161(06)26026-7

13. Nefedov, S. N. (2017). Toulmin’s proof model and quantitative indicators of the reliability of the conclusion. Issues of criminalistics and forensics, 1(41), 87−93. (In Russ.).

14. Oliveri, M. E., Mislevy, R. J. (2019). Introduction to “Challenges and Opportunities in the Design of ‘NextGeneration Assessments of 21st Century Skills’ Special Issue. International Journal of Testing, 19(2), 97–102. https://www.doi.org/10.1080/15305058.2019.1608551

15. Toulmin, S. E. (2003). The Uses of Argument. Cambridge: Cambridge University Press.

16. Uglanova, I. L., Brun, I. V., Vasin, G. M. (2018). Evidence-Centered Design methodology for measuring complex psychological constructs. Modern foreign psychology, 7(3), 18–27. https://www.doi.org/10.17759/jmfp.2018070302 (In Russ.).

17. Weideman, A. (2012). Validation and Validity Beyond Messick. Per Linguam, 28(2), 1–14. https://doi.org/10.5785/28-2-526

18. Yefremova, N. F. (2018a). Evidence centred design model in the design of measuring instruments. Pedagogical dimensions, 3, 8−15. (In Russ.).