Evaluation of a Maturity Model for the Adoption of Learning Analytics in Higher Education Institutions

Authors

DOI:

https://doi.org/10.17398/1695-288X.19.2.101

Keywords:

Higher Education, Learning Analytics, Politics of Education, Questionnaires, Models

Abstract

Learning Analytics (LA) aims to analyze the data generated by both students and teachers in online environments in order to promote actions to improve teaching and learning processes. The results of such analyzes can help teachers to know their students’ study processes, as well as being able to assist with the verification and correction of both educational activities and practices. For students, LA can help with reflection and self-regulation of learning. However, despite its benefits, institutions have difficulties in adopting it. In this sense, an instrument that can support the use of LA is the Maturity Model (MM), which has been used in different knowledge areas in order to indicate an improvement roadmap for organizations. Hence, this paper aims to present the assessment results of a MM proposed for the adoption of LA in Higher Education Institutions, called MMALA. The evaluation focused on the model composition and was carried out through a questionnaire addressed to LA researchers and professionals. After conducting analyzes, both qualitative and quantitative, suggestions for improvement for the proposed model were identified, and the model was validated, supporting its further development.

Downloads

Download data is not yet available.

Author Biographies

  • Elyda Laisa Soares Xavier Freitas, Federal University of Pernambuco (UFPE) / Center of Informatics

    Graduated in Information Systems from the University of Pernambuco (UPE) and Master in Computer Science from the Federal University of Pernambuco (UFPE). Has experience in Oracle Database Management, SQL, PL / SQL, Data Warehouse and Business Intelligence development, Cloud Database and programming. She is currently an assistant professor at the University of Pernambuco at the Caruaru campus and a doctoral student at the UFPE Computer Center, conducting research in the areas of Learning Analytics and Non-Relational Databases.

  • Fernando da Fonseca de Souza, Federal University of Pernambuco / Center of Informatics

    He holds a Bachelor's degree in Civil Engineering from the Federal University of Pernambuco (1976), master's degree in computer science from the Federal University of Pernambuco (1982) and a Ph.D. in Computer Sscience from the University Of Kent At Canterbury (1990). He is currently an Associate Professor in the Informatics Center of the Federal University of Pernambuco. He has experience in the area of computer science with an emphasis in databases, working mainly on the following themes: Strategic Information Management, Cloud Databases, Virtual Learning Environments and Accessibility.

  • Vinicius Cardoso Garcia, Federal University of Pernambuco / Center of Informatics

    Dr. Vinicius Garcia is an adjunct professor in the Software Engineering area at Federal University of Pernambuco, Brazil, where he is member of the LABES (Software Engineering Lab), is a senior member of the RiSE (Reuse in Software Engineering) Group and actually is the leader of the ASSERT (Advanced System and Software Engineering Research Technologies) Lab.

  • Taciana Pontual da Rocha Falcão, Federal Rural University of Pernambuco (UFRPE) / Department of Computing

    is a professor and researcher at the Federal Rural University of Pernambuco (UFRPE) in the area of ​​Human-Computer Interaction (IHC) and a member of the Graduate Programs in: Technology and Management in Distance Education (PPGTEG-UFRPE) and Applied Informatics (PPGIA) -UFRPE) PhD from the Institute of Education (IoE) - University of London, UK, her doctoral research dealt with how tangible interfaces can contribute to the learning of children with intellectual difficulties. Graduated (2004) and master's (2007) in Computer Science from the Federal University of Pernambuco. He was an interaction designer at CESAR Recife from 2010 to 2013, and a researcher at the London Knowledge Lab (University of London) in the area of ​​tangible interfaces for education, from 2008 to 2010. He has also worked at NAAH / S-Recife, activities center for children with high skills. He did post-doctoral studies at McGill University - Canada, working with the development of technologies to support the communication of the elderly. His interests include: tangible interfaces for education, technologies for education, teaching programming, computational thinking, digital games in educational contexts, usability, user experience and accessibility.

  • Elaine Cristina Moreira Marques, Federal Rural University of Pernambuco (UFRPE) / Department of Computing
        PhD student in Biometrics and Applied Statistics at the Federal Rural University of Pernambuco (UFRPE) - Recife (2019). Master in Biometrics and Applied Statistics from the Federal Rural University of Pernambuco (UFRPE) - Recife (2018). Graduated in Statistics (Bachelor) from the Federal University of Paraíba - João Pessoa (2015). She is currently a Distance Tutor at the Federal Institute of Education, Science and Technology of Pernambuco - (IFPE) - Recife (2019). She was substitute professor at the Federal Rural University of Pernambuco (UFRPE) - Recife (2018). She was an intern at the Brazilian Micro and Small Business Support Service Company - SEBRAE / PB, at the Strategic Management and Monitoring Unit (UGEM) - João Pessoa (2015). Works in the area of ​​Probability and Statistics, Biological Testing and Bioequivalence, Experiment Planning and in the area of ​​Artificial Intelligence (AI) with an emphasis on Text Mining (MT), with a focus on information retrieval (IR) and text classification (TC) ).
  • Rafael Ferreira Mello, Federal Rural University of Pernambuco (UFRPE) / Department of Computing

    Rafael Ferreira Mello is a doctor in computer science with research interests that include natural language processing, learning analytics and educational technology. He is a professor at the Federal Rural University of Pernambuco, Brazil, where he is one of the coordinators of the Artificial Intelligence Laboratory (https://aiboxlab.org/), develops several initiatives related to learning analytics (LA) and supervises undergraduate students and postgraduate studies. Dr. Mello held a postdoc at the University of Edinburgh School of Informatics in 2018, where he continues to have collaborations. He has worked on several multinational research projects, involving academic partners and companies in Europe, Australia, the United States and Latin America. Dr. Mello's previous experience also includes a research project funded by Hewlett-Packard, where he served as a technical leader. A key theme of his recent research has been the use of natural language processing applied as education and themes more focused on text analysis such as summarization and classification.

References

Almeida Neto, H., Magalhães, E., Moura, H., Teixeira Filho, J., Cappelli, C., & Martins, L. (2015). Avaliação de um Modelo de Maturidade para Governança Ágil em Tecnologia da Informação e Comunicação. iSys – Revista Brasileira de Sistemas de Informação, Rio de Janeiro, 8(4), 44-79. Recuperado a partir de http://www.seer.unirio.br/index.php/isys/article/viewFile/5176/4938

Arnold, K., & Pistilli, M. (2012). Course signals at Purdue: using learning analytics to increase student success. Proceedings of the International Conference on Learning Analytics and Knowledge - LAK '12, New York, NY, USA, 267-270. https://doi.org/10.1145/2567574.2567621

Becker, J., Knackstedt, R., & Pöppelbuß, J. (2009). Developing maturity models for IT management – A Procedure Model and its Application. Business & Information Systems Engineering, 1(3), 213–222. https://doi.org/10.1007/s12599-009-0044-5

CMMI. (2010). CMMI para Desenvolvimento (v1.3). Software Eng. Institute, Carnegie Mellon.

Dawson, S., Joksimovic, S., Poquet, S., & Siemens, G. (2019). Increasing the Impact of Learning Analytics. Proceedings of the International Conference on Learning Analytics and Knowledge - LAK’19, Tempe, Arizona, USA, 446-455. https://doi.org/10.1145/3303772.3303784

DAMA International. (2009). The DAMA guide to the data management body of knowledge (DAMA-DMBOK), Tech. Publications.

DMM. (2014). Data management maturity model – 1.0 version. CMMI Institute.

Ferguson, R. (2012). Learning analytics: drivers, developments and challenges. International Journal of Technology Enhanced Learning, 4 (5/6), 304-317. https://doi.org/10.1504/IJTEL.2012.051816

Freitas, E. L. S. X., Souza, F. F., & Garcia, V. C. (2019). Learning Analytics em Ação: Uma Revisão Sistemática de Literatura. Anais do Simpósio Brasileiro de Informática na Educação - SBIE, Brasília. https://doi.org/10.5753/cbie.sbie.2019.1581

Freitas, E. L. S. X., Souza, F. F., Garcia, V. C., Mello, R. F., & Gasevic, D. (2020). Towards a Maturity Model for Learning Analytics Adoption: An Overview of its Levels and Areas. Proceedings of the International Conference on Advanced Learning Technologies (ICALT), Tartu, Estonia, 2020. https://doi.org/10.1109/ICALT49669.2020.00059

Gallego, F. O., & Corchuelo, R. (2020). An encoder–decoder approach to mine conditions for engineering textual data. Engineering Applications of Artificial Intelligence, 91, 103568. https://doi.org/10.1016/j.engappai.2020.103568

Gewerc, A., Rodríguez-Groba, A., & Martínez-Piñeiro, E. (2016). Academic Social Networks and Learning Analytics to Explore Self-Regulated Learning: a Case Study. IEEE Revista Iberoamericana de Tecnologias del Aprendizaje, 11(3), 159-166. https://doi.org/10.1109/RITA.2016.2589483

Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E., & Tatham, R. L. (2009). Análise multivariada de dados. Bookman Editora.

Halper, F., & Stodder, D. (2014). TDWI analytics maturity model guide. TDWI Research. Recuperado a partir de https://tdwi.org/pages/maturity-model/analytics-maturity-model-assessment-tool.

Hiles, A. (2010). The Definitive Handbook of Business Continuity Management, 3ª Ed., Wiley.

Johnson, L., Smith, R., Willis, H., Levine, A., & Haywood, K. (2011). The 2011 Horizon Report. Austin, Texas, The New Media Consortium. Recuperado a partir de https://library.educause.edu/-/media/files/library/2011/2/hr2011-pdf.pdf.

Keystone Strategy. (2016). Data & analytics maturity model & business impact. White Paper. Recuperado a partir de https://info.microsoft.com/rs/157-GQE-382/images/EN-CNTNT-SQL-Data%20Analytics%20Maturity%20Model-en-us.pdf

Kitto, K., Cross, S., Waters, Z., & Lupton, M. (2015). Learning analytics beyond the LMS: the connected learning analytics toolkit. Proceedings of the International Conference on Learning Analytics And Knowledge - LAK '15. New York, NY, USA, 11-15. https://doi.org/10.1145/2723576.2723627.

Li, M., & Smidts, C. (2003). A ranking of software engineering measures based on expert opinion. IEEE Transactions on Software Engineering, 29(9), 811–824. https://doi.org/10.1109/TSE.2003.1232286

Lockyer, L., Heathcote, E., & Dawson, S. (2013). Informing Pedagogical Action: Aligning Learning Analytics With Learning Design. American Behavioral Scientist, 57(10), 1439-1459. https://doi.org/10.1177/0002764213479367.

Pedhazur, E. J., & Schmelkin, L. P. (2013). Measurement, design, and analysis: An integrated approach. Psychology Press.

Rau, M. A., Aleven, V., & Rummel, N. (2014). Sequencing Sense-Making and Fluency-Building Support for Connection Making between Multiple Graphical Representations. En J. Polman, E. Kyza, D. K. O'Neill, I. Tabak, W. R. Penuel, A. S. Jurow, K. O'Connor, T. Lee, L. D'Amico (Eds.). Learning and Becoming in Practice: The International Conference of the Learning Sciences (ICLS), (2, 977-981).

Tempelaar, D. T., Rienties, B., & Giesbers, B. (2015). In search for the most informative data for feedback generation: Learning Analytics in a data-rich context. Computers in Human Behavior, 47, 157-167. https://doi.org/10.1016/j.chb.2014.05.038.

Tlili, A., Essalmi, F., Jemni, M., & Kinshuk. (2015). An educational game for teaching computer architecture: Evaluation using learning analytics. Proceedings of the International Conference on Information & Communication Technology and Accessibility (ICTA), Marrakech, 1-6. https://doi.org/10.1109/ICTA.2015.7426881.

Tsai, Y. S., & Gašević, D. (2017). The State of Learning Analytics in Europe – Executive Summary – SHEILA. Recuperado a partir de http://sheilaproject.eu/2017/04/18/the-state-of-learning-analytics-in-europe-executive-summary.

Tsai, Y., Moreno-Marcos, P., Jivet, I., Scheffel, M, Tammets, K., Kollom, K., & Gasevic, D. (2018). The SHEILA framework: informing institutional strategies and policy processes of learning analytics. Journal of Learning Analytics, 5(3), 5-20. https://doi.org/10.18608/jla.2018.53.2

União Europeia. (2014). Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions. Towards a thriving data-driven economy, SWD(2014) 214 final. Brussels. Recuperado a partir de https://ec.europa.eu/information_society/newsroom/cf/dae/document.cfm?doc_id=6216.

Yassine, S., Kadry, S., & Sicilia, M. A. (2016). A framework for learning analytics in moodle for assessing course outcomes. Proceedings of the IEEE Global Engineering Education Conference (EDUCON), Abu Dhabi, 261-266. https://doi.org/10.1109/EDUCON.2016.7474563.

Downloads

Published

2020-12-13

Issue

Vol. 19 No. 2 (2020)

Section

Articles

License

Authors who publish in this journal accept the following conditions:

Authors retain copyright over their works and grant the journal the right of first publication. Articles are published under the Creative Commons Attribution 4.0 International License (CC BY 4.0), which allows third parties to share, copy, distribute, publicly communicate, adapt, transform, and reuse the work in any medium or format, including for commercial purposes, provided that authorship is properly acknowledged, the original source is cited, a link to the license is included, and any changes made are indicated. Note: This license applies to articles published from Vol. 25, No. 2, 2026 onwards.
Authors may enter into separate and additional contractual arrangements for the non-exclusive distribution of the published version of the article —for example, its deposit in an institutional repository or its subsequent inclusion in a book—, provided that it is clearly stated that the work was first published in this journal.
Authors are permitted and encouraged to deposit and disseminate their work on the Internet, for example, in institutional repositories, institutional websites, or personal websites before, during, and after the review and publication process, as this may foster scholarly exchange, increase the visibility of the work, and enable broader and faster dissemination of the published research.

How to Cite

Freitas, E. L. S. X., Souza, F. da F. de, Garcia, V. C., Falcão, T. P. da R., Marques, E. C. M., & Mello, R. F. (2020). Evaluation of a Maturity Model for the Adoption of Learning Analytics in Higher Education Institutions. Latin American Journal of Educational Technology - RELATEC, 19(2), 101-113. https://doi.org/10.17398/1695-288X.19.2.101

Similar Articles

1-10 of 201

You may also start an advanced similarity search for this article.

Most read articles by the same author(s)