Abstract

With the prevalence of learning technologies, learning analytics plays an important role in driving new insights on how people learn, but lacks a theoretical grounding to enable learning design to enhance learning outcomes effectively. On the other hand, there is a need for learning design to become more data-driven by using the insight from learning analytics. This paper aims to explore the alignment between learning analytics and learning design and how learning analytics has been leveraged to inform learning design in the unique context of higher education. Current research suggests a need to create more integration between learning analytics and learning design in higher education in order to not only ground learning analytics on learning sciences but also enable data-driven decisions in learning design to improve learning outcomes. In addition, multiple conceptual frameworks have been proposed to enhance the synergy and alignment between learning analytics and learning design. Finally, three areas of future research have been proposed to foster this synergy further in higher education: identifying learning analytics metrics in higher education that can offer insight into learning processes, evaluating the effect of learning analytics outcomes on learning design decision-making, and designing learning environments that make the capturing and deployment of learning analytics outcomes more efficient.

Keywords

learning analytics, learning design, big data in higher education

Introduction

With the prevalence of learning technologies, learning analytics plays an important role in driving new insights on how people learn. It can not only identify and validate how processes, outcomes, and activities are measured, but also fosters evidence-based approaches through evaluation of learning progress, motivation, and attitudes. However, learning analytics lacks a theoretical grounding in learning sciences to explain inconsistencies and clarify conditions that affect learning (Gasevic et al., 2017). In addition, a challenge of learning analytics is to generate data-driven interventions in order to enhance learning outcomes (Corrin et al., 2016).

Learning design enables the analysis and interpretation of learner data and learning patterns. It defines learning objectives and pedagogical approaches for instructors to make improvements on the learning experience. Learning design applies frameworks, techniques, and resources to achieve a particular learning objective in a certain learning context, and outlines the sequence of teaching methods and learning tasks (Mor & Craft, 2012). However, studies in the past have focused on creating principles in learning design, rather than evaluating learning outcomes beyond the learning design process (Rienties & Toetenel, 2016). Thus, there is a need to enhance the learning design practice using the insight from learning analytics.

This paper aims to explore the alignment and integration between learning analytics and learning design in order to not only ground learning analytics on a theoretical foundation but also help make learning design decisions that are driven by data to improve learning outcomes. It explores areas of alignment and integration between learning analytics and learning design, as well as in what ways learning analytics can be leveraged to inform learning design in the unique context of higher education effectively. The research question this paper hopes to answer is: how could learning analytics inform learning design to drive learning outcomes in higher education? To answer this question, first, the definition, lifecycle, and taxonomies of learning analytics will be introduced, followed by the definition and taxonomies of learning design. Then, learning analytics and learning design will be put into the unique context of higher education, and the benefits as well as challenges of learning analytics in this context will be discussed. Finally, how learning analytics can be utilized to inform the design of learning experiences in the unique context of higher education will be discussed and potential areas for future research will be explored.

Learning Analytics

Learning analytics has emerged in recent decades due to the increasing digitization in how people teach and learn, the creation of online learning environments, and the experiences of online learning by the public. Learning analytics is “the measurement, collection, analysis and reporting of data about learners and their contexts for purposes of understanding and optimizing learning and environments in which it occurs” (Conole et al., 2011, para. 3). Learning analytics is also an interdisciplinary field that makes use of the power of technologies and big data to develop, use, and integrate new procedures and tools for improving educational practices (Siemens & Gašević, 2012). Many of its benefits include identifying unexpected learning behaviors and successful learning patterns, increasing understanding of learner actions and progress, and creating interventions to improve learning experience.

The learning analytics lifecycle includes 4 stages: (1) generation of data, (2) data storage, (3) analysis, and (4) act (Khalil & Ebner, 2015). Learning analytics starts by capturing and collecting data from a variety of sources including online learning environments, learning management systems, forums, and web portals (Tseng et al., 2016). After being stored for further analysis, the educational datasets will be applied quantitative and/or qualitative analytics methods to uncover hidden patterns and create new insights. The analysis outcomes then get interpreted into actions, such as predictions, interventions, or recommendations.

Several taxonomies have been proposed for classifying and conceptualizing learning analytics; such taxonomies often include the purpose of learning analytics application such as predicting students at risk, the measures being used in the analysis such as the learner behavior, and the tools and techniques being used such as regression analysis (Law & Liang, 2020). For example, Peña-Ayala’s taxonomy comprises functionalities or purposes of deploying learning analytics, learner analysis, and resources which include interfaces and tools necessary for learning analytics implementation (Peña-Ayala, 2018). In another example, the taxonomy created by Nguyen et al. (2017) includes four layers: the objective layer, the data layer showing what data sources are needed or available, the stakeholder layer including people informed by and benefiting from learning analytics, and the instrument layer including theories and techniques underpinning an learning analytics application.

Learning Design

The term learning design was used to enhance the practice of instructional design that is based on behaviorism and cognitivism and to introduce interventions that are based on approaches in social constructivism enhanced by technology (Mor & Craft, 2012). Learning design is defined as a methodology that instructors use to make informed choices in creating learning experiences, including activities and interventions, that can allow them to effectively use materials, technologies, and processes (Conole, 2012). Compared with instructional design, learning design focuses more on the learning context and the application of social constructivism to develop learning experiences. Despite the fact that a shared language in this emerging field is needed, learning design can make a positive impact in developing, facilitating, and improving educational processes. Instructors then need to orchestrate all the learning design activities and interventions to achieve maximum learning effect (Prieto et al., 2018).

The most widely used learning design taxonomy is developed by OULDI that characterizes learning activities into 7 types, including assimilative, communicative, productive, experiential, and interactive or adaptive (Cross et al., 2012). It can provide an overall picture of how different learning activities may influence learning outcomes and help instructors visualize the distribution of learning activities. Another taxonomy created by Law et al. (2017) is based on the Learning Designer software platform, which includes 3 levels including learning approach, teaching and learning activity, and learning process. There are 9 learning approaches including didactic instruction, inquiry-based learning, social constructivism, experiential learning, constructionism, collaborative learning, guided discovery learning, problem-based learning, and cognitive apprenticeship (Law et al., 2017). Each learning approach is related to several teaching & learning activities and each activity in turn comprises some learning processes including acquisition, discussion, inquiry, production, and practice.

Learning Analytics in Higher Education

To navigate uncertainty and fierce competition, universities need to not only increase financial and operational efficiency, but also gain a deep understanding of the student body to serve them effectively. Higher education institutions also face the rapidly changing technology landscape and have huge amounts of data collected about students on web portals and Learning Management Systems. Additional data can be collected using Intelligent Tutoring Systems, educational games, and other online learning platforms. These data can be leveraged utilizing learning analytics to surface students’ learning progress, predict potential problems and future learning behaviors of students, as well as enable timely support for students who need it. Improving student retention has been another focus of learning analytics so that higher education institutions can counter financial losses and low graduation rates (Palmer, 2013).

The top three learning analytics techniques that have been applied to higher education are making predictions, distilling data for human judgment, and detecting outliers (Leitner et al., 2017). Each of these three analytics techniques will be introduced with an example for illustration. The prediction technique can predict student performance and learning behaviors. For example, researchers used clustering to create communities with shared career aspirations for career-specific competencies’ development (AbuKhousa and Atif, 2016). Distilling data for human judgment can help teachers understand, examine, and interpret ongoing students’ learning activities and their use of learning materials. For example, researchers developed a system that gives warning to academic advisors to help them identify students who are struggling academically (Aguilar et al., 2014). The outlier detection technique can help detect students with academic challenges or abnormal learning processes. For example, Sinclair and Kalvala (2015) used a model to detect which students in massive open online courses are more likely to drop out.

There are many benefits of leveraging learning analytics in the unique context of higher education. The techniques of learning analytics can help universities identify specific courses that can match student needs more closely and predict graduate numbers for enrollment planning (Althubaiti & Alkhazim, 2014). It can provide insight on student learning outcome, behavior, and process, allowing instructors to improve curriculum and adjust learning design accordingly. Learning analytics can also foster personalized learning with real-time feedback. Last but not least, learning analytics can support post-educational employment, surfacing vocational prospects for students and assessing occupational compatibility (Kostoglou et al., 2013). Some challenges using learning analytics in higher education involve legal and ethical considerations, including consent, data accuracy, privacy, and anonymity. Also, data ownership, preservation, and the ability to share data with parties outside of the university can generate additional concerns (Sclater, 2014).

Learning Analytics for Learning Design

In learning analytics, an important task is to develop measures that can help promote the awareness of learning processes and interpret these measures for driving actionable insights and inform how we design learning environments and experiences. Learning analytics should be aligned with learning design, because optimizing learning does not only involve collecting and processing data, but also informing the design of learning activities that meet learners’ needs. For example, when learning behavior deviates from the design intention, how we should redesign the learning experience using evidence-based recommendations.

Thus, the human aspect of understanding and improving the learning design is as important as the technical aspect of collecting and analyzing data. By creating synergy between learning analytics and learning design, we can (1) utilize learning analytics to assess learning design and (2) utilize learning design to translate learning analytics findings into effective educational interventions. Conceptual frameworks can be helpful in aligning learning analytics with learning design, with the goal of better understanding how specific design elements influence learners’ learning behaviors as well as creating alignment and enhancing communication among various stakeholders on adopting learning analytics platforms and tools.

Researchers have demonstrated the potential of utilizing learning analytics to inform learning design and since then several conceptual frameworks have been proposed to integrate learning analytics and learning design. The framework proposed by Bakharia et al. (2016) includes five learning analytics dimensions, including temporal analysis and cohort dynamics or patterns, that can support instructors with evaluations of their learning design, emphasizing the role of instructors in bringing context to data interpretation. Hernández-Leo et al. (2019) suggested a framework that includes three components (i.e., learning analytics, design analytics, and community analytics) to provide guidance on how advantages of learning analytics can be channeled into the field of learning design. Persico and Pozzi (2015) proposed a framework focusing on the representations, approaches, and tools to connect learning analytics to learning design, aiming to foster inquiry-based evaluation and learning design scaffolding. Another conceptual framework called Orchestrating Learning Analytics, or OrLA, aimed to offer guidelines to assist instructors to incorporate learning activities using learning analytics in authentic learning environments (Prieto et al., 2018).

Implications for Future Research

Despite the fact that multiple conceptual frameworks have been proposed to enhance the integration between learning analytics and learning design, future research should explore this synergy further in higher education. Since the current frameworks provide a generalizable approach across educational settings, there are some benefits for creating more nuanced and tailored approaches that fit the characteristics of adult learning in higher education. First, frameworks that explore how learning analytics can be used to inform learning design in higher education can help drive adoption given its specifically targeted learners and educational settings. Second, such frameworks are more likely to be implemented and improved as part of an academic research in the university setting. Researchers can team up with instructional designers in universities, designing experiments to test some or all parts of the framework and providing a feedback loop to evaluate and enhance how learning analytics can be utilized to inform learning design.

To create such frameworks that offer new insight into the integration of learning analytics and learning design in higher education, here are potential areas that researchers can focus on exploring further. First, it is important to identify concrete learning analytics metrics in higher education that can shed new light into learning processes and add unique value to learning designers. Currently, what we measure in learning analytics is designed on a case-by-case basis. It could be helpful to create frameworks that categorize different metrics in alignment with various educational settings and objectives.

Second, it is crucial to evaluate the effect of learning analytics outcomes on learning design decisions in higher education. Most of the existing studies on learning analytics reached certain conclusions based on data analysis without informing learning decision decisions and evaluating these decisions in further studies. In order to foster a tighter feedback loop between learning analytics and learning design, future research should clearly define the learning decision decisions that come out of learning analytics outcomes and come up with new studies to evaluate the effect of learning decision decisions on learning outcomes.

Third, it is valuable to think beyond the design of learning experiences to include the design of learning environments. Thus, it can be useful for future research to focus on designing learning environments in higher education institutions that make the capturing and deployment of learning analytics outcomes more efficient. Ideally, learning design decisions should be embedded in the learning environments directly, so that depending on the types of data that are generated by the learners, certain learning design decisions can be made in an automatic manner. Learning environments design can help enable this tight coupling of learning analytics results and learning design decisions.

Conclusion

In conclusion, this paper explores areas of alignment between learning analytics and learning design as well as how learning analytics can be leveraged to inform learning design in higher education effectively. Current research suggests a need to create more tightly integration of learning analytics and learning design in higher education to not only ground learning analytics on learning sciences but also enable data-driven decisions in learning design to improve learning outcomes. Despite the fact that multiple conceptual frameworks have been proposed to enhance the synergy between learning analytics and learning design, future research should explore this synergy further in higher education. Designing frameworks that create a tight coupling between learning analytics and learning design in higher education can not only drive adoption and implementation, but also enable a quick feedback loop to evaluate and enhance learning design decisions given outcomes from learning analytics in higher education. More specifically, future research should identify learning analytics metrics in higher education that can offer insight into learning processes, evaluate the effect of outcomes in learning analytics on learning design decisions in higher education, and design learning environments in higher education that make the capturing and deployment of learning analytics outcomes more efficient.