Basser Seminar Series

Opportunities and Challenges of the New Field of Multimodal Learning Analytics

Speaker: Dr Sharon Oviatt
Incaa Designs

Time: Monday 16 September 2013, 12-1pm

Location: The University of Sydney, School of IT Building, Lecture Theatre (Room 123), Level 1

Add seminar to my diary


Multimodal learning analytics (MMLA) is an emerging field within big data learning analytics. It analyzes natural communication modalities that represent rich multimodal information sources (speech, visual images, digital pen) as students work on learning activities. From an educational viewpoint, MMLA aims to identify domain expertise and consolidation of expertise rapidly, reliably, and objectively. The emergence of MMLA also is compatible with rapid adoption of cell phones with multimodal input capabilities, which now are the dominant educational technology platform worldwide.

From an engineering viewpoint, MMLA provides an opportunity to develop faster, more robust, and more continuous techniques for evaluating students’ learning and the impact of educational technologies and other resources. MMLA aims to achieve predictive power based on high quality data, including coherent patterns available in multimodal information streams, rather than simply large volumes of data. It also aims to combine empirical and machine learning techniques to bootstrap their combined advantages (e.g., transparency, causal inference, consensual metrics).

To support the development of relevant new analysis, modeling, and prediction techniques, MMLA data resources such as the Math Data Corpus have become publicly available to participants in data-driven grand challenge workshop events - such as the upcoming Second Intl. Grand Challenge Workshop on Multimodal Learning Analytics at the ICMI Conference in Sydney on Dec. 9, 2013. I’ll describe these MMLA data resources, their ground-truth coding, initial research findings and benchmark prediction results, and future studies now in the planning stages. This work represents a collaborative multidisciplinary effort among scientists at Incaa Designs, UCSD, USC, Stanford, and international partners. To conclude, discussion will be encouraged on opportunities, challenges, and implications of MMLA research for both education and computer science.

Speaker's biography

Sharon Oviatt is internationally known for her extensive work in human-centered interface design, multimodal and mobile interfaces, educational interfaces, and interface design and evaluation. She has been recipient of a National Science Foundation Special Creativity Award for pioneering research, and has published over 140 scientific articles in a wide range of multidisciplinary venues - including a new book on The Design of Future Educational Interfaces (Routledge Press, 2013). She is an Associate Editor of the main journals and edited book collections in the field of human interfaces, including the journal of Human Computer Interaction, ACM Transactions on Intelligent Interactive Systems and The Human-Computer Interaction Handbook. She also is the founder of Incaa Designs, a Seattle-area nonprofit that researches, designs, and evaluates the impact of new educational interfaces. She was a founder the ACM conference series on Multimodal Interfaces (ICMI), and currently is Co-Chair of ICMI 2013 in Sydney with colleagues Fang Chen (NICTA) and Julien Epps (UNSW).