MLTI Lab
 ⟶  MLTI Lab / Projects

Inquiries
in motion.

04
ACTIVE GRANTS
$482,243+
COMBINED
40+
AWARDS RECEIVED
19+
SPONSORS
2026 · 01
ACTIVE

Multimodal STEM+AI Classrooms: A Sociocultural Inquiry into Participatory Design for Collaborative Learning

SPONSOR
University of Virginia
PROGRAM
IDEAs (Innovative, Developmental, Exploratory Awards)
AWARD
$10,000
PERIOD · ROLE
Mar 2026 – Mar 2027
PI
This participatory design initiative develops an AI-enhanced Multimodal Learning Analytics (MMLA) platform for collaborative K-12 STEM classrooms, co-designed with a transdisciplinary team of researchers, educators, and developers. Grounded in sociocultural perspectives, the project examines how MMLA platforms can be responsibly integrated into secondary learning environments, attending to the cultural, social, and material contexts in which collaborative STEM+AI learning unfolds. The project also builds a foundational multimodal dataset to support computer vision research for video-based MMLA.
CONTRIBUTORSview
Miguel Lujan
AI Developer, Data Analyst
United States Department of War
Camyla Gonzalez
Undergraduate Research Assistant
University of Virginia
Dong-Geon Kim
Visiting Undergraduate Research Assistant
Korea Advanced Institute of Science & Technology (KAIST)
Hyun G Kwon
Hyun G Kwon
Research Professor · Community Partner
Korea University · The Society of Technology for Education and Learning Analytics
Jongheon Kim
Jongheon Kim
School Partner
Sejong Academy of Science and Arts · South Korea
Jeehun Sung
AI Developer
MooWee Co., Ltd
Jandi Choi
Jandi Choi
Junior Designer & Developer
Korea Institute of Curriculum and Evaluation
Hyeri Mel Yang
Hyeri Mel Yang
Incoming Ph.D. Student · Education (Major)
Harvard University · United States
Chaeyeon Kim
Chaeyeon Kim
Ph.D. Student · Learning Sciences (Major)
University of Wisconsin–Madison · United States
OUTCOMESview
Active project · early outputs, more emerging.
RESEARCH01
  • Lee, H. H., Kwon, H. G., Kim, J., Sung, J., Choi, J., Yang, H., Kim, C., & Lujan, M. (2026, June). Influence and negotiation in co-designing an AI-enhanced Multimodal Learning Analytics platform for secondary STEM classrooms. In Proceedings of the Impactful and Responsible AI Systems for Education Workshop (IRAISE) at the Festival of Learning 2026. Proceedings of Machine Learning Research Research
TOOLS02
  • The co-designed AI-enhanced multimodal learning analytics platform for collaborative STEM classrooms.
  • Multimodal Analytics Support Studio: Turning observational video into structured, shareable insight for research.
2026 · 02
STARTING

Multimodal Futures: Responsible AI for Equitable, Multilingual STEM Education Across Global Contexts

SPONSOR
University of Virginia Center for Global Inquiry and Innovation
PROGRAM
Global Programs of Distinction Grants
AWARD
$70,000
PERIOD · ROLE
Aug 2026 – Jul 2027
PI
AI is reshaping education, yet most AI in Education (AIED) systems are built on Western, monolingual, and individualistic models of learning that overlook the diverse ways students engage in STEM practices. The Global Responsible AI in Education Collaborative reimagines AIED through multimodal, multilingual, and globally grounded lenses, leveraging AI to make the invisible visible and the silenced heard. This participatory design project brings together researchers, practitioners, community partners, and families from four hubs, Virginia, New Mexico, Brazil, and South Korea, to examine how AI can support more equitable K-12 STEM education and develop research priorities and actionable guidelines for stakeholders.
2026 · 03
STARTING

Building Capacity in Computer Vision and Spatiotemporal Deep Learning for AI-Enhanced Multimodal Learning Analytics

SPONSOR
University of Virginia
PROGRAM
AI Catalyst Program 2026–27
AWARD
$8,000
PERIOD · ROLE
Aug 2026 – Aug 2027
PI
This project builds methodological capacity in computer vision and spatiotemporal deep learning for AI-enhanced Multimodal Learning Analytics in K-12 STEM classrooms.
2026 · 04
ACTIVE

Creating Open Access Textbook for Learning Theories and Practices: Past, Present, and Future

SPONSOR
The Virtual Library of Virginia (VIVA)
PROGRAM
VIVA Open Creation Grants
AWARD
$37,000
PERIOD · ROLE
Jul 2026 – Jun 2029
Co-PI (PI: Jinhee Kim, Old Dominion University)
This project develops an openly licensed, interactive textbook on learning theories to advance affordability, equity, and student success across education programs. Existing commercial texts are costly, fragmented, and misaligned with digital and AI-enhanced learning contexts. Replacing multiple required textbooks with a single open resource will reduce student costs and improve instructional coherence. Aligned with VIVA Open priorities, the textbook integrates diverse perspectives, inclusive practices, and interactive case-based learning. Its modular, open design supports sustainability, regular updates, and cross-institutional adoption.

Past projects

Prior grants that shaped the current research program.
Equity Learning Analytics: A Multimodal Approach to Understanding Group Dynamics through Verbal, Nonverbal, and Object-Based Interactions in Collaborative Learning
Jan 2024 – May 2024
Learnest · Learnest Ethical AI in Education Fellowship · $2,000 · PI
2025
University Graduate Continuing Fellowship (Outstanding Graduate Research Fellowship) · The University of Texas at Austin · $64,170
OUTCOMESview
RESEARCH10
More journal articles from this project are forthcoming.
  • Lee, H. H., Sung, H., Celedón-Pattichis, S., & Pattichis, M. S. (2026). Toward an inclusive understanding of collaborative learning using MMLA: Exploring multimodal participation dynamics by gender and linguistic diversity. Learning and Instruction, 102, 102317. https://doi.org/10.1016/j.learninstruc.2026.102317
    · Special issue on Implementing multimodal learning analytics (MMLA) in ecological settings for generating actionable insights
    Research
  • Lee, H. H. (2026, June). AI-Enhanced sociocultural, multimodal approaches for equitable collaborative learning: Learning, power, and multimodality. In General Proceedings of the ISLS Annual Meeting 2026. International Society of the Learning Sciences Research
  • Lee, H. H., Lujan, M., Jatla, V., Egala, U., Pattichis, M. S., & Celedón-Pattichis, S. (2026, June). Integrating AI and interaction analysis: A mixed-methods approach to tracing object-based participation in CSCL. In Proceedings of the 19th International Conference on Computer-Supported Collaborative Learning - CSCL 2026. International Society of the Learning Sciences Research
  • Lee, H. H. (2025, June). AI-enhanced sociocultural analysis of keyboard access in CSCL: Proxy, gatekeeper, or mediator of intellectual influence? In General Proceedings of the ISLS Annual Meeting 2025. International Society of the Learning Sciences Research
  • Lee, H. H. (2025, March). Table dynamics: AI-enhanced analysis of keyboard access and power dynamics in collaborative STEM learning groups. In Companion Proceedings of the 15th International Conference on Learning Analytics & Knowledge (LAK25) (pp. 325-331). Society for Learning Analytics Research Research
  • Lee, H. H., Lujan, M., Jatla, V., Egala, U., Pattichis, M. S., Celedón-Pattichis, S., & Cantú, E. (2026, April). Multimodal participation analysis in CSCL through AI-enhanced Activity Mapping and screen recording video analysis systems [Conference presentation]. The Annual Meeting of the American Educational Research Association (AERA 2026), Los Angeles, CA, United States. Research
  • Lee, H. H., Celedón-Pattichis, S., Jatla, V., & Pattichis, M. S. (2025, April). Exploring keyboard interactions and power dynamics in collaborative groups using AI for tracking human activity [Conference presentation]. The Annual Meeting of the American Educational Research Association (AERA 2025), Denver, CO, United States.
    🏆 Nominated for the Advanced Technologies for Learning/Learning Sciences (ATL/LS) Special Interest Groups’ Best Student Paper Award.
    Research
  • Lee, H. H. (2025, April). AI-enhanced sociocultural analysis of keyboard access in collaborative computer science learning using 3D convolutional neural networks [Conference presentation]. 2025 Korean American Educational Researchers Association (KAERA) Conference, Denver, CO, United States. Research
  • Lee, H. H. (2024, May 31). Equity Learning Analytics: A Multimodal Approach to Understanding Group Dynamics through Verbal, Nonverbal, and Object-Based Interactions in Collaborative Learning [Invited presentation]. Learnest Inaugural Ethical AI in Education Colloquium at Stanford University, CA, USA (Invited to give a presentation as a fellowship recipient). Research
  • Lee, H. H., Borchers, C., Liu, X., & Zhang, J. (2024, May 31). Ethical AIED and AIED Ethics: Toward Synergy Between AIED Research and Ethical Frameworks [Invited presentation]. Learnest Inaugural Ethical AI in Education Colloquium at Stanford University, CA, USA (Invited to give a presentation as a fellowship recipient). Research
An In-Depth Investigation of Student Interest Growth in STEM
Feb 2024 – May 2024
The University of Texas at Austin, Department of STEM Education · Graduate Student Research Scholarship · $500 · PI
OUTCOMESview
RESEARCH05
  • Lee, H. H., & Celedón-Pattichis, S. (revised manuscript submitted February 2026; initial submission September 2024). Toward an improved understanding and modeling of interest development. Cognition and Instruction
  • Lee, H. H., Celedón-Pattichis, S., LópezLeiva, C., Pattichis, M., & Song, Y. (2024, June). Emergent interests to well-developed interests in STEM. In R. Lindgren, T. I. Asino, E. A. Kyza, C. K. Looi, D. T. Keifert, & E. Suárez (Eds.), Proceedings of the 18th International Conference of the Learning Sciences - ICLS 2024 (pp. 1295–1298). International Society of the Learning Sciences. https://doi.org/10.22318/icls2024.111198 Research
  • Lee, H. H., Celedón-Pattichis, S., Pattichis, M. S., Johnson, A. R., Cantú, E., Tovar, I., & LópezLeiva, C. A. (2023, April). Knowing and enjoying: Expanding Latinx students’ experiences with an integrated computer programming and mathematics curriculum [Conference presentation]. The Annual Meeting of the American Educational Research Association (AERA 2023), Chicago, IL, United States. https://doi.org/10.3102/2012330 Research
  • Lee, H. H. (2024, April). An in-depth investigation of student interest growth in STEM [Conference presentation]. 2024 Korean American Educational Researchers Association (KAERA) Conference, Philadelphia, PA, United States. Research
  • Lee, H. H., Celedón-Pattichis, S., Pattichis, M. S., LópezLeiva, C. A., & Cantú, E. (2024, April). Integrating computer science and mathematics for enhanced learning using object-oriented programming and robotics: mixed methods [Conference presentation]. The Annual Meeting of the American Educational Research Association (AERA 2024), Philadelphia, PA, United States. https://doi.org/10.3102/2110022 Research
Responsible AI: Why did mentors hide from mentees they had seen their mentees’ learning analytics data?
Jun 2022 – Aug 2022
The University of Texas at Austin, Good Systems Ethical AI at UT Austin · Good Systems Graduate Fellowship Award · $6,000 · PI
OUTCOMESview
RESEARCH05
  • Lee, H. H., & Gargroetzi, E. (2023). “It’s like a double-edged sword”: Mentor perspectives on ethics and responsibility in a learning analytics–supported virtual mentoring program. Journal of Learning Analytics, 10(1), 85-100. https://doi.org/10.18608/jla.2023.7787 Research
  • Lee, H. H. *, & Kwon, H. G.* (2023). Practicing learning analytics and ethics. In J. Moon, G. Choi, H. Bae, & J. Byun (Eds.), Educational/instructional technology and learning sciences: Korean open access guide. EdTech Books. https://doi.org/10.59668/431.13536
    *Authors contributed equally to this work.
    Research
  • Lee, H. H. (2022, September 28). Responsible AI: Why did mentors hide from mentees they had seen their mentees’ learning analytics data? [Invited presentation]. 2022 Good Systems Graduate Fellowship Lightning Talks, Austin, TX, USA (Invited to give a presentation as a fellowship recipient). Research
  • Lee, H. H. (2022, December 3). Mentor Perspectives on Ethics and Responsibility in a Learning Analytics-Supported Virtual Mentoring Program [Invited presentation]. Korean Edutech / Learning Sciences Researcher Network (KELS) 2022 December Webinar Series – Learning Analytics Chapter, Virtual Event. Research
  • Lee, H. H., & Gargroetzi, E. (2023, April). Learning Analytics in a Mentoring Context: Through a Sociocritical Lens [Invited presentation]. 2023 Good Systems Annual Symposium: Shaping the Future of Ethical AI, Austin, TX, USA. Research
2026
ISLS Early Career Fellow
The International Society of the Learning Sciences (ISLS)
2025
Semi-Finalist, NAEd/Spencer Dissertation Fellowship
National Academy of Education/Spencer Foundation
Selected as one of 63 semi-finalists from over 400 applicants across more than 100 institutions; Respectfully withdrew from the final selection process in light of faculty appointment at the University of Virginia
2025
University Graduate Continuing Fellowship (Outstanding Graduate Research Fellowship)
The University of Texas at Austin · $64,170
2025
Michael B. Salwen Scholarship
The Korean-American Educational Researchers Association
2025
ISLS Doctoral Consortium Fellow
The International Society of the Learning Sciences (ISLS)
2025
SoLAR LAK Doctoral Consortium Fellow
The Society for Learning Analytics Research (SoLAR)
2025
AERA Learning Sciences/Advanced Technologies for Learning SIG Mentoring Program Travel Grant
Selected
2025
Graduate School Professional Development Awards
The University of Texas at Austin · $1,440
2024 – 2025
Long Joe and Teresa Graduate Fellowship, STEM Education
The University of Texas at Austin · $1,000
2024 – 2025
Health Communication Scholars Program Research Grant
“Optimizing Academic Success and Well-being Through AI: An Interdisciplinary Longitudinal SEM (Structural Equation Modeling) Study” · The University of Texas at Austin Moody College of Communication · $2,500 · Co-PI (PI: Fan Wang, The University of Texas at Austin)

Fellowships and grants

Highlighted scholarships and fellowships that supported the research journey.