Research & Technology



In a collaborative learning environment, teachers must conduct multiple forms of learning activities which require significant skill to orchestrate, including helping groups coordinate their collaborations, managing whole-class activity, transitioning between activities, and supporting individuals in their uptake of productive learning practices. This complex environment places significant demands on teachers. The I-ACT project seeks to build an orchestration assistant, or OA, which combines the data students generate while playing the video game with sophisticated artificial intelligence technologies to support the teacher in orchestrating learning.

The I-ACT orchestration assistant will be designed to directly support K-12 science teachers by providing context-sensitive guidance. It will be driven by AI-based multimodal learning analytics from six data streams: learning environment interaction traces, motion tracking, facial expression tracking, gesture tracking, gaze tracking, and assessment data. I-ACT science classrooms will track students’ problem-solving interaction data. I-ACT will track not only the location and movement of teachers and students in the classroom but also students’ facial expressions, gestures, and gaze to supply a rich set of learner information. Additionally, the I-ACT orchestration assistant will collect assessment data during the course of students’ problem solving. Finally, the orchestration assistant will provide tools for teachers to plan lessons, adjust them in real time, and reflect on them to make productive changes for the future. Envisioning the future of teaching, this project seeks to design and evaluate the I-ACT orchestration assistant to support teachers by helping them orchestrate classrooms while also automating some processes, freeing them to do things at which teachers are experts: supporting students in deep, collaborative, and meaningful learning. We accomplish this through two related threads of research and technology design:

  1. Design and develop I-ACT cognitive assistants for K-12 STEM teachers and implement them in public school classrooms. Utilizing AI-based multimodal learning analytics and a social constructivist theory of pedagogy, I-ACT cognitive assistants will use three complementary families of machine-learned models of teacher orchestration to provide guidance throughout the full teaching workflow: Bayesian linear regression, probabilistic graphical models, and deep neural networks. I-ACT cognitive assistants will operate in a tight feedback loop in which collected data will drive successive iterations of machine learning to train refined teacher support models for improved I-ACT cognitive assistant functionalities.
  1. Investigate how I-ACT cognitive assistants improve K-12 STEM teacher performance and teacher quality of work-life. Each year, we will conduct focus groups, case studies, semi-structured interviews, and observations of teachers using I-ACT cognitive assistants in school implementations with middle school science teachers at the project’s partner schools in North Carolina and Indiana. In the final two years of the project, we will conduct quasi-experimental studies to determine (1) I-ACT impact on teacher performance using the Reformed Teaching Observation Protocol instrument for Student-Teacher Interaction, Student-Student Interaction, Lesson Design & Implementation, Propositional Pedagogical Knowledge, and Procedural Pedagogical Knowledge, and (2) I-ACT impact on teacher quality of work-life as reflected by teachers’ ratings of job satisfaction and teacher self-efficacy.


OpenPose Classroom Capture 1

Comments are closed