The JavaTutor System

JavaTutor is an online learning environment to teach computational concepts in introductory Computer Science. Students progress through a series of units where they build an interactive text adventure using the Java programming language. The units were designed to supplement introductory course content, progressively bringing the students from programming statements and variables to higher-level control flow constructs, such as if-statements and while loops. The content, activities, and support provided by the system were specifically designed to both motivate and engage novice programmers.

There are two versions of the JavaTutor system, the human tutoring version, and the automated tutoring version. In the human tutoring version students interact through a chat system with an expert tutor who supports cognitive and affective aspects of learning. The dialogue model for the automated tutoring version was built on cognitive and affective utterances of human tutors. For the automated tutoring version, the system provides automated feedback created through machine learning on dialogue of human tutors. The automated feedback system encourages students as they encounter successes and failures, while also providing specific guidance based on task progress and errors.

JT Interface 2 JT Interface
Human Tutoring Version of JavaTutor
Automated Tutoring Version of JavaTutor

The automated JavaTutor system was developed through modeling cognitive processes and affective states utilized by the learner throughout a human-human tutoring session.

Cognitive Scaffolding

A key function of tutoring is providing cognitive scaffolding that is tailored to the knowledge of each learner. A central research question that the team addressed over the course of the project was how to represent the cognitive scaffolding in human­-human tutorial dialogue in a way that supported acquiring machine­-learned models. Building on dialogue act theory and computational linguistics technologies, the team explored and compared novel supervised and unsupervised approaches to learning cognitive scaffolding models. These approaches included supervised and reinforcement learning and the incorporation of task information to enrich the dialogue features, yielding better performance than previously available with automated dialogue models.

Affective Scaffolding

Highly effective human tutors simultaneously address cognitive and affective states of learners, adapting to the appropriate level of content difficulty and improving learner motivation through personalized instruction. Just as human tutors consider more than task performance of the student, incorporating computational mechanisms for drawing inferences about learner affect into tutorial interventions is a promising line of investigation. The JavaTutor project utilizes both verbal and nonverbal behavior to ascertain student affect. The team developed machine-­learning models to predict learning and affect by combining multimodal nonverbal behaviors and task actions from the observational tutoring corpus. This body of research findings regarding how students experience affect during tutoring, and how these experiences shape both the learning and affective outcomes of the learner, has advanced the state of knowledge in affective computing.


Comments are closed