Scaling up collaborative dialogue analysis: an AI-driven approach to understanding dialogue patterns in computational thinking education

Abstract

Pair programming is a collaborative activity that enhances students' computational thinking (CT) skills. Analyzing students' interactions during pair programming provides valuable insights into effective learning. However, interpreting classroom dialogues is a challenging and complex task. Due to the simultaneous interaction between interlocutors and other ambient noise in collaborative learning contexts, previous work heavily relied on manual transcription and coding, which is labor-intensive and time-consuming. Recent advancements in speech and language processing offer promising opportunities to automate and scale up dialogue analysis. Besides, previous work mainly focused on task-related interactions, with little attention to social interactions. To address these gaps, we conducted a four-week CT course with 26 fifth-grade primary school students. We recorded their discussions, transcribed them with speech processing models, and developed a coding scheme and applied LLMs for annotation. Our AI-driven pipeline effectively analyzed classroom recordings with high accuracy and efficiency. After identifying the dialogue patterns, we investigated the relationships between these patterns and CT performance. Four clusters of dialogue patterns have been identified: Inquiry, Constructive Collaboration, Disengagement, and Disputation. We observed that Inquiry and Constructive Collaboration patterns were positively related to students' CT skills, while Disengagement and Disputation patterns were associated with lower CT performance. This study contributes to the understanding of how dialogue patterns relate to CT performance and provides implications for both research and educational practice in CT learning.