How pairs of students teach a robot

Two lenses on conversations in which two students teach Emma together: whose words they share, and whether they agree on how to solve the problem

These studies analyzed conversations in which two students taught Emma together: U.S. undergraduates teaching a NAO robot middle-school ratio problems by voice. This Emma used pre-scripted, pattern-matched dialogue. The research is separate from the LLM version of Emma, which one student teaches at a time, though its findings shaped that design.

I study these conversations through two lenses. Lexical alignment asks whose words the group ends up sharing. Semantic (mis)alignment asks whether the students and Emma agree on how to solve the problem.

Every example below comes from the same problem: Emma uses 1/20 of her battery in 2/3 of an hour. How much does she use in 1 hour?

Lens 1: Lexical alignment

People in conversation start using each other’s words. Between humans, this tends to signal rapport. With a teachable robot, it’s more complicated: earlier work found that rapport was higher when the robot aligned to students, but lower when students aligned to the robot. So it matters who aligns to whom, and with three speakers, existing measures either look only at adjacent turns or blur individuals together.

I extended a measure of shared expressions to multi-party conversation. An expression is initiated the first time anyone says it and established once every speaker has used it, however many turns later. This credits each speaker with their part in the group’s shared vocabulary.

Who said “two thirds” first?

From a real session (Asano et al., AIED 2025). Some turns omitted. Green italic: Emma initiates the expression. Blue: students reuse it. Bold blue: every speaker has now used it, so it is established.

Student B

Okay. So Emma, first, you’re going to take the battery usage, which is one over twenty and divide that by the time, which is two over three. So you’re gonna do one over twenty divided by two over three.

Emma

I get it. So we’re trying to figure out what times two thirds of an hour will give us one hour. But then I’m not sure what to do.

Student B said “two over three,” but Emma says “two thirds.” This is the first use of the expression, so Emma initiates it.

Student B

So then should we just tell her to multiply the battery usage by two thirds?

Student B adopts Emma’s wording while talking to the partner, not to Emma.

Student A

Let’s try.

Student A

It’s not all you’re gonna do, for this one is you’re gonna multiply the battery usage in step zero by two thirds to get what it is in step one.

Now all three speakers have used it, so the expression is established. Analyzing each pair of speakers separately would miss that it started with Emma and traveled through the group.

Student B

So on the battery usage side, you’re gonna multiply one over twenty times two thirds to get the new value.

The new measure predicts rapport better than existing multi-party and pairwise measures, and it separates Emma’s alignment from the students’. When more expressions were established by Emma, rapport with both Emma and the human partner was higher; when more were initiated by Emma, it was lower. Released as the open-source package dialign_python.AIED 2025
Students mirror a teachable robot’s wording more when teaching it alone than in pairs, and the link between that mirroring and rapport is more complex than human-human theory predicts.SIGDIAL 2022

Lens 2: Semantic (mis)alignment

Many math problems have more than one correct solution, so students and an AI partner can disagree even when both are right. I call this a solution-path conflict. In these sessions, conflicts happened when Emma misread which solution the students were teaching and then stuck to her own. I annotated each problem as having no conflict, a resolved conflict (the students explicitly reconciled their approach with Emma’s), or an unresolved one.

A resolved solution-path conflict

From a real session. Some turns omitted. The students planned to halve 1/20 and then triple it; Emma proposed scaling by the ratio of 2/3 of an hour to 1 hour.

Student B

Yeah. Emma, you should divide one twentieth in half to get one fortieth and then multiply that times three to get a full hour. Since one twentieth is two thirds of an hour.

Emma

We will multiply. Can you clarify though, what we should multiply?

Student B

You should divide one twentieth by in half to get. You could equal to one third.

Emma

So we can set up the ratio and I multiply two over three by three over two, right?

Emma has mapped the students’ explanation onto a different, also valid, solution. This is a solution-path conflict.

Student A

So, she did two over three. I don’t know why. Two over three by three over two that would be six over six, which is one. So one hour, so right?

Instead of ignoring Emma, the students work out why her approach makes sense.

Student B

I think so.

Student A

So could you use that same right ratio and ...

Student B

Multiply one twentieth by three over two? Like, if you multiply. Yeah.

Student B

You should multiply one twentieth times three over two to get the one hour.

Resolved: the students explicitly switch to Emma’s solution.

Emma

We know one over twenty times three over two is three over forty?

An unresolved solution-path conflict

From a real session. Some turns omitted. The students wanted the battery usage for 1/3 of an hour, then converted to decimals; Emma stayed with the ratio of 2/3 of an hour to 1 hour.

Student B

Try finding just the battery usage of one third of an hour and you would usually do that just by dividing by two.

Emma

I think I understand what you’re saying. So we have two over three because we have two thirds of an hour for one twentieth of the battery. Student A. Do we multiply it by something?

Emma has committed to a different solution and assumes the students are following it.

Student A

No, let’s try another way.

Emma

Okay. Student A. Can you explain what you mean?

Student A

Emma, one divided by twenty is zero point zero five, two divided by three is zero point six six, divide zero point zero five by zero point six six.

Emma

I think I get it. We’re trying to figure out what times two thirds of an hour will give us one hour. But then I’m not sure what to do.

This Emma was scripted to keep her solution, so she cannot follow the students’ approach.

Student A

Yeah, so I guess based on her response to that, she doesn’t quite understand like the decimal conversion. So like I mentioned, just try solving it step by step and as simple as possible.

Student B

Emma try dividing by battery usage by time in step zero.

Emma

Okay. Student A, why do we do that though?

Student A

So that you can find the ratio and apply that to step one and step two.

Emma

Okay. So then for one hour, do we use three fortieths of the battery?

Unresolved: the students never take up Emma’s solution, even though both reach the same answer.

Unresolved conflicts, not conflict in general, were associated with lower rapport with both Emma and the human partner. Resolved conflicts were not.
Resolving a conflict did not by itself mean students learned more. What likely matters is whether they actually worked through the competing ideas.
Part of the link between students’ lexical alignment and rapport is explained by unresolved conflict. When students repeat Emma’s words, they may be working to correct her, not signaling affiliation.

These findings motivated the LLM version of Emma: it introduces a second, verified solution on purpose and then guides the student to compare the two.

Other findings from these sessions

Compared metrics of participation balance for predicting group learning, including ones a robot could compute automatically and act on in real time.EDM 2024
When speech recognition errors changed the course of the conversation, those changes were not related to learning gains or rapport.AIED 2023, best poster award

How the data is built

I design and run lab and classroom studies end to end: experimental design and power analysis, IRB protocols, participant sessions, dialogue and audio collection, annotation, and statistical analysis. For solution-path conflicts, I developed the coding scheme, trained a second annotator, and refined the scheme through disagreement review, reaching Cohen’s κ = .72 and Gwet’s AC1 = .83.

Collaborators

Diane Litman, Erin Walker, Timothy Nokes-Malach, and Adriana Kovashka at the University of Pittsburgh, and Nikki Lobczowski at McGill University.