AI-Fueled Testing, From the Mouths of Babes

One of the hidden advantages of video games is that they offer automatic assessments: Winning one shows a user that she has mastered all she needs to know — no pesky final exam required. 

That has long been a dream of testmakers: to embed assessments in student work and, in a sense, make them indistinguishable.

For very young children, however, that’s a challenge. Much of what they know is revealed not through easy-to-interpret writing, but talk and play. To assess these kids effectively, one needs to be able to turn their quirky utterances into data.

That’s the basic idea behind Curriculum Associates’ 2023 acquisition of Dublin-based SoapBox Labs. The Irish startup has spent the past decade developing software that understands the unique speech of children and translates it reliably into text. As schools focus on the Science of Reading, that could be the key to making assessments a more seamless part of teachers’ workflow, especially for those who instruct children as young as pre-kindergarten.

“The future of assessment is invisible because it is integrated with instruction,” said Kristen Huff, Curriculum Associates’ head of assessment and research. “It is not disruptive. It’s authentic. And it helps the teacher personalize the learning path for each student.”

The future of assessment is invisible because it is integrated with instruction.

Kristen Huff, Curriculum Associates

Like virtually every other educational publisher, Massachusetts-based Curriculum Associates, founded in 1969, is trying to figure out how to offer teachers more data about student learning.

The publisher’s popular i-Ready reading and math programs are used by an estimated 13 million students nationwide. Curriculum Associates now says its reading program will soon feature speech recognition technology that can be operated not just by teachers but by the youngest students, with artificial intelligence listening and revealing exactly how well they understand the words they read and, some day, the math they do. 

The new tool will likely roll out next fall, the publisher says. 

For years, educators have puzzled over how to effectively assess the work of young children. They typically can’t just sit down, read texts and answer questions. They need hands-on instruction through different kinds of media — watching, listening and reading in equal measure — to understand what they’re learning. They act out stories, they sing, they chant rhymes, they talk and move around. 

Paper-and-pencil tests are mostly out of the question. 

To those who have studied it, voice offers the quickest means of assessing a child’s abilities, since in all but the most special cases there’s little space between a child’s thoughts and his or her utterances. “It’s the most natural way for most children to convey information,” said Amelia Kelly, SoapBox’s chief technology officer. 

But putting a keyboard, mouse, trackpad or even a touch screen in front of many students creates “confounding factors” that limit their ability to show what they know, she said.

By capturing students’ voices as they read independently on a tablet or laptop, then translating that into text and comparing it to what’s on screen, teachers can get valuable insights into kids’ understanding. Good voice assessments can help teachers see gaps in children’s learning so schools can challenge them with appropriate work. 

But processing kids’ voices accurately is another challenge altogether. 

‘They shout, they whisper, they sing’

SoapBox founder Patricia Scanlon, an engineer with a Ph.D in speech recognition technology, has said the company grew out of her personal experience watching her own child struggle to learn how to read. 

One day in 2013, she opened an email from the maker of a game her 3-year-old daughter was using for help. The app automatically sent parents updates, and this one told Scanlon her child had completed seven levels in the game, a major achievement. 

“Suitably impressed,” Scanlon asked her daughter to show her the game. She soon realized that the child hadn’t actually mastered the material — she’d simply guessed at the correct answers and gathered rewards without mastering the skills. “She had learned to hack the game,” Scanlon said, impressed with her daughter’s ingenuity — but steamed at a wasted opportunity.

(Kids) shout, they whisper, they sing, they elongate, they over-pronounce the words.

Patricia Scanlon, SoapBox Labs

What was missing, she realized, was a way for the game to hold her daughter accountable, to “invisibly and continuously” quiz and assess her progress, despite the fact that, at age 3, she and most kids can’t hold a pencil, control a mouse or type on a keyboard.

With her background, Scanlon knew that even in 2013, speech recognition technology worked well for adults but not for younger children, who have higher pitched voices and rarely follow standard language rules: “They shout, they whisper, they sing, they elongate, they over-pronounce the words,” she said.

Of course, children come to school with regional accents and years of learning distinctive dialects at home. And millions of kids are learning English as they enter school. So she began building a proprietary “voice engine” that would accurately record what young children say in real-world, noisy environments and on ordinary consumer devices like Chromebooks and iPads.

At the time, the biggest AI voice recognition systems such as Apple’s Siri (Amazon’s Alexa was still about a year away) were being trained almost exclusively on adult voices, in “grown-up” situations: consumers purchasing products, drivers seeking directions or hikers asking about the weather. 

Siri and other systems worked well for these nominal tasks, but they weren’t built for school, where children are struggling to learn. Kelly, SoapBox’s CTO, compared it to training an AI-guided self-driving car on a Formula 1 racetrack instead of a crowded, congested street. When you finally got the car out onto the streets, it wouldn’t work.

So Scanlon and her colleagues spent the next decade training SoapBox’s AI to learn from children in both Europe and the U.S. That meant teaching the AI that a word said by an English language learner in Dublin is the same one spoken by one in Philadelphia or a kid from the American South.

“If it doesn’t work for every student equally, then it doesn’t work,” said Kelly.

(Speech) is the most natural way for most children to convey information.

Amelia Kelly, SoapBox Labs

She sees that functionality as an ethical concern. Voice-activated AI “can be the great equalizer here,” she said. “I think it can help solve the literacy crisis — but only if people use it. And people are only going to use it if they trust it. And they’re only going to trust it if it works.”

The terms of the November sale weren’t disclosed, but it will almost certainly create a huge competitive advantage for Curriculum Associates, which gets exclusive access to a technology that has been widely used by other publishers.

Before the acquisition, SoapBox had licensed its technology to dozens of education providers such as McGraw Hill, Scholastic and Amplify, essentially enabling them to outsource voice recognition for their own products. With the 2023 deal, those partnerships stopped, Curriculum Associates said.

According to recent filings, before the acquisition, Soapbox had raised $10.4 million in funding since 2017. Its most recent investor last year was the Bill & Melinda Gates Foundation, which provided an undisclosed sum to underwrite development of a Spanish language voice engine for U.S. students.

By next fall, Curriculum Associates envisions that the technology will be so simple to use that even the youngest students could work independently, putting themselves through the paces of self-guided games and activities that evaluate their reading skills on an ongoing basis. While it’s still piloting the technology in schools, one teacher who has seen a preview said she’s eager to see it in action. 

LaTanya Renea Arias of Kingsland Elementary School in Kingsland, Ga., said having better data about students is key not just to learning but equity — especially when 55% of students are people of color but 80% of teachers are white.

Though she has taught for a decade, she said, “I don’t have an ear to pick up every single dialect, to have great understanding of how a word that I pronounce sounds differently” when a particular student says it. “But I still need to credit them with their learning and their knowledge.”

Disclosure: The Bill & Melinda Gates Foundation provides financial support to The 74.

Get stories like these delivered straight to your inbox. Sign up for The 74 Newsletter