Computers alone aren’t making kids smarter, but some educational software does work.
“Education technology is an area where innovation has outpaced rigorous research,” said Vincent Quan, who runs the North American education unit at J-PAL. “We wanted to find all the studies and distill the main lessons so that decision makers can decide which programs to scale up and invest in.”
To meet JPAL’s high standards, the study either had to be a randomized controlled trial, in which students were randomly selected to try a technology and studied alongside students who didn’t try it, or it had to be a “regression discontinuity design,” in which students with statistically similar test scores were studied, but those just below a test-score threshold tried the technology, for example, and those just above it didn’t. Both types of studies are expensive and typically take two years or more to conduct — time and money that ed-tech entrepreneurs usually don’t have. Technology can become obsolete by the time the results come out.
J-PAL threw out some other quasi-experimental studies because it wasn’t confident that the students trying the technology weren’t already higher achieving or more motivated to learn than the ones who didn’t try the technology. J-PAL also didn’t include observational studies, in which results were based on teacher surveys of how much they thought their students had learned. They tend to show much more positive results. That doesn’t mean the technologies don’t work, it just means they haven’t been rigorously tested.
1. Computers and internet access alone don’t boost learning
Handing out laptops, providing high-speed internet access or buying most other kinds of hardware doesn’t on its own boost academic outcomes. The research shows that student achievement doesn’t rise when kids are using computers more, and it sometimes decreases. The J-PAL researchers did find that students who have computers use them more, and become more adept at clicking and typing. It remains an open question whether tech-savvy students will be better workers in the future, even if they’re not better students now.
While hardware alone isn’t making kids smarter, students need computers and the internet to use educational software. And some of that does work.
2. Some math software shows promise
Exactly 29 software studies met J-PAL’s standards and 20 of those showed at least some measure of learning improvement (see table 2, pp. 30-37). The ones that tended to show positive results were mostly in one subject: math.
3. Cheap can be effective
Low-cost technological interventions, particularly text message reminders, were surprisingly effective with students and parents. “It’s not necessarily the most expensive or complicated technologies that make a difference,” J-PAL’s Quan told me.