Can just-in-time advice keep more college students on track?

Professor Tim McKay has measured millions of galaxies in his 20 years as a big data cosmologist, seeking to understand how they’ve developed.

In 2009, he realized the University of Michigan had its own vast sets of data related to another aspect of his job: teaching introductory physics to hundreds of students each semester. High school grades, test scores and socio-economic backgrounds could help him understand his students, or at least explain their disparate academic performances.

McKay admits his first insights shouldn’t have been a surprise.

“It’s simple to predict student performance in a class,” he said. “If you want to know what grade a student is going to get in a class, you should ask what grades are they getting in other classes. Students with excellent grades typically get excellent grades.”

But it also mattered where students came from and what kind of high school science experience they had. Having that simple information laid out so clearly made him rethink teaching.

“I approached class imagining every student was a blank slate, starting in the same place,” McKay said. It was clear to him now, “on its face, it’s a ridiculous assumption.”

He wanted to be responsive to each of his students. But with as many as 700 introductory physics students each semester, that seemed impossible.

Then McKay learned about the University of Michigan’s Center for Health Communications Research. The center had already built the kind of tool he was looking for: the Michigan Tailoring System can send personalized messages to patients about health goals like quitting smoking or controlling or preventing diabetes.

McKay used that framework to build ECoach, a system to help students manage coursework and deliver the advice they needed at the right time, from the right source.

“If Professor McKay says you need to study 10 hours a week for his class, the student thinks ‘McKay doesn’t know about me. He doesn’t know what I need to do,’” McKay said. “But if your roommate says, ‘no, you really have to study 10 hours a week.’ Students will take that advice seriously because it comes from an identity salient peer — a person who is like me in a way that is important to my identity.”

McKay first used the ECoach in a 2011 class. In the last four years the system has grown more sophisticated and spread to other courses. About 5,000 students use it each semester.

As a sophomore, Hanna Zlotnick used ECoach in an introductory statistics course that stood between her and the upper-division courses she needed for her economics major.

“We took a survey at the beginning of the year that asked you what you wanted, it could cater to you,” she said. “It would tell you about your performance and it would tell you what to work on. ECoach told you which material was best to go over and if you were struggling with something, which problems sets were best for that given topic.”

For Zlotnick, ECoach was an easy place to access all of the course materials — recorded lectures, problem sets, practice exams — which she otherwise would have had to hunt for on different university websites. It also gave students a customizable weekly checklist, which could show mandatory assignments and exams, or include extra practice work.

“Had I not had something that brought everything together and showed me all the resources available, I would perhaps not have used all of them and done as well,” Zlotnick said.

McKay and others are finding online course management tools like the ECoach, which can track students’ work and grades for a class, can also change their behavior and improve performance.

At the University of Maryland Baltimore County, John Fritz is studying how performance changes when students are given information showing how their time working on class material and their grades compared to their classmates.

Students who access those comparisons can be “nudged” to put in more effort, according to Fritz, the university’s assistant vice president of instructional technology. His early results show a 4 percent bump in those students’ final grades.

Knowing just when to deliver that nudge is also important. Getting the timing right — to deliver a reminder to pay a bill or go to an organized study session — depends on pulling all the data a student creates on a campus, whether in the classroom or the financial aid office, into one centralized system.

“We’re seeing really consistent 3 to 7 percentage point changes in persistence just by getting the data at the right point to the right student,” said Mark Milliron, co-founder and chief learning officer of Civitas Learning, a company that works with schools to build data analysis systems.

Increasing persistence just means keeping more students coming back to campus semester to semester. Milliron thinks that’s likely to pay off most for more vulnerable students.

“We’ve got more data at our fingertips now to ask and answer those questions like, ‘what is going to help these students learn well?’ and ‘what’s going to actually help them navigate this system?” he said. “Second-, third-, fourth-generation [college students] that come from high-income families are scaffolded by the stories of their families and friends. First-generation students, who are the biggest group that is coming into higher ed right now, have no scaffold.”

That’s part of why the University of Michigan is expanding its ECoach tool. This summer, graduate students are working with McKay on a coaching platform for freshmen, which will deliver advice from older students with similar backgrounds on navigating the campus and developing successful habits.

Hanna Zlotnick, who graduated in May, said a tool like that could have helped her acclimate to the Ann Arbor campus as a freshman from San Diego.

“Michigan has absolutely incredible resources, whether it’s academics or career planning, and it extends to lifestyle and extracurriculars,” she said. “But there isn’t a comprehensive list or accessible way to find out these things exist.

“If there was a way to find out about them before I needed them, I would probably have used them more and that would have made my time here even better,” she said. “It would really be an incredible resource.”

Putting every student on the same page about how much work it will take to pass a class or what office they can go to for help applying for scholarships is part of what Tim McKay thinks will make his pre-data vision of his students — that everyone has the same chance at success — a reality.

“The relationship between where students came from and outcomes, we would like that relationship to be very weak,” he said. “We’re not there yet. The goal is to get closer to the blank slate idea that any student who comes here can have any outcome determined by what they do here, not by what happened before they came here.”

PBS NewsHour coverage of higher education is supported by the Lumina Foundation and American Graduate: Let’s Make it Happen, a public media initiative made possible by the Corporation for Public Broadcasting.