Towards Sustainable Learning in Online Education: A Reinforcement Learning Approach

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Next we'll be talking about the paper "Towards Sustainable Learning in Online Education: A Reinforcement Learning Approach".

Jane: The paper was written by Chaofan Zhai, Yicheng Song, Ravi Bapna and Junyao Ye from University of Minnesota and MaiMemo Inc..

Tom: Stay tuned as we take you through the paper and discuss its implications.

Jane: We also have Lu with us today — senior AI researcher at Tsinghua.

Tom: We also have Meng with us today — lead engineer at a mysterious AI startup.

Jane: We also have Lalam with us today — the in-house Large Language Model.

Tom: Alright, let's get started.

Title: Tom: Welcome back, everyone. I’m Tom, and with me is the brilliant Jane. We’ve got a fascinating paper on our desk today, and the title alone tells you it’s ambitious: “Towards Sustainable Learning in Online Education: A Reinforcement Learning Approach.”

Jane: Tom, I love this title because it packs two huge ideas into one. You’ve got “sustainable learning,” which sounds like a buzzword, but they mean something specific: keeping students engaged long-term and making sure they actually retain what they learn. And then you’ve got “reinforcement learning,” which is the AI technique that decides what to recommend next.

Tom: Right, and that’s the core problem, isn’t it? Online courses have massive dropout rates. The paper cites that MOOC completion rates dropped from about six percent in two thousand fourteen–fifteen to just over three percent by two thousand seventeen–eighteen. So the system is scaling, but it’s not keeping people.

Jane: Exactly. And the authors, Chaofan Zhai, Yicheng Song, Ravi Bapna from the University of Minnesota, and Junyao Ye from MaiMemo, they’re trying to fix that. They built a system called AI-Tutor that doesn’t just push you forward through new material. It also decides when to bring back old material for review, so you don’t forget it.

Tom: And that’s the “sustainable” part. It’s not just about finishing the course. It’s about finishing it with knowledge that sticks. They tested this on twenty-three million learning records from thirty-three thousand seven hundred learners on a language platform, and the results are pretty striking.

Jane: We’ll get into those numbers soon, but let me just say this: the idea that a recommendation system should care about your motivation, not just your test scores, feels like a shift. Most systems I’ve seen just try to maximize the chance you get the next question right.

Tom: That’s the old way. This paper says, “Wait, if you burn people out, they leave, and then all those future rewards are gone.” So they actually build the probability of you quitting into the AI’s math. That’s a really human way to think about it.

Jane: It is. And it’s not just about keeping you on the couch longer. It’s about making sure the time you spend is actually good for you. We’ll see how they balance that in the methodology, but for now, I’m hooked.

Tom: Same here. Let’s dig into how they actually built this thing.

Summary: Tom: So, Jane, we’ve got the title unpacked. Now let’s talk about what the paper actually does. They frame this as a reinforcement learning problem, which means the AI is making a sequence of decisions, and each decision gets a reward.

Jane: And the reward is the clever part. They don’t just give points for getting a word right. They split the reward into two pieces. One piece is about learning something new, which they call knowledge acquisition. The other piece is about strengthening your memory of something you already learned, which they call knowledge retention.

Tom: Why does that matter? Because if you only reward new learning, the AI will just keep throwing new words at you. You’ll feel like you’re making progress, but you’ll forget the old stuff. And the paper shows that actually happens. They have this concept of a forgetting curve, which is a real psychological idea. You learn something, and if you don’t review it, your ability to recall it drops off over time.

Jane: Right, and that’s where the second big innovation comes in. They don’t assume you’ll stick around. They explicitly model the probability that you’ll drop out after each interaction. That’s the engagement piece. So the AI is constantly weighing: “If I give this hard word, they might learn a lot, but they might also quit. If I give an easy review, they might stay, but they won’t learn as much.”

Tom: That’s a real trade-off, and they put it directly into the math. They modified the standard Bellman equation, which is the core equation in reinforcement learning, to use this dropout probability instead of a fixed discount factor. That’s a big deal technically.

Jane: And to make this work, they built a learner simulator. They can’t just test the AI on real students and watch them quit. That would be unethical and slow. So they trained a model on historical data to predict three things: will the student recall this word, will they stay engaged, and how strong is their memory after this interaction.

Tom: And the simulator is really good. On the recall prediction task, it beats the baselines by a huge margin. We’re talking about a thirty-three percent improvement in AUC over the next best model. That’s not a small bump.

Jane: No, it’s not. And it makes sense because they’re using a Transformer model, which is great at capturing long sequences of behavior, and they’re feeding it knowledge graph embeddings, so the AI understands that some words are related to other words. It’s not just treating each word as an isolated fact.

Tom: So you’ve got a smart simulator, a reward function that cares about both new learning and memory, and an engagement model that tries to keep you from quitting. That’s the whole package. And in the next segment, we’ll see how they actually tested it and what the numbers look like.

Jane: Can’t wait. Because the real question is, does it actually work on people, or just in the simulation?

Improvements: Tom: Alright, Jane, let’s talk results. They set up a twenty-eight-day course with one thousand vocabulary words and one thousand simulated learners. They compared AI-Tutor against several baselines, including a greedy approach, a rule-based spaced repetition system called FSRS, and a deep reinforcement learning model called DRL-SRS.

Jane: And the headline number is the course completion rate. The greedy approach, which just always picks the hardest, newest word, basically burned everyone out. By day twenty almost all users had quit. That’s the failure mode we were talking about.

Tom: Yeah, that greedy model is a cautionary tale. It maximizes immediate learning but ignores motivation. And the paper shows that strategy backfires completely. Now, the rule-based FSRS does better, but the full AI-Tutor crushes it. Completion rate jumps from about eleven percent with DRL-SRS to over thirty-one percent with AI-Tutor. That’s a one hundred seventy-seven percent improvement.

Jane: And it’s not just about finishing. The final exam scores are even more telling. AI-Tutor gets a fifty-five point nine percent average on the final test, compared to twenty-two point four percent for DRL-SRS. So people are staying longer, and they’re actually learning more. That’s the sustainable part in action.

Tom: But here’s what I find really interesting. They ran ablation studies, which means they removed parts of their model to see what mattered. And the biggest drop in performance came when they removed the engagement modeling. That tells you that the dropout probability isn’t just a nice add-on. It’s the engine that makes the whole thing work.

Jane: And the second biggest drop came from removing the knowledge graph. Without it, the AI is just picking words randomly from a huge pool. It doesn’t understand that you need to learn “cat” before you learn “catastrophe.” That structure matters.

Tom: Right. And then they also looked at the actual learning paths. They visualized them on the knowledge graph, and you can see the difference. The greedy model scatters words all over the place. AI-Tutor follows the edges of the graph, building a coherent path. It’s like the difference between a tour guide and someone just pointing at a map.

Jane: And they also showed that AI-Tutor adapts to different learners. Low performers get easier words and more reviews. High performers get harder words and fewer reviews. It’s not a one-size-fits-all policy. It’s genuinely personalized.

Tom: That’s the kind of system that could actually change how online education works. Instead of just dumping content and hoping for the best, you have an AI that’s actively managing the learner’s experience, like a good tutor would.

Jane: Exactly. And that’s what we’ll wrap up with in the conclusion. But before that, I want to hear what our other guests think about the practical side of this.

Conclusion: Tom: We’re back for the final segment on “Towards Sustainable Learning in Online Education: A Reinforcement Learning Approach.” Jane, we’ve covered the method and the results. Let’s talk about what this means for the real world.

Jane: For me, the biggest implication is that we now have a proven framework for building tutors that care about the whole learner, not just their test scores. The paper shows that engagement and retention aren’t optional extras. They’re core to the learning outcome.

Lu: I’d push that even further, Jane. This framework isn’t just for vocabulary apps. The core idea of balancing new knowledge with review, and modeling dropout risk, applies to any sequential learning task. Think about medical students learning anatomy, or engineers learning a new programming language. The structure is the same.

Meng: And from an engineering standpoint, I’m impressed that they made this work with a simulator. Training a reinforcement learning agent directly on users is risky and slow. By building a reliable learner simulator first, they made the whole process safe and efficient. That’s the kind of design I’d want to copy.

Tom: And the numbers back it up. A one hundred seventy-seven percent increase in completion rate and a one hundred forty-nine percent increase in final exam performance over the best baseline. Those aren’t incremental gains. Those are transformative.

Jane: And I love that they showed the learning paths. It’s one thing to say “our model works.” It’s another to show that it’s recommending coherent, sensible sequences that adapt to the learner’s level. That visual evidence is really convincing.

Lalam: If I may add a broader perspective: this paper points toward a future where online education isn’t just a scalable version of a lecture, but a personalized companion. The AI-Tutor doesn’t replace the teacher. It replaces the feeling of being lost and alone in a sea of content. That could make education more humane, not just more efficient.

Tom: That’s a beautiful way to put it, Lalam. And it’s a good note to end on. This paper gives us a roadmap for building learning systems that respect the learner’s time, motivation, and memory.

Jane: We’ll be watching to see if this gets deployed on a large scale, and whether it generalizes beyond language learning. For now, let’s say goodbye to this paper and get ready for the next one.

Tom: Thanks for listening, everyone. We’ll see you on the next episode.