SquareCat
Miles Corran · 
AI can now draft a set of flashcards from a page of notes, suggest practice questions for a lesson, or check a deck for mistakes. For anyone building an education product, that’s a real saving. It’s also a real risk. A wrong answer in a learning app isn’t a typo. It’s something a learner studies, remembers and repeats on the exam.
We use AI in our own learning products, and we hold their content to strict rules. Here’s what we’ve learned about using one without undermining the other.
A person approves every change
On Braineos, the premium AI features can generate a deck from your notes, suggest more cards for a deck, and fact-check a deck. In every case the deck’s owner approves each change. The AI drafts, and a person decides what goes in.
That one rule does most of the work. A model that’s wrong one time in a hundred is still useful if a person sees every suggestion before it’s saved. It’s not useful if its output goes straight in front of learners.
For a product where you, not your learners, write the content, the same idea applies with a different reviewer: your subject experts. AI can make them faster. It shouldn’t replace them.
Every item has a source
Review only works if the reviewer has something to check against. For CertBrain, our study app for the AWS Certified AI Practitioner exam, we set three content rules:
- Original content only, written from AWS’s public exam guide and documentation.
- Every item has a source. Each lesson, question and flashcard records its exam domain, its task statement, its difficulty and the AWS documentation page that backs it up.
- Retired services are never the answer.
Every item was reviewed against its cited source, with a sceptical second pass after the fixes. Most problems the review caught weren’t wrong answers. They were sources that didn’t actually say what the question claimed. That’s exactly the kind of mistake AI-drafted content tends to make: confident, plausible, and not quite supported. It’s much easier to catch when every item has to point at the page it came from.
If you’re planning AI-drafted content, decide what counts as a source before anything is generated, and make the source part of each item, not a note somewhere else.
Facts go out of date
Subjects change. Services close, rules are revised, guidance is updated. Models are trained on older material and won’t know which facts have since been retired.
CertBrain handles this with a specific rule. AWS has closed several AI services to new customers and retired some models. They can still come up in study material, so lessons mention them with a note that they’ve closed, and no question uses one as the correct answer. A reviewer checking a question looks at more than whether the answer is right. They also check whether it’s still right.
For your own subject, it’s worth listing up front what tends to go stale, such as product names, regulations, figures or recommended practice, and making those part of every review.
Keep AI where it’s checked
Not every site needs every AI feature. On our platform, each site has feature flags enforced on the server, and AI tools are one of the things a focused site can switch off entirely. Finance Brain, a free financial literacy course site, turns off some of the AI tools that Braineos offers.
That’s a useful option for any education product. If learners use your content as given, rather than making their own, AI may belong in your authoring workflow, with your reviewers, and not in front of learners at all.
Cap the cost before launch
The other surprise with AI features is the bill. Every request costs money, and the people making requests are users you’ve never met. We’ve written about the patterns we use in more detail. The short version:
- Keep the key on your server. Apps call our API, and our server calls the model. A key shipped in an app can be extracted.
- Quota by cost, not just count. Generating a whole deck costs far more than suggesting a few tags, so heavy and light features have separate monthly allowances.
- Use small models for small jobs. The expensive model is for the work learners notice.
- Gate the expensive parts. On Braineos, the AI features are part of the paid plan, and the quota still applies within it, so even the heaviest subscriber has a known ceiling.
Plan for no answer
AI services time out, hit limits and sometimes return something that isn’t in the format you asked for. In an education app, a broken or half-parsed answer is worse than none. We validate every response before using it, and treat “unparseable” as just another way of getting no answer. Then each feature has a plain, ordinary fallback: say generation failed and let the user try again, or let them type their own.
The short version
AI is a good drafting tool for learning content, and a poor authority. Put a person between the draft and the learner, make every item carry its source, watch for facts that have gone stale, and put a ceiling on what it can cost.
If you’re thinking about where AI fits in your own course or training product, our education apps page explains how we approach it, or get in touch to talk it through.
