New AI tutor achieves 0.71-1.30 SD effect size in Dartmouth course [pdf]

What did the Dartmouth study actually find?

Researchers linked Phosphor, an AI tutor built on Claude and used in a Dartmouth statistics course, to a 0.71 to 1.30 standard deviation improvement in exam performance among students who engaged fully with it. That is a large effect for education research. It was an observational study, not a randomised trial, so some of the gain may reflect that motivated students were more likely to use the tool in the first place.

The more interesting number, and the one most coverage has skipped, is adoption: 97% of students voluntarily used the optional tool, against an estimated 10 to 15% who reliably do the assigned textbook reading. That gap between "available" and "actually used" is the real story.

Why should a UK small business owner care about a US university study?

Because the underlying technology, an AI system that reads a free-form answer, checks it against a set of rules, and gives instant feedback, is not limited to grading essays. It is the same mechanism you would use to answer customer questions, qualify a lead, or train a new member of staff without your input every time.

Phosphor works because it does not just spit out multiple-choice results. It judges open-ended answers against a rubric the instructor defined. Swap "instructor" for "business owner" and "exam answer" for "customer enquiry" and you can see where this is going.

What's the practical takeaway for a small business right now?

Start by identifying the repetitive, judgement-based conversations in your business, the ones a good employee handles well but a script cannot, such as "do you do X for a wedding" or "what's included in the basic package". These are the exact shape of task rubric-based AI now handles reliably.

• List your 10 most common customer questions that need more than a yes/no answer.
• Write out how your best member of staff would answer each one, including the follow-up questions they'd ask.
• Use that as the basis for an AI-assisted chat or booking flow, rather than a static FAQ page nobody reads.
• Check monthly whether the AI's answers match what you'd actually say. Rubrics drift if you don't review them.

The 97% adoption figure matters here too. If a tool is genuinely useful and low-friction, people use it without being told to. If your current customer-facing setup needs constant chasing (a WhatsApp number nobody checks, an enquiry form that goes to a dead inbox), that is a signal the tool is wrong, not the customers.

Is this going to replace my staff?

No, and the data backs that up: 95% of UK SMEs already using AI reported no change to headcount over the past year, and 86% said job roles stayed the same. AI is being used to take on repetitive first-line work, not to replace the people doing skilled or judgement-heavy jobs.

What it does change is the gap between businesses that adopt and those that wait. AI adoption among UK SMEs rose from 23% in 2023 to 54% in 2026, but only 14% of micro-businesses use it, against 36% of large firms. Businesses already using AI report a net productivity expectation 25 percentage points higher than those still planning to start.

Braynex Services' point of view: what everyone else is missing

The lesson from Phosphor isn't "buy an AI tutor". It's that rubric-based, instant-feedback AI tools drive far higher genuine engagement than passive information (a textbook, a static FAQ, a PDF menu). We see the same pattern locally: a Manchester restaurant saw online orders rise 34% within four weeks simply by replacing a PDF menu with a live web menu people could actually interact with.

Here's the part a generic AI news piece won't tell you: whatever customer Q&A or booking tool you build on this shift, own it. Do not build your rubric, your customer conversations, and your training data inside a rented platform you cannot export from. We have watched businesses discover too late that their booking history, reviews, and customer data sat inside someone else's system with someone else's terms.

There's a second, bigger shift here too. As AI assistants like ChatGPT, Gemini and Google's AI Overviews increasingly answer "who does X near me", being recommended by them is becoming the new local SEO. A business with a real website, clear service pages and an accurate Google Business Profile is machine-readable in a way a Linktree or a Facebook-only presence is not.

What should you actually do this week?

Audit where your customer conversations currently happen and who owns that data: your website, your Google Business Profile, or a third-party platform. If it's the latter, start planning your exit before it becomes urgent.

If you want a second opinion on where your business stands, and which of these changes would genuinely move the needle for you, book a free audit at braynexservices.com. We'll tell you plainly what's working, what's costing you money, and what to fix first.