Creuto is now an OpenAI Select Partner Read More

AI & Machine Learning

AI in corporate training: buy an LMS or build?

AI in corporate training lifts novices most: 34% more output in the NBER study. When an LMS add-on is enough, and when to build on your own data.

AI in corporate training: buy an LMS or build?

AI in corporate training pays off most for new and less-skilled staff. In the largest field study so far, an AI assistant raised issues resolved per hour by 14% on average and 34% for the least-skilled agents, by passing on what top performers do. That gain depends on your own performance data, and that is where buying an LMS stops and building starts.

This post is for the head of L&D or training-product lead choosing between an AI add-on for an existing learning management system (LMS), an AI roleplay tool, or a platform built around your own product content and certification rules. By the end you will know what the research measured, what the vendor data shows about how people actually practice, and which of the three routes fits your network.

Where AI in corporate training helps most: new and less-skilled staff

The study behind the numbers is "Generative AI at Work" by Erik Brynjolfsson, Danielle Li and Lindsey Raymond. The NBER working paper (w31161, April 2023, revised November 2023) follows the staggered rollout of a chat assistant to 5,179 customer support agents at a Fortune 500 software firm, most of them working from the Philippines. The tool, built on a GPT model, suggested replies in real time; agents stayed in charge and could ignore it.

Its findings, in the working paper's own terms:

  • 14% more issues resolved per hour on average across all agents.
  • 34% more for agents in the lowest skill quintile, and 46% for agents with less than a month of tenure.
  • Two months with AI matched six months without it. "Treated agents with two months of tenure perform just as well as untreated agents with more than six months of tenure."
  • No productivity gain for the most skilled, and small but significant declines in their resolution rates and customer satisfaction.
  • About 40% lower attrition among agents with less than six months of experience, against a 25% baseline.

The tenure result is the one an L&D team should hold on to. It is a time-to-competence number, which is what onboarding programs are funded to reduce.

The published version moved the numbers

The 14% and 34% figures that circulate are from the 2023 working paper. The peer-reviewed version in the Quarterly Journal of Economics (volume 140, issue 2, 2025) reports 5,172 agents, a 15% average gain, "a 30% increase in the number of issues resolved per hour" for less skilled and less experienced workers, and 36% for the lowest skill quintile. The direction is the same and the two-months-equals-six result survives. If you cite the study in a business case, cite the journal figures; a finance team that checks will find them.

Why the gain depends on your own top performers' data

The assistant did not help novices because it was a general-purpose chatbot. According to the working paper, it was trained on customer conversations labeled with whether each was resolved, how long it took, and whether the agent was "considered a 'top' performer" by the firm. The journal version adds that the AI firm "up-weights the value of training chats if the chat was conducted by a top performer." The authors' reading is that the tool spread the best practices of able workers down the experience curve.

That is a data requirement, not a feature. To repeat the effect in your sales or service network, a training system needs three things a generic LMS usually does not hold: outcome data on real work (deals closed, issues resolved, audit scores), a record of who your top performers are, and your current product content. A course catalog with completion rates has none of them.

Two further results matter for a training team. During software outages, when the assistant gave no suggestions at all, agents still performed above their pre-AI baseline, and the gains were larger for those who had followed the AI more closely. That is evidence of learning, not just assistance. The authors also flag an incentive problem: "Top workers are generally not paid for their contributions to the training data." If your best sellers' calls become the training set, decide up front how they are credited, before you ask them to record anything.

It also changes what you measure. The study counted resolutions per hour, an operational outcome. Most training dashboards count completions, and a platform that cannot see the operational number cannot tell you whether the AI helped.

What vendor data shows about AI roleplay practice

The most useful public dataset on AI practice comes from a vendor. VirtualSpeech analyzed 47,043 AI roleplay sessions run from 1 January to 13 October 2026 by 5,321 accounts at 465 organizations and universities, across 1,380 scenarios. (The page carries a July 2026 date, so the data has been extended since it was first published.)

FindingFigure
Sessions per learner, median2
Cases where someone tried a scenario only once65%
Average score, first attempt to best by fifth attempt5.5 to 7.0 out of 10
People who improved on their second try60%
Company and public-body sessions on custom scenarios47%

Two things follow. Practice works when people repeat it, and most people do not repeat it unprompted; VirtualSpeech's own summary is "AI makes practice possible; programme design makes it happen." And nearly half of organizational practice ran on scenarios customers built themselves, which says buyers want their own products and objections in the roleplay, not a generic library. The dataset has limits the authors state: scores come from AI rather than human assessors, and the people who practiced more chose to.

What the leading LMS guide leaves out

Docebo's guide to AI in corporate training, last updated 9 January 2026, lists the uses buyers will recognize: adaptive personalization, natural language interaction, predictive analytics for knowledge gaps, agentic workflow automation and AI content generation. Its outcome figures, such as onboarding time cut in half and course completion up 40%, come from Docebo's own customer stories. The guide never discusses building; its step is "Select an AI-powered platform." It does put a pilot at 2-4 months and an enterprise deployment at 6-12 months, without a source, which is worth noting: buying is not instant either.

Should you build or buy an AI training platform?

Most teams have three routes, and each is right for a different network.

RouteRight whenWhat you give up
AI add-on to your existing LMSTraining is mostly content and compliance, and your LMS vendor already ships the add-onAccess to operational outcome data outside the LMS
Buy an AI roleplay toolThe skill is conversational (sales calls, objections, service) and you can author custom scenariosOne record across practice, exams and certification; you integrate it yourself
Build a training platform on your own dataCertification rules, product launches and identity checks are specific to your network, and outcome data lives in your own systemsSpeed to a first version, and you own maintenance

The strongest case for buying is real. Docebo already lists AI content generation and authoring tools, VirtualSpeech sells a scenario library that carried 64% of all sessions in its dataset, and a build takes engineering time you could spend elsewhere. If your training is mostly compliance courses for office staff, an off-the-shelf LMS with an AI add-on is the right answer, and we would tell you so. The same logic is in our guide to which SaaS layers to own in build vs buy decisions: own the layer where your rules are the product, rent the rest.

Building pays when the rules are the hard part. Distributed sales and dealer networks in automotive, insurance, pharma field forces and retail tend to share three traits: launches on fixed dates, certification that gates who may sell what, and staff who sit in partner businesses outside your HR system. Those are workflow and identity problems first, and an AI layer on top only works if they are solved underneath.

This is the wrong choice when…

A custom platform is the wrong choice if you cannot name the operational metric it should move, if nobody will own scenario content after launch, or if your certification rules fit an LMS's standard course-and-quiz model. In those cases, buy, and spend the money on program design: assigned practice, deadlines and required retries, the things the VirtualSpeech data says make practice happen.

What we built for a dealership network

For Škoda Auto Volkswagen India we built a sales training, examination and certification platform for the dealership network, used to train and certify sales staff for each new model launch. It is two connected apps on one shared, verified employee record. Sales professionals verify their identity with their LMS ID or mobile number, apply for a training batch, upload documents, take in-app training and exams, and receive a certificate automatically when they complete the exam. Event managers scan M-Pass QR codes at the door against the same record.

We are not claiming AI features for this platform; we mention it because of what it shows about the layer underneath. The hard parts were identity, batch logistics, identical exams for every batch, and certificates issued on completion. Any AI coaching or roleplay layer added to a network like this would need that verified record to know who is practicing, for which launch, and what they are certified to sell. If certificates need to travel with the person, our post on what Open Badges 3.0 leaves to you covers the credential side.

How to measure ROI of AI in corporate training

Measure it the way the study did: on the work, not the course. Pick one operational outcome per role (resolutions per hour, conversion on test drives, audit pass rate), and compare new staff who get AI practice or assistance with those who do not, at fixed tenure. The study's own headline result is a time-to-competence comparison, and attrition among new hires is the second number worth tracking. If AI-generated answers have to cite your product documents, our breakdown of RAG implementation cost prices the retrieval layer.

Scope the platform around your certification rules

Before you compare vendors or quotes, write down the certification rules for one role, where that role's outcome data lives, and how staff in partner businesses are verified. If all three fit inside an LMS, buy one. If they do not, our custom software development team can scope a training and certification platform around those rules, with a clear place for AI practice once the record underneath exists. A focused MVP typically takes 6-10 weeks, and larger enterprise platforms 3-6 months depending on integrations and scope.

Frequently asked questions

AI in corporate training is used for adaptive personalization, AI-generated course content, predictive analytics on knowledge gaps, and AI roleplay practice for conversations such as sales calls. Field evidence suggests the largest gains come from AI that passes top performers' practices to new staff, which depends on a company's own performance data.

Yes. In the Generative AI at Work study of over 5,000 support agents, AI assistance raised issues resolved per hour by 34% for the least-skilled agents in the working paper, and 30% for less skilled and newer workers in the published version, with little or no gain for the most experienced.

Buy an LMS with an AI add-on when training is mostly content and compliance. Buy an AI roleplay tool when the skill is conversational. Build a training platform when certification rules, launch schedules, identity checks and outcome data are specific to your network and live outside any LMS.

Measure the ROI of AI in corporate training on an operational outcome per role, such as resolutions per hour or conversion rate, comparing staff with and without AI at the same tenure. Time to competence and attrition among new hires are the two numbers the main field study found AI moved most.

VirtualSpeech's analysis of 47,043 AI roleplay sessions in 2026 found the median learner ran only two sessions, and in 65% of cases people tried a scenario once. Scores rose from 5.5 to an average best of 7.0 out of 10 by the fifth attempt, so program design that requires retries matters.

Written by

Akash Mohapatra

Akash Mohapatra

Co Founder & Director

19 Nov 2026

·

9 min read

Share

LET'S CONNECT

Connect with Creuto!

Ready to take the first step towards unlocking opportunities, realizing goals, and embracing innovation? We're here and eager to connect.