ServiceNow pitches AI that writes its own homework to train enterprise agents

ServiceNow’s AI research arm has published details of a new system called AutoSynthData, designed to solve a problem that anyone building “AI agents” for business software keeps running into: these models are often fine in general, but hopeless at the specific, fiddly workflows of a particular company’s IT systems.

What ServiceNow is actually claiming

According to a blog post on Hugging Face, AutoSynthData watches where an AI agent fails - say, mishandling a particular combination of tools, or breaking a rule in a company’s workflow - and compares that against what a “stronger teacher” model would have done instead. It then automatically generates fresh practice tasks targeting exactly that weakness, checks whether those tasks are actually solvable and realistic, and feeds the results back into training. As the agent improves, the difficulty supposedly shifts to whatever it’s still struggling with - a kind of self-adjusting curriculum for a machine.

The company frames each task as three parts: a “system specification” (the rules and starting conditions), a user request the agent must fulfil, and a verifier that checks whether it actually succeeded. ServiceNow says the pipeline was demonstrated using something called EnterpriseOps Gym, a benchmark environment released by other researchers, and that it involves both automated repair of flawed generated tasks and a batch-level review step to keep quality up.

What’s missing from the pitch

This is a company blog post, not an independent study, and it reads like one. There are no numbers in the published material showing how much agents actually improved after training on AutoSynthData’s output, no comparison against simpler or cheaper alternatives, and no mention of cost, compute requirements, or how long the process takes to run. There’s also no indication of whether, or when, this becomes something customers can actually use - it currently looks like an internal research tool tied to ServiceNow’s own CoreAI work, illustrated via an ITSM (IT service management) use case, which happens to be ServiceNow’s core business.

Crucially, the “verifier” step - the thing that decides whether an AI agent’s answer counts as a success - is itself built by the same company training the agent. Self-graded homework is a reasonable research technique, but it’s not the same as independent proof that the resulting agents are more reliable in a live enterprise environment.

So who does this actually affect?

Right now: nobody, directly. There’s no product launch, no pricing, and no general release mentioned. This is a research methodology write-up aimed at AI practitioners and ServiceNow’s own customers who use its agent platforms, not a tool or feature ordinary users will notice appearing on their desktop.

If you work somewhere that already uses ServiceNow’s agentic AI products, this hints at where the company’s training pipeline is heading - more automated, more self-correcting, and increasingly tailored to each customer’s specific systems. If you don’t, this is a glimpse into how AI labs are trying to solve the “works in the demo, falls apart in production” problem that has dogged enterprise AI agents since they became a marketing buzzword.

The bottom line

AutoSynthData is a plausible-sounding engineering idea with a sensible rationale, but it’s being described entirely in the vendor’s own words, with no outside testing and no concrete deployment details. Worth watching if you’re in this space professionally; not yet something that changes anything for the rest of us.