A Gold-Medal Mathematician That Can’t Read a Clock

Why AI’s jagged intelligence is the enterprise’s biggest risk and its biggest opening.

By Shobhit Gupta

The same AI systems that win gold at the International Mathematical Olympiad read an analog clock correctly only half the time (50.1%, according to Stanford’s 2026 AI Index). Every enterprise deploying AI is betting on which side of that gap its own work falls on.

By Stanford’s latest count, 88% of organisations used AI in at least one business function in 2025, up from 78% a year earlier. Generative AI was in use in 70% of them. Yet the same Index found agents, meaning systems that carry out multi-step tasks on their own, still in single digits across nearly every business function. Most companies have put AI in front of their staff. Few have let it own part of a process.

The models are not the obvious bottleneck. The unevenness is, because nobody can tell in advance which side of that line a given task falls on. For a business, that is the real difficulty.

The Jagged Edge: Where AI Helps, and Where It Quietly Hurts

The evidence on productivity is better than the sceptics suggest and more mixed than the marketing implies. In a study of 5,179 customer-support agents, AI assistance raised productivity by 14% on average and by 34% for the least experienced. Among 444 professionals given writing tasks, ChatGPT cut the time taken by 0.8 standard deviations and raised quality by 0.4. Stanford summarises this body of work as gains of roughly 14–15% in customer support and 26% in software development, and cautions that the studies are directional rather than definitive.

Now the exceptions. In a Harvard–BCG experiment with 758 consultants, those using GPT-4 on tasks inside its capability completed 12.2% more tasks and worked 25.1% faster. On a task designed to fall outside it, they were 19 percentage points less likely to reach the right answer than colleagues working without AI. In a 2025 randomised trial by METR, experienced open-source developers took 19% longer with AI tools while believing they had been faster. Both studies used early-generation tools, but the pattern matters more than the vintage.

None of this shows that AI is unreliable. It shows that reliability belongs to the pairing of a system with a task, and has to be measured for each pairing. Documented AI incidents rose from 233 to 362 in 2025. Some of that reflects wider deployment and better reporting, so it is not a failure rate. But it points one way: as use spreads, so do failures, and companies need ways to catch them.

The Price Falls. The Bill Doesn’t

The price of a fixed level of AI performance has collapsed. Stanford’s 2025 Index put the fall for GPT-3.5-level output at $20 to $0.07 per million tokens in about 18 months, a 280-fold drop. Enterprise spending has moved the other way. The FinOps Foundation’s 2026 survey found that 73% of organisations exceeded their AI cost projections last year, and that the average enterprise AI budget grew from $1.2 million in 2024 to $7.0 million in 2026. Gartner estimates that agentic workflows use five to thirty times more tokens per task than a simple chatbot query.

Unit prices fall while the units consumed multiply. A back-of-envelope case shows why architecture matters. A service handling 10 million interactions a day at about 3,000 tokens each burns 30 billion tokens daily. At $1 per million tokens that is roughly $11 million a year, and at $10 per million, it is roughly $110 million. The gap between those bills is a design decision: which requests need a frontier model, which can go to a small specialised one, and which need a person. Cost per token is the wrong measure. Cost per successfully completed task is the useful one.

The Model Is Not the Moat. Your Data Is

Retrieval-augmented generation, introduced by Patrick Lewis and colleagues in 2020, lets a model draw on an external source rather than rely only on what it learned in training, and it improved factual specificity on knowledge-intensive tasks. In a company, that source is sales records, contracts, policies and manuals. Ask a model why sales fell in the southern region last quarter and it can offer a plausible story or an answer traced to actual transactions. Only the second is usable.

That depends on unglamorous questions. Which system holds the authoritative figure? How current is it? Who may see it? Gartner has forecast that more than 40% of agentic AI projects will be cancelled by the end of 2027, citing rising costs, unclear business value and inadequate risk controls.

Boring Is the Point

Engineering has solved a version of this problem before. Over 18 years of building and running production platforms, the pattern I have seen is consistent. Reliability comes from unglamorous practices: monitoring, automated testing, disciplined releases and clear ownership of incidents. They work because they are in place before the problems appear. AI systems need their equivalents: task-specific test sets, tracking of hallucination rates and data freshness, drift detection, cost per completed task, and clear triggers for handing over to a human.

Early results from applying this thinking are encouraging but should not be over-read. In the workflows where we have applied AI-assisted development, defects fell by roughly 30%. That is a single-organisation figure, gathered where the data and the process were ready for it.

Three Questions Before the Next AI Budget

What does a completed task cost, and what happens to that number when volume grows tenfold? Where can the system fail, who is told, and how quickly? What data can it see, how current is it, and who is accountable when it is wrong?

Companies that can answer those questions are the ones that will move from pilots to agents. The rest will keep reporting adoption figures that look healthy and change little.