Enterprise AI’s next leap will be an architecture layer

After months of arguing that enterprise AI is constrained by the mismatch between how large language models work and how companies actually run, the prediction is clear: before the end of the year, a new layer will emerge above today’s models—making intelligen

For months, I’ve been writing about what I believe is the central problem in enterprise AI—one that has little to do with the models themselves.

It’s not prompts. It’s not context windows. It’s not even agents.

The problem, as I’ve been arguing across a series of articles, sits in the architecture.

Large language models, in my view, were never designed to run companies. Companies don’t operate by predicting the next token. Companies operate through memory, context, state, constraints, permissions, incentives, workflows, and feedback loops. That mismatch helps explain why enterprise AI adoption looks widespread while business transformation remains elusive. It also helps explain why many organizations report productivity gains but struggle to produce meaningful operational impact.

Deployments frequently lean on consultants, systems integrators, and—more recently—forward deployed engineers embedded inside customer organizations. The experience can feel revolutionary and incomplete at the same time, even when the technology works.

Taken together, these observations point to a shift that could arrive soon: enterprise AI is approaching a discontinuity.

The prediction is straightforward. Before the end of this year, someone will launch a product that fundamentally changes how companies think about AI. Not a better chatbot. Not a more capable copilot. Not an agent with a longer context window. A new layer.

Once that layer appears, much of today’s enterprise AI landscape will start to look transitional.

The pattern is familiar. The internet worked before the web. In 1991, TCP/IP already moved packets, email connected institutions, and FTP transferred files. Universities and technically sophisticated organizations could use the network effectively.

But it wasn’t yet the web. The breakthrough wasn’t more networking. It was the emergence of a layer—URLs, HTTP, HTML, browsers, and servers—that made the infrastructure understandable, usable, and buildable by ordinary organizations.

The same kind of “layer” moment shows up in enterprise software. Relational databases became transformative after Edgar F. Codd formalized the relational model. ERP emerged when enterprise operations gained a common representation. CRM emerged when customer relationships became a manageable system rather than a collection of disconnected interactions.

In each case, the underlying technology mattered enormously—but the category-defining breakthrough arrived when someone introduced an abstraction that organized it.

That’s the direction I believe enterprise AI is moving in now.

The models will still matter. The frontier models being developed by OpenAI, Anthropic, Google, Meta, xAI, and others are improving at an extraordinary pace, and each new generation expands the amount of intelligence available to organizations.

But when technology improves faster than organizations can absorb it, attention shifts away from the technology itself and toward the architecture that organizes it.

Companies don’t buy ERP because they’re fascinated by databases. They don’t choose Salesforce because they admire SQL. They don’t pick cloud platforms because they enjoy thinking about virtualization. The underlying technology remains essential, but gradually becomes infrastructure—something the business leans on while it builds value somewhere else.

That’s why the business question is changing. In my view, it’s becoming less about which model is smartest, and more about how intelligence is organized, deployed, governed, measured, and continuously improved inside the enterprise.

The next breakthrough, I suspect, will be simpler than most people expect. Major abstractions often feel obvious only after they’re explained—“everything is a file. ” the web as a collection of resources identified by URLs. or the idea that business operations can be represented as processes and transactions. Inventing those ideas wasn’t simple, but explaining them often was. And once explained, they can feel inevitable.

The kind of innovation described here wouldn’t land as a pile of new features. It would arrive as an abstraction that suddenly makes a fragmented landscape coherent.

That’s the emotional center of the argument: the reaction won’t be “how extraordinary.” It will be “of course. How else could it have worked?”

For the last two years. the enterprise AI conversation has been dominated by prompts. copilots. agents. context windows. orchestration frameworks. memory architectures. and model benchmarks. Those discussions are important, but they increasingly feel like conversations about components rather than systems.

The next layer, in that framing, comes with a different focus:

persistent state instead of sessions;
formal representations instead of metaphors;
governance instead of improvisation;
optimization instead of generation;
outcomes instead of outputs.

The organizations that eventually win won’t necessarily be the ones with access to the smartest models. They’ll be the ones that learn how to organize intelligence most effectively—how to turn it into something the company can run, measure, and refine as part of its operating fabric.

This is already visible in research from McKinsey, Deloitte, MIT, Gartner, Microsoft, and others. Across different vocabularies and industries. the same pattern keeps showing up: isolated productivity gains are relatively easy. while transforming enterprise performance is much harder. The difference often comes down to workflows, systems, measurement, feedback, and organizational architecture.

The conversation is drifting away from intelligence itself and toward the structures that make intelligence useful. That’s also why the next breakthrough, as I see it, won’t arrive from making AI more human-like. It will arrive from making enterprise intelligence more structured.

Predictions are dangerous in technology, and most of them don’t age well. Still, after months studying the evolution of enterprise AI and trying to explain it in eight articles, the conviction has grown stronger: before the end of this year, a new layer will emerge.

A layer that sits above the models rather than competing with them.
A layer that benefits from every improvement in underlying intelligence.
A layer that makes intelligence part of the operating fabric of the company rather than a separate tool employees occasionally consult.

When that happens, the architectures many organizations built today won’t look wrong. They’ll look incomplete.

And, if the prediction holds, the most important question in enterprise AI may turn out to have been the one people asked too late: not how to make models smarter, but how to make intelligence behave like software.

enterprise AI architecture large language models copilots agents governance persistent state workflows measurement outcomes business transformation