transformer scaling – Misryoum reports on skepticism that transformer model scaling alone will deliver true AGI, as startups pursue new approaches.
Transformer models may dominate today’s AI spending, but Misryoum says the industry’s biggest bet may not be enough to reach true general intelligence.
The commercial AI race has largely centered on pre-trained transformer systems. with major labs investing heavily in the same overall paradigm.. The logic is straightforward: improve performance by scaling training compute and expanding data. while relying on backpropagation to optimize model behavior.. However. Ben Goertzel. who helped popularize the term “AGI. ” argues that much of the sector is effectively copying and recombining GPT-style systems. rather than building fundamentally different foundations for human-like generality.
This matters because the economics of scaling are getting harder to justify as marginal gains become more costly. When budgets are tightly tied to one approach, it can also limit room for serious experimentation with alternative architectures.
Goertzel’s critique goes beyond the training method.. He points to the way today’s models learn during use: instead of updating their internal parameters in real time from new experiences. they typically operate within a fixed baseline learned during training.. In his view, that gap could be a blocker for the kind of continual, experience-driven adaptation that humans demonstrate.
Meanwhile, the search for different technical paths is showing up in both research agendas and product launches.. Misryoum notes that researchers at organizations including DeepMind and Microsoft. along with work associated with Ilya Sutskever’s Safe Superintelligence. are exploring architectures aimed at continual learning.. The emphasis is on moving beyond scale alone. toward systems designed to absorb new information more dynamically rather than simply repeating patterns learned earlier.
In practice, this shift also changes how investors and buyers assess progress. If competitive advantage depends on architecture and learning behavior as much as raw compute, then the next phase of AI spending may reward teams that can demonstrate qualitatively different capabilities.
On the commercial side, Misryoum highlights Tokyo-based startup Sakana AI and its beta product, Sakana Fugu.. The system is built as a multi-agent orchestration layer that coordinates multiple frontier foundation models into a single workflow.. Instead of requiring manual switching between models. Fugu is designed to route subtasks autonomously and use looping mechanisms to recover when progress stalls.
Sakana’s approach also reflects a broader trend: rather than assuming one model does everything. products increasingly try to combine strengths across models to tackle tasks like coding. mathematics. and scientific reasoning.. While benchmarks are used to signal performance improvements. the business question remains whether these systems translate into durable. cost-effective advantages as the market gets crowded.
As AI companies map out their next investments, Misryoum says the key issue is strategic focus.. The industry can keep scaling what works. but if breakthroughs toward genuine general intelligence require new learning mechanisms or architectures. today’s concentration of resources could determine how quickly the sector catches up.