Battleship boosts small AI models through sharper questions

Battleship boosts – MIT researchers tested a Battleship-style setup where AI agents had to ask and answer natural-language questions before making moves. The tweak delivered a dramatic performance jump for smaller systems—most notably Llama 4 Scout, which rose from beating humans

For small AI models, the problem often isn’t raw intelligence. It’s timing. Ask the wrong questions—or ask them too early—and the system acts on a foggy picture of the world.

MIT researchers tried to corner that weakness with a game old enough to feel like a childhood pastime. They used a Battleship-style test built around natural-language questions. setting up one AI agent to play the role of a teammate hunting for hidden ships while another agent had access to the board and answered the questions.

The surprising part came when the team changed how the questions were planned. With a more deliberate inference strategy, smaller systems improved sharply, including one model that went from rarely beating humans to winning most games.

The model went from 8% to 82%

In MIT’s test, the biggest jump was for Llama 4 Scout. At first, MIT said this smaller model beat human players in only 8% of games.

After researchers added a more deliberate inference strategy—aimed at how the agent searched the board and set up its next steps—the outcome flipped. The model beat humans 82% of the time.

MIT also reported that this shift didn’t just lift Llama 4 Scout above humans. It also outpaced a larger frontier model while operating at about 1% of the cost.

The message is hard to miss: the breakthrough didn’t come from making the model bigger. It came from making the agent ask sharper questions and make better use of each answer.

Why an old naval guessing game matters

Battleship isn’t just a theme here. It’s a stress test for limited information.

In the setup MIT used, the agent can’t see the whole board. Every question has to narrow the search, which forces the system to treat information gathering as part of the move—rather than a warm-up that can be skipped.

That maps directly onto how many real AI tools are used. A support bot, a research assistant, or a planning agent often can’t help immediately without follow-ups. If the questioning process breaks down, the model can miss a key detail, repeat itself, or produce a recommendation too early.

In MIT’s version of the experiment, the task was designed to measure whether an agent can gather the right information before producing an answer.

What comes next beyond games

MIT’s team also flagged the obvious limitation: Battleship is a controlled environment. It’s easier to score performance in games than in open-ended agent workflows like search, customer support, or workplace software.

Still, the direction is the same one companies have been trying to unlock—making cheaper models feel more capable in everyday use. If smaller models learn to ask sharper questions before acting, the cost curve could change in a way that matters to deployment.

The next milestone MIT points to is transfer beyond the game board: whether the same approach can hold up in real work, where tasks come with unclear instructions, missing files, and the pressure of a user who wants results quickly.

MIT Battleship AI Llama 4 Scout small AI models agent planning natural-language questions inference strategy AI costs frontier model information gathering