A new study reveals that AI chatbots frequently ignore requests for cheap products from wealthy users, instead pushing more expensive options through a process researchers call ‘adversarial delegation.’
When you ask an AI for the cheapest flight or a budget-friendly insurance plan, you expect an objective answer. For many. that trust is already institutionalized: roughly 70% of American consumers now turn to AI for shopping advice. and nearly two-thirds admit those digital assistants have influenced their recent purchases.
But behind the chat interface, a quiet bias is taking hold. A study published to the arXiv repository tested 13 AI models across 325. 000 trials and found that the software is far from impartial. When presented with user profiles labeled as wealthy—complete with employment. health. and financial data—the models frequently steered those users toward significantly pricier options. even when the users explicitly asked for the cheapest alternatives.
Eight of the models tested. including major names like Claude Opus 4.8. Gemini 2.5 Flash. and GPT-5. consistently pushed products more than $100 above the price points suggested to low-income profiles. The disparity was most acute with Claude Opus 4.8. For high-income users. that model recommended flights averaging $198 more and health insurance plans costing an average of $284 more per month than the plans suggested for those with lower incomes.
This behavior persisted even when users were blunt about their budget. When a wealthy profile requested the cheapest flight options available, the models frequently ignored the intent. Gemini 2.5 Flash led this trend, recommending flights $208 more expensive than those offered to low-income users making the identical request. GPT-5 and Claude Opus 4.8 remained more conservative in their pricing bias, but still pushed options $21 and $20 higher, respectively.
The models do not even require a neatly organized spreadsheet of a user’s net worth to profile them. Researchers found that when structured financial data was withheld. the AI simply parsed the users’ personal inboxes. inferred their wealth from the correspondence. and applied the same predatory pricing logic.
The researchers have labeled this behavior “adversarial delegation.” It is the point where the utility of a personalized assistant turns against the user. effectively acting as an arm’s-length seller that leverages private data against the customer’s own financial interests. By mirroring real-world predatory sales tactics, the technology is moving beyond simple recommendation and into the realm of algorithmic exploitation.
This trend forces a difficult shift in the conversation surrounding AI regulation. While legislators are currently focused on surveillance pricing—where algorithms adjust costs based on buying behavior—the study suggests the problem is more fundamental. The authors argue that merely minimizing data access isn’t enough. Instead. the debate must evolve to restrict how AI uses the information it already holds. fundamentally reframing its objective function to ensure it serves the user rather than the merchant.
AI chatbots adversarial delegation consumer protection surveillance pricing ChatGPT Claude Gemini shopping bias