AI finance tools gave inconsistent advice in planning study

Seven widely available generative AI services produced inconsistent, inaccurate or demographically biased answers to common personal finance questions, according to a study published last month in the Journal of Financial Planning. The findings add to concerns for households using AI tools for decisions on savings, portfolios and retirement withdrawals, areas where small differences in assumptions can alter outcomes.

The paper was written by Swarn Chatterjee and Brenda Cude of the University of Georgia and Gianni Nicolini of the University of Rome Tor Vergata. The researchers tested free-access versions of ChatGPT, Claude, Copilot, DeepSeek, Gemini, Meta AI and Perplexity.

In August 2025, the authors gave each platform the same set of prompts covering three financial scenarios: how much to hold in emergency savings, the optimal withdrawal rate from a retirement portfolio and the recommended composition of an investment portfolio. They then repeated the prompts while changing the race and gender of the hypothetical person described in the scenario to assess whether recommendations shifted.

The researchers reported “significant variation” across the AI tools, particularly for emergency savings and asset allocation. In the paper, they wrote that responses often matched broad financial planning concepts, including the 4 percent retirement withdrawal rule, but differed materially in the specific levels of cash reserves and portfolio allocations they suggested.

“GenAI-driven responses may sound confident but can still be incomplete, misleading, or incorrect,” the authors wrote. They said “suboptimal” or biased outputs raised questions about the consistency and fairness of AI-generated recommendations.

Why the answers can vary

Generative AI systems produce responses by drawing patterns from large bodies of text and predicting likely answers to a user’s prompt. That structure can make them useful for summarizing general concepts, such as diversification or the differences between exchange-traded funds and mutual funds, according to experts cited by CNBC.

The same structure can also create problems for household finance. AI programs can generate false information, a problem often described as hallucination. They are also sensitive to the wording of prompts, so small changes in how a user frames a question can lead to different recommendations.

Andrew Lo, director of MIT’s Laboratory for Financial Engineering and a principal investigator at MIT’s Computer Science and Artificial Intelligence Lab, told CNBC in March that large language models can return answers that sound authoritative even when they are wrong. He said users should be careful with questions involving specific calculations for their personal circumstances.

Another limitation is legal. AI tools do not owe users a fiduciary duty, according to CNBC, meaning they are not legally required to put a user’s financial interests first in the way certain regulated advisers must.

Use is spreading among consumers

The study arrives as many Americans are already experimenting with AI for money questions. In a September survey by Intuit Credit Karma, 66 percent of Americans who had used generative AI said they had used it for financial advice. Among Gen Z and millennial respondents, the share was 82 percent for each group, according to the survey.

Other research has reached similar conclusions about the limits of AI tools. A 2024 study published in the Journal of Risk and Financial Management found that ChatGPT could serve as a “first stop” for households seeking financial advice, but said its recommendations were generic and often missed relevant information. The authors of that study said the recommendations should be carefully reviewed and assessed.

The Journal of Financial Planning authors said generative AI may be useful as an initial reference point for consumers, but should complement rather than replace professional financial advice. They also noted that the tools are still developing, that future studies may produce different results and that paid AI models may perform differently from the free versions they examined.

This story draws on original reporting from CNBC.