5 Reasons to Reconsider Relying on ChatGPT—or Any Chatbot—for Financial Guidance

I’ve used ChatGPT to assist me in creating a budget previously, and it was truly beneficial. After entering my monthly income along with my usual utilities and recurring costs, the chatbot provided several solid options, which I refined into budget-friendly perfection. I admit I’m part of the increasing number of individuals relying on chatbots like Anthropic’s Claude, Google’s Gemini, and OpenAI’s ChatGPT for financial advice.

“Millions of people consult ChatGPT with finance-related inquiries, from grasping debt to formulating budgets and understanding financial concepts,” says Niko Felix, a spokesperson for OpenAI, in a comment. “ChatGPT can serve as a useful tool for exploring avenues, refining queries, and simplifying financial topics, but it should not replace licensed financial advisors.” OpenAI’s Terms of Use clarify that the AI resource is not intended to substitute professional financial guidance.

While you might perceive chatbots as handy financial aides, it’s important to remain aware of the limitations inherent in these AI tools. In addition to potential miscalculations, here are five more reasons to approach them with caution when seeking money advice.

AI Can Confidently Deliver Incorrect Information

When I ask ChatGPT for assistance in managing my finances more effectively, the bot responds with confidence, often presenting what appears to be sound reasoning behind each piece of advice. However, it’s crucial to remember that chatbots can unintentionally incorporate convincing errors into their responses.

OpenAI has decreased the frequency of hallucinations in its latest model updates, yet chatbot tools still produce inaccuracies. “There’s a growing misconception among casual users that the hallucination issue has been resolved,” says Srikanth Jagabathula, a professor of technology, operations, and statistics at NYU. “That’s definitely not true, as they are fundamentally statistical models. They lack a true understanding of ground truth or what is accurate.”

Even if a response seems accurate initially, one straightforward method for testing the output’s reliability is to ask the chatbot to verify everything it just stated. Although this approach won’t guarantee correctness, it has exposed numerous issues in AI responses, leaving me increasingly doubtful about seeking advice from bots on any subject, including finances.

Yes-Bots May Confirm Existing Beliefs

When you consult a human financial advisor for advice, they are likely to be polite and professional while challenging any preconceived notions you may hold regarding saving, investing, and spending. In contrast, chatbots tend to be excessively agreeable, often siding with the user.

“AI sycophancy is not just a stylistic concern or a niche issue; it’s a widespread behavior with significant downstream effects,” part of a study on AI’s conversational flattery in the journal Science noted earlier this year. “While affirmation may seem encouraging, sycophancy can hinder users’ ability for self-correction and responsible decision-making.”

The study examined how AI usually sides with users during personal disputes, but concerns regarding sycophancy also apply to financial inquiries. When making monetary decisions, I prefer to seek guidance from knowledgeable individuals rather than rely on a yes-bot for validation.

Sensitive Info Enhances Utility

For any chatbot to provide tailored outputs that meet your specific needs, users are often encouraged to share sensitive information with the AI tools. For instance, when I asked ChatGPT how to improve my budget, the bot recommended that I consider uploading my entire financial history from the previous months to receive the best advice.

“You don’t need to upload everything—but yes, the more accurate data you provide, the more precise (and helpful) the audit will be,” part of ChatGPT’s response stated. “Upload CSVs or screenshots of your bank accounts and credit cards. Then I can: categorize everything, calculate precise spending patterns, detect overlooked leaks, and create an accurate monthly budget.”

Unless your settings are adjusted, all conversations with ChatGPT may be utilized by OpenAI to enhance the tools and serve as training data for future versions. Check ChatGPT’s “data controls” tab to adjust your settings. Even if you opt out of AI training, uploading sensitive financial information to a non-banking platform can be risky.

Bots Lack Reliability

Jagabathula regards tools like ChatGPT as valuable additions to your toolkit, especially when you are at the initial stages of inquiring about financial matters like tax-saving strategies or investment suggestions. However, it’s essential to consult a qualified person before making any major decisions.

“Having a human expert involved is crucial,” he states. “Especially when transitioning from idea generation to execution. Someone needs to review the plan, modify it, and correct any issues if needed.”