AI Chatbots Could Be Putting You at Risk of Scams, Warns Report
AI is now a regular part of our everyday life and is being used more than ever before. People now use AI tools like ChatGPT and Perplexity to get quick answers, find websites, and even solve technical problems. Many major search engines like Google and Bing have also started integrating AI features to stay ahead in the tech race. But as these tools grow in popularity, experts are warning that they can also open doors to new kinds of online threats.
A recent report by cybersecurity firm Netcraft has raised concerns that AI chatbots can sometimes provide incorrect or fake website links. These errors, known as “hallucinations,” happen when the AI confidently gives wrong or made-up information. In this case, it could be a dangerous website that tricks users into giving away personal details—a type of scam known as phishing.
According to the study, OpenAI’s GPT-4.1 model was asked to provide login links for 50 well-known brands across different industries. While it gave the correct link in 66% of the cases, it provided a wrong or misleading URL in 34% of them. Clicking such links could lead users to phishing websites that look real but are designed to steal sensitive information like passwords or credit card details.
One serious example included in the report showed that Perplexity AI gave out a phishing link instead of the real website for Wells Fargo, a major U.S. bank. This highlights how easy it is for users to fall into a trap if they trust AI responses without double-checking the information.
Netcraft also found that scammers are taking advantage of this problem by creating thousands of fake websites. Over 17,000 fake websites designed to trick users were found on platforms such as GitBook. These pages are well-written, fast-loading, and designed to appear legitimate, making it easier for both users and AI systems to trust them. Many of these scams are targeting people in the cryptocurrency world by pretending to be official support pages or software documentation.
Smaller companies are at even greater risk. Since AI tools often rely on large volumes of training data pulled from the internet, well-known brands are more likely to be recognized correctly. Lesser-known brands, on the other hand, may not be represented in the training data, increasing the chances that an AI will guess or “hallucinate” their web address—and this is where scammers jump in, registering these fake URLs to launch phishing attacks.
The report also mentioned another worrying trend: attackers are actively trying to “poison” AI systems used by software developers. One such attack involved a fake API pretending to be part of the Solana blockchain. Developers included this API in their projects without realizing it was fake, which caused users’ transactions to be redirected to a hacker’s wallet. In another case, attackers created dozens of fake GitHub projects, blog posts, and online tutorials under the name “Moonshot-Volume-Bot” to trick AI systems into learning and recommending these harmful tools.
What Can You Do?
While AI is a helpful tool, this report shows that it’s not always reliable when it comes to sharing links or technical instructions. Always double-check URLs before clicking, especially if they come from an AI chatbot. Use trusted sources and official websites, and install a good antivirus or anti-phishing extension in your browser.
For developers, it’s essential to manually verify APIs and software libraries, rather than blindly trusting AI-generated code or recommendations.
As artificial intelligence becomes more deeply integrated into our digital experience, it’s crucial to stay alert and remember that even the smartest machines can make mistakes—ones that hackers are more than ready to exploit.
