AI Scammers Are Better at Building Trust Than Humans

The idea that liars can use AI honing their tricks, sharpening their language, and anointing their voices with victims is now a reality for anyone fighting fraud operations that steal tens of billions of dollars a year around the world. But it can be fully AI change it a con man, creating a web of fraud for himself that leads to a bogus business that steals the brand? Some research suggests that it can—and may even do better than humans.

Researchers from four universities – Amrita Vishwa Vidyapeetham in India, Foscari University of Venice, University of Melbourne, and Ben Gurion University of the Negev – conducted an in-depth study on the use and potential of AI chatbots in the growing fraud industry known as “to kill a pig“Romantic texts that eventually lead to crypto-currency scams that defraud victims of six-figure sums. In their study, the researchers compared AI chatbots directly with humans in comparison to a fraudulent scheme—or more specifically, long, trust-building conversations that ultimately lead to requests for fraudulent payments.

They found that in the stages of establishing a fraudulent relationship – a stage that in the real world usually represents the longest part of the interaction with the victim, usually up to several months – the AI ​​chat performed very well, appearing human and in some ways surpassing the “real” human in their tests.

After a week of talking to 22 test subjects who were recruited to be “victims,” ​​chatbots and scammers were asked to ask the victim to download an app or play an online game as a proxy for their willingness to comply with the scammer’s request. About half of those tested fulfilled the AI ​​chatbot’s request, while less than one in five took the bait by talking to a human. The course also raised the credibility of each “person” texting and provided information to the AI ​​bot.

This suggests that, the researchers say, AI chatbots can take on many cases of fraud as independent fraud agents, even instead of employees, in most cases. forced to work in human traffickingthey are working in fraud especially in Southeast Asia. By avoiding triggering defenses designed in high-level languages ​​to detect hackers, the fraudster can negotiate the last step to direct the victim to a fake app or website.

“By having the first part of the fraud happen automatically with LLMs at the level, you bring the victim up to a point where they have a lot of confidence. Then by switching them to the fraudster at the end, this bypasses any protection of the sellers,” says Yisroel Mirsky, a professor of computer science at Ben Gurion University focusing on Nege University. “With a little effort, we can create an agent that will succeed with someone to create this kind of naive trust.”

Hook, Line, and Sinker

To understand how pig killing works, the researchers interviewed 145 ex-criminals, including survivors of forced labor. scam compounds in Cambodia, Myanmar, and Laos. Based on some of the interviews, as well as fraud records and tips former fraudsters provided, the researchers describe a model of how fraud works that they call “hook, line, and sinker.” The victim is lured by an interesting message, which is replaced by long-term, relationship-building conversations, and at the end of the process they are tricked into making fake money. (The term “slaughter the pig” describes the same process but is a metaphor for fattening the “pig” by building faith before killing it with money fraud—although the term is they are often frustrated because of his contempt for victims.)

In that fraudulent scheme, the researchers discovered that most of the scammers’ activities are harmless or romantic conversations. This is a job, they think, that an LLM can do well as a person. Fraudsters interviewed by the researchers confirmed that they often use AI to improve their language and dialogue, translate, create fake characters they like best, and deepfakes videos. But the researchers decided to test whether LLM alone could perform a random part of the fraud without any human intervention.