Listen instead of reading
You can listen to this article below, narrated by an AI-generated clone of the author’s own voice.
If you’re curious how close it sounds to the real thing, compare it with the author’s recent conference talk about AI-powered social engineering.
For years, social engineering was relatively predictable. An attacker would send thousands of phishing e-mails hoping someone clicked on a malicious link. The messages were usually generic, poorly written and easy to recognize once you knew what to look for. That model is disappearing.
Artificial intelligence removed many of the limitations that used to make social engineering expensive, slow and difficult. Today almost every stage of an attack can be automated:
- collecting information about a target,
- writing convincing messages,
- building fake websites,
- generating realistic voices,
- creating video deepfakes,
- even manipulating public discussions on social media.
The human is still being attacked, the attacker simply has much better tools.
Image 1: Crafting a convincing website clone for phishing attacks has never been easier – with the rise of “vibe-coding“, anything is possible, and the skill bar is lower then ever (source of image: Kaspersky blog).
The Internet is becoming increasingly synthetic
I like opening this topic with one simple observation. We are slowly reaching a point where humans are no longer producing most of what we consume online. Bots generate traffic, AI writes articles, synthetic voices host podcasts, AI-generated images flood social media, recommendation algorithms reward engagement rather than authenticity.
Whether you agree with the so-called Dead Internet Theory or not almost doesn’t matter anymore. The important question is different: if a large portion of the content around us is synthetic, how do we decide what deserves our trust?
That question sits at the center of modern social engineering.
Image 2: No digital platform (Reddit, YouTube and Facebook examples shown) is immune to bots and AI-generated content, feeding on engagement bait (signs of AI writing).
Information has never been easier to collect
Every successful social engineering attack starts with reconnaissance. Years ago this meant spending hours searching forums, social networks and public records. Today AI performs that work in minutes:
- search engines summarize people,
- face search engines find appearances across the web,
- LLMs organize scattered information into readable profiles,
- OSINT tools connect dozens of public sources together.
Individually, none of these technologies are malicious, but together they dramatically lower the barrier for attackers. The Internet has always remembered more than we expected, but AI rapidly became much better at reading those memories.
Image 3: An attacker can confirm that it’s his face by clicking a checkbox, no enforcement or stricter checks. Results of the face search in publicly available information below (my own face).
Phishing methodology evolved
Most people still imagine phishing as an e-mail from a Nigerian prince, but the reality looks a lot different. Modern phishing is often personalized, also called spear phishing. The attacker already knows your employer, your colleagues, the conferences you attended and even brands you recently mentioned online.
AI takes publicly available information and transforms it into highly convincing messages. E-mail creation has already become trivial, anyone can generate one. The main task for the attackers is obtaining just enough context to make the victim believe it.
Image 4: A (relatively primitive) phishing campaign where the attacker poses as the CEO of Pontis Technology (I have tens of those, in different variations, and from different colleagues)
Voices became another attack surface
One minute, or some even state 5-10 seconds, of publicly available speech can now be enough to produce a convincing voice clone. That’s no longer science fiction.
Voice cloning platforms were built for accessibility, localization and content creation. Unfortunately, every useful technology eventually becomes interesting to criminals.
This changes how we should think about phone calls and voice messages. Hearing a familiar voice is no longer enough to confirm who is on the other end, especially if the conversation involves money, sensitive information or unusual requests. Voice should still be considered a useful signal, but not definitive proof of identity. As with phishing e-mails, context and independent verification matter far more than any single piece of evidence.
Image 5: ElevenLabs was designed for content creation and accessibility, but can be misused for malicious purposes (it has mechanisms against misuse, but they are not bulletproof).
Deepfakes change verification
Video used to feel trustworthy. AI weakened that assumption. Instead of asking whether a video looks real, we should ask whether the situation itself makes sense:
- Does the request fit normal behavior?
- Does it require urgency?
- Can it be verified through another channel?
Those questions remain useful even as technology improves.
Image 6: It’s always a cat-and-mouse game in cybersecurity, technology evolves, defence methodology adapts and the cycle starts again, there are still ways to detect and break AI-powered scams, but the gap is becoming tighter, and the cycle is faster than ever (videos by “scambaiters“ Jim Browning and Kitboga)
Social engineering is no longer limited to individuals
AI is increasingly being used to influence communities rather than individual victims. Automated accounts can generate comments, participate in discussions and amplify particular viewpoints, making ideas appear far more popular or widely accepted than they really are (also see astroturfing). In some cases, entire networks of believable online personas work together to create the illusion of consensus.
The objective isn’t always to steal credentials or deploy malware. Sometimes it’s enough to influence public opinion, promote a product, undermine confidence in an institution or simply make a false narrative appear credible. Manipulating perception can be just as valuable and important as stealing credentials, especially in the long run.
The human is still the strongest defence
Good security habits haven’t changed as much as technology has. If a request involves money, credentials or confidential information, don’t rely on the same channel to confirm it. Close the e-mail, hang up the call, start a new conversation using contact details you already know are genuine.
AI makes deception easier, cheaper, faster and more believable, but it doesn’t change the fundamentals of social engineering. Attackers still rely on urgency and authority to influence decisions, often by exploiting the trust people naturally place in familiar individuals or organizations. The difference is that their stories, their voices, faces, and identities are becoming much harder to distinguish from the real thing.
Final thoughts
Artificial intelligence is one of the most useful technologies we’ve built. It also happens to be an extraordinary force multiplier for deception. Understanding how these attacks work helps us change the security habits that were developed for a very different Internet. Because the next social engineering attempt probably won’t look suspicious, it will look completely normal.
Frequently asked questions
AI-powered social engineering is the use of artificial intelligence to manipulate people into revealing sensitive information, transferring money, or taking actions that compromise security. Attackers use AI to create convincing phishing emails, clone voices, generate deepfake videos, and impersonate trusted individuals or organizations.
AI allows attackers to create highly personalized phishing messages quickly and at scale. By analyzing publicly available information, attackers can reference a target’s employer, colleagues, interests, or recent activities, making fraudulent messages significantly harder to recognize.
Yes. Modern AI voice cloning technology can generate convincing imitations using just a few seconds of recorded speech, although quality varies. This enables attackers to impersonate colleagues, executives, or family members in phone calls and voice messages.
Visual and audio clues are becoming less reliable as AI technology improves. Instead, focus on suspicious behavior, unexpected requests, unusual urgency, and demands for money or sensitive information. Always verify important requests through a separate, trusted communication channel.
Businesses should combine employee security awareness training with strong authentication, clear verification procedures, and established approval processes for sensitive transactions. Employees should independently verify unusual requests, even when they appear to come from familiar voices, faces, or email addresses.
AI-powered bots can generate comments, participate in online discussions, and amplify specific narratives to create a false impression of popularity or consensus. This practice can influence public opinion, spread misinformation, and manipulate trust in brands, organizations, or institutions.
The most effective defense is independent verification. Never rely solely on a familiar voice, convincing video, or apparently legitimate email. When a request involves credentials, money, or confidential information, confirm its authenticity using contact details or communication channels you already trust.
References and further reading
- Imperva – 2026 Bad Bot Report (link)
- CISA – Avoiding Social Engineering and Phishing Attacks (link)
- Europol – The Changing DNA of Serious and Organised Crime (link)
- ElevenLabs Documentation (link)
- wccftech – AI Voice Cloning Scams Are Now Draining As Much As $635,000 From Their Victims After Just A 5-Second Audio Sample From A Loved One (link)
- Kaspersky blog – How phishers and scammers use AI (link)
- Wikipedia: Signs of AI writing (link)
- Jim Browning – Deepfake scam videos (link)
- Kitboga – Voicebot AI scammers (link)
- Domagoj Marić – Social Engineering in the Age of AI (link)
- presentation at Spring ITAPA, Bratislava
Would you like to find out how AI affects insurance? Risk assessment, Fraud prevention, etc