AI Code Hallucinations Increase the Risk of ‘Package Confusion’ Attacks

image via WIRED

April 30, 2025, 7:08 PM

• •AI-generated code often references non-existent third-party libraries.
• •This creates opportunities for supply-chain attacks.
• •Researchers found that nearly 20% of package dependencies in generated code were 'hallucinated'.
• •Open-source models hallucinated more dependencies than commercial models.
• •JavaScript code showed a higher hallucination rate than Python code.
• •These package hallucinations can be exploited to install malware.

Research reveals that AI-generated code frequently includes references to non-existent software libraries, creating a vulnerability for supply-chain attacks. These "hallucinated" package dependencies can be exploited by malicious actors to inject malware. Open-source models and JavaScript code are particularly prone to this issue, highlighting the need for increased caution. With AI poised to generate a significant portion of future code, developers must be vigilant about verifying its outputs to prevent potentially devastating attacks.

Read original article