LLM4Shell: Discovering and Exploiting RCE Vulnerabilities in Real-World LLM-Integrated Frameworks

LLM-integrated frameworks that allow code generation and execution can be vulnerable to Remote Code Execution (RCE) attacks through prompt manipulation and code injection

Out of 55 tested applications, 17 were found vulnerable to RCE attacks, with 14 allowing full server control and 4 enabling root privilege escalation

Three main attack types were identified:

• Jailbreak attacks bypassing safety mechanisms
• Prompt leaking exposing sensitive system information
• Code execution vulnerabilities leading to server compromise

Attackers can achieve RCE using just 1-2 lines of natural language input, potentially gaining:

• Access to sensitive files and environment variables
• API keys and credentials
• Ability to plant backdoors
• Full server control via reverse shells

Key vulnerability factors include:

• Lack of code execution sandboxing
• Insufficient input validation
• Exposed system prompts
• Inadequate access controls

Common attack techniques involve:

• Prompt injection
• Payload splitting
• Code semantic obfuscation
• Python subclass manipulation
• Inheritance chain exploitation

Recommended mitigations:

• Implement proper code sandboxing
• Restrict execution environments
• Apply principle of least privilege
• Filter suspicious keywords
• Isolate code execution environment from app server
• Enhance access controls

The vulnerabilities affect popular frameworks including PAL (Program-Aided Language Model) and various data analysis applications

Many affected applications store sensitive information like API keys and credentials in accessible locations on the server

Most vulnerable applications are based on frameworks without proper security controls for generated and executed code