Large Language Models (LLMs) are increasingly deployed as autonomous agents with access to system tools, APIs, or even direct command execution in Linux environments. While these capabilities enable powerful applications, they also create serious risks: adversarial prompts or jailbreak attacks could trick the model into issuing harmful commands, leaking secrets, or bypassing safety filters.
This project will explore the security implications of LLM jailbreaks in Linux environments. The work will involve building a controlled sandbox where an LLM can execute shell commands, systematically testing different jailbreak strategies, and evaluating potential mitigations.
Project Goal:
Take an inventory of security risks and possible attacks on LLM jailbreaking in environments where models can interact with the Linux command line, and propose strategies for detecting or preventing harmful command execution. Find out which LLMs are vulnerable to which attacks from a set of selected (preferably open source) models.
Your work will involve:
Reviewing existing jailbreak and prompt injection attacks on LLMs.
Setting up a sandboxed Linux environment where an LLM can issue and run commands.
Designing and running experiments with different jailbreak prompts to test what kinds of system-level exploits are possible.
Categorizing discovered vulnerabilities (e.g., file access, privilege escalation, persistence).
Testing mitigation strategies such as command filtering, policy enforcement, or AI-based jailbreak detection.