AI-powered chatbots are increasingly used in customer service, healthcare, banking, and government services. While these systems are designed to follow policies and protect sensitive data, they are vulnerable to social engineering attacks: carefully crafted conversations that trick the bot into revealing restricted information, bypassing safeguards, or performing unintended actions. Such attacks are difficult to defend against because they exploit human-like interaction rather than technical bugs.
This project will explore the risks of social engineering attacks on AI chatbots. Using open-source chatbot frameworks, you will design and test attack strategies, categorize the types of manipulations that succeed, and evaluate possible defense mechanisms such as stricter intent recognition, role conditioning, or adversarial training.
Project Goal:
The goal is to systematically study what are the tactics that that can be used to manipulate AI chatbots, through social engineering tactics or prompt injection. Build an inventory of attacks and map which popular models are vulnerable to them.
Research Steps:
Conducting a literature review on chatbot vulnerabilities and social engineering techniques.
Setting up one or more open-source chatbots (e.g., Rasa, Botpress, or LLM-based systems) in a sandbox environment where they have access to a database for example.
Designing attack scenarios (e.g., phishing for sensitive data, escalating privileges, bypassing safety filters).
Running controlled experiments to measure success rates of different tactics.
Proposing and testing defensive strategies (e.g., stricter input validation, dialogue-level anomaly detection).