Philipp Spitzer

Motivation

Large Language Models (LLMs) are increasingly deployed in agentic settings, where they autonomously interact with digital environments. A prominent application is the web, where browser drivers (e.g., Playwright) allow seamless interaction with websites, and models’ native understanding of JavaScript enables task automation without fine-tuning. Together with an industry partner, Scieneers has developed a prototype agentic web system where users delegate tasks to an LLM-based agent that executes them fully autonomously in a browser environment. While the capabilities—especially with evolving frontier models—are promising, so far, little attention has been paid to security in this setup. When an LLM-driven agent operates without human supervision, it opens up new attack surfaces that traditional web automation doesn’t face. A key concern is prompt injection, where malicious instructions are fed to the agent through its inputs or environment. These injections can come from a user’s prompt or be hidden in webpages the agent visits (e.g,. a site embedding deceptive instructions in text or ads). Research has flagged prompt injection as one of the most pressing threats for LLM agents [1]. For example, one study demonstrated that hidden prompts in a webpage’s HTML can hijack a web agent’s behavior, causing unintended actions like credential theft or fake clicks [2].In essence, an attacker could manipulate the agent by injecting commands into content that the agent trusts, leading it to ignore its original instructions. Beyond prompt injections, emerging threat models for agentic AI range from context manipulation attacks to model backdoors and plugin exploits [3], highlighting the need for a structured analysis of the system’s vulnerabilities and the development of robust defenses.

Research Goal

This thesis will tackle the problem of securing an autonomous LLM web agent through three stages:

Threat Modeling: Systematically analyze the current agentic web system to identify possible attack vectors and abuse scenarios. This involves mapping out how an adversary might exploit the agent’s inputs, the websites it interacts with, or its use of tools/plug-ins, in order to anticipate potential threats.

Mitigation Strategies: Propose and evaluate security measures grounded in theory (drawing on established security principles and recent research) but tailored to the agent’s practical context. Possible strategies might include prompt-hardening techniques (to make the agent less susceptible to malicious instructions ), sandboxing or restricting the agent’s actions, input validation and filtering, or runtime monitors that detect anomalous behavior. Each proposed defense will be assessed for its effectiveness and impact on the agent’s performance.

Practical Implementation: Implement the most critical defenses in our autonomous web agent prototype. This will involve coding the chosen security measures into the system and then testing them against a benchmark suite of attack scenarios. The student will create or curate realistic attack examples (e.g., malicious web pages or prompts) to systematically evaluate how well the defenses work, ensuring that the agent can resist or safely handle the threats identified in the first stage.

The thesis is in collaboration with Scieneers (https://www.scieneers.de/). The student will be working hands-on on a cutting-edge project in collaboration with industry engineers. The concrete use case and system access will be provided at the start of the thesis, allowing direct experimentation on a real agentic web platform.

[1] Design Patterns for Securing LLM Agents against Prompt Injections: https://arxiv.org/pdf/2506.08837
[2] Manipulating LLM Web Agents with Indirect Prompt Injection Attack via HTML Accessibility Tree: https://arxiv.org/pdf/2507.14799
[3] From Prompt Injections to Protocol Exploits: Threats in LLM-Powered AI Agents Workflows: https://arxiv.org/pdf/2506.23260v1

Your Profile

We are looking for candidates who:

• Passionate about agentic AI
• Technical foundation: Experience with Python and machine learning
• Self-motivated researcher: Able to work independently while contributing creative ideas
• Strong communication: Excellent English skills for writing and presenting your research

How to Apply

We offer you a cutting-edge research topic at the intersection of agentic AI, safety, close mentorship from experienced researchers, and access to modern LLMs and computational resources. You’ll gain both theoretical insights and practical skills that are highly valued in today’s AI-driven job market.
Ready to advance the future of trustworthy AI?

Please send your current transcript of records, a short CV, and a brief motivation (3–4 sentences) to:

spitzer@kit.edu