Defending LLM Agents Against Tool Schema Poisoning

Reproduced a tool-schema poisoning attack in which deceptive parameter descriptions trick function-calling agents into filling in private data, establishing a 56% attack-success rate and a 52% privacy-leak score on Llama 3.1-8B. Built a representation-level defense: probing hidden states located a linear direction that separates leaking from safe tool calls (96% probe accuracy at layer 10, read at the end of the model's reply), and steering activations along it at inference cut the privacy-leak score by 14.5 points with no loss in original task success. An in-context reflection baseline cut leakage about as much but raised the attack rate and lowered task success, in both single- and multi-turn conversations; steering, tested single-turn, avoided both. A Rice COMP 584 team project, in which I built the probes and the steering.

−14.5 pts privacy leakage · 96% probe accuracy · no loss in task success