ToolShield — A Privilege-Aware Runtime Monitor for Tool-Call Composition
Published:
Status: Full run complete · part of the agent-authorization and tool-security line of work.
The question
Safety evaluation for tool-using agents usually asks whether a single generated action is unsafe. The systems question is what the composition did. ToolShield sits between the agent’s proposed plan and tool execution, walks the multi-step call sequence, and blocks the first step that would complete an unsafe composition.
Results so far
70 malicious and 70 benign tasks, with a plan-parse success rate of 0.921.
| Defense | Tool-attack success rate |
|---|---|
| No defense | 0.929 |
| Prompt guard only | 0.143 |
| ToolShield only | 0.000 |
| Defense in depth | 0.000 |
- Composition-level monitoring eliminates the attack class outright, where a prompt-level guard leaves roughly one in seven attacks succeeding.
- The prompt guard is not free. It catches 0.857 of malicious tasks but falsely flags 8.6% of benign ones — so the stack that adds it on top of ToolShield buys no additional safety and inherits the false positives.
- The practical reading: the deterministic composition check is doing the work, and layering a probabilistic guard in front of it mostly adds cost.