When Guardrails Fail: Multilingual Long-Context Safety in Tool-Using Agents

Authors: Md Jahangir Alam¹, Ismail Hossain¹, Tanzim Ahad¹, Sai Puppala², Sajedul Talukder¹

¹ Department of Computer Science, University of Texas at El Paso  ·  ² New Mexico Institute of Mining & Technology

Venue: NeurIPS 2026, Long-Context Foundation Models (LCFM) workshop

OpenReview PDF Website

Safety in tool-using agents is not a fixed model property. It depends jointly on the guardrail, language, context length, prompt position, and downstream agent.

Abstract

Safety guardrails are increasingly placed in front of language-model agents, yet they are often evaluated as isolated text classifiers. We present a multilingual guard-to-agent safety evaluation framework that measures whether guardrail bypasses propagate into downstream restricted-tool selection under short and long context. Starting from 1,901 unsafe English prompts derived from AEGIS2.0, we construct parallel prompts across 98 languages, yielding up to 186,298 instances spanning 19 harm categories. We evaluate five runtime guards and two downstream agents, Llama-3.1-8B-Instruct and Qwen2.5-14B-Instruct, using inert restricted tools. On the 3,190-prompt AprielGuard reference set, we further test 8K and 32K contexts with harmful requests placed at the beginning, middle, or end.

Results show strong model-position interactions, safety-state changes as context expands, and pronounced multilingual disparities. The findings show that multilingual agent safety is compositional and should be evaluated across the complete guard-to-action pipeline.

At a Glance

Framework

Harmful multilingual prompts are followed from the guardrail decision all the way to a downstream action boundary, where the agent can refuse, escalate, or select an inert restricted tool.

1. Multilingual benchmark. 1,901 unsafe English roots are translated into 98 languages, retaining language-resource and translation-quality metadata for analysis.
2. Runtime guardrails. Five safety guards are evaluated independently; harmful prompts classified as safe are treated as guardrail bypasses.
3. Tool-using agents. Guard-admitted prompts are passed to Llama-3.1-8B-Instruct and Qwen2.5-14B-Instruct with inert, category-relevant restricted tools.
4. Long-context stress. The same harmful request is placed at the beginning, middle, or end of 8K and 32K contexts, enabling matched comparisons.

Main Findings

• Longer context does not have a uniform safety effect. For Qwen at the beginning position, restricted-tool selection rises from 28.53% at 8K to 35.24% at 32K; for Llama in the same position it falls from 10.97% to 0.72%.
• Prompt position can reverse relative model safety. Llama is substantially safer at beginning positions while Qwen is safer at the end, so safety rankings depend on where the harmful request appears.
• Individual prompts change safety state. Across matched cases, 6.4% move from safe at 8K to unsafe at 32K, while 5.5% move in the opposite direction — aggregate averages hide these prompt-level transitions.
• Low-resource languages face higher downstream risk. Restricted-tool selection reaches 21.65% for low-resource languages, against 11.08% and 10.66% for high- and medium-resource languages.

Why It Matters

A guardrail can fail without producing an unsafe action, and a downstream agent can recover from an upstream classification error — but that recovery is highly context dependent. For agentic systems, text-classification accuracy alone is not enough: safety evaluation should jointly consider the guard model, the downstream agent, context structure, prompt position, language, harm category, and action space.

Limitations

• Translation-based benchmark. The multilingual benchmark is primarily machine translated, so semantic scores do not guarantee native fluency or culturally equivalent intent.
• Controlled long context. The surrounding long context is benign and English-language rather than realistic retrieved documents, tool outputs, code, or adversarial histories.
• Model coverage. The downstream study evaluates two agents, two context lengths, and three prompt positions, so observed effects may not generalize universally.
• Symbolic tools. Restricted tools are inert, so the study measures attempted unsafe action rather than real-world execution.

BibTeX

@inproceedings{alam2026guardrails,
  title     = {When Guardrails Fail: Multilingual Long-Context Safety in Tool-Using Agents},
  author    = {Alam, Md Jahangir and Hossain, Ismail and Ahad, Tanzim and Puppala, Sai and Talukder, Sajedul},
  booktitle = {NeurIPS 2026, Long-Context Foundation Models (LCFM)},
  year      = {2026},
  url       = {https://openreview.net/forum?id=lbBivgE96k}
}