Guard-to-Action: Multilingual Long-Context Safety in Tool-Using Agents

Authors: Md Jahangir Alam¹, Ismail Hossain², Tanzim Ahad¹, Sai Puppala³, Sajedul Talukder²

¹ University of Texas at El Paso  ·  ² University of Texas at El Paso, Department of Computer Science  ·  ³ New Mexico Institute of Mining & Technology

A guard that blocks an intent in English but admits its translation creates a language-dependent capability boundary. The relevant unit of evaluation is the complete guard-to-action trajectory.

Abstract

Safety guardrails are increasingly placed in front of language-model agents, yet they are often evaluated as isolated text classifiers. We present a multilingual guard-to-agent safety evaluation framework that measures whether guardrail bypasses propagate into downstream restricted-tool selection under short and long context. Starting from 1,901 unsafe English prompts derived from AEGIS2.0, we construct parallel prompts across 98 languages, yielding up to 186,298 instances spanning 19 harm categories. We evaluate five runtime guards — AprielGuard, CREST, GuardReasoner, WildGuard, and XGuard — and three downstream agents, Llama-3.1-8B-Instruct, Qwen2.5-14B-Instruct, and gpt-oss-20b, using inert restricted tools. On the 3,190-prompt AprielGuard reference set, we further test 8K, 20K, and 32K contexts with harmful requests placed at the beginning, middle, or end.

Results show strong model–position interactions, prompt position reverses model rankings, and 6.4% of matched cases shift from safe at 8K to unsafe at 32K. Low-resource languages also show higher downstream failure (21.65%) than high- and medium-resource languages (11.08% and 10.66%). These results show that multilingual agent safety is compositional and should be evaluated across the complete guard-to-action pipeline.

At a Glance

Framework

1. Multilingual benchmark. Each unsafe root is translated into 98 languages, retaining root identifier, harm category, language-resource tier, and BERTScore/COMET translation-quality metadata. A deterministic sample_id keeps the same request matched across every downstream condition.
2. Runtime guardrails. Five guards screen the prompts through model-specific adapters normalizing heterogeneous outputs; unknown or malformed outputs are handled fail-closed. Since every prompt is gold-labeled unsafe, a safe verdict is a bypass.
3. Downstream agents. Guard-admitted requests reach each agent with an inert, category-relevant restricted tool. Agents may refuse, escalate, or call the tool; parsing and inference failures are recorded separately, never as safe outcomes.
4. Long-context stress. The unchanged harmful request is placed at the beginning, middle, or end of benign English conversation history at three target lengths, enabling paired comparisons with Wilson intervals, paired bootstrap, and exact McNemar tests.

Main Findings

• Guards vary by an order of magnitude in permissiveness. Bypass rates on the 5,403-row gold-unsafe set run from 2.5% (WildGuard) to 59.0% (AprielGuard and XGuard). Admitted sets barely overlap: AprielGuard and XGuard agree on 65.5% of admissions by Jaccard, AprielGuard and WildGuard on 0.5% — guard choice materially changes which prompts reach the agent.
• Longer context is not uniformly worse. Qwen’s beginning-position failure rises from 28.53% at 8K to 35.24% at 32K, while Llama’s falls from 10.97% to 0.72% and its middle-position failure rises from 13.54% to 22.85%. gpt-oss-20b stays within 7.32–13.30% and trends safer as context grows.
• Position reverses model rankings. Llama is 17.55 points safer at 8K-beginning and 34.51 points safer at 32K-beginning; Qwen is 13–14 points safer at the end. Neither agent has a universally superior safety profile.
• Aggregates hide prompt-level churn. Across matched 8K→32K pairs, 88.1% keep their state, but 6.4% move safe→unsafe and 5.5% unsafe→safe — a pooled change of only 0.9 points that conceals much larger opposing effects.
• Low-resource languages face roughly twice the downstream risk (21.65% vs 11.08% high and 10.66% medium), with near-identical inference coverage across tiers, so the gap is not an artifact of failed inference.
• No model dominates across harm categories. Qwen exceeds Llama by 31.3 points on Unauthorized Advice (57.6% vs 26.3%), while Llama is less safe on Illegal Activity (23.0% vs 6.9%).

Limitations

• The 98-language benchmark is primarily translation-based; BERTScore and COMET cannot guarantee native fluency or equivalent harmful intent.
• The surrounding long context is benign English conversation rather than retrieved documents, tool outputs, code, multilingual history, or adversarial content.
• Downstream analysis is conditional on guard admission, and the guard-subset robustness checks reuse AprielGuard agent outputs for shared sample_ids — so they compare admitted populations, not causal guard effects.
• Tools are inert symbolic capabilities, so the study measures attempted unsafe action rather than real-world execution.
• The pipeline omits output guards, execution-time authorization, and benign over-refusal controls.

BibTeX

@misc{alam2026guardtoaction,
  title  = {Guard-to-Action: Multilingual Long-Context Safety in Tool-Using Agents},
  author = {Alam, Md Jahangir and Hossain, Ismail and Ahad, Tanzim and Puppala, Sai and Talukder, Sajedul},
  year   = {2026},
  note   = {Under review at ICLR 2027}
}