Language acts as a security surface for agent guards

Research shows that translating user requests into different languages can change the safety verdicts of many agent guards.

Step-level guards decide if an agent's next action is safe before each tool call. These guards use English for prompts and benchmarks, but users write requests in many languages.

Researchers built LinguaGuardBench using 1,220 labelled steps from TS-Bench AgentDojo-Traj. They translated user instructions into German, French, Turkish, Swahili and Amharic while keeping tool calls and observations byte-identical.

Most open, small and purpose-built guards change verdicts under translation beyond their own noise floor. Among nine guards below 13B parameters, the mean placebo-adjusted flip rate rises from +1.5 pp on French to +4.4 pp on Amharic.

Purpose-built guards are the most sensitive to these changes. Open guards of 12-30B parameters block more benign steps by up to +5.0 pp false positives, while every significant loss of recall is in a guard below 10B by up to -7.4 pp.

Frontier guards shift by at most about one point. Gemini 3.8 Flash returns identical verdicts with and without the request.

Key facts

  • The mean placebo-adjusted flip rate rises from +1.5 pp on French to +4.4 pp on Amharic for guards below 13B parameters.
  • Open guards of 12-30B parameters block up to +5.0 pp false positives.
  • Significant loss of recall reaches up to -7.4 pp in guards below 10B parameters.

All sections

Edition