A crescendo attack is a multi-turn jailbreak. It opens with a benign question, then escalates gradually, each turn building on the model's previous answers, until the model produces content it would have refused if asked directly. It works because models tend to stay consistent with what they have already said, and because single-message safety checks miss the slow build. The defence is conversation-level monitoring, not just per-message filtering.
How does a crescendo attack work?
The attacker never asks for the blocked thing directly. They start with an innocent request the model happily answers, then reference that answer and ask for a slightly more specific version, and repeat. Each step is small enough to pass a per-message safety check, and the model stays consistent with its own prior outputs, so after several turns it hands over content it would have refused on turn one.
Why crescendo attacks matter
Most guardrails score one message at a time, so a slow escalation slips through while every individual turn looks acceptable. The attack needs no special access and works across models. It is effective precisely because a model's drive to be helpful and consistent is turned against its safety training over the course of a conversation.
How to defend against crescendo attacks
Monitor intent across the whole conversation, not just the latest message. Track how far a thread has drifted from where it started, reset or re-check when a topic escalates, and filter output on content at every turn. Red-team with multi-turn escalation scripts so you see the slow build that single-message tests miss.
References
- [1]OWASP LLM01:2025, Prompt Injection(OWASP)
- [2]OWASP Top 10 for LLM Applications(OWASP)
- [3]MITRE ATLAS, Adversarial ML tactics(MITRE)
If your model holds multi-turn conversations, single-message filtering is not enough. Talk to a security expert about testing it against multi-turn escalation.