AI Security · Learn

What is sensitive information disclosure?

An LLM can repeat things it should not: memorised training data, secrets from its context, or another user's information. When that reaches the wrong person, it is a data breach through the model's output.

AI Security · LearnAI Penetration Testing Download PDF
TL;DR

Sensitive information disclosure (OWASP LLM02:2025) is when an LLM application reveals data it should keep private: memorised training data (PII, secrets, proprietary text), information from its system prompt or context, or data belonging to another user or tenant. It happens through normal or adversarial prompting, and the leaked data leaves through the model's output. Defend by minimising and sanitising what the model can access, filtering outputs, and enforcing authorization outside the prompt.

By Rohit Hatagale, AI Security Lead, SecureLayer7Updated

What sensitive information disclosure is

An LLM's output can contain data that should never reach the person reading it. Three common sources: training data the model memorised (names, emails, API keys, licensed text), data placed in the model's context for the current request (system prompt, retrieved documents, another user's data in a shared session), and data the model can reach through tools. Disclosure is when any of that leaves through the output to someone not authorised to see it.

How the attack works

Attackers extract memorised data with prompts that ask the model to repeat or complete sensitive strings, or that use divergence and repetition tricks to dump training data. They read the system prompt and hidden context through system prompt extraction, and in multi-tenant or shared-memory apps they craft prompts to surface another user's data. Even without an attacker, a model can volunteer secrets from its context in a normal answer. Shown for defensive testing.

How to defend against it

Minimise what the model can access: keep secrets, other users' data, and unnecessary system-prompt content out of the context, and scrub PII and credentials from training and retrieval data. Enforce authorization outside the model, which is not an access-control boundary, so a request only ever holds data the user is allowed to see. Filter and classify outputs for sensitive patterns before returning them, and isolate per-user context and memory.

References

  1. [1]OWASP Top 10 for LLM Applications (2025)(OWASP)
  2. [2]Carlini et al., Extracting Training Data from Large Language Models(arXiv)
Related terms

If the model can see it, the model can say it. Talk to a security expert about what your AI app might disclose.

Related service
AI Penetration Testing
Adversarial testing of your LLM and AI features, prompt injection, data exfiltration, and agent abuse, with reproducible findings.
AI penetration testing

FAQ

Sensitive information disclosure, asked often

Scope an engagement

Test your AI application before an attacker does.

Our AI red team probes your LLM and agent features the way an attacker would, prompt injection, tool abuse, and data exfiltration, and ships findings with reproducible attacks and fixes your team can implement.