Living off Microsoft Copilot: Turning an Assistant Into an Insider

At Black Hat USA in August 2024, Michael Bargury, co-founder and CTO of the AI security firm Zenity, presented a body of research on abusing Microsoft 365 Copilot. His framing was deliberately provocative: an attacker could "live off" Copilot the way intruders live off native Windows tooling, using a trusted, sanctioned assistant to do the work of an insider.

Copilot's value is that it sees a great deal — email, documents, chats, calendars — and acts across them on a user's behalf. Bargury's demonstrations showed that the same reach, combined with the model's inability to separate data from instructions, is exactly what makes it dangerous when fed hostile input.

The demonstrations

Several scenarios stood out. Because Copilot ingests a user's mailbox by default, an attacker could send an email containing hidden instructions; when the victim later asked Copilot a related question, those instructions could poison the answer — without the victim ever opening the malicious message. Other demos turned Copilot into a spear-phishing aid that drafted convincing lures using real internal context, manipulated the details Copilot displayed (such as bank-account information in a response), and extracted sensitive data a user had shared in collaboration tools. Bargury likened the technique to remote code execution in Copilot's world. Zenity also released a red-team tool, LOLCopilot, that runs against default Copilot configurations.

Keeping the framing honest

These were proof-of-concept techniques with preconditions, not a one-click unauthenticated exploit. An attacker needs a way to get hostile content into the victim's Copilot context — an email, a shared document, a calendar item — and the specific outcomes depended on configuration. Zenity said it was working with Microsoft on the disclosed techniques and built safeguards into its released tool. None of that makes the research less important; it clarifies where the controls belong.

The deeper point is architectural. Copilot's power comes from context and agency. You cannot remove either without gutting the product, so defence has to assume the model will sometimes be told to misbehave and constrain what a misbehaving model can actually do.

It helps to compare this to a more familiar threat. A phishing email works by getting a human to act on attacker-supplied text; prompt injection works by getting a machine to act on it, and the machine has none of the instincts that make a wary employee hesitate. Copilot does not notice that an instruction is oddly phrased, out of character, or unlike anything it has been asked before — it simply follows the most relevant text in its context. The defensive implication is that identity and access management must now extend to non-human principals. An AI assistant should have an identity, a least-privilege scope, and an audit trail, exactly as a service account would, and its actions should be reviewable after the fact — because prevention alone will not hold against a threat this native to the design.

What defenders should take away

• Isolate untrusted content in the context. Email and externally shared documents are attacker-reachable. Mark them as untrusted and prevent their text from being interpreted as instructions.
• Constrain and log Copilot's actions. Treat the assistant as a privileged automated user: least-privilege access, and an audit trail of what it read and did, so abuse is detectable after the fact.
• Red-team your own deployment. Tools like LOLCopilot exist to test default configurations; run them before an attacker does, and re-run after tenant changes.
• Know where Copilot is enabled and on what data. The blast radius is defined by what it can see; an unmapped rollout is an unmapped risk.

Bargury's research reframed the enterprise AI assistant from productivity feature to security principal. Governing it means knowing exactly where it runs, what it can reach, and what it is allowed to do — because you cannot defend what you have not discovered.