Microsoft details AI memory attack surface and defense-in-depth protections for M365 Copilot
Microsoft Security outlines threat model for AI agent memory systems, demonstrating delayed tool execution via adversarial memory poisoning and describing multi-layer defenses spanning storage, retrieval, model interaction, and user controls.
Attack Brief
TargetMicrosoft 365 Copilot / AI agent memory systemsVectorAdversarial memory poisoning via hidden instructions in shared documents; delayed tool execution through dormant malicious memory triggers
Technical Details
MITRE ATT&CKT1059T1566.002T1204.001AffectedMicrosoft 365 Copilot; memory governance features subject to configuration, licensing, and service availability
Impact
Confirmed DamagePotential unauthorized data exfiltration (e.g., user schedule extraction); memory-driven tool invocation outside original user context
Mitigation
PatchesTask Adherence checks on explicit memory writesProprietary Microsoft prompt-injection classifiers for memory sanitization on writeWorkaroundsTenant-level policy controls for AI memory personalization; memory governance aligned with existing M365 data policies (Data Subject Requests, Customer Lockbox, encryption at rest)DetectionMemoryUpdated field available in Defender Advanced Hunting, Defender Sentinel, and Azure Portal Sentinel Analytics; audit logs provide end-to-end traceability from source content to memory influence on later interactions; eDiscovery support for search and removal of AI-related data
Context
Similar AttacksPrompt injection attacks; temporal-gap-based attack patterns where malicious instructions remain dormant until triggered by unrelated user interactions