In Depth
Protected Health Information is individually identifiable health data: a diagnosis tied to a name, a medication list tied to a record number, a visit note that could be traced back to a person. HIPAA, through the federal Privacy and Security Rules at 45 CFR Parts 160 and 164, governs how that data may be used and disclosed, and it does not care whether the entity doing the disclosing is a person, a legacy system, or an AI agent. If your agent handles PHI on behalf of a covered entity, you are almost certainly a business associate, and the disclosure rules bind you directly.
AI makes a PHI disclosure both easier to cause and harder to detect. An agent that summarizes charts can carry a patient's details into the wrong record. A model fine-tuned on clinical notes can surface one patient's information in another's session. A retrieval agent can pull a record the requesting clinician isn't authorized to see. Because the leak path is the agent's normal output rather than a breach, no alarm fires. The disclosure looks like a helpful answer until someone recognizes whose data it contained.
This is the healthcare-specific, highest-stakes form of AI data disclosure. The mechanism is the same; the consequences are not. A leak of ordinary business data is a contract problem. A leak of PHI is a contract problem plus a federal-compliance problem plus a patient-trust problem.
What It Looks Like
A health system deploys an AI scribe that drafts visit notes and answers clinicians' questions across the chart. A doctor asks the agent to summarize a patient's medication history. The retrieval layer, indexed across the full record set without a per-encounter access check, pulls and summarizes entries from a different patient with a similar name. The summary lands in the active note. No system was breached, no malware ran, yet one patient's PHI has now been disclosed into another patient's record, and the vendor's business-associate agreement just became a live liability.
Why It Matters For AI Vendors
Healthcare buyers run the most demanding security and compliance reviews of any sector, and PHI handling is the first thing they scrutinize. They will want a business-associate agreement, and they will want evidence that the agent cannot leak across patients. "Our cyber policy covers a breach" does not answer the question, because a disclosure through normal model operation may never qualify as a breach under that policy.
The regulatory exposure compounds the commercial one. HIPAA carries tiered civil monetary penalties, and a disclosure can also trigger breach-notification obligations. For an early-stage vendor, the cost of responding to a single PHI event, from investigation to notification to legal defense, can dwarf the deal that introduced the risk.