The Conventional Wisdom
Ask any compliance officer about AI and HIPAA, and you'll hear the same checklist: Get the enterprise tier, sign the BAA, turn off model training. Done.
This approach has become the standard because it's straightforward and fits neatly into existing vendor management workflows. The logic seems airtight: if the AI provider can't train on your data and you've got a signed contract making them liable, you're covered.
Major consultancies repeat this framework. Vendor sales teams build their pitches around it. Even OCR guidance emphasizes the BAA as the primary compliance mechanism for Business Associates.
Why It's Incomplete
The conventional checklist addresses liability transfer but ignores operational reality. You're solving for "who gets fined if something breaks" while missing "how do we prove what happened when it does break."
Here's the gap: HIPAA Security Rule § 164.312(b) requires you to maintain Audit Controls. You must be able to record and examine activity in systems that touch Protected Health Information. That obligation doesn't vanish when you sign a BAA. It doesn't get satisfied by turning off model training.
The audit requirement creates a paradox with modern AI privacy features. Many enterprise AI platforms now offer Zero Data Retention, they process your prompt in volatile memory and immediately delete it. No logs. No trace. This sounds like privacy gold, and vendors market it that way.
But if your AI provider instantly deletes the data and your organization captures nothing internally, you've eliminated your audit trail. When an employee accidentally pastes a patient's medication list into Claude or a claims processor uploads a spreadsheet of Social Security numbers to ChatGPT, you'll have no forensic record of what was sent, when, or by whom.
That's not a theoretical risk. It's a compliance violation waiting to be discovered during your next risk assessment or breach investigation.
The Evidence
No AI tool is inherently HIPAA compliant. That's not our opinion, it's structural reality. Compliance depends on three elements working together: the right subscription tier, a signed Business Associate Agreement, and precise backend configuration.
The configuration piece is where most organizations stumble. Under the HIPAA Security Rule § 164.312(b), you're required to retain compliance documentation, including access logs, for a minimum of six years. Some state laws extend that timeline further.
Consider what happens when you enable Zero Data Retention on your enterprise AI platform:
- What the vendor does: Processes your prompt, generates a response, and immediately purges both from their systems.
- What you're left with: No record of who accessed the tool, what data was submitted, or when the interaction occurred.
- What OCR expects during an investigation: Complete audit trails showing access patterns, data flows, and user activity.
The gap between those last two points is where penalties happen.
This isn't about vendors being dishonest. Google Workspace Enterprise tiers and Google Cloud Platform (Vertex AI) are eligible for a BAA. So are ChatGPT Enterprise, Claude Enterprise, Microsoft 365 Copilot with commercial plans, and AWS Bedrock. The vendors are doing their part.
The problem is that turning off model training protects you from one risk (your data improving a public model) while Zero Data Retention creates a different risk (no audit trail for your mandatory six-year retention).
What to Do Instead
You need a layered approach that balances data privacy with audit requirements.
First, implement internal logging before you enable Zero Data Retention. Your IT team must deploy a secure proxy or API gateway that sits between your users and the AI platform. This architecture allows the AI vendor to immediately drop the data payload while your organization captures the necessary metadata: user identity, timestamp, session duration, and endpoint accessed.
You're not logging the actual prompt content or the AI's response. You're logging the fact that Employee X accessed the system at Time Y. That's sufficient for audit purposes and doesn't compromise the privacy benefits of Zero Data Retention.
Second, enforce access controls at the organizational level. Don't rely on training alone. Use your admin console to restrict AI tool access to employees who've completed your internal AI policies training and have documented business justification for using these platforms with Protected Health Information.
For Google Workspace, that means configuring Gemini access controls through your Admin Console and ensuring Google Vault retains required communications data. For Microsoft 365 Copilot, you'll need Data Loss Prevention rules in the Compliance Admin Center to prevent the tool from pulling unencrypted files or sharing sensitive information with unauthorized users.
Third, adopt a Minimum Necessary posture. Just because you can use AI with Protected Health Information doesn't mean you should. Before inputting any protected data, ask whether the task genuinely requires it. Can you de-identify the dataset first? Can you accomplish the task without AI assistance? Technology is only as compliant as the human operator using it.
When the Conventional Wisdom Is Right
The standard checklist isn't wrong, it's just incomplete.
You absolutely need the enterprise tier. Consumer versions of these tools (the free Gemini website, ChatGPT Plus, Claude Pro, the free web version of Copilot) cannot support a Business Associate Agreement. Period. If you're using a personal account or a $20-per-month subscription, you're exposing Protected Health Information to a vendor with zero liability and zero compliance infrastructure.
You absolutely need the signed BAA. Without it, your AI vendor isn't legally a Business Associate, which means they have no contractual obligation to protect your data or report breaches. The BAA establishes liability, defines permitted uses, and creates the legal framework for your vendor relationship.
And yes, you absolutely need to turn off model training. If you don't explicitly disable this feature, your prompts and outputs could resurface in responses generated for external users. That's not a compliance risk, that's a guaranteed breach.
The conventional wisdom gets you 80% of the way there. The missing 20% is the audit infrastructure that proves you actually did all of this when OCR comes asking.
