HIPAA and generative AI: what the rules require for PHI

HIPAA and generative AI can work together, but HIPAA has no separate rule for AI. If a covered entity or business associate sends protected health information (PHI) to a model, the existing rules apply: the use has to be permitted, each vendor that handles the PHI generally needs a business associate agreement, only the minimum necessary data should be shared, and the Security Rule’s risk analysis and safeguards must cover the new system.

This article is general information, not legal advice. HIPAA questions turn on your facts, and state laws may add to them, so confirm your position with your own compliance and legal advisers before PHI goes near a model.

HIPAA and generative AI: which rules apply

The closest official analogy is cloud computing. In its guidance on HIPAA and cloud computing, HHS says that when a covered entity engages a cloud service provider to create, receive, maintain or transmit electronic PHI on its behalf, that provider is a business associate. HHS adds that this holds even when the provider stores only encrypted data and has no key.

A hosted model that processes prompts containing PHI is doing a comparable job. That guidance was written about cloud services, not AI, so ask your counsel how it applies to each tool you plan to use.

One more point from the same guidance: the HHS Office for Civil Rights does not endorse, certify or recommend specific technology or products. A vendor’s “HIPAA compliant” label is a marketing statement about its controls. It is not a government approval, and it does not make your use of the tool compliant.

Four questions to answer before PHI reaches a model

1. Is this use of PHI permitted?

The Privacy Rule decides whether a use or disclosure is allowed at all. That is a question for your privacy officer, and it comes before any architecture work. A feature that drafts a visit summary for the treating clinician and a feature that mines records for marketing are very different cases.

2. Is there a business associate agreement with every vendor in the path?

HHS states that a covered entity may disclose PHI to a business associate only after obtaining satisfactory assurances, in the form of a business associate agreement (BAA), that the information will be safeguarded. According to the HHS business associate FAQs, the agreement must describe the permitted and required uses of PHI and bar other uses. Subcontractors that handle PHI for a business associate need their own agreements.

For a generative AI feature, trace the whole path, not only the model:

• The model provider or the cloud platform that hosts the model
• Vector databases and search indexes that store embedded documents
• Logging, tracing and evaluation tools that capture prompts and outputs
• Any annotation or review service that sees real records

Check that the agreement covers the exact product and features you will use, and get a written answer on whether your data can be used to train or improve the vendor’s models.

3. Are you sending the minimum necessary?

The Privacy Rule generally requires reasonable steps to limit uses, disclosures and requests of PHI to the minimum necessary for the purpose. HHS lists exceptions, including disclosures to a health care provider for treatment. For prompt design this argues for selecting the fields a task needs instead of passing a whole chart, and for retrieval that returns relevant passages, not full documents.

4. Does your risk analysis cover the AI system?

The cloud guidance says covered entities and business associates must conduct risk analyses that identify threats and vulnerabilities to all electronic PHI they create, receive, maintain or transmit. A new AI feature creates new places where PHI lives: prompts, model outputs, embeddings, caches and test sets. Add them to the analysis before launch.

Security Rule safeguards mapped to an AI feature

The HHS summary of the Security Rule describes administrative, physical and technical safeguards. The technical ones translate directly into design requirements.

De-identification: when data stops being PHI

Teams often plan to remove identifiers so that HIPAA no longer applies. The HHS de-identification guidance describes two methods under the Privacy Rule:

• Expert Determination. A qualified expert applies statistical and scientific principles, concludes that the risk of identifying an individual is very small, and documents the analysis.
• Safe Harbor. Eighteen listed types of identifiers are removed, and the organization has no actual knowledge that the remaining data could identify a person.

HHS states that data de-identified by either method is no longer PHI. The bar is higher than deleting names. The Safe Harbor list includes all elements of dates other than the year, geographic units smaller than a state with a limited ZIP code exception, device identifiers and any other unique identifying code.

Clinical notes are the hard case, because identifiers sit inside free text. If you rely on automated scrubbing, measure how often it misses, and have an expert or your counsel confirm that the method meets the standard.

Common mistakes

• Staff paste patient details into consumer chat tools that have no agreement in place.
• The agreement covers one product, and the team uses a different feature or tier.
• Prompt logs, caches and evaluation datasets are left out of the risk analysis.
• Encryption is assumed to remove a vendor’s business associate status, which HHS says it does not.
• AI output goes into the record with no human review.

If your AI feature reads from an EHR, the data access side has its own planning work, covered in the FHIR API integration guide. For the wider preparation, see the AI readiness assessment checklist.

How YLinkTech helps

We build AI features for healthcare and life sciences organizations with privacy and security designed in from the first sprint. Our AI and Data practice maps where PHI flows, designs retrieval that respects existing permissions, and sets up logging and evaluation. Our responsible AI and governance work gives your compliance team the documentation it needs. We are engineers, not lawyers, so we work alongside your advisers. Talk to us about your use case.

Frequently asked questions

Sources

• HHS: Guidance on HIPAA and Cloud Computing
• HHS: Business Associates Frequently Asked Questions
• HHS: Summary of the HIPAA Security Rule
• HHS: Guidance Regarding Methods for De-identification of Protected Health Information
• HHS: Minimum Necessary Requirement

This article is general information, not professional advice. See our Disclaimer.