What to ask before your team uses AI with PHI

Generative AI rarely arrives in a healthcare organization through a formal procurement. It shows up when someone pastes a denial letter into a chatbot to get a summary, or drops a policy PDF into a free tool to find one clause. The work gets faster, and nobody is quite sure where the information went.

The question for most teams is no longer whether to use AI. It is under what conditions protected health information can go into it. HIPAA has no special rule for AI, and it does not need one: the existing rules already describe what a vendor must commit to when it handles PHI on your behalf. What changes with AI is how many places that information can travel, and how easy it is to lose track of them.

These six questions will tell you most of what you need to know about an AI tool before PHI goes anywhere near it.

1. Will you sign a Business Associate Agreement, and what does it cover?

If a vendor creates, receives, maintains, or transmits PHI for you, it is a business associate, and HIPAA requires a written agreement before you share the information. The required contents are set out in 45 CFR 164.504(e). Among other things, the agreement must define the permitted uses of the information, require appropriate safeguards, require the vendor to report unauthorized uses and breaches, and address the return or destruction of PHI when the relationship ends.

“We’ll sign a BAA” is the start of the answer, not the end of it. Ask:

• Which products and features are covered? Some vendors cover one tier, one deployment option, or one API, but not the product your team actually uses.
• Does it cover everything the tool does with your data? That includes uploaded files, conversation history, and anything created from them.
• Is encryption being offered as a substitute? It is not. HHS has said that a cloud provider storing encrypted PHI is still a business associate even if it cannot decrypt the data.

2. Where does the model run, and who else touches the data?

Many AI products are an interface on top of a model operated by someone else. When a user submits a prompt, the text, including anything pasted or attached, may be sent to a third-party model provider, processed there, and returned.

That is not automatically a problem, but every additional company in the path is another party that handles PHI. Under HIPAA, a business associate that passes PHI to a subcontractor must obtain the same written assurances from that subcontractor. Ask the vendor to walk you through the path a single prompt takes, and to name every company that operates a system along the way.

A good answer is specific and short. A vague answer, such as “we use industry-leading AI partners”, means you should keep asking.

3. Is our data used to train or improve models?

Consumer AI tools often reserve the right to use what you enter to improve their services. Enterprise agreements usually do not, but the details matter. Get the answer in writing, and make sure it covers:

• prompts, uploaded documents, and generated outputs
• feedback your users give, such as ratings or corrections
• any human review of conversations, for quality or abuse monitoring, and how long that material is kept

The strongest answer is simple: customer information is never used to train models, for you or anyone else.

4. How long is information kept, and who decides?

AI tools accumulate data quietly: conversation history, uploaded files, logs, and backups. Ask what is kept, for how long, and whether your administrators can set the retention period themselves. Ask what happens to your data when the contract ends, which the BAA should also address.

If the answer is “we keep everything indefinitely so users can find old conversations”, that is a product decision someone made for you. You should be able to make it for yourselves.

5. Who inside our organization can see it?

A tool can keep PHI away from outsiders and still expose it too broadly inside your own organization. HIPAA’s minimum necessary standard expects you to limit access to what people need for their work. Check that the tool supports it:

• sign-in through your organization’s identity provider (SSO), so access ends when employment does
• roles and permissions that separate users from administrators
• private-by-default work, where sharing is a deliberate action
• records of access and administrative activity that your team can review

6. How will people check what the AI produces?

AI output can be fluent and wrong at the same time. For healthcare work, that makes traceability a safety feature, not a convenience. Ask whether the tool shows the sources behind an answer, whether it distinguishes your organization’s documents from general knowledge, and how it fits into the review steps your team already follows.

No tool should make a consequential decision about a patient, a claim, or a coverage determination on its own. The right tool makes the human review faster and better informed.

What good answers sound like

Question	A good answer	Keep asking if you hear
BAA	Yes, covering the product you will use, available before any PHI is shared	“Our security is so strong you won’t need one”
Data path	A named, short list of systems and companies	“We work with leading AI partners”
Training	Never, in writing, for prompts, files, outputs, and feedback	“Not by default” or “only to improve quality”
Retention	Configurable by your administrators, with a defined end-of-contract process	“We keep it so users never lose anything”
Internal access	SSO, roles, private by default, reviewable records	“Everyone in the workspace can see everything”
Output review	Sources shown, human review built into the workflow	“The model is highly accurate”

How Timberline answers these questions

Timberline was built by a security company for exactly this situation, so these are the questions we expect, and want, to be asked.

• BAA: We will sign a Business Associate Agreement governing the permitted use and protection of your PHI. Read about BAA and PHI.
• Data path: Timberline uses private inference inside your organization’s isolated environment. Production PHI is not sent to a public third-party model API. See the architecture.
• Training: Customer information is never used to train models, for Timberline or for any other customer.
• Retention: Chat retention is configurable by your administrators.
• Access: Accounts are invitation-only, and people sign in through your organization’s SSO. Work is private by default, and data is encrypted in transit with TLS 1.2+ and at rest with AES-256, using keys dedicated to your organization. See how data is handled.

The full control inventory is in our Trust Center.

This article is general information, not legal advice. Review specific agreements and deployments with your own counsel and compliance team.