Three months of building Timberline

At the end of July, Timberline was a query pipeline and a sign-in service. Three months later it is a private AI workspace that healthcare teams use to read long policies, check coverage criteria against their sources, draft letters, and share work with colleagues, without their information going anywhere it shouldn’t.

We haven’t said much publicly while we built it. Now that we have a place to publish, here is the whole story: fifteen releases, the reasoning behind them, and what we learned along the way. Each release has its own entry in Product updates.

July: security first, features second

Most AI products start with a model and add security later. We started from the other end. Timberline is built by Sidechain Security, and the first question we asked was not “what can the model do?” but “where does the information go, and who can see it?”

So the first release was a foundation more than a feature list. Answers come from a model that runs privately, inside Timberline’s own environment. Sign-in runs through your organization’s identity provider. Conversations are stored in your environment rather than the browser, with retention settings and an audit trail from day one. Everything since has been built on top of that boundary, not bolted around it.

August: working with real documents

Healthcare work lives in documents: coverage policies, manuals, denial letters, procedures. A useful assistant has to read them, so August was about getting documents in and reading them well.

• Attachments arrived with a deliberate constraint: Timberline keeps the text extracted from a file, never the original, and deletes it with its conversation.
• Long documents stopped being a problem. A larger model with a 131,000-token context window means a 134-page manual fits in one conversation, read in full rather than in fragments.
• A calmer reading experience made careful reading easier: answers appear at a steady pace, and the page no longer scrolls out from under you.
• Word documents joined PDFs, because that’s where many policies actually live.

September, part one: answers you can check

An answer about coverage criteria is only useful if you can verify it. Much of September went into making Timberline’s answers traceable and faithful to their sources.

• CMS coverage references became part of the workspace: a versioned library in which national coverage determinations take precedence and tables keep their structure.
• Checked citations and source previews verify each citation against its source and let you read the passage without leaving the conversation.
• More file types, including spreadsheets, presentations, and scanned pages, and each of several attached documents keeps its own identity in citations.
• More faithful answers capped the month. Citations to the wrong page fell by more than half in our testing, and the policy conditions an answer restates are now checked against the source’s own wording.

That last release reflects a principle we settled on early: a useful answer first, traceability second, and never block normal work in pursuit of perfect verification. Timberline doesn’t make decisions. It makes the person reviewing the case faster and better informed, and it shows its work.

September, part two: working as a team, safely

The rest of September turned a personal workspace into one a whole team can use, without loosening any of the controls underneath.

• Invitation-only access and the Access panel: signing in no longer creates an account, and administrators manage members without ever seeing anyone’s conversations.
• Projects and secure sharing: ongoing work keeps its context, and sharing is a deliberate act with specific colleagues.
• Drafts ready to copy: letters and replies arrive in their own block, ready to send.
• Password sign-in with MFA for organizations without single sign-on, with breached-password checks and account lockout.
• Governed web search: off by default, enabled only by an administrator, and identifying details are removed from a query before it leaves your environment.
• Security hardening: per-tenant separation at the model layer, short-lived credentials, and security patches applied on every build.

What we learned

Measure, don’t assume. When we moved to a larger model, measurement caught three problems a demo never would have. The grounding work shipped only after we could show it moved the numbers.

Test what you actually deploy. More than once, a change that passed every test failed on the deployed system, and testing the real thing is how we caught it.

Security features are product features. Invitation-only accounts, administrators who can’t read conversations, web search that’s off until someone decides otherwise: for our customers, these aren’t settings buried in an admin page. They are why the product can be used at all.

Follow along

We’ll post new releases in Product updates as they ship. If you’re evaluating AI for work involving PHI, our Trust Center lists the controls behind everything above, and we’re glad to walk your team through it.