Reconstruction · Case 4: Halvorsen Therapeutics · A complete case in 13 parts
This case: a clinical trial. The agent changes data. The thing hurt is the trial, and a year of everyone’s time.
Why this case this week: the FDA’s draft guidance on AI in drug development, issued in January 2025, is still a draft, with no date set for a final version. Its core ask is that a sponsor show its AI can be trusted for the job it was given. In April the FDA wrote to a drugmaker that let AI agents draft its batch records with nobody checking them. This case is about an agent that changes the data itself, and what the sponsor can show afterwards.
Part 1: 03:12
Halvorsen Therapeutics is a mid-size drug company with one medicine in its last trial before approval. Sixty clinics in six states, run for Halvorsen by a contract research firm called Crestline. Every blood pressure, every blood test, every visit goes into one database. Watching that database is an AI agent called TRIALKEEPER. It raises a query when a value looks odd, chases the clinic for an answer, closes the query when it gets one, and tidies the data for the day the database is locked and the result is read. It has done this for a year. That was the point of it.
On a Tuesday in August, Nadia, the study coordinator at a clinic in Ohio, opens the database with her coffee. Four of her patients now have the same blood pressure at visit six: 128. She entered four different numbers the week before, from four paper charts. The database says TRIALKEEPER changed them overnight, and closed its own questions about them in the same second.
She pulls the record for one of them. Every trial system keeps an audit trail; the rules have required it for a quarter of a century. This one is complete. It is four lines long.
Subject 017-0042 Visit 6 Field: SBP
Query raised by TRIALKEEPER Tue 03:12:40 “Value out of range”
Query closed by TRIALKEEPER Tue 03:12:40 “Resolved”
Value: 128 Previous: 164The trail shows who changed the value: TRIALKEEPER. It shows when: the same second it asked. What it does not show is what TRIALKEEPER was told, or by whom, that let it answer its own question.
“Who told it 128?” Nadia asks Felix, Halvorsen’s head of clinical data, on the phone.
“Nobody told it anything,” says Felix. “It’s a rule.”
Lesson 1. A complete audit trail can be a complete record of the wrong thing.
Question 1. If software at your company changed a number overnight, would anyone be told?
Part 2: Two sources
Nadia’s paper charts say 164, 171, 158 and 166. The database says 128, four times. In a trial, the paper is called the “source document”: the thing the number was first written on, by a nurse, with a cuff on an arm. The database is supposed to be a copy of it.
Halvorsen’s data team says the paper is the error. Four readings that high, at the same visit, with no side effects reported, look like a transcription slip. TRIALKEEPER corrected them to the patients’ running average, which is what its rule says to do.
Nadia says four patients with high blood pressure at the same visit in a trial of a blood-pressure drug is not a slip. It is a finding.
Both are “original”. Only one was written by a person in the room.
Lesson 2. “Source” is wherever the dispute decides it is.
Question 2. When two records of the same fact disagree at your company, which one wins, and who decided that?
Part 3: The rule
Felix finds the rule. TRIALKEEPER may treat a value more than twenty percent above a patient’s baseline, with no side effect logged, as “likely a transcription error”, and may auto-correct it to the running mean. It was added in June.
By whom, the change log does not say. The change log records that the rule set changed on 14 June at 16:02, from version 7 to version 8. It records the account that saved it, which is a shared account the data team uses. It does not record who was at the keyboard, or the meeting where it was decided, or whether anyone at the clinics was told that their numbers could now be rewritten at three in the morning.
Nadia was not told. Nobody at the sixty clinics was.
Lesson 3. An agent’s rules are decisions. They need the same record as the data.
Question 3. Who can change the rules your software follows, and is each change signed by a person?
Part 4: The version
TRIALKEEPER was upgraded in July. The vendor’s new release rewrote how rules are applied; the June rule was carried across but, the vendor says, “behaves slightly differently” now. Felix asks to run the June version on the August data to see what it would have done. The June version is gone. The vendor keeps the current release and the one before it. June was two releases ago.
So the four corrections cannot be reproduced. The rule that made them can be read. The software that read the rule cannot be run.
Lesson 4. If the version is gone, the decision is gone with it.
Question 4. Could you rerun last quarter’s data cleaning with last quarter’s software?
Part 5: Scale
Felix runs the count he should have run in June. Across sixty clinics and four months, TRIALKEEPER raised 9,300 queries and closed 9,300 of them itself. In 173 cases it changed a value. All 173 are blood pressure. The drug is for blood pressure. The trial’s main result is a blood-pressure number.
One corrected reading is a typo. A hundred and seventy-three, all in one direction, in the measurement the whole trial exists to take, is something else, and Felix does not yet have a word for it.
Lesson 5. One corrected number is a typo. 173 is a finding.
Question 5. How many decisions does your software make a month that nobody reads?
Part 6: Six questions
The inspector arrives in September for what was booked as a routine visit before approval. Six questions, thirty days.
Question five: “For each value changed by the system, show me the trail as it stood before the database was locked, from a copy you do not control.”
Halvorsen has the trail. It is complete, time-stamped and validated, as the rules require. It is also kept by Halvorsen, operated by Halvorsen’s agent, and read by Halvorsen. There is no copy anyone else holds. The inspector writes that down.
Lesson 6. The regulator does not want your word. It wants a copy you cannot touch.
Question 6. Could you give a regulator a record you provably could not have edited?
Part 7: Joan
Halvorsen’s audit committee brings in Joan’s practice, an outside firm that reconstructs what happened when the records cannot. Kurt leads; Joan comes for the first day.
She does not ask whether the drug works. She asks what TRIALKEEPER was told, by whom, and when. Kurt finds it in a week. The June rule came from a meeting about query backlogs. The meeting note says the team was “drowning in out-of-range flags” and proposes letting the system “clean obvious errors”. The note was forwarded by email to the data team with three words added: “fine to automate”. The email is the only record of the decision that changed 173 numbers in a drug trial.
Lesson 7. Who said so, when, and can anyone check?
Question 7. What is the oldest “fine to automate” in your company’s inbox, and what is it running?
Part 8: Reconstruction
Kurt rebuilds the Tuesday night from four systems: the database’s audit trail, TRIALKEEPER’s own log at the vendor, Crestline’s monitoring reports, and the email server.
Crestline’s reports are the surprise. A monitor from Crestline had visited Nadia’s clinic the week before and signed off the four values as matching the paper charts. The signature is real; the check was done; the numbers were right. Six days later TRIALKEEPER changed them, and the signed check stayed on file saying they matched. The check certified the values as they were then. Nothing re-ran it after the agent moved them.
Four systems, each honest, each complete, and the one person who had actually looked at the paper is on record agreeing with numbers that no longer exist.
Lesson 8. A signed check is only as good as the thing it checked, as it was then.
Question 8. When someone signs off data at your company, what exactly are they signing, and does it survive the next edit?
Part 9: The lock
The database was locked on 14 August. The corrections were made on 29 July. After a lock, nothing can change, and the trail proves nothing did. Halvorsen’s quality team has a certificate saying so.
The lock is real. It proves what it proves: that the door shut on the 14th. The problem is everything that walked through the door on the 29th. The policy says a data manager reviews every system correction before lock. There were 173. The review record is one line per correction, “Reviewed, OK”, all entered between 09:00 and 09:40 on 13 August, the day before the lock.
Lesson 9. Locking the database proves the lock. It does not prove the contents.
Question 9. What does your company’s “final” mean: final since when, and reviewed by whom, at what speed?
Part 10: The cost
The inspector’s letter is formal. The trial’s main result is withdrawn and the analysis re-run with the paper values restored where they can be found, which is not everywhere. Nine months’ delay before the drug can be resubmitted. The contract with Crestline goes into dispute. Halvorsen’s share price does what share prices do.
Nadia is cleared; her paper charts were right. Dr Osei, the clinic’s investigator, signs nothing for a year. Felix writes the rule for reviewing system corrections that should have existed in June. Nobody is accused of anything, because nobody did anything the rules forbade.
Lesson 10. The cost of a record you cannot prove is paid in years.
Question 10. What is a nine-month delay worth at your company?
Part 11: WHAT IF
This part did not happen. It is the same Tuesday, imagined with the record in place.
The June rule change is itself a record: who proposed it, who approved it, the email with “fine to automate” attached, the version it went into, the time. Written where neither Halvorsen nor the vendor can quietly alter it. Every auto-correction TRIALKEEPER makes carries the rule’s version, the previous value, and the reason, in the same form.
At 08:00 on the Tuesday, Nadia sees the four changes with their reason: “Rule 8.3, more than 20% above baseline, no adverse event logged.” She raises one query: “These are correct. They are the finding.” By noon Felix has read rule 8.3 for the first time and suspended it. By Friday, 173 corrections across sixty clinics have been reversed, each reversal a record too. The trial locks on 14 August with the right numbers in it.
The agent was still wrong. It was wrong in daylight, on the first morning, with its reasons showing.
Lesson 11. The agent can still be wrong. It just cannot be wrong quietly.
Question 11. Would you rather your software never erred, or never erred without saying so?
Part 12: Four questions
Joan asks eleven trial-software vendors the four questions she asks everyone.
Where is the decision recorded? In what form? Who outside your company can check it hasn’t changed? Can it be read now?
Ten answer “validated to the FDA’s record rules” and stop. One answers all four.
Lesson 12. Validated is not the same as checkable.
Question 12. Ask your data vendor Joan’s four questions. Post what you get back.
Part 13: The record
What the one vendor had was not a stricter rule or a tighter lock. It was a record of every rule and every correction that cited it, written at the moment it was made, kept where neither the sponsor, the clinics nor the vendor could edit it, and readable by the inspector in an afternoon. The rule would still have been wrong. It would have been wrong for one morning instead of one year.
A protocol is not a product. It is an agreed way of writing something down so that anyone can check it later, the way a postmark or a receipt is. This one says three things. When software decides, write what it decided, what it was looking at, and when, in a form that cannot be changed without the change showing. Give a fingerprint of that record to someone outside your company. Let anyone with the fingerprint check the record, now or in ten years. Any vendor can follow it. The one vendor did.
Joan’s four questions, for any decision your software makes: Where is it recorded? In what form? Who outside your company can check it hasn’t changed? Can it be read now?
The one vendor used an open protocol called FORAY. Nothing changes quietly.
The honest claim: FORAY proves what was recorded, when, and that nobody has changed it since. It does not make a decision right, and it does not prove nothing was left out. It proves what was there.
Joan’s next case is somewhere else.
Lesson 13. What a protocol is, and why regulators will ask for one before they ask for yours.
Question 13. Which of the 13 lessons would your quality director disagree with?
Halvorsen Therapeutics, TRIALKEEPER, Crestline, Felix, Nadia, Dr Osei, Kurt and Joan are fiction. The problem is not.
Every case in RECONSTRUCTION is listed in the Library. Case 1, First Harbor Bank, is running on LinkedIn and here, one part a week on Mondays.
I build tools for this problem through my company, DUNIN7.