In Part 1, we shared findings from a preprint discussing how the validation study, tested against 21 real clinical trials instead of one or two, and the model came out ahead on 16 of them — a 0.80 F1 score overall. That's the topline. Here's the finding underneath it that matters.
Trials Nobody Expected the Model to Get Right
Six of those 21 trials involved drugs with no real-world usage history to learn from — novel or repurposed agents for which the model had to lean entirely on knowledge-graph-derived mechanism data instead of prescribing patterns. Those are exactly the cases where real-world-data models typically fall apart. Instead, the QuantHealth UC model predicted all six with perfect concordance (F1 = 1.0) — its best performance in the entire validation set.
That speaks to the power of our knowledge graph and the differentiator worth sitting with.
Most real-world-data models share the same structural weakness: they're only as good as the historical usage data behind them, which makes them weakest exactly where development teams need help most: assessing a first-in-class or repurposed asset before there's a track record to lean on. Building the knowledge graph specifically to close that gap was a deliberate design choice, and this validation is the first data to prove it.
It Wasn't Just Predicting Yes or No
The model also produced actual effect-size estimates, which held up under Bland-Altman analysis – a small, consistent underestimation bias, not scattered noise. The distinction matters: a model that's wrong in a predictable direction is one a biopharma’s clinical development teams can calibrate around; a model that's wrong randomly is one nobody should trust with a go/no-go call.
The team also applied the model prospectively to one trial (ELEVATE UC-12), flagging which patient subgroups would respond best, which included patients without prior biologic exposure and lower baseline inflammation. Those predictions lined-up with a post-hoc analysis that wasn't even available when the modeling was done.
The Takeaway
This isn't a claim that AI replaces trials -- the gold standard remains the gold standard. What this validation supports is narrower and more concrete: a properly validated RWD-plus-knowledge-graph model earns a seat in the room when stress-testing eligibility criteria, sizing cohorts, and building the case for a go/no-go decision, particularly for novel mechanisms, where historical data alone leaves teams guessing. This is what we at QuantHealth – and the industry – is increasingly calling simulation-first clinical development and it’s how organizations are derisking and optimizing their entire clinical development program.
What makes me excited is that our model continues to get better, especially for UC – and this study is a strong story supporting the use in UC and other therapeutic areas.
The full preprint is available here.