Evidence-based Probability of Success (PoS) in late-stage drug development: experience from an applied statistician in industry
Rutgers Biostatistics and Epidemiology Lecture Series
2026-10-12
The opinions expressed in this presentation and on the following slides are solely of the presenter, and not necessarily those of Novartis.
What exactly are we predicting? "Success" has to be defined clearly and consistently across programs.
Historical benchmarks, Phase-2 data, and expert judgement: how do we choose and utilize the most relevant evidence?
Decision-makers need an assessment they can trust, probe, and compare, not a black box.
Bringing a new drug to market is a highly competitive, low-probability endeavor.

Hofmann (2026)
Our most rigorous statistical thinking is applied before Phase 3: when data is richest, and when the required investment is largest.
Probability of Success (PoS) is the probability that a program ultimately achieves regulatory approval and meets its Target Product Profile (TPP).
Developed in 2019 for pivotal-stage PoS (Hampson et al. 2022) and since applied across the portfolio. Program-specific evidence is used wherever it exists: a generic benchmark alone would discard it.

Advocacy for black-box, AI-driven prediction is increasingly loud (Beasley 2026).

Example of an LLM-driven pipeline (Zhang et al. 2026).
Our stakeholders value a “white box” approach: transparent, traceable, and open to challenge by the people who must act on it.
Our default stance is often skeptical, driven by industry benchmarks for (often low) success rates of similar programs.
The program benchmark alone does not comprise the assessment. It sets what we believe before looking at the program’s own data.
The prior is a Gaussian mixture: the weight on optimistic vs. pessimistic components is calibrated to match the benchmark success rates.
When the Phase 3 endpoints match Phase 2, predicting Phase 3 is a straightforward Bayesian update.
We simulate the program many times from the efficacy model, combined with a consensus assessment of non-efficacy risks (regulatory, safety, quality, technical), and classify each replicate.
Phase 3 ongoing, pre-readout. Success rule: significance (1 dose) on both exacerbations & FEV1, in both trials.
| Endpoint | Quantity of interest | Available Phase 2 evidence |
|---|---|---|
| Exacerbations | Treatment effect on annualised rate of moderate-to-severe exacerbations: the Phase 3 primary endpoint | None directly: only a surrogate (reduction in sputum eosinophil counts) in one Phase 2 trial |
| FEV1 | Treatment effect on FEV1: a Phase 3 key secondary endpoint | Several Phase 2 studies, but shorter trials in a milder asthma population |
Workshop structure of the basic SHELF protocol.

Four experts give range, median, and tertiles independently; and a follow-up facilitated discussion produces a single group judgement.

Repeating the exacerbations elicitation at several assumed eosinophil effects (50-75%) traces out the conditional distributions needed to capture dependence between the two endpoints.
Predictive distribution for each endpoint’s Phase 3 treatment effect, with P(effect beats target).

Elicited prior (top) vs. realized Phase 3 results by dose and study (bottom), for exacerbations (left) and FEV1 (right) (Holzhauer et al. 2022).
| Stakeholder | Decisions | What they need from PoS |
|---|---|---|
| Portfolio / Finance | Valuation, prioritisation, resource allocation | Comparability across assets; a consistent, benchmarkable definition |
| Governance | Tollgate decisions | A defensible, reproducible, auditable assessment |
| Leadership | Risk evaluation and communication, go/no-go | Where the risk sits, and how it changes with new evidence |
| Project teams | Design decisions: sample size, endpoints, TPP, number of trials | Sensitivity of PoS to design levers; the value of more data |
Conditional on a positive pivotal program on the key endpoints, with no major safety showstopper, what is the remaining probability of ultimate success?
Five factors are each scored via consensus among the project team
| Risk factor | Low | Medium | High |
|---|---|---|---|
| Alignment with key regulator | Clear agreement or strong positive feedback | Generally positive feedback, but unresolved issues | Inadequate discussion or important disagreement |
| Unaccounted safety risks | No known unaccounted safety risks | Some known risks, but limited relevance to the program | Significant risks likely to apply to the program |
| Quality & compliance risks | Low expected risk | Medium expected risk | High expected risk |
| Technical development risks | No major known technical risks; pre- and post-pivotal products very similar | Some meaningful technical risks or product differences | Multiple or significant technical risks |
| Unaccounted TPP risks | Low risk of missing important additional TPP features | Medium risk | High risk, even assuming key pivotal endpoints are met |
To account for any information not captured in Steps 1-3, teams may override the computed PoS with a written, documented, agreed-upon rationale

A standard Bayesian update for the surrogate: prior 24% → posterior 80% probability that the effect exceeds the target.

Four experts give range, median, and tertiles independently; these are pooled into a single group judgement distribution.
Probability of Success in Late-Stage Drug Development