A brief recap: the CHALLENGE trial enrolled patients treated for colon cancer and adjuvant chemotherapy and randomized them to receive either health-educational materials only (control patients), or similar materials plus a 3-year supervised-exercise program (experimental patients). The trial showed a Disease-Free Survival (DFS) benefit, and an Overall Survival (OS) gain in patients randomized to receive the 3-year exercise program.

The trial was presented during the annual American Society of Clinical Oncology 2025 ASCO meeting by Pr Christopher Booth, MD FRCPC, on 1st June, and published the same day in the New England Journal of Medicine. The results triggered enthusiasm and headlines in media outlets.

In a first X Article, entitled "Can we challenge the CHALLENGE results?", also covered in the drugdevletter, I had a more cautionary take, mainly due to concerns about the possibility of informative censoring, with a focus on overall survival.

My first analysis led to two types of reactions

1. Many were happy to see a critical appraisal of the CHALLENGE study, which is a well-conducted, academic trial, and not studying a new, costly drug. We should appraise any trial without preconceived biases.

2. Others were unhappy, mainly, in my interpretation, because they were happy with the results (DFS and OS gain), and were happy with the intervention being studied (exercise), and therefore unhappy to see such a study – and results – criticised.

After all, why would one criticize that doing more exercise is beneficial?

Appraisal shouldn't be driven by personal preferences

I don’t see any justification for applying different standards to evaluate evidence regarding the intervention being studied. Some even advocated for changing practice the day after the presentation. Others downplayed the relevance of any critic of CHALLENGE, simply (I believe) because they love the supervised exercise programme and seem convinced it can only be beneficial.

I have 3 points to make here :

1. Why run the study if we knew for sure exercise was better in the first place ?

2. A 3-year supervised exercise program can be burdensome for patients (more on that later).

3. Imagine the guilt one may impose on a patient that is unable to undergo the adjuvant programme, for whatever reason. This is exactly the same with other adjuvant therapies in general. Poor communication can lead patients to believe that their cancer will return—or has already returned—because they couldn’t follow the adjuvant program.

Before discussing and proposing any additional treatment, or intervention, based on a survival gain, we must be confident that such gain is real. The same principles should apply whether it involves eating nuts daily, receiving chemotherapy, or committing to a three-year exercise program. As physicians, we cannot decide what is convenient or acceptable for patients; they should decide for themselves.

A burdensome program, at least for some patients

A 3-year supervised exercise program can be burdensome for many reasons.

1. This can come with time-toxicity: the time you spend doing exercise is not spent with your family, friends, etc. (more on time-toxicity in our work led by Vinay Prasad, here). You may have to travel to be able to attend the programme, etc.

2. Second, the supervised program can come along with financial toxicity: who will pay for your transportation, parking, etc.?

3. Additionaly, the program may be associated with side effects, and this is one finding of the CHALLENGE trial: "Musculo-skeletal adverse events occurred more often in the exercise group than in the health-education group (in 18.5% vs. 11.5% of patients)."

Burdensome, really?

From the CHALLENGE trial report, we can see that adherence to the program is far from easy. For instance, only 20% of patients attended the 12 recommended supervised exercise sessions of Part 1.

It’s likely that informative censoring occurred, let’s look at DFS now.

My core hypothesis is that patients randomized to the 3-year supervised program who left and were censored in excess within the trial, both for DFS and OS endpoints, may have been different from those remaining in the trial.

If these patients were frailer and more at risk of presenting the event, this could retain patients with better outcomes in the 3-year exercise group, potentially creating an artificial benefit.

Let’s consider a group race, which is relevant here. If we remove the running-time of patients who didn’t complete the course, the average time will logically – and artificially – increase.

While a 3-year exercise programme isn’t inherently toxic, it does come with a potentially significant burden, including time, toxicity, physical, and other challenges. This may select for healthier patients who are able to commit to the program.

Two BREAKING-ICE App© DFS sensitivity analyses.

When reporting time-to-event endpoints, it’s good practice to report both the time when each patient was censored (either with tick marks or circles) and the number of censored patients in brackets after the number at risk at each time interval. This type of reporting has been almost systematic in journals of The Lancet group for years.

The New England Journal of Medicine sometimes reports it, sometimes not.

It’s disappointing that the CHALLENGE trial report didn’t include the number of censored patients, nor tick marks or circles of censoring events: as shown below, the curves are perfectly smooth. Additionaly, there is no discussion about censoring as a potential limitation in the results’ validity.

Fortunately enough, we have tools to estimate the numbers and distribution over time of censoring events, after digitizing curves, like the BREAKING-ICE App© (see previous post). I already conducted sensitivity analyses based on the OS results (detailed here). Here is the DFS, go on: https://www.timotheeolivier-research.com/breaking-ice, and select "CHALLENGE-DFS".

Firstly, you can see an excess of censored patients at early time points in the exercise group. Cumulatively, at 36 months, 19 additional patients have been censored in the exercise group as compared to the control group. If one hypothesises that only a fraction of those patients censored in excess would have presented a DFS event instead of being censored (n = 12), the results are becoming non-significant. (select "CHALLENGE-DFS-Sens1").

This is hardly an extreme assumption: it only changes the status of 15 % of censored patients in the arm with excess censoring during the first 36 months.

You can see that if you apply similar assumptions over a longer period of time, both curves now appear very close (select "CHALLENGE-DFS-Sens2"): it becomes difficult to fit a laser pointer between the red and orange curves!

Below is one screenshot, but the best is to visit the BREAKING-ICE App© and play with it: https://www.timotheeolivier-research.com/breaking-ice

The Reverse Kaplan-Meier Plots : another way to explore informative censoring.

The Reverse Kaplan-Meier plots basically invert all events into censoring ones, and vice-versa, and run the analyses again. Even though it is usually used to estimate the median follow-up duration in trials, it is a useful tool to explore whether censoring occurs. Tomer Meirson et al. elegantly used this method in their work published in the European Journal of Cancer (here).

In CHALLENGE, both the DFS and OS reverse Kaplan–Meier plots and the Cox model outputs suggest that informative censoring may have occurred (see: https://www.timotheeolivier-research.com/reverse-km, select "CHALLENGE-DFS", and "CHALLENGE-OS" and look at the statistical outputs).

These results reinforce the concerns that informative censoring could have occurred in CHALLENGE.

A surprising deviation from the “Common Sense Oncology” principles

“Common Sense Oncology” – or CSO – is a movement born in 2023, with a mission I fully share: “To ensure that cancer care focusses on outcomes that matter to patients”. Many friends and researchers I admire belong to this movement for which I have respect. Amoung the CSO founding members, Pr Christopher Booth is considered the “father” of CSO. He is also the last and corresponding author of the CHALLENGE trial report.

Earlier this year, CSO published guiding principles in The Lancet Oncology for the “design, analysis, and reporting of phase 3 randomized clinical trials”. Although aimed primarily at systemic anticancer therapy trials, the principles can – as the authors note – "be adapted for trials evaluating other interventions”. One principle I strongly endorse is:

“(7) censoring should be detailed, and sensitivity analyses done to determine its possible effects”

Yet the CHALLENGE publication falls short on this key point – one that is especially relevant to the trial.

Since my initial posts, I wrote a letter to the New England Journal of Medicine precisely asking for clarification, but it was rejected: “Many worthwhile communications must be declined for lack of space.” Let’s see which letters will be chosen: my only hope is that the potential for informative censoring will be addressed in the correspondence. I also see this as an opportunity for the CSO movement to offer clarification (see “Takeaway points” below).

Let’s run the “Common Sense Oncology” sensitivity analysis

Amoung other guiding principles, CSO proposes that:

“One of the sensitivity analyses should assume that patients progress at time of censoring (if progression-free survival or disease-free survival are endpoints), thereby providing curves for time to treatment failure, in which treatment failure represents dropping out of the trial as well as disease progression or recurrence”

Even though such analysis is not exactly reflecting time-to-treatment failure (TTF), as there can be censoring in TTF endpoint (for instance at data analysis for patients still under therapy), I have performed the analysis, now look at the HR = 0.94 (0.82- 1.08), and the green and red curves are essentially superimposable.

In short, this additional sensitivity analysis – conducted in line with the CSO principles – adds another layer reinforcing concerns for informative censoring.

Takeaway points

There is a difference between, on the one hand, believing that sedentarism is detrimental to one’s health and that physical exercise should be promoted – which was the control arm of CHALLENGE – and, on the other hand, asserting that a 3-year supervised exercise program will reduce your risk of recurrence and even your risk of death.

Patients with cancer, particularly after adjuvant therapy, have already experienced enduring suffering and may feel exhausted: a 3-year intensive exercise program is not trivial. We should not downplay it, in the same manner we do not trivialise 3-years of osimertinib in the ADAURA trial. Every intervention can have downsides, including those appearing to be automatically beneficial.

My concern is that informative censoring could have driven, at least partly, the reported results of the CHALLENGE trial, both for DFS and OS. This is not intended to be a criticism of the investigators. Informative censoring can occur in well-conducted and well-designed studies, and should be handled through sensitivity analyses or acknowledged as a potential limitation. I have already congratulated the authors, investigators, and patients for this impressive trial.

Finally, as noted earlier, even though the initial report falls short on addressing informative censoring, I believe the CHALLENGE trial offers an important opportunity for the CSO movement to demonstrate – unambiguously – that it applies the principles it advocates, transparently, whether or not its members are involved in the study. In my view, such transparency would be essential for a movement that aims to grow.

With such uncertainty, and until these concerns are resolved, I feel uncomfortable asserting to patients that a 3-year structured exercise program may save their lives.

The online BREAKING-ICE App© has been published recently in The Lancet Oncology, co-authored with Vinay Prasad and Alyson Haslam.

Informative censoring will no longer be a dry concept, you'll be able to visualize it live!

This freely accessible App - https://www.timotheeolivier-research.com/breaking-ice - best used on a computer, could revolutionize the way you approach censoring in survival analysis. Here is why.

What is censoring ?

In a clinical trial, each patient either experiences an event, or is censored if they have not experienced the event. This is why, on Kaplan-Meier curves, you will be seeing either tick marks or circles: they indicate patients that have been censored. What is usually expected in a trial is late censoring, and equal censoring.

1. Late censoring because patients who are still in a trial without having presenting the event will be censored at data cut-off.

2. Equal censoring because the rate of censoring should not be influenced by the arm to which the patients were allocated upon randomization. Censoring should be non-informative.

Let's have a look at the ADRIATIC trial (durvalumab after front-line therapy in limited-stage small-cell lung cancer).

In both curves, you can see tick marks mostly appearing after 27 months, probably due to censoring at data cut-off (median follow-up in both groups = 37.2 months), which is what is expected. However something is missing from the NEJM publication: where are the number of patients censored at each time point ?

The Lancet approach, for years, is to report in brackets, after the number at risk, the number of patients that have been censored. Below is an example from the TransMet trial.

A first feature of the BREAKING-ICE App©

The App will provide estimates of censored numbers, following the Lancet approach, whenever those data are lacking in publications or presentations, which is unfortunately common.

Let's have a look at ADRIATIC again. By using reconstructed Kaplan-Meier curves, the App allows you to estimate those numbers, which were not reported in the NEJM publication. You can also do it yourself: https://www.timotheeolivier-research.com/breaking-ice and in "Select Trial", choose, “ADRIATIC-OS”.

Two types of informative censoring: toxicity and disappointment.

When censoring is unbalanced, particularly early censoring, there may be reasons related to the allocated arm. How does this affect the survival estimates? Let's exemplify this with the two main reasons for informative censoring.

In a race with many runners, if you exclude those who dropped out before the finish, your average time will be skewed towards a faster pace. These analogies are not intended to trivialise being diagnosed with cancer, which is always a significant life event, and as a practising oncologist, I am fully aware of this. However, they can be very effective in understanding concepts.

When considering survival curves, individuals censored excessively might have been frailer and at higher risk of experiencing the event compared to those who remain in the study. If they had continued on the trial, they could have presented the event, but were censored before. This could retain individuals with a lower risk of the event, potentially favouring the experimental arm when it is more toxic.

Another common type of informative censoring is “disappointment-driven censoring”. Here again, let’s imagine you invite a group of people to run a race, and when they arrive, they find slippers… Some won’t accept to run such a race. If those who drop out are the ones who would have run faster, the average time will again be lower.

In a trial, particularly in open-label ones, where patients know the allocated arm, they may be disappointed. This typically happens in cases of a suboptimal control arm. In such situations, some patients will quit the study to seek other treatment opportunities outside the trial. A plausible hypothesis is that those leaving the trial are more connected, have more options, and are in better condition as compared to those remaining in a poor, suboptimal control arm. If this happens, the control arm will retain individuals at higher risk of presenting the event, and this will further penalize the control arm.

Both toxicity and disappointment can occur during a trial.

This is a key point, and a key feature in the BREAKING-ICE App© : you can modelize one or the other type of censoring in one or the other arm, depending assumptions in relation to the trial design.

In the CONTACT-02 trial, patients were allocated to either a toxic experimental arm or a suboptimal control arm. Here is the trial design, followed by the primary endpoint results— a statistically significant Progression-Free Survival benefit.

We performed a very conservative sensitivity analysis based on reconstructed Kaplan-Meier curves. We selected 15% of patients censored over the first six months in the experimental arm (more toxic), and hypothesized they would have presented the event instead of being censored. We then selected 15% of patients censored over the first six months in the control arm (disappointing), and hypothesized they would have stayed in the trial and not presented the event, and been late censored instead. With these two conservative and plausible assumptions, the curves are changing and the PFS benefit is no longer statistically significant.

This analysis has been published, and examplifies how censoring can still be informative even with equal rates of censoring, but different reasons for censoring (full article available here).

The BREAKING-ICE App©: 3 features.

The App uses synthetic individual patient data (IPD) from published Kaplan-Meier curves. One advantage is that there is a set of trials already available in the App, and many others will come! So there is no need to digitize curves yourself (which you can also do, of course).

1. Gives you estimates of censored patients at each time interval (including in a table below the curves)

2. This allows you to perform sensitivity analyses and visualise the live impact on the curves.

3. The statistical output of your analysis is also available.

Informative censoring is no more a dry concept. Use the App, if you are interested in a trial to be digitized, feel free to contact me. There is also a concise step-by-step tutorial here : https://www.timotheeolivier-research.com/breaking-ice-tutorial. The Lancet Oncology publication is also available here.

You can find an example of the sensitivity analyses I ran with the BREAKING-ICE App© in Part 1 of my CHALLENGE-trial analysis. This will also be useful for Part 2 – stay tuned!

I want to start by congratulating the authors, the investigators, and the patients for participating and conducting the CHALLENGE trial, presented during the ASCO 2025 meeting, one of the biggest oncology conference worldwide.

The CHALLENGE results were presented in early June 2025, and published in the New England Journal of Medicine the same day.

In CHALLENGE, patients were randomized after adjuvant chemotherapy for colon cancer to either a 3-year structured exercise program or only health education materials.

The findings are impressive and making the headlines: there is an overall survival benefit in the structured exercise program!

Reasons for censoring in CHALLENGE?

The NEJM report is informative in terms of reasons for censoring, and again I thank the authors for that. They are provided in Table 3.

However, the censoring patterns cannot be properly explored because there are no tick marks or circles to indicate when patients were censored, nor the number of patients censored in brackets after the number of patients at risk (see below the Overall Survival curves from the NEJM publication, 2025)

Why censoring could matter in CHALLENGE?

In the CHALLENGE trial, it is possible that patients who dropped out from the experimental, more intensive arm, did so because they were frailer patients, not able to sustain the programme, and at the same time they could have been those presenting a higher risk for the event. This would constitute, if patients were censored in excess in this arm, the type of informative censoring that we usually observe because of toxicity.

In CHALLENGE, we know the number of events, and the number and reasons for censoring in each arm.

1. Exercise arm: 41 patients died, 22 patients were censored after withrawing consent, and 8 were censored due to loss to follow-up.

2. Control arm: 66 patients died, 16 patients withdrew consent, 6 were loss to follow-up.

Overall, 8 additional patients were censored due to withdrawal of consent or loss to follow-up in the experimental arm as compared to the control arm. Is this relevant?

One assumption, and two BREAKING-ICE sensitivity analyses.

Let's assume that those 8 additional patients were the frailer ones, unable to maintain the exercise programme, left the trial, and later presented the event (death) which was uncaptured. We can model this sensitivity analysis by randomly choosing 8 censored patients and modifying their censoring status into an event.

After digitizing the curves, we can obtain synthetic Individual Patient Data and explore those assumptions, using the BREAKING-ICE App© available here: https://www.timotheeolivier-research.com/breaking-ice

You can select the trial "CHALLENGE-OS-Sens1" for a first sensitivity analysis where 8 censored patients, in the experimental arm, over the whole study period (corresponding to 2.3% of censored patients over this period of time) have their status changed from being censored to the event (i.e. death): the survival gain is no longer statistically significant (see green curve below and statistical output with the upper bound of the HR now = 1.04).

The reverse Kaplan-Meier analysis further support the possibility of informative censoring.

In this method, each event is artificially modified into a censoring event, and vice versa: the KM curves are displayed and a Cox analysis is performed. If a statistical difference is seen (HR, CI, and p-value), this suggests informative censoring could have occurred.

You can see the analysis for CHALLENGE, based on the same synthetic Indivical Patient Data, by selecting "CHALLENGE-OS" here : https://www.timotheeolivier-research.com/reverse-km

As you can see, the HR = 1.31 (CI from 1.13 to 1.52, p-value 0.00), further suggesting the possibility that informative censoring may have occurred.

Conclusion - reasons and time of censoring are needed.

In CHALLENGE, why such phenomen could happen? In the exercise arm, 30 patients were censored due to withdrawal or loss to follow-up, while 41 patients died. Both numbers are close, so it’s logical that informative censoring, if it occurred, could have influenced the results, at least partly.

It's not possible to conclude that the results of CHALLENGE are not valid based on these sensitivity analyses, however reasons for and the time of censoring, including performing similar sensitivity analyses by the authors themselves, would provide more robustness to the reported survival gain.

I hope this first take on CHALLENGE demonstrates how censoring is key in interpreting clinical trial results. Stay tuned for part 2 where I will explain why I would not (yet) implement the CHALLENGE stragegy, and why!

The LUNAR trial

Our new paper, co-authored with Vinay Prasad, is out in the Journal of Cancer Policy. It is an analysis of the LUNAR trial and raises broader questions about Tumor Treating Fields.

Tumor Treating Fields (TTFs) are a medical device therapy that uses electric fields delivered through adhesive arrays placed on patients’ skin. In the LUNAR trial, patients with metastatic non-small cell lung cancer (NSCLC) who had previously received platinum-based chemotherapy were treated with TTFs in addition to standard therapy (either checkpoint inhibitors or docetaxel). This combination led to a median survival benefit of 3.3 months.

However, the LUNAR trial has many limitations, making it of little to no value for informing current clinical practice.

Patients enrolled in LUNAR do not mirror current practice

First-line immunotherapy has been the standard of care for years. However, 68% of patients in LUNAR did not receive prior immunotherapy. Second, among patients allocated to docetaxel — likely because this “choice” was based on local availability —41% had no prior exposure to immunotherapy.

In other words, a significant proportion of patients did not receive first-line standard-of-care (immunotherapy), were enrolled in LUNAR, and again did not receive immunotherapy but docetaxel instead!

Survival analysis: p-hacking?

During LUNAR, it was decided to reduce the sample size after the Data Monitoring Committee (DMC) requested an unplanned interim analysis due to slow accrual. However, the DMC may have departed from pre-specified statistical rules. Unfortunately, even when asked for clarification. the DMC analysis remain unshared, and no explanation has been provided.

In the absence of prespecified and stated reasons to alter sample size, concern arises that the decision was influenced by a chance deviation in data that favored the intervention. It may be tempting to “quit while you are ahead”, and seek to curtail a trial trending in a favorable direction.

Why the analysis remain unshared after the end of the trial has no justification.

Survival analysis: informative censoring?

When a higher proportion of patients drop out, this may be due to increased toxicity. Then, it is plausible that frailer patients drop more easily when facing toxicity, and these same patients may be at higher risk of progressive disease or dying. This is the core concept of informative censoring. Because these patients are uncaptured, the final results may be artificially skewed.

In LUNAR, we suspect this happened, with both more dropouts and more toxicity in the TTFs group (serious adverse events occurred in 53% of patients receiving TTFs, compared to 38% in the control group).

Following a correspondence published in The Lancet Oncology with the LUNAR investigators (also available here), our concerns were confirmed: they actually shared reasons for censoring in their reply (also reply and supplement here). After excluding patients censored due to data cut-off, a 10 percentage-point higher rate of censoring was observed in the TTFs arm, mainly driven by withdrawal of consent.

We decided to explore, based on digitized curves, whether informative censoring could have affected the results. By assuming that 50% of the censored patients in the TTFs group over the first 12 months dropped out due to toxicity and would have experienced the event if not censored, the survival benefit would no longer be statistically significant. The analysis is available here: https://www.timotheeolivier-research.com/breaking-ice (select “LUNAR-OS”).

Even if only hypothesis-generating, this sensitivity analysis — combined with the sample size modification — raises questions about the reliability of the presented results.

Open-label design, and the lack of a sham-control

LUNAR was an open-label trial and not a sham-controlled study. What would a sham-controlled study look like? It would involve “false” TTFs for controls — but with the same level of care around patients.

Indeed, patients in the TTFs group received a significant level of extra attention, including continuous 24/7 support, frequent home-based visits for array changes every three days, and the convenience of direct home deliveries (slide from ASCO 2023 below).

The main issue is that TTFs automatically come with “increased vigilance.” To what extent this could lead to better survival in a very poor prognosis setting is a key question. We know from a randomized trial led by Jennifer Temel that early palliative care for patients with metastatic lung cancer led to a survival gain of 2.7 months, close to the 3.3 months gain reported in LUNAR.

In the case of LUNAR — and TTFs more broadly — without sham-controlled trials, we cannot exclude that any observed survival gain is driven by the by-product of the extra vigilance associated with the device.

Final thoughts

Current data on TTFs in NSCLC have limited clinical relevance, as today’s patients differ from those enrolled in LUNAR. Several factors may have inflated the survival gain: higher censoring in the TTFs arm, reduced sample size increasing the risk of chance findings, and the lack of sham control.

A sham-controlled trial would be key in answering whether TTFs are beneficial. In the meantime, regulators should lean toward protecting poor prognosis patients who will naturally seek for more options. Those should not be exposed to procedures with significant toxicities before reaching sound confidence that they provide a clinical benefit.

Check out our visual abstract! (and full paper here)

Here is an excerpt on evaluating the control arm in oncology randomized clinical trials – a key element in judging trial validity and relevance.

Just out in Nature Reviews Clinical Oncology, my latest on Surrogate Endpoints from the patient's perspective. Despite writing on this topic for 15 years, hope to offer new ideas in this piece.

In the essay I describe the fact that there is some relationship between incentive and developing new and innovative drugs, but the precise nature of this relationship is unknown.

The full paper is also available here.

Our new paper, entitled “Adding Adjuvant Drugs From Distinct Breast Cancer Trials”, co-authored with Vinay Prasad, is out in the Journal of Cancer Policy today ! The full paper is available here (pre-proof).

This couldn’t be more timely, as the 19^th St. Gallen International Breast Cancer Conference 2025 begins in 3 days – on 12th March in Vienna. St Gallen is an internationally renowned medical symposium, held biennially, focusing on the latest evidence-based strategies for breast cancer treatment. It gathers global experts to establish consensus guidelines influencing clinical practice worldwide.

Here is the tweetorial I posted on X to explain our paper:

The figure below shows the structures of two CDK4/6 inhibitors: the upper structure is palbociclib, a drug developed by Pfizer, the first company to create this category of drugs, while the lower structure is ribociclib, a drug from Novartis with a similar structure.

Three “me-too”, costly drugs

A new class of drugs, used for the treatment of hormone-sensitive breast cancer, is CDK4/6 inhibitors. There are three of them on the market: palbociclib (Pfizer), ribociclib (Novartis) and abemaciclib (Eli Lilly). The similarity of the chemical structure of two of them are shown in the figure. All three have shown similar outcomes on the primary endpoint of progression-free survival; and although palbociclib did not meet the secondary endpoint of overall survival, there is no statistically significant difference in overall survival between these therapies. In addition, the side effects appear to be somewhat similar, though the frequency of each side-effect may vary between drugs. None of these therapies are cost-effective. In fact, claims data from the United States shows that adding CDK4/6 inhibitors to treatment costs approximately $70,000 for each extra month the disease is delayed.

An effort to create market dynamics in the Netherlands

Up until January 2025, the Dutch government had negotiated price arrangements for all three drugs. This meant that the health system received a discount but paid the same price for each drug, as the government was not allowed to pay one company less than another for similar drugs.

To create a market dynamic, the government recently delegated the negotiations to the payers.

But things went wrong…

In the Dutch system, as long as a drug is under government control (especially therapies with a high budget impact), companies are obliged to offer discounts to ensure their drug is cost-effective; otherwise, the drug is not reimbursed. However, now that the responsibility has been handed over to the payers, companies no longer feel obligated to make financial arrangements. As a result, only one company, Pfizer—the one with the largest market share—stepped forward to the payers to make financial arrangements.

Novartis went to court

Novartis decided not to compete by making a price arrangement with the payers or directly with the hospitals, but instead took the matter to court to stop payers from creating competition. They claimed that their drug was superior because it had overall survival data and a different side effect profile.

Last week, the court ruled in Novartis' favour.

The judge decided that based on the most recent scientific research (i.e., palbociclib's secondary endpoint for overall survival was not significant), it could not be maintained that the three CDK4/6 inhibitors were equal. As a result, payers had to immediately stop their financial arrangements. This was a major victory for Novartis, as the company could now set its own price with no interference from payers or governmental obligation to offer any discounts.

In a culture with a collective ethos and a healthcare system based on solidarity, as the Netherlands has, irrespective of the final decision, the judge could have said, "Your drug should still be cost-effective; otherwise, it will displace other care and disadvantage other patients." But the judge didn’t.

Superiority cannot be claimed

Reading the court ruling brings to mind a proceeding by the National Advertising Division (NAD), a self-regulation program of the industry in the United States. In 2022, when the overall survival data for Novartis' drug was published, the company launched a campaign referring to their CDK4/6 inhibitor as the only drug in class with a consistently proven overall survival benefit in breast cancer. Eli Lilly, the third competitor, filed a challenge, claiming that Novartis' message was unsupported due to the absence of a head-to-head study. NAD found that although Novartis' drug, ribociclib, had the longest median overall survival, they couldn't demonstrate that the methodologies in the studies, patient characteristics, or other elements were similar enough to allow a valid comparison between drugs. Superiority could therefore not be claimed.

Shortly after, the FDA also fired back regarding the promotional communication of Novartis on ribociclib, as it created a misleading and false impression about the drug’s efficacy and impact on quality of life.

The Duck Test

The development of a me-too product is less risky and therefore more profitable for companies. The needed comparative data between drugs are often never delivered to discourage any comparison. In this case, the largest real-world study, involving nearly 10,000 patients in the United States, shows no significant overall survival differences between the three CDK4/6 inhibitors.

For us these questions remain: how can the healthcare system ensure competition rather than turning me-too drugs into separate monopolies? Does stopping competition and causing higher prices violate antitrust law? And finally, how can the legal system support and stimulate competition when faced with a complex legal case like this?

Do you remember Huey, Dewey and Louie, Donald Duck's identical triplet nephews? Perhaps the court could use the 'duck test': if it walks like a duck, swims like a duck and quacks like a duck, it is… a duck! If a me-too drug has a comparable structure, if the mechanism of action is similar, and if the indication granted by EMA or FDA is similar, then it’s reasonable to conclude that the drugs are close enough to be treated as the same.

Historically, postlicensure safety monitoring has relied on passive surveillance and ad hoc epidemiological studies. Both are useful due to their relatively low operational costs.

However, observational studies based on passive surveillance data suffer from several methodological limitations, which may hinder the ability to draw reliable conclusions. Notably, passive reporting systems do not allow for comparisons between vaccinated and unvaccinated persons.

More recently…

More recently, large-linked databases, such as the Vaccine Safety Datalink (VSD) in the United States and the Vaccine Adverse Event Surveillance and Communication in Europe, have gained popularity. However, vaccination coverages are very high in these databases and few unvaccinated control subjects are left available for comparative analyses.

While postlicensure observational studies of vaccine safety are of critical importance to public health by generating evidence on incidence, prevalence, associations, and prognosis of certain conditions, their nonexperimental nature renders them particularly susceptible to unmeasured confounding and different kinds of bias.

Specifically, differential health-seeking behaviours between vaccinated and unvaccinated individuals often give rise to the healthy vaccinee effect – a situation when patients, who are in better health conditions, are more likely to adhere to the vaccination.

Our study

In our work out in the American Journal of Medicine (also fully available here), we systematically reviewed studies published in 6 top medical journals from their inception through March 2024 and found several interesting findings: 37 studies were examined.

Here are our main findings:

• Authors often acknowledge residual confounding – residual confounding was conceded in 54% of studies, and an additional 24% did so implicitly.

• Authors rarely use negative control outcomes (NCO) – NCO were used in a single study.

• Spin is common – spin practices were noted in 48.6% of the studies.

All the 37 included studies focused on vaccines for the prevention of infectious diseases and concerned a variety of products for children and adults. Unsurprisingly, a large number of studies (30%) focused on COVID-19 vaccines. Most studies (41%) were conducted in the United States.

Despite the intrinsic limitation of observational study designs in eliminating residual confounding, some methods, such as negative control outcomes (NCO), can improve the methodological robustness of such studies by detecting unmeasured confounding. We found that only one study used NCO. NCO are also called falsification testing because they can identify spurious correlations in observational datasets.

Notably, nearly a third of the studies acknowledged the healthy vaccinee effect.

Despite these limitations, spin was found in over 50% of the studies, most commonly of the type “over-interpretation or inappropriate extrapolation of results”. Mind that we exclusively assessed studies published in high-impact medical journals, where, theoretically, the highest standards of quality and scrutiny are upheld.

Our study is not free from limitations. Only one reviewer handled the data extraction and analysis, which may introduce subjectivity. Also, top-tier journals might encourage authors to over-interpret their findings, though these journals may also scrutinize observational studies more carefully during peer review.

To conclude

Our results suggest that observational studies could improve their use of proper methods for detecting confounding factors and, importantly, that editors and reviewers of high-impact journals should make sure that the language used in these studies clearly reflects both the findings and their limitations.

Okay, sit tight! This story involves an old drug, a bile acid, from the 1970s that now cost €153,300 per patient per year.

Chenodeoxycholic acid (CDCA), is originally sold under the brand name Chenofalk. In 2008, the company Leadiant acquires the drug, rebrands it as Xenbilox, and increases its price nearly twentyfold. Then, in 2014, the company applies for 'orphan drug designation' for an off-label indication of the drug and raises the price again, this time by fourfold. This off-label indication, the rare genetic metabolic disorder cerebrotendinous xanthomatosis (CTX), is not new, as its use has been documented in the literature since at least the mid-1980s.

In 2017, the European Medicines Agency (EMA) grants marketing authorisation for CTX. The price is increased again, this time by more than fourfold.

Figure 1 shows the increase in the price of a package of 100 capsules between 2009 and 2017.

It is noteworthy that the supporting clinical data for the EMA authorisation did not belong to the company, but were generated over the years by independent researchers, with some even relating to the use of pharmacy compounding. In other words, the company did not develop or design the drug, nor conduct the clinical trials, yet it received orphan designation, which secured 10 years of market exclusivity in the EU, effectively blocking competitors. In 2021, the Dutch Competition Authority fined the company for violating competition rules by charging an excessive price.

Appeal Against the Fine

The pharmaceutical company appealed to the court in Rotterdam against the €17,044,000 fine imposed by the Competition Authority. They argued that not only were procedural errors made by the Authority (a.o., some documents were redacted and the cost of translating the penalty report was not covered), but they were also wrongly accused of abusing their market power. The list price of their drug was considered too high, and they were criticised for not doing enough to negotiate a lower price with the insurance companies.

The company argued that the list price is the wrong benchmark for assessing the fairness of the price. It was never their intention to charge the list price and their ultimate goal was to charge a lower, negotiated price. In fact, they were willing to offer a 50% discount. However, they did not do so because the health insurers and the Ministry of Health imposed restrictions on the negotiations.

The company also argued that the legal interpretation was unforeseeable for them and that the Competition Authority miscalculated the fine by using the list price; the fine should be reduced at least by 50%, as they had intended to offer a 50% discount. They claimed to have recorded a 50% provision in their books and always intended to refund half of the revenue. Furthermore, the company blamed the health insurers for contributing to the situation that led to the fine, as the insurers continued to cover the full list price.

The “Intentions” Are Not Relevant for Assessing the Company's “Actual Behaviour”

Last week the court ruled that the Competition Authority rightfully based its assessment on the list price of the drug because the company actually billed and received that price. The stated intentions of the company were not relevant when evaluating its actual behaviour. The healthcare budget is limited; the excessively high price meant that fewer financial resources were available for other healthcare services. Additionally, CTX patients were disadvantaged because of the uncertainty surrounding the reimbursement of their medication

Market exclusivity comes with responsibility

The court rejects the company's argument that there has been an unforeseeable legal interpretation under which “the behaviour qualifies as abuse”. Several previous cases by the European Commission and national competition authorities regarding excessive prices have resulted in fines. Therefore, the methodology (price-to-costs comparison) and the fine were foreseeable (lex certa).

The granted market exclusivity comes with a “special responsibility” regarding pricing. After all, this medicine has been prescribed to CTX patients for years. Due to the patients' dependence on this treatment, health insurers were required to continue reimbursement. The court adds: regardless of the company’s views on the insurers' behaviour, it has always been the company’s own choice to bill and collect an excessively high price. This conduct is the sole reason for the fine.

What Else Happened in the Past Years?

For the first 2.5 years after market authorisation, the Dutch healthcare system paid the full list price for the drug. However, after that, a coalition of health insurers and the University Medical Centre of Amsterdam initiated the pharmacy compounding of CDCA. The process was not without challenges, as the company prohibited the supplier of the raw material from providing CDCA for compounding. Eventually, though, the result was a pharmacy-compounded product that met the requirements of the European Pharmacopoeia, serving as a more affordable alternative to the commercial variant.

In 2022, both the Italian Competition Authority (AGCM) and the Spanish Competition Authority (CNMC) imposed fines of €3.5 million and €10.25 million, respectively, bringing the total fines for the company to €31 million.

The question remains: does granting market exclusivity to an existing drug benefit anyone at all, especially in this case when the company played no key role in creating the evidence?

The authors of this globally-run industry-sponsored trial demonstrated that a higher number of women with locally advanced (but not incurable) cervical cancer would be alive after 3 years of follow-up when treated with 2 years of pembrolizumab (experimental treatment with immunotherapy) in addition to receiving several weeks of chemotherapy and radiation therapy (the standard of care).

In the trial, nearly 13 patients need to be treated with immunotherapy to keep 1 additional patient alive after three years.

The reported findings of the KEYNOTE-A18 have generated enthusiasm and have already led to approval for pembrolizumab for treating women with locally advanced cervical cancer by both the Food and Drug Administration and European Medicines Agency (and will most likely be incorporated in clinical practice guidelines).

In our new Cancer journal paper, we argue that the reported findings should not change clinical practice. We question their reliability and doubt the survival benefit would hold if post-protocol care were optimal. We detail our reasoning here! (Full paper available here.)

Why shouldn't KEYNOTE-A18 change practice?

Findings from an earlier clinical trial (KEYNOTE-826) demonstrated that in women with incurable cervical cancer, the addition of the same immunotherapy drug, pembrolizumab, to the standard of care, prolonged survival. This was presented in September 2021 and incorporated in practice guidelines in October 2021. KEYNOTE-A18 started accrual in June 2020.

General oncology principles…

A basic principle in cancer treatment: moving a treatment to an earlier phase of disease, based on disease biology and patient eligibility/physical condition, often results in more patients being exposed to the treatment for a longer time.

When a treatment improves survival in an incurable phase (like in KEYNOTE-826), it will often subsequently be tested in a clinical trial in patients who will be treated with curative intent – aiming to increase the fraction of patients cured – before a patient becomes incurable. This is the adjuvant setting, like in KEYNOTE-A18.

In the adjuvant setting, by definition, an (often large) number of patients will be exposed to the experimental treatment who would have been cured anyway (but nonetheless exposed to the toxicity of the experimental treatment). A number of patients will be exposed to the experimental treatment but die of their cancer anyway. In a positive clinical trial, a (usually small) fraction of patients will be cured (or live longer) due to the experimental treatment.

… applied to pembrolizumab in locally advanced cervical cancer

• If at the time of initiation, the KEYNOTE-826 results were not yet known, the KEYNOTE-A18 should not have been initiated yet.

• If the results were known (trials were run by the same sponsors and overlapping authors), cross-over to pembrolizumab should have been guaranteed per protocol for patients in the control arm of the trial.

• When the KEYNOTE-826 results became available most patients participating in the KEYNOTE-A18 would not yet have recurred and as such, an amendment to the KEYNOTE-A18 protocol should have been made that guaranteed optimal post-recurrence care with cross-over per protocol (pembrolizumab next to chemotherapy).

When a patient's disease recurs or progresses, the study treatment (whether experimental or control) will be discontinued. The treatment that patients then receive after the study treatment is called the post-protocol treatment.

The KEYNOTE-A18 should have addressed the key question at hand: “Do we improve survival by exposing all patients in the curative setting to pembrolizumab, or would survival be the same if we limit pembrolizumab exposure to only those patients that, in time, become incurable”.

However, this question would only be answered if patients randomised to placebo would have all received optimal post-protocol treatment, including pembrolizumab, in case they became incurable.

Sadly, in KEYNOTE-A18, when control-group patients recurred, access to pembrolizumab was not guaranteed. At the time of the reported analysis, only 51 of 193 (26%) patients that became incurable in the control group received immunotherapy as a post-protocol treatment (41 of 51 were treated with pembrolizumab). Also, no data regarding post-protocol treatment with chemotherapy and bevacizumab were made available. Thus, most patients in the control arm (treated with chemotherapy and radiation therapy) did not receive immunotherapy when their cancer relapsed or progressed, whereas by definition, every patient in the experimental arm initially received immunotherapy. The results are thus based on exposing everyone in the experimental arm to immunotherapy, as compared to only a quarter of patients in the control arm at the later stage.

The results are therefore, alas, non-informative. The modest survival benefit may well be only the result of withholding the standard of care in the control arm patients. Those benefits could have evaporated if the control arm patients had proper access to pembrolizumab upon recurrence.

A difference between White and non-White patients?

Interestingly, the investigators did an analysis per protocol to compare White to non-White patients.

It seemed that the reported OS benefit was limited to non-White patients (51% of all patients). It did not work at all in White patients.

This finding supports our hypothesis. It is well known that access to any treatment (let alone costly immunotherapy) is higher in White patients. It may well be that White patients with cancer progression in the control arm of the KEYNOTE-A18 had better access to any post-protocol treatment as compared to non-White patients. This suggests that indeed there is no advantage to administering pembrolizumab early in the treatment landscape of women with cervical cancer.

Next to the doubt of the validity of the reported findings, a calculation shows that it costs nearly 3 million dollars to prevent one death after three years ($2.729.375 based on the methodology described here). The treatment duration – 2 years – also raises concerns.

Concluding words

The main reason for suboptimal post-protocol care is that trials are conducted globally, including in countries with limited or no access to optimal treatment upon recurrence. This was described in many trials leading to marketing authorisations. Taken together and based on the above, we feel that pembrolizumab should not be adopted in clinical practice guidelines for treating patients with locally advanced cervical cancer based on the data as reported. Read our full analysis here (also available here).

In a blog post titled "Government Fails, Not the Market," they argue that investing in the development of homegrown CAR-T therapies is a waste of taxpayers' money, unduly exposes patients to experimental treatments, and insults the biotech ecosystem. They find it particularly concerning because it follows a previous €30 million government subsidy to “cobble together” a CAR-T for non-Hodgkin lymphoma (NHL), which they consider a waste of scarce public funds.

Reading their statement today, several things go through my mind. Why are biotech companies reacting so strongly to a research grant, and why are they suddenly concerned about the waste of public money? Public resources are constantly wasted on authorised drugs with excessively high prices, unfunded lengthy treatment durations and ambiguous effectiveness. I’m also wondering why biotech companies are undermining a research programme with hurtful, damaging words and essentially unfounded allegations, such as “cobbling together” and “exposing vulnerable patients,” when this in-house CAR-T has already undergone a phase 2 trial (providing a similar level of evidence as some commercial products), and the current grant aims to deliver phase 3 data for regulatory approval in the second-line setting.

If anything, the research team deserves to be applauded for their noble mission to enhance accessibility and the steps they are taking to drive future innovations. After all, we shouldn't forget that CAR-T originated as an academic achievement

Limited access and displacement of other care

Currently, six CAR-T cell products are approved in the European Union, but access to these products has been limited. The combination of high costs and limited clinical evidence (typically based on small patient groups in single-arm trials) makes it difficult for HTA bodies to assess cost-effectiveness or negotiate financial agreements with the companies providing these therapies, without negatively impacting other healthcare services.

For the first authorised indication of Cilta-cel for MM patients who have had at least three prior treatments, the Dutch HTA advised the government in 2022 against covering the therapy due to its high costs and the negative impact on the healthcare budget.

To contextualize what is meant by “high costs”, the cost of one infusion bag of Cilta-cel is 420,000 euro based on the report of our HTA from 2022.

The HTA was unable to recommend a cost-effective price because the analysis submitted by the manufacturer was of insufficient quality. Recently, based on a new HTA review, the government has been advised to negotiate a price reduction of at least 35%, as the budget impact of Cilta-cel could cause a significant displacement of other healthcare services, “equivalent to about 800 full-time district nurses”. This is for a country with a population roughly double that of New York City.

While the first authorised indication of Cilta-cel is expected to become available soon following a successful financial arrangement, the reimbursement dossier for the second-line indication of this drug, which was authorised last year, still needs to be submitted by the company to the Dutch HTA body. This indication concerns a much larger patient population and therefore has a higher budget impact.

Many countries are investing in in-House CAR-T production

Worldwide, a growing number of academic centres are establishing the necessary infrastructure and expertise to develop and produce CAR-T therapies. In Europe, the EMA is actively supporting academia and non-profit organisations in developing CAR-T or other advanced therapy medicinal products (ATMPs) to help them meet regulatory requirements. The first CAR-T pilot is ARI-0001, a second-generation CAR developed at Hospital Clínic de Barcelona for Acute Lymphoblastic Leukaemia (ALL) and NHL. A second CAR-T (ARI-0002h) for multiple myeloma (MM) is also being developed through an academic network, including the initiative in the Netherlands.

Building academic networks for collaboration, alongside effective regulatory oversight, is essential for the sustainable development and production of CAR-T therapies.

Six reasons for countries to invest in CAR-T development

Investing in in-house development and production of CAR-T therapies offers several strategic, operational and financial advantages:

• 1. Production of CAR-T at academic facilities can be achieved at a fraction of the commercial cost, making therapies more affordable and accessible.

• 2. There is no need for patient materials to be shipped to a centralised facility, often outside the country, for processing. This reduces the time required to start therapy and potentially has a positive impact on the quality of the cells and the outcome of the therapy.

• 3. Greater flexibility to innovate, like rapid prototyping and testing of new CAR-T modifications, enables better-tailored treatments with improved safety and efficacy.

• 4. Academic centres can prioritise treatments for underserved populations, like paediatrics or rare conditions, as there's no pressure for high financial returns.

• 5. Greater transparency regarding the product characteristics, which is important for the reproducibility of research, predicting efficacy and for investigating long-term effects, such as the potential development of secondary malignancies.

• 6. Manufacturing process between clinical trials and routine treatment is more consistent for in-house CAR-T; this can be a challenge for commercial CAR-T cell therapies, as companies need to significantly scale up production after authorisation. In its assessment report regarding uncertainties and limitations of Cilta-cel, the EMA notes: “Regarding the proposed commercial manufacturing process, comparability of commercial product with clinical trial lots could not be fully established at the quality level. Currently, only few clinical data are available for patients treated with lots from the commercial process, supporting the efficacy of these lots but not excluding slight differences.”

From the Hospital Clínic de Barcelona, where CAR-T products are manufactured, Urbano-Ispizua et al. note that, while it may seem we are competing with pharmaceutical companies, “a symbiotic relationship is plausible.” “ Building Point-of-care manufacturing infrastructure does not have to serve as a competition for pharmaceutical companies but can instead narrow the inequality gap that currently exists with commercial CAR T-cell programs.”

Immune checkpoint inhibitors (ICIs) have changed the landscape of cancer therapy due to their ability to bind and block the action of cancer cells, thus allowing a person’s own immune system to attack and destroy invading cancer cells. This has led to durable responses being produced in cancers that previously had limited response to treatment, namely melanoma and non-small cell lung cancer.

2018 estimates

Early approvals for ICIs were primarily in these tumor types, in addition to others, such as Hodgkin’s lymphoma and urothelial cancer. As of 2018, seven years after the initial ICI approval for melanoma, we estimated that the percentage of US patients with advanced cancer eligible for ICIs had grown from 1.5% to 43.6%, and the percentage of US patients with advanced cancer with a response had grown from 0.14% to 12.5% (see the figure below).

Updated 2023 estimates…with regulatory context

Since then, there have been a number of additional approvals for ICIs, including new drug approvals (e.g., dostarlimab), new mechanisms (e.g., relatlimab, which is an anti-LAG-3), and approvals for tumor types without prior ICI approvals (e.g., triple-negative breast cancer). With these recent approvals, we realized that our first estimates in 2019 needed to be updated, so we set out to do just that. We also were interested to know whether an increase in response mirrored the increase in eligibility that we would likely find.

We update our estimates through 2023, and by then eleven ICI drugs had been approved for 88 different indications for 20 general tumor types. In contrast, there were only six ICIs approved for 14 tumor types as of 2018.

In the updated figure, the estimated eligibility in 2023 was 56.4%, and the estimated response was 20.1%. We can also visualize the number of approvals over time for additional context.

The tumor types that had the highest percentage of people who responded to or were eligible for ICIs were tumor types that had ICI approved prior to 2018, including non-small cell lung cancer and renal cell carcinoma.

Our results suggest that although there have been a number of additional approvals since our first estimates, many approvals have been for tumors with low occurrence and/or have a low response to ICIs. This is indicated by our findings of an 83% increase in the number of ICIs compounds since 2018, versus increases in estimated eligibility and response of only 27% and 61%, respectively.

Who are the patients who respond?

In 2023, the estimated response to ICIs was 20.1% . In Figure 2, we detailed the percentage of patients who respond by tumor type and oncology indications.

Durability of responses

As part of our analysis, we also evaluated durability of responses. We found that only 16 drug approvals had follow-up for at least 10% of patients at 3 years and only 2 had follow-up for at least 10% of patients at 5 years. Of those with patients at risk, the median percentage of patients progression-free at 3- and 5-years was 18.9% and 25.5%. The long-term improvement in response and survival was best for melanoma indications.

We have several thoughts on our findings

• First, as more approvals have been granted, a number of them are biomarker specific. This can be a good thing since it allows for therapies being administered to patients who will likely benefit the most. While this may reduce the eligibility to a certain extent, it will be offset by an increase in response. We encourage drug evaluation that uses appropriate subgroup analysis, which puts patient outcomes before maximizing company profits.

• Second, while there have been a number of biomarker specific approvals, there have also been a number of approvals for tissue-agnostic indications, which may be less favorable for the patient. TMB-high tumors had one of the highest contributions towards eligibility and response estimates. For our estimates, we assumed that response was evenly distributed across all tumor types with this mutation, but this is likely overestimates response since there is wide variability in responses to ICIs between different TMB-high tumor types. We have previously noted this limitation in drug approvals for tissue agnostic indications.

• Third, while this was beyond the scope of our analysis, more approvals are being granted in earlier lines, including in the adjuvant and neoadjuvant setting, which greatly increases the number of people eligible, but the estimates of response are largely unknown, and cannot be assessed in the adjuvant setting by definition, as no disease is visible. The goal of ICI therapy in the metastatic setting is different than in the adjuvant setting, where the tumor has been removed and patients are often cured. Only a few patients benefit, but all patients may experience side-effects, and so many patients may choose to opt out of treatment.

Concluding words

ICI development has greatly altered the landscape of cancer therapy, and many patients have benefited. However, nearly 80% of patients do not respond to them, and 43% are not even eligible to receive them, which indicates that there is still work to be done to determine who benefits most from these therapies, the long-term benefits of them, and how to maximize the benefits, while minimizing side-effects. (Full publication here).

But what if I told you this happens often in clinical trials? And, surprisingly, everyone seems fine with it.

Moving the primary endpoint “target”: one example

The QUANTUM-First trial is a notable example of a clinical trial where the primary endpoint was modified, not just once, but twice. Had the trial adhered to its original endpoint, it would have been considered a negative trial.

In this trial, between September 27, 2016, and August 14, 2019, 539 newly diagnosed patients with acute myeloid leukaemia (FLT3-ITD positive) were enrolled and randomised into two groups to receive either quizartinib or placebo. The primary endpoint was event-free survival (EFS), with overall survival (OS) as one of the secondary endpoints. This meant that OS data would only be analysed if EFS was positive.

The final change occured 1 year after the enrolment of the last patient

On April 7, 2020, the primary endpoint of the study was suddenly changed to dual primary endpoints: EFS and OS.

The rationale given was: "Changing to two primary endpoints of EFS and OS ensured that OS would be analysed regardless of EFS results."

This can be seen as a risk mitigation strategy: if one endpoint failed to show a significant effect, the other might still demonstrate the treatment's benefit, providing a backup measure of success.

After discussing these changes, the FDA indicated that the agency’s definition of EFS was different and stricter. As a result, on October 28, 2020, more than one year after the enrolment of the last patient in the trial, the primary endpoint was changed for a second time—this time to OS as the sole primary endpoint. Remarkably, OS became the only positive endpoint of the study.

Unanswered Questions and Regulatory Approval

In its assessment report, the EMA raises concerns regarding the uncertainties in this study:

“The key secondary endpoint EFS per FDA definition was not statistically significant different between the quizartinib and placebo arms (p = 0.24).” Furthermore, “The difference in overall survival of quizartinib compared to placebo is not supported by other secondary endpoints; rates of CR, CR with FLT3-ITD MRD negativity, and CRc with FLT3-ITD MRD negativity were similar between treatment arms.”

At first glance, you might think, "Does that really matter? Isn’t overall survival the ultimate goal?" And you’re right—OS is indeed the gold standard endpoint. But these other endpoints provide crucial insight into the biological activity of the compound and its ability to control the disease at a deeper, molecular level. The negative results of these secondary endpoints raise questions about how the drug is actually improving survival, as it does not seem to be eradicating the disease more effectively than the placebo.

Several questions remain unanswered,* yet quizartinib has been authorised by both the EMA and FDA since 2023. This approval follows the rejection of the drug earlier in 2019 for the same indication, but as monotherapy in the relapsed or refractory setting, based on a randomised trial comparing it to chemotherapy (EMA, FDA).

Changing primary endpoints happens all too often

The quizartinib case is not unique. A cross-sectional analysis of 755 phase 3 cancer studies found that the primary endpoints were changed in 19% of trials after initiation. Less than a third discussed the changes in the published articles. Not surprisingly, the changes to primary endpoints appeared to be independently associated with trial positivity. Another study by Richters et al., which examined 38 randomised trials involving immunotherapies, found that 63% of the trials changed at least one aspect of their primary endpoint. In 6 out of 38 trials, the assessment of protocols was hindered due to redacted sections related to primary endpoints or the absence of original protocols.

Florez et al. suggest that journals should require the publication of protocols and amendments, as well as mandate the disclosure of any changes to endpoints along with the reasons for those changes, to ensure fairness to trial participants, the public and the competitors who conduct a fair trial.

Just as organisers of a sports match wouldn’t change the rules mid-game, trialists should stick to a validated primary endpoint to ensure fair play and scientifically valid results.

Footnotes:

* Several questions remain unanswered regarding the QUANTUM-First trial. For instance, could the imbalances between the two study arms (e.g., higher rates of discontinuation due to adverse events in the quizartinib group) and/or the differences in other treatments received by patients lead to a survival advantage? These are important questions, especially considering the limited statistical strength of the evidence for overall survival (p=0.03). The EMA notes that “not all subsequent (non-protocol) therapies were recorded for all patients” and that “during follow-up for OS, in addition to receiving already approved AML therapies (Note: Midostaurin received US approval 7 months after the start of QuANTUM-First and EU approval 12 months after) and Hematopoietic Stem Cell Transplantation as subsequent treatment options, patients in the study also moved to a new clinical trial for another investigational drug or received treatments that may not be available to all patients in Europe.”

Global rise in launch prices.

During this time, annualized inflation-adjusted launch prices for new anticancer drugs rose sharply—from $114,000 in 2012 to $256,000 in 2021—representing a compounded annual growth rate (CAGR) of 9.4% (see the figure).

No relation with clinical value.

While factors such as overall survival (OS) or progression-free survival (PFS) were not predictors of price, the study identified the “year of approval” as the strongest factor influencing pricing. This finding suggests that pharmaceutical companies use a price-referencing strategy, setting new drug prices based on those of prior approvals.

This approach challenges the industry’s claim that drug prices reflect clinical value.

To better understand pricing dynamics, our work also explored a wide range of potential predictors, including hazard ratios, response rates, incidence, mortality, trial size, line of therapy, biomarker testing, development time, boxed warnings, and more. Despite this broad approach, the analysis revealed that neither OS nor PFS hazard ratios were statistically significant predictors of drug prices. Again, the drugs “year of approval” emerged as the dominant factor.

Factors associated with higher prices: smaller market share and single-arm trials.

We also found that factors influencing negatively the potential size of the patient population – such as rare disease classification and biomarker testing requirements – were positively associated with higher prices. These findings suggest that pharmaceutical companies compensate for smaller market sizes by setting higher prices.

The study also identified a notable trend: the growing reliance on single-arm trials as the basis for drug approvals. These trials, which lack comparator arms, are unsuited for thorough and reliable evaluations. More than half of the newly approved drugs analyzed in the study were supported by data from single-arm trials. On average, these drugs were priced higher than those assessed through randomized trials with comparator arms.

To address this concern, we recommend that the Centers for Medicare and Medicaid Services (CMS) consider the absence of a comparator arm when determining reimbursement policies.

Conclusion

The study’s findings highlight the significant implications of price-referencing strategies employed by pharmaceutical companies for newly approved anticancer drugs in the U.S. By linking the prices of new anticancer drugs to those of recently approved ones – without regard for their clinical value – these strategies contribute to rising prices that are often disconnected from patient outcomes. This pricing approach risks failing to adequately reward genuine clinical value and may send unclear signals to drug developers, potentially misaligning incentives and deprioritizing innovations that enhance patient care. When prices do not reliably reflect clinical value, the focus on improving patient outcomes can be diminished, ultimately impacting public health. Addressing this issue will require collaborative efforts from regulators, payers, and policymakers. Read our full paper here!

What is the goal of ASCO Featured Voices?

A significant portion of conference discussions occurs on Twitter/X and other social media platforms. To help attendees navigate the 2024 ASCO conference, ASCO identified 25 individuals as “Featured Voices” promoting their Twitter/X handles as key references. The aim was to provide attendees with a curated list of people to follow on social media during the event. ASCO announced these Featured Voices on Instagram, Twitter/X, and LinkedIn, underscoring this intent.

Previous studies (linked here and here) have shown that many prominent oncologists who tweet have conflicts of interest. Our study's originality lies in focusing specifically on the “Featured Voices” – individuals explicitly highlighted by the conference organizers for their social media value during the event.

An in-depth analysis of Tweets from “Features Voices”

In our study out in the Journal of Cancer Policy (freely available here), we abstracted how often the Featured Voices posted about research compared to topics unrelated to research. If they did post about research, we looked at how often they were critical, neutral, or supportive of the research.

We analyzed 810 posts from the 25 Featured Voices during the timeframe of the ASCO 2024 conference and found that 229 (28.3%) were related to research. Notably, the median number of tweets per individual was 20, ranging from a minimum of 1 to a maximum of 104. This raises questions about the effectiveness of the Featured Voices selection, as some individuals barely tweeted, and others did not post a single research-related tweet.

Another important observation was that tweets discussing research largely endorsed the conclusions of the studies presented (65.3%), with criticism being relatively rare, appearing in only 9.3% of the tweets.

Given the inherent difficulty and complexity involved in interpreting oncology clinical trials, often affected by limitations, such level of criticism appears very low, while the level of support seems relatively high. In contrast, other researchers – such as Vinay Prasad, Timothée Olivier, and others – have offered more nuanced insights into many of the conference’s featured studies (see their videos before and after ASCO 2024).

While there could be several factors at play, one possible explanation for the relatively low level of criticism directed at research findings is the networking dynamic of the conference. Attendees may be reluctant to openly criticize research from individuals they know or have just met, as the social nature of the event might discourage overt negativity.

Our analysis may suggest this explanation could have been at play for the Featured Voices Tweets, as our study found that a higher proportion of tweets (34.6%) featured images – primarily of people or selfies – compared to those discussing research findings (28.3%).

This is shown in the figure.

While the conference serves an academic purpose, it also provides opportunities for networking and socializing, making some level of selfies and personal photos understandable. However, the Featured Voices have a distinct role in representing ASCO in an unofficial capacity, which suggests a potential mismatch between their professional role and the nature of some of the shared content.

We concluded: “For a conference that has a mission statement consisting of, `conquering cancer through research, education, and promotion of the highest quality, equitable patient care’, the perceived lack of publications about research and overabundance of photos shows that there is room for improvement”.

How Features Voices are selected?

The current system lacks transparency regarding the selection process for ASCO Featured Voices. Our findings indicated that some of the selected individuals rarely tweet, and most research-related posts either express praise for the research or remain neutral.

A transparent and inclusive selection process for “Featured Voices” could help amplify diverse perspectives, encourage more critical appraisal approaches, and foster richer discussions about research advancements and their implications.

Here is the story...

The AMPHIA trial

Eighteen years ago, around this time, I was having a challenging pregnancy, with concerns about premature birth. During one of the biweekly visits to the hospital, my kind and experienced obstetrician asked me to participate in one of their clinical trials.

The AMPHIA trial was a double-blind study, where women, like me, with a multiple pregnancy were randomised to receive either weekly intramuscular injections of 250 mg 17-hydroxyprogesterone caproate (17‑OHPC) or placebo injections. The idea that progesterone could prevent preterm birth actually dates back decades. In a 1930 study, Allen and Corner demonstrated that corpus luteum extracts from swine, rich in progesterone, could sustain pregnancy in rabbits after early castration.

Back then, I read through the patient information leaflet of the trial and then turned to PubMed, only to find contradictory research and a lack of evidence or reassurance about the safety of progesterone during pregnancy. What worried me most was that several studies providing evidence against the use of progesterone were not cited in the patient information leaflet. I compiled a summary of these studies and sent it to my obstetrician, expressing my concerns that mothers should have access to all available information in order to make truly informed decisions.

I remember, it wasn’t a difficult decision for me. I chose not to take part in the study.

Several negative trials

The AMPHIA trial eventually recruited 671 women with multiple pregnancies between 2006 and 2009 from 55 obstetric clinics in the Netherlands. A total of 1,322 children were born as part of this study. However, the trial turned out to be negative, as 17-OHPC did not prevent neonatal morbidity or preterm birth.

The authors, who published the results in 2011, stated that only 36% of the women who were counselled agreed to participate in the trial. Remarkably, the number of women who came to these obstetric clinics and met the criteria for counselling was at least twice as high. This raises the question of what motivated the obstetricians to present the option of the trial to some women and not others.

One possible reason could be that some obstetricians were not convinced of the efficacy of progesterone, as two randomised studies published in 2007 had already shown that 17-OHPC and vaginal progesterone do not reduce the rate of preterm birth.

FDA Accelerated Approval of 17-OHPC in 2011

In 2003, the National Institute of Child Health and Human Development (NICHD) published the results of a double-blind, placebo-controlled trial involving pregnant women with a history of preterm birth across 19 clinical centres in the United States. A total of 463 women were randomised in a 2:1 ratio to receive weekly injections of 17-OHPC or caster oil as placebo. 17-OHPC resulted in a significant reduction in preterm births: 36.3% compared to 54.9% in the placebo group.

However, there were concerns about these trial results, particularly the higher preterm birth rate in the control group, which was much higher than the national average and even exceeded the authors’ own a priori estimate of 37% preterm birth rate for the control group.

Nevertheless, the American College of Obstetricians and Gynecologists recommended the use of progesterone for women with a history of preterm delivery in their guideline, while also highlighting unresolved issues, including its efficacy in multiple gestations and long-term safety.

A few years later, in 2011, the FDA granted accelerated approval for Makena (hydroxyprogesterone caproate injections) for women with a singleton pregnancy who had a history of spontaneous preterm birth.

The results of the NICHD trial were considered “sufficiently persuasive (p < 0.001) to meet the level of statistical significance generally expected to support approval based on the findings of a single trial.”

This FDA approval caused the cost of an inexpensive drug, with questionable added benefits and unknown long-term safety, to rise 100-fold, from $300 (for the compounded version) to $30,000 for the Makena brand.

Negative confirmatory trial and FDA withdrawal in 2023

The confirmatory study required by the FDA for full market approval, the PROLONG trial, was conducted between 2009 and 2018 and involved 1,130 women receiving 17-OHPC and 578 receiving placebo. The results, published in 2019, showed that 17-OHPC does not decrease preterm birth. However, the FDA took a few more years before withdrawing the accelerated approval in 2023. This delay came at a high cost, as it is reported that only between 2018 to 2021, $700 million of the US healthcare budget (Medicare and Medicaid) was wasted on Makena.

Potentially higher cancer risk and EMA withdrawal in 2024

Sadly, over the course of 12 years, Makena was marketed to a vulnerable population that invested its time, money and hope in a drug that ultimately proved ineffective and potentially harmful. A recent study on a population from Oakland, California, shows that offspring exposed in utero to 17-OHPC (n=234) between 1959 and 1966 had a higher risk of developing cancer compared to those not exposed (n=18,517).

Finally, just a few months ago, on 28 June 2024, more than a year after the FDA withdrawal, the EMA announced that 17-OHPC medicines are to be suspended from the EU market, as studies found no effect in preventing premature birth and raised safety concerns. Their review concluded that “there is a possible but unconfirmed risk of cancer in people exposed to 17-OHPC in the womb.”

Some Thoughts for Your New Year’s Resolutions

The withdrawal of 17-OHPC highlights critical lessons for regulators, medical societies and researchers, underscoring the importance of evidence-based decision-making and patient safety. Here are some thoughts for your New Year's resolutions:

• Regulators: Why not mandate more robust evidence before granting approval, especially for treatments with significant public health implications that could even affect future generations.

• Medical Societies: Same goes for you. Why not recommend therapies only when there is robust evidence to support their use? Please also, ensure that those writing the guidelines have no competing interests.

• Researchers: As Vinay Prasad often says, ask a guiding, ethical question before designing a clinical trial: "Would I recruit my mother or father for this study?" Or, in the context of 17-OHPC, "Would I have recruited my wife, sister, daughter and my unborn child?" This reflective approach ensures trials are ethically designed, scientifically sound and genuinely prioritise patient well-being.

Best wishes for 2025, and much strength and wisdom!

The landscape

We set out to evaluate the collective benefit, in terms of overall survival and progression-free survival, of all approved checkpoint inhibitors. Our work was recently published in the European Journal of Cancer (openly available here).

Of the 103 checkpoint inhibitor approvals, 56 (54.4%) reported overall survival information. Almost half of trials do not report survival data. Of those that do report, the median improvement in survival was a modest 2.8 months. The breakdown of improvement time in survival is shown in the figure.

Sixty-five (63.1%) of trials reported data on progression-free survival, and the median improvement was 0.9 months. Of those reporting data on progression-free survival, 20% had a difference of less than 0, meaning that patients in the control arm had a numerically longer time until tumor progression or death.

Another important finding is that the number of people in the control arm who crossed over to the investigative treatment was high (~20%). Crossover can bias the study results, making it difficult to reliably interpret efficacy findings. Moreover, crossover is only appropriate when the drug has shown to be beneficial when there is tumor progression.

Conclusion

The take-home message from our study show that checkpoint inhibitors have been tremendously valuable for some tumor indications (10-plus months of improved survival for 7% of indications), but for most indications, the improvement in survival is modest (less than 3-month benefit). Excitement for these drugs should be tempered to more realistically reflect the actual benefit from them. Read our full article here.