Transportability of Comparative Effectiveness Evidence Across Countries – Healthcare Economist

Let’s say you have an international clinical trial that shows that a new drug (SuperDrug) works better than the previous standard of care (OldDrug). Let’s also assume that individuals with a specific comorbidity (let’s call it EF) respond less well to SuperDrug treatment. If you live in a country where comorbid EF is common, how well do you think SuperDrug will work in your population?

This is the question posed by Turner et al. (2023) in his recent Pharmacoeconomics paper. The general problem faced by decision makers in the countries is the following:

When study populations are not randomly selected from a target population, external validity is more uncertain and the distributions of effect modifiers (characteristics that predict variation in treatment effects) are likely to differ across the trial sample. and the target population.

Many of you may have guessed that my EF comorbidity actually means an effect modifier. Four classes of effect modifiers that the authors consider include:

• Patient/disease characteristics (e.g. prevalence of biomarkers),
• Environment (e.g. location and access to care),
• Treatment (e.g., timing, dosage, comparative therapies, concomitant medications)
• Results (for example, monitoring or
• time of measurements)

See Beal et al. (2022) for a potential checklist for effect modifiers.

In their article, the authors examine the problem of transportability. What is transportability?

While generalizability relates to whether a study’s inferences can be extended to a target population from which the study’s data set was sampled, transportability relates to whether
Inferences can be extended to a separate (external) population from which the study sample was not derived.

Key differences between countries that can make transportability problematic include effect modifiers.
such as disease characteristics, comparative therapies, and treatment settings.

What is the problem of interest?

Typically, decision makers are interested in the target population average treatment effect (PATE): the average treatment effect if all individuals in the target population were assigned the treatment. However, researchers commonly have access to only a sample and must estimate the study sample average treatment effect (SATE).

Key assumptions for estimating PATE are included below:

Primarily, there are two key elements to address (at least for RCTs): (i) are there differences in the distributions of characteristics between the study and the population of the target country/geography and (ii) are these characteristics effect modifiers? [or for single arm trials with external controls, prognostic factors].

Differences in the distribution of covariates can be tested using mean differences in propensity scores, examining propensity score distributions, as well as formal diagnostic tests to identify the absence of overlap. Univariate standardized mean differences (and relevant tests) can then be used to examine the determinants of the overall differences. If only aggregate data are available, one can simply compare differences in mean values.

To test whether a variable is an effect modifier, the authors recommend the following approaches:

Parametric models with treatment-covariate interactions can be used to detect effect modification. When small study samples result in power problems or when functions are unknown
shapes increase the risk of model misspecification, machine learning techniques such as Bayesian additive regression trees and the use of directed acyclic methods could be considered.
Graphics can be particularly crucial for selecting effect modifiers in this case.

Approaches to adjusting effect modifiers vary depending on whether an investigation has access to individual patient data.

• With DPI: Use methods based on outcome regression, matching, stratification, inverse probability of participation weighting, and doubly robust methods that combine matching/weighting with regression adjustment.
• No DPI. Use population-adjusted indirect treatment comparisons (e.g., matching-adjusted indirect comparisons).

To determine which country data (typically real-world data) should be used as the target population, a variety of tools could be considered such as EUnetHTA APPLICATION o Data Adequacy Assessment
Tool (SAT Data) NICE tool.

You can read more recommendations on how to better validate transportability issues in the full document. here.