Kelly: When and how should manufacturers integrate economic evaluation into their clinical development programs?
The Top-Line Overview
Steve: This is a pertinent question important for manufacturers to be considering throughout the entire spectrum of their clinical development program as to when to do what in terms of economic evaluation. It’s not an easy question for me to answer for any particular manufacturer because it will certainly vary by disease, the type of product being brought to market, and the reimbursement and competitive landscape. That's one way to say “it will vary.” But I think we can outline a very top-line overview of one possible way to think about the sequencing and possible research steps along the clinical development pathway. After, I'll describe a more specific example of a type of economic evaluation and when and how that type of evaluation can be integrated.
Starting in late phase one, I recommend that clients at least have done some research on the possible economics of the disease or condition of interest. Having an understanding, and there might be published evidence to assist with this, of the economics of disease and the cost of illness will assist with considering how the asset’s clinical advantages or disadvantages might translate into the economic picture. Also, as manufacturers are starting to think about moving into phase two studies, it will behoove them to start to plan for and conduct more meaningful research that either directly or indirectly relates to economic evaluation. Those could be claims analyses, literature reviews, etc. Again, that can be to inform work that will be happening downstream or a standalone economic evaluation. In addition, in phase two, I encourage my clients to be thinking about whether cost-effectiveness analysis will be something that they're going to need to pursue to demonstrate the value of their product. And if they determine that's something they're going to need as part of their pricing, reimbursement, and/or marketing efforts, then phase two is the time to start conceptualizing those projects--especially as some products now are being approved based on phase two trial data alone. In those circumstances, you certainly have to be thinking about this, and implementing that kind of research, in phase two. Also in phase two, some early conceptual economic modeling could be explored.
As phase three trials are initiated, this is another time point at which manufacturers will be doing some aspects of economic evaluation. How much time you have during that phase three trial program will vary, especially given the disease. If it's an acute disease, you may have only a year or two as you conduct phase three trials, or for oncology or chronic disease, the runway will be longer to be working on economic studies. Those timeline considerations certainly can dictate the trajectory and aggressiveness with which you pursue this type of research; but phase three is the time to get more serious about developing a cost-effectiveness analysis. Late in phase three also is typically when clients then develop budget impact models or analyses, especially as work commences on product dossiers for payers.
That's all a very top line overview and certainly, there are several reasons why that kind of timing and trajectory would be modified. But I think it's a reasonably good example. The take-home message is that I encourage manufacturers to think about the economic value story throughout the clinical development program.
An Example – Economic Evaluation Alongside A Clinical Trial
As I previously alluded to, there's one particular example that I think we can delve into a bit deeper because it does require a lot more consideration and planning. It's an approach that isn’t suitable for every product, but it is something that clients need to be aware of and be thinking about—that is, a clinical economic trial or an economic evaluation conducted as a component of a trial. I'd like to walk through some of my thoughts about that type of research endeavor.
Clinical development and trials, in particular, are designed to better understand the efficacy and safety of a manufacturer's product. In most cases that means comparing the investigational product to placebo or some specific standard of care. But sometimes there is no comparator as single-arm trials are not that unusual (although certainly not optimal). Although there can be trade-offs with clinical trials, such as their somewhat artificial nature, having already created the trial infrastructure, it does beg the question of whether that should be tapped into as a vehicle for either additional data collection or possibly as the framework for a standalone economic evaluation itself.
Before discussing the when’s and the how’s of your original question, I think we'll just touch on a few of the pros and cons posed by economic evaluation alongside a clinical trial. This will help convey some of the advantages of integrating economic evaluation and data collection into a clinical trial program, but also some of the challenges and pitfalls.
First the advantages:
Trials, if designed correctly, tend to provide an opportunity for high internal validity and possibly reasonable external validity. So that is always an advantage, by virtue of their design.
Clinical economic trials can provide timely information for pricing and reimbursement decisions, especially in jurisdictions that require economic evaluation. Where pricing and reimbursement aren't dependent on a cost-effectiveness analysis, a clinical economic trial still does afford the possibility to collect data that can inform budget impact or other economic analyses that may be important for market access or product adoption. That benefit shouldn't be underestimated as, often, the critique of economic analysis, especially models, is the lack of robust data to inform them. So, having collected a lot of that information in a clinical trial can help to address that critique.
Finally, there is an advantage especially related to rare diseases. Assembling a cohort of patients for a phase three trial in a rare disease is not straightforward. In some ways, I see it as almost a scientific duty to collect as much data as possible from that cohort. Manufacturers in that rare disease space need to be thinking, “Are there opportunities for me to expand my data collection that could inform an economic evaluation, whether stand-alone or post hoc?”
Those are some of the advantages. On the other hand, the challenges also need to be mentioned for balance.
Certainly, costs are always going to be an issue associated with data collection. The trial infrastructure provides some efficiencies, but there will be additional investigator and site personnel time, which will add cost to research budgets that may already be quite constrained. Also, during and after the trial there will be personnel that will manage and use that data, and their time also is usually constrained.
As mentioned earlier, to have reasonable external validity, the trial design needs to consider several factors that threaten generalizability. For example, the artificially increased compliance that comes along with trials, possible protocol-driven resources use, and the potential for homogenous patient populations, both in terms of their clinical and demographic characteristics, are all challenges inherent in clinical trial design. If accommodations cannot be made in the trial design, then the effort of conducting a parallel economic evaluation may not be warranted because the results will be insufficiently generalizable. This is a reason that an economic investigator should be tasked early on with helping guide trial design, data collection, and other key elements of a clinical economic trial.
Another challenge is that, in many diseases, there can be multiple relevant comparators and a typical trial would be comparing the investigational product usually to only one other treatment and often to placebo. In reality, there are usually multiple treatment options for a specific disease and this will hamper the credibility or utility of an economic evaluation conducted alongside a clinical trial when, at best, you can usually only compare to one other product.
Finally, I'll mention trial sample size and power, which are calculated for clinical endpoints, and typically would not be revised for economic outcomes. So it is a usual expectation that the analysis of economic outcomes alongside a clinical trial will be underpowered and not necessarily be expected to yield statistically significant differences.
Those are some preliminary thoughts on the pros and cons. I think the 2015 ISPOR Good Research Practices Task Force does a nice job of laying out a lot more detail about this.
In terms of the timing for this type of research, typically data collection related to either a post-hoc economic evaluation or for a standalone economic evaluation would be considered in a phase three trial--that certainly would be most common. But because approvals of some therapies are being made based on phase two data, it is conceivable that, in some circumstances, you would conduct an economic evaluation alongside a phase two trial. Possible collection of exploratory data in phase two, especially if you're thinking about a phase three economic evaluation, may be an advantageous approach.
However, earlier is not always better. You would not want to consider capturing that type of data in dosing studies or studies that require a design that's too artificial to be representative. So in terms of timing, I think phase two or phase three is when you need to be thinking about this design. Just as important as timing is under what circumstances a manufacturer would potentially conduct a clinical economic evaluation. One consideration goes back to how many relevant treatment comparators exist. If there are one or two or no other truly effective therapies for a disease, then the clinical trial infrastructure might be more likely to yield a meaningful representation of clinical practice than when there are four or five therapies available. The availability of few competitors might signal a good candidate for a clinical economic trial. Also, when a therapy is hypothesized to have a substantial quality of life impact, then use of those tools or measures, especially if they can be used to calculate quality-adjusted life years, for collection of that data needs to be seriously considered as part of the clinical economic trial.
Those are a few examples of when you might consider a clinical economic trial. In terms of how to collect the data and then how to use the data, I'll just make a few points here. First, you need to consider what to collect, which will or should be driven largely by what is expected to differ between treatment arms. It's not feasible to collect everything, so some time and effort needs to be devoted beforehand to determine what's important and what's feasible to collect whether it's certain aspects of medical resource use, quality of life, or preference-based data, and the timing of when to collect it during the trial. All of that needs to be thought through; the mechanics of collecting the information are a bit more straightforward.
Given that the trial will be collecting clinical information as part of its design, quality of life or other patient-reported outcome measures can be administered as part of the in-person visits or possibly as remote evaluation. Medical resource use can be collected via a clinical database or a case report form or an electronic medical record. Occasionally, manufacturers will attempt acquisition of cost data during a trial. My experience has been that that's usually a luxury and that that data tends to be closely guarded information at the site level and, ultimately, is subject to a lot of variability anyway. So direct acquisition of cost data is not something I recommend.
Finally, a word about how to use the data. Just as I recommend that there needs to be advanced planning and research conducted on what to collect; there also needs to be consideration regarding how the collected data are going to be used. Are they going to be analyzed in a standalone cost-effectiveness analysis or are they going to simply inform future economic modeling efforts? For the former, a standalone cost-effectiveness analysis, then the client needs to be utilizing good research methods and practices, similar to how the clinical trial data would be treated, in order to maximize the robustness and credibility of any economic analysis that they ultimately perform. That should include a predefined analysis plan, an intention to treat analysis approach, and designation of how to handle missing data. For the latter, when the data are intended to be used in a somewhat less rigorous fashion, for example in post-hoc economic models, then less attention may be needed for extensive analysis planning.
Summary
To summarize: economic evaluation and research can be considered along the continuum of the clinical development program. More specifically, when considering an economic evaluation within a clinical trial, I'd reiterate that a manufacturer needs to invest time, energy, and funding up front to analyze whether a clinical economic trial is the right research vehicle. Do your due diligence!
Also, keep in mind the strengths and limitations related to the clinical trial design and the impact that may have on your economic evaluation. In addition to a key clinical investigator, there needs to be a researcher designated to spearhead the economic evaluation component and that person or team needs to be brought in early enough to have some input on design and data collection. Finally, carefully consider the right time to conduct the study. Is there a possibility for some exploratory work earlier on, for example, in a phase two study? Could that benefit the full clinical economic trial later?
About the Steve Duff
I have spent nearly 25 years providing health economic and reimbursement consulting services to pharmaceutical, biotechnology, medical device, and diagnostic companies. I provide to my clients a unique combination of health economics expertise with clinical knowledge and product development experience gained through various positions in the consulting and pharmaceutical industries.
Prior to founding my own consulting practice, I spent eight years as a principal consultant with Covance Health Economics and Outcomes Services where I focused on medical technology assessment, economic modeling, and development of dossiers, manuscripts, and strategic plans. My clients ranged from small start-ups to Fortune 500 companies with technologies in various stages of development and commercialization.
In addition to my consulting experience, I also have held various positions in pharmaceutical research and clinical development. I spent seven years in research and development at Kendall McGaw and Allergan, primarily in the field of pharmacokinetics.
Over the course of my 25-year career providing health economics consulting services, I have been privileged to assist hundreds of clients achieve their market access and commercialization goals. Examples of these accomplishments include:
Conceptualization and programming of sales and pricing models for dozens of pharmaceutical, device, and diagnostic products to inform successful market adoption strategies;
Development and publication of the first cost-utility analysis of robotic surgery and six comparator treatments of prostate cancer informed by an extensive literature review and synthesis;
Supporting a cutting-edge ophthalmology product for over a decade with budget impact and cost-effectiveness models, dossiers, publications, and literature reviews across five approved indications; and
Development of several economic models used to support successful listing recommendations by Australia’s Pharmaceutical Benefits Advisory Committee (PBAC).
I have a Bachelor’s Degree in Biology from the University of California, San Diego and a Master’s Degree in Health Policy and Management from the Harvard University School of Public Health.