Improving the Quality of 'Personalized Medicine' Research and Practice

Through an Ethical Lens

George P Browman; Jochen Vollmann; Alice Virani; Jan Schildmann


Personalized Medicine. 2014;11(4):413-423. 

In This Article

Ethics, Quality & Methodological Standards in PM

Proposals for innovative clinical trial designs in PM such as cohort enrichment strategies and adaptive designs for validation of biomarkers in both the laboratory (analytic validity) and clinic (efficacy and clinical utility) have in general respected the traditional evidence-based medicine (EBM) paradigm as the preferred methodological approach.[30–32] However, the feasibility of the evidence-based approach as represented by the randomized controlled trial (RCT) is coming under scrutiny when it comes to evaluating targeted therapeutic interventions in PM.[33–35] Emboldened by the 'comparative effectiveness research' (CER) movement[9] researchers and industry are looking for alternatives to the RCT using large databases and prospective monitoring of patients selected for various therapeutic interventions. Both the WIN consortium[25] and American Society of Clinical Oncology's (ASCO's) CancerLinq© initiative[36] are based on this population-based evaluation paradigm.

These initiatives, which are based on prospective collection and storage of individual data from populations, its aggregation and subsequent analysis to inform clinical policy decisions is appealing. How it differs from the well-worn path of using administrative data and how the limitations of that approach will be overcome needs further explication. Furthermore, it is not clear that this approach, properly executed from a systems perspective, will be any more affordable than RCT programs, although it is likely that costs will be shared more widely among different stakeholders. Finally, it remains unclear at what point in the evaluation journey of new technologies, including drugs, the CER paradigm is intended to be applied. Certainly, population-based CER evaluation methods in the postapproval phase of new technologies can better describe how the technologies perform under 'real-world' circumstances. In addition CER can more efficiently detect adverse events signals from newly introduced technologies. However, whether CER is appropriate to replace the RCT for evaluating the efficacy of new technologies and informing approval decisions is another matter. Furthermore, it is unclear that this evaluation approach would be acceptable to regulatory authorities accountable for approving such new technologies for use in the population at large.

The evidence-based evaluation paradigm has been the cornerstone of informative healthcare policy and practice for decades. From an ethical perspective the principle of nonmaleficence is an important value underlying evidence-based evaluation. It is only by collecting enough and suitable data that we are able to be precise about reporting the benefit:harm ratio of a new intervention to allow stakeholders, including patients, to determine what is acceptable to them. The popularization and widespread adoption of the EBM approach was prompted by the realization that what came before (sometimes known as 'speculative medicine') resulted in too much avoidable harm to too many people. When applied to large populations, the level of harm from premature dissemination of incompletely or inappropriately evaluated interventions could be amplified (for example see[37–39]).

The evidence-based paradigm, simply put, advocates replacement of practice based on what we think we know from our 'understanding' of biological mechanisms, with practice that is based on direct observations of what interventions actually do through experimental testing. The evidence based paradigm is in essence a form of protection from the very real human frailty that leads us to make cause–effect inferences about observed associations, colored by our preconceived notions of how we believe the world (including biological systems) works (bias). Taken to its logical conclusion, the evidence-based paradigm protects us from our 'hubris' (often defined as excessive or over-weaning pride or confidence in our knowledge (i.e., what we think we know).[40] This characteristic is also the source of the biases we unconsciously bring to the design and interpretation of studies. So powerful are these biases that special methods are needed to protect ourselves from them, and ultimately those who will be affected by our decisions and actions. It is this fundamental issue that is responsible for the rather complex framework and set of rules that characterize the rigor of the evidence-based movement.

While the RCT is only one among many different methodological tools of the evidence-based approach,[9] it remains the one most closely identified with EBM. In essence, the RCT creates the conditions for making confident causal inferences between exposures (interventions) and outcomes because within a single methodological framework it has the advantage of addressing all of the established Bradford Hill criteria for causal inference[41] and at a minimum it represents a due diligence approach to evaluating innovations that can produce unintended harm as well as intended benefit on the scale of whole populations.

There are legitimate criticisms of the RCT design, especially where multiple interventions are to be compared within complex environments that include multiple variables that could act as confounders. RCTs are relatively expensive to conduct, and can take a lot of time to yield the desired results. Many who strongly promote PM have heeded these limitations of the RCT, and propose alternative methods for evaluating the performance of predictive or diagnostic tests and therapeutic interventions. Based on the assumption that within the framework of PM it is possible to translate biological concepts into clinical practice it has been argued that the usual cascade of Phase I–III clinical trials required for approval of some new technologies may not be necessary to prove the benefit of certain PM related interventions. This may be especially true when the interventions are informed by a compelling biological rationale based on known rather than speculative mechanisms of action. Among the frequently mentioned arguments against the application of established evaluation standards is that, in light of the multitude of markers that have been detected to be associated with cancer and the limited number of resources that are available for clinical trials to test targeted substances, it will simply not be possible to test all interventions in Phase III clinical trials. A second argument frequently mentioned is that the often small number of patients with a specific biomarker (or biomarker profile) requires changes in trial designs. Instead, it has been suggested that the setup of infrastructures that allow us to gather genetic, clinical and further data on a global level prospectively will accelerate discovery and allow therapeutic choices based on the carefully monitored outcomes of such research and reliable capture of such data.

Scientific rigor is a key element of ethical research.[42] However, the current debate on methodological rigor in the context of PM shows how value judgments underlie the different thresholds for an acceptable, or appropriate evidentiary level. These thresholds differ depending on the perspective, whether from a clinician, a biomedical researcher, the pharmaceutical industry or the body responsible for making decisions about drug approval and/or their funding. And, of course, pressures on decision bodies for approval of new drugs come from patients, whose threshold for acceptable rigor will be different, depending on their own circumstances.

Factors relevant here are, for example, the positions held regarding the required level of efficiency of mechanisms which shall translate findings of basic research into clinical practice, the concept of clinical utility we hold or thresholds regarding the risk of harm we accept when approving a drug.[43]

While there are legitimate criticisms of the RCT design, there is a risk in replacing the established RCT design in situations where it is a feasible method for evaluating interventions. If the RCT is to be replaced by other methods, we argue that it is necessary to make explicit the ethical implications of the choice of different methodological approaches with regards to risks to validity, benefits and harms for patients, the resources which are needed to implement the different strategies as well as the interests of the players concerned. It would also be worthwhile to know what industry partners, who are making considerable investments in shared 'big data' initiatives[44] expect from this approach in terms of either better understanding the adverse events of their products, and/or, as a legitimate strategy for accelerating drug approval decisions and access to markets.