In a letter to FDA published in the April 13, 2007, Cancer Letter, Howard Scher of Memorial Sloan-Kettering Cancer Center presented valid and compelling arguments that FDA await the completion of an ongoing 500 patient (9902B) Phase 3 trial before deciding whether to approve sipuleucel-T in prostate cancer patients. Reportedly, Scher felt motivated to write the letter after being kept awake the night following the March 29, 2007, FDA Cellular, Tissue and Gene Therapies Advisory Committee by the thought that if sipuleucel-T were approved, patients may well forego more effective treatment alternatives. He also struggled with what he might communicate to patients about sipuleucel-T’s safety and efﬁcacy when discussing therapeutic options with them.
I also was kept awake the night following the panel. I had been invited by FDA to be screened to serve on the March 29, 2007, FDA Advisory Committee, but declined because I had had limited interactions with the sponsor in the capacity of critiquing available data. Now that the FDA Clinical and Statistical Brieﬁng Documents are in the public domain, I am at liberty to express my own serious concerns about some of the signiﬁcant ﬂaws and limitations in the 9901 and 9902A clinical trials evaluating Sipuleucel-T in prostate cancer patients.
As noted by Scher, the 9901 and 9902A trials provide evidence that the effect of sipuleucel-T on the pre-speciﬁed primary endpoint, progression-free survival, was 1-2 weeks, far less than the 15 week improvement targeted in the 9901 protocol. Therefore, not only did the trials fail to achieve statistically persuasive evidence for beneﬁt, the estimates of effect on that measure indicate that clinically meaningful effects were not achieved. The 9901 trial also failed to establish beneﬁt on measures of pain or other prespeciﬁed secondary endpoints.
Major concerns arise when interpreting the survival data from the 9901 and 9902A trials. Overall survival was not a primary or secondary endpoint in 9901 (speciﬁcally, only a “descriptive” analysis of overall survival was to be performed), and also was not the pre-specified primary endpoint in 9902A.
The concerns regarding the unreliability of post-hoc analyses are far more profound than that they simply provide a violation of statistical “rules”, as one might believe from comments by the sponsor’s consulting biostatistician, Brent Blumenstein, (see O’Neill RT, “Secondary Endpoints Cannot be Validly Analyzed if the Primary Endpoint Does Not Demonstrate Clear Statistical Signiﬁcance.” Controlled Clinical Trials 18:550-556, 1997). Estimates of effect of sipuleucel-T on overall survival are biased and p-values reported from such analyses convey a false sense of reliability of that evidence. An explanation for this bias was presented in a recent article discussing why proper adjustments must be made when multiple testing arises over the course of the trial, (Fleming et. al., “Maintaining Conﬁdentiality of Interim Data to Enhance Trial Integrity and Credibility.” Annals of Internal Medicine, under review). That article states:
“This bias (a form of “regression to the mean” bias) occurs because there is true signal and random noise in every estimate of treatment effect and, when many analyses are conducted, there is a tendency for those results that appear to be most favorable to be, at least in part, due to random overestimates of true effect”.
The risk for “regression to the mean” bias is very substantial in the reported estimates of the survival effect in the Sipuleucel-T trials. A clear illustration of this bias is provided by the recent experiences from the GIPF-001 and the GIPF-007 trials conducted by InterMune to evaluate Actimmune in patients with idiopathic pulmonary ﬁbrosis (IPF). Like Dendreon, InterMune conducted exploratory analyses after their primary analysis of GIPF-001 established Actimmune did not provide a beneﬁcial effect on the primary endpoint (relating to pulmonary function). When a survival advantage (2-sided p=0.004) was found in patients with mild to moderate impairment in lung function, the sponsor provided a press release indicating “The mortality beneﬁt is very compelling and represents a major breakthrough in this difficult disease.”
Fortunately, the sponsor eventually recognized that their post-hoc analyses of overall survival did not provide reliable evidence of beneﬁt and conducted GIPF-007, a conﬁrmatory trial in 826 IPF patients having mild to moderate impairment in lung function, precisely the same population in which beneﬁt was suggested by the post-hoc survival analysis of the GIPF-001 trial.
The GIPF-007 trial (called INSPIRE) was recently terminated since, according to the sponsor’s March 5, 2007 press release, “the DMC found the overall survival result crossed a predeﬁned stopping boundary for lack of beneﬁt of Actimmune relative to placebo” and where overall mortality was “14.5% in the Actimmune group as compared to 12.7% in the placebo group.” Many parallels between this setting and Dendreon’s evaluation of Sipuleucel-T strongly illustrate the need to await the results of Dendreon’s 9902B trial.
Important concerns with the sponsor’s covariate adjusted survival analyses of the 9902A trial also should be highlighted. The covariate analysis in 9902A that changed the two-sided from p = 0.33 to p<0.05 was invalid in that the reported covariate analysis not only provided the intended adjustment for potential confounding, but also inappropriately excluded 10% of study patients, where the patients excluded from the Sipuleucel-T arm had less favorable survival and those excluded from the placebo arm had more favorable survival, as illustrated by the FDA Statistical Brieﬁng Document.
FDA should bring consistent scientiﬁc and ethical standards to the oversight and evaluation of clinical research much like a court of law should bring consistent standards to legal justice. FDA approval of Sipuleucel-T would set an unfortunate precedent for accepting lack of rigor, including giving undue credence to post-hoc analyses that very likely reﬂect misleading estimates of efﬁcacy due to regression to the mean-type bias, and to invalid analyses, such as the covariate adjustment of the 9902A trial that inappropriately excluded many patients who did not have missing outcome data. Furthermore, in light of FDA’s recent consideration of DN101 in prostate cancer that is discussed in Scher’s letter to FDA, how would one defend internal consistency at FDA if Sipuleucel-T were to be approved before availability of the 9902B trial? Like Dendreon, Novacea had obtained a two-sided p<0.05 in supportive analyses of survival in their ASCENT1 trial evaluating DN101 in 250 prostate cancer patients. Extensive available data from ASCENT1 and other investigations of vitamin D also suggest a potential additional beneﬁcial mechanism of DN101 through reduction in the risk of thromboembolic events, (Venner, ASCO, 2006). Nevertheless, ODAC and FDA have recognized the need for Novacea to conduct a 900 patient trial to conﬁrm effects of DN101 on overall survival in prostate cancer patients.
Issues of safety and ethics also deserve further discussion. In clinical trials, sipuleucel-T has nearly three-fold higher rate of cerebrovascular events (17/345 on sipuleucel-T versus only 3/172 on placebo patients).
Furthermore, sample sizes in the completed trials are too small to rule out that other important risks exist. In the absence of established beneﬁt, sipuleucel-T may readily provide more harm than beneﬁt. Hence, one should reexamine the reasoning by FDA Advisory Committee member, Francesco Marincola. He supported approval of Sipuleucel-T by stating:
“Even if we make a mistake, even if the [therapy] is not this effective, there is so much to learn by starting to see patients being treated with this and see what else can be added. We should not underestimate the importance of this decision. I don’t think it’s just about the drug and what the drug does, but it’s about opening a ﬁeld, and the investigation on that ﬁeld.”
One does not need marketing approval in order to continue clinical research studies evaluating sipuleucel-T Marincola’s position is tantamount to advocating that regulatory approval be provided for interventions that have not been established to provide a favorable beneﬁtto-risk proﬁle, in order to enable a sponsor to market potentially ineffective and even harmful products to patients, without a requirement for obtaining informed consent, in order to further investigation in the ﬁeld.
Such use of patients for research purposes without obtaining full informed consent is illegal as well as unethical. Such practice would be in direct violation of federal law, (45 CFR 46.116 and 21 CFR 601.25(d)(2) and (3)).
I do not know whether sipuleucel-T in truth has a favorable beneﬁt-to-risk proﬁle. The current data are inadequate to make a reliable assessment. The 9901 and 9902A trials do not provide “substantial evidence of efﬁcacy.” Rather, at best, these trials provide plausibility of efficacy that would justify the conduct of a confirmatory survival trial. That trial (9902B) is well underway. If there is a pre-mature approval of Sipuleucel-T by FDA, how would the Agency proceed in the likely scenario that the 9902B trial, when completed, would indicate that sipuleucel-T does not provide survival beneﬁt, as recently happened in the similar situation with Actimmune in the IPF setting? Or what if a pre-mature approval of sipuleucel-T by FDA compromises the ability or commitment of the sponsor to successfully complete the 9902B trial? The patient advocate on the Advisory Committee, Robert Samuels, stated;
“I look upon (sipuleucel-T) as an opportunity for me to make a choice. That’s all the patients want: an opportunity to make a choice.”
As a fellow person living with prostate cancer, I strongly disagree with his statement that all patients want is a “choice.” Patients want an “informed choice.” How then would pre-mature approval of sipuleucel-T that could diminish the likelihood of obtaining reliable results from the 9902B trial be in the best interests of prostate cancer patients?
Dr. Thomas R. Fleming, Ph.D. is Professor and Chair of Biostatistics and Statistics in the School of Public Health and Community Medicine, University of Washington, Seattle, WA. Fleming's research and teaching interests are in survival analysis, sequential analysis, cancer clinical trials, AIDS research, and methods for design, conduct and analysis of clinical trials. He has been invited to give the Bernard G. Greenberg Distinguished Lecture Series at University of North Carolina School of Public Health May 9-11. He has received the Outstanding Teaching Award from the School of Public Health and the FDA Commissioner's Special Citation Award for Extraordinary Contributions to the Agency. Dr. Fleming served as an advisor to the FDA for 18 years and has been a voting member of numerous advisory committees for the Center for Drug Evaluation Research and the Center for Biologics Evaluation Research. Since 1987, Dr. Fleming has been a member of the DAIDS Data Monitoring Committee that oversees trials conducted by ACTG and CPCRA, and has served as the chair or a member of data monitoring committees for over 100 industry- and government-sponsored clinical trials
General Disclaimer: PSA Rising is designed for informational purposes only and is not engaged in rendering medical advice or professional services. News and information provided through PSA Rising should not be used for diagnosing or treating a health problem or a disease. It is not a substitute for professional care. If you have or suspect you may have a health problem, consult your healthcare provider.Last updated May 5, 2007