Predicting poor peripheral blood stem cell collection in patients with multiple myeloma receiving pre-transplant induction therapy with novel agents and mobilized with cyclophosphamide plus granulocyte-colony stimulating factor: results from a Gruppo Italiano Malattie EMatologiche dell’Adulto Multiple Myeloma Working Party study
Stem Cell Research & Therapyvolume 6, Article number: 64 (2015)
A still not well defined proportion of patients with multiple myeloma (MM) and eligible for autologous stem cell transplantation (AuSCT) fails to mobilize CD34+ peripheral blood stem cells (PBSC) at all or to collect an adequate number for a safe procedure or sufficient for multiple transplants. These so-called “poor-mobilizers” are difficult to be predicted, due to marked difference across previous heterogeneous studies.
We aimed to develop a method based on simple clinical parameters for predicting unsuccessful (<2 × 106/kg) or sub-optimal (<5 × 106/kg) collections of CD34+ PBSC in newly diagnosed MM patients eligible for AuSCT, treated with novel agents and receiving an homogeneous mobilizing therapy with cyclophosphamide and granulocyte-colony stimulating factor (G-CSF). To this purpose, 1,348 patients enrolled in five consecutive Italian clinical trials were retrospectively analysed. Age, baseline low peripheral blood cell counts, use of lenalidomide, and haematological toxicity developed during induction were taken into account as possible factors associated with poor mobilization.
Overall, 280 patients (20.8%) showed either sub-optimal (167 patients, 12.4%) or unsuccessful (113 patients, 8.4%) collections. All analysed parameters negatively influenced the procedure, but only age and haematological toxicity during induction maintained their significance at multivariate analysis. Based on ordinal logistic regression model, we constructed a risk heat-map where the four parameters were pooled and weighted according to their relevance as single or combined variables. This model was predictive for different probabilities of failure, suboptimal or optimal outcomes.
We found that about one fifth of newly diagnosed MM fails to collect an adequate number of PBSC. Our model, based on a large group of patients treated frontline with novel agents and receiving the most popular mobilizing approach currently employed in Europe, is applicable in individual subjects and may contribute to the early identification of “poor mobilizer” phenotypes.
Multiple myeloma (MM) is a neoplastic plasma cells disorder accounting for approximately 13% of all hematologic malignancies . In Europe, melphalan-prednisone (MP) plus thalidomide (MPT) or bortezomib (MPV) represent the initial standards of care for elderly MM patients [2,3], while induction therapy including novel agents followed by autologous stem cell transplantation (AuSCT) is the gold standard for frontline treatment of younger, transplant-eligible subjects [3,4]. Notwithstanding, a variable proportion of these last patients fails to mobilize CD34+ peripheral blood stem cells (PBSC) at all, or to collect an adequate number of these cells for a safe AuSCT or sufficient for additional transplants [5-7]. The percentage of these ‘poor mobilizers’, however, differs across studies, depending on definitions, parameters utilized to evaluate collections, age, disease type, phase and characteristics, treatments applied, objectives to reach, and practices for mobilization and apheresis [8,9]. Due to such heterogeneity, data are difficult to analyze and to compare.
This study aimed to evaluate the rate of unsatisfactory CD34+ PBSC collections and to investigate the possible role of some easily available clinical parameters in predicting this phenomenon in MM patients eligible for AuSCT, treated at diagnosis with novel agents and homogeneously mobilized with cyclophosphamide and granulocyte-colony stimulating factor (G-CSF), which is the principal approach currently employed in Europe.
Overall, 1,348 newly diagnosed patients with MM enrolled in five consecutive clinical trials conducted by GIMEMA/Multiple Myeloma Italian Network were retrospectively evaluated [10-14]. According to the different study protocols, induction regimens consisted of: 1) thalidomide + dexamethasone (TD: 316 patients; clinical trial number NTC01341262 – clinicaltrials.gov, registered 11 January 2004); 2) bortezomib, thalidomide and dexamethasone (VTD: 258 patients; clinical trial number NCT01134484 – clinicaltrials.gov, registered 28 May 2010); 3) lenalidomide and dexamethasone (RD, 396 patients; clinical trial numbr NCT00551928 – clinicaltrials.gov, registered 30 October 2007); 4) pegylated liposomal doxorubicin, bortezomib, and dexamethasone (PAD, 86 patients; clinical trial Eudract number 2005-004714-32 – AIFA, registered 30 September 2005); 5) a further group of 292 patients receiving a modified VAD induction regimen (doxorubicin, vincristine and dexamethasone), not including novel agents (clinical trial number AOGIBAT-III-2004-002, registered 07 May 2004) was also evaluated. In all these patients the impact on CD34+ PBSC collection of: 1) age (>60 years), 2) initial use of lenalidomide (up to four cycles), 3) cytopenia at diagnosis (Hb <10 g/dl, neutrophil count <1 × 109/L, platelet count <100 × 109/L: at least one), and 4) grade 3 to 4 hematological toxicity during induction therapy (CTCAE v4) was analyzed. All these parameters are generally believed potentially able to negatively affect releasing of PBSC and to impair their collection in MM [15-18]. Initially, we also considered extensive radiotherapy to marrow bearing tissue, which was, however, excluded in the final analysis, as the number of patients who underwent such a treatment was very small and almost entirely comprised within the group developing cytopenia during induction.
In all cases, the mobilizing regimen was cyclophosphamide (3 to 4 g/sqm) + G-CSF (10 mcg/kg), given in two daily divided doses, without the addition of the mobilizing agent plerixafor, a drug still not available at the time of the evaluated trials. The morning dose of G-CSF was omitted on the harvest day. An absolute CD34+ PBSC number >20/μl was the threshold to start apheresis. Due to the retrospective nature of the study, which included patients enrolled over a prolonged period of time, apheresis machines, characteristics and methods of collection were progressively adjusted, according to available standards. In most patients, however, the amount of blood processed was twice the calculated total blood volume, by using Cobe Spectra or Fresenius AS series machines, and the target for CD34+ PBSC harvest was at least 5 × 106/kg. CD34+ PBSC total amounts after a single mobilization procedure <2 × 106/kg (the minimum target cell dose usually required to proceed to AuSCT) and >2/<5 × 106/kg (a cell dose ensuring a safe and rapid marrow recovery, as well as the possibility to perform at least two AuSCT) were considered ‘failures’ or ‘sub-optimal’ results, respectively. Data were collected from two databases (Turin and Bologna). No formal ethical approval and/or consent form were needed for this study, according to current Italian law, as it was a retrospective, observational study, which was, however, performed according to Helsinki Declaration principles for experimental research on humans.
All risk factors were analyzed by univariate and multivariate logistic regression, taking into consideration two models having as outcome the risk to have a ‘failure’ or that of a ‘sub-optimal’ collection. The results were internally validated using the bootstrap method. In addition, receiver operating characteristic (ROC) curves were constructed to assess model discriminatory power for the predictive probability. The proportional odds assumption was evaluated by the Brant test. Finally, predicted probabilities of outcomes, based on the ordinal logistic regression model (optimal versus suboptimal versus failure), were used to generate a patient-based risk heat-map where the four parameters were pooled and weighted according to their relevance as single or combined variables. Statistical analysis was performed using the STATA analysis program, version 11.0 (Stata Corp., College Station, TX, USA).
The distribution of analyzed parameters in the whole population is reported in Table 1. Among the 1,348 patients evaluated, 402 (29.8%) had no ‘negative’ parameters, while 946 (70.2%) showed at least one of them; in particular, 630 patients (46.7%) had only one parameter, while 319 patients showed a combination of two (252, 18.7%), three (54, 4.0%) or four (10, 0.7%) parameters, respectively. Overall, 560 patients (41.5%) were more than 60 years old, 332 patients (24.6%) had baseline cytopenia, 356 patients (26.4%) were treated with lenalidomide and 88 patients (6.5%) developed grade 3 or 4 hematological toxicity under induction therapy.
After a single mobilizing procedure (median of leukapheresis: 2; range: 0 to 4), 280 patients (20.8%) collected an insufficient number of CD34+ PBSC, including 167 patients (12.4%) with unsuccessful and 113 patients (8.4%) with sub-optimal collections, respectively (Table 2). An ordinal logistic regression model showed that each single parameter negatively influenced collections at univariate analysis (Table 3); at multivariate analysis, however, only hematological toxicity developed during induction and age ≥60 years maintained a statistically significant negative effect on mobilization (Table 3). Data regarding the effect of other induction regimens on PBSC collection indicated that the use of agents such as bortezomib, thalidomide and doxorubicin did not negatively influence PBSC collections (data not shown). Figure 1a illustrates the risk heat-map obtained based on an ordinal logistic regression model. According to the cumulative probability of unsuccessful collections (failures + suboptimal), four different areas were identified, respectively, at low (range 14% to 18%), intermediate-1 (21% to 30%), intermediate-2 (39% to 46%), and high (50% to 63%) risk. In particular, the possibility of a complete failure ranged from 9% for lowest-risk, to 40% for highest-risk patients. Interestingly, the number of parameters also paralleled the risk of unsatisfactory collections in 1,203 patients (89.24% of the entire population) reaching 20/μl (or more) circulating CD34+ PBSC at the time of apheresis (Figure 1b). All these findings did not differ significantly when the 294 patients who had not received novel agents were excluded from the analysis (data not shown). Finally, ROC curves, based on the logistic regression model (Additional file 1), depicted an area under the curve (AUC) of 0.63 and 0.62, respectively, in discriminating patients with failure or sub-optimal outcome (Additional file 2).
An insufficient availability of PBSC may be a relevant clinical problem in patients otherwise eligible for AuSCT, a procedure that represents the best therapeutic approach in MM. We quantified this phenomenon, showing that a significant proportion of previously untreated MM patients (at least one out of five) completely fails CD34+ PBSC collection or does not reach an adequate number after a standard mobilizing procedure. Our results also show that older age, baseline cytopenia and severe hematological toxicity during the induction treatment (which likely suggest a more compromised marrow function) may be useful to highlight patients at risk of an impaired release of PBSC in newly diagnosed MM. Of interest, in our series, some patients with hematological toxicities due to induction treatment also showed one or more cytopenias at the time of diagnosis, before any treatment; both of these parameters had singly a negative impact on harvest at univariate analysis. However, when the results were investigated in multivariable analysis using two different assays (ordinal outcome and logistic regression), only hematopoietic toxicity remained statistically significant, while low baseline peripheral blood cell counts did not. The use of lenalidomide has also been considered a possible risk factor, although patients receiving lenalidomide-based regimens can be mobilized with appropriate strategies . In particular, a short course of lenalidomide did not affect PBSC harvest in patients treated with cyclophosphamide plus G-CSF as mobilizing therapy . In that study, however, the percentage of inadequate yield (<4 × 106 CD34+ PBSC) after a single mobilization was similar to that (21%) observed in our study. Overall, we confirm a possible negative effect of lenalidomide on CD34+ PBSC collection and this should be taken into consideration when planning mobilization in patients with other potential risk factors, such as cytopenia.
Predicting mobilization failure using clinical variables has been demonstrated to be often inaccurate . In this setting, algorithms based on the so-called ‘pre-emptive’ or ‘just in time’ evaluation of circulating CD34+ PBSC have been shown to be useful in predicting failure, allowing an immediate and appropriate addition of plerixafor to the original mobilization regimen and resulting in improvement of PBSC collection rates [21-25]. However, even among subjects with an apparently adequate number of circulating CD34+ PBSC at the time of apheresis (>20/μl), there may be a not negligible percentage of patients (5% to 10% in the literature, about 12% in our series) still achieving poor or sub-optimal results [26,27]. As a consequence, the rate of harvest failure in MM is usually higher than the rate of mobilization failure. The majority of these discrepancies can probably be ascribed to variability in mobilization, collection efficiency, premature termination of apheresis dictated by clinical problems arising during the procedure, reduced flow from central venous catheter, or poor intra-apheresis mobilization . In any case, our ‘composed’ risk model was able to identify these patients at risk of a poor harvest, although potentially considered good-mobilizers, thus selecting a further population in which the use of plerixafor could be considered.
A possible definition of ‘predicted/proven poor mobilizers’ has been recently proposed by GITMO, which employed an analytic hierarchy process able to analyze quantitative and qualitative aspects of a decision need when a poor information base is available [7,29]. This proposal, however, based on clinical and laboratory criteria, included lymphomas, as well as patients with advanced/refractory disease, multiple lines of prior treatments, failure of previous mobilization attempts, extensive radiotherapy and reduced marrow cellularity or large neoplastic involvement at the time of mobilization. Indeed, our study, in which all patients had newly diagnosed MM achieving at least a partial response after induction treatment, represents a completely different scenario.
Based on four simple clinical parameters, our model was, not unexpectedly, not completely predictive (see AUC reported in Additional file 2). Taking into consideration other variables (that is, the pharmacogenetic background of mobilizing and induction regimens) could certainly improve its performance, probably at the expense, however, of a wide application in daily practice.
To conclude, about 20% of newly diagnosed MM fail to collect an adequate number of PBSC. Our simple ‘risk card’, based on the largest group of patients treated frontline with novel agents and receiving the most popular mobilizing approach currently employed in Europe so far reported in this setting, is applicable in individual patients with MM and may contribute to the early identification of ‘poor mobilizer’ phenotypes. Although certainly ameliorable, this model increases our awareness in this field, representing a novel possible framework to select patients in whom to plan alternative mobilizing strategies [30,31]. Therefore, its validation in prospective series is required, possibly even in the setting of potentially more effective chemotherapy-free-mobilizing regimens, which have been recently reported as superior to the cyclophosphamide and G-CSF combination [27,32].
area under the curve
autologous stem cell transplantation
granulocyte-colony stimulating factor
Gruppo Italiano Trapianto Midollo Osseo
melphalan + prednisone
melphalan-prednisone + thalidomide
melphalan + prednisone + bortezomib
pegylated liposomal doxorubicin + bortezomib + dexamethasone
peripheral blood stem cells
lenalidomide + dexamethasone
receiver operating characteristic
thalidomide + dexamethasone
bortezomib + thalidomide + dexamethasone
Howlader N, Noone AM, Krapcho M, Neyman N, Aminou R, Waldron W. SEER Cancer Statistics Review, 1975–2010, National Cancer Institute. Bethesda, MD, based on November 2012 SEER data submission, posted to the SEER web site; 2013. http://seer.cancer.gov/csr/1975_2010.
Palumbo A, Rajkumar SV, San Miguel JF, Larocca A, Niesvizky R, Morgan G, et al. International Myeloma Working Group consensus statement for the management, treatment, and supportive care of patients with myeloma not eligible for standard autologous stem-cell transplantation. J Clin Oncol. 2014;32:587–600.
Engelhardt M, Terpos E, Kleber M, Gay F, Wäsch R, Morgan G, et al. European Myeloma Network recommendations on the evaluation and treatment of newly diagnosed patients with multiple myeloma. Haematologica. 2014;99:232–42.
Gertz MA, Dingli D. How we manage autologous stem cell transplantation for patients with multiple myeloma. Blood. 2014;124:882–90.
Gertz MA. Current status of stem cell mobilization. Br J Haematol. 2010;150:647–62.
To LB, Levesque JP, Herbert KE. How I treat patients who mobilize hematopoietic stem cells poorly. Blood. 2011;118:4530–40.
Olivieri A, Marchetti M, Lemoli R, Tarella C, Iacone A, Lanza F, et al. Proposed definition of “poor mobilizer” in lymphoma and multiple myeloma: an analytic hierarchy process by ad hoc working group Gruppo ItalianoTrapianto di Midollo Osseo. Bone Marrow Transplant. 2012;47:342–51.
Giralt S, Costa L, Schriber J, Dipersio J, Maziarz R, McCarty J, et al. Optimizing autologous stem cell mobilization strategies to improve patient outcomes: consensus guidelines and recommendations. Biol Blood Marrow Transplant. 2014;20:295–308.
Duong HK, Savani BN, Copelan E, Devine S, Costa LJ, Wingard JR, et al. Peripheral blood progenitor cell mobilization for autologous and allogeneic hematopoietic cell transplantation: guidelines from the American Society for Blood and Marrow Transplantation. Biol Blood Marrow Transplant. 2014;20:295–308.
Cavo M, Zamagni E, Tosi P, Tacchetti P, Cellini C, Cangini D, et al. Superiority of thalidomide and dexamethasone over vincristine-doxorubicindexamethasone (VAD) as primary therapy in preparation for autologous transplantation for multiple myeloma. Blood. 2005;106:35–9.
Cavo M, Tacchetti P, Patriarca F, Petrucci MT, Pantani L, Galli M, et al. Bortezomib with thalidomide plus dexamethasone compared with thalidomide plus dexamethasone as induction therapy before, and consolidation therapy after, double autologous stem-cell transplantation in newly diagnosed multiple myeloma: a randomised phase 3 study. Lancet. 2010;376:2075–85.
Palumbo A, Bringhen S, Bruno B, Falcone AP, Liberati AM, Grasso M, et al. Melphalan 200 mg/m(2) versus melphalan 100 mg/m(2) in newly diagnosed myeloma patients: a prospective, multicenter phase 3 study. Blood. 2010;115:1873–9.
Palumbo A, Gay F, Falco P, Crippa C, Montefusco V, Patriarca F, et al. Bortezomib as induction before autologous transplantation, followed by lenalidomide as consolidation-maintenance in untreated multiple myeloma patients. J Clin Oncol. 2010;28:800–7.
Palumbo A, Cavallo F, Gay F, Di Raimondo F, Ben Yehuda D, Petrucci MT, et al. Autologous transplantation and maintenance therapy in multiple myeloma. N Engl J Med. 2014;371:895–905.
Lacativa CP, Lacativa PG, Garnica M, Portugal RD, Schaffel R, Dos S, et al. Risk factors for unsuccessful peripheral blood stem cell harvesting using granulocyte-colony stimulating factor mobilization in patients with multiple myeloma. Transfus Apher Sci. 2012;47:331–5.
Donmez A, Yilmaz F, Gokmen N, Tombuloglu M. Risk factors for a poor hematopoietic stem cell mobilization. Transfus Apher Sci. 2013;49:485–8.
Pozotrigo M, Adel N, Landau H, Lesokhin A, Lendvai N, Chung DJ, et al. Factors impacting stem cell mobilization failure rate and efficiency in multiple myeloma in the era of novel therapies: experience at Memorial Sloan Kettering Cancer Center. Bone Marrow Transplant. 2013;48:1033–9.
Costa LJ, Nista EJ, Buadi FK, Lacy MQ, Dispenzieri A, Kramer CP, et al. Prediction of poor mobilization of autologous CD34+ cells with growth factor in multiple myeloma patients: implications for risk-stratification. Biol Blood Marrow Transplant. 2014;20:222–8.
Sinha S, Gertz MA, Lacy MQ, Dispenzieri A, Hayman SR, Buadi FK, et al. Majority of patients receiving initial therapy with lenalidomide-based regimens can be successfully mobilized with appropriate mobilization strategies. Leukemia. 2012;26:1119–22.
Cavallo F, Bringhen S, Milone G, Ben-Yehuda D, Nagler A, Calabrese E, et al. Stem cell mobilization in patients with newly diagnosed multiple myeloma after lenalidomide induction therapy. Leukemia. 2011;25:1627–31.
Sancho JM, Morgades M, Grifols JR, Juncà J, Guardia R, Vives S, et al. Predictive factors for poor peripheral blood stem cell mobilization and peak CD34(+) cell count to guide pre-emptive or immediate rescue mobilization. Cytotherapy. 2012;14:823–9.
Jantunen E, Varmavuo V, Juutilainen A, Kuittinen T, Mahlamäki E, Mäntymaa P, et al. Kinetics of blood CD34(+) cells after chemotherapy plus G-CSF in poor mobilizers: implications for pre-emptive plerixafor use. Ann Hematol. 2012;91:1073–9.
Milone G, Tripepi G, Martino M, Ancora F, Bartolozzi B, Spadaro A, et al. Early measurement of CD34+ cells in peripheral blood after cyclophosphamide and granulocyte colony-stimulating factor treatment predicts later CD34+ mobilisation failure and is a possible criterion for guiding “on demand” use of plerixafor. Blood Transfus. 2013;11:94–101.
Ozsan GH, Micallef IN, Dispenzieri A, Kumar S, Lacy MQ, Dingli D, et al. Hematopoietic recovery kinetics predicts for poor CD34+ cell mobilization after cyclophosphamide chemotherapy in multiple myeloma. Am J Hematol. 2012;87:1–4.
Rossi G, Skert C, Morello E, Almici C, Arcaini L, Basilico C, et al. PBSC mobilization in lymphoma patients: analysis of risk factors for collection failure and development of a predictive score based on the kinetics of circulating CD34+ cells and WBC after chemotherapy and G-CSF mobilization. Hematol Oncol 2014. doi:10.1002/hon.2148.
Gertz MA, Wolf RC, Micallef IN, Gastineau DA. Clinical impact and resource utilization after stem cell mobilization failure in patients with multiple myeloma and lymphoma. Bone Marrow Transplant. 2010;45:1396–403.
Milone G, Martino M, Spadaro A, Leotta S, Di Marco A, Scalzulli P, et al. Plerixafor on-demand combined with chemotherapy and granulocyte colony-stimulating factor: significant improvement in peripheral blood stem cells mobilization and harvest with no increase in costs. Br J Haematol. 2014;164:113–23.
Abuabdou A, Rosenbaum ER, Usmani SZ, Barlogie B, Cottler-Fox M. Analysis of CD34+ cell collection using two mobilization regimens for newly diagnosed multiple myeloma patients reveals the separate impact of mobilization and collection variables. J Clin Apher. 2014;29:251–5.
Piccirillo N, Vacca M, Lanti A, Ipsevich F, Maresca M, Fiorelli E, et al. Poor mobilizer: a retrospective study on proven and predicted incidence according to GITMO criteria. Transfus Apher Sci. 2012;47:217–21.
Lanza F, Lemoli RM, Olivieri A, Laszlo D, Martino M, Specchia G, et al. Factors affecting successful mobilization with plerixafor: an Italian prospective survey in 215 patients with multiple myeloma and lymphoma. Transfusion. 2014;54:331–9.
Andreola G, Vanazzi A, Radice D, Babic A, Rabascio C, Negri M, et al. Who should be really considered as a poor mobilizer in the plerixafor era? Transfus Apher Sci. 2012;47:27–32.
Afifi S, Adel NG, Duck E, Devlin SM, Landau H, Lendvai N, et al. Upfront plerixafor plus G-CSF versus cyclophosphamide plus G-CSF for autologous stem cell mobilization in multiple myeloma patients: an update on cost analysis study at Memorial Sloan Kettering Cancer Center. Blood. 2014;124:848. abstract.
This study was supported by funds from Current Research, Italian Ministry of Health and presented as Oral Communication at the 19th Congress of the European Haematology Association, Milan, 15 June 2014.
PM: Honoraria from Celgene, Janssen, Novartis, Advisory committee for Sanofi and Sandoz; FG: Honoraria from Celgene and Janssen-Cilag, Advisory committee for Celgene and Sanofi; SB: Honoraria from Celgene, Janssen-Cilag and Novartis, Advisory committee for Merck Sharp & Dohme, Consultancy from Onyx; AL: Honoraria from Celgene and Janssen-Cilag; MTP: Honoraria from Celgene, Janssen-Cilag, Sanofi and Bristol-Myers Squibb; MO: Honoraria from Janssen, Celgene, Sanofi, Mundipharma, Amgen, Novartis; AP: Consultancy fees from Amgen, Bristol-Myers Squibb, Genmab A/S, Celgene, Janssen-Cilag, Millennium Pharmaceuticals Inc., Onyx Pharmaceuticals, Honoraria from Amgen, Array BioPharma, Bristol-Myers Squibb, Genmab A/S, Celgene, Janssen-Cilag, Millennium Pharmaceuticals Inc, Onyx Pharmaceuticals, Sanofi Aventis; MC: Honoraria from Janssen, Celgene, Novartis, Onyx, Bristol-Myers Squibb, Sanofi, Consultancy for Janssen, Celgene, Novartis, Sanofi and Millennium Pharmaceuticals; MB: Consultancy and Scientific advisory board from Celgene, Janssen-Cilag, Onyx, and Sanofi. All other authors declare that they have no competing interests.
PM, AP, MC, AG, and MB designed the study. PM, AP and MB wrote the paper. VS performed statistical analysis. FG, SB, AL, RG, GP, OV, GD, FM, MTP, MO, EZ, PT, CC, GM, CCu and CM enrolled and/or followed the patients, managed local databases and contributed to acquisition, analysis and interpretation of final data. CCu and CM also checked the quality of data of all stem cell collections. All authors read and approved the final manuscript.
Logistic regression model. Univariate and multivariate analysis of risk factors, taking into consideration two models having as outcome the risk to have a ‘failure’ or that of a ‘sub-optimal’ collection. Statistical significance was defined as P <0.05.
Receiver operating characteristic curves (ROC) of logistic regression model. ROC curves assessing the model discriminatory power for the predictive probability of failure and suboptimal harvests, respectively. (PDF 160 kb)