Creatinine clearance versus serum creatinine as a risk factor in cardiac surgery

Abstract

Background

Renal impairment is one of the predictors of mortality in cardiac surgery. Usually a binarized value of serum creatinine is used to assess the renal function in risk models. Creatinine clearance can be easily estimated by the Cockcroft and Gault equation from serum creatinine, gender, age and body weight. In this work we examine whether this estimation of the glomerular filtration rate can advantageously replace the serum creatinine in the EuroSCORE preoperative risk assessment.

Methods

In a group of 8138 patients out of a total of 11878 patients, who underwent cardiac surgery in our hospital between January 1996 and July 2002, the 18 standard EuroSCORE parameters could retrospectively be determined and logistic regression analysis performed. In all patients scored, creatinine clearance was calculated according to Cockcroft and Gault. The relationship between the predicted and observed 30-days mortality was evaluated in systematically selected intervals of creatinine clearance and significance values computed by employing Monte Carlo methods. Afterwards, risk scoring was performed using a continuous or a categorical value of creatinine clearance instead of serum creatinine. The predictive ability of several risk score models and the individual contribution of their predictor variables were studied using ROC curve analysis.

Results

The comparison between the expected and observed 30-days mortalities, which were determined in different intervals of creatinine clearance, revealed the best threshold value of 55 ml/min. A significantly higher 30-days mortality was observed below this threshold and vice versa (both with p < 0.001). The local adaptation of the EuroSCORE is better than the standard EuroSCORE and was further improved by replacing serum creatinine (SC) by creatinine clearance (CC). Differential ROC analysis revealed that CC is superior to SC in providing predictive power within the logistic regression. Variable rank comparison identified CC as the best single variable predictor, even better than the variable age, former number 1, and SC, previously number 9 in the standard set of EuroSCORE variables.

Conclusion

The renal function is an important determinant of mortality in heart surgery. This risk factor is not well captured in the standard EuroSCORE risk evaluation system. Our study shows that creatinine clearance, calculated according to the Cockcroft and Gault equation, should be applied to estimate the preoperative renal function instead of serum creatinine. This predictor variable replacement gains a significant improvement in the predictive accuracy of the scoring model.

Background

Several models for preoperative risk assessment in cardiac surgery were introduced in the past 15 years, all of which considered the renal function as one of the predictors of mortality. The following criteria were used to estimate the renal function: acute renal failure, necessity of dialysis, categorical serum creatinine value with the threshold set in the interval between 1.6 and 2.3 mg/dl (140 and 200 μmol/l) [1].

The European System for Cardiac Operative Risk Evaluation (EuroSCORE) is one of the established risk score systems in Europe [2–4]. In Germany, the EuroSCORE is applied as a nation-wide tool for the risk adjustment of 30-days mortality in cardiac surgery quality control. This scoring system is based on a logistic regression model for mortality and considers all adult cardiac surgery procedures within a defined 30-days post-operative time interval. The impaired renal function is included in this score system as one of 18 independent predictors by testing on serum creatinine values above threshold 200 μmol/l (see Table 4). According to the standard, so-called "simple additive" EuroSCORE model, an integer number called weight is assigned to each risk parameter present in the scored patient. The individual weights are added to give the expected risk score of death. The EuroSCORE value usually lies between 0 and 20. One main attraction of this model is its simplicity and its ease of usage and of explanation for people without statistical training. The newer, published version of the EuroSCORE, also called the "full logistic" EuroSCORE, assigns a weight of continuous value to each risk factor. These values can enter directly the probability calculation of the mean multicenter mortality risk.

In 1976, Cockcroft and Gault introduced an equation to estimate the glomerular filtration rate from the serum creatinine value, also considering such variables as age, gender and body weight [5]. We have studied a large group of our patients with the aim of establishing whether it is appropriate to replace serum creatinine in the EuroSCORE based preoperative risk estimation by creatinine clearance, calculated according to the Cockcroft-Gault formula.

Patients and Methods

A total of 11878 patients underwent cardiac surgery in our hospital in the period between January 1996 and July 2002. The 18 EuroSCORE risk variables could retrospectively be determined in a group of 8138 patients. In our institute, the postoperative course is systematically checked six months after surgery. Thanks to this approach, the 30-days mortality rate could easily be determined. In all our patients scored, creatinine clearance was calculated according to Cockcroft and Gault. The following equation was used for men:

The proportion of muscle mass on body weight is relatively lower in women than in men. Therefore, the calculated value of creatinine clearance was multiplied by the factor of 0.85 in female patients. Although serum creatinine is one of the dichotomous variables in the EuroSCORE it is impossible to linearly transform this to Equation (1).

Earlier studies, comparing our patient population with the original EuroSCORE study, revealed, on the one hand, that our group of patients exhibited a risk profile which was shifted towards higher risk values and, on the other hand, significantly lower mortality in our patients (see Table 1). Missing value analysis revealed two anomalies of the data set: the mortality was significantly higher when the variable "chronic pulmonary disease" or "neurological dysfunction" was missing and therefore patient excluded from this study. But it turned out that this is due to the incomplete data collection in cases of very emergent operations. In other words the present data set underrepresents the small group of emergency operations (prevalence 1.6% instead of 3.6% total).

Table 1 Differences between the EuroSCORE and Lahr patient population

Table 2 Deviations of expected and observed mortality frequencies and their significance for several creatinine clearance intervals.

We have developed several approaches based on the standard EuroSCORE model with the aim of establishing our own institutional risk scoring system. All our models used the 18 standard preoperative EuroSCORE risk variables. The added weights, which were assigned to the risk factors present, gave the final predictive risk value. In the first calibration step, the simple additive model was adjusted to the observed institutional mortality by regression analysis. In this way, the individual risk, expressed as expected mortality (EM), could be determined for each patient as the death probability. For all patients, this probability was in the range of 0.002 to 0.1.

The studied population of 8138 patients was divided into 13 groups on the basis of creatinine clearance (CC) calculated according to the Cockcroft-Gault formula. In each group, the sum of the expected mortality (EM) was compared with the sum of the observed mortality (OM). The difference between EM and OM, the negative residuals, are sometimes called "net life saved" (NLS) values (particularly in the context of temporal performance evaluations, e.g. "VLAD" [6]). A positive NLS value indicates the operation results are better than expected. Since the NLS value scales with the group size, a second invariant measure, the OM-to-EM ratio, also called the "risk adjusted mortality quotient" (RAMQ), was determined. A RAMQ value below one indicate a surgical performance better than the average.

Due to the low probability of death and the finite group sizes considerable fluctuations of the NLS and RAMQ values are encountered. To determine the significance of deviations, we employed the method of Monte Carlo simulation in order to gain EM distributions and from there measurements of significance. Here the null hypothesis is the correctness of the underlying risk model in all groups studied. The alternative hypothesis is that the risk model is incomplete and systematic deviations in the CC groups are present. We found the latter true and subsequently determined the most effective binary threshold for the CC value. By systematic testing (in the obvious range 45–65) we found the threshold CC<55 ml/min. On the one hand this aggregation in two groups improves the significance by increasing the number of observations per group, and on the other hand, it is in concordance with the spirit of the EuroSCORE as a simple, easy usable scoring system.

The consequence is the suggestion to modify the EuroSCORE system by the replacing serum creatinine value by the creatinine clearance value in order to better capture the risk from impaired renal function.

In the next step we analysed the potential improvement of this step by comparing various logistic risk models using the measure of the area under the receiver operating characteristic (ROC) curves. (i) The first model studied the simple additive EuroSCORE model with 18 fixed integer weights (see Table 4 and http://www.euroscore.org/euroscore_scoring.htm) calibrated for our data set. (ii) The second score model examined was the full logistic EuroSCORE with fixed and continuous weights (see http://www.euroscore.org/logisticEuroSCORE.htm). (iii) In the third model, the weighted values of all 18 EuroSCORE risk factors were directly determined by the logistic regression fit. (iv) In the fourth model, serum creatinine was replaced by a continuous CC value as a risk variable characterising the renal function. (v) This procedure was repeated in the fifth model, however, instead of the continuous CC value the dichotomous value with the threshold CC<55 ml/min was used.

Improving mature risk models is inherently difficult and changes in ROC area are difficult to judge since the differences are often smaller than the computed standard deviations. Therefore we examined the predictive power of all 18 EuroSCORE variables in addition to the CC variable by comparing logistic regression models. The first set of models capture the predictive power contribution of a variable by contrasting the full EuroSCORE (Type (iii) in Table 3) and the reduced model, generated by leaving out the considered variable. The ROC area reduction is computed and the rank order determined. The second set of models examines the single predictor ability by computing and ranking the gain in ROC area compared to the uninformed dummy predictor with area = 0.5. The ranking results allow to judge the newly suggested predictor variable in comparison to the widely established EuroSCORE variable set.

Table 3 Predictive ability of the EuroSCORE risk score model and comparision with suggested modified systems.

Table 4 EuroSCORE model and ist scoring weights; standard "simple additive" version, as http://www.euroscore.org and Tab. 3 (i).

Results

Figure 1 shows the distribution of patients scored (with model 1 in Table 3), as well as the distribution of observed (OM) and expected mortalities (EM) for several CC intervals. The maximum EM values occurred at lower CC values than (or left of) the cases curve maximum. This means that the EuroSCORE did identify low CC values as being associated with a higher risk. Nevertheless, the risk was underestimated, which becomes apparent at the excess of the observed mortality curve (OM > EM) for all CC intervals below 55 and the clear reverse (OM < EM) for all higher intervals. The differences between the observed and expected mortalities were significant in two groups: 45≤CC<55 with p = 0.011 and 55≤CC<65 with p = 0.006 found by Monte-Carlo simulated mixture of individual expected risks (Table 2). After the formation of two aggregate groups with the systematically identified threshold CC<55 ml/min the significance became even more pronounced with p < 0.001 (see Table 2 and Figure 2).

Figure 1

Distribution of cases, observed and expected mortalities in 13 creatinine clearance intervals (numbers in percent of total, see Table 2; underlying risk model type (i) in Table 3).

Figure 2

Similar to Table 2, here as bar plot and split in two CC groups with threshold 55 ml/min.

The predictive ability of the CC-modified EuroSCORE risk score is better than the predictive accuracy of the original EuroSCORE models (i+ii) and the locally adapted logistic regression (iii). The replacement of serum creatinine by creatinine clearance either as a continuous (iv) or categorical value (v), both improve the predictive power of the EuroSCORE models. As the ROC area results in Table 3 show, categorical CC model (v) is best (0.776 s.d. 0.018), closely followed by the continuous CC model (iv).

Figure 3 displays the results of the individual contribution of the 18 well established predictors to the predictive ability of the EuroSCORE. The marginal ROC area contributions and their rank number is listed for the set removal and the single predictor case. The binarized and the continuous CC value gain both the 5^th rank for replacing serum creatinine and the 1^st rank as single variable predictor, while the SC variable holds the 14^th and 9^th rank only.

Figure 3

Contribution of single predictors of the EuroSCORE measured by change in area under the Receiver Operating Characteristics (ROC) curve (times 100) in two ways: The "Leave out Predictor" results show the reduction of ROC area from the full set of EuroSCORE variables ROC = 0.776 (model 3 in Table 3). The "Single Predictor" numbers indicate the predictive power of each isolated variable (i.e. ROC gain above 0.5, times 100). Furthermore the binarized creatinine clearance variable ("CC<55") is added (as a replacement for serum creatinine) and its insertion position in the rank ordering 1..18 indicated by an extra decimal digit (5.0 and 1.0 before 5 and 1 resp.). Note, that in the "Leave out" case the CC-variable reaches the 5^th most contributing predictor and in the single predictor experiment it even outperforms "Age" and gains the top rank. Serum creatinine holds only position 14 and 9 respectively.

Discussion

In the Framingham Heart Study performed on 6233 subjects (mean age 54 years, 54% females), 8% of women and 8.7% of men exhibited mild renal insufficiency, which was defined according to the serum creatinine value (120 to 256 μmol/l, i.e., 1,4 to 3,0 mg/dl in women and 136 to 265 μmol/l, i.e., 1,5 to 3,0 mg/dl in men) [7]. The data analysis of 18790 patients in the HOT (Hypertension Optimal Treatment) Study revealed that the impaired renal function was a predictor of increased cardiovascular morbidity and mortality and that patients suffering from renal failure exhibited a higher cardiovascular risk than patients with an intact renal function [8]. In patients with chronic heart failure, the renal function is a prognostic risk value [9, 10] which can be regarded as a predictor of mortality in this patient group [11]. Multiple models for preoperative risk evaluation in patients undergoing heart surgery also confirmed the significance of the renal function as a predictor of mortality. In these models, acute renal failure, the necessity of dialysis and serum creatinine, in form of a categorical value, were applied as risk criteria [1].

The serum creatinine level is influenced by many factors which are independent of the glomerular filtration rate: tubular secretion and reabsorption, endogenous production, variable intake, extrarenal elimination and interference, caused by the laboratory diagnostic techniques and medicaments used [12, 13]. Since the assessment of the renal function, based on the determination of serum creatinine, is associated with several limitations [13, 14] and the measurement of creatinine clearance by urine collection is rather time-consuming, several formulas estimating the renal function from serum creatinine, body weight, age and sex, as well as ethnic features, have been developed. All these formulas exhibit certain limitations. The most commonly used equation for estimating creatinine clearance, e.g., in the Medicare programme and in the transplantation waiting lists in the USA [15], is the Cockcroft-Gault formula. Although this formula also does not provide absolutely accurate results (e.g., in elderly patients) and it may over- or underestimate the true renal function [12, 16–19], several studies on cardiac insufficiency and renal impairment have shown a good correlation between the creatinine clearance values calculated according to Cockcroft and Gault and the measured glomerular filtration rate [20–24]. Because of this broad acceptance of the Cockcroft-Gault formula, we have decided to use it in our model.

The initial steps in our present study concentrated on the proper selection of a threshold creatinine clearance value. It was apparent that in all patient groups with a CC value lower than 55 ml/min the observed mortalities were higher than the ones predicted by the standard EuroSCORE model. This indicates a poor modelling of the renal impairment variable in the EuroSCORE model, which is defined by a binary serum creatinine variable, i.e. SC level above 200 μmol/l (Table 4).

The determined CC threshold of 55 ml/min is in accordance with many findings of other authors. The large HOT Study defined a CC value of 60 ml/min as a criterion for the impaired renal function [8]. Hillege and co-workers [11] divided their population of 1906 patients suffering from chronic heart failure into four groups according to the CC values estimated with the Cockcroft-Gault formula. The following intervals of CC values were used: <44, 44–58, 59–76 and >76 ml/min. The overall mortality (calculated according to Kaplan-Meyer) in the four groups studied was 36.5%, 24.8%, 17.6% and 13.7% whereby significant differences were determined between the first two and last two groups.

Because of the differences in the risk profile between the patient population examined in the EuroSCORE study and our own (see Table 1), we have developed several own institutional score models by means of logistic regression analysis. To be able to secure comparability with other institutions and because of the broad acceptance of the EuroSCORE model, we have used its 18 determinants as the basis of our model as well. The predictive power of the score model can be improved by readapting the EuroSCORE regression model to the large patient population in our institution (Table 3). Furthermore, we have shown that the variable selection of the EuroSCORE can be significantly improved by better capturing the major risk factor impaired renal function. We found either the continuous or the binarized preoperative creatinine clearance value is an easy to assess measurement which encoded renal function much better than the binarized serum creatinine value as called for in the EuroSCORE procedure.

We showed this by significance analysis of the binarized CC<55 value with p < 0.001 using Monte-Carlo methods. Furthermore we demonstrated the improvement of the predictive power by calculating the area gain under the ROC curve. The final experiments examined the individual contribution of each EuroSCORE variable to the ROC area. Only one variable ("age") showed a marginal ROC area contribution (0.0291) larger than one standard deviation of the best ROC value (0.0018). All other 17 well established predictors would be doubted when judged by ROC area improvement only. This exhibits the difficulty of the uncertainty measures (s.d.) of the ROC area in comparison to model differences. To circumvent this hurdle we evaluated the rank ordering of the predictors contribution. This can be done in two ways: either the marginal contribution measured by leaving the predictor out of the regression, or by employing only the isolated predictor. The resulting numbers and rankings are certainly varying since in the set of 18 parameter the information is partially provided by other variables.

The rank numbers expose the superiority of the proposed creatinine clearance value compared to the standard EuroSCORE choice. While the serum creatinine variable ranks number 14, the CC replacement would gain rank 5 within the (modified) set of EuroSCORE predictors. The reverse view gives even more favourable figures: the creatinine clearance value (binary or continuous) surpasses age in the top rank as a single variable predictor.

The risk model based estimation of the expected mortalities, the grouping (EM) and comparison with the total observed mortality (OM) by the two parameters, NLS and RAMQ, represent effective analytical tools in assessing the potential further influences for mortality (occurrence of preoperative disease, choice of surgical procedure, etc.). Using Monte-Carlo methods for testing the significance of deviations as well as ranking of predictor variable are a valuable addition to conventional statistical methods. These analytical methods give us the opportunity to better study, e.g., the effects of the renal function, diabetes and body mass index on the outcome of patients undergoing heart surgery.

On the basis of our results we summarize that the renal function is an important determinant of 30-days mortality in cardiac surgery. This risk factor is not well captured in the standard EuroSCORE model. Creatinine clearance calculated according to the Cockcroft and Gault equation should be considered in the preoperative assessment of the renal function instead of serum creatinine. This procedure results in a significant improvement of the risk estimation.