To whom do the results of the multicenter, randomized, controlled INSECT trial (ISRCTN 24023541) apply? - assessment of external validity
© Fischer et al; licensee BioMed Central Ltd. 2012
Received: 17 May 2011
Accepted: 8 February 2012
Published: 8 February 2012
A response to Seiler et al: Interrupted or continuous slowly absorbable sutures for closure of primary elective midline abdominal incisions: a multicenter randomized trial (INSECT: ISRCTN24023541). Ann Surg 2009, 249(4):576-582.
Existing evidence suggests that the transfer of results of randomized controlled trials into clinical practice may be limited. Potential reasons can be attributed to aspects of external validity. The aim of this study is to investigate issues related to the external validity of the INSECT trial.
All participating surgical departments were categorized and the clinical and baseline characteristics of randomized patients were evaluated. In addition, demographic and clinical data of all screened and randomized patients at the Departments of Surgery in Heidelberg and Erlangen were analyzed.
Twenty-five centers enrolled a total of 625 patients. These centers included eight primary, 11 secondary, and six tertiary care centers. The tertiary care centers enrolled the most patients (n = 237, 38%) followed by the primary care centers (n = 199, 32%) and the secondary care centers (n = 189 patients; 30%). The mean number and baseline data of randomized patients did not differ between the three types of care centers (p = 0.09). Overall, the treatment according to protocol was at least 92%. At the Department of Surgery, University of Heidelberg, 307 patients were screened and 60 out of 130 eligible patients were randomized. There were no differences in demographic and clinical baseline data between included and non-included patients. In Erlangen, 351 patients were screened and 57 out of 106 eligible patients randomized.
Results of the INSECT trial are applicable to a broad spectrum of patients treated at different hospital levels.
Incisional hernias of midline incisions are the most common long-term complication after major abdominal surgery with an incidence ranging from 5% - 24% . Although randomized clinical trials (RCTs) and meta-analyses  have been performed in order to define optimal closure strategies, clinical uncertainty still has not been resolved.
Issues that potentially affect external validity adapted to Rothwell (10).
Setting of the trial
Recruitment from primary, secondary, or tertiary care
Selection of participating centers
Selection of participating clinicians
Selection of patients
Patients declining randomization
Characteristics of randomized patients
Baseline clinical characteristics
Severity of disease
Differences between trial protocol and routine praxis
Therapeutic or diagnostic advances since the trial was completed
Outcome measures and follow-up
Who measured outcome
Frequency of follow-up
Adequate length of follow-up
Adverse effects of treatment
Completeness of reporting
Selection of trial centers/clinicians
Intensity of trial safety procedures
The internationally registered INSECT trial was designed and conducted as a multicenter RCT (mRCT) comparing two continuous suture techniques with slowly absorbable monofilament materials (PDS™ or MonoPlus™) and one interrupted suture (Vicryl™) for fascial closure after primary elective midline incisions . The primary endpoint of the INSECT trial was the occurrence of incisional hernias within one year after the surgical intervention. The trial was conducted on 625 randomized patients and its primary analysis showed no differences in the occurrence of incisional hernias between the three groups after one year. Further, there were no significant differences with regard to burst abdomen, wound infection, pulmonary infections, serious adverse events, and one-year mortality . The objective of this analysis was to investigate issues of external validity of the INSECT trial focusing in particular on institutional and selection biases.
During the INSECT trial, 625 patients were randomized at 25 surgical sites between July 11, 2004, and September 26, 2006. The detailed study protocol of the INSECT trial describes several strategies to ensure high internal validity including primary hypothesis, randomization, sample size calculation, harmonization and standardization of treatment, assessment and bias . In addition, issues that potentially affect external validity were recorded, namely detailed information describing the process from screening to randomization in two participating centers.
Setting of the trial
The types of all participating surgical departments were categorized into primary, secondary or tertiary care centers. Primary care centers (PCC) were defined as hospitals with basic diagnostic and therapeutic options offering general surgical interventions. Secondary care centers (SCC) were hospitals with a broader surgical spectrum (i.e. more specialized departments) and advanced diagnostic and therapeutic tools to which a patient had been referred by primary care providers. Tertiary care centers (TCC) act as referral centers and are often academic hospitals with the highest standards of care, including access to most specialists and the necessary equipment that may be lacking in primary and secondary care centers. The selection of participating centers and surgeons was purely based on the self-motivation of the according centers. Once they agreed to adhere to the protocol, signed a contract, received ethical approval from their local ethics committee and finalized training, they were able to start patient recruitment. In order to assess whether this process caused imbalances between the patient populations at the different hospital levels, analyses concerning the recruitment of patients and demographics (i.e. age and gender) as well as clinical parameters (body mass index) and surgery-related parameters (type of procedures) were performed. In addition, the level of expertise of surgeons and the adherence to the protocol were assessed.
Selection of patients
Inclusion and exclusion criteria of the trial were shown previously . Due to the incomplete recording of screening lists, the according data was not available from other participating centers. Patients at the Department of Heidelberg were analyzed for demographics (age and gender), clinical parameters (body mass index, ASA score, duration of hospitalization, presence of malignant diseases), and surgery-related parameters (incision technique, closure technique) according to three specified subgroups: included patients, eligible patients who declined participation and eligible patients randomized to one other RCT.
The random ratio was calculated by dividing the number of randomized patients/participants by the number of primarily eligible patients/eligible for participation for the Department of Surgery of the University of Heidelberg and Erlangen. The random ratio thus equals the enrollment fraction as described by Gross et al. . According to Gross et al.  and Jones et al. , the enrollment fraction equals the proportion of people who are eligible for participation and who actually enroll in the RCT.
Data analysis was done using SAS™ 9.1 Win (Release 9.1, SAS Institute, Inc., Cary, NC). A description of the data included absolute and relative frequencies for categorical data and mean and standard deviations for continuous data. Possible differences between groups were calculated with Fisher's exact test of categorical parameters, Kruskal-Wallis test for continuous variables, and binomial test for proportion of randomized patients. The explorative statistical significance level was set at 5% (0.05); no adjustment for multiple comparisons was done due to the descriptive nature of the study.
This article was structured based on the recommendations according to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) in order to include an accurate and complete report of an observational study . The original INSECT trial was published in accordance with the CONSORT criteria .
Setting of the trial
Analysis of participating centers.
Type of care center
Randomized patients (n)
Percentage of randomized patients (%)*
Mean of randomized patients
Selection of patients
During the recruitment phase of the INSECT trial, all patients admitted with a diagnosis requiring a midline laparotomy were screened to be randomized to the INSECT trial. After controlling all inclusion and exclusion criteria and receiving informed consent, patients were included for randomization.
Baseline characteristics of all randomized patients.
Body Mass Index (kg/m 2 )
Surgical Procedure (n)
Treatment according to protocol (%)
Lost to follow-up (%)
At the Department of Surgery, University of Heidelberg, 307 patients scheduled for elective abdominal surgery were screened at admission (24 hours before surgery) for eligibility from January 13 to August 25, 2005. At the Department of Surgery, University of Erlangen, 351 patients were screened between October 21, 2004, and February 24, 2006.
Based on the given data, the overall eligibility fraction was 37.5% (Heidelberg 45.1%, Erlangen 31.1%) and the overall recruitment fraction was 18.6% (Heidelberg 20.8%, Erlangen 16.7%).
Characteristics of randomized patients
From the 625 randomized patients, there were 210 patients randomized to interrupted suture using an absorbable braided suture material, 205 patients randomized to continuous suture techniques using PDS™, and 210 patients randomized to continuous suture techniques using MonoPlus™. All 625 patients were of Caucasian origin. After analyzing demographics (age and gender), clinical parameters (body mass index), and surgery-related parameters (surgical procedures) of all randomized patients, there were no differences between the three randomized groups (table 3). The indication for the surgical procedures did not differ between the three hospital types. In 83% of PCC, 82% of SCC, and 83% of TCC, the primary diagnosis was associated with the rectum, colon, and stomach/esophagus. Further, the incision length did not differ significantly between the randomized groups whereas the closure times for the continuous groups were significantly shorter compared to the interrupted group (P < 0.01; ).
Surgeons training for standardizing abdominal wall closure
Midline incision of the abdominal wall is a preferred strategy to open the abdominal cavity. However, there is no consent within the surgical community concerning the optimal abdominal wall closure after midline incision. For the INSECT trial, the two most often used methods of abdominal wall closure were compared using three different suture materials. In order to harmonize the closure techniques between the different centers and participating surgeons as much as possible, all participating centers were trained in how to perform the different suturing techniques adequately before randomization of the first patient. The results for the three different hospital categories were excellent with more than 90% of all patients treated according to protocol.
Outcome measures and follow-up examinations
Patients were observed up to 30 days after the primary surgical intervention to record adverse and serious adverse events and early complications defined as secondary endpoints (burst abdomen, wound infection, and postoperative pulmonary complications). The definition for secondary endpoints is published elsewhere . The 30-day follow-up was conducted by coordinating personnel. The frequency of incisional hernias was examined after one year. A surgeon performed the clinical examination and at minimum an ultrasound of the abdominal wall had to be performed by a radiologist. Coordinating personnel as well as surgeons and radiologists involved in follow-up examinations were blinded to the method of closing the abdominal wall.
Adverse effects of treatments
During the INSECT trial, adverse events were monitored and classified by an adverse events committee (AEC) consisting of three surgeons. All information on adverse events were initially documented and verified by coordination personnel and eventually submitted to the AEC. All adverse events were defined in the published protocol . All serious adverse events were reported to the according regional councils.
External validity describes to which extent the results of RCTs are applicable to patients treated in general practice. Together with the primary publication , the focus of this analysis was to investigate further aspects relevant for the external validity of the INSECT trial mainly according to the issues raised by Rothwell . As the data presented here comprises a significant amount of new information, it seemed reasonable to publish this data as a separate article.
Because of prospective data acquisition, external monitoring and external data analysis the INSECT trail is an optimal source for analyzing external validity. The INSECT trial was a multicenter German RCT. Thus, there were no differences based on different healthcare systems or countries. The participation was voluntarily suggesting that only motivated surgeons got involved. Even though the number of participating TCC was the lowest, the mean number of randomized patients did not differ between PCC, SCC and TCC, respectively.
One main conclusion of this paper would be that smaller hospitals such as PCC (or SCC) should be involved in clinical trials whenever possible because these hospitals are important resources in terms of patient recruitment. Not only that, but the here presented analyses also revealed that the randomization process occurred at similar speeds in all three types of hospitals; the adherence to the protocol was significantly better in PCC and SCC; the number of included patients and the main characteristics of included patients did not differ between the three hospital types; and the number of patients lost to follow-up did not differ significantly between PCC, SCC and TCC. Thus, the message of the INSECT trial (i.e. the rate of incisional hernias after midline incision is not dependent on the used suture material/technique) is truly independent of the hospital type in which the patient is operated.
Reasons for not participating in a surgical study are multifactorial and were not assessed systematically within this study. However, they include well-known reasons such as the dislike of randomization procedures, strict protocols for performance of interventions and unwillingness for follow-up investigations. In order to detect potential selection bias, the screening processes at the Departments of Surgery at the Universities of Heidelberg and Erlangen were assessed in detail. The process from screening until randomization could be described in detail in 116 patients out of 625 randomized patients (18.5%). Even though this alone may not allow us to draw general conclusions about external validity, other parameters such as the progress of randomized patients between the different hospitals and the clinical data of included patients were also similar. This suggests that the process from screening to randomization may be also comparable in all participating centers.
The largest group of excluded patients had a prior laparotomy. Because this patient group may not benefit from the results achieved during the INSECT trial, further studies are needed to define the optimal method of abdominal wall closure for patients with prior laparotomy. Out of the remaining potentially eligible patients, another 119 patients had to be excluded because of two reasons: firstly, patients refused randomization; and secondly, patients were already included in another RCT, one of which was the POVATI trial .
Baseline characteristics of the 60 included patients compared to the 70 patients who had to be excluded due to non-study related reasons.
Patients in INSECT trial
(n = 60)
Patients not consenting
(n = 44)
Patients in POVATI trial (n = 26)
Body Mass Index
Duration of hospitalization
In general, the random ratio ("enrollment fraction" according to Gross et al.) describes the proportion of randomized patients to the primarily eligible patients. The random ratio per se does not say anything as to whether the results of a study are widely acceptable. It is more likely to be a marker of the efforts necessary to randomize a patient. Even though the random ratio at the Department of Surgery at the University of Erlangen was similar to that at the University of Erlangen, the recruitment period lasted longer in Erlangen (16 months versus 6 months at Heidelberg). One possible explanation could be that all surgical trainees in Heidelberg rotate into the clinical study centre and residents who already finished their rotation are part of an on-call program. This setting allows screening and inclusion of patients even at night or on weekends. In contrast, one clinical investigator was primarily responsible for patient recruitment in Erlangen. Considering missing days due to holidays, weekends, days off after being on call, etc., screening and recruiting of patients was only possible on 120 days.
Most of the criteria for external validity as suggested by Rothwell  have been fulfilled by the INSECT trial.
I nterrupted or con tinuous slowly absorbable s uture - Design of a multi-center randomized trial to e valuate abdominal c losure t echniques
Primary care center
Randomized controlled trial
Secondary care center
Tertiary care center
Supported by BBD-Aesculap, GmbH, Tuttlingen, Germany; Infrastructure of the SDGC is funded by the German Federal Ministry of Education and Research (Bundesministerium für Bildung und Forschung (BMBF); 01GH99033) and the German Surgical Society (Deutsche Gesellschaft für Chirurgie). Additional industrial funding for the infrastructure has been provided by Johnson & Johnson Medical GmbH, Norderstedt, Germany, and Covidien Healthcare Deutschland GmbH, Neustadt, Germany. Presented in part at the 29th Annual Meeting Society of Clinical Trials, St. Louis, USA, May 2008.
The original study on which the present article is based was published 2009 in Annals of Surgery .
- Seiler CM, Bruckner T, Diener MK, Papyan A, Golcher H, Seidlmayer C, Franck A, Kieser M, Buchler MW, Knaebel HP: Interrupted or continuous slowly absorbable sutures for closure of primary elective midline abdominal incisions: a multicenter randomized trial (INSECT: ISRCTN24023541). Ann Surg. 2009, 249 (4): 576-582. 10.1097/SLA.0b013e31819ec6c8.View ArticlePubMed
- Hodgson NC, Malthaner RA, Ostbye T: The search for an ideal method of abdominal fascial closure: a meta-analysis. Ann Surg. 2000, 231 (3): 436-442. 10.1097/00000658-200003000-00018.PubMed CentralView ArticlePubMed
- Rothwell PM: Factors that can affect the external validity of randomised controlled trials. PLoS Clin Trials. 2006, 1 (1): e9-10.1371/journal.pctr.0010009.PubMed CentralView ArticlePubMed
- Steg PG, Lopez-Sendon J, Lopez de Sa E, Goodman SG, Gore JM, Anderson FA, Himbert D, Allegrone J, Van de Werf F: External validity of clinical trials in acute myocardial infarction. Arch Intern Med. 2007, 167 (1): 68-73. 10.1001/archinte.167.1.68.View ArticlePubMed
- Antes G: [The evidence base of clinical practice guidelines, health technology assessments and patient information as a basis for clinical decision-making]. Z Arztl Fortbild Qualitatssich. 2004, 98 (3): 180-184. discussion 190-182, 214-185PubMed
- Gross CP, Mallory R, Heiat A, Krumholz HM: Reporting the recruitment process in clinical trials: who are these patients and how did they get there?. Ann Intern Med. 2002, 137 (1): 10-16.View ArticlePubMed
- Jones R, Jones RO, McCowan C, Montgomery AA, Fahey T: The external validity of published randomized controlled trials in primary care. BMC Fam Pract. 2009, 10: 5-10.1186/1471-2296-10-5.PubMed CentralView ArticlePubMed
- von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vandenbroucke JP: The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet. 2007, 370 (9596): 1453-1457. 10.1016/S0140-6736(07)61602-X.View ArticlePubMed
- Seiler CM, Deckert A, Diener MK, Knaebel HP, Weigand MA, Victor N, Buchler MW: Midline versus transverse incision in major abdominal surgery: a randomized, double-blind equivalence trial (POVATI: ISRCTN60734227). Ann Surg. 2009, 249 (6): 913-920. 10.1097/SLA.0b013e3181a77c92.View ArticlePubMed
- Rothwell PM: External validity of randomised controlled trials: "to whom do the results of this trial apply?". Lancet. 2005, 365 (9453): 82-93. 10.1016/S0140-6736(04)17670-8.View ArticlePubMed
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2482/12/2/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.