Article: Indications for Computed Tomography in Patients with Minor Head Injury
Authors: Haydel, M, Preston CA, Mills, TJ, Luber S, Blandeau E, DeBlieux PMC
Publication: New Engl J Med. 2000; 343(2): 100-105.
Step 1: Evaluate the study design
Step 1a: Will the rule will serve a purpose if it is valid? Does the rule make clinical sense?
The investigators state their purpose several times throughout the paper: "We conducted a study to derive and validate a set of clinical criteria that could be used to identify patients with minor head injury in whom CT could be forgone." They desired the rule to have a 100% negative predictive value. They state this goal in the abstract and introduction. Twice in the discussion they indicate that the purpose of the decision rule is to identify patients with minor head injury who should undergo CT scanning with 100% sensitivity. These 2 purposes are not identical and it would have been better to pick one (probably to decide "in whom CT could be forgone" as discussed later) and stick with it.
The CDR simply stated would be: Patients who have experienced a minor head injury (defined by the authors as loss of consciousness with a GCS of 15 and a normal neurological exam) do not need to undergo CT unless they have one or more of the following findings: age over 60 years old, intoxication, headache, vomiting, short-term memory deficits, evidence of trauma above the clavicles, and seizure activity following the trauma.
The rule makes clinical sense and would serve a purpose. Prior to the publication of this paper no similar rule with a reported 100% negative predictive value existed. A CDR that would help clinicians decide whom with a minor head injury could safely avoid having a CT scan does seem worthwhile. This particular clinical scenario seems ideal for the development of a CDR because it is complex (there are many different factors which go into deciding who needs a CT), the stakes are high (even one missed epidural hematoma has huge negative consequences), and there is the potential for savings. Excessive CT scanning is expensive and significantly slows down the flow of emergency departments (delaying the care of other patients who require a CT scan). If the rule proves to be valid, clinicians would order roughly 20% less CT scans without untoward outcomes, which would reduce costs and save time in busy trauma centers (according to the authors).
Step 1b: Is the outcome of interest clearly defined and clinically important?
The outcome of interest, a positive CT scan, is clearly defined as one showing a subdural, epidural, or parenchymal hematoma, subarachnoid hemorrhage, cerebral contusion, or depressed skull fracture. The question of the clinical importance of a positive CT scan is controversial. The majority of patients with these lesions are observed and discharged. Less than 1% of patients with positive CT scans require neurosurgery. Some clinicians may believe it is acceptable to not identify some of these lesions if they are not clinically apparent, because no intervention would be undertaken anyway. Other clinicians may feel it is important to detect every lesion because a small percentage of these lesions may evolve into something more serious and because they may explain future morbidity (headaches, cognitive changes, etc). Since this controversy will not resolve soon, we will consider a positive head CT to be clinically important.
Step 1c: Are the predictors clearly defined, sensible, reliable, and reproducible as determined by different clinicians?
Here we must look at the predictors individually. The authors proposed 8 predictors to derive the CDR. The investigators did test interobserver reliability for the determination of predictors on 50 patients, however they do not report the results for individual predictors.
1) age over 60: yes
2)presence or absence of headache: yes
3) presence or absence of vomiting: yes
4) drug or alcohol intoxication: not as clearly defined (" determined on the basis of history obtained from the patient or a witness and suggestive findings on physical examination, such as slurred speech or the odor of alcohol on the breath"). They mention that alcohol and drug tests could be ordered at the discretion of the clinicians, but not whether they were used when collecting data on predictors. The presence or absence of intoxication seems a sensible predictor to include, considering the known higher predisposition of alcoholics to form subdural hematomas with minor head injury. The assessment of intoxication in a head injured patient is probably not very reliably or reproducibly determined by the same clinician let alone different clinicians. Many of the findings of a head-injured and intoxicated patients are similar.
5) deficits in short-term memory: even less clearly defined than intoxication ("persistent anterograde amnesia" in a patient with a GCS 15). This predictor does not seem as sensible or as reliably or reproducibly determined by different clinicians. To discover "persistent anterograde amnesia" one would have to examine the patient multiple times and know what their mental status was like prior to the traumatic event.
6) post-traumatic seizure: defined as suspected or witnessed seizure after the traumatic event. There are some potential pitfalls here--syncope jerks may appear like seizures. What defines a suspected seizure is not mentioned (tongue laceration? incontinence?) For these reasons seizure activity may not be reliably or reasonably determined by different clinicians. Even so, it seems sensible to include seizure activity as a potential predictor for a positive CT scan
7) physical evidence of trauma above the clavicles: clearly defined as any external evidence of injury (contusions, abrasions, lacerations, deformities and signs of facial or skull fractures). For the most part this predictor seems sensible, but one can imagine the scenario of a patient with a small abrasion on their nose and wonder...This predictor should be reliably and reproducibly determined by different clinicians apart from the possibility of posterior head trauma that is not evident until removal of a c-spine collar.
8) coagulopathy: yes
Step 1d: How did the authors construct the list of potential predictors? Did the authors include all elements that might have predictive power when creating the rule?
The authors constructed the list of predictors based on "extensive review of the literature on minor head injury." They don't specify which literature yielded these predictors. They include all of the expected predictors of positive head CT except for possibly infancy and mechanism of injury. However infants were not studied in their population. Coagulopathy as a predictor was excluded from the rule. There were not enough patients in their study sample with this predictor to show its relationship with a positive head CT. Most clinicians would still adopt the policy of scanning patients with coagulopathy and minor head injury. The other 7 predictors were chosen for use in the rule after phase 1 of the study based on the recursive partitioning analysis.
Step 1e: Were those assessing predictors blinded to outcomes (and vice versa)?
The investigators mention that data on predictors were collected prior to CT scanning, so that we can assume the study was blinded in this aspect. They do not explicitly say that the neuroradiologists interpreting the scans were unaware of the patients clinical status, so we must assume that the study was not blinded in this regard.
Step 1f: Was the study site described?
The study site is well described as a level 1 trauma center and teaching institution. The clinical setting is well described as an emergency department. The referral filter was consecutive patients with minor head injury seen in the emergency department during the specified study period.
Step 1g: Was the data collected prospectively?
Yes, for both phases 1 and 2 of the study.
Step 2: Evaluate the results
Step 2a: Does the rule have predictive power? Look at the results and statistical method(s) used to derive the rule. Were all of the relevant results reported and described well? Was the statistical analysis appropriate?
The rule's predictive power is reported in terms of sensitivity and specificity. The rule showed 100% sensitivity (95% CI 95-100%), which means it does have strong negative predictive power (meaning if the patient lacks the 7 predictors plus coagulopathy as described above, it is pretty much guaranteed that they will have a negative head CT). The specificity of the rule was only 25% (95% CI 22-28%). Almost all of the relevant results were reported and displayed well for phase 1. The predictor data was not displayed for phase 2. It would have been useful for the authors to report any associations among the predictor variables with one another. If 2 predictors were highly associated with each other (eg intoxication and anterograde amnesia), then the inclusion of both in the rule might not be necessary. The statistical analysis, which included recursive partitioning, was very appropriate considering that the investigators wanted to derive a rule with 100% sensitivity.
Step 2b: Was there an adequate sample size, including an adequate number of positive outcomes?
The answer to this question most likely is no. Even though 520 patients were included in the first phase and 909 in the second, there was a low number of positive outcomes (36 in phase 1 and 57 in phase 2). The number of patients in this study was probably not enough for the clinical subject at hand. There were only 5 positive outcomes for each predictor in phase 1 and 8 positive outcomes for each predictor in phase 2. The authors' goal was to create a CDR with 100% sensitivity and this was not statistically possible (ie, the confidence interval is still too wide) without larger numbers of subjects. This problem could be remedied by a larger validation sample.
Step 2c: Were the important predictors present in a significant proportion of the study population?
Certain predictors (intoxication, physical evidence of trauma above the clavicles and headache) were present in a sizable proportion of the study sample while others (age over 60, coagulopathy, vomiting, seizure and short-term memory deficit) were not. 697 (~77%) of patients in phase 2 had at least 1 of the 7 predictors. This data is not reported for phase 1.
1) age over 60: Only 42 (8%) patients in phase 1.
2) intoxication: 180 (35%) patients in phase 1.
3) physical evidence of trauma above the clavicles: 338 (65%) patients in phase 1.
4) short-term memory deficits: 9 (2%) patients in phase 1.
5) seizure: 24 (5%) patients in phase 1.
6) headache: 123 (24%) patients in phase 1.
7) vomiting: 47 (9%) patients in phase 1.
8) coagulopathy: 1(<1%) patients in phase 1.
Step 2d: Were important patient characteristics reported?
Age and sex were reported. Mechanism of injury was not reported.
Step 2e: Was interobservor variability of the rule assessed and reported?
As mentioned above interobserver reliability for predictors was assessed and reported. The investigators tested interobserver variability by having 2 different radiologists interpret the CT scans on a sample of 50 patients.The kappas for determination of predictors (k=0.78) and outcomes (k=0.94) showed good interobserver reliability.
Step 3: Has the rule's validity been tested?
Yes, in phase 2 of the study the rule was applied to 909 patients. All of the patients had CT scans. The authors recommend that further validation studies be performed.
Step 3a: Was the rule applied to a narrow or broad patient sample?
The validation sample was larger (909) than the derivation sample (520). However, the validation sample would still be considered a narrow population because it was similar to the derivation sample (same clinical setting, institution, and most likely clinicians). In fact the validation sample had the same mean age (36 years) and proportion of males (65%) as the derivation sample. A broad validation sample will be extremely important for this CDR for the following reasons. The rule, if applied, may actually increase the number of CT scans in certain settings, such as hospitals which are not level 1 trauma centers and in Europe and Canada where clinicians do not order CT scans on all patients with minor head injury, loss of consciousness, a GCS of 15 and a normal neurologic exam (1, 3). Other clinicians feel it may not be safe to practice according to the rule based on their personal clinical experience (of patients having positive head CTs despite lacking all 7 predictors) (4).
Step 3b: Were the patients in the validation sample chosen in an unbiased manner?
Yes, consecutive patients were studied.
Step 3c: Were different clinicians involved in the validation study? Was interobserver reliability of the rule assessed and reported?
We can assume that the same clinicians were involved in the validation phase of the study, because it is not mentioned that they were not. It is unclear whether interobserver variability was assessed for phase 1 or 2 or both, but it is assumed here that this was only done in the derivation sample.
Step 3d: Were those assessing predictors blinded to outcomes (and vice versa)?
The answer here is the same as for step 1e.
Step 3e: Was the rule applied in a prospective or retrospective fashion?
Prospective. However, all patients, regardless of number of predictors, underwent CT.
Step 3f: Was the rule consistent and valid when applied?
Yes, the rule yielded a 100% negative predictive value for a negative CT scan in patients without any of the 7 predictors. Threats to the rule's validity as mentioned above were: the CDR was applied to a narrow population in the validation phase, coagulopathy as a predictor was excluded, and the sample size may not have been large enough.
Step 3g: Was there excessive loss to follow-up?
There was no loss to follow-up for the determination of the chosen outcome (CT result). There was no follow-up of patients who had negative head CTs in either phase of the study.
Step 4: Did the rule have an impact? Has an impact analysis been performed?
No. An impact analysis was not performed.
Step 5: Grade the rule
This CDR receives a 3 on the hierarchy of evidence, because it has not been validated in a broad sample and an impact analysis has not been performed. A clinician could reasonably consider using it with caution in patients similar to those studied.
References
1. Stiell IG, Laupacis A, Wells GA, et al. Indications for computed tomography after minor head injury. Canadian CT Head and Cervical-Spine Study Group. New Engl J Med. 2000;343(21):1570-1571.
2. Haydel M. Indications for computed tomography after minor head injury. New Engl J Med. 2000;343(21):1571.
3. Kohn M, Newman TB. Indications for computed tomography after minor head injury. New Engl J Med. 2000;343(21):1571.
4. Stein SC. Indications for computed tomography after minor head injury. New Engl J Med. 2000;343(21):1570.