RLO Transcript: Interpreting Confidence Intervals
This is a transcript of the Reusable Learning Object (RLO) entitled "Interpreting Confidence Intervals", online at:

http://www.nottingham.ac.uk/nursing/sonet/rlos/ebp/confidence_intervals
Introduction
In a clinical trial, there is always a possibility that the effect seen may have occurred by chance.  Confidence intervals are a useful approach to assessing the role of chance.

Consider the following scenario.  A trial is being carried out to investigate the effectiveness of drug A in preventing heart attacks.  10 patients receive drug A, and 2 of these patients have heart attack within the next 12 months. 10 patients receive placebo and 3 of these patients have a heart attack within the next 12 months.  

Drug A has reduced the risk of a heart attack by one third, in other words, a relative risk reduction of 33%. It’s looking good.  On the basis on this trial, would you be happy to recommend this treatment for your patients? 
The role of chance
If this trial was repeated in another group of 10 similar patients, would you expect to see exactly the same results again? Would it be surprising if, for example, 1 patient in each group had a heart attack?  Would it be surprising if 2 patients in the placebo group and 3 patients in the treatment group had a heart attack?

Try changing the values in the table and look at the effect on the relative risk reduction. Note how values for the relative risk reduction changes. The true effect on relative risk reduction is somewhere within this range of values and may be larger or smaller than the effect observed in the original  trial. This range, within which the true value could plausibly lie given the size of the difference actually observed, is called the confidence interval. 
95% Confidence Intervals
Confidence intervals should be reported in a clinical trial, but see the resources at the end if you want to know how  to calculate them yourself. The actual calculated confidence interval for the relative risk reduction in the trial described above is -4.4% - 70%.  

Confidence intervals are created at the 95% level – this means that there is a 5% chance that the true value lies outside this range.
Assessing statistical significance from a confidence interval
Confidence intervals can be calculated for any effect measure, not just relative risk reduction, and can be used to determine if the size of the effect is significant.  If the confidence interval includes the value reflecting “no effect” we cannot rule out the possibility that the intervention has no effect and the result is statistically non-significant. In the table opposite, drag and drop the values of “no effect”  into the appropriate place.(Pause)

Now let us go back to our trial where the relative risk reduction was 33% (display white box).  It looked good, but was it statistically significant, or might it have occurred by chance? Click on “yes” or “no” to indicate your opinion.
What about continuous data?
Some trials don’t measure effect in terms of “did it happen or not”, so called dichotomous data.  Instead they measure the effect on a continuous measure such as blood pressure, or months of survival.  Confidence Intervals can also be calculated for these measures, and used to assess significance of the result.

Consider a trial comparing the effect of drug X and placebo on FEV1¬  in patients with chronic obstructive airways disease.  At the end of the trial, the FEV1 of patients who received drug X has, on average, declined 50ml less than in those who received placebo.  The 95% confidence interval is 5mL-100mL (display white box).    Is this result statistically significant? Click on “yes” or “no” to indicate your opinion.

Is this result clinically significant? Click on “yes” or “no” to indicate your opinion.
The width of the confidence interval
In our heart attack example, the true effect of the intervention lies within a wide range and the trial is not very informative.  In the table on the right, look at what happens when our trial is repeated in a larger number of patients. The relative risk reduction has stayed the same, but try manipulating the figures as before and see what happens. (Pause)

You can see that not only has the trial now reached statistical significance, but the confidence intervals become narrower with a bigger trial.  The narrower the confidence interval, the more sure you can be of the true size of the effect of the intervention.
Tables
The tables below are referred to in the transcript above. 

	No. of patients	No. of patients with heart attack	Relative risk reduction	
Drug A	10	2	33%	
Placebo	10	3		
Table One

	No. of patient	No. of patients with heart attack	Relative risk reduction	
Drug A	10	1	0%	
Placebo	10	1		
Drug A	10	1	67%	
Placebo	10	3		
Drug A	10	1	80%	
Placebo	10	5		
Drug A	10	2	-100%	
Placebo	10	1		
Drug A	10	2	33%	
Placebo	10	3		
Drug A	10	2	60%	
Placebo	10	5		
Drug A	10	3	-200%	
Placebo	10	1		
Drug A	10	3	0%	
Placebo	10	3		
Drug A	10	3	40%	
Placebo	10	5		
Table two

Measure of effect	
This column would be blank. The student needs to drag and drop the appropriate figure into the box. The figure only stays there is it is correct

No effect	
Relative risk	1	
Relative risk reduction	0%	
Absolute risk reduction	0%	
Number needed to treat	infinity	
Table three

0%

1

0%

∞

		No. of patients	No. of patients with heart attack	Relative risk reduction	
Bigger trial	Drug A	100	20	33%	
	Placebo	100	30		
Even bigger trial	Drug A	1000	200	33%	
	Placebo	1000	300		
Table 4

	No. of patient	No. of patients with heart attack	Relative risk reduction	
Drug A	100	20	33%	
Placebo	100	30		
Drug A	100	25	17%	
Placebo	100	30		
Drug A	100	15	50%	
Placebo	100	30		
Drug A	1000	200	33%	
Placebo	1000	300		
Drug A	1000	230	23%	
Placebo	1000	300		
Drug A	1000	170	43%	
Placebo	1000	300		

Figure 1 

Table 5