RLO Transcript: Determining the Clinical Importance of Trial Results
This is a transcript of the Reusable Learning Object (RLO) entitled "Determining the Clinical Importance of Trial Results", online at:

http://www.nottingham.ac.uk/nursing/sonet/rlos/ebp/trial_results/
Introduction

When making decisions in health care it is important to be able to identify and interpret the clinical significance of trial results. Depending on the method of presenting the data, the same intervention can be seen to have a large or small effect. This learning object assumes you are already familiar with the concepts of surrogate outcomes, absolute and relative risk reduction, number needed to treat, and confidence intervals. If you need a reminder of these, follow the links to the respective RLOs [see the Resources at the end of this transcript]. 
Scenario

Consider the following scenario, outlined in a hypothetical news article:

‘NEW DRUG OFFERS HOPE TO MIGRAINE SUFFERERS’
Migraine, a severe disabling headache, brings untold misery to many thousands of people in the UK, and each year millions of working days are lost.  Doctors today have announced that compared to standard treatment, a new drug, NoMig, reduces the number of patients who still have a headache after 2 hours by 2/3rd

Would you recommend this drug on the basis of this report?  Move the cursor on the thermometer.

The newspaper is only reporting the relative benefit of treatment. This can be misleading as it does not indicate the actual size of effect. More information is needed.

Search for information

The GP decides to look for further information using the internet and finds the following two articles:

1.	Binks, J (2005). A comparison of NoMig with standard treatments for the relief of migraine. Annals of Migraine; 3: 131-135.
2.	Smith, P (2003). Pharmacokinetics of NoMig in patients with migraine. International Journal of Headache. Suppl 1, 7-11.

Reference 1 looks at an outcome that is important to the patient and is clinically relevant. Reference 2 looks at a surrogate outcome. This is of academic interest but does not provide information on outcomes for the patient. Reference 1 is therefore more useful.
Data

The trial showed that 30 out of 300 patients still had migraine symptoms 2 hours after talking NoMig, compared with 81 out of 270 patients who took the standard treatment. 

Using this information complete the table of results shown:
	Migraine present after 2 hours 	Total No. pts	
	Yes	No		
NoMig	30	270	300	
Standard treatment	81	189	270	

What is the risk of having a migraine present after 2 hours in the standard treatment group - Control Event Rate (CER)? [Answer: 30%]

What is the risk of having a migraine present after 2 hours in the NoMIG group -Experimental Event Rate (EER)? [Answer: 10%]
Calculate the ARR and NNT

Use the experimental event rate and control event rate to calculate the absolute risk reduction and the number needed to treat. [Answer: ARR 20%, NNT 5]

Would you now recommend this drug with this further information? Adjust your recommendation on the thermometer. 
Further information
Other information may help substantiate or refute your decision including those shown here. Rank these in order with the one that you think is most useful first.

confidence intervals for ARR………rank =  [Answer: 2]
RRR ………………………………….rank =  [Answer: 3]
NNH (safety)….……………………  rank = [Answer: 1]

Then click on the information button to find out what these values are:

confidence interval for ARR is 13.5% - 26.5%
and confidence interval for NNT is 4 – 7
RRR = 67%
NNH = 3

Rate your confidence regarding your decision on the thermometer again.

Explanations
Has the thermometer changed?  Why is this? See if you agree with the explanations. 

Information	Useful	Explanation	
NNH	Tick	If 100 patients are given NoMig rather than standard treatment, 15 more would have their headache relieved at 2 hours, but 33 would suffer a side effect.  	
Confidence Interval	Tick	The confidence intervals for ARR are narrow and don’t include zero. This indicates a significant difference between the two treatments. The confidence intervals for NNT are 4 to 7 so up to 7 patients would need to be treated with the new drug rather than the standard treatment in order for one patient to benefit.  You and the patient would need to decide if this is clinically acceptable.	
RRR	Tick ??	A 67% reduction in the chance of still having a headache at 2 hours sounds impressive, but is misleading as it doesn’t tell you anything about the size of the effect.	

Summary
It is important to consider all of the measurements discussed in combination, rather than in isolation. In doing this we can interpret information on the relevance, size and precision of the effect of an intervention. However this is not the whole story. Other factors we would need to consider are cost of treatment, what treatments the patient has tried before, individual patient risks, patient attitudes and beliefs………

Resources
Title/link
	Description
	
Using Evidence to Guide Practice MeReC Briefing No. 30. September 2005 National Prescribing Centre (PDF)		
Using Evidence to Guide Practice - Supplement MeReC Briefing No. 30. September 2005 National Prescribing Centre		
Confidence Intervals (RLO)	Defines the term 'confidence intervals' and demonstrates how they can be used to determine the significance and range of possible sizes of a treatment effect.
	
Number needed to treat and number needed to harm (RLO)	Considers how to measure and interpret the magnitude of effect in clinical trial results using number needed to treat (NNT) and number needed to harm (NNH).	
Positive and predictive value of diagnostic tests (RLO)	Explains how diagnostic test results are a combination of true and false positive, or true and false negative.	
Relative Risk Reduction and Absolute Risk Reduction (RLO)	This RLO considers how to measure and interpret the magnitude of effect in clinical trial results using relative risk reduction (RRR) and absolute risk reduction (ARR).
	
Surrogate Outcomes (RLO)	Considers the type of evidence which should be used when making decisions about patient care.

Glossary
Surrogate Outcome - should reflect the treatment effect on an important clinical endpoint, but does not directly measure the clinical endpoint itself.

Control event rate (CER) = proportion of patients who experience an outcome in the control group

Experimental event rate (EER) = proportion of patients who experience an outcome in the experimental group

Relative risk reduction (RRR) = difference in event rates relative to (or proportional to) the control event rate, expressed as a percentage 
                                       RRR = (CER – EER) / CER

Absolute risk reduction (ARR) = difference between the control event rate and the experimental event rate, expressed as a percentage
                                        ARR = CER – EER

Number needed to treat (NNT) = the number of patients that would need to be treated in order for ONE on them to have the beneficial outcome.
It is calculated by calculating the reciprocal (or inverse) of the ARR
                                       NNT = 1 / ARR
Note: if the ARR is expressed as a percentage (%) then:
                                       NNT = 1/ARR x 100 
                                              or 100/ARR 

Number needed to harm (NNH) = the number of patients that would need to be treated in order for ONE on them to experience the adverse outcome. 
It is calculated by calculating the reciprocal (or inverse) of the absolute risk increase (ARI)
                                       NNH = 1/ARI
Note: if the ARI is expressed as a percentage (%) then:
                                       NNH = 1/ARI x 100
                                               or 100/ARI 

Confidence Interval – tells you the range of values within which the true value could feasibly lie, given the size of the difference observed.

Document updated: 14/03/2007