Do we know the effectiveness of behavioural interventions?

Sevgi O Aral, Thomas A Peterman

Lancet 1998; 351 (suppl III): 33-36

Division of STD Prevention, National Center for HIV, STD, and TB Prevention, Centers for Disease Control and Prevention, Atlanta, GA 30333, USA (SO Aral PhD, TA Peterman MD)

Correspondence to: Dr Sevgi Aral

During the past decade the increased emphases on "evidence-based medicine", "quality of evidence", and "prevention effectiveness studies" raised the question of how well we know the effectiveness of our prevention efforts. Ultimately, our knowledge of the effectiveness of behavioural interventions is determined by the comprehensiveness of the behavioural-intervention research that has been done, the representativeness and completeness of published research findings, and the scientific rigour and appropriateness of the research. Behavioural-intervention research into the prevention of sexually transmitted diseases (STDs) and HIV started only in the late 1980s and has been greatly under funded. In fiscal year 1996, only 8·1% of the funds allocated to AIDS research by the US National Institutes of Health were for behavioural intervention research, and in fiscal year 1997 this proportion decreased to 7·7% (J Auerbach, Office of AIDS Research, NIH, Bethesda, MD, USA, personal communication). It is reasonable to expect that other research institutes in the USA and elsewhere allocated even fewer resources to evaluating the effectiveness of behavioural interventions that prevent the spread of STDs and HIV.

The research that has been conducted on behavioural interventions to date is spread across a variety of populations (eg, heterosexual, homosexual, and drug-using populations), outcome measures (eg, knowledge, attitudes, behaviours, and STD or HIV incidence or prevalence), types of intervention (eg, delivered one-on-one, in dyads, in small or large groups), content of intervention (eg, delivery of information, delivery of materials, skills training, delivery of services, referrals to professional services), intervention duration (eg, less than an hour, 2-4 hours, or 5 hours or more), and number of intervention sessions (eg, one, two, three, or more sessions). Thus, in general, findings are not sufficiently replicated.

The quality and completeness of reporting of behavioural intervention research in the peer-reviewed scientific literature is also an issue; published reports may not cover fully all relevant aspects of the studies, and published studies may be a select sample of all studies reflecting publication bias. First, when published, behavioural and biomedical intervention studies need to be completely and reliably reported.¹ One methodological review found that only 72% of studies provided information to replicate the intervention, 85% stated the numbers recruited, 44% provided preintervention data, 76% discussed attrition, 76% discussed all outcomes, and 65% provided postintervention data for all study groups.² In the absence of adequate reporting, assessing quality becomes impossible.³ Journals and editors could help by allowing space for explicit descriptions of the study population, recruitment and sampling methods, participation rates, power calculations, randomisation procedures, treatment of control groups, blinding procedures, attrition, content of the intervention, exposure to the intervention, cost of the intervention, intention-to-treat analyses, intervention exposure analyses, and outcome measurement.¹

Second, all well-conducted intervention studies, both those with null findings and those that obtain statistically significant intervention effects need to be published. Any tendency on the parts of investigators or editors to fail to publish study results on the basis of the direction or strength of the study findings leads to publication bias.⁴ The earliest study of publication bias reviewed four prominent psychology journals and reported that more than 95% of articles that focused on hypothesis testing reported statistically significant, positive findings.⁵ Results were very similar when study was replicated in the same four journals in 1986 and 1987, and 85% of articles in medical journals were found to report statistically significant results.⁴ Since that time evidence has accumulated to show that studies reaching publication are a select sample of all studies conducted.^6,7 There is now considerable evidence indicative of reporting bias in all stages of study findings including acceptance of abstracts for presentation at meetings, publication of studies initially presented as abstracts, publication of initiated studies, and reporting of published results by lay press. Although no studies have been conducted of the STD/HIV prevention literature, our knowledge of the effectiveness of behavioural interventions may be greatly affected by bias in the reporting of findings from intervention research.  

What we need to know

The effectiveness of an intervention is the impact an intervention achieves in the real world, under resource constraints, in entire populations, or in specified subgroups of a population. It is the improvement in a health outcome achieved in a typical community setting. Efficacy is the improvement in health outcome achieved in a research setting, in expert hands, under ideal circumstances. Thus, effectiveness of an intervention is the product of its efficacy, the penetration or reach of the intervention into the population, and the compliance of the population with the intervention.⁸ In general, effectiveness is lower than efficacy, though theoretically it could be higher, particularly for infectious diseases, due to the effect of transmission dynamics.

When any intervention is evaluated at the community level, the impact of the intervention can be measured directly in terms of the reduction in the incidence of a specific adverse health outcome; for example, in the community randomised trial conducted in the Mwanza Region in Tanzania, improved STD treatment reduced HIV incidence by about 40%.⁹ Most studies do not explicitly measure impact or effectiveness. In these cases, potential impact needs to be estimated to provide an indicator of effectiveness. Preventable fraction can be estimated based on attributable risk or the incidence of the health outcome that can be attributed to a specific risk factor. Adjustments need to be made for compliance of the population with the intervention, penetration of the intervention into the population, and risk incurred before implementation of the intervention.

The estimation of potential impact in the community is not straightforward in the case of HIV and STDs in general (panel). Prevention-effectiveness-estimation methods have mostly been worked out for chronic diseases and tend not to take into account the dependence of one individual's health outcome on the health outcome of other individuals.¹⁰ If an individual's sexual partners are uninfected, the individuals's risk of infection is zero. Each prevented infection also prevents a variable number of secondary infections. To obtain a more accurate estimate of the impact of STD-prevention strategies on morbidity, an additional correction that reflects the transmissible nature of STDs is necessary; without this correction the impact of behavioural interventions tend to be underestimated.

In the absence of community randomised trials, it is difficult to determine what infection rates would be with and without the intervention; therefore, attribution of changes in morbidity to the effects of an intervention is problematic. Prevalence and incidence of STDs or HIV may increase or decline as a reflection of natural progression of epidemics,¹¹ or as a result of changing behaviours where behaviour changes may be unrelated to the intervention. The optimal way to control for the effects of the natural progression of the epidemic and the behaviour changes unrelated to the intervention is to employ randomised controls.

What we need to know is the impact of behavioural interventions on STD/HIV morbidity in the population. This impact is ideally measured directly through community randomised trials. In the absence of such trials, potential impact may be estimated based on considerations of attributable risk, penetration, compliance, previous risk, transmission dynamics of communicable disease, and intervention efficacy, where efficacy is measured through individual or group randomised trials with STD incidence as an outcome measure. Moreover, intervention impact probably varies by the profile of the epidemic, behavioural patterns of the population, and existing policies, and needs to be assessed specifically in these three contexts.  

What we know

Available evidence on behavioural interventions indicates that in many study populations, interventions of variable content may lead to desired changes in knowledge, attitudes, perceptions, self-efficacy, skills, and self-reported behaviours including male condom use, number of sex partners, and practice of unprotected sex.^12-23 Most behavioural interventions are delivered to individuals and change in self-reported behaviour is measured at the individual level. Only a few behavioural-intervention studies have measured effect on the incidence of HIV or STD. Project Respect and Project Light are two important studies that have measured biomedical outcomes.^24,25. Thus, in most cases we have evidence of efficacy of behavioural interventions in changing self-reported behaviours, which may not be valid surrogates for incidence of infection. We need further evidence on: efficacy of behavioural interventions to change the incidence or prevalence of STDs including HIV, indirect assessment of potential impact of behavioural interventions on community infection rates, and evidence of impact of behavioural interventions on STD/HIV morbidity in populations.

Design issues

Efficacy of behavioural interventions can be optimally measured through randomised controlled trials.²⁶ Bias in the adoption of behaviour change and in intervention assignment is inherently a part of the practice of medicine and public health and cannot be dealt with by simply adjusting for the effects of known confounding factors.²⁷

Issues of outcome measurement

During the past decade, self-reported measurement of sensitive behaviours has improved greatly.²⁸ Nevertheless, use of self-reports of behaviour is problematic in that interventions may change reports of behaviour without changing behaviour itself.²⁸ Perhaps more importantly, the nature of the relation between self-reported sexual or preventive behaviours and STD/HIV outcomes is not straightforward. At the individual level, the magnitude of risk associated with particular behaviours depends on the infectiousness of the partner. At the population level, the relation between behaviour and STD incidence varies across the phases of the epidemic.¹¹ Moreover, individuals who change one risk behaviour often change other risk behaviours in countervailing directions, necessitating the development and validation of summary measures that reflect the net effects of these changes.²⁹ In this context, it is important to remember that the purpose of behavioural interventions is prevention of disease, not behaviour change.

Individual and community trials

Most studies of behavioural interventions focus on individuals. Individual-level effects, even when they measure biomedical outcomes, cannot be translated directly into population-level impact. Individuals who participate in studies may differ from those who do not, and, depending on their placement in sexual and social networks, individuals who change their behaviours and their infection risk may have a greater or a lesser effect on the infection risk of others. Focusing behavioural interventions on individuals has other practical implications. Individuals, after successfully changing their behaviours to reduce risk in the study context, may return to their typical life situation and relapse into earlier risky behaviours necessitating repeat applications of sometimes very costly behavioural interventions. Individual members of new cohorts who continually enter societal situations that foster risk also require repeated application of behavioural interventions.

Behavioural-intervention research in comparative historical perspective

The field of behavioural-intervention research is remarkable in the extent to which it has produced self-critical reviews and responded to them by great improvements in methodology and focus (figure). Among abstracts from international AIDS conferences between 1989 and 1992, there were ten reporting on randomised controlled trials of behavioural interventions.²⁶ An assessment of articles included in the Cochrane systematic database (CSD) indicated that between 1994 and 1997 there were 23 articles focusing on HIV and STD and five articles focusing on STD excluding HIV that employed randomised controlled designs. Over the past decade, use of self-reported behaviours, rather than only perceptions, attitudes, intentions, and knowledge as outcome measures has also increased: of studies included in the CSD, only two conducted between 1990 and 1991, and none conducted since then, failed to include behaviours as outcome measures. Unfortunately the use of biomedical outcome measures has not shown a similar trend: since 1990 only two studies in the CSD have measured incident STDs as their outcome.

Evolution of behavioural-intervention development and implementation

Failure to use STD and HIV incidence as the outcome measure constitutes a major weakness in the behavioural-intervention area. Another important weakness is our failure to evaluate basic social structural interventions. We cannot know the effectiveness of interventions
that have not been addressed. There is overwhelming evidence that oppression contributes to STDs and many other maladies. Yet we have failed to see prejudice against single-sex marriage, racism, and human-rights violations as issues that require public-health interventions.

It was recently suggested that research in general should be evaluated based on how often it is cited in the literature and how much it cost ("citations attracted per million pounds spent on research").³⁰ This idea is attractive for its simplicity, but the goal of prevention research is disease prevention, not citation. Prevention research has a big impact when interventions that prevent disease are developed and then adopted by prevention programmes. Thus, a more meaningful evaluation of the impact of research would be "amount of disease prevented in the population per pound spent on research". HIV and STDs are tremendously important health problems and behavioural-intervention research could score well on this scale. Some interventions have been shown to prevent disease in research settings. They need to be implemented to have an effect on the population. Many more interventions need to be developed, tested, implemented, and refined. Greater resources need to be invested toward this goal. We know more about the effectiveness of behavioural interventions than we did a few years ago, but we still have a lot to learn.

References

1 O'Leary A, DiClemente, Aral SO. Reflections on the design and reporting of STD/HIV behavioral intervention research. AIDS Educ Prev 1997; 9 (suppl A): 1-14.

2 Oakley A, Fullerton D, Holland J. Behavioural interventions for HIV/AIDS prevention. AIDS 1995; 9: 479-86.

3 Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. JAMA 1995; 273: 408-12.

4 Dickersin K. Publication bias: the problem that won't go away. Ann N Y Acad Sci 1993; 703: 135-45.

5 Sterling TD. Publication decisions and their possible effects on inferences drawn from tests of significance--or vice versa. J Am Stat Assoc 1959; 54: 30-34.

6 Begg CB, Berlin JA. Publication bias: a problem in interpreting medical data. Stat Soc 1998; annex 151: 419-63.

7 Dickersin K. The existence of publication bias and risk factors for its occurrence. JAMA 1990; 263: 1401-05.

8 Haddix AC, Teutsch SM, Shaffer PA, Dunet DO. Prevention effectiveness. Oxford: Oxford University Press, 1996

9 Grosskurth H, Mosha F, Todd J, et al. Impact of improved treatment of sexually transmitted diseases on HIV infection in rural Tanzania: randomised controlled trial. Lancet 1995; 346: 530-36.

10 Koopman J. Emerging objectives and methods in epidemiology. Am J Public Health 1996; 86: 630.

11 Wasserheit JN, Aral SO. The dynamic topology of sexually transmitted disease epidemics: implications for STD prevention strategies. J Infect Dis 1996; 174 (suppl 2): S201-13

12 Boyer CB, Barrett DC, Peterman TA, Bolan G. Sexually transmitted disease (STD) and HIV risk in heterosexual adults attending a public STD clinic: evaluation of a randomized controlled behavioral risk-reduction intervention trial. AIDS 1997; 11: 359-67.

13 O'Donnell L, San Doval A, Duran R, O'Donnell CR. The effectiveness of video-based interventions in promoting condom acquisition among STD clinic patients. Sex Transm Dis 1995; 22: 97-103.

14 Eldridge GD, St Lawrence JS, Little CE, et al. Evaluation of the HIV risk reduction intervention for women entering inpatient substance abuse treatment. AIDS Educ Prev 1997; 9 (suppl A): 62-76.

15 Peterson JL, Coates TJ, Catania J, et al. Evaluation of an HIV risk reduction intervention among African-American homosexual and bisexual men. AIDS 1996; 10: 319-25.

16 St Lawrence JS, Brasfield TL, Jefferson KW, Alleyne E, O'Bannon RE III, Shirley A. Cognitive-behavioral intervention to reduce African-American adolescents' risk for HIV infection. J Consult Clin Psychol 1995; 63: 221-37.

17 St Lawrence JS, Jefferson KW, Alleyne E, Brasfield TL. Comparison of education versus behavioral skills training interventions in lowering sexual HIV-risk behavior of substance-dependent adolescents. J Consult Clin Psychol 1995; 63: 154-57.

18 Kalichman SC, Sikkema KJ, Kelly JA, Bulto M. Use of a brief behavioral skills intervention to prevent HIV infection among chronic mentally ill adults. Psychiatr Serv 1995; 46: 275-80.

19 Kelly JA, Murphy DA, Washington CD, et al. The effects of HIV/AIDS intervention groups for high-risk women in urban clinics. Am J Public Health 1994; 84: 1918-22.

20 Jemmott JB, Jemmott LS, Fong GT. Reductions in HIV risk-associated sexual behaviours among black male adolescents: effects of an AIDS prevention intervention. Am J Public Health 1992; 82: 372-77.

21 Rotheram Borus MJ, Koopman C, Haignere C, Davies M. Reducing HIV sexual risk behaviors among runaway adolescents. JAMA 1991; 266: 1237-41.

22 Leviton LC, Valdiserri RO, Lyter DW, et al. Preventing HIV infection in gay and bisexual men: experimental evaluation of attitude change from two risk reduction interventions. AIDS Educ Prev 1990; 2: 95-108.

23 Kelly JA, Murphy DA, Sikkema KJ, et al. Randomised, controlled, community-level HIV-prevention intervention for sexual-risk behaviour among homosexual men in US cities. Lancet 1997; 350: 1500-05.

24 Kamb ML Bolan G, Zenilman J, Douglas JM. Does HIV/STD prevention counseling work? Results from a multi-center randomized trial (Project RESPECT) (abstract). 12th Meeting of the International Society for STD Research; 1997 October 19-22; Seville, Spain.

25 Methodological overview of a multisite HIV prevention trial for populations at risk for HIV. NIMH multisite HIV prevention trial. AIDS 1997; 11 (suppl 2): S1-63.

26 Peterman TA, Aral SO. Evaluating behavioral interventions: need for randomized controlled trials. JAMA 1993; 269: 2845.

27 Green S. Editorial: the eating patterns study--the importance of practical randomized trials in communities. Am J Public Health 1997; 87: 541.

28 Aral SO, Peterman TA. Measuring outcomes of behaviourial interventions for STD/HIV prevention. Int J STD AIDS 1996; 7 (suppl 2): 30.

29 Peterman TA, Lin L, Bolan G, et al. Does measured behavior change reflect change in STD/HIV risk? [abstr]. 12th World AIDS Conference; 1998; Geneva, Switzerland.

30 May RM. The scientific wealth of nations. Science 1997; 275: 793.

BUHUS INFO PAGES, main page, subgroup of AIDS and STDs, subgroup of hepatitis, subgroup of nosocomial diseases and tuberculosis, subgroup of tobacco abuse, subgroup of substance abuse, subgroup of social activities, history and aims, activities, members, researches, links to the related sites on the web, administrative committee, our statute(in Turkish).

© 1999 Last updated on September, the 29th, 1999

Sign Guestbook View Guestbook