Since the last newsletter, we’ve made steady progress on most of our research projects and will commence with field work in the new year.  In this report, I would like to focus on the issue of programme evaluation.

There are two major research projects that are concerned with the evaluation of an intervention. Firstly, the Gender Programme is conceptualising an innovative approach towards tackling the intersection of gender inequality and poverty. They are currently adapting Stepping Stones (a gender transformative intervention) to include a livelihoods strengthening intervention and plan to test the effectiveness of this at reducing risky sexual behaviour beginning in 2012.

Secondly, we are also involved in evaluating a risk reduction intervention with men attending taverns in KwaZulu-Natal. This project focuses on violence, alcohol and sexual risk reduction activities. Programme evaluation is becoming an increasingly important part of our research activities. Researchers, funders and policy makers want to know:

1) Whether a programme is effective or not.

2) Under what conditions certain interventions will prove to be effective.

3) Is the programme cost effective.

4) Can it make a difference in the real world? 

In this respect, interventions can only have a significant impact on health and health care, if they are shown to be effective when tested, are capable of being widely implemented and can be normalized into routine practice.

Traditional approaches to monitoring and evaluation of interventions have primarily adopted the ‘black box’ approach. This means, the focus has been, in a mechanistic manner, on issues related to isolating input factors in order to demonstrate its linear association with output factors. While this method had merit in being attentive to understanding causality, its logic was rather simplistic. One of the inherent problems with this approach is that it made minimal effort to inquire into the ‘processes’ of the programme (that is, what is going on in the box). Recent approaches to Programme Evaluation (PE) have however made significant strides with respect to the latter. The evaluations of Complex Interventions have approached the issue of causality in a more synergistic manner. Here PE is grounded in relation to the contextual realities of programme implementation and is also concerned with multiple components and consequences (both intended and unintended) of the subsequent intervention.  In 2007, the British Medical Research Council has put out a revised Framework for the Development and Evaluation of RCTs for Complex Interventions to Improve Health[i]. These guidelines have been highly influential and the accompanying BMC Medicine paper provides an informative framework (called Normalisation Process Theory) that addresses factors needed for successful implantation and integration of interventions into practice[ii].

Here are some key points of the recent report that may be useful programme evaluators:

1) Early phases of a trial should be seen as iterative rather than linear.

2) Both intervention and evaluation require a strong theoretical component.

3) Detailed descriptions of the intervention and the context of the evaluation are needed.

4) Modelling to estimate the potential benefits is important before proceeding to a trial.

5) Qualitative methods can assist with understanding the processes involved in the intervention and evaluation. 

These points are important in closing the gap between research and practice (normalisation processes).

Follow HEARD On

 

If you are having problems viewing this email, View it online
To ensure that our emails reach your inbox, please add heardnews@ukzn.ac.za to your address book.
© HEARD 2011