How does risk prevention (not) work? Learning from the realist, pragmatic, cluster-randomised trial of the RISKIT-CJS programme (with a null result)

Background: RISKIT-CJS was a cluster-randomised trial of a multi-component risk reduction programme for adolescents (aged 13 -17), involved with the criminal justice system in England. Its main quantitative finding was acceptance of the null hypothesis of no effect on alcohol and substance use. Here we report on the realist, qualitative evaluation of the RISKIT-CJS programme to explore what can be learnt about how such programmes may or may not produce effects for their target group.

Methods: The qualitative evaluation used: records of meetings and focus groups (purposive sample), practitioner interviews and observational fieldnotes of programme delivery. Data were abductively coded, using a realist framework to identify the configurations of contexts, mechanisms, moderators and outcomes through which the programme worked in practice.

Results: This paper presents the refined realist programme theory. Relevant contexts included: geographical and institutional setting of the intervention sites (organising group work, staff ability, motivation and capacity), and the individual participant contexts (complex needs, maturity, motivation and risk profile). The intended mechanisms included: motivation to change; learning consequences; educational engagement; better communication; improved self-efficacy; and positive peer influence. The triggering of these mechanisms was affected by positive moderators (strong local programme champions, ageing out of crime for older participants) and negative moderators (disorganised, understaffed setting, and ageing into crime for younger participants). Potential negative mechanisms, included: negative peer effects and treatment fatigue. Different combinations of contexts, mechanisms and moderators led to different outcomes for different participants which are not adequately explained by an interpretation of the null quantitative result that the programme had ‘no effect’.

Conclusion: The paper demonstrates how realist, qualitative research can provide useful learning for policy and practice, even when the overall, quantitative result of a trial of a preventative intervention is null.