How to evaluate complex drug policies? A case study of the Belgian drug policy

Evaluation is widely considered an essential part of drug policy. Nevertheless, evaluators are often confronted with methodological and theoretical challenges, especially when dealing with complex policy initiatives like a national drug policy. After all, drug policies are characterized by high degrees of uncertainty and emergence, but also by gradual adaptation and dynamical patterns, bringing high levels of unpredictability. To match the evaluation framework to this complexity, we rely on a theory-driven evaluation (TDE) framework that explains how drug policy works rather than solely focusing on whether it works.

By presenting the findings of a TDE of the Belgian drug policy, we discuss the challenges and benefits of designing and conducting a TDE of complex drug policy initiatives.

This study relies on various methods to design and test the policy theory. Through a document analysis of the central policy documents, the policy theory is constructed. Semi-structured interviews and focus groups with practitioners, experts, and civil servants (n=39) as well as people with lived experiences (n=22) were then conducted to explore how the policy theory is translated into practice.

Based on the experience of a TDE of the Belgian drug policy, we draw lessons learned of how complex policy initiatives benefit from a TDE framework. These lessons include, amongst others, the visualization of the complexity and central mechanisms of the policy theory, identification of gaps in the policy theory, and the potential to demonstrate how the central mechanisms interact with context.

The added value of a theory-based framework primarily lies in the theoretical understanding of mechanisms bringing about change, how they interact, and how they are operationalized in a (Belgian) context. Although a theory-driven framework has advantages for evaluating a complex drug policy, the models should still be understood as a simplification to guide the process examination.