Addressing Adherence and Success Rates of a Digital Self-Help Intervention for Alcohol and Substance Use with Machine Learning

Digital self-help interventions for reducing the use of alcohol, tobacco and other drugs (ATOD) have generally shown positive but small effects in controlling substance use and improving participants’ quality of life. Nonetheless, these interventions have low adherence and mixed results in (prolonged) participation and outcome. We aimed 1) to predict intervention adherence and success, 2) to determine possible solutions to improve these rates, and 3) to explore acceptability and ethical issues around application of machine learning in eHealth.

To identify predictors, we included participants from a widely used, evidence-based ATOD intervention from the Netherlands (Jellinek Digital Self-help). Participants were considered successful if they completed all intervention modules and reached their substance use goals. Machine learning models were trained, validated and tested using a nested k-fold cross-validation strategy. Subsequently, we held two focus groups among intervention 1) researchers and clinicians and 2) intervention participants (each focus group n=5-8) 1) to determine possible solutions to improve adherence and success rates, and 2) to explore opinions around application of machine learning in eHealth.

From the >32K participants who enrolled, around 20% completed the first module. Success rates were 30% for alcohol, 22% for cannabis, and 24% for tobacco. Quitting substance use instead of moderation as a goal, initial daily consumption, no drinking on the weekends as a goal, and intervention engagement were strong predictors of success. Focus group results led to implementation of changes involving 1) promoting stopping rather than reducing, 2) sending daily emails, and 3) abandoning the minimal 5 days of practice in a module before continuation. First results of these changes will be available summer 2022 and presented; along with further focus group results.

Machine learning models better predicted drop-out than that they predicted success rates, which had consequences for solutions discussed in the focus groups and following implementation.

This work was supported by ZonMw, project number 555003024.