Do Self-Report Drug Use Surveys Produce the Same Results in Print and Online Mode? Findings from a Dual Mode Survey.
Background: Online surveys are increasingly becoming a standard mode of data collection in the social sciences. They have important advantages but also imitations. Regarding self-report drug use, one limitation is population coverage; thus, EMCDDA currently does not recommend online General Population Surveys. This study focuses on another issue; namely, the willingness of participants to disclose sensitive information online. The issue is critical, especially in societies where drug use attitudes are extremely negative, drug laws strict and law enforcement aggressive. In such contexts, strict anonymity is necessary; thus, surveys using ballots or sealed envelopes are preferred. Online surveys can be anonymous too, but not "visibly" so: respondents may not trust anonymity promises or that the data will be safe. Thus, the question of mode comparison (print vs. online) needs empirical investigation. Studies have so far produced mixed results. This study addresses the question using data from a dual mode survey of undergraduate students in Cyprus.
Methods: The survey was conducted in the fall of 2021. COVID-19 restrictions were in effect and universities were allowed to decide whether to run classes in traditional, online or hybrid mode. We selected 120 audiences from 32 institutions by probability-proportional-to-population sampling. All audiences were physically visited by researchers and, depending on each institution's COVID-19 policy, students either answered printed questionnaires and dropped them in a ballot or were directed to answer the questionnaire later anonymously online. Questions about lifetime, recent and current use of tobacco, heat cigarettes, alcohol, tranquilizers, marijuana, MDMA, amphetamines, cocaine, heroin, LSD and new drugs, and questions of theoretical interest and demographics were included. Mode comparison is addressed in a quasi-experimental way: 1559 completed questionnaires (54.1%) come from in-class and 1323 (45.9%) from online participation.
Results: Mode comparison was assessed with 31 logistic regression equations where the dependent variables were self-reported drug use behaviors (coded 0 for none and 1 otherwise) and the focal independent variable was the mode of administration. Controlling for demographic and attitudinal correlates of drug use, several items produced statistically significant differences: about half (5/11) of lifetime, about one third (4/11) of recent and about one third (3/9) of current drug use behaviors were underreported online. There was no statistically significant online overreporting. Safe conclusions cannot be drawn for several illegal drugs, because of very low frequencies of use.
Conclusions: With some reservation, the findings are overall encouraging for online self-report surveys. Most of the behaviors were not underreported online. Still, doubt stays about a significant number of items. Methodological implications, including the need for more similar studies, are discussed.
