Abstract
The purpose of this study was to develop and validate an observation measure designed to capture teachers’ use of interactive teaching skills within the delivery of the All Stars substance use prevention program. Coders counted the number of times teachers praised and encouraged students, accepted and used students’ ideas, asked questions, self-disclosed personal anecdotes, and corrected student misbehavior. These teacher behaviors loaded on three factors: classroom management, acknowledgment, and student-centered methods. Classroom management was negatively related to student engagement. Acknowledgment was negatively related to students’ normative beliefs. Student-centered methods were positively related to student idealism and normative beliefs, and marginally predicted decreases in student marijuana use. Editors’ Strategic Implications: The authors provide a promising approach to studying pedagogical prevention approaches, and they also link teaching processes to student outcomes. This study of program delivery should be of general interest (i.e., not limited to substance use prevention) to practitioners and researchers.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abbott, R. D., O’Donnell, I., Hawkins, D., Hill, K. G., Kosterman, R., & Catalano, R. F. (1998). Changing teaching practices to promote achievement and bonding to school. American Journal of Orthopsychiatry, 68, 542–552. doi:10.1037/h0080363.
Bosworth, K., & Sailes, J. (1993). Content and teaching strategies in 10 selected drug abuse prevention curricula. Journal of School Health, 63, 247–253.
Botvin, G. J., Dusenbury, L., Baker, E., James-Ortiz, S., & Kerner, J. (1989). A skills training approach to smoking prevention among Hispanic youth. Journal of Behavioral Medicine, 12, 279–296. doi:10.1007/BF00844872.
Dusenbury, L., Brannigan, R., Hansen, W. B., Walsh, J., & Falco, M. (2005). Quality of implementation: Developing measures crucial to understanding the diffusion of preventive interventions. Health Education Research, 20, 308–313. doi:10.1093/her/cyg134.
Dusenbury, L., & Falco, M. (1995). Eleven components of effective drug abuse prevention curricula. Journal of School Health, 65, 420–425.
Flanders, N. (1970). Analyzing teacher behavior. Reading, MA: Addison-Wesley.
Hansen, W. B. (1996). Pilot test results comparing the All Stars program with seventh grade D.A.R.E.: Program integrity and mediating variable analysis. Substance Use and Misuse, 31, 1359–1377. doi:10.3109/10826089609063981.
Hansen, W. B., Graham, J. W., Wolkenstein, B. H., & Rohrbach, L. A. (1991). Program integrity as a moderator of prevention program effectiveness: Results for fifth-grade students in the Adolescent Alcohol Prevention Trial. Journal of Studies on Alcohol, 52, 568–579.
Hansen, W. B., & McNeal, R. B. (1999). Drug education practice: Results of an observational study. Health Education Research, 14, 85–97. doi:10.1093/her/14.1.85.
Harachi, T. W., Abbott, R. D., Catalano, R. F., Haggerty, K. P., & Fleming, C. B. (1999). Opening the black box: Using process evaluation measures to assess implementation and theory building. American Journal of Community Psychology, 27, 711–731. doi:10.1023/A:1022194005511.
Harrington, N. G., Giles, S. M., Hoyle, R. H., Feeney, G. J., & Yungbluth, S. C. (2001). Evaluation of the All Stars character education and problem behavior prevention program: Pretest-posttest effects on mediator and outcome variables for middle school students. Health Education & Behavior, 28, 533–546. doi:10.1177/109019810102800502.
McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1, 30–46. doi:10.1037/1082-989X.1.1.30.
McNeal, R. B., Hansen, W. B., Harrington, N. G., & Giles, S. M. (2004). How All Stars works: An examination of program effects on mediating variables. Health Education & Behavior, 31, 165–178. doi:10.1177/1090198103259852.
Muthén, L. K., & Muthén, B. O. (2007). Mplus statistical software (Version 5.0). Los Angeles, CA: Muthén & Muthén.
Pentz, M. A., Trebow, E. A., Hansen, W. B., MacKinnon, D. P., Dwyer, J. H., Johnson, C. A., et al. (1990). Effects of program implementation on adolescent drug use behavior. Evaluation Review, 14, 264–289. doi:10.1177/0193841X9001400303.
Ringwalt, C. L., Pankratz, M. M., Hansen, W. B., Dusenbury, L., Jackson-Newsom, J., Giles, S. M., Brodish, P. (2007). The potential of coaching as a strategy to improve the effectiveness of school-based substance use prevention curricula. Health Education & Behavior, July 25, 2007. doi:10.1177/1090198107303311.
Shrout, P. E., & Fleiss, J. J. (1979). Intraclass correlations: Uses in assessing reliability. Psychological Bulletin, 86, 420–428. doi:10.1037/0033-2909.86.2.420.
Sobol, D. F., Dent, C. W., Gleason, L., Brannon, B. R., Johnson, C. A., & Flay, B. R. (1989). The integrity of smoking prevention curriculum delivery. Health Education Research, 4, 59–67. doi:10.1093/her/4.1.59.
Substance Abuse and Mental Health Services Administration. (2008). SAMHSA model programs: Effective substance abuse and mental health programs for every community. http://modelprograms.samhsa.gov/model.htm. Retrieved 22 July 2008.
Tobler, N. (1986). Meta-analysis of 143 adolescent drug prevention programs: Quantitative outcome results of program participants compared to a control or comparison group. Journal of Drug Issues, 16, 537–567.
Tobler, N. S. (1992). Drug prevention programs can work: Research findings. Journal of Addictive Diseases, 11(3), 1–28. doi:10.1300/J069v11n03_01.
Tobler, N. S. (2000). Lessons learned. Journal of Primary Prevention, 20, 261–274. doi:10.1023/A:1021362620740.
Tobler, N. S., Lessard, T., Marshall, D., Ochshorn, P., & Roona, M. (1999). Effectiveness of school-based drug prevention programs for marijuana use. School Psychology International, 20, 105–137. doi:10.1177/0143034399201008.
Tobler, N. S., & Stratton, H. (1997). Effectiveness of school-based drug prevention programs: A meta-analysis of the research. Journal of Primary Prevention, 18, 71–128. doi:10.1023/A:1024630205999.
Acknowledgments
This study was supported by a grant from the National Institute on Drug Abuse, Grant No. 5 R01 DA016098, “Promoting Fidelity Using Remote and Onsite Support.” We would like to acknowledge the contribution of our coders—Terry Dumansky, Sara Patterson, and Grant Bollmer. The work was tedious but their insights and acumen were invaluable. We are also indebted to Dr. Inez Drummond of Chicago Public Schools for her support and to all the teachers who collaborated with us on this study.
Author information
Authors and Affiliations
Corresponding author
Appendix
Appendix
Coding instrument
Category | Tally |
|---|---|
Praises or encourages | Genuine: |
Non-genuine: | |
Accepts or uses ideas of pupils | Accepts: |
Uses: | |
Asks questions | Original: |
Repeated: | |
Probing: | |
Self-discloses personal anecdotes | |
Corrects students misbehavior | Student specific, appropriate: |
Class specific, appropriate: | |
Student specific, inappropriate: | |
Class specific, inappropriate: | |
Comments: | |
Rights and permissions
About this article
Cite this article
Giles, S., Jackson-Newsom, J., Pankratz, M.M. et al. Measuring Quality of Delivery in a Substance Use Prevention Program. J Primary Prevent 29, 489–501 (2008). https://doi.org/10.1007/s10935-008-0155-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10935-008-0155-7