Skip to main content

Advertisement

SpringerLink
Log in

Therapist Self-Report of Evidence-Based Practices in Usual Care for Adolescent Behavior Problems: Factor and Construct Validity

  • Original Article
  • Published:
Administration and Policy in Mental Health and Mental Health Services Research Aims and scope Submit manuscript

Abstract

This study introduces a therapist-report measure of evidence-based practices for adolescent conduct and substance use problems. The Inventory of Therapy Techniques—Adolescent Behavior Problems (ITT-ABP) is a post-session measure of 27 techniques representing four approaches: cognitive-behavioral therapy (CBT), family therapy (FT), motivational interviewing (MI), and drug counseling (DC). A total of 822 protocols were collected from 32 therapists treating 71 adolescents in six usual care sites. Factor analyses identified three clinically coherent scales with strong internal consistency across the full sample: FT (8 items; α = .79), MI/CBT (8 items; α = .87), and DC (9 items, α = .90). The scales discriminated between therapists working in a family-oriented site versus other sites and showed moderate convergent validity with therapist reports of allegiance and skill in each approach. The ITT-ABP holds promise as a cost-efficient quality assurance tool for supporting high-fidelity delivery of evidence-based practices in usual care.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Aarons, G., Hurlburt, M., & Horowitz, S. (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38, 4–23.

    Article  PubMed Central  PubMed  Google Scholar 

  • American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th edn., text revision). Washington, DC: APA.

  • Asparouhov, T. (2005). Sampling weights in latent variable modeling. Structural Equation Modeling, 12(3), 411–434.

    Article  Google Scholar 

  • Ball, S. A., Martino, S., Nich, C., Frankforter, T. L., Van Horn, D., Crits-Christoph, P., Woody, G. E., Obert, J. L., Farentinos, C., Carroll, K. M. (2007). Site matters: Multisite randomized trial of motivational enhancement therapy in community drug abuse clinics. Journal of Consulting and Clinical Psychology, 75, 556–567.

    Google Scholar 

  • Battjes, R. J., Gordon, M. S., O’Grady, K. E., Kinlock, T. W., Katz, E. C., & Sears, E. A. (2004). Evaluation of a group-based substance abuse treatment program for adolescents. Journal of Substance Abuse Treatment, 27, 123–134.

    Article  PubMed  Google Scholar 

  • Bearsley-Smith, C., Sellick, K., Chesters, J., & Francis, K. (2008). Treatment content in child and adolescent mental health services: Development of the treatment recording sheet. Administration and Policy in Mental Health and mental Health Services Research, 35, 423–435.

    Article  PubMed  Google Scholar 

  • Becker, S., & Curry, J. (2008). Outpatient interventions for adolescent substance abuse: A quality of evidence review. Journal of Consulting and Clinical Psychology, 76, 531–543.

    Article  PubMed  Google Scholar 

  • Bond, G. R., Becker, D. R., & Drake, R. E. (2011). Measurement of fidelity of implementation of evidence based practices: Case example of the IPS fidelity scale. Clinical Psychology: Science and Practice, 18, 126–141.

    Google Scholar 

  • Breslin, C., Li, S., Sdao-Jarvie, K., Tupker, E., & Ittig-Deland, V. (2002). Brief treatment for young substance users: A pilot study in an addiction treatment setting. Psychology of Addictive Behaviors, 16, 10–16.

    Article  PubMed  Google Scholar 

  • Brookman-Frazee, L., Haine, R. A., Baker-Ericzen, M., Zoffness, R., & Garland, A. F. (2010). Factors associated with use of evidence-based practice strategies in usual care youth psychotherapy. Administration and Policy in Mental Health and Mental Health Services Research, 37, 254–269.

    Article  PubMed Central  PubMed  Google Scholar 

  • Brosan, L., Reynolds, S., & Moore, R. G. (2008). Self-evaluation of cognitive therapy performance: Do therapists know how competent they are? Behavioural and Cognitive Psychotherapy, 36, 581–587.

    Article  Google Scholar 

  • Browne, M., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. Bollen & J. Long (Eds.), Testing structural models. Newbury Park, CA: Sage Publications.

    Google Scholar 

  • Carroll, K. M., Ball, S. A., Nich, C., Martino, S., Frankforter, T. L., Farentinos, C., Kunkel, L. E., Mikulich-Gilbertson, S. K., Morgenstern, J., Obert, J. L., Polcin, D., Snead, N., & Woody, G. E. (2006). Motivational interviewing to improve treatment engagement and outcome in individuals seeking treatment for substance abuse: A multisite effectiveness study. Drug and Alcohol Dependence, 81, 301–312.

    Google Scholar 

  • Carroll, K. M., Nich, C., & Rounsaville, B. J. (1998). Utility of therapist session checklists to monitor delivery of coping skills treatment for cocaine abusers. Psychotherapy Research, 8, 307–320.

    Google Scholar 

  • Carroll, K. M., Nich, C., Sifry, R., Nuro, K. F., Frankforter, T. L., Ball, S. A.,…Rounsaville, B. J. (2000). A general system for evaluating therapist adherence and competence in psychotherapy research in the addictions. Drug and Alcohol Dependence, 57, 225–238.

    Google Scholar 

  • Chambers, D., Ringeisen, H., & Hickman, E. (2005). Federal, state, and foundation initiatives around evidence-based practices for child and adolescent mental health. Child and Adolescent Mental Health, 14, 307–327.

    Google Scholar 

  • Chorpita, B., Daleiden, E., Ebesutani, C., Young, J., Becker, K., Nakamura, B,…Phillips, L. (2011). Evidence-based treatments for children and adolescents: An updated review of indicators of efficacy and effectiveness. Clinical Psychology: Science and Practice, 18, 154–172.

    Google Scholar 

  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Cook, J. M., Biyanova, T., Elhai, J., Schnurr, P. P., & Coyne, J. C. (2010). What do psychotherapists really do in practice? An internet study of over 2,000 practitioners. Psychotherapy: Theory, Research, Practice, Training, 47, 260–267.

    Article  Google Scholar 

  • D’Amico, E. J., Miles, J. N. V., Stern, S. A., & Meredith, L. S. (2008). Brief motivational interviewing for teens at risk of substance use consequences: A randomized pilot study in a primary care clinic. Journal of Substance Abuse Treatment, 35I, 53–61.

    Article  Google Scholar 

  • Damschroder, L., & Hagedorn, H. (2011). A guiding framework and approach for implementation research in substance use disorders treatment. Psychology of Addictive Behaviors, 25, 194–205.

    Article  PubMed  Google Scholar 

  • Dennis, M. L., Godley, S. H., Diamond, G., Tims, F. M., Babor, T., Donaldson, J., et al. (2004). The Cannabis Youth Treatment (CYT) study: Main findings from two randomized trials. Journal of Substance Abuse Treatment, 27, 197–213.

    Article  PubMed  Google Scholar 

  • Diamond, G. M., Diamond, G. S., & Hogue, A. (2007). Attachment-based family therapy: Adherence and differentiation. Journal of Marital and Family Therapy, 33, 177–191.

    Article  PubMed  Google Scholar 

  • Eyberg, S. M., Nelson, M. M., & Boggs, S. R. (2008). Evidence-based psychosocial treatments for children and adolescents with disruptive behavior. Journal of Clinical Child and Adolescent Psychology, 37, 215–237.

    Article  PubMed  Google Scholar 

  • Fixsen, D. L., Naoom, S. F., Blasé, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida, Louis de la parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231).

  • Garland, A. F., Bickman, L., & Chorpita, B. F. (2010a). Change what? Identifying quality improvement targets by investigating usual mental health care. Administration and Policy in Mental Health and Mental Health Services Research, 37, 15–26.

    Article  PubMed Central  PubMed  Google Scholar 

  • Garland, A. F., Hurlburt, M. S., Brookman-Frazee, L., Taylor, R. M., & Accurso, E. C. (2010b). Methodological challenges of characterizing usual care psychotherapeutic practice. Administration and Policy in Mental Health and Mental Health Services Research, 37, 208–220.

    Article  PubMed Central  PubMed  Google Scholar 

  • Gifford, E. V., Tavakoli, S., Weingardt, K. R., Finney, J. W., Pierson, H. M., Rosen, C. S.,…Curran, G. M. (2012). How do components of evidence-based psychological treatment cluster in practice?: A survey and cluster analysis. Journal of Substance Abuse Treatment. 42(1), 45–55.

    Google Scholar 

  • Hawkins, E. H. (2009). A tale of two systems: Co-occurring mental health and substance abuse disorders treatment for adolescents. Annual Review of Psychology, 60, 197–227.

    Article  PubMed  Google Scholar 

  • Hedeker, D., Gibbons, R. D., & Flay, B. R. (1994). Random-effects regression models for clustered data with an example from smoking prevention research. Journal of Consulting and Clinical Psychology, 62, 757–765.

    Article  CAS  PubMed  Google Scholar 

  • Henggeler, S. W., Sheidow, A. J., Cunningham, P. B., Donohue, B. C., & Ford, J. D. (2008). Promoting the implementation of an evidence-based intervention for adolescent marijuana abuse in community settings: Testing the use of intensive quality assurance. Journal of Clinical Child and Adolescent Psychology, 37, 682–689.

    Article  PubMed  Google Scholar 

  • Hepner, K. A., Azocar, F., Greenwood, G. L., Miranda, J., & Burnam, M. A. (2010). Development of a clinician report measure to assess psychotherapy for depression in usual care settings. Administration and Policy in Mental Health and Mental Health Services Research, 37, 221–229.

    Article  PubMed Central  PubMed  Google Scholar 

  • Hogue, A. (2010). When technology fails: Getting back to nature. Clinical Psychology: Science and Practice, 17, 77–81.

    Google Scholar 

  • Hogue, A., Dauber, S., Chinchilla, P., Fried, A., Henderson, C. E., Inclan, J.,…Liddle, H. A. (2008a). Assessing fidelity in individual and family therapy for adolescent substance abuse. Journal of Substance Abuse Treatment, 35, 137–147.

  • Hogue, A., Dauber, S., Samuolis, J., & Liddle, H. (2006). Treatment techniques and outcomes in multidimensional family therapy for adolescent behavior problems. Journal of Family Psychology, 20, 535–543.

    Article  PubMed Central  PubMed  Google Scholar 

  • Hogue, A., Henderson, C. E., Dauber, S., Barajas, P., Fried, A., & Liddle, H. (2008b). Treatment adherence, competence, and outcome in individual and family therapy for adolescent behavior problems. Journal of Consulting and Clinical Psychology, 76, 544–555.

    Article  PubMed Central  PubMed  Google Scholar 

  • Hogue, A., & Liddle, H. (2009). Family-based treatment for adolescent substance abuse: Controlled trials and new horizons in services research. Journal of Family Therapy, 31, 126–154.

    Article  PubMed Central  PubMed  Google Scholar 

  • Hogue, A., Liddle, H., Dauber, S., & Samuolis, J. (2004). Linking session focus to treatment outcome in evidence-based treatments for adolescent substance abuse. Psychotherapy: Theory, Research, Practice, and Training, 41, 83–96.

    Article  Google Scholar 

  • Hogue, A., Liddle, H., Rowe, C., Turner, R., Dakof, G. A., & LaPann, K. (1998). Treatment adherence and differentiation in individual versus family therapy for adolescent drug abuse. Journal of Counseling Psychology, 45, 104–114.

    Article  Google Scholar 

  • Hurlburt, M. S., Garland, A. F., Nguyen, K., & Brookman-Frazee, L. (2010). Child and family therapy process: Concordance of therapist and observational perspectives. Administration and Policy in Mental Health and Mental Health Services Research, 37, 230–244.

    Article  PubMed Central  PubMed  Google Scholar 

  • Kazak, A., Hoagwood, K., Weisz, J. R., Hood, K., Kratochwill, T., Vargas, L., et al. (2010). A meta-systems approach for evidence-based practice for children and adolescents. American Psychologist, 65, 85–97.

    Article  PubMed  Google Scholar 

  • Kelley, S. D., de Andrade, A. R. V., Sheffer, E., & Bickman, L. (2010). Exploring the black box: Measuring youth treatment process and progress in usual care. Administration and Policy in Mental Health and Mental Health Services Research, 37, 287–300.

    Article  PubMed  Google Scholar 

  • Kolko, D. J. (2006). Commentary: Studying usual care in child and adolescent therapy: It’s anything but routine. Clinical Psychology: Science and Practice, 13, 47–52.

    Google Scholar 

  • Martino, S., Ball, S., Nich, C., Frankforter, T. L., & Carroll, K. M. (2009). Correspondence of motivational enhancement treatment integrity ratings among therapists, supervisors, and observers. Psychotherapy Research, 19, 181–193.

    Article  PubMed Central  PubMed  Google Scholar 

  • McDonald, R., & Ho, M. (2002). Principles and practice in reporting structural equation analyses. Psychological Methods, 7, 64–82.

    Article  PubMed  Google Scholar 

  • McHugh, R. K., & Barlow, D. H. (2010). The dissemination and implementation of evidence-based psychological treatments. American Psychologist, 65, 73–84.

    Article  PubMed  Google Scholar 

  • McHugh, R., Murray, H., & Barlow, D. H. (2009). Balancing fidelity and adaptation in the dissemination of empirically supported treatments: The promise of transdiagnostic interventions. Behavior Research and Therapy, 47, 946–953.

    Article  Google Scholar 

  • Miller, W. R., & Rose, G. S. (2009). Toward a theory of motivational interviewing. American Psychologist, 64, 527–537.

    Article  PubMed Central  PubMed  Google Scholar 

  • Miller, W. R., Yahne, C. E., Moyers, T. B., Martinez, J., & Pirritano, M. (2004). A randomized trial of methods to help clinicians learn motivational interviewing. Journal of Consulting and Clinical Psychology, 72, 1050–1062.

    Article  PubMed  Google Scholar 

  • Miranda, J., Azocar, F., & Burnam, M. A. (2010). Assessment of evidence-based psychotherapy practices in usual care: Challenges, promising approaches, and future directions. Administration and Policy in Mental Health and Mental Health Services Research, 37, 205–207.

    Article  PubMed Central  PubMed  Google Scholar 

  • Nakamura, B. J., Daleiden, E. L., & Mueller, C. W. (2007). Validity of treatment target progress ratings as indicators of youth improvement. Journal of Child and Family Studies, 16, 729–741.

    Article  Google Scholar 

  • Perepletchikova, F. (2011). On the topic of treatment integrity. Clinical Psychology: Science and Practice, 18, 148–153.

    Google Scholar 

  • Perepletchikova, F., & Kazdin, A. E. (2005). Treatment integrity and therapeutic change: Issues and research recommendations. Clinical Psychology: Science and Practice, 12, 365–383.

    Google Scholar 

  • Rabe-Hesketh, S., & Skrondal, A. (2005). Multilevel and longitudinal modeling using Stata. College Station, TX: StataCorp LP.

    Google Scholar 

  • Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Thousand Oaks, CA: Sage.

    Google Scholar 

  • Schoenwald, S. K. (2011). It’s a bird, it’s a plane, it’s…fidelity measurement in the real world. Clinical Psychology: Science and Practice, 18, 142–147.

    Google Scholar 

  • Schoenwald, S. K., Garland, A. F., Chapman, J. E., Frazier, S. L., Sheidow, A. J., & Southam-Gerow, M. A. (2011). Toward the effective and efficient measurement of implementation fidelity. Administration and Policy in Mental Health and Mental Health Services Research, 38, 32–43.

    Article  PubMed Central  PubMed  Google Scholar 

  • Schoenwald, S. K., Sheidow, A. J., & Chapman, J. E. (2009). Clinical supervision in treatment transport: Effects on adherence and outcomes. Journal of Consulting and Clinical Psychology, 77, 410–421.

    Article  PubMed Central  PubMed  Google Scholar 

  • Waldron, H. B., & Turner, C. W. (2008). Evidence-based psychosocial treatments for adolescent substance abuse. Journal of Clinical Child and Adolescent Psychology, 37, 238–261.

    Article  PubMed  Google Scholar 

  • Webb, C. A., DeRubeis, R. J., & Barber, J. P. (2010). Therapist adherence/competence and treatment outcome: A meta-analytic review. Journal of Consulting and Clinical Psychology, 78, 200–211.

    Article  PubMed  Google Scholar 

  • Weersing, R. V., Weisz, J. R., & Donenberg, G. R. (2002). Development of the therapy procedures checklist: A therapist-report measure of technique use in child and adolescent treatment. Journal of Clinical Child Psychology, 31, 168–180.

    Google Scholar 

  • Winters, K. C., Stinchfield, R., Latimer, W. W., & Lee, S. (2007). Long-term outcome of substance-dependent youth following 12-step treatment. Journal of Substance Abuse Treatment, 33, 61–69.

    Article  PubMed  Google Scholar 

  • Winters, K. C., Stinchfield, R. D., Opland, E., Weller, C., & Latimer, W. W. (2000). The effectiveness of the Minnesota Model approach in the treatment of adolescent drug abusers. Addiction, 95, 601–612.

    Article  CAS  PubMed  Google Scholar 

  • Zazzali, J. L., Sherbourne, C., Hoagwood, K., Greene, D., Bigley, M. F., & Sexton, T. (2008). The adoption and implementation of an evidence based practice in child and family mental health services organizations: A pilot study of Functional Family Therapy in New York State. Administration and Policy in Mental Health and Mental Health Services Research, 35, 38–49.

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

Preparation of this article was supported by grant R01DA023945 from the National Institute on Drug Abuse. The authors would like acknowledge the dedicated work of the clinical research staff for this project: Molly Bobek, Candace Johnson, and Emily Lichvar. We are also extremely grateful to the community therapists working at six study sites who generously agreed to participate in a research study, attend two focus groups and one instrument training, and complete numerous post-session checklists in the hopes of advancing the clinical science in their field.

Author information

Authors and Affiliations

  1. Treatment Research Division, National Center on Addiction and Substance Abuse (CASA) at Columbia University, 633 Third Avenue, 19th floor, New York, NY, 10017, USA

    Aaron Hogue & Sarah Dauber

  2. Department of Psychology, Sam Houston State University, Huntsville, TX, USA

    Craig E. Henderson

Authors
  1. Aaron Hogue
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sarah Dauber
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Craig E. Henderson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aaron Hogue.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Hogue, A., Dauber, S. & Henderson, C.E. Therapist Self-Report of Evidence-Based Practices in Usual Care for Adolescent Behavior Problems: Factor and Construct Validity. Adm Policy Ment Health 41, 126–139 (2014). https://doi.org/10.1007/s10488-012-0442-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10488-012-0442-8

Keywords

Search

Navigation