Abstract
Outcome monitoring has become a focus of accountability for public and nonprofit human service agencies. Besides providing answers to funders' questions about the services' impact, outcome monitoring helps administrators improve program effectiveness. After a three-year development period and a one-year implementation experience, SumOne for Kids represents a technically advanced outcome-monitoring system for children's mental health and/or child welfare services. Initiated, designed, and tested by 31 children's service agencies throughout Pennsylvania, and with state bureaucrats' and policy makers' encouragement, SumOne for Kids represents an effort to create a bottom-up/top-down process for implementing a statewide outcome-monitoring system. This article describes the genesis of this outcome-monitoring system, primary design principles, use of social validation for outcome selection, resolution of methodological difficulties, and reasons for selecting functional over clinical outcomes. The article reviews lessons learned through the development experience instructive to children's service managers, program evaluators, and industry leaders interested in establishing outcome-monitoring systems.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
United States House of Representatives Select Committee on Children, Youth, and Families:No Place to Call Home: Discarded Children in America. Washington, DC: U.S. Govt. Printing Office, 1989.
Nadel MV:Residential Care. Report No. GAO HEHS 94 56. United States Government Accounting Office, Health, Education, and Human Services Division, Washington, DC, 1994.
Kaplan LW:Making Quality Count: A New United Way of Allegheny County Program Review and Allocation System. Pittsburgh, PA: Allegheny County United Way, 1993.
Cole RF, Poe SL:Partnerships for Care: Systems of Care for Children With Serious Emotional Disturbances and Their Families. Interim Report of the Mental Health Services Program for Youth. Washington, DC: Washington Business Group on Health, 1993.
Governor's Select Commission:Memorandum on the Care of Dependent Children in Pennsylvania. Commission report (suppl.), 1923.
SumOne for Kids: An Outcomes-Based Measurement Software System. Pittsburgh, PA: Corporation for Standards & Outcomes, 1994.
VanDenBerg J, Beck S, Pierce, J: The Pennsylvania outcome project for children's services. In: Kutash K, Liberton CJ, Algarin A, et al. (Eds.):A System of Care for Children's Mental Health: Expanding the Research Base. Tampa: Mental Health Institute, University of South Florida, 1992, pp. 233–238.
Fabry BD, Hawkins RP, Luster WC: Monitoring outcomes of services to children and youths with severe emotional disorders: An economical follow-up procedure for mental health and child care agencies.The Journal of Mental Health Administration 1994; 21(3):271–282.
Usher L: Balancing Stakeholder Interests in Evaluations of Innovative Programs to Serve Families and Children. Paper presented at the annual meeting of the Association for Policy Analysis and Management, Washington, DC, October 30, 1993.
Mathison S: What do we know about internal evaluation?Evaluation and Program Planning 1991; 14:159–165.
Hawkins RP, Fremouw WJ, Reitz AL: A model useful in designing or describing evaluation of planned interventions in mental health. In: McSweeny AJ, Fremouw WJ, Hawkins RP (Eds.):Practical Program Evaluation in Youth Treatment. Springfield, IL: Charles C Thomas, 1982, pp. 24–48.
Rosenblatt A, Atkisson CC: Assessing outcomes for sufferers of severe mental disorder: A conceptual framework and review.Evaluation and Program Planning 1993; 16:347–363.
Wolf MM: Social validity: The case for subjective measurement or How applied behavior analysis is finding its heart.Journal of Applied Behavior Analysis 1978; 11:203–214.
Barth RP:Social and Cognitive Treatment of Children and Adolescents. San Francisco: Jossey-Bass, 1986.
Gold N: Stakeholders and program evaluation: Characterizations and reflections. In: Bryk AS (Ed.):Stakeholder-Based Evaluation. San Francisco: Jossey-Bass, 1983, pp. 63–72.
VanDenBerg J, Beck S, Howarth D, et al.:What Pennsylvanians Want From Children's Services: Summary Report on the Social Validation Study. Pittsburgh, PA: The Pressley Ridge Center for Research and Public Policy, 1992.
Data collection for foster care and adoption; statewide automated child welfare information systems; final rule and interim final rule.Federal Register 1993; 58(244).
Service Efforts and Accomplishments Reporting: Concepts Statement No. 2. Governmental Accounting Standards Board No. 109-A. Norwalk, CT: Government Accounting Standards Board, April 1994.
Author information
Authors and Affiliations
Corresponding author
Additional information
Shirley A. Beck, M.S.S.A., was a research coordinator, Center for Research & Public Policy, The Pressley Ridge Schools, Pittsburgh, Pennsylvania at the time this article was written.
Pamela Meadowcroft, Ph.D., is deputy executive director, Center for Research & Public Policy, The Pressley Ridge Schools, Pittsburgh, Pennsylvania.
Edward S. Kiely, Ph.D., is a project associate, Center for Research & Public Policy, The Pressley Ridge Schools, Pittsburgh, Pennsylvania.
Rights and permissions
About this article
Cite this article
Mason, M., Beck, S.A., Meadowcroft, P. et al. Multiagency outcome evaluation of children's services: A case study. The Journal of Behavioral Health Services & Research 25, 163–176 (1998). https://doi.org/10.1007/BF02287478
Issue Date:
DOI: https://doi.org/10.1007/BF02287478