Abstract
Meta-analysis is a statistical method that is increasingly utilized to combine and compare the results of previous primary studies. However, because of the lack of comprehensive guidelines for how to use meta-analysis, many meta-analysis studies have failed to consider important aspects, such as statistical programs, power analysis, publication bias, model selection, test of heterogeneity, and identification of heterogeneity. Therefore, the current study reviewed 84 meta-analysis studies conducted in Korea to examine proper application of the six categories named above. With regard to the issue of effect sizes, it was found that most of the meta-analysis studies obtained more than ten effect sizes, which seem to be an adequate number for representing an issue. However, many studies failed to consider the other issues: power analysis, publication bias, model selection, test of heterogeneity, and identification of heterogeneity.
Similar content being viewed by others
References
Note: References marked with an asterisk indicate studies included in review of the meta-analysis studies.
*Ahn, H. J., & Lee, S. H. (2005). A meta-analysis of the variables related to parent–child communication. The Korean Family Welfare Association, 10(1), 65–81.
*Ahn, S. K., Yang, J. H., & Chung, I. J. (2012). Meta-analysis on protective and risk factors in the family domain affecting child abuse. Korean Journal of Family Social Work, 38, 331–358.
*Bae, J. M., Ji, S. H., Kim, D. M., Park, J. Y., Kim, E. Y., Kim, T. S., et al. (2007). A study on models of protection support for recovery operations (pp. 73–86). Korea Youth Counseling Institution.
Bangert-Drowns, R. L. (1986). Review of developments in meta-analytic method. Psychological Bulletin, 99(3), 388–399. doi:10.1037/0033-2909.99.3.388.
Becker, B., & Morton, S. (2002). Publication bias: Methods and practice in evidence-based medicine and social science. Berkeley, California: Mathematical Sciences Research Institute.
Begg, C. B., & Mazumdar, M. (1994). Operating characteristics of a rank correlation test for publication bias. Biometrics, 50(4), 1088–1101.
Borenstein, M., Hedges, L. V., Higgins, J., & Rothstein, H. R. (2009). Introduction to meta-analysis. Wiley Online Library.
*Byun, M. H. (2002). A meta-analysis on child, youth counseling effect. Korean Association of Youth Welfare, 4(2), 51–65.
Cafri, G., Kromrey, J. D., & Brannick, M. T. (2010). A meta-meta-analysis: Empirical review of statistical power, type I error rates, effect sizes, and model selection of meta-analyses published in psychology. Multivariate Behavioral Research, 45(2), 239–270.
Chalmers, T. C., Frank, C. S., & Reitman, D. (1990). Minimizing the three stages of publication bias. JAMA, 263(10), 1392–1395.
*Cho, Y. J., & Yoo, S. H. (2010). A meta-analysis on the effect of group counseling programs for the children of parentless family. Korea Youth Counseling Institution, 18(2), 37–55.
*Choi, W. S. (2005). A meta-analysis of the variables related to school violence. The Korean Family Welfare Association, 10(2), 95–111.
*Choi, S., & Choi, S. W. (2009). Meta-analysis of executive function in unipolar depression and bipolar depression (pp. 272–273). Korean Psychology Association.
*Choi, S., & Choi, S. W. (2010). Comparison of cognitive function between unipolar depression and bipolar depression through meta-analysis: Focus on verbal memory and frontal-executive function. Korean Psychology Association, 29(2), 607–620.
Cooper, H., Hedges, L. V., & Valentine, J. C. (2009). The handbook of research synthesis and meta-analysis. New York: Russell Sage Foundation.
Gerard, P. D., Smith, D. R., & Weerakkody, G. (1998). Limits of retrospective power analysis. The Journal of wildlife management, 62(2), 801–807.
Hedges, L. V., & Olkin, I. (2014). Statistical methods for meta-analysis. Cambridge: Academic press.
Hedges, L. V., & Pigott, T. D. (2004). The power of statistical tests for moderators in meta-analysis. Psychological Methods, 9(4), 426.
Hedges, L. V., & Vevea, J. L. (1998). Fixed-and random-effects models in meta-analysis. Psychological Methods, 3(4), 486.
Higgins, J., & Thompson, S. G. (2002). Quantifying heterogeneity in a meta-analysis. Statistics in Medicine, 21(11), 1539–1558.
Higgins, J., Thompson, S., Deeks, J., & Altman, D. (2003). Measuring inconsistency in meta-analyses. BMJ, 327, 557–560. (Find this article online).
*Hong, Y. K., & Lee, K. S. (2013). A meta-analysis on the variables related with social phobia: 2011–2013. Journal of Emotional & Behavioral Disorders, 29, 295–310.
*Hong, K. J., Yim, J. S., & Kim, W. S. (2001). A meta-analysis comparison on the effects of humanistic and cognitive-behavioral counseling groups for the improvement of adolescents’ self-esteem. Korean Counseling Association, 2(1), 23–44.
*Im, S. I., & Bak, B. G. (2013). Meta-analysis study of test anxiety on a domestic and international level. Journal of Educational Psychology, 27, 529–553.
*Im, S. A., & Jung, W. J. (2013). A meta-analysis of the effects of self-efficacy on academic achievement. Korean Journal of Educational Research, 51, 83–105.
*Jeon, B. U., & Koh, J. B. (2007). A meta-analysis of phonics-based reading instruction and meaning-based reading instruction for children with special education need. The Korean Society of Special Education, 4, 151–169.
*Jeon, M. R., & Park, B. G. (2011). A meta-analysis on the effect of art activities for preschool children. Korean Educational Psychology Association, 25(2), 295–316.
*Jin, H. M., & Bae, S. W. (2012). A meta-analysis on the variables related with juvenile delinquency. Journal of Adolescent welfare, 14, 93–221.
*Jin, H. M., & Park, B. S. (2013). A meta-analysis on the variables related with job satisfaction of social workers. Journal of Korea Academy of Social Welfare, 65, 107–130.
*Jo, H. S., Kim, C. B., Lee, H. W., & Jeong, H. J. (2004). A meta-analysis of health related behavior study based on health belief model in Korean. Korean Psychology Association, 9(1), 69–84.
*Jung, A. K., Kim, K. Y., & Kim, D. M. (2008). A meta-analysis on the studies of career indecision and its related variables. Korean Counseling Association, 9(2), 551–564.
*Jung, Y. J., & Lim, S. A. (2013). A meta-analysis of the correlation effects between self-efficacy and school adjustment. Korean Journal of Educational Research, 27, 53–873.
*Jung, E. H., Lim, K. W., & Jung, B. U. (2012a). The trend of single subject research on the picture exchange communication system pecs of children with autistic disorders: Existing studies for the years 1990 to 2011. Journal of Emotional & Behavioral Disorders, 28, 243–276.
*Jung, E. H., Lim, K. W., & Jung, B. U. (2012b). Trend of video modeling studies of children with autistic disorders. Journal of Emotional & Behavioral Disorders, 28, 109–141.
*Jung, P. Y., & Yim, D. S. (2011). Meta-analysis of parent training: Focus on children with language impairment. The Korean Academy of Speech-Language Pathology and Audiology, 16, 521–539.
Kalaian, H. A., & Raudenbush, S. W. (1996). A multivariate mixed linear model for meta-analysis. Psychological Methods, 1(3), 227.
*Kang, H. S., & Bak, B. G. (2014). A meta-analysis of the relationship between creativity and cognitive, emotional, and environmental variables. Korean Journal of Educational Research, 28, 371–404.
*Kang, M. C., Kim, S. I., & Kim, D. M. (2012a). A meta-analytic review of prevalence of postpartum depression and its related factors: Social support and parenting stress. Korean Counseling Association, 13(1), 149–168.
*Kang, M. K., Lee, S. B., Kim, Y. T., & Yim, D. S. (2012b). A meta-analysis of lexical processing in groups with autism spectrum disorders. Communication Sciences and Disorders, 17, 274–289.
*Kang, J. R., & Son, H. D. (2004). A meta-analysis of the effects of group counseling programs on academic achievement. The Korea Journal of Youth Counseling, 12(1), 81–90.
*Kang, H. Y., & Son, C. N. (2007). A meta-analysis on the variables related internet and game addiction. Korean Psychology Association, 12(4), 733–744.
*Kim, M. S. (2012). The effect of suicide prevention programs: A meta-analytic review. Yonsei University Social Welfare Studies Association, 30, 27–56.
*Kim, H. T., & Baek, Y. M. (2007). A meta-analysis on the efficacy of self-growth group counseling. Korean Counseling Association, 8(3), 965–978.
*Kim, B. J., & Han, M. W. (1995). A meta-analysis; effects of mental practice on motor performance in Korea. Korea Society of Sport Psychology, 6, 67–88.
*Kim, H. S., & Kim, D. I. (2007). The effective construction of self-regulated learning program based on meta-analysis. Korean Counseling Association, 8(2), 719–736.
*Kim, J. M., & Kim, H. J. (2012). Behavioral intervention for individual with ASD; meta-analysis of single subject researches. Journal of Emotional & Behavioral Disorders, 28, 293–316.
*Kim, S. M., Ko, H. Y., Park, S. H., & Yang, E. J. (2012a). Factors related to depression of children and adolescents in South Korea: A meta-analysis. Korean Journal of Culture and Social Issues, 18, 533–555.
*Kim, M. Y., Lee, S. H., & Heo, S. Y. (2012b). A meta-analysis of pivotal response treatments interventions for the children with autism spectrum disorders: Based on single subject research. The Journal of Special Children Education, 14, 1–23.
*Kim, J. M., & Lim, H. J. (2013). A single-subject meta-analysis of the effectiveness of social skill interventions for children with autism spectrum disorder. Journal of Emotional & Behavioral Disorders, 29, 311–339.
*Kim, S. S., & Woon, M. S. (2012). Trend and meta-analysis for self-determination theory and its relation to academic achievement. Korean Journal of Educational Research, 50, 77–106.
*Kim, K. H., Lee, Y. J., & Wang, E. J. (2002). A meta-analysis on the efficacy of group counseling conducted in Korea. Korean Counseling Association, 3(1), 47–62.
Kisamore, J. L., & Brannick, M. T. (2007). An illustration of the consequences of meta-analysis model choice. Organizational Research Methods, 11(1), 35–53.
*Ko, Y. S., & Kim, E. K. (2007). A meta-analysis of single-subject design studies for improving socialization and communication skills for autistic spectrum disorders. Korean Society for the Emotional & Behavioral Disorders, Korean Counseling Association, 23(1), 1–27.
*Ko, Y. N., & Lee, M. W. (2011). A meta-analysis on the effectiveness of group biblio-therapy. Korean Counseling Association, 12(2), 489–504.
*Koo, H. J., Choi, S. M., & Kwon, J. H. (2012). Youth psychotherapy outcome research: Methodological review and meta-analysis. Korean Psychology Association, 31(1), 43–73.
*Kwon, H. S. (2013). A meta-analysis on variables related to suppression of depression of in-home elderly. Journal of Welfare for the Aged Institute, 61, 241–266.
*Kwon, K. I., & Jo, S. Y. (2013). A meta-analysis on the effects of adolescent stress group counseling programs in Korea. Korean Journal of Counseling and Psychology, 25, 41–62.
*Kwon, T. H., & Kim, J. R. (2012). Meta-analysis of the effects of content-integrated English education in knowledge-based subjects and skill-based subjects in elementary school. Korean Association for Learner-Centered Curriculum and Instruction, 12(1), 29–49.
*Kwon, J. E., & Lee, S. H. (1999). The meta-analysis of juvenile delinquency variable. The Korean Family Welfare Association, 4(1), 79–101.
*Kwon, Y. B., & Shin, M. S. (2012). Meta-analysis of the effects of academic counseling program. Journal of Learner-Centered Curriculum and Instruction, 12, 23–41.
*Lee, Y. M., & Cho, Y. J. (2014). A meta-analysis of the effects of creativity training programs in elementary practical arts. Journal of Practical Arts Education, 27, 19–40.
*Lee, H. S., Choi, G. H., Yoo, Y. K., & Lim, S. I. (2012a). Meta-analysis of study of the education and the effective treatment programs for the aged. Journal of Welfare for the Aged Institute, 56, 93–114.
*Lee, S. Y., & Chung, H. J. (2009). Research trend of enneagram and a meta-analysis study on the effects of enneagram programs. The Korean Family Welfare Association, 14(2), 121–137.
*Lee, M. A., Kim, J. E., & Hwang, S. Y. (2013). A meta-analysis on efficacy of self-management strategies for students with disabilities. Journal of Emotional & Behavioral Disorders, 29, 293–310.
*Lee, S. Y., Lee, Y. J., & Jeong, H. S. (2002). The effects of parent education programs through the meta-analysis. Korean Psychology Association, 14(3), 637–653.
*Lee, H. S., Son, J. N., Lee, K. D., Lim, S. I., & Choi, K. M. (2012b). Meta-analysis of research on art therapy programs for the disabled. Korea Academy of Disability and Welfare, 17, 91–109.
*Lee, G. O., & Yeo, T. C. (2012). The meta-analysis on the effects according to mediation variables of career counseling programs for elementary school students. The Journal of Korea Elementary Education, 23, 1–17.
*Lee, K. S., & Yeun, Y. R. (2012). A systematic review of research on acculturative stress among married immigrant women in Korea. Korean Journal of Health Psychology, 17, 697–709.
Light, R. J., & Pillemer, D. B. (1984). Summing up. Cambridge: Harvard University Press.
Light, R. J., Singer, J. D., & Willett, J. B. (1994). The visual presentation and interpretation of meta-analuses. In H. M. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis. New York: Russell Sage Foundation.
*Lim, H. J., Jun, B. U., & Kim, J. M. (2013). Literature review of smart learning for individuals with autism spectrum disorder ASD: Single subject researches. Journal of Emotional & Behavioral Disorders, 29, 355–381.
*Lim, E. M., & Lim, C. O. (2003). A meta-analysis on the effectiveness of group career guidance and counseling programs in Korea. Korea Youth Counseling Institution, 11(2), 3–11.
*Moon, D. K. (2012a). A meta-analysis on related protective variables on the suicidal ideation of older adults. Korean Council on Social Welfare Education, 17, 144–166.
*Moon, D. K. (2012b). A meta-regression analysis on related triggering variables on the suicidal ideation of older adults. Korean Society of Welfare for the Aged, 55, 133–158.
*Moon, D. K., & Kim, Y. H. (2011). A meta-regression analysis on related triggering variables of adolescents’ suicidal ideation. Korean Counseling Association, 12(3), 945–964.
National Research Council. (1992). Combining information: Statistical issues and opportunities for research. Washington, DC: National Academy of Science Press.
*Oh, S. S., & Goo, B. D. (1999). Korean-type academic achievement related variables in a search through the meta-analysis. The Journal of Educational Research, 3(4), 99–122.
*Oh, I. S., & Kim, C. (2009). Meta-analysis on the effects of the prevention and intervention programs for internet addiction. Korean Association of Information Education, 13(4), 529–537.
Orwin, R. G. (1983). A fail-safe N for effect size in meta-analysis. Journal of Educational Statistics, 8, 157–159.
Orwin, R. G., & Boruch, R. F. (1982). RRT meets RDD: Statistical strategies for assuring response privacy in telephone surveys. Public Opinion Quarterly, 46(4), 560–571.
*Park, J. A. (1998). Features and utility of the educational non-graded system. The Journal of Educational Research, 36(2), 67–98.
*Park, S. M. (2009a). A meta-analysis on the effects of adolescent internet addiction group counseling program in Korea. Korean Psychology Association, 21(3), 607–623.
*Park, S. M. (2009b). Towards an evidence-based practice: The effectiveness, durability, and cost-effectiveness analysis of smoking cessation programs for pregnant women. Far East University Social Welfare Studies Association, 5, 7–31.
*Park, H. J. (2011). The relationship of team learning behavior to team performance: Meta-analysis. Korean Psychology Association, 24(3), 651–672.
*Park, J. W., & Lee, E. J. (2012). A research synthesis of milieu language teaching and PECS for individuals with autism spectrum disorders. Communication Sciences and Disorders, 17, 466–487.
*Park, B. G., & Lim, S. I. (2010). A meta-analysis of test anxiety with related variables. Korean Educational Psychology Association, 24(4), 875–894.
*Park, H. G., Nam, S. K., & Yang, E. J. (2011). Relationships of burnout with job attitudes and turnover intention among Koreans: A meta-analysis. Korean Psychology Association, 24(3), 457–491.
Raudenbush, S. W. (1988). Estimating change in dispersion. Journal of Educational and Behavioral Statistics, 13(2), 148–171.
Raudenbush, S. W. (1994). Random effects models. In The handbook of research synthesis (pp. 301–321).
*Rho, S. D., & Kim, K. H. (2004). A meta-analysis on the effectiveness of peer counseling conducted in Korea. Korea Youth Counseling Institution, 12(2), 3–10.
Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638.
*Ryu, C. H. (1995). A meta-analysis on the effectiveness of courseware. The Journal of Educational Research, 33(3), 343–372.
*Ryu, S. H. (2001). The meta-analysis of the variables related to children’s self-concept. Korean Association of Practical Arts Education, 14(2), 115–126.
Schmidt, F. L., & Hunter, J. E. (2014). Methods of meta-analysis: Correcting error and bias in research findings. Thousand Oaks: Sage publications.
Schmidt, F. L., Oh, I. S., & Hayes, T. L. (2009). Fixed-versus random-effects models in meta-analysis: Model properties and an empirical comparison of differences in results. British Journal of Mathematical and Statistical Psychology, 62(1), 97–128.
*Shin, S. I. (2008). The effects of children’s exposure to domestic violence on juvenile delinquency: A meta-analytic review. Korean Academy of Family Social Work, 23, 153–182.
*Shin, J. H. (2011). Meta-analysis of the effects of lessons using a scaffolding strategy. The Korean Society for Child Education, 24(2), 25–46.
*Shin, S. I. (2012). Research trends in neuro-linguistic programming and meta-analysis on counseling effect. Korean Counseling Association, 13(1), 291–310.
*Shin, M. S., & Kang, C. Y. (2009). A meta-analysis on the effect of creativity education programs in elementary school. The Korean Society for Child Education, 22(3), 113–135.
*Shin, S. I., & Kim, Y. H. (2013). A meta-analysis on related variables of elders’ quality of life. Korea Journal of Counseling, 14, 3673–3690.
*Son, M. H., & Song, Y. H. (2009). Study on group play therapy program effect by meta-analysis. Korean Association for the Psychological Rehabilitation Children, 13(1), 19–43.
*Song, J. Y. (2012). A meta-analysis of achievement goal-academic achievement relationships. Korean Educational Psychology Association, 26(1), 225–250.
Sterne, J. A. C., & Egger, M. (2005). Regression methods to detect publication and other bias in meta-analysis. In H. Rothstein, A. J. Sutton, & M. Borenstein (Eds.), Publication bias in meta-analysis: Prevention, assessment and adjustments (pp. 99–110). Chichester: Wiley.
Suh, M.-O. (2011). The review of meta-analysis and research methodology proposed. Korean J Educ Res, 49(2), 1–23.
*Won, K. S., & Park, S. H. (2010). A meta-analysis of the effects of art therapy. Korean Counseling Association, 11(2), 429–447.
*Yang, H. K., & Park, W. H. (2012). A meta-analysis of PECS interventions using the improvement rate difference. The Korean Society of Special Education, 14(2), 73–90.
*Yeun, Y. R., & Lee, K. S. (2012). Meta-analysis of the effectiveness on psychosocial interventions for alcohol addiction. Korean Journal of Health Psychology, 17, 911–926.
*Yeun, Y. R., & Lee, K. S. (2013). Meta-analysis of the effectiveness on psychosocial rehabilitation program. Korean Journal of Health Psychology, 18, 285–302.
*Yoo, S. J., & Kim, H. J. (2011). Meta-analysis of aggression reduction group counseling for children and adolescents. Korean Association for Learner-Centered Curriculum and Instruction, 11(4), 267–292.
*Yoo, S. K., Park, E. S., Kim, S. J., & Cho, H. J. (2014). Meta-analytic review on the relationship between meaning in life and mental health mainly of domestic studies between 1996–2013. Korean Journal of Psychology, 33, 441–468.
*Yoon, H. B., Choi, Y. J., & Kang, M. C. (2011). Mata-analysis of single-subject design studies on positive behavior support in Korea. The Korean Society of Special Education, 13(4), 605–624.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Park, S., Hong, S. The empirical review of meta-analysis published in Korea. Asia Pacific Educ. Rev. 17, 313–324 (2016). https://doi.org/10.1007/s12564-016-9433-x
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12564-016-9433-x