In the fifth edition of Diagnostic and Statistical Manual of Mental Disorders (DSM-5; American Psychiatric Association 2013), Internet gaming disorder is included in Section III, describing clinical conditions possibly to be defined as formal disorders in future editions of DSM. According to the revised manual, Internet gaming disorder is compulsive preoccupation with online games. This new inclusion was based on a comparably comprehensive literature on the pathological Internet use of online games. Resembling substance-related addiction disorders and pathological gambling, the nine diagnostic criteria proposed for Internet gaming disorder are organized into the areas of excessive use, withdrawal, tolerance, mood modification and negative repercussions. Despite the different labels, Internet gaming disorder is often referred as Internet addiction and Internet use disorder. Although the disorder is restricted to Internet gaming in DSM-5, studies on Internet addiction and Internet use disorder commonly implicate multiple forms of Internet use (e.g. Fu et al. 2010; Griffiths 2000; Müller et al., 2013). Nevertheless, some studies suggested that excessive computer gaming is only a subtype of Internet addiction (Block 2008; Young 2000). Owing to the absence of an official diagnostic criterion in the earlier editions of DSM, no assessment instruments are established as the gold standard of screening or diagnosing Internet addiction (Shaw and Black 2008).

Based on the constructs of impulse control disorder and substance use disorder, Chen et al. (2003) recommended Internet Addiction Disorder (IAD) as a diagnosis characterized by the core symptoms of Internet overuse and associated problems. The core symptoms of Internet overuse included tolerance, compulsiveness and withdrawal symptoms, whereas the problems associated with Internet overuse included neglect of family and social life, compromised work and study, concealment of the use and physiological discomforts (e.g., headaches, sleep disturbances, backaches, sleeping irregularities). The Chen Internet Addiction Scale (CIAS; Chen et al. 2003) was developed to assess pathological and addictive Internet use on the basis of this conceptual framework. The revised version, the CIAS-R, consisting of 26 items was modified from the original CIAS. In exploratory factor analysis (EFA) of the CIAS-R, Core Symptoms of Internet Addiction (IA-Sym) and Related Problems of Internet Addiction (IA-RP) emerged as two clear factors underlying CIAS-R among Taiwan university students (Chen et al. 2003). As Chen et al. hypothesized, items tapping Tolerance (Sym-T), Compulsive Use (Sym-C) and Withdrawal Symptoms (Sym-W) emerged as separate factors under IA-Sym, and items tapping Interpersonal and Health-Related Problems (RP-IH) and Time Management Problems (RP-TM) loaded separately on two different factors under IA-RP. In total, the revised instrument CIAS-R consists of five factors. Among the body of assessing instruments of Internet addiction, the CIAS-R is the only instrument that was specifically developed for the Chinese populations across the straits. Bai and Fan (2005) later suggested that CIAS-R, with modifications, is an appropriate assessment of Internet dependence of Mainland Chinese college students. After removing two items with unclear meaning (i.e. Item 4 and 7) and five items with ambiguous factor loadings in the EFA (i.e. Item 2, 11, 17, 22 and 25), 10 and 9 items were retained for the components IA-Sym and IA-RP, respectively. Compulsive Use and Withdrawal Symptoms (Sym-CW) and Sym-T, and RP-IH and RP-TM were identified respectively for the components IA-Sym and IA-RP. Although shorter in length, the abridged CIAS-R reported strong internal consistency (α = 0.90), and satisfactory convergent validity as evidenced by its significant correlations with Young’s (1996) Diagnostic Questionnaire (YDQ) and Morahan-Martin and Schumacher’s (2000) Pathologic Use Scale (PUS).

The CIAS-R is a popular tool for the assessment of Internet addiction or pathological Internet use, in particular among the Chinese populations (e.g. Cheung and Wong 2011; Ko et al. 2009; Lin et al. 2011; Tokunaga and Rains 2010; Tsai et al. 2009; Yen et al. 2007, 2008). To identify individuals with pathological Internet addiction behaviors, Ko et al. (2005) derived cutoffs for the CIAS-R. A cut-off point of 57/58 and 63/64 for the 26-item CIAS-R was used for the two-stage diagnostic and one-stage epidemiological screenings in adolescents. Thus, according to the findings of the studies among Taiwan and mainland Chinese university students, the CIAS-R per se is a reliable instrument with either a four-factor or five-factor structure reflecting the core symptoms and related problems of Internet addiction in adult populations. Past results also revealed a negative impact of Internet use on social involvement and psychological well-being primarily in adolescents, but not adults (Kraut et al. 1998, 2002). It was speculated that Internet addicted adolescents are more likely to leave real-life social relationships for virtual ones. From the developmental perspective, adolescence is the critical stage to resolve the crisis of identity (Erikson 1963) and increased capacity for cognitive control (Luna et al. 2004). When compared with mature adults, adolescents may be more reliant on the cyber world to achieve role experimentation and interpersonal exploration.

This study aimed to examine the applicability of the CIAS-R in Hong Kong Chinese adolescents. The fitness of factor structure of Internet addiction assessed by CIAS-R was tested with a confirmatory approach. The criterion-related validity of the CIAS-R in explaining adolescents’ general health and social functioning was examined. Moreover, the sensitivity and specificity of the CIAS-R cutoff points in Hong Kong Chinese adolescents were determined. Furthermore, the prevalence of Internet addiction defined by the CIAS-R was compared with the Internet Addiction Test (IAT) scale (Young 1998).

Methods

Students in Grade 7 to Grade 13 from local coeducation secondary schools were invited to participate in a survey in 2012. They were briefed by the teachers to complete a structured questionnaire in a 40-min class. A total of 920 questionnaires were distributed and 896 were returned, resulting in a response rate of 97.4 %. Among the students who completed the questionnaire, 38.0 % were boys and 62.0 % were girls, with mean age of 15.90 ± 3.48 years. The questionnaire included the Chinese version of the CIAS-R, the IAT, the Health of the Nation Outcome Scales for Children and Adolescents (HoNOSCA), and items assessing the demographic background, Internet use, and related health behaviors. The quality of the translated items was assured by conducting a forward and backward translation procedure and a face validity test before the final approval. Sixty-six cases with insufficient completion of the CIAS-R were excluded and 860 cases were used for the analysis. Participation was voluntary with informed consent and ethical approval was sought from the University Research Ethics Committee.

Instruments

The Revised Chen Internet Addiction Scale (CIAS-R)

The CIAS-R is a 26-item self-report questionnaire with a four-point Likert scale ranging from 1 Does not match my experience at all to 4 Definitely matches my experience. The minimum and maximum scores of the CIAS are 26 and 104, respectively. Higher scores indicate a more severe level of Internet addiction. Adopting the recommended cutoffs for Taiwanese adolescents (Ko et al. 2005), respondents with CIAS-R scores of 64 or above were classified as Internet addicted. Cronbach’s alpha of the total score of CIAS-R was reported to range from 0.90 (Bai and Fan 2005) to 0.95 (Lin et al. 2011) in university students, and was 0.97 in the present study.

The Internet Addiction Test (IAT)

The IAT (Young 1998) consists of 20 items in Likert scale from 1 (rarely) to 5 (always), and is a widely used instrument in the West (e.g., Durkee et al. 2012; Johansson and Götestam 2004). Validation studies on its psychometric properties have been concluded in both university students (Jelenchick et al. 2012; Widyanto et al. 2011) and adults (Widyanto and McMurran 2004). Recent work in Chinese adolescents (Lai et al. 2013) and adults (Chang and Man Law 2008) has consistently indicated a 3-factor structure of IAT – Withdrawal and Social Problems, Time Management and Performance, and Reality Substitute. The possible range of the total IAT is 20 to 100. Respondents who obtained a total IAT score of 70 or above were classified as addictive Internet users (i.e. excessive Internet use causing significant problems in life); those with a score between 40 and 69 were classified as problematic Internet users (i.e. excessive internet use experiencing frequent problems in life because of the Internet); and those with a score of 39 or below were classified as average Internet users (i.e. having no problem of controlling Internet use) (Young 1998). A high reliability of the IAT was consistently reported in adolescents with a Cronbach’s alpha over 0.80 in the past studies (Bayraktar and Gün 2007; Milani et al. 2009; Wang et al. 2011), and 0.93 in the present study.

The Health of the Nation Outcome Scales for Children and Adolescents (HoNOSCA) Section A

HoNOSCA was developed by the Royal College of Psychiatrist in UK to measure the health and social functioning of children and adolescents with severe mental illness at admission, review and discharge in inpatient and outpatient mental health services. The HoNOSCA consists of 13 items rated on a 5-point scale (0 = no problem; 5 = severe to very severe problem), belonging to four domains: Behaviour (Item 1–3), Impairment (Item 4–5), Symptoms (Item 6–8), and Social (Item 9–12). Subscale scores and a total score can be created by summation of individual item scores. Its construct, concurrent, predictive validity, and internal, inter-rater and test-retest reliability, feasibility and utility, and levels of acceptability in multi-disciplinary practice are satisfactory (Gowers et al. 1999, 2000; Harnett et al. 2005). A Cronbach’s alpha value of 0.93 is reported in this study.

Statistical Analysis

SPSS version 19.0 and EQS 6.1 (Bentler 2005) were used to analyze the data. Confirmatory factor analysis (CFA) was performed to validate the potential models of CIAS-R obtained by EFA in the literature. The first model tested was the 26-item five-factor model proposed by Chen et al. (2003). The second model was the 19-item four-factor model proposed by Bai and Fan (2005). Due to the findings of high inter-factor correlations (r = 0.78–0.91 for Chen et al.’s (2003) five-factor model; r = 0.75–0.95 for Bai and Fan’s (2005) four-factor model), the corresponding second-order structures were submitted for analyses as well. In every model, each item of the CIAS-R was modeled as the reflective indicator with no inter-related error variances, and one of the indicators of each factor was fixed to one. Variances and covariances of all factors were freely estimated. For the second-order model, all factors on the first level were fitted freely on a single second-order factor. By convention, a value of 0.90 or more in the comparative fit index (CFI), the normed fit index (NFI) and the non-normed fit index (NNFI), and a value less than 0.08 in the root mean square error of approximation (RMSEA) indicate a well-fitted model (Bentler 1992; MacCallum et al. 1996). For the nested model, chi-square difference tests were conducted to evaluate whether there is significant decrease in the goodness-of-fit in the unrestricted model. Akaike’s (1987) information criterion (AIC) were used to evaluate the relative fit of non-nested models. A smaller value of AIC indicates a better fit of the model. To understand the sources of model misspecification, residual statistics and modification indices of the best-fit model were examined.

Furthermore, Cronbach’s alpha coefficient, item-total and inter-item correlations were computed to examine the internal consistency of the CIAS-R. Incremental validity of the CIAS-R was then provided by illustrating that the CIAS-R is associated with psychological wellbeing beyond the effect of amount of Internet use. Two hierarchical multiple regression analyses were conducted with HoNOSCA-assessed psychological well-being as the criterion variable and CIAS-R scores (Block 3) entered as predictors after age, sex (Block 1) and amount of time spent online during weekday and weekend or IAT total score (Block 2).

Results

Factor Structures

The mean subscale and total scores of CIAS-R, IAT and HoNOSCA are presented in Table 1. Chen et al.’s (2003) first-order five-factor (Model 1) and its second-order five-factor (Model 2), as well as Bai and Fan’s (2005) first-order four-factor (Model 3) and its second-order four-factor (Model 4) were tested. After excluding cases with missing answers in any items of the 19-item and 26-item version of CIAS-R, a final sample of 800 participants were included for Model 1 and 2, and 812 participants for Model 3 and 4. Goodness-of-fit indices of the four models are shown in Table 2. No non-significant chi-square test results were obtained from the proposed models (Model 1–4), but the chi-square test was not relied on as the major index for rejection of models due to its sensitivity to sample size. When other fit indices were considered, they further confirmed that all models provided acceptable fit to the data well and met the cut-off criteria of NFI, NNFI, CFI, and RMSEA.

Table 1 Mean and standard deviation of subscale and total scores of the CIAS-R, IAT, and HoNOSCA
Table 2 Goodness of fit indices of hypothesized CIAS-R models in confirmatory factor analysis

In comparison, the two first-order models provided better fit than their corresponding higher order structures. The chi-square tests indicated significant differences between Model 1 and Model 2, Δχ2 = 77.50, Δdf = 5, p < 0.001, and between Model 3 and Model 4, Δχ2 = 30.72, Δdf = 4, p < 0.001. The value of AIC was lower for Model 3 (ΔAIC = 472.69), indicating that it fit better than Model 1 when both the model fit and model parsimony were considered. Given a model describing the data well, the residual values of the model should be small and evenly distributed. Standardized residuals of Model 3 were in the range of −0.083 for Item 1 and 3 to 0.18 for Item 9 and 20 (mean = 0.035) suggesting no salient localized areas of misfit; no relationships among the indicators are substantially under- or over-estimated by the model’s parameter estimates. Furthermore, results of the Wald test suggested that no estimated parameters of Model 3 should be dropped. Multivariate Lagrange Multiplier (LM) test was then conducted to locate the potential parameters to be added in order to improve the model fit, and no cross-factor loadings were suggested to be included. The final model resulted in moderate to strong factor loadings, ranging from 0.53 to 0.88, ps < 0.05. The standardized statistics of factor loadings and factor correlations of model 3 are presented in Fig. 1.

Fig. 1
figure 1

Standardized parameter estimates for the first-order four-factor confirmatory factor analysis of the CIAS-R

Internal Consistency

The internal reliability (Cronbach’s alpha) of the short form CIAS-R were satisfactory (CIAS-Total = 0.95, Sym-CW = 0.89, Sym-T = 0.85, RP-IH = 0.84, RP-TM = 0.83). The internal reliability of the items was further supported by the strengths of the item-total correlations and inter-item correlations among the individual items. Item-total correlations for the 19 items ranged from 0.58 for Item 1 to 0.77 for Item 24. These indicated that excluding these items could further improve the Cronbach’s alpha estimates. Moreover, the inter-item correlations (mean = 0.51) ranged from 0.35 between Item 1 and Item 13, to 0.75 between Item 23 and Item 26 (ps < 0.001).

Incremental Validity

Correlation analyses yielded significant positive correlations of the short-form CIAS-R with the hours spent weekly on Internet activity (r = 0.28, p < 0.01 for weekdays; r = 0.39, p < 0.01 for weekends). Results of the hierarchical multiple regression analyses (see Table 2) indicated that all the three sets of variables, demographic background, time of Internet use, and CIAS-R subscale scores were all significant predictors, which together accounted for 23.5 % of variability in the HoNOSCA score. The incremental validity of the CIAS-R was shown as the set of CIAS-R subscale scores accounted for 11.7 % of variability in the HoNOSCA score (which is above the variability accounted for by demographic background and amount of Internet use of the participants) (see Table 3), and 4.4 % of total variances of HoNOSCA after controlling for the effects of demographic backgrounds and the IAT total score (see Table 4).

Table 3 Hierarchical multiple regression analyses predicting the score of the HoNOSCA by sex, age, amount of internet use and CIAS-R scores (N = 808)
Table 4 Hierarchical multiple regression analyses predicting the score of the HoNOSCA by sex, age, IAT total score and CIAS-R scores (N = 852)

Classification Sensitivity and Specificity

A moderate to strong zero-order correlation of the CIAS-R and IAT total scores was found, r = 0.51, p < 0.001, suggesting the concurrent validity of the CIAS-R. While a total of 155 participants (18 %) were identified as Internet addicts according to the total score of the CIAS-R, 271 (31.5 %) and 25 (2.9 %) participants were classified as excessive and addictive Internet users, respectively. Treating CIAS-R as the predictor of IAT-determined excessive and addicted Internet user membership (i.e., the IAT total score of 40 or above), the sensitivity, specificity, positive predictive value and negative predictive value were 0.42, 0.95, 0.83, and 0.76, respectively. When only the group membership of IAT-determined addicted Internet user was treated as the standard (i.e., the IAT total score of 70 or above), the respective figures became 0.80, 0.84, 0.13, and 0.99.

Discussion

DSM-5 has responded to the need to define Internet addiction problems for psychological and psychiatric practices (American Psychiatric Association 2013). A number of researchers have developed their own instrument for assessing Internet addiction deriving from different conceptual frameworks (Davis 2001; Huang et al. 2007; Young 1998). Among them, the CIAS is a well-established and widely-used scale in the Chinese community. In this sample, according to the 63/64 cutoff of the CIAS-R, the prevalence of Internet addiction was 18 % in Hong Kong adolescents. This prevalence is similar to 17.2 % as reported in a previous studies assessed by CIAS-R in Hong Kong adolescents (Cheung and Wong 2011). Concerning concurrent validity, the CIAS-R could correctly classify more than 80 % of respondents who would also be diagnosed as addicted Internet user or non-addicted users by Young’s (1998) IAT. The poor positive predictive value, but good negative predictive value of CIAS-R further suggest that the CIAS-R is more inclusive in detecting Internet addicted users than the IAT. These results alert future researchers to interpret the prevalence of Internet addiction assessed by the CIAS-R with caution. We believe that individuals being classified as Internet addicts by the CIAS-R may be better defined as problematic Internet users who are at an early stage of Internet addiction. With the DSM-5 definitions for diagnosis of Internet gaming disorder, the validity of CIAS-R could be tested further in well-designed clinical studies.

Results of this study further suggest the construct validity and internet consistency of the CIAS-R scale in adolescents. In terms of its clinical utility, we found that the CIAS-R could account for additional variance in HoNOSCA score reflecting one’s health and social functioning, after demographic characteristics and amount of internet use had been taken into account. This suggests that CIAS-R may tap dimensions underlying the construct of Internet addiction aside from excessive Internet use. Furthermore, the unique contribution of the CIAS-R in accounting for HoNOSCA scores suggests that the CIAS-R may merit a consideration as an Internet addiction assessing instrument.

Resembling the factor structure found in Taiwanese and mainland Chinese adults (Bai and Fan 2005; Chen et al. 2003), the construct of Internet addiction was confirmed to be a four-factor structure with the dimensions Compulsive Use and Withdrawal, Tolerance, Interpersonal and Health-related Problems, and Time Management Problems in this study. Such a structure was largely consistent with the original conceptual model of Internet addiction proposed by Chen et al. (2003) characterizing Internet addiction with factors subsumed under the realm of its core symptoms and related problems. At the same time, this may suggest that compulsive use and withdrawal symptoms should be merged, instead of treated separately as two factors. In the past studies (Chen et al. 2003; Ko et al. 2009), the total score of the CIAS-R was significantly associated with amount of time spent on the Internet. Replicating these findings, we have further shown that adolescents spending more internet time during weekends had poorer mental health and social-behavioral functioning. Nevertheless, we did not differentiate the purposes of Internet use. With the popularity of e-learning in Hong Kong, spending a great deal of time online to learn and finish school assignments during weekdays may be inevitable. This may explain the fact that time spending online during weekends was more reflective of addictive internet use and associated with poorer mental well-being.

In the literature, CIAS-R-assessed internet addiction was found to be related to aggressive behaviors (Yen et al. 2011), attention-deficit/hyperactivity disorder (Yen et al. 2009b), alcohol use (Yen et al. 2009a), and negative psychophysiological outcomes (Lu et al. 2010) in Taiwanese young adults; poorer frustration tolerance in Taiwanese adolescents (Ko et al. 2008); and sleep problems in Hong Kong adolescents (Cheung and Wong 2011). These findings are consistent with our correlations between CIAS-R subscales and the general psychological outcomes assessed by the HoNOSCA. To determine the major domain most impacted by internet overuse, Pearson’s correlation coefficients between CIAS-R total score and each HoNOSCA item were computed. This follow-up analysis revealed that r ranged from 0.28 for over-activity, attention or concentration to 0.37 for peer relationships (ps < 0.001). The magnitude of the correlation coefficients suggested that peer relationships and school attendance (r = 0.35) may be the major areas that warrant extra attention from parents and teachers of adolescents with high CIAS-R scores.

Several limitations of our study are noteworthy. Results of a previous study in Taiwan (Chen et al. 2003) suggested that the CIAS yielded temporarily stability (2-week test-retest reliability = 0.83), although no such information has been reported using the CIAS-R. The cross-sectional nature of the current data does not allow for test-retest reliability. Moreover, the factor structure of CIAS-R-assessed Internet addiction may vary across samples. The structure of the CIAS-R found in the present study may not be generalisable to other populations. Therefore, psychometric validation of the CIAS-R in Western populations is warranted. Given that face-to-face diagnostic interviews are costly and time-consuming, and no diagnostic interview has been established, the diagnostic efficiency of the CIAS-R was examined with the other self-reported measures. We chose Young’s 20-item IAT as a reference because of its popularity (Durkee et al. 2012; Huang 2010) and good psychometric properties (Jelenchick et al. 2012; Widyanto et al. 2011; Widyanto and McMurran 2004). Nonetheless, future investigations are needed to examine the validity of the proposed cutoffs for the CIAS-R, as the positive predictive value of the CIAS-R was not satisfactory in this study. To our knowledge, this is the first study examining the psychometric properties of CIAS-R in Hong Kong Chinese adolescents. The confirmatory factor analytic approach of this study allows a test of the latent factor structure of CIAS-R derived from exploratory factor analysis (EFA) approach (Bai and Fan 2005; Chen et al. 2003).

As an important contribution to the literature, this study articulated the findings of the core symptoms and problems pertaining to Internet addiction in Hong Kong adolescents with those reported in Taiwanese and mainland Chinese adults. The CIAS-R could possibility to be used in non-Chinese populations after robust modifications and validation. The 4-factor model (Compulsive Use and Withdrawal, Tolerance, Interpersonal and Health-related Problems, and Time Management Problems) found in other Chinese communities, represented a satisfactory fit to the current data in Hong Kong Chinese adolescents. The CIAS-R also exhibited satisfactory internal consistency, concurrent and incremental validity. Nonetheless, CIAS-R may have over-estimated the prevalence of Internet addiction, which may lower its value to be a screening instrument of Internet addiction. Further studies are needed to establish the norms and diagnostic cutoff points for the CIAS-R for different populations.