Validity and reliability of the Thai version of the Autism Treatment Evaluation Checklist: A two-phase diagnostic accuracy study

Kanitha Sunakarach; Pattapong Kessomboon

doi:10.12688/f1000research.14537.1

Home Browse Validity and reliability of the Thai version of the Autism Treatment...

ALL Metrics

Views

Downloads

Get PDF

Get XML

Export

▬

✚

Research Article

Validity and reliability of the Thai version of the Autism Treatment Evaluation Checklist: A two-phase diagnostic accuracy study

[version 1; peer review: 1 approved, 1 approved with reservations]

Kanitha Sunakarach¹, Pattapong Kessomboon ¹

PUBLISHED 03 May 2018

Author details Author details

¹ Department of Community Medicine, Faculty of Medicine, Khon Kaen University, Khon Kaen, 40002, Thailand

Kanitha Sunakarach
Roles: Conceptualization, Data Curation, Formal Analysis, Methodology, Project Administration, Resources, Software, Validation, Writing – Original Draft Preparation

Pattapong Kessomboon
Roles: Conceptualization, Formal Analysis, Supervision, Validation, Writing – Review & Editing

OPEN PEER REVIEW

REVIEWER STATUS

Abstract

Background: This study aimed to evaluate the psychometric properties of the Thai version of the Autism Treatment Evaluation Checklist (Thai-ATEC); a tool which has been developed for Thai parents and caregivers who have children with autism spectrum disorder (ASD).
Methods: Approval for this study was first obtained from the appropriate Ethics committee and from the original Autism Treatment Evaluation Checklist (ATEC) developers. This was a two-phase study. Phase 1 consisted of the forward–backward translation of the ATEC and phase 2 included the testing of psychometric properties, i.e. the validity and reliability of the final draft of the tool. The validity of the tool was assessed by comparing Thai-ATEC scores of parents and caregivers of 160 children with ASD with the assessment of a child and adolescent psychiatrist using DSM-V criteria on the same group of children. The inter-rater reliability of the tool was tested using a two-way model of intra-class correlation coefficient (ICC) for two-parent/caregivers’ assessment of 50 children with ASD.
Results: The validity of the Thai-ATEC was moderate to high. A cut-off point of ≤38 scores was used to distinguish between children with ASD with mild symptoms and the rest of the children (sensitivity = 94%, specificity= 61.9%, and the area under ROC curve = 90%). A cut-off point of ≥68 scores was used to distinguish between children with ASD with a severe degree of symptoms and the rest (sensitivity = 94%, specificity = 62.8%, area under receiver operating characteristic curve = 85%). The inter-rater reliability was very strong (ICC = 0.97).
Conclusions: The Thai-ATEC has moderate to high validity and high reliability.

Keywords

Autism Treatment Evaluation Checklist, Thai version Autism Treatment Evaluation Checklist, validity, reliability, autism spectrum disorder

Corresponding author: Pattapong Kessomboon

Competing interests: No competing interests were disclosed.

Grant information: The study received partial financial support from the Graduate School of Khon Kaen University.
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Copyright: © 2018 Sunakarach K and Kessomboon P. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Data associated with the article are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).

How to cite: Sunakarach K and Kessomboon P. Validity and reliability of the Thai version of the Autism Treatment Evaluation Checklist: A two-phase diagnostic accuracy study [version 1; peer review: 1 approved, 1 approved with reservations]. F1000Research 2018, 7:538 (https://doi.org/10.12688/f1000research.14537.1) First published: 03 May 2018, 7:538 (https://doi.org/10.12688/f1000research.14537.1) Latest published: 03 May 2018, 7:538 (https://doi.org/10.12688/f1000research.14537.1)

Introduction

Autism spectrum disorders (ASD) are a cluster of neurodevelopmental disorders that are characterized by abnormalities in social interactions and verbal and nonverbal communication, and restricted, repetitive and stereotyped patterns of behavior, interests and activities¹. ASD symptoms begin in the early years after birth, generally before 3 years of age². The prevalence of ASD has increased over recent years in many parts of the world³. In Thailand, it was estimated that there were 180,000 children with ASD nationwide (i.e. 2.8 per 1,000 population or 15 per 1,000 children age <15)⁴. However, this may underestimate the true scale. Accurate diagnosis needs adequately trained health professionals and appropriate tools⁵.

Many assessment tools have been developed to screen and diagnose children with autism, such as the Autism Diagnostic Interview-Revised (ADI-R)⁶, the Pre-Linguistic Autism Diagnostic Observation Schedule (PL-ADOS)⁷, and the Childhood Autism Rating Scale (CARS)⁸. These tools are mostly used by intensively trained mental health professionals. These tools are time-consuming, however; for example, the ADI-R takes 2 h to complete and the PL-ADOS takes 30 min. Scores derived from the tools had low sensitivity with older children with ASD⁹ and these tools are not readily available to caregivers. Although these tools are appropriate for research, because of these limitations, other tools have been investigated that have been standardized for evaluating the changes of symptoms of children with autism, one example of which is the ATEC¹⁰.

The ATEC was developed by Rimland and Edelson at the Autism Research Institute¹¹. The tool has been designed to enable parents to assess their children with ASD. It has increased the effectiveness of home caring because it allows for confirmation of the effectiveness of treatments provided^10,12. This tool can be used by parents, caregivers, teachers, professionals and researchers. The ATEC has been translated to 18 languages and is widely used globally^10,13.

The ATEC is a questionnaire consisting of four subscales: 1) speech/language/communication (14 items); 2) sociability (20 items); 3) sensory/cognitive awareness (18 items); and 4) health/physical/behaviour (25 items). The ATEC total score ranges from 0 to 179 with maximum scores on the subscales of 28 (speech/language/communication), 40 (sociability), 36 (sensory/cognitive awareness), and 75 (health/physical/behavior). The higher the subscale and total scores, the more impaired the child.

After testing 1,358 children, results showed a high internal consistency (Pearson correlation coefficient, split-half method) of the total ATEC scores (r=0.94) and high internal consistencies of the subscales (speech/language: r=0.92; sociability: r=0.84; sensory/cognitive awareness: r=0.88; health/physical behavior: r=0.82)¹⁴. Compared with the CARS utilized by a health care professional, the result showed that ATEC scores and CARS scores had a significant correlation (ρ =0.71, p-value <0.0001)¹⁵. It also showed that the predictive value of the ATEC was good. This could be used to continuously follow up the response to treatments¹⁰.

In Thailand, there is a high need for an efficient assessment tool, and a high interest in the ATEC. However, no formal standardized translation of the original ATEC has been done and no research to assess its psychometric properties in the Thai context has been conducted. Therefore, this study aimed to translate the ATEC into Thai and evaluate its psychometric properties.

Methods

This was a two-phase study. Phase 1 consisted of the forward–backward translation of the ATEC and phase 2 included the testing of psychometric properties, i.e. the validity and reliability of the final draft of the tool. The full study protocol can be accessed at the Khon Kaen University graduate school or from the corresponding author upon request.

Phase 1: Forward–backward translation of the ATEC

The authors obtained permission from Dr. Stephen M. Edelson, the original developer of the ATEC at the Autism Research Institute (San Diego, CA, USA), for the use of the ATEC in this study. The forward–backward translation included six processes: 1) studying and gaining understanding of the assessment tool, after which a child psychiatrist work at Child and Adolescent Mental Health Rajanagarindra Institute, Bangkok and clinical psychologist work at Psychiatric Hospital Nakhon Ratchasima Rajanagarindra translated it into the Thai language; 2) synthesis of the first translation; 3) backward translation into the English language by a British translator who had been living in Thailand for over 10 years(Private business at his home in Bangkok), and a professional translator Director and Tutor at Samarthome, Nakhonratchasima, both employed for this particular task; 4) comparing the back translation with the original version for content, sematic, idiomatic, and cultural equivalences by an expert committee comprising six members, including the child psychiatrist, the clinical psychologist, two backward translators, an expert in communication with children with ASD work at Faculty of Medicine Ramathibodi Hospital, Mahidol University, Bangkok, and a parent of a child with ASD with a high English proficiency, Vice president Autism Society of Thailand, Bangkok; 5) trying out the assessment tool, during which the processes were undertaken iteratively until the tool had reached acceptable understandability by stakeholders; and 6) submitting the final tool and documentations to the original ATEC developers. The complete Thai-ATEC can be found in Supplementary File 1.

Phase 2: Testing of the validity and reliability psychometric properties of the final draft tool

The validity of the tool was assessed by comparing the Thai-ATEC scores from parents and caregivers of 160 children with ASD, with the assessment of the same child and adolescent psychiatrist using DSM-V criteria² of the same group of children with ASD. The psychiatrist was blinded to the results of the parallel measure.

The validity indexes included sensitivity, specificity, predictive value, likelihood ratio positive (LR+), likelihood ratio negative (LR−), accuracy, and area under receiver operating characteristic (ROC) curve. These were calculated by exploring an appropriate cut-off point score that was able to distinguish between children with ASD with mild degree of symptoms and the rest of the children. Another appropriate cut-off point score was assessed for its ability to distinguish children with ASD with a severe degree of symptoms from the rest of the children.

The tool’s inter-rater reliability was tested by comparing the Thai-ATEC scores from 50 pairs of parents/caregivers of 50 children with ASD. Each pair of parents/caregivers was independently assess the child with ASD.

Participants

Inclusion criteria were parents/caregivers of children with ASD who were able to read and write. The age of the children with ASD was ≥3 years old. The sample size was calculated based upon data from a pilot study with expected sensitivity = 0.81, expected specificity = 0.60, prevalence = 0.4, desired pecision = 0.10 and a confidence level = 95%. Therefore, 160 parents and caregivers of children with ASD were needed. They were recruited simple random sampling during an appointment with the psychiatrist at the outpatient clinic at the North Eastern Institute of Child and Adolescent Mental Health, Khon Kaen province, Thailand, during May–August 2017.

The tool’s inter-rater reliability was tested using a two-way model of intra-class correlation coefficient (ICC) rater assessment of the same children with ASD. The participants of the inter-rater reliability test consisted of 50 pairs of parents and caregivers who were subsets of the 160. Each pair was selected for this test because both were caring for the same child and accompanied the child on the day of the assessment. Therefore, a child with one parent or one caregiver on those days at the hospital was not selected. After a child was examined by the psychiatrist, each pair of parents and caregivers was invited into a separate room and informed about how to complete the Thai-ATEC. The pairs were not allowed to meet and talk to each other. There was no time limit to fill in the checklist. Parents and caregivers spent an average of 9.21 minutes completing the tool.

Statistical analysis

The validity indexes including sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), accuracy, likelihood ratio (LR+/−), area under receiver operating characteristic (ROC) curve and the appropriate cut-off point of the tool were calculated by comparing scores obtained from the parents’ and caregivers’ Thai-ATEC assessment with the results of the psychiatrist’s assessment using DSM-V criteria. The 95% confidence intervals of these indexes were also estimated. An appropriate cut-off point score was explored based on its ability to distinguish between children with ASD with mild degree of symptoms and the rest of the children. Another appropriate cut-off point score was assessed for its ability to distinguish children with ASD with a severe degree of symptoms from the rest of the children. The tool’s inter-rater reliability was calculated using a two-way model of intra-class correlation coefficient (ICC) rater assessment of the same children with ASD. The SPSS statistical software version 19.0 was used to perform these data analyses.

Ethical approval and consent

Approval for the study was obtained from the Khon Kaen University Ethics Committee in Human Research (HE601001). Parents of children with ASD provided written informed consent for their participation in the research.

Results

Validity of the Thai-ATEC

A total of 160 parents/caregivers met the criteria of selection. Most parents/caregivers were female (113, 70.6%). Their ages ranged from 18–60 years old. Most were mothers (82, 51.2%). Most of the ASD children were male (102, 63.7%). The average age of children with autism was 7.86 years old (SD = 3.43, min = 3 years, max = 18 years).

At a cut-off point of a score of ≤38, to distinguish between children with ASD with mild symptoms and the rest of the children with moderate to severe symptoms, it was found that the tool’s sensitivity was 94% (95% CI, 0.91–0.98), the specificity was 61.9% (95% CI, 0.48–0.78), the PPV was 88.3% (95% CI, 0.82–0.93), the NPV was 81.2% (95% CI, 0.67–0.94), the accuracy was 86.9%, the LR+ was 2.59, the LR− was 0.07, and the area under the ROC curve was 90% (Table 1).

Table 1. Validity of the Thai-Autism Treatment Evaluation Checklist (ATEC) at the cut-off score of ≤38.

Thai-ATEC Score	Cut-off	Sensitivity (95% CI)	Specificity (95% CI)	PPV (95% CI)	NPV (95% CI)	Accuracy	LR+	LR−
Total score	38	94.9% (0.91–0.98)	61.9% (0.48–0.78)	88.3% (0.82–0.93)	81.2% (0.67–0.94)	86.9%	2.59	0.07
Speech/Language/ Communication	5	95.8% (0.79–0.92)	64.3% (0.68–0.92)	92.7% (0.87–0.97)	66% (0.52–0.79)	84.3%	4.39	0.17
Sociability	11	93.2% (0.86–0.96)	57.1% (0.43–0.76)	86.5% (0.80–0.92)	70.5% (0.55–0.85)	83.1%	2.20	0.14
Sensory/Cognitive Awareness	8	94.9% (0.83–0.94)	64.3% (0.54–0.82)	89% (0.83–0.95)	68.2% (0.64–0.92)	83.7%	2.80	0.15
Health/Physical/ Behavior	16	89.8% (0.72–0.87)	54.8% (0.46–0.75)	85.5% (0.79–0.92)	51% (0.37–0.65)	75.0%	2.04	0.33

PPV, positive predictive value; NPV, negative predictive value; LR+, likelihood ratio positive.

At a cut-off score of ≥68 to distinguish between children with ASD with a severe degree of symptoms and the rest of the children with mild to moderate symptoms, it was found that the tool’s sensitivity was 91.3% (95% CI, 0.67–0.98), the specificity was 62.8% (95% CI, 0.55–0.74), the PPV was 28.8% (95% CI, 0.17–0.39), the NPV was 95.7% (95% CI, 0.91–0.99), the accuracy was 68.1%, the LR+ was 2.40, the LR− was 0.26, and the area under ROC curve was 85% (Table 2).

Table 2. Validity of the Thai-Autism Treatment Evaluation Checklist (ATEC) at the cut-off score of ≥68.

Thai-ATEC Score	Cut-off	Sensitivity (95% CI)	Specificity (95% CI)	PPV (95% CI)	NPV (95% CI)	Accuracy	LR+	LR−
Total score	68	91.3% (0.67–0.98)	62.8% (0.55–0.74)	28.8% (0.17–0.39)	95.7% (0.91–0.99)	68.1%	2.40	0.26
Speech/Language/ Communication	15	73.9% (0.50–0.88)	68.6% (0.63–0.78)	28.5% (0.16–0.44)	93.2% (0.88–0.98)	84.3%	4.39	0.42
Sociability	17	87% (0.86–0.96)	52.6% (0.43–0.76)	25.8% (0.80–0.92)	92.8 (0.55–0.85)	66.8%	2.07	0.45
Sensory/Cognitive Awareness	16	82.39% (0.67–0.98)	63.5% (0.53–0.70)	26.7% (0.16–0.37)	95.5% (0.91–0.99)	65.0%	2.17	0.28
Health/Physical/ Behavior	30	87% (0.67–0.98)	56.9% (0.53–0.70)	26.7% (0.16–0.37)	95.5% (0.91–0.99)	65.0%	2.17	0.28

PPV, positive predictive value; NPV, negative predictive value; LR+, likelihood ratio positive.

Therefore, the above findings suggest that children with ASD can be assessed using the Thai-ATEC and the severity of ASD symptoms can be grouped according to the cut-off points found in this study as follows:

Level of mild symptoms: Thai-ATEC scores in the range of 1–38
Level of moderate symptoms: Thai-ATEC scores in the range of 39–68
Level of severe symptoms: Thai-ATEC scores in the range of 69–179

The flow of participants can be seen in Supplementary File 2.

Reliability of the Thai-ATEC

ICCs were calculated to evaluate the inter-rater reliability of the tool. It was found that ICC for the total score was 0.97 (95% CI, 0.97–0.98). When we analyzed each subset of the tool, that is speech/language/communication, sociability, sensory/cognitive awareness and health/physical/behavior, it was found that the ICCs were 0.94, 0.86, 0.89 and 0.97, respectively. These were all more than 0.75, meaning that the assessment was highly consistent¹⁶ (Table 3).

Table 3. Inter-rater reliability of the Thai-Autism Treatment Evaluation Checklist (ATEC).

The first observer was one of the parent/caregiver pair, and the second observer was the other person in the parent/caregiver pair.

Domain	Thai-ATEC scores (mean ± SD)		ICC (95% CI)
Domain	First observer	Second observer	ICC (95% CI)
Total score	69.4 ± 24.65	67.8 ± 24.23	0.97 (0.95–0.98)
Speech/Language/Communication	11.64 ± 7.91	11.32 ± 7.66	0.94 (0.91–0.97)
Sociability	15.98 ± 5.27	16.56 ± 5.90	0.86 (0.75–0.92)
Sensory/Cognitive Awareness	13.38 ± 6.80	14.16 ±6.49	0.89 (0.81–0.94)
Health/Physical Behavior	28.40 ± 10.36	25.82 ± 11.22	0.97 (0.95–0.98)

ICC, intra-class correlation coefficient.

Dataset 1.The Thai-ATEC scores for each child from each parent/caregiver and the corresponding diagnosis from the child psychiatrist.

Discussion

To our knowledge, this was the first study of its kind that attempted to formally translate the ATEC into Thai. Standardized and rigorous translation processes were employed. Beaton et al. proposed guidelines for the process of cross-cultural adaptation of self-report measures, including forward and backward translation to ensure that tools are appropriate and comparable¹⁸. This study has intentionally followed this guideline and employed additional processes which were needed. A child and adolescent psychiatrist, a Thai professional translator, a professional ASD communicator, health professionals responsible for the care of children with ASD, a British professional translator and a parent with a high English proficiency who had a child with ASD were all involved in the translation process to ensure that the final tool was of high quality and that the meaning of the Thai-ATEC was consistent with the original ATEC tool.

The psychometric properties of the Thai-ATEC were moderate to high. The sensitivity of the tool was 94%, its specificity was 62%, and the area under ROC curve was ≥85%. The discovered cut-off points were used to divide the severity of ASD symptoms into three levels: mild, moderate and severe. These levels mild were similar to the original ATEC, However, the results showed that moderate and severe levels were different from the original¹⁴. (Table 4).

Table 4. Symptom severity levels of the original Autism Treatment Evaluation Checklist (ATEC) and the Thai-ATEC.

A comparison of mild, moderate and severe symptoms between the original and new Thai version of the ATEC.

Level of severity of the original ATEC	Level of severity of the Thai-ATEC
Mild (score 0-30)	Mild (score 1–38)
Moderate (score 31-103)	Moderate (score 39–67)
Severe (score 104-179)	Severe (score 68–179)

Parents and caregivers can use the ATEC to monitor children with ASD to assess the severity of impairment in different domains and the progress in response to intervention over time. The ATEC covers not only behavioral issues, but also health and systemic issues, such as sleep problems, seizures, eating, gastrointestinal problems, hyperactivity, self-injuries and sleep disturbance. Health and systemic issues are of high concern among parents¹⁹.

A study in India comparing the CARS and clinical diagnoses found that at a CARS score of ≥33, the sensitivity of the scale was 81.4%, the specificity was 78.6% and the area under the curve was 81%¹⁶. Thus, the Thai-ATEC had higher sensitivity but lower specificity than the Indian CARS. Geier et al. found that the CARS correlated well with the ATEC¹⁵.

The inter-rater reliability of the Thai-ATEC was very strong. The ICC of the total score was 0.97 and the ICCs of subscale 1, 2, 3 and 4 were 0.94, 0.86, 0.84 and 0.97, respectively. The Indian study that compared the CARS with clinical diagnoses by psychiatrists¹⁶ found that inter-rater reliability was only 0.74. An ICC of >0.9 is perceived as indicating high reliability²⁰.

Limitations of this study included the time consuming nature of the processes, owing to the limited number of child and adolescent psychiatrists involved in the study. Child and adolescent psychiatrists are rare in Thailand.

In this study, the experienced child and adolescent psychiatrist who acted as the gold standard, followed the DSM-V criteria and spent 5–10 minutes per person. This relatively short length of time might have caused some mistakes and may have led to misclassification of symptoms that could have affected sensitivity and specificity. However, the single-psychiatrist approach taken in this study may have minimized subjective differences in scoring methods and provided superior consistency to a multi-psychiatrist approach.

This study was conducted in the north-eastern part of Thailand. Therefore, further studies are needed to explore the usability of the new Thai-ATEC in other parts of Thailand, where local dialects are present.

Conclusion

The Thai-ATEC has moderate to high validity and high reliability. The cut-off points at scores of ≤38 and ≥68 were appropriate to distinguish the level of severity between the mild, moderate, and severe ASD symptoms. Parents and caregivers of children with ASD can efficiently use the tool to assess their children at home and communicate the results with health professionals.

Data availability

Dataset 1. The Thai-ATEC scores for each child from each parent/caregiver and the corresponding diagnosis from the child psychiatrist. DOI: 10.5256/f1000research.14537.d202175¹⁷.

Competing interests

No competing interests were disclosed.

Grant information

The study received partial financial support from the Graduate School of Khon Kaen University.

The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Acknowledgements

We wish to thank all the parents, caregivers and children who participated in the Thai-ATEC study. We would also like to thank all the translators and stakeholders who were involved in the study. We sincerely thank Dr. Charnnarong Chaiudomsom and Dr. Suthra Auapisithwong for their hard work in making this study possible. We very much appreciate the strong support provided by the director and staff of the North Eastern Institute of Child and Adolescent Mental Health in Khon Kaen. We express our gratitude to S. M. Edelson of the Autism Research Institute for the permission to use the original ATEC and invaluable suggestions he gave for this study.

Supplementary material

Supplementary File 1. The Thai-Autism Treatment Evaluation Checklist questionnaire used in the current study.

Click here to access the data.

Supplementary File 2. Flow diagram of participants.

Click here to access the data.

Faculty Opinions recommended

References

1. American Psychiatric Association: Diagnostic and statistical manual of mental disorder: DSM-IV-TR. 4^th ed. Washington, DC: APA; 2000. Reference Source
2. American Psychiatric Association: Diagnostic Criteria From DSM-5. Washington, DC: APA; 2013. Reference Source
3. World Health Organization: Meeting report: autism spectrum disorders and other developmental disorders: from raising awareness to building capacity. 2013 Sep 16–18; Geneva, Switzerland. [cited 2017 Jan 22]. Reference Source
4. Kopetz PB, Endowed ED: Autism worldwide: prevalence, perceptions, acceptance, action. J Soc Sci. 2012; 8(2): 196–201. [cited 2017 Dec 15]. Publisher Full Text
5. National Institute for Health and Clinical Excellence: Autism: Recognition, Referral and Diagnosis of Children and Young People on the Autism Spectrum. London: NICE; 2011. PubMed Abstract
6. Lord C, Rutter M, Le Couteur A: Autism Diagnostic Interview-Revised: a revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. J Autism Dev Disord. 1994; 24(5): 659–85. PubMed Abstract | Publisher Full Text
7. Lord C, Risi S, Lambrecht L, et al.: The Autism Diagnostic Observation Schedule-Generic: a standard measure of social and communication deficits associated with the spectrum of autism. J Autism Dev Disord. 2000; 30(3): 205–23. PubMed Abstract | Publisher Full Text
8. Schopler E, Reichler RJ, DeVellis RF, et al.: Toward objective classification of childhood autism: Childhood Autism Rating Scale (CARS). J Autism Dev Disord. 1980; 10(1): 91–103. PubMed Abstract | Publisher Full Text
9. Green J, Charman T, McConachie H, et al.: Parent-mediated communication-focused treatment in children with autism (PACT): a randomised controlled trial. Lancet. 2010; 375(9732): 2152–60. PubMed Abstract | Publisher Full Text | Free Full Text
10. Magiati I, Moss J, Yates R, et al.: Is the Autism Treatment Evaluation Checklist a useful tool for monitoring progress in children with autism spectrum disorders? J Intellect Disabil Res. 2011; 55(3): 302–12. PubMed Abstract | Publisher Full Text
11. Rimland B, Edelson SM: Autism Treatment Evaluation Checklist (ATEC) [online]. 2000; [cited 2016 Dec 20]. Reference Source
12. Weiner RH, Greene RL: Intention-based therapy for autism spectrum disorder: promising results of a wait-list control study in children. Explore (NY). 2014; 10(1): 13–23. PubMed Abstract | Publisher Full Text
13. Jarusiewicz B: Efficacy of neurofeedback for children in the autistic spectrum: a pilot study. J Neurother. 2002; 6(4): 39–49. Publisher Full Text
14. Autism Research Institute: Studies confirm validity of ATEC - 2018. 2018; [cited 2018 Jan 10]. Reference Source
15. Geier DA, Kern JK, Geier MR: A Comparison of the Autism Treatment Evaluation Checklist (ATEC) and the Childhood Autism Rating Scale (CARS) for the Quantitative Evaluation of Autism. J Ment Health Res Intellect Disabil. 2013; 6(4): 255–67. PubMed Abstract | Publisher Full Text | Free Full Text
16. Russell PS, Daniel A, Russell S, et al.: Diagnostic accuracy, reliability and validity of Childhood Autism Rating Scale in India. World J Pediatr. 2010; 6(2): 141–7. PubMed Abstract | Publisher Full Text
17. Sunakarach K, Kessomboon P: Dataset 1 in: Validity and reliability of the Thai version of the Autism Treatment Evaluation Checklist: A two-phase diagnostic accuracy study. F1000Research. 2018. Data Source
18. Beaton DE, Bombardier C, Guillemin F, et al.: Guidelines for the process of cross-cultural adaptation of self-report measures. Spine (Phila Pa 1976). 2000; 25(24): 3186–91. PubMed Abstract | Publisher Full Text
19. Dale E, Jahoda A, Knott F: Mothers' attributions following their child's diagnosis of autistic spectrum disorder: exploring links with maternal levels of stress, depression and expectations about their child's future. Autism. 2006; 10(5): 463–79. PubMed Abstract | Publisher Full Text
20. Nunnally JC: Psychometric theory. 2^nd ed. New York: McGraw-Hill; 1978.

Comments on this article Comments (0)

Version 1

VERSION 1 PUBLISHED 03 May 2018

Author details Author details

¹ Department of Community Medicine, Faculty of Medicine, Khon Kaen University, Khon Kaen, 40002, Thailand

Kanitha Sunakarach
Roles: Conceptualization, Data Curation, Formal Analysis, Methodology, Project Administration, Resources, Software, Validation, Writing – Original Draft Preparation

Pattapong Kessomboon
Roles: Conceptualization, Formal Analysis, Supervision, Validation, Writing – Review & Editing

Competing interests

No competing interests were disclosed.

Grant information

The study received partial financial support from the Graduate School of Khon Kaen University.
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Article Versions (1)

version 1

Published: 03 May 2018, 7:538

https://doi.org/10.12688/f1000research.14537.1

© 2018 Sunakarach K and Kessomboon P. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Data associated with the article are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).

Download

Export To

metrics

	Views	Downloads
F1000Research	-	-
PubMed Central Data from PMC are received and updated monthly.	-	-

Citations

SEE MORE DETAILS

CITE

how to cite this article

Sunakarach K and Kessomboon P. Validity and reliability of the Thai version of the Autism Treatment Evaluation Checklist: A two-phase diagnostic accuracy study [version 1; peer review: 1 approved, 1 approved with reservations] F1000Research 2018, 7:538 (https://doi.org/10.12688/f1000research.14537.1)

NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.

track

receive updates on this article

Track an article to receive email alerts on any updates to this article.

Open Peer Review

Current Reviewer Status: ?

Key to Reviewer Statuses VIEW HIDE

ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions

Version 1

VERSION 1

PUBLISHED 03 May 2018

Views

Reviewer Report 11 Jun 2018

Manote Lotrakul, Department of Psychiatry, Ramathibodi Hospital, Mahidol University, Bangkok, Thailand

Approved with Reservations

https://doi.org/10.5256/f1000research.15824.r34245

The advantages of the ATEC are that it is shorter than other measures, easy to understand and designed to be completed by parents, teachers, or caregivers, making it a suitable tool for a country where there is a shortage of psychiatrists. It provides quantitative measurements on important domains of ASD severity which increased the effectiveness of home-base care for children with ASD.

The authors have translated the ATEC into Thai following Beaton's Guidelines for the cross-cultural translation and adaptation of self-report instruments. The psychometric properties of the Thai-ATEC have been tested and reported as acceptable to good. The methods section and results of the study have sufficient and clear details. The authors are skeptical whether the Thai-ATEC can be used in other parts of Thailand but this may not be the case because the wording in each item is natural, clear and easy to understand.

Comments:

Please change “DSM-V” to “DSM-5“
It would be better if the author also present the other type of reliability, i.e., the internal consistencies of the ATEC scores and subscales.
Page 6 paragraph 4: It is not relevant to compare the Thai-ATEC with the CARS study in India as the purpose of that study was to develop the screening tool whereas the ATEC is designed to measure ASD severity.
Page 6 paragraph 6: This research is time consuming or slow-paced but it is not the limitation of the study.

Is the work clearly and accurately presented and does it cite the current literature?

Yes
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

Yes
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Yes

Competing Interests: No competing interests were disclosed.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

CITE

Report a concern

Author Response 21 Jun 2018

Pattapong Kessomboon, Department of Community Medicine, Faculty of Medicine, Khon Kaen University, Khon Kaen, 40002, Thailand

21 Jun 2018

Author Response

Dear Reviewer (Manote Lotrakul),

Thank you very much for the review. We would like to respond to the comments as follows:
1. Yes. Please change "DSM-V" to "DSM-5"
2. The internal consistencies as ... Continue reading Dear Reviewer (Manote Lotrakul),

Thank you very much for the review. We would like to respond to the comments as follows:
1. Yes. Please change "DSM-V" to "DSM-5"
2. The internal consistencies as measured by Cronbach's alpha coefficient of the Thai-ATEC total score and the sub-scales are as follows: Total score (alpha=0.946); speech/language (alpha=0.930); sociability (alpha=0.845); sensory/cognitive awareness (alpha=0.884) and health/physical/behavior (alpha=0.842).
3.The authors agree with the reviewer on this point.
4.The authors agree with the reviewer on this point.

Yours sincerely,

Pattapong Kessomboon
Kanitha Sunakarach
Dear Reviewer (Manote Lotrakul),

Thank you very much for the review. We would like to respond to the comments as follows:
1. Yes. Please change "DSM-V" to "DSM-5"
2. The internal consistencies as measured by Cronbach's alpha coefficient of the Thai-ATEC total score and the sub-scales are as follows: Total score (alpha=0.946); speech/language (alpha=0.930); sociability (alpha=0.845); sensory/cognitive awareness (alpha=0.884) and health/physical/behavior (alpha=0.842).
3.The authors agree with the reviewer on this point.
4.The authors agree with the reviewer on this point.

Yours sincerely,

Pattapong Kessomboon
Kanitha Sunakarach
Competing Interests: We declare no competing interests. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 21 Jun 2018

Pattapong Kessomboon, Department of Community Medicine, Faculty of Medicine, Khon Kaen University, Khon Kaen, 40002, Thailand

21 Jun 2018

Author Response

Dear Reviewer (Manote Lotrakul),

Thank you very much for the review. We would like to respond to the comments as follows:
1. Yes. Please change "DSM-V" to "DSM-5"
2. The internal consistencies as ... Continue reading Dear Reviewer (Manote Lotrakul),

Thank you very much for the review. We would like to respond to the comments as follows:
1. Yes. Please change "DSM-V" to "DSM-5"
2. The internal consistencies as measured by Cronbach's alpha coefficient of the Thai-ATEC total score and the sub-scales are as follows: Total score (alpha=0.946); speech/language (alpha=0.930); sociability (alpha=0.845); sensory/cognitive awareness (alpha=0.884) and health/physical/behavior (alpha=0.842).
3.The authors agree with the reviewer on this point.
4.The authors agree with the reviewer on this point.

Yours sincerely,

Pattapong Kessomboon
Kanitha Sunakarach
Dear Reviewer (Manote Lotrakul),

Thank you very much for the review. We would like to respond to the comments as follows:
1. Yes. Please change "DSM-V" to "DSM-5"
2. The internal consistencies as measured by Cronbach's alpha coefficient of the Thai-ATEC total score and the sub-scales are as follows: Total score (alpha=0.946); speech/language (alpha=0.930); sociability (alpha=0.845); sensory/cognitive awareness (alpha=0.884) and health/physical/behavior (alpha=0.842).
3.The authors agree with the reviewer on this point.
4.The authors agree with the reviewer on this point.

Yours sincerely,

Pattapong Kessomboon
Kanitha Sunakarach
Competing Interests: We declare no competing interests. Close
Report a concern

Views

Reviewer Report 29 May 2018

Mark R Geier, Institute of Chronic Illnesses, Inc., Silver Spring, MD, USA

Approved

https://doi.org/10.5256/f1000research.15824.r33726

The researchers have undertaken an important and timely study to evaluate the validity and reliability of the Autism Treatment Evaluation Checklist (ATEC) for diagnosis and evaluate children diagnosed with autism in Thailand. They have demonstrated that they were able to translate the ATEC from English into the Thai language, and then utilize the ATEC as a means to identify and evaluate the severity of children diagnosed with autism in Thailand. They have utilized appropriate methods for data collection and data analyses. The study results are consistent with previously cited published studies examining the validity of the ATEC on different populations using different metrics. The researchers appropriately consider study limitations, and the study conclusions are supported by the data.

Is the work clearly and accurately presented and does it cite the current literature?

Yes
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

Yes
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Yes

Competing Interests: No competing interests were disclosed.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

CITE

Report a concern

Respond or Comment

Comments on this article Comments (0)

Version 1

VERSION 1 PUBLISHED 03 May 2018

Open Peer Review

Reviewer Status

Reviewer Reports

	Invited Reviewers
	1	2
Version 1 03 May 18	read	read

Mark R Geier, Institute of Chronic Illnesses, Inc., Silver Spring, USA
Manote Lotrakul, Ramathibodi Hospital, Mahidol University, Bangkok, Thailand

Comments on this article

All Comments(0)

Add a comment

Browse by related subjects

Back to all reports

Reviewer Report

23 Views

11 Jun 2018 | for Version 1

Manote Lotrakul, Department of Psychiatry, Ramathibodi Hospital, Mahidol University, Bangkok, Thailand

23 Views Cite this report Responses(1)

Approved With Reservations

Please change “DSM-V” to “DSM-5“
It would be better if the author also present the other type of reliability, i.e., the internal consistencies of the ATEC scores and subscales.
Page 6 paragraph 4: It is not relevant to compare the Thai-ATEC with the CARS study in India as the purpose of that study was to develop the screening tool whereas the ATEC is designed to measure ASD severity.
Page 6 paragraph 6: This research is time consuming or slow-paced but it is not the limitation of the study.

Is the work clearly and accurately presented and does it cite the current literature?

Yes
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

Yes
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Yes

Competing Interests

No competing interests were disclosed.

Respond to this report

Responses (1)

Author Response

21 Jun 2018

Pattapong Kessomboon, Department of Community Medicine, Faculty of Medicine, Khon Kaen University, Khon Kaen, 40002, Thailand

Dear Reviewer (Manote Lotrakul),

Thank you very much for the review. We would like to respond to the comments as follows:
1. Yes. Please change "DSM-V" to "DSM-5"
2. The internal consistencies as measured by Cronbach's alpha coefficient of the Thai-ATEC total score and the sub-scales are as follows: Total score (alpha=0.946); speech/language (alpha=0.930); sociability (alpha=0.845); sensory/cognitive awareness (alpha=0.884) and health/physical/behavior (alpha=0.842).
3.The authors agree with the reviewer on this point.
4.The authors agree with the reviewer on this point.

Yours sincerely,

Pattapong Kessomboon
Kanitha Sunakarach

View more View less

Competing Interests

We declare no competing interests.

Back to all reports

Reviewer Report

24 Views

29 May 2018 | for Version 1

Mark R Geier, Institute of Chronic Illnesses, Inc., Silver Spring, MD, USA

24 Views Cite this report Responses(0)

Approved

Is the work clearly and accurately presented and does it cite the current literature?

Yes
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

Yes
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Yes

Competing Interests

No competing interests were disclosed.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

Alongside their report, reviewers assign a status to the article:

Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions

Click here to access the data.

Downloaded data do not display as expected? Download the data

[1] 1. American Psychiatric Association: Diagnostic and statistical manual of mental disorder: DSM-IV-TR. 4^th ed. Washington, DC: APA; 2000. Reference Source

[2] 2. American Psychiatric Association: Diagnostic Criteria From DSM-5. Washington, DC: APA; 2013. Reference Source

[3] 3. World Health Organization: Meeting report: autism spectrum disorders and other developmental disorders: from raising awareness to building capacity. 2013 Sep 16–18; Geneva, Switzerland. [cited 2017 Jan 22]. Reference Source

[4] 4. Kopetz PB, Endowed ED: Autism worldwide: prevalence, perceptions, acceptance, action. J Soc Sci. 2012; 8(2): 196–201. [cited 2017 Dec 15]. Publisher Full Text

[5] 5. National Institute for Health and Clinical Excellence: Autism: Recognition, Referral and Diagnosis of Children and Young People on the Autism Spectrum. London: NICE; 2011. PubMed Abstract

[6] 6. Lord C, Rutter M, Le Couteur A: Autism Diagnostic Interview-Revised: a revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. J Autism Dev Disord. 1994; 24(5): 659–85. PubMed Abstract | Publisher Full Text

[7] 7. Lord C, Risi S, Lambrecht L, et al.: The Autism Diagnostic Observation Schedule-Generic: a standard measure of social and communication deficits associated with the spectrum of autism. J Autism Dev Disord. 2000; 30(3): 205–23. PubMed Abstract | Publisher Full Text

[8] 8. Schopler E, Reichler RJ, DeVellis RF, et al.: Toward objective classification of childhood autism: Childhood Autism Rating Scale (CARS). J Autism Dev Disord. 1980; 10(1): 91–103. PubMed Abstract | Publisher Full Text

[9] 9. Green J, Charman T, McConachie H, et al.: Parent-mediated communication-focused treatment in children with autism (PACT): a randomised controlled trial. Lancet. 2010; 375(9732): 2152–60. PubMed Abstract | Publisher Full Text | Free Full Text

[10] 10. Magiati I, Moss J, Yates R, et al.: Is the Autism Treatment Evaluation Checklist a useful tool for monitoring progress in children with autism spectrum disorders? J Intellect Disabil Res. 2011; 55(3): 302–12. PubMed Abstract | Publisher Full Text

[11] 11. Rimland B, Edelson SM: Autism Treatment Evaluation Checklist (ATEC) [online]. 2000; [cited 2016 Dec 20]. Reference Source

[12] 12. Weiner RH, Greene RL: Intention-based therapy for autism spectrum disorder: promising results of a wait-list control study in children. Explore (NY). 2014; 10(1): 13–23. PubMed Abstract | Publisher Full Text

[13] 13. Jarusiewicz B: Efficacy of neurofeedback for children in the autistic spectrum: a pilot study. J Neurother. 2002; 6(4): 39–49. Publisher Full Text

[14] 14. Autism Research Institute: Studies confirm validity of ATEC - 2018. 2018; [cited 2018 Jan 10]. Reference Source

[15] 15. Geier DA, Kern JK, Geier MR: A Comparison of the Autism Treatment Evaluation Checklist (ATEC) and the Childhood Autism Rating Scale (CARS) for the Quantitative Evaluation of Autism. J Ment Health Res Intellect Disabil. 2013; 6(4): 255–67. PubMed Abstract | Publisher Full Text | Free Full Text

[16] 16. Russell PS, Daniel A, Russell S, et al.: Diagnostic accuracy, reliability and validity of Childhood Autism Rating Scale in India. World J Pediatr. 2010; 6(2): 141–7. PubMed Abstract | Publisher Full Text

[17] 17. Sunakarach K, Kessomboon P: Dataset 1 in: Validity and reliability of the Thai version of the Autism Treatment Evaluation Checklist: A two-phase diagnostic accuracy study. F1000Research. 2018. Data Source

[18] 18. Beaton DE, Bombardier C, Guillemin F, et al.: Guidelines for the process of cross-cultural adaptation of self-report measures. Spine (Phila Pa 1976). 2000; 25(24): 3186–91. PubMed Abstract | Publisher Full Text

[19] 19. Dale E, Jahoda A, Knott F: Mothers' attributions following their child's diagnosis of autistic spectrum disorder: exploring links with maternal levels of stress, depression and expectations about their child's future. Autism. 2006; 10(5): 463–79. PubMed Abstract | Publisher Full Text

[20] 20. Nunnally JC: Psychometric theory. 2^nd ed. New York: McGraw-Hill; 1978.

Validity and reliability of the Thai version of the Autism Treatment Evaluation Checklist: A two-phase diagnostic accuracy study

Abstract

Keywords

Introduction

Methods

Phase 1: Forward–backward translation of the ATEC

Phase 2: Testing of the validity and reliability psychometric properties of the final draft tool

Participants

Statistical analysis

Ethical approval and consent

Results

Validity of the Thai-ATEC

Table 1. Validity of the Thai-Autism Treatment Evaluation Checklist (ATEC) at the cut-off score of ≤38.

Table 2. Validity of the Thai-Autism Treatment Evaluation Checklist (ATEC) at the cut-off score of ≥68.

Reliability of the Thai-ATEC

Table 3. Inter-rater reliability of the Thai-Autism Treatment Evaluation Checklist (ATEC).

Discussion

Table 4. Symptom severity levels of the original Autism Treatment Evaluation Checklist (ATEC) and the Thai-ATEC.

Conclusion

Data availability

Competing interests

Grant information

Acknowledgements

Supplementary material

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Browse by related subjects

The problem

How to fix it

Competing Interests Policy

Stay Updated