1. Introduction
Language assessment has been widely researched across varying educational and professional contexts [
1,
2,
3] and is considered vitally important to students’ academic growth [
4,
5,
6]. Effective assessment enables language teachers to achieve and maintain high teaching quality [
7,
8], while inadequate assessment weakens language learning quality, resulting in a loss of student motivation and confidence [
9,
10]. Assessing language skills is challenging because many factors can influence the reliability and validity of designing, administering, interpreting, utilizing, and reporting procedures. Therefore, language assessment literacy (LAL) is vital to solving these problems and benefiting students [
11,
12,
13].
Academic focus on “assessment literacy” stems from the test-based accountability system that emerged in many educational settings in the 1980s [
14,
15]. In these educational settings, teachers had to monitor and report all student learning outcomes according to established standards [
9,
16]. After comprehensively considering informal language assessments, such as classroom observation, random testing, and oral feedback, Brindley [
17] distinguished LAL to reflect language disciplines’ unique characteristics and requirements.
The LAL research that has emerged since has explored conceptual frameworks and clarifying relationships among the elements examined. Davies [
18] and Inbar-Lourie [
19] developed and explained the earliest conceptual framework, consisting of principles (why to evaluate), knowledge (what to evaluate), and skills (how to evaluate). Taylor [
20] constructed a LAL appraisal framework for different stakeholder groups that included eight elements: knowledge of theory, technical skills, principles and concepts, language pedagogy, sociocultural values, local practices, personal beliefs/attitudes, and scores and decision making. Baker and Riches [
21] further explored this LAL framework with Haitian teachers and narrowed its elements down to seven: theoretical and conceptual knowledge, task performance, language pedagogy, collaboration, awareness of local practices, awareness of personal beliefs/attitudes, and decision making. With empirical data’s continuous enrichment and validation in different assessment contexts, new conceptual frameworks are constantly being constructed [
21,
22,
23].
Different stakeholder groups have varying interests, needs, and expectations about language testing and assessment; therefore, their understanding of LAL frameworks and mastery levels of components are different [
20,
22,
24]. Taylor [
20] categorized LAL stakeholders into four groups: test writers, classroom teachers, university administrators, and professional language testers. Stakeholder groups’ LAL profiles have different components and values because they do not participate in assessment activities identically. Brindley [
17] proposed that, compared to other stakeholder groups, language teachers must master three extra components: the ability to evaluate language tests, develop language curriculum assessments, and put assessment into practice. Taylor [
20] stated that although the four stakeholder groups need to master the same components of LAL, the requirements in each component are different because of the characteristics of the stakeholder group. For example, professional language testers are expected to have a thorough knowledge of every aspect of the assessment process and to achieve value 4 (the highest level) in components of knowledge of theory and principles and concepts. At the same time, classroom teachers focus more on designing practical assessment methods and are expected to only achieve level 2 in the same components. Because of these differences, LAL is not usually researched as a holistic concept and has been researched separately within different stakeholder groups. Classroom teachers are the most investigated group in the literature.
Stakeholders’ LAL is the product of social influence and numerous factors affect it. Previous studies have found that internal factors closely related to stakeholders and external factors dominated by the environment jointly affect the development of LAL [
9,
25]. Internal factors, such as teachers’ self-confidence [
26], willingness to participate in assessment training [
27,
28], and teaching experience, [
9] are those that directly affect their LAL. As for external factors, LAL is easily influenced by educational environments, teaching and assessment administrative orders, educational policies, and socio-cultural values of language teaching [
8,
23,
29].
The existing literature significantly improves the understanding of LAL’s conceptual nature, analytical framework, and social contexts; however, mainstream research papers focus on empirical investigation and theoretical discussion. New systematic review articles on LAL are relatively limited but necessary. First, emerging trends and thematic patterns should be scientifically explored because LAL is a complex research topic that keeps changing [
30]. Second, more reviews have been conducted on assessment literacy (AL) in general education [
31,
32,
33,
34] than LAL in language assessment and testing, but LAL should be researched separately due to its complexities regarding assessment knowledge and communicative competence [
4,
35,
36]. Third, existing reviews on LAL employ a conventional scoping approach that assists with discovering foci, themes, and contributions of LAL research. Still, few studies adopt the scientometric method, which has excellent benefits when reviewing a field of research [
37]. Compared to the conventional approach, scientometrics can logically investigate emerging trends and thematic patterns and dynamically explore the intellectual landscape and citation linkage by analyzing metrics and indicators in bibliographic records. Additionally, broader literature can be reviewed in a visual format, enabling researchers to find influential patterns in complex networks more precisely and efficiently than a conventional review, which tends to include fewer than 100 articles [
35,
36].
While existing literature reflects that there is increased attention on LAL, there is no consensus on an optimal LAL theoretical framework. LAL is highly contextual, and its stakeholders are significantly involved, which means that LAL is a complex research topic with complex networks. In this way, scientometric reviews should be used to explore its emerging trends, intellectual landscape, critical changes, and thematic patterns.
2. Methods
This study employed a scientometric approach to review emerging trends, intellectual landscape, critical changes, and thematic patterns in LAL research. Within the scope of this study, data was retrieved from one of the most widely used library resources, the Web of Science (WoS) [
38]. To ensure the visibility of state-of-the-art research, the databases used for scientometric analyses in this article include the Social Science Citation Index (SSCI), the Art & Humanities Citation Index (A&HCI), and the Emerging Sources Citation Index (ESCI). SSCI is a multidisciplinary index including over 3400 journals across 58 social science disciplines from 1985 to the present. A&HCI features abstracting and indexing for more than 1700 arts and humanities journals and covers fields including social and natural science journals. Finally, ESCI extends the scope of publications in WoS to include high-quality and peer-reviewed publications in all disciplines. Therefore, this study included these three databases comprised of almost all authoritative academic journals in the social sciences and arts to retrieve comprehensive literature on LAL.
Researchers queried the ‘topic’ section on WoS, which searches titles, abstracts, keywords, and Keywords Plus, to search for data for the scientometric review. Publications on LAL from the WoS core collection on the SSCI, A&HCI, and ESCI databases were searched using both the combined phrase “assessment literacy” (AL) and the word “language” as search items. This is because “LAL” is written as either “language assessment literacy,” “assessment literacy of teachers of languages,” or “assessment literacy of EFL teachers” in the literature. Initially, the period was set to 2021, 20 years from when Brindley [
17] first differentiated LAL from AL in 2001. However, LAL was first indexed into the WoS core collection via Inbar-Lourie’s [
19] journal article in 2008; therefore, 14 years from 2008 to 2021 were selected on WoS via “articles” and “reviews” as document types. Next, the language type was set to English. Concerning literature categories, multidisciplinary studies were included due to the interdisciplinary nature of LAL and the increased willingness of journals to accept articles from various fields. For instance, Sustainability has been categorized as a journal of environmental sciences and green technology; however, many of its social science articles have been cited in applied linguistics. Thus, to comprehensively understand LAL’s overall dynamics and disciplinary structure, no categories or citation topics were excluded while searching and collecting data sources.
163 articles and review articles were collected from 77 journals across 10 WoS categories. These included education and educational research; language and linguistics; educational psychology; interdisciplinary applications of computer science; cultural studies; multidisciplinary psychology; environmental sciences; environmental studies; and green and sustainable science and technology. The final three categories were sourced from Sustainability. None of the 163 studies were duplicated or removed.
CiteSpace, a Java application created by Chaomei Chen, was employed to analyze and visualize the data. CiteSpace is widely applied in analyzing and visualizing co-citation networks, enabling researchers to identify co-authorship, co-occurrence, and co-citation network algorithms [
39]. To thoroughly review LAL’s emerging trends, intellectual landscape, critical changes, and thematic patterns, the following CiteSpace analyses were conducted: (1) dual-map overlay; (2) co-authorship analysis among researchers, organizations, and countries; (3) co-occurrence analysis among terms, keywords, and categories; and (4) co-citation analysis among references, authors, and journals. Publication and citation trends and highly cited papers and authors based on analyses from WoS and CiteSpace were also adopted to understand emerging trends in LAL and its intellectual landscape profoundly and comprehensively.
To comprehensively explore LAL’s emerging trends, intellectual landscape, critical changes, and thematic patterns, this study examined publication and citation trends, dual-map overlay, highly cited papers, co-authorship, co-occurrence analysis, and co-citation analyses. Specifically, publication and citation trends and dual-map overlay primarily help clarify emerging trends and critical changes; highly cited papers and co-authorship help understand the intellectual landscape; and co-occurrence analysis and co-citation analyses help understand thematic patterns. However, these analyses are not independent of each other nor only belong to one function; the analyses can support each other. For example, author co-citations in co-citation analyses can also support an understanding of the intellectual landscape.
4. Discussion
A bibliometric analysis was conducted, and 163 English articles and reviews published over the past 13 years were chosen from the SSCI, A&HCI, and ESCI databases to review LAL’s emerging trends, the intellectual landscape, critical changes, and thematic patterns.
4.1. Emerging Trends and Critical Changes
In 2001, seven years before LAL was first indexed in the WoS core collection, the CEFR introduced assessment into the language assessment field and challenged language teachers’ assessment literacy. Brindley [
17] proposed that language teachers’ assessment literacy should be independent of the general education field in 2001, but the idea of LAL was not explicitly mentioned in their outline. 2001 to 2008 can be considered as a pre-stage of LAL research, during which researchers noticed the importance of differentiating LAL from AL but were still in the exploration stage. Through the scientometric analysis of LAL literature between 2008, when LAL was first indexed, and 2021 (20 years after the first time it was deemed necessary to separate LAL from AL), it is clear that LAL research has shown a fluctuating upward trend since the initial stagnation, thereby demonstrating a broad development prospect. This study divides the entire time range into 2008 to 2012, 2013 to 2017, and 2018 to 2021. Inbar-Lourie’s [
19] article marked the coming of the first stage of LAL research, where LAL was clearly defined for the first time. Inbar-Lourie further proposed that LAL needs to address three key questions: the “how-to” (skills), the “what” (knowledge), and the “why” (principles) of assessment. In the same year, ALTE discussed the research and development of foreign language teachers’ LAL, and members of the association began to focus on foreign language teachers’ LAL in their own countries. During the first stage, researchers put significant effort into identifying the concept of LAL and constructing the theoretical framework. However, since LAL is a multilayered entity and defining it is a significant challenge [
70], relevant studies have progressed slowly and delivered few results. The second stage of LAL research has had the most considerable impact. In 2013, Taylor [
20] discussed LAL stakeholder groups and introduced theoretical frameworks for each group. This article was published in Language Testing’s featured special issue on LAL in 2013, a key outcome of the 33rd LTRC. The conference brought more attention to LAL, and field research moved into the second stage. From 2008 to 2017, related studies showed a fluctuating upward trend. However, at the beginning of 2018, the associated studies dramatically increased. This indicates that LAL is increasingly recognized as an independent concept, and that more attention is being paid to language discipline characteristics. During the third stage of LAL research, the theoretical framework of LAL has been developed further, and many researchers have conducted empirical studies on the status of teachers’ LAL and their training needs [
23,
29,
45]. Professional conferences in language assessment and testing have significantly contributed to the development of LAL and are closely related to emerging and critical milestones.
4.2. Intellectual Landscape of Authors
Authors are those who conduct and publish research. Past work in a particular field can be tracked and understood through their work. Focusing on important scholars in an area is a way to learn about the field. In this context, the condition of importance in the LAL research is complex.
Section 3.4.1 and
Section 3.6.2 discuss LAL scholars from the categories of publication, citation, and co-citation. Four lists referring to authors are included in this paper: authors who are most cited (
Table 1, e.g., Yueting Xu and Gavin Brown), authors who published the most articles (
Table 2, e.g., Frank Giraldo), authors who are most cited together with other scholars (
Figure 11, e.g., Glenn Fulcher), and cited authors with the most significant citation bursts (
Figure 12, e.g., Karin Vogt and Dina Tsagari). The overlap between the four lists is exceptionally low, which means “most valued authors by other scholars ≠ authors having most articles ≠ authors having most links with others ≠ authors leading research upsurge.” One possible explanation for this situation is that research work by LAL scholars’ is relatively independent (
Section 3.4.1), indicating that scholars usually have their own research emphasis and citation principles and resulting in a diverse research context. Hence, when authors and author linkage networks are used in building the LAL research network, the work of scholars who are important for various reasons should be considered to guarantee the completeness of the network. These are especially the case as visualization research is increasingly employed, and scholars may focus on one dimension, like author linkage, to carry out discussions. When exploring influential authors or other intellectual landscapes, scientometric reviews contribute significantly as they can be used to analyze various indicators.
4.3. Thematic Patterns and Research Focus on Teachers
LAL is closely related to testing and most LAL research subjects have been language teachers. “Teacher” appears as one of the most frequently used words in abstract sections and as a keyword in many related studies. As essential stakeholders in teaching activities, language teachers directly participate in the assessment process and take direct responsibility for assessment results. In the context of school- and class-based assessments, teachers’ LAL is highly valued by academia [
8,
28,
70]. There are numerous factors leading to such bias. One is that teachers have versatile talents. Teachers carry out teaching, design courses, and create standards for judging and assessing students and their work. Teachers take on multiple roles in the LAL stakeholder structure. The complexity of this group endows them with higher research and practical values and more study perspectives that can be further explored. However, LAL research is still developing, and research on other stakeholders has increased [
4,
22,
24]. The appearance of “assessment reform” in the keywords shows that attention has also been paid to student initiatives during the worldwide curriculum reform process. In other words, the evaluation of LAL has gradually shifted from only the teacher to the teacher and the student. “Peer assessment” is also used in many language learning practices, such as writing and speaking, making students’ LAL a new trend in LAL research.
4.4. Interdisciplinary Nature of LAL
Through a co-occurrence analysis among terms, keywords, and categories, results show that LAL research is not only focused on language and education but also psychology, computer science, environmental sciences, and green and sustainable science; fields not directly related to LAL. As LAL research continues to advance, researchers’ concerns are becoming more diverse and research topics continue to be enriched. Moreover, the journal co-citation network shows that the co-citation relation between documents is complicated. These findings indicate that LAL is a complex concept with multiple dimensions, and the current LAL research and development trends have interdisciplinary and multi-perspective characteristics. In addition, the dual-map overlay suggests that LAL is underdeveloped in various fields, and the number of multidisciplinary studies is limited. For example, testing is still the most closely related sub-discipline. Language Testing, Language Assessment Quarterly, and Educational Assessment Evaluation and Accountability have become the most recognized journals in this field. As such, the interdisciplinary nature of LAL research still leaves room for exploration.
5. Conclusions
Through the scientometric visual analysis of 163 articles and review articles from 77 journals across 10 WoS categories, the current study identifies the stage changes of LAL research by analyzing LAL-related articles published from 2008 to 2021. This study suggests that the theoretical framework of LAL is currently being constructed. LAL stakeholders, who directly influence the components and needs of LAL, are significant perspectives in exploring and developing the framework. “Teacher” appears as one of the most frequently researched stakeholder groups and research on other stakeholders has increased, making students’ LAL a new trend in LAL research. Hence, future studies should encourage the analysis of different stakeholders.
It is worth noting that cross-disciplinary cooperation requires paying attention to research differences between various fields as different disciplines have different research backgrounds and methods. Issues related to LAL are encouraged to be more thoroughly and comprehensively studied and reviewed. Furthermore, precise measurement instruments, criteria, and channels must be utilized for efficient assessment. Finally, scientometric visual analyses should be encouraged in language assessment and testing. This study demonstrates the benefits and suitability of using a visualization method for clearly and systematically sorting complex disciplinary concepts and their development processes.
The research method used in this study does have certain limitations. First, the data for this study was collected from SSCI, A&HCI, and ESCI in WoS. While this study incorporated some of the most influential journals, prestigious publications from other databases may have been excluded. In addition, this study only focused on English language journal articles in the database, excluding non-English articles and different types of research such as dissertations and conference papers. Moreover, to ensure the accuracy of search results, namely collecting and analyzing articles highly focused on LAL, this study performed a more precise search, which may have resulted in fewer search results. As more data and documents are available for a more comprehensive understanding of developments in the field of LAL, future studies should consider expanding the scope of the database search and target language to create a richer visualization of LAL by rigorously screening relevant data.