https://orcid.org/0000-0003-3896-6049
Figures
Abstract
This article introduces Interdisciplinary Research Maps as a novel visualization technique to assist with interdisciplinary research analytics and to map common (and distinct) topics across publications from different disciplines. We detail the method for this technique which is based on entity linking and illustrate its application to a sample of articles sourced from the top business/management and environmental sciences journals. Both fields have separately been criticized for a lack of interdisciplinary research to co-create insights for tackling pressing environmental issues such as climate change. Our mapping approach provides a starting point for exploring similarities and differences in research topics across these fields. The mapping technique introduced here has broader applicability to facilitate the creation and exchange of knowledge across fields. We discuss avenues for visualization techniques to bridge the different fields by focusing on identifying common concepts to provide a basis for future analysis.
Citation: Marrone M, Linnenluecke MK (2020) Interdisciplinary Research Maps: A new technique for visualizing research topics. PLoS ONE 15(11): e0242283. https://doi.org/10.1371/journal.pone.0242283
Editor: Edmond Sanganyado, Shantou University, Guangdong, CHINA
Received: July 5, 2020; Accepted: October 29, 2020; Published: November 24, 2020
Copyright: © 2020 Marrone, Linnenluecke. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The data underlying the results presented in the study are available from: Marrone, Mauricio; Linnenluecke, Martina (2020), “Interdisciplinary Research Maps - Scattertext input”, Mendeley Data, V1, doi: 10.17632/g8xz5vdds8.1.
Funding: M.L. acknowledges funding from the Australian Research Council, grant DP160103425. URL arc.gov.au. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
1 Introduction
The premise that interdisciplinary research can tackle pressing environmental issues such as climate change holds forth great promise–the implicit assumption is that interdisciplinary efforts can overcome limited and narrow disciplinary understandings of solutions and contribute to better problem-solving efforts. However, the sheer magnitude of information and knowledge embedded in individual disciplines, the lack of a common shared understanding of disciplinary knowledge, differences in research interest, methods, data, analytical approaches, as well as the complexity of coordinating research amongst researchers from different disciplines are still limiting factors [1, 2]. Tools for evidence synthesis are beginning to emerge, but are not yet commonly used [3], especially for interdisciplinary mapping reviews [4, 5].
To advance interdisciplinary research endeavors, our objective is to provide a new visualization technique to identify and display common (and distinct) topics in interdisciplinary research efforts. Specifically, we introduce Interdisciplinary Research Maps as a novel visualization technique to assist with interdisciplinary research analytics across publications from different disciplines. We discuss how Interdisciplinary Research Maps can be applied to bridge disciplinary areas by mapping and visualizing where disciplines have separate concepts, ideas and topics, where boundaries separating disciplines might be dissolving, and where there might be possibilities for new hybrid fields to emerge [6]. Prior research has shown that a focus on shared concepts, ideas and topics across disciplines can greatly help researchers to improve collaboration in interdisciplinary research [1].
The Interdisciplinary Research Maps technique builds on entity linking. Entity linking (further detailed in section 3.2) is based on the use of algorithmic technology that allows for the identification of meaningful concepts and text sequences (here referred to as ‘topics’) in unstructured text, and thus for the identification of common topics from the articles included for analysis. Here, we illustrate the application of Interdisciplinary Research Maps to a sample of articles sourced from the top business/management and environmental sciences journals to provide a starting point for exploring similarities and differences in research topics across fields.
The article seeks to contribute to the mapping of scientific discourses by bringing together knowledge from across disciplines to inform research debates. We offer a method that allows for an unbiased synthesis of available evidence to contribute to dialogue across different research communities. The is structured as follows. First, we provide a brief background on mapping and visualizing academic discourses [7]. Next, we detail our new method and show its application to our sample data to create an interdisciplinary research map. The technique developed here has broader applicability to facilitate the creation and exchange of knowledge across fields. We discuss avenues for Interdisciplinary Research Maps to bridge different fields by focusing on defining common concepts and to provide a basis for future analysis.
2 Mapping academic discourses
Researchers have developed a variety of different approaches for mapping and visualizing academic discourses. Many of these approaches focus on the analysis of academic publications within a particular field (often based on topic modelling or topic synthesis) [8], however, there are still few studies on the analysis of interdisciplinary academic discourses. Van Leeuwen and Tijssen [9], for instance, study the extent to which disciplines are interrelated through citation links, analyzing journal-to-journal citation data from the Journal Citation Report database. However, the authors note that their analysis provides limited insights in terms of mapping interdisciplinary connections between scientific areas. Other authors have focused on analyzing the content of grant proposals in order to document interdisciplinary research endeavors [10]. A substantial effort has also gone into identifying cross-citation networks across fields [11], however, the outputs generated are typically in the form of citation networks, with limited insights into topic overlaps.
The approach introduced here offers an improved approach to topic modeling. Topic modelling typically focuses on identifying a group of words (i.e., topic) from a collection of textual data [12–14], but has a number of limitations. In topic modelling, the user establishes the number of topics that will be extracted, which is based on an arbitrary selection [15]. Second, the researcher needs to label the correlated groups of terms that are generated by the statistical algorithms underlying the method [16], which is also based on user preferences. The labeling of topics is usually undertaken based on the most frequently occurring words in a group of terms, but word frequency and probability distributions do not provide a straightforward basis for naming a topic [17]. To carry out topic modelling, a researcher therefore needs to have a good understanding of the text corpora being analyzed and/or requires access to experts who can assist with this process [17, 18]. Given that the researcher is largely responsible for deciding on the extraction and labelling of topics, topic modelling approaches can therefore be biased and limited in their usefulness [18].
In the next section we detail our approach which is an automated approach to a mapping review and can therefore overcome user limitations associated with manual decisions in topic extraction and labelling [12, 13]. Furthermore, the approach is applicable to textual data from different disciplines, and therefore suitable for the development of Interdisciplinary Research Maps.
3 Method
The method introduced here involves several different steps (see also Fig 1), detailed in this section. First, we selected a sample of articles to demonstrate the visualization technique in this study. We sourced data through the Scopus database, selecting articles published in the top four journals by impact factor in the areas of business/management and environmental sciences (published between 2011-01-01 and 2020-09-24) and downloaded information (abstracts, titles, publication years). The following journals were included for the area of business/management: The Academy of Management Review, the Journal of Management, the Academy of Management Journal, and the Journal of International Business Studies. In addition, the following journals were included for the area of environmental science: Nature Climate Change, Global Environmental Change, Annual Review of Environment and Resources, and Wiley Interdisciplinary Reviews Climate Change. We decided not to include the journal Tourism Management due to its narrower focus on the planning and management of travel and tourism. We selected 2011-01-01 as the starting date as this provided a common timeframe across all journals. We recognize that a focus on high impact factors has limitations; however, we see the approach here as a first attempt to visualize different knowledge domains which can be expanded and refined in future research.
In a next step, we downloaded associated journal citation records from the Scopus database which fully indexes all eight journals, and extracted the abstracts, titles and years for further analysis. We decided to only use the titles and abstracts for our analysis for a number of reasons: (1) The decision to include only the academic abstracts rather than the entire articles was based on the view that such abstracts are likely to contain the most important aspects of the articles they describe [19]. Abstracts of academic articles are typically very concise and contain an overview of the core topics contained in the associated articles. More specifically, when comparing the usefulness of abstracts and full-length articles for textual analysis, Crawford et al. [19] were unable to find a clear advantage of using the full-length versions. (2) We decided not to use author keywords as prior research has raised several issues regarding the use of author keywords in entity linking. Entity linking works best with surrounding text as context [20], but authors sometimes only use acronyms as keywords, and provide a very limited number of 3–5 words that are at times not reflective of the article. However, it is certainly possible to just use the full text of articles or author keywords with the technique presented here, and other researchers using our proposed technique would be able to make this decision based on their own preferences.
We manually inspected the downloaded citation data to ensure that no duplicates were included and that no articles were accidentally missed. These data were exported to two separate Comma Separated Values (CSV) files, one for each of the areas of interest. The titles of the publications from each body of literature were then merged with their respective abstracts. Our final datasets for further analysis comprised the titles and abstracts of 4,827 articles in the environmental sciences, and the titles and abstracts of 2,671 articles in the business/management literature (total of 7,498 records). Table 1 summarizes the sources and years of publication for the articles.
3.1 Entity linking
We proceeded with analyzing key research topics emerging from the titles and abstracts of the academic articles using an entity linker. An entity linking system uses algorithmic technology to identify meaningful text sequences (here referred to as ‘topics’) in the unstructured abstract text and title of each article and assigns unambiguous identifiers to them. Entity linkers can recognize and merge strings that describe the same concept, such as the U.S., USA and the United States of America [21]. Entity linking tools follow three steps. First, the tool identifies a list of possible topics, sometimes also referred to as mentions. Second, these topics are then disambiguated by linking them to a large catalogue of text. The disambiguation is an essential step as words often have multiple meanings–for example, the term “Mercury” can refer to the chemical element, the planet, or the Roman god [22], but is also used as company name, in the context of computing, and even for fictional characters and places in films and literature. The disambiguation step analyses the context in which the term is used to then label identified text segments using designated tags based on information gathered from the catalogue of text, for instance, to decide on “Mercury (planet)”. Lastly, the algorithm assigns scores to each possible topic to reflect the relevance of the respective topic for the overall context in which the topic is used. This score can be thought of as a confidence measure. In the last stage, the algorithm removes topics that receive low confidence scores. A further comprehensive discussion of entity linking is offered by Cornolti et al. who also discuss the application of entity linking to different types of datasets [23].
Here, we used the tool TAGME for the entity linking analysis. TAGME is an entity linker that is widely available and has been viewed by other researchers as highly effective in extracting key topics from academic texts [23, 24]. Whilst TAGME was initially designed for short, unstructured texts, it has subsequently been applied to longer texts, and was deemed to be a suitable tool for annotating text segments on-the-fly and with high precision [25]. Wikipedia was selected as the catalog of choice as it offers a trade-off between a catalog with a rigorous structure but low coverage (for example, the high-quality entity catalog WordNet and CYC), and a large collection of texts with broad coverage but unstructured and noisy content (for example, the worldwide web) [26].
To carry out the various entity linking steps detailed above, we ran the analysis in Python 3 to analyze the CSV file created in the previous step. The Python code (available on Github [27]) calls on the TAGME tool [26], which executes a process that scans the text and compiles a list of all possible topics (not ranked by any importance in the first instance). To illustrate how TAGME categorizes text and assigns identifiers, we show an example from one article included in our analysis (randomly selected) in Table 2 below. Table 2 shows the input and output of the TAGME annotation process [26]. The text on the left side served as an input into the process. The topics on the left side are the outcome of entity linking system.
To ensure that the annotation process was rigorous and precise, we applied parameter choices for TAGME that have yielded good results in previous analyses [29]. Relevant parameters in the code allow to adjust the specificities of the annotation task (for example, the length of the text window to be annotated) [29]. The parameters we adopted are long_text 10, epsilon 0.427, rho = 0.1613. The value of long_text specifies the shifting window of the text to be annotated (which can be thought of as a scanner that runs over the text). This value can range from 1 to 10 (i.e., defining a shorter versus a longer shifting window). The value of epsilon can range from 0 to 0.5 and defines whether the annotation process will favor the immediate context of the text that is analyzed (if a lower value is set) or will assign more common/general topics (if a higher value is set). The value of rho can range from 0 to 1 and is used to indicate annotations above and below a given confidence score threshold, representing the likelihood that the annotations are appropriate given their context in the input text [29]. Given that these parameters were tested in prior research [14], we are confident that the choice of parameter settings helped to reduce annotation errors while also ensuring that the output (i.e., the number of topics identified in the respective texts) was useful for the analysis.
The application of TAGME initially 7,952 identified unique topics in the environmental sciences articles, and 4,333 topics in the business/management articles. Articles can be tagged with more than one topic, for example, climate change and greenhouse gas emissions. We reviewed the tagging to remove false positives. Topics that make little meaningful sense given the context in which they were used were deleted. Examples of these include phrases such as for but not with, and incorrect tags such as Trapped (Islandic TV series). After cleaning results, we retained 7,915 unique topics in the environmental sciences articles, and 4,293 unique topics in business/management articles.
3.2 Visualization of key topics
This stage of the method involves the visualization of the topics extracted by TAGME. Rather than mapping all topics, the technique here first examines them for “keyness.” The python library scattertext, used for purposes of visualizing the results in Fig 2 (below), employs a Scaled F-Score to determine the saliency of topics identified [30]. For our study, scattertext visualizes the topics within and across the business/management and environmental sciences literature to determine topics that are salient in each of the two fields, and topics that are common, or joint topics across the two fields. The term salient thereby refers to topics that are mentioned more frequently within each field (as determined by the Scaled F-Score, see above) and can thus be considered as relevant for that literature.
4 Results
The results of the analysis are shown in Fig 2 below. The axes correspond to the rank-frequency of a specific term [30]. In addition, Fig 2 shows association based on the color coding of topics. The topics (represented as dots) are colored either in red or blue based on their association with either the business/management or the environmental sciences literature. The higher up a dot is on the vertical axis, the more frequent it appears in environmental sciences literature. The further right a point is on the horizontal axis, the more it is frequent in business/management. In other words, topics that appear in the lower right corner are used frequently in the business/management literature (but not the environmental sciences literature), while topics that appear in the upper left corner are used frequently in the business/management literature (but not the environmental sciences literature). Topics that are more common to both literatures can be found closer to the diagonal. The top upper right points thus represent topics that are most frequently associated with both literatures.
An interactive version of Fig 2 is included in the appendix and allows for a closer interrogation of all topics beyond those displayed in the static version of Fig 2. Table 3 offers a summary of the topics most frequently associated with the environmental sciences literature only (column 1), with the business/management literature only (column 2), and with both literatures (column 3). While Fig 2 and Table 3 do not display the full citation information for specific journal articles connected to each dot, this information can be extracted from the program for purposes of further analysis.
4.1 Topics in environmental journals
As evident from Fig 2, core topics represented almost exclusively in the environmental sciences articles are related to concern about climate change. The topics “climate”, “climate change”, “global warming”, “greenhouse gas”, “carbon dioxide”, “temperature”, and “climate change mitigation” are among the most frequent topics associated with this body of literature. Articles that are tagged as belonging to these topics focus, for example, on developing cost-optimal and equitable mitigation scenarios across different countries, especially with a view of limiting global mean temperature increase below 1.5°C [31–36], specific options such as reforestation and forest-based climate mitigation [37–39], changes in consumer choices and demand [40–42], emissions pricing of food commodities [43], as well as carbon sequestration and CO2 capture and storage solutions [44, 45].
Attention to topics related to climate change is closely followed by attention to closely related topics, such as “human impact on the environment”, “biodiversity”, “deforestation” and “land use”. Research on these topics focuses on the adverse consequences of human impact on ecosystems, covering topics such as the negative impacts of deforestation, biodiversity and habitat loss, degradation of ecosystems and the degradation of water resources [46–50]. Several publications in this area are connected to the Planetary Boundaries framework [51–53], which tracks the risk that human activity will generate large-scale abrupt or irreversible environmental changes by interfering with nine earth system processes (biosphere integrity, land system change, freshwater use, biogeochemical flows, ocean acidification, atmospheric aerosol leadings, ozone depletion, the release of novel entities such as plastic pollution and climate change) [54].
Many articles in environmental research are tagged as belonging to more than one topic, which reflects the interconnectedness of socio-economic drivers and environmental outcomes. For example, articles within the environmental sciences field focus on Shared Socioeconomic Pathways (SSPs) and offer an the integrated analysis of future climate impacts, vulnerabilities, adaptation, and mitigation [55–62], while others examine the implications of policy decisions on future climate change outcomes [63–67], which means that they are tagged as belonging to more than one category.
4.2 Topics in business/management journals
Core topics represented almost exclusively in the business/management journals included in this analysis are related to firm structure and expansion, such as “multinational corporation”, “foreign direct investment”, and the role of the “Chief Executive Officer”. Articles examine, for example, how the selection and characteristics of the CEO and board impact outcome variables–such as firm performance, investment in R&D (research and development), or strategic change [68–73]. Articles interested in the impacts of internationalization focus on overseas expansion and the growth of overseas market potential, especially in emerging economies [74–76]. It should be noted here that one of the journals selected for inclusion is the Journal of International Business Studies, which traditionally features articles with an international focus, thus including a high percentage of articles with international topics.
Other common topics focus on the importance of corporate members, as evidenced by the topics “human resource management” and “human capital”. Several articles focus on specific topics such as the importance of “creativity”, “cognition”, and “performance”, Within the literature, there is also a significant focus on theoretical work related to the analysis of the behavior of individual members within organizations, such as “emotion” and “abusive supervision”. The literature is also showing an interest in corporate social responsibility, but without direct connections to the environmental sciences.
4.3 Joint topics
As evident from Fig 2, several topics are common to both the environmental and business/management journals. Of particular interest is the topic “decision-making”, reflecting discussions about the creation of new insights on managing environmental impacts, and the applicability of these findings to policy, organizational and other decision-makers [77, 78]. Of interest are also the topics “knowledge” and “innovation”, reflecting the endeavor in both disciplinary areas to pursue new knowledge generation and innovative ideas [79].
Research in both business/management and environmental domains has focused on the topic “China” and the role of China as one of the major developing economies–the difference is that articles in the business/management field address management challenges and economic opportunities in China [80], while the environmental articles address the role of emerging economies in climate adaptation and mitigation efforts [81]. A similar observation arises when examining the “globalization” topic–articles in the environmental sciences domain analyze the environmental implications of issues such as global population growth and socio-technological transformations [82, 83] while articles in the business/management domain analyze internationalization and globalization theories that are largely focused on managing and exploring international growth opportunities in overseas markets [84].
Of joint interest are also behavioral theories, as well as more general topics that relate to future developments, including: “risk”, “uncertainty”, and “technology”. A common topic in both fields is the concern about economic development (topic “economy”). However, a key difference here is that articles published in the environmental sciences domain analyze the economic impacts of climate change [85–88], ecosystem and resource degradation [89–91], decarbonization (e.g., through the adoption of carbon sinks as well as low carbon/zero carbon technologies) and cleantech uptake [92–94], while articles in the business/management areas focus on the economic growth through organizational (firm and industrial) activities [95], without necessarily paying attention to the environmental implications of organizational growth.
Another common theme is “institution”–related to the role of institutions. Articles in the environmental sciences domain focus on the role of institutional actors (e.g., policy actors, management agencies) in fostering or inhibiting change in social, economic and environmental systems (also related to the themes “system” and “social change”) [56]. Environmental sciences articles also focus on the role of institutional actors in the management of land resources of other commons, often with references to the work of Ostrom and other commons researchers, and outline mechanisms leading to adaptive governance and community resilience, such as cooperation and social learning [96]. Articles on institutions in the business/management domain predominantly use institutional theory as the foundation, studying a range of cultural-cognitive, normative, and regulative actors and elements that shape the development of corporate and industrial activities over time or in different cultural or international contexts [97].
5 Discussion
We have presented a method to map the convergence and divergence of research topics across different journals and fields to facilitate the creation and exchange of knowledge across fields. Our analysis is based on a sample of high-impact factor journals; however, the method presented here can be easily expanded to different journals, fields, and collections of articles. Below, we discuss how mapping the convergence and divergence of topics can allow for the exploration of new topic areas in interdisciplinary research.
5.1 Identification of areas for interdisciplinary research collaboration
Interdisciplinary Research Maps can be used to identify areas for interdisciplinary research collaboration in a number of different ways: (1) by investigating topics that are frequently associated with both literatures, and (2) by investigating topics that are frequently identified in one literature, but not the other. We use our sample of articles sourced from the top business/management and environmental sciences journals as well as the resulting map in Fig 2 as the context to discuss these options.
As evident from Fig 2, there are several areas where the fields we examine already converge, with joint interest in topics such as decision-making, the creation of knowledge, China-related research as well as globalization. These topics are frequently associated with both literatures and thus provide possible fruitful avenues for future interdisciplinary research. For example, research could explore how economic opportunities in China can be combined with the pursuit of climate adaptation and mitigation efforts. Future research can examine similarities and differences in conceptualizations of these joint topics. For example, or analysis shows a joint interest in economic development; however, articles in the business/management areas predominantly focus on economic growth, without necessarily paying attention to the environmental implications of organizational growth. Such conceptual differences do not have to be limiting factors. Understanding similarities and differences in key topics (e.g., economy, risk, uncertainty, decision-making) and realizing their importance to both fields will allow for the identification of domains and topics that represent important fundamental concepts in the development of knowledge across fields.
In addition, opportunities also exist in connecting topics across literatures that are not yet connected (i.e., a topic associated predominantly with one field). For example, our mapping approach shows that a frequent topic in the environmental sciences is concern about climate change; however, this topic is virtually absent from the business and management field. A key question is why this is the case as it stands to reason that climate change has not just impacts on the natural environment, but also on economic entities such as firms and industries. Some articles have started to notice this disconnect and are urging for further interdisciplinary research [98]. Vice versa, key management decisions (e.g., maintaining profitability) are not a core topic represented in the environmental sciences. Nonetheless, there are opportunities here to bridge such divergent topics. Some recent articles (e.g. in Nature Climate Change and Global Environmental Change) have started to engage with questions about how climate change impacts business and management decisions, including asset valuations and investment decisions [99, 100]. Future opportunities exist to broaden research at the intersection of the two fields.
5.2 Limitations
As with every approach, there are also limitations with the approach presented here. First, entity linking is still a new approach and will require further robustness testing across different contexts and research settings. While it has found applications in understanding topic evolution within a particular field of research, there are undoubtedly opportunity to fine-tune the approach for interdisciplinary settings, in particular to ensure that the entity linking approach correctly tags topics in very specialized research areas. One opportunity for further analysis is to examine improvements in natural language processing, and to use those to undertake further refinements to improve the quality of tagging and entity linking to overcome possible limitations. Future research can also investigate the use of a different reference catalogue.
Second, we have only used a sample of articles and journals to illustrate our analysis here–we recognize that a focus on high impact factors has limitations; however, the analysis here was aimed at a first attempt to illustrate the application of Interdisciplinary Research Maps to visualize different knowledge domains, rather than an attempt to map and review all available literature on the topic. Future research applying Interdisciplinary Research Maps can overcome such limitations by considering other sampling techniques.
6 Conclusion
Without access to robust time‐saving tools and visualizations, researchers will be challenged to synthesize research outputs effectively [3]. Our approach offers a first attempt at visualizing conceptual commonalities and differences across two different fields of research. We suggest that interdisciplinary research endeavors can focus on (1) investigating topics that are common across fields and to develop joint research endeavors at this interaction, or (2) on investigating topics that are frequently associated with only one field to see if these topics could be meaningfully integrated into other research fields. Future work can build upon the tool presented here to develop meaningful work that sits at the boundaries of science, business, ethics and politics, and bridges different disciplinary areas.
References
- 1. Jeffrey P. Smoothing the waters: Observations on the process of cross-disciplinary research collaboration. Social Studies of Science. 2003; 33(4):539–62.
- 2. Urbanska K, Huet S, Guimond S. Does increased interdisciplinary contact among hard and social scientists help or hinder interdisciplinary research? PLoS One. 2019; 14(9):e0221907. pmid:31483810
- 3. Westgate MJ. revtools: An R package to support article screening for evidence synthesis. Research Synthesis Methods. 2019.
- 4. Shemilt I, Simon A, Hollands GJ, Marteau TM, Ogilvie D, O'Mara‐Eves A, et al. Pinpointing needles in giant haystacks: use of text mining to reduce impractical screening workload in extremely large scoping reviews. Research Synthesis Methods. 2014; 5(1):31–49. pmid:26054024
- 5. Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal. 2009; 26(2):91–108. pmid:19490148
- 6.
Bordons M, Morillo F, Gómez I. Analysis of cross-disciplinary research through bibliometric tools. Handbook of Quantitative Science and Technology Research: Springer; 2004. p. 437–56. pmid:15373984
- 7. Donnelly CA, Boyd I, Campbell P, Craig C, Vallance P, Walport M, et al. Four principles to make evidence synthesis more useful for policy. Nature. 2018; 558(7710):361–5. pmid:29925978
- 8. Asmussen CB, Møller C. Smart literature review: a practical topic modelling approach to exploratory literature review. Journal of Big Data. 2019; 6(1):93.
- 9. Van Leeuwen T, Tijssen R. Interdisciplinary dynamics of modern science: analysis of cross-disciplinary citation flows. Research Evaluation. 2000; 9(3):183–7.
- 10. Mutz R, Bornmann L, Daniel H-D. Cross-disciplinary research: What configurations of fields of science are found in grant proposals today? Research Evaluation. 2014; 24(1):30–6.
- 11. Karunan K, Lathabai HH, Prabhakaran T. Discovering interdisciplinary interactions between two research fields using citation networks. Scientometrics. 2017; 113(1):335–67.
- 12. Moro A, Raganato A, Navigli R. Entity linking meets word sense disambiguation: a unified approach. Transactions of the Association for Computational Linguistics. 2014; 2:231–44.
- 13. Hachey B, Radford W, Nothman J, Honnibal M, Curran JR. Evaluating entity linking with Wikipedia. Artificial intelligence. 2013; 194:130–50.
- 14. Cai CW, Linnenluecke MK, Marrone M, Singh AK. Machine Learning and Expert Judgement: Analysing Emerging Topics in Accounting and Finance Research in the Asia–Pacific. Abacus. 2019.
- 15. Blei DM, Ng AY, Jordan MI, editors. Latent dirichlet allocation. Advances in Neural Information Processing Systems; 2002.
- 16. Schober A, Kittel C, Baumgartner RJ, Füllsack M. Identifying dominant topics appearing in the Journal of Cleaner Production. Journal of Cleaner Production. 2018; 190:160–8.
- 17. Lee S-G, Kang I, Lim C-S. Optimizing the Construction of LDA-based Science & Technology Knowledge Base. International Information Institute (Tokyo) Information. 2018; 21(1):23–32.
- 18. Westgate MJ, Barton PS, Pierson JC, Lindenmayer DB. Text analysis tools for identification of emerging topics and research gaps in conservation science. Conservation Biology. 2015; 29(6):1606–14. pmid:26271213
- 19. Crawford L, Pollack J, England D. Uncovering the trends in project management: Journal emphases over the last 10 years. International Journal of Project Management. 2006; 24(2):175–84.
- 20. Marrone M, Linnenluecke MK, Richardson G, Smith T. Trends in environmental accounting research within and outside of the accounting discipline. Accounting, Auditing & Accountability Journal. 2020.
- 21.
Khalid MA, Jijkoun V, De Rijke M, editors. The impact of named entity normalization on information retrieval for question answering. European Conference on Information Retrieval; 2008: Springer.
- 22.
Dang DB, Nguyen HS, Nguyen TB, Duong TH. A framework of faceted search for unstructured documents using wiki disambiguation. In: Núñez M, Nguyen NT, Camacho D, Trawiński B, editors. Computational Collective Intelligence: Springer; 2015. p. 502–11.
- 23.
Cornolti M, Ferragina P, Ciaramita M, editors. A framework for benchmarking entity-annotation systems. Proceedings of the 22nd International Conference on World Wide Web; 2013: ACM.
- 24. Marrone M, Hammerle M. Relevant Research Areas in IT Service Management: An Examination of Academic and Practitioner Literatures. CAIS. 2017; 41:23.
- 25.
Ferragina P, Scaiella U, editors. Tagme: on-the-fly annotation of short text fragments (by wikipedia entities). Proceedings of the 19th ACM International Conference on Information and Knowledge Management; 2010: ACM.
- 26.
Piccinno F, Ferragina P, editors. From TagME to WAT: a new entity annotator. Proceedings of the 1st International Workshop on Entity Recognition & Disambiguation; 2014: ACM.
- 27.
Marrone M. Automated Literature Tagging. Available at: https://github.com/mmarrone/Literature_Tagging. 2020.
- 28. Torvanger A, Meadowcroft J. The political economy of technology support: Making decisions about carbon capture and storage and low carbon energy technologies. Global Environmental Change. 2011; 21(2):303–12.
- 29. Cuzzola J, Jovanović J, Bagheri E, Gašević D. Evolutionary fine-tuning of automated semantic annotation systems. Expert Systems with Applications. 2015; 42(20):6864–77.
- 30.
Kessler JS. Scattertext: a browser-based tool for visualizing how corpora differ. Available at: https://arxiv.org/pdf/1703.00565.pdf. 2017.
- 31. Du Pont YR, Jeffery ML, Gütschow J, Rogelj J, Christoff P, Meinshausen M. Equitable mitigation to achieve the Paris Agreement goals. Nature Climate Change. 2017; 7(1):38.
- 32. Rao ND, van Ruijven BJ, Riahi K, Bosetti V. Improving poverty and inequality modelling in climate research. Nature Climate Change. 2017; 7(12):857.
- 33. Rogelj J, Popp A, Calvin KV, Luderer G, Emmerling J, Gernaat D, et al. Scenarios towards limiting global mean temperature increase below 1.5 C. Nature Climate Change. 2018; 8(4):325.
- 34. van Vuuren DP, Stehfest E, Gernaat DE, van den Berg M, Bijl DL, de Boer HS, et al. Alternative pathways to the 1.5 C target reduce the need for negative emission technologies. Nature Climate Change. 2018; 8(5):391.
- 35. Le Quéré C, Korsbakken JI, Wilson C, Tosun J, Andrew R, Andres RJ, et al. Drivers of declining CO2 emissions in 18 developed economies. Nature Climate Change. 2019; 9(3):213.
- 36. Krausmann F, Wiedenhofer D, Haberl H. Growing stocks of buildings, infrastructures and machinery as key challenge for compliance with climate targets. Global Environmental Change. 2020; 61:102034.
- 37. Grassi G, House J, Dentener F, Federici S, den Elzen M, Penman J. The key role of forests in meeting climate targets requires science for credible mitigation. Nature Climate Change. 2017; 7(3):220.
- 38. Nanni AS, Sloan S, Aide TM, Graesser J, Edwards D, Grau HR. The neotropical reforestation hotspots: A biophysical and socioeconomic typology of contemporary forest expansion. Global environmental change. 2019; 54:148–59.
- 39. Busch J, Engelmann J, Cook-Patton SC, Griscom BW, Kroeger T, Possingham H, et al. Potential for low-cost carbon dioxide removal through tropical reforestation. Nature Climate Change. 2019; 9(6):463.
- 40. Rai V, Henry AD. Agent-based modelling of consumer energy choices. Nature Climate Change. 2016; 6(6):556.
- 41. Creutzig F, Fernandez B, Haberl H, Khosla R, Mulugetta Y, Seto KC. Beyond technology: demand-side solutions for climate change mitigation. Annual Review of Environment and Resources. 2016; 41:173–98.
- 42. Gerlagh R, Heijmans RJ. Climate-conscious consumers and the buy, bank, burn program. Nature Climate Change. 2019; 9(6):431.
- 43. Springmann M, Mason-D’Croz D, Robinson S, Wiebe K, Godfray HCJ, Rayner M, et al. Mitigation potential and global health impacts from emissions pricing of food commodities. Nature Climate Change. 2017; 7(1):69–74.
- 44. Mac Dowell N, Fennell PS, Shah N, Maitland GC. The role of CO2 capture and utilization in mitigating climate change. Nature Climate Change. 2017; 7(4):243.
- 45. Deng L, Liu S, Kim DG, Peng C, Sweeney S, Shangguan Z. Past and future carbon sequestration benefits of China’s grain for green program. Global Environmental Change. 2017; 47:13–20.
- 46. Samaniego L, Thober S, Kumar R, Wanders N, Rakovec O, Pan M, et al. Anthropogenic warming exacerbates European soil moisture droughts. Nature Climate Change. 2018; 8(5):421.
- 47. Smith MR, Myers SS. Impact of anthropogenic CO2 emissions on global human nutrition. Nature Climate Change. 2018; 8(9):834.
- 48. Bruno JF, Bates AE, Cacciapaglia C, Pike EP, Amstrup SC, van Hooidonk R, et al. Climate change threatens the world’s marine protected areas. Nature Climate Change. 2018; 8(6):499.
- 49. Pacifici M, Visconti P, Butchart SH, Watson JE, Cassola FM, Rondinini C. Species’ traits influenced their response to recent climate change. Nature Climate Change. 2017; 7(3):205.
- 50. Gauzens B, Rall BC, Mendonça V, Vinagre C, Brose U. Biodiversity of intertidal food webs in response to warming across latitudes. Nature Climate Change. 2020; 10(3):264–9.
- 51. Heck V, Gerten D, Lucht W, Popp A. Biomass-based negative emissions difficult to reconcile with planetary boundaries. Nature climate change. 2018; 8(2):151.
- 52. Heck V, Hoff H, Wirsenius S, Meyer C, Kreft H. Land use options for staying within the Planetary Boundaries–Synergies and trade-offs between global and local sustainability goals. Global environmental change. 2018; 49:73–84.
- 53. Dao H, Peduzzi P, Friot D. National environmental limits and footprints based on the Planetary Boundaries framework: The case of Switzerland. Global environmental change. 2018; 52:49–57.
- 54. Steffen W, Richardson K, Rockström J, Cornell SE, Fetzer I, Bennett EM, et al. Planetary boundaries: Guiding human development on a changing planet. Science. 2015; 347(6223):1259855. pmid:25592418
- 55. Riahi K, Van Vuuren DP, Kriegler E, Edmonds J, O’neill BC, Fujimori S, et al. The shared socioeconomic pathways and their energy, land use, and greenhouse gas emissions implications: an overview. Global Environmental Change. 2017; 42:153–68.
- 56. O’Neill BC, Kriegler E, Ebi KL, Kemp-Benedict E, Riahi K, Rothman DS, et al. The roads ahead: narratives for shared socioeconomic pathways describing world futures in the 21st century. Global Environmental Change. 2017; 42:169–80.
- 57. Popp A, Calvin K, Fujimori S, Havlik P, Humpenöder F, Stehfest E, et al. Land-use futures in the shared socio-economic pathways. Global Environmental Change. 2017; 42:331–45.
- 58. Fujimori S, Hasegawa T, Masui T, Takahashi K, Herran DS, Dai H, et al. SSP3: AIM implementation of shared socioeconomic pathways. Global Environmental Change. 2017; 42:268–83.
- 59. Fricko O, Havlik P, Rogelj J, Klimont Z, Gusti M, Johnson N, et al. The marker quantification of the Shared Socioeconomic Pathway 2: a middle-of-the-road scenario for the 21st century. Global Environmental Change. 2017; 42:251–67.
- 60. Kriegler E, Bauer N, Popp A, Humpenöder F, Leimbach M, Strefler J, et al. Fossil-fueled development (SSP5): an energy and resource intensive scenario for the 21st century. Global environmental change. 2017; 42:297–315.
- 61. Calvin K, Bond-Lamberty B, Clarke L, Edmonds J, Eom J, Hartin C, et al. The SSP4: A world of deepening inequality. Global Environmental Change. 2017; 42:284–96.
- 62. Bauer N, Calvin K, Emmerling J, Fricko O, Fujimori S, Hilaire J, et al. Shared socio-economic pathways of the energy sector–quantifying the narratives. Global Environmental Change. 2017; 42:316–30.
- 63. Bauer N, McGlade C, Hilaire J, Ekins P. Divestment prevails over the green paradox when anticipating strong future climate policies. Nature Climate Change. 2018; 8(2):130.
- 64. Estrada F, Botzen WW, Tol RS. A global economic assessment of city policies to reduce climate change impacts. Nature Climate Change. 2017; 7(6):403.
- 65. McGrath LF, Bernauer T. How strong is public support for unilateral climate policy and what drives it? Wiley Interdisciplinary Reviews: Climate Change. 2017; 8(6):e484.
- 66. Martin G, Saikawa E. Effectiveness of state climate and energy policies in reducing power-sector CO2 emissions. Nature Climate Change. 2017; 7(12):912.
- 67. Barron AR. Time to refine key climate policy models. Nature Climate Change. 2018:1.
- 68. Oh W-Y, Barker VL III. Not all ties are equal: CEO outside directorships and strategic imitation in R&D investment. Journal of Management. 2018; 44(4):1312–37.
- 69. Takacs Haynes K, Campbell JT, Hitt MA. When more is not enough: Executive greed and its influence on shareholder wealth. Journal of Management. 2017; 43(2):555–84.
- 70. Le S, Kroll M. CEO international experience: Effects on strategic change and firm performance. Journal of International Business Studies. 2017; 48(5):573–95.
- 71. Bilgili TV, Calderon CJ, Allen DG, Kedia BL. Gone with the wind: a meta-analytic review of executive turnover, its antecedents, and postacquisition performance. Journal of Management. 2017; 43(6):1966–97.
- 72. Gangloff KA, Connelly BL, Shook CL. Of scapegoats and signals: Investor reactions to CEO succession in the aftermath of wrongdoing. Journal of Management. 2016; 42(6):1614–34.
- 73. Blagoeva RR, Mom TJ, Jansen JJ, George G. Problem-solving or self-enhancement? A power perspective on how CEOs affect R&D search in the face of inconsistent feedback. Academy of Management Journal. 2020; 63(2):332–55.
- 74. Gaur AS, Ma X, Ding Z. Home country supportiveness/unfavorableness and outward foreign direct investment from China. Journal of International Business Studies. 2018; 49(3):324–45.
- 75. Ramamurti R, Hillemann J. What is “Chinese” about Chinese multinationals? Journal of International Business Studies. 2018; 49(1):34–48.
- 76. Buckley PJ, Chen L, Clegg LJ, Voss H. Risk propensity in the foreign direct investment location decision of emerging multinationals. Journal of International Business Studies. 2018; 49(2):153–71.
- 77. Hahn T, Preuss L, Pinkse J, Figge F. Cognitive frames in corporate sustainability: Managerial sensemaking with paradoxical and business case frames. Academy of management review. 2014; 39(4):463–87.
- 78. Hartmann J, Inkpen AC, Ramaswamy K. Different shades of green: Global oil and gas companies and renewable energy. Journal of International Business Studies. 2020:1–25.
- 79. Low S, Buck HJ. The practice of responsible research and innovation in “climate engineering”. Wiley Interdisciplinary Reviews: Climate Change. 2020; 11(3):e644.
- 80. Xie Z, Li J. Exporting and innovating among emerging market firms: The moderating role of institutional development. Journal of International Business Studies. 2018; 49(2):222–45.
- 81. Lo K, Broto VC. Co-benefits, contradictions, and multi-level governance of low-carbon experimentation: Leveraging solar energy for sustainable development in China. Global Environmental Change. 2019; 59:101993.
- 82. Merino G, Barange M, Blanchard JL, Harle J, Holmes R, Allison EH, et al. Can marine fisheries and aquaculture meet fish demand from a growing human population in a changing climate? Global Environmental Change. 2012; 22(4):795–806.
- 83. Rai V, Schultz K, Funkhouser E. International low carbon technology transfer: Do intellectual property regimes matter? Global Environmental Change. 2014; 24:60–74.
- 84. Petricevic O, Teece DJ. The structural reshaping of globalization: Implications for strategic sectors, profiting from innovation, and the multinational enterprise. Journal of International Business Studies. 2019; 50(9):1487–512.
- 85. Rao ND, Min J. Less global inequality can improve climate outcomes. Wiley Interdisciplinary Reviews: Climate Change. 2018; 9(2):e513.
- 86. Hufkens K, Keenan TF, Flanagan LB, Scott RL, Bernacchi CJ, Joo E, et al. Productivity of North American grasslands is increased under future climate scenarios despite rising aridity. Nature Climate Change. 2016; 6(7):710.
- 87. Carrao H, Naumann G, Barbosa P. Mapping global patterns of drought risk: An empirical framework based on sub-national estimates of hazard, exposure and vulnerability. Global Environmental Change. 2016; 39:108–24.
- 88. Lemoine D, Kapnick S. A top-down approach to projecting market impacts of climate change. Nature Climate Change. 2016; 6(1):51.
- 89. Kuipers KJ, van Oers LF, Verboon M, van der Voet E. Assessing environmental implications associated with global copper demand and supply scenarios from 2010 to 2050. Global Environmental Change. 2018; 49:106–15.
- 90. Bleischwitz R, Nechifor V, Winning M, Huang B, Geng Y. Extrapolation or saturation–Revisiting growth patterns, development stages and decoupling. Global Environmental Change. 2018; 48:86–96.
- 91. Haberl H, Erb K-H, Krausmann F. Human appropriation of net primary production: patterns, trends, and planetary boundaries. Annual Review of Environment and Resources. 2014; 39:363–91.
- 92. Fankhauser S, Jotzo F. Economic growth and development with low‐carbon energy. Wiley Interdisciplinary Reviews: Climate Change. 2018; 9(1):e495.
- 93. Peters GP, Andrew RM, Canadell JG, Fuss S, Jackson RB, Korsbakken JI, et al. Key indicators to track current progress and future ambition of the Paris Agreement. Nature Climate Change. 2017; 7(2):118.
- 94. Vandyck T, Keramidas K, Saveyn B, Kitous A, Vrontisi Z. A global stocktake of the Paris pledges: implications for energy systems and economy. Global environmental change. 2016; 41:46–63.
- 95. Mariotti S, Marzano R. Varieties of capitalism and the internationalization of state-owned enterprises. Journal of International Business Studies. 2019; 50(5):669–91.
- 96. Vasconcelos VV, Santos FC, Pacheco JM. A bottom-up institutional approach to cooperative governance of risky commons. Nature Climate Change. 2013; 3(9):797–801.
- 97. Cardinale I. Beyond constraining and enabling: Toward new microfoundations for institutional theory. Academy of Management Review. 2018; 43(1):132–55.
- 98. Nyberg D, Wright C. Climate-proofing management research. Academy of Management Perspectives. 2020; (ja).
- 99. Dietz S, Bowen A, Dixon C, Gradwell P. ‘Climate value at risk’of global financial assets. Nature Climate Change. 2016; 6(7):676.
- 100. Van Renssen S. Investors take charge of climate policy. Nature Climate Change. 2014; 4(4):241.