Discussion paperOut for the count: some methodological questions in ‘publications counting’ literature
Introduction
Measurement of the publications output of academics as a means of eliciting some assessment of individual and institutional productivity and equality is not a new phenomenon, although in the hospitality and tourism context it is thus far principally a US one. A number of studies focusing on the tourism and hospitality areas have appeared in the last dozen years or so (Weaver and McCleary, 1989; Weaver et al., 1990; Rutherford and Samenfink, 1992; Samenfink and Rutherford, 1996). Related studies have been undertaken on faculty selection and review criteria (Sheldon and Collinson, 1990; Schmidgall and Woods, 1992); trends in the authorship and content of published articles in hospitality (Chon et al., 1989; Roberts, 1998); the rating of different kinds of publications output (Schmidgall and Woods, 1993); the rating of journals in the hospitality and tourism fields (Ferreira et al (1994), Ferreira et al (1998)); and perceptions of the value of single, co-authored and multiple authored articles in hospitality (Woods, 1998). The ‘publications counting’ literature has been further extended by citation analysis of published hospitality research (Woods and Schmidgall, 1995; Howey et al., 1999).
Although, ostensibly, the purpose of the research studies described above has been to investigate the most productive and influential scholars in the fields concerned, precise research objectives are noticeable by their absence although there are passing references to the benefits of benchmarking (Rutherford and Samenfink, 1992) and to the usefulness of such research in terms of “tenure, promotion, and salary issues...recruiting purposes and for promotional reasons…for placement or relocation purposes” (Woods and Schmidgall, 1995, p. 33). Perhaps more remarkable has been the lack of apparent critique of the methodologies of these studies or their general tenor. In an early and still important paper Smith (1983), by the way of outlining approaches to the production of academic articles, charted much of the political context and terrain attendant on such activity. More recently, Seaton (1996) has commented upon the claustrophobic nature of the refereeing process in tourism journals and Wood (1995) has examined some of the conceptual and methodological problems raised by the hospitality ‘publications counting’ literature. These contributions, while valuable, are characterised by brevity and leave considerable scope for further reflection on a range of issues, issues that this paper seeks to address.
The rationale for what follows does not simply emanate from the nature of the studies concerned. In many countries higher education is subject to increasing public and political scrutiny in the name of fiscal accountability and cost efficiency. The workings of the contemporary university in particular are constantly subjected to new tests of social, business and community ‘relevance’. In America, where publication has always had a central role in individual and institutional politics in higher education, these and additional pressures are reflected in semi-public critical discourse on the intellectual practices of higher seats of learning (Sykes, 1988; Hughes, 1994; Paglia (1993), Paglia (1994)). Fears about the quality of academic research recur, as is most recently seen in the controversy over Alan Sokal's duping of a social science journal, where the latter published a ‘phoney’ article by the physicist purporting to deconstruct theories of quantum gravity (Sokal and Bricmont, 1998). These and other pressures have led to a measure of defensiveness on the part of the higher education sector. More important, perhaps, are the mechanisms that some governments have introduced to place checks on quality. In the last 10–15 years in the UK both teaching and research have come to be periodically peer-assessed, the latter every five years in a ‘Research Assessment Exercise’ (RAE) with resultant ratings in subject areas being widely published (a similar exercise is undertaken in Hong Kong). The circulation of such information and the competitiveness it creates means that measures such as publications counting and citation analysis, when they come from academic scholars, are particularly important as they might exert influence on external assessments. Thus, the main objective of this paper is to offer a conceptual critique of existing studies of ‘publications counting’ in the hospitality field.
Section snippets
Conceptual critique of ‘publications counting’
Within the hospitality field, publications counting research is, as already intimated, an ethnocentric affair, the majority of published work emanating from the USA. Of the literature cited earlier, Rutherford and Samenfink (1992) remains the seminal template for the field, a point noted by Wood (1995) who illustrates his own discussion with examples from this study. A similar approach is undertaken here, but examples are drawn from a slightly larger number of sources. The main areas for
Some ways forward?
This paper has sought to unravel by means of a critique some of the assumptions underlying ‘publications counting’ literature in hospitality. On the basis of the foregoing discussion it would be tempting to suggest clear recommendations as to future methodological procedures for ‘publications counting’ research. Such a strategy would, however, be flawed. Much of the literature alluded to in this paper evidences, as the critique offered suggests, many methodological flaws and problems, some of
Conclusion
Somewhere between the two approaches described in the previous section lies, probably, a means by which research in hospitality can be benchmarked in a fashion that commands reasonable agreement. Some hope resides in the emergent consensus concerning the ‘best’ journals in the field. For the reasons described earlier, ‘bracketing’ the Cornell Hotel and Restaurant Administration Quarterly leaves Annals of Tourism Research, the Journal of Hospitality and Tourism Research, the International
References (25)
- et al.
Rating the hospitality journals
International Journal of Hospitality Management
(1994) - et al.
Tourism and hospitality research journalscross-citations among research communities
Tourism Management
(1999) Blowing the whistle on tourism referees
Tourism Management
(1996)- et al.
Faculty review criteria in tourism and hospitality
Annals of Tourism Research
(1990) - et al.
Hospitality researchthe emperor's new clothes?
International Journal of Hospitality Management
(1996) Assessing publications output as an indicator of academic productivitythe case of hospitality management
Tourism Management
(1995)Hospitality research and scholarship
- et al.
Trends in hospitality management literaturea content analysis
Hospitality Education and Research Journal
(1989) - et al.
Hospitality program directors ratings on hospitality journals
Journal of Hospitality and Tourism Education
(1998) Culture of Complaintthe Fraying of America
(1994)
Sex, Art and American CultureEssays
Vamps and TrampsNew Essays
Cited by (22)
Spanish research into tourism with an international impact (1997-2011). A perspective from the economy and company management
2013, Cuadernos de Economia y Direccion de la EmpresaThemes and trends in Australian and New Zealand tourism research: A social network analysis of citations in two leading journals (1994-2007)
2009, Journal of Hospitality and Tourism ManagementA citation analysis of tourism scholars
2008, Tourism ManagementThe role of research in the hospitality industry: A content analysis of the IJHM between 2000 and 2005
2008, International Journal of Hospitality ManagementAn investigation of academic leadership in tourism research: 1985-2004
2007, Tourism Management