Researching Students’ Experiences of Sexual and Gender-Based Violence and Harassment: Reflections and Recommendations from Surveys of Three UK HEIs

Abstract

In the US, ‘campus climate surveys’ are an established measure of the prevalence of, and students’ awareness of and attitudes to sexual and gender-based violence and harassment (SGBVH). They are regularly carried out by universities to assist SGBVH prevention and responses. Such surveys have only recently started to be carried out within UK higher education institutions (HEIs) and the three authors of this article all independently undertook such surveys in different HEIs. Comparing our experiences of undertaking these surveys across three HEIs allows us to explore similarities and differences in our experiences of this type of research, in particular the challenges which arose in carrying out such research in three very different types of HEI. This article presents reflections on the methodological and political challenges of such work. We discuss our rationales for initiating these projects, the methodological approaches we employed, the governance structures navigated in pursuing the research and the difficulties that arose in conducting and reporting on the research. This article will be of interest to academics, activists, and policy-makers—domestically and internationally—who wish to carry out such research. By comparing approaches, we draw attention to issues and potential impediments of relevance to others wanting to embark on similar work within their own HEI.

1. Introduction

The last decade has seen a marked growth in awareness of the incidence and prevalence of student and staff sexual and gender-based violence and harassment (SGBVH) in UK higher education institutions (HEIs), with concerns about institutional responses and outcomes. As noted by other authors (such as Cantalupo 2014), an important way to enhance visibility and awareness of SGBVH is carrying out staff and student surveys, as these can highlight the gap between experiencing and reporting rates. Robust SGBVH survey data are increasingly necessary as a counterpoint to data relating to formal reporting that may be available within institutions. Moreover, survey data can help make a positive change in determining institutional priorities in relation to both student support and discipline (Follingstad et al. 2021).

In the US, Ireland, and Australia, large-scale national surveys of SGBVH in HEIs have been carried out (Australian Human Rights Commission 2017; Cantor et al. 2015, 2019; Heywood et al. 2022; Active Consent & Union of Students in Ireland 2020; MacNeela et al. 2022a, 2022b). North America has the longest history of ‘campus climate surveys’1, i.e., surveys of SGBVH in HEIs surveys, presumably reflecting the ‘productive confluence of research and activism in the US, set in a responsive political climate, and most importantly a legislative structure [Title XI2] which has both mandated the collection of information and made funding available’ (Phipps and Smith 2012, p. 13). Pritchard et al. (2019) outlined that many North American campus survey instruments were established following the 2013 Federal Sexual Violence Elimination Act (SaVE), the establishment of the 2014 White House Task Force to Protect Students from Sexual Assault, and Kirby Dick’s 2015 documentary The Hunting Ground. US institutions developed a plethora of survey instruments (Wood et al. 2017), mostly focusing on student sexual misconduct, which assess students’ perceptions of safety on campus, fears of and/or experiences of sexual assault (de Heer and Jones 2017), and a range of associated victimisation issues. As Cantalupo (2014) argued, there are benefits ‘not only for students, prospective students, parents, and the general public but also for schools [HEIs] themselves’ in carrying out such surveys:

Schools surveying their students do not have to rely on victim reporting to determine whether they need to respond to a widespread campus sexual violence problem or to develop response systems hurriedly in the shadow of a rapidly developing and potentially high publicity case. Instead, these institutions can employ such surveys to assess the extent and dynamics of the sexual violence problem among their students and use the survey results to inform their institutional responses to the specific manifestations of the problem on their campuses, well ahead of any high-profile or ultimately high-liability report of sexual violence.

(Cantalupo 2014, p. 228)

Thus, these surveys act as a ‘useful, but imperfect, sexual violence barometer’ (Sutton 2020, p. 9). However, what has been covered by these surveys has often been determined by governmental requirements (Follingstad et al. 2021) for example, the US mandates the Federal Department of Education to oversee HEIs’ collection and reporting on crime, including sexual offences, occurring on or near their campuses (GAO 2020). Yet, the absence of a standardised approach to such data collection means there is no consistency or agreed-upon terminology, even of the term ‘campus climate’, which Wood et al. (2017, p. 1253) noted ‘incorporates a large range of behaviours, environmental factors, and occurrences that promote or hinder student safety, acceptance, and ability to learn’ (see also Henry et al. 2011).

In the UK, such coordinated work—governmentally mandated or otherwise—does not (yet) exist. No survey tool has been adopted as the sector standard, yet Chantler et al.’s (2019) study of how British universities were tackling SGBVH found that 31 out of 71 respondents reported their institutions were doing prevalence surveys to establish baseline data. Only a handful of these appear to have publicly reported their findings. Making tools and data more widely available offers the opportunity to assess methods and findings allowing for nuanced comparisons. Currently, however, there are limited data about SGBVH in UK HEIs (see also Steele et al. 2021b); instead, existing insights come from students’ unions’ and third sector organisations’ surveys (Imperial College Union 2022; Brook 2019; National Union of Students 2011, 2018), academics undertaking their own research into the issue (Bovill and White 2022; Steele et al. 2021b; Roberts et al. 2019, 2022; Fenton and Jones 2017), and (more recently) learned societies such as the British Society of Criminology (Duggan and Walton 2022)3. Government data indicate that female students in the UK are three times more likely than women in other occupational groups to be subject to sexual assault and are far more likely than male students to experience victimisation (Office for National Statistics 2021). Regardless of gender, and more pressingly, there appear to be large discrepancies between the numbers of students (and staff) subjected to sexual misconduct, those who report it, and those who find action is taken by the institution after reporting. Journalists’ investigations have found that between 2017 and 2020, across 125 HEIs who responded to a Freedom of Information request, there were 1655 recorded complaints of sexual misconduct (1403 misconduct by students and 252 by staff) (Howlett and Davies 2021). Of these, 522 (487 complaints about students and 35 about staff) were actively investigated and only 213 of the complaints about student misconduct resulted in disciplinary proceedings against the respondent (Howlett and Davies 2021; see also Bull and Page 2022). This example illustrates that for HEIs to plan effective prevention and response work that meets staff and student needs, it is crucial they collect sexual misconduct data and compare it with complaint monitoring data.

In the absence of a UK national or HE sector-wide approach, a growing number of academic staff (often already engaged in SGBVH activism) have conducted independent SGBVH in HEIs studies, including the authors of this article, who undertook three surveys independently of each other into students’ attitudes towards, and experiences of, SGBVH at three HEI institutions. As more work on this issue was undertaken by UK-based academics, a network of like-minded scholars emerged who began discussing their experiences of undertaking such research. After comparing their individual experiences of undertaking student SGBVH surveys in HEIs, the three authors of this article felt it would be a useful exercise to compare and share these insights. Initially, we had sought to compare our survey findings, but as will be outlined below, this proved to be challenging due to our varying methodological approaches. This article explores our experiences of undertaking this work, focusing on rationales for data collection, governance structures navigated in pursuing the research, and the survey tools used. We reflect across the three studies, outlining similarities, differences, and the challenges which arose in conducting such research in three very different HEIs. By comparing approaches, we draw attention to methodological and political issues of relevance to others who want to embark on similar work. This article is, therefore, of interest to academics, activists, and policy-makers internationally and particularly those working in the UK context.

1.1. US ‘Campus Climate’ Surveys

Like in the UK, many US campus climate surveys are conducted in single institutions. Single-campus surveys produce useful data, but having different survey instruments makes comparisons across datasets difficult due to variations in methodology, definitions, and sample populations. Analysing ten such survey tools, Wood et al. (2017) found the surveys reviewed contained similar elements but substantial variation, which made it difficult to accurately comparatively assess institutional findings. Concern around a single-institutional focus prompted the Association of American Universities (AAU) to partner with academics and an external research firm (Westat) to design and implement the AAU Campus Climate Survey on Sexual Assault and Sexual Misconduct (Cantor et al. 2015). The resulting survey was administered to 27 universities and yielded a 19.3% response rate equating to 150,072 students (Cantor et al. 2015). Even between the three UK studies discussed here, differences in adaptations of surveys caused issues with comparability. Nonetheless, Wood et al. noted that despite impediments to comparison, there are other possible and desirable aspects to inquiries therefore it is important ‘to collectively and collaboratively analyse the existing surveys and their implementation processes’ and they go on to argue that ‘future inquiry should assess the quality of various climate survey tools and their application in an assortment of campus settings’ (Wood et al. 2017, p. 1265).

Giroux et al. (2020, p. 468) noted that ‘current methodologies for the collection of this sensitive data may lead to prevalence figures that misestimate the problem of sexual assault among college students, especially among men and other groups that may be more prone to under-reporting’. However, the growth in popularity of SGBVH in HEIs surveys has created the ability to generate and analyse data on specific sub-populations, and researchers have now begun to adopt approaches that address overlooked or less well examined sub-groups’ experiences. Such work has proved to be particularly important to discern different experiences for ethnic minority students (Solinas-Saunders 2021; Gomez 2022), disabled students (Kirkner et al. 2020), community college students (Howard et al. 2019), graduate students (McMahon et al. 2021), and international students (Fethi et al. 2022). This echoes Cowan and Munro’s (2021, p. 313) concern, i.e., in the UK context of sexual violence and misconduct in HEIs, rates of victimisation are ‘reflecting wider patterns of disadvantage, oppression, and abuse, it is also clear that factors such as race, disability, sexuality, and ethnicity intersect to create further differential exposure to risk’.

Various studies in the US have explored whether demographic factors influence university students’ experiences of sexual violence or victimisation. These include studies focusing on students from specific gender, sexual, and racial groups (Solinas-Saunders 2021); ethnic minority communities (Gomez 2022); disabled students Kirkner et al. (2020); and community college students, who are more likely to be from working-class backgrounds and which comprise greater numbers of women and minorities (Howard et al. 2019). In addition, McMahon et al. (2021) and Rosenthal et al. (2016) focused their analysis on graduate students, noting that while many campus climate surveys include graduate students, few studies focus specifically on this cohort’s experiences. Methodological explorations have focused on issues such as the relationship between survey timing and findings to discover whether releasing an online campus quality of life survey in four waves (over four weeks) had any impact on when survey participants chose to ‘tell their story about sexual assault and intimate partner violence’ (Pritchard et al. 2019). Pritchard et al. (2019) also indicted the use of a free text box in their survey, noting that 14% of participants took the opportunity to provide additional information. However, a number of issues have been identified in the reliability and validity of US campus climate surveys, including low response rates, lack of access to expertise on survey design, and lack of resourcing for administration and analysis of surveys (GAO 2020).

1.2. UK Policy and Data Collection Context

Recent UK policy developments, which are the driver for institutional data-collection around SGBVH, have largely been a top-down exercise, led by national representative bodies such as Universities UK, an advocacy group representing university leadership across the UK. This is despite decades of extensive grassroots work by staff and students. Prior to the 2010s, the last significant piece of work for universities on the issue was the release of the 1994 ‘Zellick’ guidance on complaints about student misconduct, which might also constitute the basis of criminal proceedings. A more recent stimulus was, arguably, the co-incidental timing of ‘revelations’ about Jimmy Savile, a prominent figure in the entertainment industry who was revealed after his death to have been a prolific sexual offender, alongside the publication of a survey by the National Union of Students (NUS) revealing the high levels of sexual and gender-based violence and harassment experienced by students in UK universities (National Union of Students 2011; Phipps and Smith 2012). Both subsequent inquiries into Savile’s offending (Lampard and Marsden 2015; Scott-Moncrieff 2015; Smith 2016) and the NUS Report highlighted how organisational cultures contributed to problematic traditions which enabled and protected harmful and victimising experiences, whilst not providing adequate institutional support for victims. The National Union of Students (2011) found that 68% of respondents had experienced sexual harassment, 16% had experienced unwanted kissing, touching, or molesting, and 14% had experienced serious physical or sexual assault while at university, the majority of which took place in public (National Union of Students 2011; see also Phipps and Young 2013). In 2015, the government instructed Universities UK to investigate SGBVH in HEIs (including domestic abuse, stalking, sexual harassment and violence, and hate crimes), and their subsequent report (Universities UK 2016) stated that universities urgently needed to acknowledge and respond to student experiences in this area. The report offered extensive guidance to institutions on implementing changes and substantial grants were made available via Higher Education Funding Council of England (HEFCE) from a £2.45m so-called ‘Catalyst’ funding pot to help embed initiatives. Alongside this, new guidance was commissioned from Pinsent-Masons (Universities UK/Pinsent-Masons 2016) to replace the 1994 ‘Zellick’ guidance (Zellick 1994).

Whilst this funding encouraged HEIs to conduct various work addressing SGBVH, much of the awareness raising work was undertaken by groups of activist students and staff, such as NUS Women’s Campaign, Revolt Sexual Assault, Pro-Empower, Empowered Campus, The 1752 Group, and Our Streets Now, often prompted by their members’ own victimisation experiences (see for example Empowered Campus 2020; Page et al. 2019). Furthermore, the Catalyst fund can be critiqued for (amongst other things) the decision to fund individual institutional projects rather than taking a national or regional approach, and for failing to incorporate standardised evaluation at either the institutional or sector level. Therefore, funding served to silo knowledge about contributing factors to different prevalence rates rather than enabling comparisons between institutions. While Universities UK and the Office for Students (which replaced HEFCE) did publish Catalyst funding consolidation reports on the projects and themes arising from successive rounds of funding (Universities UK 2018; Office for Students 2018, 2019; Baird et al. 2019), these reports did not include information on whether and how HEIs were measuring prevalence rates, comparative prevalence rates themselves, nor the attitudes and cultures around SGBVH in HEIs. Instead, these reports provided a range of case studies from institutions who submitted information for inclusion. However, relating to surveys of SGBVH, the reports only briefly mention ‘internal surveys’ being carried out within institutions to understand where to target prevention campaigns (Universities UK 2018, pp. 30, 33), while noting that ‘there is a need for better accuracy and consistency in data collection’ and recommending that ‘a set of data standards which could be used by individual providers’ be devised to ‘enable HE providers to undertake their own campus climate surveys and compare these with their peers’ (Office for Students 2019, pp. 13–14).

However, there are a growing number of academic articles and some grey literature presenting survey data relating to student experiences of SGBVH in UK HEIs. These can be organised into three types. The first are reports from activist or third sector organisations who conducted surveys to boost visibility of campaigns on sexual harassment and violence, which include Pro-Empower (2022), Brook (2019), Revolt Sexual Assault/The Student Room (2018), and the National Union of Students (2011). These surveys do not use standardised methodologies or questions and tend not to discuss their methods or analysis in detail (for example, lacking information on survey instrument design; participant recruitment; incentives to participate; response rates; and decisions around implementation and analysis). As such, their findings are not comparable and do not establish a baseline that can be tracked over time (nor was this their intention), nor provide data for individual institutions to effectively address the issue. Most focus solely on sexual violence and harassment rather than all forms of SGBVH, thus prioritising certain forms of SGBVH and invisibilising others, most notably domestic abuse (Khan 2021).

The second group of survey reports come from students’ unions; Bristol University Students’ Union (2021) and Imperial College Union (2022) both published public reports based on their survey findings on topics such as attitudes towards consent, awareness of support structures, and experiences of sexual violence and harassment. Similar to the activist group surveys noted above, these reports are helpful for orienting institutional priorities and actions, but they do not draw on standardised survey tools and they give variable amounts of detail on methods, and therefore, they are difficult to replicate.

Finally, peer-reviewed articles have been published detailing findings from student surveys carried out at individual UK universities (Bovill and White 2022; Steele et al. 2021b; Roberts et al. 2019; Fenton and Jones 2017) as well as a national study of staff–student sexual misconduct from research and campaign organisation The 1752 Group (Bull et al., forthcoming; National Union of Students 2018) and in Scotland, a study from the Equally Safe in Higher Education group (McCarry et al. 2021). These studies explore student attitudes towards, prevalence of, and reporting experiences of SGBVH. Fenton and Jones (2017) and Bovill and White (2022) both focused on undergraduate student attitudes and made important contributions to UK work. Fenton and Jones (2017, p. 150) noted at the time that ‘specific [SGBVH] research regarding UK undergraduate student populations is in its infancy’. They surveyed 381 undergraduate students at a university in the south west of England, exploring attitudes towards and readiness to intervene in SGBVH, using the Illinois Rape Myth Acceptance Scale (IRMAS, discussed below), a self-devised Domestic Violence and Abuse [DVA] Myth Scale, and a ‘readiness to help’ scale. They found that in some cases, rape myth acceptance predicted DVA myth acceptance, and argued that ‘rape and DVA myths need to be targeted in the development of effective prevention programmes in English Universities’ (p. 150). Similarly, Bovill and White’s (2022) article explored awareness of and confidence to intervene in SGBVH among first-year students at a university in the west of England. This study also included an experimental strand via a second survey which divided respondents into three groups who received different levels of intervention and found ‘associations between active intervention and raised awareness’ (p. 2801). Roberts et al. (2022) asked just over 1000 students at a university in the north west of England their perceptions of safety; experiences of interpersonal violence involving verbal abuse/bullying, physical violence/abuse, sexual violence/abuse, and stalking/online harassment; and their practices of help-seeking (p. 290) using questions modelled on those in the National Union of Students’ (2011) Hidden Marks study. Steele et al. (2021a) used the Sexual Experiences Questionnaire with 1608 students at Oxford University to explore SGBVH prevalence rates and experiences noting a ‘dearth of rigorous research assessing the prevalence of sexual violence among higher education students in the UK’ (p. 83). This study is unique in publishing a pre-study protocol (Steele et al. 2021a) outlining their methods in detail. However, despite strengths in gathering data on perpetration and online harassment as well as victimisation, it follows the problematic trend in the sector of focusing only on sexual violence and harassment (Khan 2021). Two national sector studies have also devised bespoke survey instruments. Campaign and research organisation The 1752 Group with the National Union of Students’ surveyed 1839 students in the UK about experiences of SGBVH perpetrated by higher education staff, reporting experiences and attitudes towards professional boundaries (National Union of Students 2018; Bull et al., forthcoming; Bull 2022). More recently, in Scotland, the Equally Safe in Higher Education project (McCarry et al. 2021) created a newly-devised survey, trialled with 2000 students and 1100 staff across four universities, asking about campus safety; attitudes to and experiences of emotional abuse, physical violence, stalking, sexual harassment, and sexual violence; impact of abuse; report and support pathways for victims/survivors; and the wider culture, particularly ‘lad culture’ and gender inequalities.

Thus, there is a growing body of survey data on attitudes to and experiences of SGBVH among students and staff in the UK, but data are of variable quality and differing levels of methodological transparency. While students’ union and activist groups’ surveys are important in raising awareness of this issue, academic researchers have carried out surveys with more ambitious aims, such as understanding how to change attitudes. Furthermore, there also now exist several different survey tools that can be drawn upon to carry out this work. It is important, therefore, to think carefully about what data are needed, and how we can work within the current political climate to gather more comprehensive, detailed, and comparable data in a transparent way. The critical discussion below of our implementation and publication experiences contributes to a developing critical conversation across the UK, and internationally, about the methods, politics, and governance of such surveys.

2. Overview and Aims of the Three Surveys

The three studies outlined here were conducted independently in separate institutions. Each was produced with different objectives, informed by different governance structures and used different partnership-working models. However, all three included academic staff as partners or leaders on the data collection and analysis. While the initial intention was to provide a snapshot of SGBVH in each of the three institutions, discussions among these academic staff (the authors of this article) yielded important insights into the factors informing the design, development, and dissemination of these surveys and their respective findings. These discussions evolved into a comparison between how these studies were undertaken, which is the focus of this paper. This section provides a brief outline of each of the three universities—A, B, and C—alongside relevant information about the evolution of each of the surveys.

University A is part of the cohort of institutions established in England in the 1960s (‘plate-glass universities’). It has over 17,000 students registered across several campus locations, making it relatively large. Before the 2016 Universities UK report, University A had already established an institutional steering group of senior management, Students’ Union representatives and selected academic staff members (including the survey lead) to design and implement SGBVH awareness and prevention initiatives including bystander awareness training for students, dedicated staff policy guidance, the employment of a (part-time) sexual misconduct welfare officer, and the production of an online reporting mechanism (released after the survey closed). By 2018, whilst these initiatives were embedding across the institution, no baseline data had been collected to assess the efficacy of initiatives and there was no centralised system for collecting data on incidents. University A received HEFCE Catalyst funding to support initiatives on preventing and responding to SGBVH among students and the survey lead applied internally to conduct an online survey to explore students’ awareness and experiences around sexual and domestic victimisation and the University’s efforts to respond to this. The funding was to employ a quantitative research assistant and offer financial incentives for completion. While the survey lead remained part of the institutional steering group, she conducted the survey independently, on a voluntary basis and in addition to a full workload, in the 2018–2019 academic year. Students were informed about the survey and their opportunity to participate via emails to their institutional accounts, yielding a 12% response rate (n = 2000) amongst undergraduate and postgraduate students.

University B is part of the post-1992 expansion of higher education in England. It is relatively small, and at the time of the survey, had around 5000 registered on-campus students and 2000 distance learners. Immediately prior to the survey (carried out during the academic year 2017–2018), there had been no institutional SGBVH initiatives. Similar to University A, University B received HEFCE Catalyst funding but, in this case, specifically to conduct a prevalence survey proposed as a way of beginning to address Universities UK (2016) report’s challenges to institutions. The grant was used to buy out two academics’ time for 0.5 days per week over six months to conduct the research and supervise a small team of student data analysts taking part out of personal interest and/or for credit bearing internship-style modules. Both researchers were relatively new to the institution (under three years employment there), although not necessarily new to higher education, and both were experienced working in the sexual violence field. Prior to the announcement of HEFCE funding, the researchers had discussed doing such a survey voluntarily and without funding or buy-out. The rationale for the prevalence survey agreed with the institution was to assess the extent of the five areas addressed by the HEFCE brief (sexual harassment, sexual violence, domestic abuse, stalking, and hate crimes) to then create an evidence-based proportional approach to the next steps and to act as a baseline to judge the efficacy of future initiatives. An internal steering group was constituted to oversee the project consisting of Student Union staff and officers, senior University management, and the researchers and this was disbanded shortly after the report was delivered. The researchers also recruited an external advisory group of relevant stakeholders from local sexual and domestic violence organisations, relevant strategic second-tier organisations, and the Police, and this was also disbanded after the report was finalised. Both the internal steering group and the external advisory group commented on various iterations of the surveys (one for staff and one for students). Participation was invited by one direct email to students and academic programme leaders and notification on internal student news sites. Ultimately, 4.1% of the campus-based University student population completed the survey with all levels of study represented proportionate to their numbers within the institution.

University C is a large post-1992 university in England. It had been awarded HEFCE Catalyst funding for earlier work on prevention and responses to SGBVH and by the time the survey was rolled out (November–December 2020), a ‘Report and Support’ system for SGBVH was in place, but without a dedicated staff member supporting students affected by SGBHV. The survey was not carried out by the university itself, but by a partnership between the Students’ Union and academic staff in the Sociology subject area developed through an initial conversation between the Students’ Union Welfare Officer, a data analyst from the Students’ Union staff, and a member of the Sociology academic staff who had previously carried out research in this area. Two further members of the Sociology team joined later to support data analysis and to develop teaching materials to disseminate survey findings across the university. The survey, analysis, report, and development of teaching materials were carried out without funding and in addition to the team’s regular workloads. Governance differed from the other two surveys because the work was carried out independently of the university, but appropriate university senior leaders were informed and consulted throughout. The aims of the survey were to assess students’ experiences of SGBVH, by whom and where it was perpetrated; to understand students’ attitudes about SGBVH to inform prevention work; and to use the data to highlight the issue across the university, including developing policy and practice. The survey was initially scheduled for March 2020, but was delayed to November 2020 by the first COVID-19 lockdown. It was then released in line with recommendations from the US National Academies that campus climate surveys should not be delayed due to COVID-19 (Holland et al. 2020). The survey, disseminated by the Students’ Union Welfare Officer via email to all students enrolled at the university, achieved a response rate of 4.2% (n = 1303), with 725 of those agreeing for their data to be reported publicly (therefore, 2.4% response rate for publicly reported data4).

2.1. Surveying Sexual and Gender-Based Violence and Harassment

All three survey instruments were based on the Administrator-Researcher Campus Climate Consortium (ARC3) survey (Tilley et al. 2020) developed by US sexual violence researchers together with university administrators in 2014–2015, focusing on US Title IX, and used in a range of US institutions (Tilley et al. 2020; Swartout et al. 2019) and elsewhere. Universities B and C used the ARC3 survey modules directly with minor amendments, and University A used a derivative survey5 as its base. ARC3 is freely available online, can be tailored to the needs of different institutions and includes 19 modules on, amongst others, sexual harassment and violence, stalking and ‘dating violence’ victimisation and perpetration, general campus climate, understandings of consent, peer norms, alcohol consumption, and sexual violence prevention programmes (Swartout et al. 2019). Developed based on research evidence, ARC3 has been psychometrically tested for internal reliability and correlation in measures (Swartout et al. 2019) and it was part of the US Government Accountability Office’s review of campus climate surveys (GAO 2020). All three research teams independently adapted the survey to fit specific institutional (and British) contexts. Universities B and C also included the updated/modified Illinois Rape Myth Acceptance Scale and University C added a scale of students’ levels of comfort with professional boundaries with staff drawing on questions from the National Union of Students (2018). The following section outlines how these tools were adapted and employed across the three institutions.

The survey conducted at University A was adapted for the British context from a survey itself derived from ARC3, which, as noted above, included questions on attitudes to and awareness and experiences of sexual and domestic victimisation. Due to the sensitive nature of the topic and to reduce ambiguity, University A reworded some questions (Fisher et al. 2000; Hamby and Koss 2003). For example, it avoided using words such as ‘rape’ based on research showing that many people who experience rape do not identify their experiences with that term (Kelly 1988)6. Instead, University A’s questions described types of behaviours that constitute sexual and domestic victimisation. Some questions employed gender-neutral language (i.e., ‘crotch’, ‘someone’) whereas others outlined specific body parts (i.e., ‘penis’) to determine in line with the UK legal definitions of rape and sexual assault exactly what type of incident had occurred. Question types ranged from multiple choice and Likert Scales to open text boxes to allow for more information (Fisher et al. 2000). The survey comprised over 100 questions (excluding demographical questions), focusing on behaviours, experiences, locations of incidents, and the relationship between the respondent and the perpetrator; however, some questions were only visible to respondents based on their prior answers (a strategy also used by University B); for example, those who indicated that they had been sexually victimised had an additional question about their relationship with the perpetrator (i.e., a stranger, an acquaintance, a romantic partner, other). Due to the substantial length of the survey, questions about numbers of incidents experienced were omitted.

Universities B and C both adapted the ARC3 Survey directly and both used the modules on sexual harassment, sexual violence and stalking victimisation. University C also used the ‘dating violence’ module, whilst University B adapted the ‘dating violence’ module to recognise that for mature students (and others) the relevant term was domestic abuse rather than dating violence. This highlights the need to recognise that students are not homogeneous and their diversity needs to be acknowledged in the wording of questions. University B also used the questions on general university culture and peer norms. Due to concerns about length, both University B and C omitted the modules on perpetration, consent, and alcohol consumption. University B’s survey was the widest in scope covering all five Catalyst funding areas (sexual harassment and violence, domestic abuse, stalking and hate crimes), asking about experiences before coming to university to assess ‘legacy’ support needs and victimization rates and asking whether respondents had witnessed or heard about incidents against others as a way of acknowledging that SGBVH impact on secondary and tertiary victims (Sellin and Wolfgang 1964). This inevitably meant a long complete survey, although many questions were only made available based on previous answers. University B’s survey also included questions to assess willingness for bystander intervention, normative views relating to the SGBVH categories (Fenton et al. 2016; Berkowitz 2013), general campus climate, willingness to report, and whether students felt they received enough information about support pathways.

2.2. Issues in Survey Design and Implementation

There were various difficulties arising in the survey design and implementation process, including the length of surveys, response rates, use of free text boxes, ethical approval, inclusion of staff, issues around the questions (definitions of ‘off campus’, frequency of occurrences of SGBVH), and incentivisation. As noted above, keeping the length of survey manageable (to encourage completion) whilst also including the most important questions was a difficult decision for each survey team. Decisions on which aspects to include were determined by the institutional political climate, the survey rationale, and the data needed to make the case for changing practices. Universities A and B had lengthy surveys that took on average 28 min to complete even though each respondents’ survey length was determined by previous answers triggering the opening up of new questions. University C omitted several questions (i.e., around reporting behaviour) to keep the survey shorter, but it still took respondents an average of 22 min to complete. All surveys followed best practice in sexual violence survey design, making all questions optional, and we did not find that this led to any particular problems with missing data. In contrast to concerns in the US around low response rates (GAO 2020), we argue that the response rates at the three sites were acceptable in comparison to other such surveys and were particularly high at University A at 12% (for example, Steele et al. (2021b) had a 6% response rate for a survey of 15–20 min). Thus, it seems acceptable to students to fill out a long survey on this topic. One influencing factor may have been that these surveys were the first of their kind in each of the universities, and that the topic is considered important by students particularly when students are asked for their experiences. Moreover, evidence suggests that such surveys do not appear to be especially distressing to respondents; while we did not ask about respondents’ distress, Swartout et al. (2019) found students rated such surveys as generally no more distressing than things they encounter on a daily basis, although in their study, 25% did report some level of distress during survey completion.

The use of free text boxes in our surveys was also a point of discussion in designing the surveys. Emulating previous research (for example, Pritchard et al. 2019) University A included a free text box for further comments at the end of the survey and several respondents used this to add comments about the survey (i.e., omitted questions, difficulties in answering some questions, appreciation for it being undertaken, etc.) or to provide more detail about what had happened to them, to raise concerns about unsafe spaces on campus, and draw attention to other specific issues. University C used free text questions as suggested in the ARC3 survey, by asking respondents after each module ‘how do you label this experience?’. University B did not include longer free text boxes, except where ‘Other’ was a possible answer for a question on the basis that the internal steering group was concerned that it would be used to disclose details of experiences (as found by University A) or request support, which was problematic as it was an anonymous survey, and thus, requests for support could not be responded to beyond Helpline numbers on Briefing and Debrief pages. This concern was probably unfounded, as neither University A nor C found that respondents used free text boxes to request support. Overall, despite the risks, we recommend the use of free text boxes in order to give respondents the opportunity to explain their experiences in their own words or to communicate what they feel is important. Risks can be mitigated against; for example, University C instructed respondents not to name any students/staff who had harmed them and any identifying data were deleted from free text comments immediately after the survey closed. However, the free text questions did produce a large amount of data that were time-consuming, challenging to code, and in some cases distressing in nature, and these issues need to be taken into account in the planning process.

In contrast to Donovan et al. (2020, pp. 135–36) and McCarry et al. (2021, p. 12), none of the researchers here found gaining institutional ethical approval to be a problem. Researchers at Universities A and B both found ethical review processes to be supportive and helpful, while at University C, it was relatively straightforward (although as discussed below, publication and dissemination was challenging for all). While there is no space here to go into detail about the ethics of carrying out such surveys, the surveys were used to inform students of available support either through debrief forms or including an option that allowed respondents to tick a box asking for the counselling service to contact them directly after the survey closed (University C).

It is also worth noting that staff were only included as respondents in one of the studies discussed here. This mirrors the wider focus of the HEI sector on supporting students in relation to SGBVH while omitting to acknowledge staff’s victimisation experiences, nor their support role, whereby they may be the recipient of disclosures. Perhaps surprisingly, data on staff experiences of SGBVH and support offered to them are not required for gender equality audit exercises, such as Athena Swan, and therefore, there is little incentive for HEIs to gather these data. By contrast, in Ireland, a national survey of staff experiences, attitudes, and knowledge of SGBVH was led by the Irish Higher Education Agency (MacNeela et al. 2022a, 2022b). Future surveys of SGBVH in universities should, therefore, include staff experiences where possible, building on findings from the Universities and Colleges Union’s national survey of staff in UK universities (UCU 2021) and the experience at University B.

There were further issues in the design, implementation, and analysis related to the wording of questions, including definitions. For example, both University A and C found it difficult to define ‘on’ and ‘off’ campus within the survey. For University A, questions around perceptions of safety on or off campus did not clarify where or what the ‘off campus’ space might be; therefore, respondents may have read this as the wider city in which the university is situated, their own home geographical region, or their own home (residential) space. University B, however, did clarify such definitions giving options of ‘on-campus’; ‘off campus but at a university-related event including Society and Sports events’; and ‘off-campus unconnected with the University’. This helped in writing up findings in defining the spaces in which students were most likely to be victimised. Another issue related to victimisation questions and whether to ask for simple yes/no answers about whether victimisation had been experienced, or to ask about the number of times that it had occurred. Universities A and B asked yes/no questions, while University C gave multiple-choice options for how many times the respondent had been subjected to a particular behaviour. The latter was more helpful in analysing data and revealed important findings around which groups were more likely to be repeatedly targeted for SGBVH in University C. As we will discuss later, these differences in adaptation meant it was very difficult to comparatively analyse the data across the three surveys.

A further issue discussed in relevant literature is the use of incentives for participation. In US campus climate surveys, incentivisation varies considerably in nature, value, and availability. Financial incentives tended to be gift cards, ranging in value from $10–20 (Gomez 2022; Howard et al. 2019) up to $50 or more (Pritchard et al. 2019), or cash prizes (McMahon et al. 2021). Prior research has demonstrated offering participants the opportunity to enter into a monetary prize draw (described as ‘lotteries’ in the US literature) often proves more effective than other forms of participation incentives (Couper and Bosnjak 2010; Pedersen and Neilson 2016; both cited in Pritchard et al. 2019), although it tends to rely on fewer, but higher value, awards. However, incentives are not necessarily required: Solinas-Saunders’ (2021) study offered no incentive or compensation, but their respondents completed paper questionnaires directly handed to, and collected from them in classroom settings. In our studies, we found mixed evidence around the use of incentivisation. University B offered no financial incentivisation, on advice from their Student’s Union that financial incentivisation ‘felt wrong’ given the topic of the survey. However, University C offered three £50 vouchers yet had similar survey completion rates to University B. By contrast, University A allocated £900 to a prize draw to win a £30 voucher and it seems possible that the larger number of incentives offered at University A contributed to their higher response rate (Pedersen and Neilson 2016 cited in Pritchard et al. 2019).

Offering financial incentivisation for participation runs the risk of bias in respondents’ answers. It may be the case that students at University A were more willing to participate due to the higher number of chances of winning a £30 voucher rather than through wanting to share their insights or experiences of SGBVH. However, having as many students from across the university as possible take part was the objective in offering thirty vouchers; to only have those with an interest in SGBVH take part would have missed a significant proportion of the available population. As the questions in University A’s survey also asked about awareness of institutional reporting processes and perspectives on the supportiveness of relevant infrastructures, it was considered valuable to obtain as broad a respondent base as possible.

2.3. Surveying Attitudes, Awareness, and Rape Myth Acceptance

Whilst there are a variety of attitudinal areas that can be covered in SGBVH surveys, in this section, we focus on one area of University B and C’s surveys that raised difficulties: rape myth acceptance. While the majority of US ‘campus climate survey’ tools do not include rape myth acceptance (Wood et al. 2017, p. 1259), Universities B and C decided to include these to inform prevention work. Both universities used the updated/modified Illinois Rape Myth Acceptance Scale (IRMAS) (McMahon and Farmer 2011). IRMAS uses explicitly gendered language based on an assumed binary gender hetero-norm (male perpetrator and female victim). University B adapted IRMAS (and similar scales for domestic abuse, stalking, and hate crimes), to remove gendered language (for example replacing ‘woman’ with ‘person’ or ‘someone’) where the myth presented could work for any gender. This was a difficult decision as the two researchers were already immersed in research and activism, which confirmed that most of these incidents are gendered (and homo-, bi-, lesbo-, and trans-phobic). However, at University B, the internal steering group (but not the external advisory group) raised concerns about the potential of the gendered nature of IRMAS to deter or alienate students who might feel that the implication of the gendered nature of such violence was provocative. Therefore, to gain institutional approval, changes were made to the wording of the questions.

Despite these changes, both Universities B and C received complaints from students arguing that the language used was ‘biased’, ‘suggestive’, and ‘narrow in its line of questioning’ and stating that the survey ‘should include how men can be raped too’. Thus, University B’s strategy of gender-neutral adaptation and the inclusion of elements of the Male Rape Myth Acceptance Scale (Struckman-Johnson 1992) was not entirely successful. These concerns may reflect the ‘gender neutral’ lens which characterises mainstream policies in this area such as the Changing the Culture report (Jackson and Sundaram 2020, p. 27). Alternatively, this may reflect some of HEIs ‘lad culture’, which includes frequent assertion of masculinity-under-threat, as thoroughly discussed elsewhere (Phipps 2013, 2017; Mott 2016; Phipps and Young 2015). Such attitudes clearly need to be challenged. The positioning of gendered problems as problematic because they are gendered is a double-edged sword. Of course, domestic abuse, stalking, sexual harassment and violence, and hate crimes are issues which need addressing irrespective of the gender identity of those victimised. However, well-established patterns of gendered perpetration and victimisation is not incidental but rather structural, driven by white, heteronormative, and homophobic hegemonies. To address the concerns raised by respondents, the survey team at University C added an explanation that this part of the survey was examining heterosexual behaviours and was based on evidence that cases including SGBVH harassment, hate crime, and violence, are most frequently committed by men against women. This was sufficient to stop further concerns being raised.

None of this is to say that IRMAS is unproblematic; however, the real issue is the exclusion of non-binary and genderqueer/gender-fluid identities and non-heterosexual relationships, which are particularly important given shifts among young people towards identifying with genders beyond the binary and sexualities beyond the heteronormative (Allen et al. 2021). University B’s adaptations partly addressed this, but low non-binary and transgender populations (and, thus, respondents) did not fully allow for the analysis of the ways in which transgender, non-binary, and genderqueer/fluid people may be targeted differently (although it did allow analysis for LGB respondents). Given the majority of SGBVH follows existing gender orders, regimes, patterns, and scripts which privilege heterosexual, cisgender men, we felt justified in including IRMAS, but would caution others to anticipate a backlash and to include an explanation similar to that used by University C. Alternatively, a ‘Gender Inclusive Rape Myth Acceptance Scale’ (Urban and Porras Pyland 2021), including myths about the experiences of cisgender women, cisgender men, transgender individuals, and gender diverse individuals has now been developed, which also includes updated language to reflect changes in discourse around rape since #MeToo. It could, therefore, be a more appropriate tool for future surveys wishing to explore this area.

2.4. Awareness of University Policies and Support

Student activist group Pro-Empower (2022) found in their UK-based survey that only 14% of their student respondents knew about their institution’s SGBVH policies and procedure and 40% lacked confidence in their university’s reporting procedures. Both Universities A and B asked respondents about their awareness of existing University policies and procedures and, at University A, about the level of interaction with several ongoing sexual violence prevention initiatives which proved a useful gauge of awareness and activity. Therefore, we suggest that these questions need to be a standard part of such surveys so institutions can assess the confidence their students have in their existing arrangements and the reach and impact of relevant initiatives (particularly as long-term evaluation strategies were not a core requirement of Catalyst funding, as noted above).

One way in which these issues were operationalised, at University A, was through respondents being presented with a series of statements pertaining to staff members’ attitudes and activities around sexual and domestic violence prevention and response and asked to indicate their levels of agreement using a five-point Likert Scale, similar to questions asked in Gomez’s (2022) research. The findings provided some insight into respondents’ levels of confidence in staff to recognise, respond to, and reduce instances of sexual and domestic victimisation among students. However, the wording grouped together all university staff, therefore not making it possible to disaggregate between professional services, support and academic staff, levels of seniority, staff demographic characteristics, and so forth. At University B, with the focus on improving institutional responses, each victimisation section asked whether the respondent had told someone at the University, and if so, what category of staff (but not staff demographics) and to rate the response received on a five-point Likert scale. This was useful for analysing both responses to particular types of reporting and responses by particular groups of staff and also highlighted the issue that respondents had often had to report to multiple (types of) staff sequentially, either because the University demanded multiple disclosures or because of failures of earlier staff to recognise and respond to what was being disclosed (see also Bull and Page 2022). The results challenged assumptions that professional staff in student help centres were always best placed and best trained to understand and help disclosing or reporting respondents.

2.5. Publishing and Disseminating Findings

In this section, in order to ensure that the institutions cannot be identified, we have adopted a more stringent anonymisation process. This means we have not linked the discussion below directly to the descriptions of Universities A, B, and C.

There were significant, but different, impediments to internally disseminating or externally publishing findings across the three surveys. In one institution, the researcher presented descriptive statistical findings to staff and students in a timely manner, but attempts to publish more widely a report on the survey were impeded because the research was done as ‘voluntary’ work on top of a heavy existing workload. This meant that the significant amount of labour that writing a report would require was not possible. At a second institution, although initial verbal assurances were given of the institution’s commitment to transparency through publishing data to both the university population and wider audiences, the institution later embargoed the results for either group. The reason given was that they could present the university negatively and deter applicants, potentially an example of ‘institutional polishing’ (Ahmed 2012; see also Phipps and McDonnell 2021, pp. 7–8). Thus, whilst a report was written in a timely fashion, its circulation was limited to a small group of senior institutional managers, Students’ Union sabbatical officers, and project funders. At this institution, there had also been a proposal to discuss the findings with groups of survivors at the institution to help with improving services for disclosing/reporting SGBVH, but this was also not possible because of the embargo. As this decision was announced late in the research process, there was too little time to challenge it, nor any mechanism through which to do so.

The researcher at the final institution had planned to publish the survey findings in a public report at the same time as a student-facing campaign would be launched. Alongside this, the researcher was working with colleagues to plan awareness-raising pedagogic work within the curriculum for first-year students in some areas of the university based on the survey findings. However, at the time of writing, a year after completion of the project report, neither of these steps have happened. While initial findings were presented to a relevant university committee early in the data analysis, the publication process has been delayed significantly, and with it any pedagogic work. While the project team were committed to disseminating the findings, there have been significant delays. It is possible that one factor here may have been a lack of institutional support for this work.

Overall, the failure to publish results of these studies stemmed from a lack of institutional prioritisation for this work, including from senior leadership, alongside a lack of resources to enable the work to take place in a timely manner. For some of the institutions, a further impediment was partnership working. At one institution, the dynamics of the relationship between the Student’s Union and the university was problematic, whilst at another, the main issue was contrasting advice being given to the researcher from different stakeholders. As noted above, within one university, there were clear indicators that institutional reputation was impeding publication. More important than institutional reputation, however, across all three sites, was lack of institutional support, resource, and, as a result, expertise for carrying out this work. Insufficient time, or in some cases no time at all, was allocated to academic staff to conduct the work and for the most part the work was carried out as ‘voluntary’ labour on top of existing workloads.

3. Discussion and Conclusions

This article has described the process of conducting SGBVH in HEIs survey research within student (and in one case, staff) populations by three UK university academic staff. Across the projects, such work was inhibited by lack of institutional resource and support, albeit in different ways. However, we would concur with McCarry et al.’s (2021) experiences across four universities in Scotland that ‘some institutions were concerned about the risk to their reputations and would not support any research. Where reputational risk was not a concern, some institutions did not have the necessary resources to implement the research’ (p. 12). As Donovan et al. (2020) noted, the tension between securing funding to carry out work on SGBVH in HEIs and doing it voluntarily mirrors the challenges faced by Violence Against Women and Girls organisations In the 1970s (p. 140) (and which continue to date). Carrying out such work on a voluntary basis proved to be unsustainable, and in all of our studies, voluntarism contributed to challenges in publication and dissemination of the findings. However, being funded for such work also leads to problems with institutional governance and control.

Across the three institutions, thousands of students spent time completing surveys, but none of the institutions have (at the time of writing) published findings in a student-friendly format. We find this hugely problematic. Whilst peer-reviewed articles are under development, academic publication formats are not accessible communication methods for reaching students, nor do they facilitate the use of findings to support awareness-raising within institutions. None of the surveys have been used as a baseline for future work or for evaluation of initiatives, as had been intended. At both Universities B and C, there were intentions and/or recommendations to carry out further biannual surveys, but the difficulties experienced with the first iteration of the research now makes this is unlikely. This demonstrates the lack of long-term planning for this work; rather than being embedded into institutional or sector frameworks, it instead relies on the energy and commitment of individuals. As such, this research risks becoming seen by the institution as a completed end in itself rather than as the start of improvement processes.

These difficulties highlight wider problems around the transparency of SGBVH in UK HEIs surveys and begs the question of what happens to data from these studies. As noted above, according to Chantler et al.’s (2019) survey of university staff involved in addressing SGBVH, by 2019 at least 31 universities in the UK had carried out such surveys to establish baseline prevalence data. Further studies, such as University C in this article and Steele et al. (2021a), have been carried out since then (and data collection is ongoing at Queen’s University Belfast, with a report due in September 2022). However, little of this work has led to accessibly published reports or academic articles. Thus, there are a large amount of unpublished data on this topic in the UK. Which poses further questions: are these data being used internally within institutions? If so, how? Are any institutions carrying out further studies to build on their baseline findings? Moreover, and perhaps most importantly, where is the knowledge sharing and collaborative learning from those of us who have carried out these studies?

In addition to these urgent questions, our article highlighted issues in the rationale and design of SGBVH in HEIs surveys; whether they should prioritise experiences, including working to assess prevalence, attitudes, in order to assess where to target intervention work, or reporting behaviours and knowledge of support systems and university processes. The surveys described here covered a combination of attitudes and experiences. This allowed for priorities to be identified to address attitudes among the student body, for correlations between attitudes and behaviours to be seen, and, for two institutions, for patterns of reporting behaviours and assessing University processes and support to be made visible. The specific questions possible within these areas are vast, and as discussed, many tools have not responded to changing understandings of gender and sexual identity or the increasing importance of digital spaces in both social and educational settings. In particular, we highlighted methodological issues with and resistance from students to the use of the updated IRMAS, and the US/UK differences in social and legal contexts which make ARC3 (and similar) survey(s) in need of adaptation. There are multiple possible standardised surveys available: Steele et al. (2021a, 2021b); Universities UK’s tool, currently in development with academic partners; ARC3, as used here; the National Union of Students’ (2011) survey tool (Roberts et al. 2022); and the Equally Safe in Higher Education survey tool. Nevertheless, we would recommend an adapted ARC3 survey as a baseline for future UK surveys, not least because it is publicly available and allows for international comparisons. However, relevant academic expertise is still needed in adapting and implementing it, as Universities B and C found that amendments to the ‘dating violence’ module were needed for the UK context, and University C also had difficulty with using the ‘stalking’ module in the UK. A shared, collaboratively developed survey tool is urgently needed to counter difficulties in merging or comparing data from surveys. In the absence of effective leadership from statutory or policy-making bodies such as the Office for Students or Universities UK, which contrasts with Ireland and Australia, where national bodies have led the designing and conducting of such surveys (Australian Human Rights Commission 2017; Heywood et al. 2022; MacNeela et al. 2022a, 2022b), the UKs’ more fragmented approach makes it incumbent upon academic researchers in this area to work with activists and agree on which, if any of these, we should be using if we are finally to move beyond Universities UK’s (2016) findings that “despite some positive activity, university responses are not as comprehensive, systematic and joined-up as they could be” (p. 4).

On a more positive note, a growing network of scholars working in and around the SGBVH arena are increasingly turning their attention to how this features in the HEI sector and are developing new knowledge and insight relevant to the UK context. Therefore, it may be prudent to look to international examples of successful coordination among scholars as a template for how to synergise these efforts and improve future data collection and dissemination. In Spain, gender equality policies have been institutionalised within universities, leading to the adoption of gender equality plans and sexual harassment protocols (Lombardo and Bustelo 2022). However, research by Lombardo and Bustelo (2022) has indicated the limited impact of these activities without investing in gender experts to facilitate them. To this end, several groups have been established across universities in Spain to raise the profile of violence-reduction work.

To conclude, the current policy context around tackling SGBVH in UK higher education suggests that surveys of students’ experiences of and attitudes towards SGBVH are likely to increase in number and frequency. However, as the discussion in this article demonstrates, knowledge sharing and expertise in this area remains at a relatively early stage. We hope that the discussion here contributes to a more joined-up conversation about these issues enabling us to learn from international experiences and implement shared protocols, develop standardised survey tools, and embrace open data practices.