https://orcid.org/0000-0003-1513-934X
Abstract
Textbooks play a critical role in schooling around the world. Small sample studies show that many books continue to under-represent women and girls, and to portray men and women in stereotypical gendered roles. In this paper, we use quantitative text analysis to assess the degree of gender bias in a newly assembled corpus of 1,255 English language school textbooks from 34 countries that are publicly available online. We find consistent patterns of under-representation of female characters and portrayal of stereotypical gendered roles. Women and girls appear less frequently, are portrayed as more passive, are less likely to be associated with work or achievement, and are more likely to be associated with the home and traditionally female occupations. Comparing across countries, female representation in books is correlated with higher GDP and more legal rights for women. Under-representation and stereotypes are a particular problem in South Asia.
Figures
Citation: Crawfurd L, Saintis-Miller C, Todd R (2024) Sexist textbooks: Automated analysis of gender bias in 1,255 books from 34 countries. PLoS ONE 19(10): e0310366. https://doi.org/10.1371/journal.pone.0310366
Editor: Andrew Kehler, UCSD: University of California San Diego, UNITED STATES OF AMERICA
Received: February 1, 2024; Accepted: August 23, 2024; Published: October 9, 2024
Copyright: © 2024 Crawfurd et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The code and data necessary to replicate all analysis can be accessed from the Harvard dataverse https://doi.org/10.7910/DVN/AJD0T4.
Funding: This research was supported by funding from the Bill and Melinda Gates Foundation and Echidna Giving. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
School textbooks play an important role in shaping identity, attitudes [1–5], and norms [6, 7]. Controversy over school book content is high in many countries including the US, where book bans continue today. The introduction of secular textbooks was critical in the political development of the religious right in America, and led to school boycotts, a shooting, and a bombing [8]. Despite the potential importance of books for shaping norms, still little is known quantitatively and systematically about the extent of biases in school books around the world.
In this paper we open the black box of global schooling, and provide quantitative insight into the content of school curricula and learning materials. Our particular focus is on girls education and gender bias. Girls education is a major global policy priority, but attention has focused primarily on the amount of schooling girls receive and the cognitive skills they obtain. If we care about equality of opportunity for boys and girls, we need to look beyond the building of human capital, to what children are exposed to in schools. Though there are many influences on children’s gender norms, school books make up a large share of children’s time spent at school, and may pass on implicit biases [9]. Many teachers make the majority of their instructional decisions based on textbooks [10].
Concretely, in this paper we ask how representation of males and females quantitatively differs across English-language textbooks globally, and to what extent textbooks portray stereotypes concerning the occupations, characteristics, and actions of male and female characters.
We use a newly compiled corpus of text documents, covering 1,255 school textbooks from 34 countries. We use ‘textbooks’ throughout as shorthand, though a small number of materials in our corpus are in fact student worksheets or slideshows. We process these documents using natural language processing and quantitative text analysis tools. We identify differences in the quantitative representation of male and female coded words, as well as in the qualitative descriptions of male and female words, using word co-occurrence, word embeddings, and dependency parsing.
Overall we find substantial gender bias in school textbooks across the 34 countries, with women and girls under-represented compared to men and boys, and portrayed in stereotypical feminine roles. In 28 of the 34 countries, women and girls occur less frequently than men and boys, and the gap for some countries is stark. Women tend to be portrayed in traditionally feminine occupations, such as nurse, teacher, and housekeeper. They are more likely than men to be associated with words relating to appearances and the home, compared to achievement and work. Women and girls are described as being more passive and less dominant than men, although they are also portrayed using more positive terms.
Comparisons between individual countries are limited by small samples, but higher income countries tend to have somewhat less unequal representation, although stereotypes still seem to be common. Countries with better representation also have better legal rights for women, even after adjusting for income levels.
Comparing across regions, South Asia has especially low representation of female characters, and somewhat stronger stereotypes, with more stereotypes in the adjectives used to describe male and female characters, a weaker association between female characters and work or achievement in general, and few women described as performing specific occupations.
Our analysis complements existing studies that have documented consistent gender bias in textbooks around the world, primarily through manual analysis and coding of texts [11, 12]. Studies on gender representation in textbooks from low- and middle-income countries come primarily from small-scale qualitative manual coding of books. Such studies typically show an under-representation of female characters, and associate women with lower-status occupations than men. This includes studies from Bangladesh [13], Cameroon [14], China [15], Ethiopia [16], Hong Kong [17], Iran [18], Nepal [19], Nigeria [20], Pakistan [21], and Palestine [22]. Some studies also present comparisons across a small number of countries, such as Bangladesh, Indonesia, Malaysia, and Pakistan [23], Australia, Hong Kong, Singapore, and Turkey [17, 24], and Rwanda, Kenya, and Uganda [25, 26].
A key advantage of our study is the ability to compare across a wider range of countries. Such larger comparative cross-country research is rare. One exception is a project at Stanford University looking at changes over decades, using books from the Georg Eckert Institute for International Textbook Research. One study reviews over 1,000 secondary school textbooks from 88 countries between 1950 and 2011, showing an increase in mentions of women’s rights over time [27]. Another study manually codes 559 secondary school textbooks from 76 countries, focusing on representations of globalization [28]. Another uses a similar approach to look at long-term trends in human rights discussion in textbooks [29].
Methodologically, our study is similar to that of some others applying natural language processing to school textbooks from high-income countries. For example, one study analyzes the representation of race and gender in 1,000 award-winning American children’s books, using both text and image analysis techniques [30]. A related study uses word embeddings in the same corpus, finding that women are more likely to be represented in relation to their appearance than in relation to their competence, and are more likely to be represented in relation to their role in the family than their role in business [31]. A study of the 247 best-selling books aimed at children under 5 years of age in the United States and Canada found that many books include gender stereotypes, such as girls being better at reading and boys at math [32]. Another study uses word embeddings to show that in American history textbooks women are discussed more in the context of the home and less in the context of achievement [33]. More broadly, the use of quantitative text analysis to document gender bias is growing across a range of topics in economics and political science, including judge’s opinions [34], Google News and Google Books [35, 36], local newspapers [37], and the Corpus of Historical American English [38].
Other studies on the impact of textbooks in developing countries have focused more on the provision of textbooks than on their specific content. For example, a randomized control trial of access to new books in Kenya found no impact on learning on average, and positive impacts only for the most able children [39]. An observational study using student fixed effects models found that textbooks had zero average effect on learning in 11 Anglophone African countries, but positive effects for children from the wealthiest families [40]. By contrast, packaged interventions including student books and teacher guides and training have been more successful, experimentally demonstrated in Kenya [41] and El Salvador [42].
In the remainder of this paper we first discuss the data and methods we use, then show our results, and finally discuss their significance.
Materials and methods
Data
Textbook corpus.
Our data comes from a systematic review of online materials. From September 2020 to February 2022, we visited every ministry of education website in the world and downloaded all available educational materials (many countries made learning materials publicly available online to facilitate online learning during the Covid-19 pandemic). We manually reviewed the materials and selected only files that were in English and that could be classified as learning materials, rather than teacher guides or subject curricula. To ensure a broad range of countries were represented, we also searched for publicly available learning materials from websites other than ministries of education, focusing on countries with sizable populations and where English is an official language. Almost all of these books are published by government bodies, with the exception of the three high-income countries in our sample, where textbooks are published exclusively by the private sector (Australia, the United Kingdom, and the United States).
Our sample contains 1,255 textbooks from 34 countries, which total over 37.2 million words. These books cover a wide range of school subjects from grades 4 to 13 (see Tables 1 and 2). The included countries are: Afghanistan, Australia, Bangladesh, Belize, Bhutan, Dominica, Ethiopia, Guyana, India, Jamaica, Kenya, Kiribati, Lesotho, Liberia, Malawi, Maldives, Namibia, Nigeria, Pakistan, Papua New Guinea, Rwanda, Samoa, Sierra Leone, Solomon Islands, South Africa, South Sudan, Sri Lanka, St Kitts and Nevis, Tonga, Uganda, United Kingdom, United States, Zambia, and Zimbabwe (Fig 1). The countries in our sample, in which schooling is at least in part conducted in English, are of course different in one important way to other countries—many having a British colonial legacy. They are not necessarily so different in all relevant dimensions though. The countries cover a range of income levels and geographic regions, and have similar average levels of gender equality to other countries (see S1 Table).
Note: Books from the 34 countries included in our corpus are coloured in teal. This includes Afghanistan, Australia, Bangladesh, Belize, Bhutan, Dominica, Ethiopia, Guyana, India, Jamaica, Kenya, Kiribati, Lesotho, Liberia, Malawi, Maldives, Namibia, Nigeria, Pakistan, Papua New Guinea, Rwanda, Samoa, Sierra Leone, Solomon Islands, South Africa, South Sudan, Sri Lanka, St Kitts and Nevis, Tonga, Uganda, United Kingdom, United States, Zambia, and Zimbabwe.
Although data on actual use of textbooks is scarce, the evidence available points to textbooks as being widely available and playing a dominant role in the curricular resources used in classroom instruction. Although there has long been concern about the scarcity of textbooks in poorer countries [43], most of the pupils in school surveys report access to at least a shared textbook. The 2019 Program for the Analysis of Educational Systems of CONFEMEN (PASEC) survey of 14 Francophone African countries (Benin, Burkina Faso, Burundi, Cameroon, Chad, Republic of the Congo, Democratic Republic of the Congo, Gabon, Guinea, Ivory Coast, Madagascar, Niger, Senegal, and Togo) found that on average 60–75 percent of 13-year-olds had their own textbooks [44]. The 2018 Programme for International Student Assessment (PISA) Development survey of 15-year-olds in seven countries (Cambodia, Ecuador, Guatemala, Honduras, Paraguay, Senegal, and Zambia) found that overall around 45 percent of students had their own textbook, and another 45 percent shared with others, with only 10 percent reporting no access at all [45]. The most recent 2012 Southern and Eastern Africa Consortium for Monitoring Educational Quality (SACMEQ) survey of 14 Anglophone African countries, found that around 40 percent of grade 6 students owned reading and mathematics textbooks [46], with that number roughly doubled through sharing [40]. A 2015 Rwandan survey found that textbooks were widely available in all schools and in approximately three out of five classrooms [47].
Textbooks are also used as a primary instructional tool in the classroom, influencing not only what teachers teach, but how they teach [48]. The 2011 Trends in International Mathematics and Science Study (TIMSS) included survey questions on the types of materials used as the basis of instruction. 70 percent of fourth-grade science teachers and 74 percent of eighth-grade science teachers used textbooks the most often as the basis for instruction [49]. In mathematics instruction, 75 percent of fourth grade teachers and 77 percent of eighth grade teachers used textbooks the most frequently. The World Bank Service Delivery Indicator (SDI) surveys conducted between 2012 and 2016 in 8 African countries found that 37 percent of children in randomly selected grade 4 classrooms had the relevant textbook [50] (this ranged from 88 percent in Morocco, through 68 percent in Togo, 66 percent in Mozambique, 43 percent in Kenya, 40 percent in Nigeria, 19 percent in Tanzania, 14 percent in Niger, to 11 percent in Madagascar). A study observing classrooms in successful reading programs found that they spent a large share of class time using textbooks (50 percent in Kenya, 45 percent in India, 42 percent in Senegal, 28 percent in Nigeria, and 8 percent in Tanzania) [51].
In the majority of countries in our sample, the government provides a set of specific approved textbooks directly to schools, rather than schools making their own choices. Therefore, particularly in these countries with centralized supply, we can expect large proportions of children in school to be actually exposed to the books in our corpus [52]. Typically, poorer countries have a single set of government-approved and provided books, whereas in richer countries, schools have more autonomy to select books from a variety of providers. Government-approved books are distributed to schools in 23 countries in our sample (Afghanistan, Bangladesh, Bhutan, Ethiopia, Kenya, Kiribati, Lesotho, Liberia, Maldives, Nigeria, Papua New Guinea, Rwanda, Samoa, Seychelles, Sierra Leone, Solomon Islands, South Sudan, Sri Lanka, Tonga, Trinidad and Tobago, Uganda, Zambia, and Zimbabwe). Schools have autonomy over textbook choice in Australia, Belize, Dominica, Guyana, Jamaica, Namibia, South Africa, St Kitts and Nevis, the United Kingdom, and the United States. India and Pakistan have a mixture of central government provision and local school choice in different states and provinces.
Gendered word lists.
We compare the content of the textbooks in our corpus with predefined lists of gendered nouns, pronouns, and names. We adapt the gendered nouns and pronouns used in a previous study [30], including words such as she/her/woman and he/his/man. In total, there are 68 male words and 68 female words (listed in S1 File).
To identify names and their gender in our text, we use damegender [53], a tool for gender detection generated using census data from 20 countries. Given the international source of the textbooks, we find that this performs better than datasets using only names from one particular country, like the United States. We only allow names above a frequency of 10,000 in this dataset, since names with a frequency below this threshold were significantly less likely to be names in context. We exclude other names that are unlikely to be names in context, including country names, city names, and months of the year. We exclude names where the gender of the individual may be ambiguous, defined as those where neither gender accounts for at least 95 percent of occurrences in the dataset.
We manually validated this process with a sample of 5 books on a range of subjects (English, Mathematics, Social Studies, Political Science, and Biology) and countries (Afghanistan, Bangladesh, Guyana, India, and Pakistan). We calculate both the rate of precision (the proportion of identified names that are in fact names in context) and recall (the proportion of names in the text which were identified). This validation process showed an average precision rate of 70 percent—meaning that from the names identified by the tool, 70 percent were actually names—and an average recall rate of 75 percent—meaning that the tool correctly identified 75 percent of the total names across the texts. We also examined the precision and recall rates between male and female names. The precision rate for female names was 65 percent compared to 78 percent for male names, meaning that the tool was able to identify male names that were actually names in the context of the text at a higher rate than female names. The tool was able to identify the number of female and male names at a similar rate, with the recall rate for female names 74 percent compared to 77 percent for male names. Because of this imprecision, we do not include names when measuring representation, instead only counting gendered nouns and pronouns. We do include names in the other parts of our analysis.
We have equal-length lists of 68 male nouns/pronouns and 68 female nouns/pronouns, but more unique female names (2,218) than male names (1,753).
Family and work word lists.
To construct lists of words in which we might expect gender bias, we adapt the lists used in several previous studies related to achievement, appearance, family, home, and work [31, 33, 54]. These include 13 words related to achievement, 6 related to appearance, 11 related to family and the home, and 20 related to work (full word lists are shown in S3 File).
To classify occupations we use a list of 1,156 occupation titles drawn from the Gazette (the official public record in the UK, and a commonly used source of text data, [55]). We then manually classify these occupations into three groups; professional, service, and manual occupations. We are left with 61 professional or managerial jobs, 23 service jobs, and 16 manual jobs (see S2 File).
Country-related gender characteristics.
We use several sources of data on gender norms and outcomes at the country level. First, the 2023 Social Institutions and Gender Index (SIGI) from the Organization for Economic Cooperation and Development (OECD). The SIGI index comprises 25 indicators covering discrimination, violence, bodily autonomy, economic rights, and civil liberties. Second, the World Bank Women, Business, and the Law Index measures women’s economic and legal rights, and comprises 35 indicators. Third, the Center for Global Development (CGD) Girl’s Education Policy Index (GEPI) measures policy effort related to girl’s education across 32 indicators, including spending, sexual health, safety, employment, and role models [56]. Fourth, to measure equality in education outcomes we use the World Bank Gender Parity Index (GPI) in Gross Secondary Enrollment. Fifth, we look at data on teenage marriage of girls, and on female MPs, both drawn from the 2023 OECD Gender, Institutions, and Development (GID) Database. We standardize all variables to a mean of zero and a standard deviation of one.
Methods
Data processing.
We begin by converting PDF books to text data, primarily using Optical Character Recognition (OCR) in Amazon Textract. This process uses a pre-trained model to infer text in an image, and recent advances in machine learning have greatly improved its accuracy; a recent benchmarking experiment found a word error rate of just 1.8% when Textract was used to convert a clear image [57]. We partially validate the conversion process by manually comparing a sub-sample of text files to the original PDFs. For several countries where textbooks had machine-readable text, we used a function in R which uses the Poppler library of PDF tools. This tool is able to extract text directly and has comparable accuracy.
We group books by subject into 14 groups; Agriculture, Arts, Business and Economics, Environmental Studies, French, General and Social Studies, Home Economics, Humanities, IT, Language, Math, Physical Education, Science, and Social Science.
Due to the limited number of books available from each individual country, we show results by three country regions; these are (i) South Asia, (ii) Sub-Saharan Africa, (iii) the three high-income countries Australia, UK, and US. Other countries not within these regions are very heterogeneous, being distributed across the Caribbean and Pacific. We also code whether books are funded by a foreign development agency. We include all countries in which textbooks include an acknowledgment of funding by a donor agency. This variable will include some measurement error if provided funding is not acknowledged.
Measuring gender representation.
We then measure the quantitative representation of male and female words in the corpus of textbooks. We do this by counting gendered nouns and pronouns. For each book, we calculate the share of female words as a percentage of all gendered words. We then show how the share of female words varies by country, subject, and grade. We also count the number of gendered words with a clear age classification (e.g., girl vs. woman and boy vs. man).
Measuring gender stereotypes
We use three different methods to assess gender stereotypes. The simplest of these is simply counting word co-occurrences within sentences. Second we use word embeddings, which are numerical representations of words to quantitatively measure the similarity between gendered words and other thematic words. Third is a method known as ‘dependency parsing’, which automatically infers the relationships between words, allowing us to identify the specific verbs and adjectives which modify gendered nouns. Using a variety of methods increases our confidence in the results and aids understanding.
Co-occurrence.
The simplest way of measuring stereotypes is counting the number of co-occurences within a single sentence of gendered words and specific target words. We count all co-occurrences of job occupations with gendered terms. For this analysis, we assume that the male or female term refers to the individual with the occupation. Although this analysis is conceptually simple, this assumption may introduce some noise and bias. For example, the sentence “he went to the doctor” would count as a co-occurrence of a male term with the occupation doctor, though in this case the sentence does not indicate the gender of the doctor. We omit cases where both male and female terms are mentioned within the same sentence. We count cases of implicitly gendered occupation terms (such as ‘salesman’) as equivalent to a co-occurrence of a gendered term and a gender-neutral version of the term (’man’ and ‘salesperson’).
Word embedding.
A more sophisticated approach to measuring stereotypes uses word embeddings. Word embeddings are numerical representations of words that capture their semantic and syntactic properties. We use the Word2vec algorithm [58], which encodes words as high-dimensional vectors or embeddings. Technically, Word2vec can be used to learn associations between any kind of token: for example, two-word phrases, or sentences. Word2vec trains a neural network on a corpus to perform one of two tasks: if implemented with a ‘continuous bag-of-words model’, to predict a token based on those surrounding it; and if implemented with a ‘skip-gram’ model, to predict the tokens surrounding a given token. Words which are mathematically closer in this vector space tend to be used in similar contexts in the training corpus and can therefore be regarded as more closely associated. Embeddings can also be combined arithmetically. An often-cited example is that the vector resulting from king − man + woman will in a well-trained model be very close to the embedding for queen.
Word embeddings can be used for various natural language processing tasks, such as text classification, sentiment analysis, and machine translation. But they also reflect human-like biases that are present in the text data on which they are trained. For example, gender bias can be observed in word embeddings when certain words are more associated with male or female attributes or occupations. In our case, we are interested in the distance between male and female words on the one hand and words relating to work, family, achievement, and appearance on the other.
Word embeddings require a relatively large sample of words and sentences, and so we train our model on our entire corpus rather than on individual countries or sub-samples of the data. We first estimate the embedding for each individual word. Second, we calculate the cosine similarity between each gendered word and each target theme word. We do this only for gendered nouns and pronouns, rather than names: many names will be very low frequency and embeddings will be correspondingly imprecisely measured. The similarity can take values between -1 and +1. Identical vectors have a cosine similarity of 1, perpendicular vectors have a 90-degree angle between them and a cosine similarity of 0, and opposite vectors have an angle of 180 degrees between them and a cosine similarity of -1. Third, we take the average similarity between all male words and each target word, and all female words and each target word. Fourth, we calculate the difference between the female similarity and the male similarity for each target word. This leaves us with a measure of female bias for each individual word themed on home, work, achievement, or appearance. Fifth, to assess the uncertainty in this measure of female bias, we bootstrap standard errors [33]. We do this by splitting the entire corpus into sentences, sampling the sentences with replacement, and training a new model on this new sample. We repeat this process 50 times, and take the standard deviation of the pro-female bias estimates across the 50 models as the estimate of the standard error.
For our main set of results, we use a continuous bag-of-words model, with a window size of 6, and generate embeddings with 300 dimensions, following standard best practice [59]. We use negative sampling which reduces the computational cost of predicting embeddings by sampling a few negative examples (words that are not likely to appear in the context of the input word), and minimizing their probability. We calculate the similarity of embeddings as their cosine similarity.
Dependency parsing.
Dependency parsing is a method of syntactic analysis that determines the grammatical structure of sentences by identifying the relationships between words. Unlike word embeddings, which capture semantic relationships based on word co-occurrence patterns, dependency parsing focuses on recovering structural relationships within sentences. Specifically, it uses pre-trained models to infer the grammatical connections between words, providing insights into sentence structure and syntactic roles.
The ‘universal dependencies’ framework [60] underpins this approach by first performing ‘part-of-speech tagging’—allocating all words in a language to one of a set of syntactic categories (such as adjective, noun, verb, etc.). This categorization depends both on the word itself and the context in which it is used. For instance, in English, ‘butter’ might be categorized as a noun or as a verb (as in ‘to butter the bread’), depending on its usage in the sentence. We make use of the UDPipe model trained on the ‘English Web Treebank’ [61, 62], a corpus of 16,621 sentences sourced from the internet that have been manually annotated. Applying this model to our corpus, we first measure the frequency with which male and female terms appear as the subject rather than the object of sentences, which may suggest that either gender is depicted in a more dominant role. We then identify the verbs and adjectives that are associated with male and female words, to explore how male and female characters are described, and what actions they are depicted doing.
Finally, in order to compare quantitatively the differences in how genders are portrayed across different country income levels, we conduct a sentiment analysis of the terms used with each gender. Specifically, we use the National Research Council Canada (NRC) Valence, Arousal, and Dominance (VAD) lexicon [63] which quantifies three aspects of words: valence—how pleasant or positive a word is; arousal—how active it is; and dominance—how dominant or submissive it is. This lexicon includes human ratings for more than 20,000 words. We calculate the average rating for each of the three dimensions for verbs and adjectives used with male and females terms, weighted by the frequency of the word.
Cross-country correlations
Does the existence of bias in textbooks correlate with other measures of gender equality at the country level? We look at the cross-country correlations between bias in textbooks and various measures of gender equality. There are several important limitations to this analysis. First, we aren’t able to make causal claims here, and causality could clearly run in both directions; from biased books to unequal norms, and vice versa from unequal norms to biased books. Nevertheless, we feel this is a useful test of the validity of the text analysis; a complete lack of correlation could raise doubts. Second, our sample of books is essentially a convenience sample and is not necessarily nationally representative of the books actually read by children in classrooms in each country. Third, we have a relatively small sample of countries at just 34, and so this analysis has low statistical power.
Results
Gender representation
Overall, across our full sample of books, there are more than twice as many occurrences of male words (178,142) as there are female words (82,113). There is also substantial variation between countries. After adjusting for book length, grade, and subject, the countries with the lowest representation of women and girls are Afghanistan, Pakistan, Sri Lanka, and South Sudan, where fewer than 1 in 3 gendered words are female (Fig 2). Inspecting the difference between subjects (across all countries), the lowest representations of female words are in religious studies and the humanities. High-paying subjects such as science and math also have less than equal representation (evidence shows relatively high labor market returns to studying math and science in India [64], the United States [65, 66] and Denmark [67].) The subject with the highest female representation is home economics. The representation of women and girls decreases with the length of the book, and there are no statistically significant differences in female representation across different grades. Among low- and middle-income countries, those with books funded by international aid projects have a higher, but still uneven female representation. All of this analysis is conducted with gendered nouns and pronouns and not with gendered names, due to the imprecision in the algorithm identifying names correctly in context. However the results are largely similar if we do include names—with the number of male words increasing to 232,502 and female words to 128,970. Representation by country only changes marginally (see S1 Fig). The share of overall variation explained by country and subject is around 8 percent each.
Note: These figures show the predicted mean share of gendered words that are female. This measure is first calculated for each individual book, which are then estimated as a function of country, subject, grade, and (log) book length. We exclude countries with fewer than five books in our corpus. The high-income countries are Australia, the UK, and the United States. The low and middle income countries with donor-funded books are Bhutan, Ethiopia, Guyana, Kenya, Lesotho, Liberia, Namibia, Nigeria, Papua New Guinea, Samoa, Sierra Leone, South Africa, South Sudan, Tonga, Zambia, and Zimbabwe.
In general, we see a slightly more equal representation of young girls than adult women. Of the gendered words with a clear age classification (e.g., girl vs. woman and boy vs. man), female words make up 40 percent of child words, but only 36 percent of adult words (this is similar to the pattern observed in the United States [30]). This gap is largest and only statistically significant for the sub-Saharan African countries (Fig 3).
Note: These figures show the mean share of gendered words that are female. This measure is first calculated for each individual book.
Gender stereotypes
How women and girls are described.
To analyse stereotypes, we begin by identifying the adjectives and nouns used to describe women and girls in our corpus, using dependency parsing. Across our entire corpus our model identifies a total of 33,670 sentences containing an adjective and a gendered word, 202,287 sentences with a verb and a gendered word, and 164,626 sentences with a noun and a gendered word. In line with the general over-representation of male gendered words, we also see more co-occurrences of male words with specific adjectives and verbs. Dependency parsing shows no meaningful difference in the proportion of male and female words that appear as the subject rather than object of the sentence: 70.9 percent of female words and 70.3 percent of male words appear as the subject.
Focusing on words which occur relatively frequently in our corpus (at least five times each in the UK/US/AUS, South Asia, and Sub-Saharan Africa regions), we show the share of mentions alongside female terms as a proportion of all mentions alongside any gendered term. The adjectives most likely to describe female and not male characters include “married”, “beautiful”, “aged”, and “quiet”, and verbs include “bake”, “cook”, “shoe”, and “sang”. The adjectives mostly likely to describe male and not female characters include “powerful”, “rich”, “wise”, “certain”, and “rich”, although also “unable” (see S2 Fig), and verbs include “rule”, “guide”, “sign”, and “order” (S3 Fig).
Exploring regional differences, we show these shares separately for the UK/US/AUS, South Asia, and Sub-Saharan Africa regions. In South Asia and Sub-Saharan Africa, “powerful” and “wise” have an even higher likelihood of being used to describe male characters compared to UK/US/AUS, although “rich” and “certain” are slightly less skewed toward male characters (Fig 4). Similarly, “aged” and “quiet” have a higher likelihood of being used to describe female characters in South Asia and Sub-Saharan Africa, although “married” and “beautiful” are slightly less skewed towards female characters. Exploring differences in verbs, “cook” is much less likely to be used to describe female characters in UK/US/AUS compared to South Asia and Sub-Saharan Africa (Fig 5).
Note: This figure shows the share of occurrences of adjectives used to describe people, which were used for female terms or nouns. We show here the 12 adjectives which were the most skewed toward either gender across UK/US/AUS, South Asia, and Sub-Saharan Africa, provided that they occurred at least 5 times in textbooks for each region. Shares are shown separately for each region.
Note: This figure shows the share of occurrences of verbs used to describe people, which were used for female terms or nouns. We show here the 12 verbs—to be precise, the lemmas of verbs—which were the most skewed toward either gender across UK/US/AUS, South Asia, and Sub-Saharan Africa, provided that they occurred at least 5 times in textbooks for each region. Shares are shown separately for each region.
In order to quantify the differences between words associated with male and female characters, we merge our annotated data with a set of quantified scores for the ‘Valence’, ‘Arousal’, and ‘Dominance’ of words (these roughly describe the positivity, activity, and power associated with a word, respectively). In general, we find that female characters are associated with more positive words (valence, +0.07 standard deviations), and male characters with more active (arousal, +0.04 standard deviation difference) and dominant words (+0.08 standard deviations). These averages mask substantial heterogeneity across countries (S5 Fig). Nigeria stands out as having particularly large differences (in favour of men) in all three scores.
Association of gendered words with stereotypes around achievement, appearance, work, and family.
In general, we find that, in line with traditional stereotypes, female words are more likely to be associated with home- or family-related words and with appearance-related words. Male words are more likely to be associated with work- and achievement-related words. The words with the strongest male bias are “leader”, “authority”, and “powerful”. The words with the strongest female bias are “wedding”, “slim”, and “cousins” (Fig 6). Almost all of the individual achievement and work-themed words have a stronger association with male words than female words. All individual appearance and home-themed words have a stronger association with female words than male words.
Note: This figure shows how terms related to four themes are associated with gender terms in our embeddings. Female bias is calculated as the difference between the average cosine similarity of the theme word with the set of female gender terms and the average similarity of the theme word with the set of male gender terms. The confidence intervals are calculated as the standard deviation for this statistic, over 50 bootstrap samples, where samples are generated by sampling all sentences in our corpus with replacement.
To assess heterogeneity, we consider the differences between regions, and between country income groups (we do not look individually at all countries due to advice that word embeddings models are run only on a corpus with at least 1.5 million ‘tokens’ or words [34]. Only ten countries in our sample have this many tokens.) South Asia shows a stronger male bias for terms associated with ‘achievement’, and both South Asia and Sub-Saharan Africa show a stronger female-bias for ‘appearance’ terms, than UK/US/AUS (Fig 7). Sub-Saharan African also shows a stronger female bias for terms relating to the ‘home’, although the region shows less male bias towards ‘work’ terms, compared to UK/US/AUS and especially South Asia.
Note: Female bias is calculated as the difference between the average cosine similarity of the theme word with the set of female gender terms and the average similarity of the theme word with the set of male gender terms. Confidence intervals are calculated as the standard deviation for this statistic, over 50 bootstrap samples, where samples are generated by sampling all sentences in our corpus with replacement, until a sample has as many sentences as the original corpus.
We pool together countries into four income groupings following the World Bank classification; low-income, lower-middle income, upper-middle income, and high-income. Overall we see similar levels of pro-male bias on average in similarity to achievement-themed and work-themed words (S6 Fig). This pro-male bias is smaller though in the upper-middle income countries in our sample (Belize, Dominica, Jamaica, Namibia, South Africa, and Tonga). All country groups have a pro-female bias in similarity with appearance- and home-themed words. This pro-female bias is largest in the books from the low-income countries in our sample (Afghanistan, Ethiopia, Rwanda, South Sudan, Liberia, Sierra Leone, and Uganda).
In S7 Fig, we also show embedding biases for the ten countries in our sample with more than 1.5 million tokens (recommended as a minimum corpus size for estimating reliable embeddings [34]). Patterns are difficult to discern, however, likely due to the still relatively small size of the corpora.
Gendered occupations.
Overall, a clear majority of job occupations co-occur with male rather than female nouns and pronouns. Of the 20 most frequently mentioned occupations, all but ‘domestic help’ and ‘nurse’ co-occur more with male than female nouns and pronouns (S9 Fig). In relative terms, among occupations which appear at least ten times across our corpus, those that occur most frequently alongside female words include ‘nurse’, ‘domestic helper’, and ‘religious official’ (see S10 Fig). The most male-dominated occupations include ‘physicist’, ‘mathematician’, and ‘salesperson’.
Across country groups, we see a particularly high share of male-bias in reference to common occupations in South Asia and Sub-Saharan Africa (Fig 8). In the high-income countries (Australia, UK, and US), 6 of the most common 15 occupations have a female-bias or approximately even gender split. By contrast 14 of the most common 15 occupations have a male-bias in South Asia. Amongst these most common occupations, those with the highest share of female mentions include ‘domestic help’, ‘teacher’, and ‘student’.
Note: This figure lists the 15 most common occupations in our full text corpus. The x-axis shows the share of all co-occurrences of the occupation and a gendered word which are female. Shares are shown separately for South Asia, Sub-Saharan Africa, and UK/US/USA.
Amongst those occupations that do co-occur with female nouns and pronouns, they are slightly more likely to be managerial and professional roles, and less likely to be manual jobs, compared to those co-occurring with male nouns and pronouns (S11 and S12 Figs). This is not the case for South Asia, however (Fig 9). In general, occupations in UK/US/AUS are more likely to be managerial and professional roles, compared to other regions (Fig 9).
Note: This figure shows the distribution of occupation categories across co-occurrences with gendered words. That is, for all co-occurrences between a male gendered word and an occupation for textbooks from South Asia, 66 percent of those co-occurrences were with managerial or professional occupations (category 1). 15 percent with service (category 2) occupations, and 18.9 percent with manual (category 3) occupations.
Country gender laws, norms, and textbooks
To what extent does gender bias in textbooks reflect gender-related social norms? We find a strong correlation between our measure of gender representation in textbooks and other measures of gender bias at the country level. A 1 standard deviation increase in the SIGI index, WBL index, or GEPI index is associated with a 5–6 percentage point increase in the share of gendered words in books that are female. A 1 standard deviation increase in the gender parity in secondary schooling index, or in the share of female members of parliament, is associated with a 2 percentage point increase in the share of female words in books. Increase in rates of girl marriage are associated with lower shares of female words in books. All of these correlations are estimated in regressions in which the outcome is the share of gendered words in a specific textbook that are female, estimated as a function of country-level attitudes, outcomes, and laws, whilst controlling for (log) GDP per capita, and characteristics of the textbook itself (its grade, subject, and word length) (Table 3).
Discussion
How much does bias in textbooks really matter?
Written materials, including textbooks, curricula, and exams, play a major role in schooling. Thus, how textbooks show girls and boys, women and men, in various different roles, and the portrayal of males and females might influence how children and adolescents view gender roles in society. But how important are they?
The main theory of gender development posits that environmental factors interact with three major social modes of influence; (i) modeling by people in one’s immediate environment and through mass media, (ii) through social reactions to one’s own behavior, and (iii) through direct tuition. All of these factors are expected to interact in complex ways, but with the largest influence coming from modeling rather than direct tuition [68]. We also know that children’s beliefs about gender stereotypes start young. One study showed that boys and girls as young as six years old thought that boys were more likely to be ‘brilliant’ than girls [69].
Evidence on the causal impact of bias in textbooks on children is limited. One exception is a study from Bangladesh that showed that providing children with gender-sensitive stories made them more likely to agree that boys and girls should do the same things when they grow up, both do chores, and both play football [70]. Small-scale lab experiments in the United States have shown that children express a preference for same-gender characters in books, and their behavior is more influenced by the behaviour of same-gender characters in storybooks [71, 72]. Children taught with gender-stereotyped books are more likely to express gender-stereotyped views on appropriate activities, personality characteristics, and career roles [73, 74]. Experimental evidence also shows that gender stereotypes can be reduced by exposure to counterexamples [75, 76].
A growing literature in economics documents the importance of role models and aspirations in determining schooling and employment outcomes, which can plausibly be provided by fictional characters as much as in-person interactions [77, 78]. Experimental evidence from Australia found that showing books with role models of the same sex to grade 3 children increased self-esteem, both for boys and girls [78]. This notion is also supported by Kenyan survey data showing that children were more likely to empathize with characters in books of their own gender [79].
Outside of the classroom, gendered language in school exams has been shown to lead to poorer performance for girls [80]. Countries with gendered national languages have worse employment and education outcomes for women [81].
Conclusion
In this paper, we gather a new corpus of 1,255 textbooks from 34 countries, and document under-representation of female characters and gendered stereotypes. Textbooks from low- and lower-middle-income countries show a greater degree of bias than upper-middle- and high-income countries. Lower female representation in books correlates with other measures of gender equality.
There are several limitations to our analysis. First, we focus only on text and not images. Advances in computer image recognition allow for the identification of different characters in pictures, though this is beyond the scope of our paper. Second, we focus exclusively on English language books, though our approach is in principle compatible with other languages. Third, while we manually validate the automated processes we use where possible, several of our analytical methods entail some imprecision in measuring bias. For example, some ‘names’ identified may not be names in context, and the co-occurrence of a gender term with an occupation term will not always indicate they are connected. Fourth, we are not able to distinguish sections of the text that are intended to be read by students from sections that are not, such as information about the book publisher. Fifth, while we recognize that gender can be expressed and perceived in a variety of ways, in this analysis we use a binary view of gender. Part of our reasoning for this more narrow view lies in the books included in our corpus, which only include binary gendered characters. We therefore use the same view in our analysis, though the absence of non-binary characters and adoption of a binary view of gender does reveal something about the norms around gender that are perpetuated by the countries included in our analysis.
Our analysis highlights which countries and subjects have particularly low female representation. This presents a new front in the agenda for girls’ education policy. Several countries in our database have achieved gender parity in school enrollment, but still see significant underrepresentation of women and girls in school materials. Paying particular attention to gender representation could be prioritized in future revisions of these books. A constraint to such reform is the likelihood that reform is most needed in countries in which it is least likely due to regressive gender norms amongst adults. This may be addressed in part in countries in which government textbooks are partly funded by donors.
In addition to highlighting the scope for revision of existing books, our analysis can also be easily replicated with new books, and so could be used as a proactive check for new drafts.
Our analysis was limited by the public availability of books, even when they were publicly funded by governments with policies in place mandating open access to materials. Donors and national governments should ensure that all publicly funded school books are easily available online in digital copy. Digital books are public goods, which should be freely available. Book availability to households is particularly important in the context of increasingly frequent school closures due to climate-related shocks, as well as shocks related to conflict and disease outbreaks. The opening of schoolbooks to researchers is just an added benefit.
Future research could easily expand our analysis to more datasets. An advantage of the quantitative approach is that our analysis is easily scalable and can be extended to new corpora of digitized books as they become available. New analyses of image data would also be valuable. As our analysis is largely descriptive, new experimental research would be useful in documenting the impact of gender bias in books on student attitudes. Replacing textbooks with more progressive ones may not be enough to change attitudes if teachers do not engage with them [82, 83]. But while debiasing school textbooks may not be sufficient to change attitudes, the clear bias demonstrated by our analysis suggests that it may be an important first step.
Supporting information
S1 Fig. Bias in gender representation, by country, subject, and income group (including names).
Note: These figures show the predicted mean share of gendered words that are female. Here, word counts for each gender include names identified as male or female. This measure is first calculated for each individual book, which are then estimated as a function of country, subject, grade, and (log) book length. We exclude countries with fewer than five books in our corpus. The high-income countries are Australia, the United Kingdom, and the United States. The low and middle income countries with donor-funded books are Bhutan, Ethiopia, Guyana, Kenya, Lesotho, Liberia, Namibia, Nigeria, Papua New Guinea, Samoa, Sierra Leone, South Africa, South Sudan, Tonga, Zambia, and Zimbabwe.
https://doi.org/10.1371/journal.pone.0310366.s001
(ZIP)
S2 Fig. Adjectives most likely to be used for each gender (full corpus).
Note: This figure shows the share of occurrences of adjectives used to describe people, which were used for female terms or nouns. We show here the 12 adjectives which were the most skewed toward either gender across UK/US/AUS, South Asia, and Sub-Saharan Africa, provided that they occurred at least 5 times in textbooks for each region.
https://doi.org/10.1371/journal.pone.0310366.s002
(TIF)
S3 Fig. Verbs most likely to be used for each gender (full corpus).
Note: This figure shows the share of occurrences of verbs used to describe people, which were used for female terms or nouns. We show here the 12 verbs—to be precise, the lemmas of verbs—which were the most skewed toward either gender across UK/US/AUS, South Asia, and Sub-Saharan Africa, provided that they occurred at least 5 times in textbooks for each region.
https://doi.org/10.1371/journal.pone.0310366.s003
(TIF)
S4 Fig. Verbs, adjectives, and nouns used alongside gendered terms (dependency parsing).
Note: These figures show the counts of the number of occurrences of gendered words alongside different specific adjectives, verbs, and nouns. The gendered words used are those contained in S1 File and the names described in Section. We identify adjectives, verbs, and nouns using the dependency parsing approach described in Section.
https://doi.org/10.1371/journal.pone.0310366.s004
(TIF)
S5 Fig. Valence, Arousal, and Dominance scores for adjectives and verbs used alongside gendered terms, by country.
Note: These figures show the differences in average scores (in Valence, Arousal, and Dominance) for adjectives and verbs used for female compared to male gender terms. The difference is shown by country.
https://doi.org/10.1371/journal.pone.0310366.s005
(TIF)
S6 Fig. Gendered word associations, by country income group.
Note: Female bias is calculated as the difference between the average cosine similarity of the theme word with the set of female gender terms and the average similarity of the theme word with the set of male gender terms. Confidence intervals are calculated as the standard deviation for this statistic, over 50 bootstrap samples, where samples are generated by sampling all sentences in our corpus with replacement, until a sample has as many sentences as the original corpus. H indicates high-income countries, which are Australia, Canada, the United Kingdom, the United States, Guyana, and St Kitts and Nevis. UM are upper-middle-income countries which are Belize, Dominica, Jamaica, Namibia, South Africa, and Tonga. LM are lower-middle-income countries, which are Bangladesh, India, Pakistan, Kenya, Nigeria, Zambia, Zimbabwe, Bhutan, Kiribati, Papua New Guinea, Samoa, Solomon Islands, Sri Lanka, and Lesotho. L are low-income countries, which are Afghanistan, Ethiopia, Rwanda, South Sudan, Liberia, Sierra Leone, and Uganda.
https://doi.org/10.1371/journal.pone.0310366.s006
(TIF)
S7 Fig. Gendered word associations, for countries with at least 1.5 million tokens.
Note: Biases are shown only for the 10 countries in our corpus with at least 1.5 million tokens, as embeddings generated for other countries will be estimated unreliably. Female bias is calculated as the difference between the average cosine similarity of the theme word with the set of female gender terms, and the average similarity of the theme word with the set of male gender terms. Confidence intervals are calculated as the standard deviation for this statistic, over 50 bootstrap samples, where samples are generated by sampling all sentences in our corpus with replacement, until a sample has as many sentences as the original corpus.
https://doi.org/10.1371/journal.pone.0310366.s007
(TIF)
S8 Fig. Gendered word associations.
Note: Words above the 45° line are discussed more frequently in relation to women and girls, words below the line more frequently in relation to men and boys.
https://doi.org/10.1371/journal.pone.0310366.s008
(TIF)
S9 Fig. Co-occurrence of job occupations with gendered terms (percent female) (full corpus).
Note: This figure lists the 15 most common occupations in our full text corpus. The x-axis shows the share of all co-occurrences of the occupation and a gendered word which are female.
https://doi.org/10.1371/journal.pone.0310366.s009
(TIF)
S10 Fig. Co-occurrence of job occupations with gendered terms (percent female), most biased jobs.
Note: This figure shows common occupation terms—classified as those with more than 10 occurrences across the corpus—which were most biased towards either gender, in the sense of co-occurring (within sentences) more frequently with terms of one than the other.
https://doi.org/10.1371/journal.pone.0310366.s010
(TIF)
S11 Fig. Co-occurrence of job occupations with gendered terms, by job type (percent).
Note: This figure shows the distribution of occupation categories across co-occurrences with gendered words. That is, for all co-occurrences between a male gendered word and an occupation, 64.8 percent of those co-occurrences were with managerial or professional occupations (category 1). 13.7 percent with service (category 2) occupations, and 21.4 percent with manual (category 3) occupations.
https://doi.org/10.1371/journal.pone.0310366.s011
(TIF)
S12 Fig. Co-occurrence of job occupations with gendered terms, by job type (absolute counts).
Note: This figure shows the distribution of occupation categories across co-occurrences with gendered words. In this case, absolute values are shown. For example, there were 3204 co-occurrences between male gendered words with a managerial or professional (category 1) occupation.
https://doi.org/10.1371/journal.pone.0310366.s012
(TIF)
S13 Fig. Co-occurrence of job occupations with gendered terms, by job type and country.
Note: This shows the results described above for the nine countries with over 250 occupation term-gender term cooccurences.
https://doi.org/10.1371/journal.pone.0310366.s013
(TIF)
S14 Fig. Gender representation by country, and country-level indicators of gender equality.
Note: This figure shows correlations between gendered word representation and country level indicators of gender equality. SIGI is the OECD Social Institutions & Gender Index. WBL is the WB Women, Business, & the Law Index. GEPI is CGD’s Girl’s Education Policy Index. Secondary Education Parity is from UNESCO. Girl marriage and female MPs are from the OECD Gender, Institutions, & Development database.
https://doi.org/10.1371/journal.pone.0310366.s014
(TIF)
S15 Fig. Gender representation by country, and country-level indicators of gender equality.
Note: This figure shows correlations between gendered word representation and country level indicators of gender equality. Support for wife-beating is from the Demographic & Health Survey. Female labour force participation is from the WB World Development Indicators. Support for basic rights and attitudes to women working are from the Gallup World Poll.
https://doi.org/10.1371/journal.pone.0310366.s015
(TIF)
S1 File. Gendered word lists.
A list of all male and female gendered nouns and pronouns for which we search.
https://doi.org/10.1371/journal.pone.0310366.s016
(DOCX)
S2 File. Occupation job type word list.
A list of all occupations for which we search.
https://doi.org/10.1371/journal.pone.0310366.s017
(DOCX)
S3 File. Theme word lists.
A list of thematic words related to ‘home’, ‘achievement’, ‘appearance’, and ‘work’.
https://doi.org/10.1371/journal.pone.0310366.s018
(DOCX)
S4 File. Sample textbook page from Sierra Leone, grade 10 English.
Note: In this sample page from a textbook we can see an uneven representation of male and female nouns and pronouns—there are 6 male themed words (he/his) and 4 female themed words (ladies/she/her/ladys).
https://doi.org/10.1371/journal.pone.0310366.s019
(TIF)
S5 File. Sample textbook page from Pakistan, grade 5 Social Studies.
Note: In this sample page we see the Prime Minister being discussed as He/him/his. The current Prime Minister does happen to be a man, but he is not named in this book, and the role is clearly not inherently gendered.
https://doi.org/10.1371/journal.pone.0310366.s020
(TIF)
S6 File. Sample textbook page from Pakistan, grade 7 Home Economics.
Note: This book discusses how “you can help your mother” with housework.
https://doi.org/10.1371/journal.pone.0310366.s021
(TIF)
S7 File. Example sentences of gender stereotypes.
https://doi.org/10.1371/journal.pone.0310366.s022
(TIFF)
S1 Table. Gender equality in countries in our sample vs rest of the world.
https://doi.org/10.1371/journal.pone.0310366.s023
(DOCX)
S2 Table. Publishing information for textbooks from each country.
Note: This table shows the sources and publishing information for the textbooks included in our corpus.
https://doi.org/10.1371/journal.pone.0310366.s024
(DOCX)
Acknowledgments
We are grateful for research assistance to Laura Moscoviz, Emma Cameron, and Theo Mitchell, for helpful comments and advice to Anjali Adukia, Susannah Hares, Alice Evans, Margaret Leighton, David Evans, Asyia Kazmi, Izzy Boggild-Jones, Shubha Jayaram, and two anonymous referees, and for help in sourcing textbooks to Ezioma Akabike and Mo Olateju.
References
- 1. Cantoni D, Chen Y, Yang DY, Yuchtman N, Zhang YJ. Curriculum and ideology. J Polit Econ. 2017;125(2):338–392.
- 2. Cantoni D, Yuchtman N. The political economy of educational content and development: Lessons from history. J Dev Econ. 2013;104:233–244.
- 3. Chen WL, Lin MJ, Yang TT. Curriculum and national identity: Evidence from the 1997 curriculum reform in Taiwan. J Dev Econ. 2023.
- 4. Lee B. Educational content, exclusive national identity, and anti-immigrant attitudes. J Polit. 2023.
- 5. Miguel E. Tribe or nation? Nation building and public goods in Kenya versus Tanzania. World Polit. 2004;56(3):328–362.
- 6. Dhar D, Jain T, Jayachandran S. Reshaping adolescents’ gender attitudes: Evidence from a school-based experiment in India. Am Econ Rev. 2022;112(3):899–927.
- 7.
Mehmood S, Naseer S, Chen DL. Transmitting rights: Effective cooperation, inter-gender contact, and student achievement; 2022.
- 8. Kincheloe JL. Understanding the new right and its impact on education. Phi Delta Kappan Educ Found. 1983.
- 9. Witt SD. The influence of school texts on children’s gender role socialization. Curric Teach. 2001;16(1):25–43.
- 10.
Sadker DM, Zittleman K. Practical strategies for detecting and correcting gender bias in your classroom. In: Gender in the classroom: Foundations, skills, methods, and strategies across the curriculum. Mahwah, NJ, US: Lawrence Erlbaum Assoc Pub; 2007. p. 259–275.
- 11. Blumberg RL. The invisible obstacle to educational equality: Gender bias in textbooks. Prospects. 2008;38(3):345–361.
- 12.
Mustapha AS, Mills S. Gender representation in learning materials: International perspectives. Routledge; 2015.
- 13. Asadullah MN, Islam KMM, Wahhaj Z. Gender bias in Bangladeshi school textbooks: Not just a matter of politics or growing influence of Islamists. Rev Faith Int Aff. 2018;16(2):84–89.
- 14.
Brugeilles C, Cromer S. Analysing gender representations in school textbooks. English edition ed. Paris: Popul Dev Centre-Unit 196 Univ Paris Descartes-INED-IRD; 2009.
- 15. Ye W. The return of “One guideline, one textbook” policy: Moral education textbook and teacher interaction in China. Comp Educ Rev. 2022.
- 16. Bachore M, Semela T. Is gender-bias in textbooks spilled over from schools to universities? Evidence from Ethiopian university English language course textbooks. Cogent Educ. 2022;9(1):2152616.
- 17. Lee JFK, Collins P. Construction of gender: a comparison of Australian and Hong Kong English language textbooks. J Gend Stud. 2010;19(2):121–137.
- 18.
Bahman M. The sexist occupational representation in English textbooks of Iranian high schools | Issues in language studies. 2020.
- 19.
Bhattarai S. Gender mainstreaming on Nepal’s pedagogy from 1995—2020; 2020.
- 20. Mustapha AS. Gender biased representation in learning materials in Nigeria educational system. Multidiscip J Gend Stud. 2014;3(2):368–392.
- 21.
Mirza M. Gender analysis of school curriculum and text books; 2004. Islamabad: UNESCO; 2004.
- 22. Karama MJ. Gender bias in school mathematics textbooks from grade 1 to 12 in Palestine. J Int Womens Stud. 2020;21(1).
- 23. Islam KMM, Asadullah MN. Gender stereotypes and education: A comparative content analysis of Malaysian, Indonesian, Pakistani, and Bangladeshi school textbooks. PLoS ONE. 2018;13(1):e0190807. pmid:29351305
- 24. İncikabi L, Ulusoy F. Gender bias and stereotypes in Australian, Singaporean, and Turkish mathematics textbooks. Turk J Educ. 2019;8(4):298–317.
- 25. Barton A, Sakwa LN. The representation of gender in English textbooks in Uganda. Pedagog Cult Soc. 2012;20(2):173–190.
- 26. Russell SG, Namatende-Sakwa L, Lewinger S. Gender representation in East African history textbooks: The cases of Rwanda, Kenya, and Uganda. Feminist Formations. 2021;33(2):233–261.
- 27. Jimenez JD, Lerch J, Bromley P. Education for global citizenship and sustainable development in social science textbooks. Eur J Educ. 2017;52(4):460–476.
- 28. Buckner E, Russell SG. Portraying the global: Cross-national trends in textbooks’ portrayal of globalization and global citizenship. Int Stud Q. 2013.
- 29. Skinner N, Bromley P. Individual and collective social justice education: Comparing emphases on human rights and social movements in textbooks worldwide. Comp Educ Rev. 2019;63(4):502–528.
- 30. Adukia A, Eble A, Harrison E, Runesha HB, Szasz T. What we teach about race and gender: Representation in images and text of children’s books. Q J Econ. 2023;138(4):2225–2285.
- 31.
Adukia A, Chiril P, Christ C, Das A, Eble A, Harrison E, et al. Tales and tropes: Gender roles from word embeddings in a century of children’s books. 2022. Available from: https://www.ssrn.com/abstract=4221213.
- 32. Lewis M, Cooper Borkenhagen M, Converse E, Lupyan G, Seidenberg MS. What might books be teaching young children about gender? Psychol Sci. 2022;33(1):33–47. pmid:34939508
- 33. Lucy L, Demszky D, Bromley P, Jurafsky D. Content analysis of textbooks via natural language processing: Findings on gender, race, and ethnicity in Texas U.S. history textbooks. AERA Open. 2020;6(3):2332858420940312.
- 34. Ash E, Chen DL, Ornaghi A. Gender attitudes in the judiciary: Evidence from U.S. circuit courts. Am Econ J Appl Econ. 2024.
- 35.
Bolukbasi T, Chang KW, Zou JY, Saligrama V, Kalai AT. Man is to computer programmer as woman is to homemaker? Debiasing word embeddings. In: Advances in neural information processing systems. vol. 29. Curran Associates, Inc.; 2016. Available from: https://proceedings.neurips.cc/paper/2016/hash/a486cd07e4ac3d270571622f4f316ec5-Abstract.html.
- 36. Garg N, Schiebinger L, Jurafsky D, Zou J. Word embeddings quantify 100 years of gender and ethnic stereotypes. Proc Natl Acad Sci USA. 2018;115(16):E3635–E3644. pmid:29615513
- 37.
Singla S, Mukhopadhyay M. Gender and education: A critical analysis of school textbooks. 2022. Available from: https://papers.ssrn.com/abstract=4183488.
- 38. Boutyline A, Arseniev-Koehler A, Cornell D. School, studying, and smarts: Gender stereotypes and education across 80 years of American print media, 1930-2009. Soc Forces. 2023. https://doi.org/10.31235/osf.io/bukdg
- 39. Glewwe P, Kremer M, Moulin S. Many children left behind? Textbooks and test scores in Kenya. Am Econ J Appl Econ. 2009;1(1):112–135.
- 40. Kuecken M, Valfort MA. When do textbooks matter for achievement? Evidence from African primary schools. Econ Lett. 2013;119(3):311–315.
- 41. Piper B, Simmons Zuilkowski S, Dubeck M, Jepkemei E, King SJ. Identifying the essential ingredients to literacy and numeracy improvement: Teacher professional development and coaching, student textbooks, and structured teachers’ guides. World Dev. 2018;106:324–336.
- 42. Maruyama T, Kurosaki T. Developing textbooks to improve math learning in primary education: Empirical evidence from El Salvador. Econ Dev Cult Change. 2022. https://doi.org/10.1086/721768
- 43.
Fredriksen B, Brar S. Getting textbooks to every child in sub-Saharan Africa. World Bank; 2015.
- 44. Bietenbeck J, Irmert N, Sepahvand MH. Teacher quality and cross-country differences in learning in francophone Sub-Saharan Africa. Econ Educ Rev. 2023;96:102437.
- 45.
Ward M. PISA for development: Results in focus. Paris: OECD; 2018. Available from: https://www.oecd-ilibrary.org/education/pisa-for-development_c094b186-en.
- 46.
Awich M. SACMEQ IV international report. The Southern and Eastern Africa Consortium for Monitoring Educational Quality; 2021. Available from: http://www.sacmeq.org/sites/default/files/sacmeq/reports/sacmeq-iv/international-reports/sacmeq_iv_international_report.pdf.
- 47. Milligan LO, Tikly L, Williams T, Vianney JM, Uworwabayeho A. Textbook availability and use in Rwandan basic education: A mixed-methods study. Int J Educ Dev. 2017;54:1–7.
- 48. Stará J, Chvál M, Starý K. The role of textbooks in primary education. e-Pedagogium. 2017;17(4):60–69.
- 49.
Martin M, Minnich CA, Mullis IVS, Stanco GM, Arora A, Centurino VAS, et al. TIMSS 2011 encyclopedia: education policy and curriculum in mathematics and science. Chestnut Hill, MA: TIMSS & PIRLS International Study Center; 2012.
- 50.
World Bank. Education service delivery indicators. 2023. Available from: https://www.worldbank.org/en/programs/service-delivery-indicators/education/interactive-results.
- 51.
Stern J, Jukes M, DeStefano J, Mejia J, Carrol B, Jordan R, et al. Learning at scale: Final report. RTI International; 2023.
- 52.
Read T. Where have all the textbooks gone?: Toward sustainable provision of teaching and learning materials in sub-Saharan Africa. Washington, DC: World Bank; 2015.
- 53. Menendez DA. Damegender: Towards an international and free dataset about name, gender, and frequency. 2022.
- 54.
Kumar V, Bhotia TS, Kumar V, Chakraborty T. Identifying and mitigating gender bias in hyperbolic word embeddings; 2021. Available from: http://arxiv.org/abs/2109.13767.
- 55.
Sheridan J. Occupations; 2015. Available from: https://github.com/johnlsheridan/occupations.
- 56.
Crawfurd L, Hares S. Introducing the girl’s education policy index; 2020. Available from: https://www.cgdev.org/blog/introducing-girls-education-policy-index.
- 57. Hegghammer T. OCR with Tesseract, Amazon Textract, and Google Document AI: a benchmarking experiment. J Comput Soc Sci. 2022;5(1):861–882.
- 58.
Mikolov T, Sutskever I, Chen K, Corrado G, Dean J. Distributed representations of words and phrases and their compositionality; 2013. Available from: http://arxiv.org/abs/1310.4546.
- 59. Rodriguez PL, Spirling A. Word embeddings: What works, what doesn’t, and how to tell the difference for applied research. J Polit. 2021. https://doi.org/10.1086/715162
- 60. de Marneffe MC, Manning CD, Nivre J, Zeman D. Universal dependencies. Comput Linguist. 2021;47(2):255–308.
- 61.
Bies A, Mott J, Warner C, Kulick S. English web treebank; 2012. Available from: https://catalog.ldc.upenn.edu/LDC2012T13.
- 62.
Silveira N, Dozat T, Manning C, Schuster S, Chi E, Bauer J, et al. Universal dependencies English web treebank; 2023. Available from: https://universaldependencies.org/treebanks/en_ewt/index.html.
- 63.
Mohammad S. Obtaining reliable human ratings of valence, arousal, and dominance for 20,000 English words. In: Proceedings of the 56th annual meeting of the association for computational linguistics (volume 1: long papers). Melbourne, Australia: Association for Computational Linguistics; 2018. p. 174–184. Available from: http://aclweb.org/anthology/P18-1017.
- 64. Jain T, Mukhopadhyay A, Prakash N, Rakesh R. Science education and labor market outcomes in a developing economy. Econ Inq. 2022;60(2):741–763.
- 65. Goodman J. The labor of division: Returns to compulsory high school math coursework. J Labor Econ. 2019;37(4):1141–1182.
- 66. Levine PB, Zimmerman DJ. The benefit of additional high-school math and science classes for young men and women. J Bus Econ Stat. 1995;13(2):137–149.
- 67. Joensen JS, Nielsen HS. Is there a causal effect of high school math on labor market outcomes? J Hum Resour. 2009;44(1):171–198.
- 68. Bussey K, Bandura A. Social cognitive theory of gender development and differentiation. Psychol Rev. 1999;106(4):676–713. pmid:10560326
- 69. Bian L, Leslie SJ, Cimpian A. Gender stereotypes about intellectual ability emerge early and influence children’s interests. Science. 2017;355(6323):389–391. pmid:28126816
- 70.
Leighton M, Mucyo K. Bangladesh children’s book initiative endline report. Save the Children; 2019.
- 71. McArthur LZ, Eisen SV. Achievements of male and female storybook characters as determinants of achievement behavior by boys and girls. J Pers Soc Psychol. 1976;33(4):467–473.
- 72. Ashton E. Measures of play behavior: The influence of sex-role stereotyped children’s books. Sex Roles. 1983;9(1):43–47.
- 73. Karniol R, Gal-Disegni M. The impact of gender-fair versus gender-stereotyped basal readers on 1st-grade children’s gender stereotypes: A natural experiment. J Res Child Educ. 2009;23(4):411–420.
- 74. Peterson SB, Lach MA. Gender stereotypes in children’s books: their prevalence and influence on cognitive and affective development. Gend Educ. 1990;2(2):185–197.
- 75. Parish TS, Bryant WT. Reversing the effects of sexism in elementary school girls through the use of counterconditioning procedures. Classroom Interact Newslett. 1976;11(2):7–15.
- 76. Nhundu TJ. Mitigating gender-typed occupational preferences of Zimbabwean primary school children: The use of biographical sketches and portrayals of female role models. Sex Roles. 2007;56(9-10):639–649.
- 77.
Serra D. Role models in developing countries; 2022.
- 78. Ochman JM. The effects of nongender-role stereotyped, same-sex role models in storybooks on the self-esteem of children in grade three. Sex Roles. 1996;35(11-12):711–735.
- 79. Mburu DNP, Nyagah G. Effects of gender role portrayal in textbooks in Kenyan primary schools, on pupils academic aspirations. Probl Educ 21st Cent. 2012;47:100–109.
- 80.
Cohen A, Karelitz T, Kricheli-Katz T, Pumpian S, Regev T. Gender-neutral language in legislation and its impact on social attitudes; 2023. Available from: https://papers.ssrn.com/abstract=4421161.
- 81.
Jakiela P, Ozier OW. Gendered language. Center for Global Development Working Paper 500; 2020. Available from: https://papers.ssrn.com/abstract=3573296.
- 82. Kostas M. Textbooks, students, and teachers talk around gender: A new materialist approach to children’s agency. Teach Teacher Educ. 2023;125:104052.
- 83. Namatende-Sakwa L. Are progressive texts necessarily disruptive? Investigating teacher engagement with gendered textbooks in Ugandan classrooms. Teachers College Record. 2021;123(1):1–26.
