Collecting event articles.

We utilize Wikidata [45] to collect events that occurred between 2016 and 2020. Wikidata collects information about entities (e.g., people or events) in the form of “items” which contain “statements”. In the case of events, such statements describe, among others, time and country the event occurred in. As an example, for the “2020 Kabul University attack”, these values are, respectively, “Afghanistan” and “2 November 2020” (see https://w.wiki/6RLk).

Retrieving events from Wikidata. To build our list of events, we use the Wikidata API [48] to retrieve all items that have a “point in time” and “country” statement, making these the prime indicators for an item being an event (i.e., something important that happened at a specific time). We utilize SPARQL queries to retrieve 56877 event items from Wikidata including their Wikidata ID, name, country, and relevant dates (“point in time” as well as “start date” and “end date”, if existent). We set the actual event date to the value of the “start date” statement if that statement is available and use “point in time” otherwise. We ignore events that have multiple “country” or “point in time” statements on Wikidata. After initial investigation of our data, we remove events with January 1^st as the set event date, as often times these entries are either faulty or include year-long events as well as summaries of seasons by sports teams and leagues (e.g., “2016 Kyrgyzstan League”).

Measuring attention to event articles. For each Wikidata event item, we retrieve the corresponding URL to the Wikipedia article in English, Spanish, German, and Italian. We choose this list of Wikipedia editions as they contain two languages spoken widely across the globe (English and Spanish) and two languages that are more localized to their country of origin (German and Italian). This allows for analysis of how both “global” (spread out across multiple continents) and “local” (focused on a single region or country) communities respond to external events. Additionally, the authors are proficient across these four languages, which facilitates our understanding of the article contents for this work. After retrieving the URL for all Wikipedia articles and dropping those for which no article could be found, we utilize the Wikimedia API [49] to collect more metadata about events and their corresponding articles, for example article title, the Wikipedia categories it belongs to, and the first paragraph of the article. Futhermore, we define the attention an event article receives in a given Wikipedia language edition with the overall pageviews (or edits) the event article and its redirect pages [50] received within the first seven days after the event date, and collect both pageviews and edits using the Wikimedia API. As robustness checks, we also compare our retrieved amount of views and edits within seven days of an event with those within thirty days and find that they are highly correlated (Spearman’s ρ = 0.96 for views and ρ = 0.94 for edits). Additionally, we further compute the maximum daily page views (edits) in the first seven days after an event, which again are strongly correlated to the seven-day views (Spearman’s ρ = 0.98) and edits (Spearman’s ρ = 0.95). Therefore, we consider the views and edits an event article received within the first week after it occurred as our main measures of attention and coverage.

News features of Wikipedia events.

To investigate event article attention and coverage, we lend from existing news value taxonomies that have included factors such as whether an event brings good or bad news, contains references to powerful individuals or organizations, affects similar demographic groups as the target audience, or impacts many people [31]. We also consider findings from research about news in the English Wikipedia [35, 36] to conceive features relevant to our research questions. Overall, we devise nine event article features:

Event category. The category of the event article, which is either sports, politics (e.g., elections, peaceful protests, speeches), disasters (e.g., natural disasters, plane crashes, shootings), or culture (e.g., award shows).

Geography. The geographic region the event happened in (East Asia & Pacific, Europe & Central Asia, Latin America & Caribbean, Middle East & North Africa, North America, South Asia, or Sub-Saharan Africa), c.f. The World Bank [47] for mapping between country and geographic region. While this categorization does not differ much from the breakdown of countries by continent, it does provide a more accurate split that better captures regional affiliation. This models “Proximity” as it occurs in news value taxonomies.

Economy. This feature set models the concept of “Power Elite” as described in news value theory in terms of economic power of an event country.

• Gross Domestic Product per Capita (GDP_pc). The event country’s GDP_pc in dollar.
• Income Class. The event country’s income categorized via the gross national income per capita (low: < $1 046, lower mid: $1 046—$4 095, upper mid: $4 096—$12 695, high: > $12 695).

Magnitude. The number of people possibly affected by this event, approximated by the event country’s population.

Uniqueness. This feature set combines the concept of “Follow-Up” and “Uniqueness” of news value theory.

• Country Uniqueness. Event articles in the same country and Wikipedia in the last 30 days (log scale).
• Category Uniqueness. Event articles in the same event category and Wikipedia in the last 30 days (log scale).

Relevance. The relevance of the language edition in the event country as the median monthly views from the country to the language edition in the five months prior to the event. Due to Wikipedia’s global popularity [1], we employ this metric as a proxy for measuring digital literacy (and thus internet penetration) of a country. For example, we find that the number of internet users per country [51] strongly correlates with Relevance for all event article countries recorded in the English Wikipedia (Spearman’s ρ = 0.71). Additionally, internet penetration per capita is highly correlated with socio-economic indicators such as GDP_pc (Spearman’s ρ = 0.82). For Relevance as well as Prominence (below), we also compute metrics for three and seven months and find high correlations between using three, five, and seven months (Spearman’s ρ > 0.98 for all comparisons), so we stick with the five-month timespan for both metrics.

Prominence. The prominence of the event country in the language edition as the median daily views of the country’s main article in the five months prior to the event. This metric reflects the approximate importance of the event country to the general visitor population of a particular language edition, such as the general presence of the country in media consumed by the target audience.

To retrieve features derived from Wikipedia metrics (e.g., Relevance and Prominence), we utilize the Wikimedia REST API [49]. Additionally, we collect economic and geographic data from The World Bank [47]. Finally, for classification of events into event categories we first consider the articles’ categories in the corresponding Wikipedia editions and then match the articles’ Wikipedia categories as well as the articles’ titles to language-specific keyword lists (see data on GitHub). For articles containing multiple keywords, we select our event category with the most matches. In case no keyword matches were found (361 events), two of the authors manually labeled these remaining events and agreed on event categories for all but 51 events (e.g., Lunar or Solar eclipses), which were consequently removed from analysis. We further apply an unsupervised approach to detect possible topical article categories using word embeddings and K-means clustering. To this end, we use the Wikimedia API to retrieve articles’ first paragraphs and extract multi-lingual SentenceTransformers embeddings [52] for the first 50 words of each article. We then perform K-means clustering on these embeddings for 2 to 100 clusters and find the best-fitting cluster size with the highest silhouette score at 9. After extracting words with the highest TF-IDF scores for each cluster, we observe that clusters largely adhere to our initial labeling, as we find clusters congruent to our labels: (i) cultural events, festivals, and award shows (culture) (ii) natural disasters, plane crashes, terrorist attacks, and murders (disasters) (iii) elections in the English Wikipedia (politics) (iv) elections in other language editions (politics) (v) tournaments, championships, and Olympics in the English Wikipedia (sports) (vi) tournaments, championships, and Olympics in other language editions (sports) (vii) cycling and other non-motor races (sports) (viii) motor sports such as Formula 1 (sports), and (ix) tennis or other similar tournament sports (sports). Therefore, we stick to our initial categorization of articles into sports, politics, disasters, or culture for the event category feature.

After all processing steps, we match the initial list of 56877 Wikidata items to 17490 Wikipedia articles (7921 English, 3278 German, 2737 Italian, 3554 Spanish).

Model setup.

We train the XGBoost models for two machine learning tasks predicting (i) attention and (ii) coverage. Both tasks consist of two subtasks: A classification subtask to predict whether articles receive any attention (> 10 views) or coverage (> 0 edits) and a regression subtask to predict the level of either attention (views) or coverage (edits). For the regression, we use the logarithm of views and edits as the target variable to better depict the scale of attention, as, for example, the difference between a hundred or a thousand views is more meaningful than between ten or eleven thousand.

For both subtasks, we utilize the features defined in Section “News Value Features”, but only use GDP_pc instead of income class as the feature bearing economic information due to high correlation across all countries (Spearman’s ρ = 0.95). We also exclude geographic region because of its relation to GDP_pc (Kruskal-Wallis p < 0.01) and instead use region to group resulting SHAP values for visualization. Furthermore, we add a categorical variable to distinguish between separate language editions and perform one-hot encoding for all categorical features. Finally, we fit XGBoost models for both classification and regression tasks.

Regarding data, the classifiers use all 17490 event articles (7921 English, 3278 German, 2737 Italian, 3554 Spanish). The view regressors are trained on the 12712 articles (6670 English, 1664 German, 1573 Italian, 2805 Spanish) that received more than 10 views, while the edit regressors are trained on 10495 articles (5725 English, 1543 German, 1292 Italian, 1935 Spanish) that were edited at least once.

For all our experiments, we split the corresponding five-year dataset (2015–2020, 60 months) into three temporally sequential sets. We use training (2016–2018, 36 months) and validation sets (2019, 12 months) for fine-tuning and hyperparameter search (see GitHub) and evaluate the models using F1-score (classification) and mean squared error (regression). We select the best-performing model regarding the train-validation split and use a hold-out test set (2020, 12 months) to assess model performance. Finally, we retrain the best-performing model using the full corresponding dataset and compute SHAP values for all features.

Model performance.

The XGBoost classifier predicting whether an article receives >10 views (>0 edits) reaches an F1-score of 0.867 (0.726) on the test set. As robustness experiments, we fit logistic regressions, support vector classifiers, and Random Forest classifiers. We find that according to the performance on the test set XGBoost performs better or is comparable to the alternative models (95% bootstrapped confidence intervals).

Furthermore, the XGBoost regression models attain a mean squared error on the test set of 3.95 for views and 1.56 for edits. We also perform robustness checks for this task using negative binomial regression, support vector regression, and Random Forest regression. Even though negative binomial regression—as a generalization of linear regression to cardinal numbers—typically has an advantage over other methods because of its explanatory power, we find that XGBoost significantly outperforms the negative binomial (95% bootstrapped CI). We observe no difference in the test error between XGBoost and the other non-linear approaches.

Estimating effects on attention via SHAP values.

SHAP values are a model-agnostic measure of feature importance based on the game-theoretic principle of Shapley values [54]. In general, SHAP values measure the contribution of features to the individual predictions of a model computed through feature permutation. They enable interpretation of local explanations and marginal contribution of features, which distinguishes them from classical measures of effect size such as negative binomial or linear regression coefficients, which interpret only global effects. However, SHAP values can also combine local explanations to better understand global model structure. In our case, we are interested in specific effects of discrete data points or groups (i.e., events or event types). Therefore, the ability to combine local and global explanations as offered by SHAP values is a more principled approach to estimating effects than alternative effect size estimators, such as regression coefficients. Moreover, our robustness analysis showed that non-linear models outperform linear regression models, so we can not make conclusive statements based on linear models. Finally, SHAP values make no assumption about feature independence—another limitation of classic dependency metrics—and can estimate feature interaction effects.

In practice, SHAP values measure how feature values affect the model’s prediction, relative to a baseline. For binary classification, the baseline is the log odds of positive (true) samples and for regression, it is the average of the target variable. While we interpret values for continuous features as provided by SHAP, for each categorical feature we sum SHAP values of the binary features generated by one-hot encoding and obtain a single value to understand the total contribution of the original categorical feature. We also compute ACV Shapley values [55], which do not change our overall findings.

Although SHAP values do not assert causality, they help interpret black-box models and provide insights into the importance of many potential mediating features. While we can not control for all mediators and confounders, by carefully selecting features in light of news value literature, we test many mediators proposed by theory. In other words, by including socio-demographic (e.g., relevance and prominence) and event-based (e.g., uniqueness) mediators as a baseline [56], we estimate the excess attention and coverage associated with GDP_pc and geographic region.

Attention by category and income.

We characterize our 17490 event articles by language edition, category, income class, and whether they receive attention in the first week of the events’ occurrence. To reduce noise, we only consider event articles with more than ten views (a proxy of attention) or at least one edit (a proxy of coverage). We summarize the data statistics in Table 1, and provide p-values for statistical tests throughout the text. Please note that, in all section that follow, we will consistently report Bonferroni-corrected p-values based on a significance threshold α/n, where α = 0.05 and n represents the number of tests conducted.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Dataset statistics.

We show our dataset by Wikipedia language edition, category of event articles, and income class of the country the events happened in. H = High, UM = Upper mid, LM = Lower mid, L = Low.

https://doi.org/10.1371/journal.pone.0289325.t001

In our four language editions, we find that most articles are viewed within the first week of the event, while the number of edited articles is slightly smaller (cf. Table 1, columns “>10 Views” and “>0 Edits”). Specifically, event articles in the global English and Spanish Wikipedia editions are considerably more likely to be viewed or edited than the German or Italian editions (chi-square tests, p < 0.025). There are also differences in attention and coverage between language editions when partitioned into the four article categories: sports, politics, disasters, or culture (chi-square tests, p < 0.0125). For example, German has a relatively large number of sports articles, but less than half of them are viewed or edited as opposed to, e.g., English where more than 73% of sports articles are viewed or edited. Additionally, politics and disasters are less likely to be viewed or edited in Italian than German, Spanish, or English (chi-square tests, p < 0.0083). Furthermore, in all languages users are more likely to view or edit articles from countries in the upper-middle or high-income classes within the first week of an event’s occurrence, and fewer from low-income countries (cf. Table 1, columns for income class), although there is no significant difference for German (chi-square tests, p > 0.0125). In Spanish, although more articles about events in high-income countries exist, events in upper-middle income countries are more likely to be viewed or edited (chi-square tests, p < 0.025), as many Latin American countries are in this income class.

When focusing on disasters, much like the Vienna and Kabul attacks seen in Fig 1, lower and lower-middle income countries are more likely to have articles (chi-square tests, p < 0.0125) and these articles are often viewed or edited within one week. This may be due to heightened news coverage or the relevance of the event to our Wikipedias’ demographics. In contrast, the investigated Wikipedia editions rarely cover political or cultural events in lower or lower-middle income countries.

Altogether, we show that Wikipedia editions differ in the numbers of articles per category. The differences we find are less pronounced between global (English and Spanish) and local (German and Italian) language editions. Our event data indicates that the number of event articles is skewed towards countries of higher economic status.

Geography and attention to events.

In Fig 2 we show the number of articles, which demonstrates that the global English Wikipedia strongly focuses on Anglophone countries, as 30.3% of event articles relate to events in the USA or United Kingdom. In the Italian and German Wikipedia we observe increased numbers for events in Europe and the USA in the Italian and German Wikipedias (54.4% and 8.3% as well as 46.1% and 9.4% of event articles, resp.). Additionally, Italian exhibits significantly higher number of articles related to Brazil than other language editions (11.2% of event articles; Z-tests, p < 0.0167) mainly driven by the 2016 Summer Olympics in Rio de Janeiro. The Spanish edition, meanwhile, has many events from the US (14.3% of articles) and significantly more articles from Hispanophone countries, such as Latin America and the Caribbean or Spain (23.1% and 13.9% of all articles), than all other languages (Z-tests, p < 0.0083).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Number of event articles per country.

We visualize the number of articles about events in a country normalized by the highest country value for each language edition. The number of events is strongly skewed towards regions which contain the language edition’s native speakers and the USA in general. White areas span countries with no event articles.

https://doi.org/10.1371/journal.pone.0289325.g002

In contrast to the number of articles, we observe no clear regional focuses for the median article views by country (Fig 3). Most notably, for English, German, and Italian the countries with the highest relative median views are those with a smaller number of articles and are all located in Sub-Saharan Africa (Sudan, Ethiopia, and Gabon, respectively). However, in general, we find higher values for views to events in countries where the language of the Wikipedia edition is an official language compared to those where it is not (Mann-Whitney U tests, p < 0.0125). For example, in the Spanish Wikipedia we find the highest median views for Honduras.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Median views to event articles per country.

We show median views to articles about events in a country normalized by the highest country value for each language edition. Median views are more equally distributed and sometimes even high in countries with non-native speakers. White areas cover countries without event articles.

https://doi.org/10.1371/journal.pone.0289325.g003

We conclude that there is a clear surplus in the number of articles in countries relevant to the user demographics of the Wikipedia editions (either in proximity, cultural context, or language), and that we observe an increased level of attention to event articles in countries relevant to the Wikipedia language, although highest median attention is registered for countries relatively foreign to the language edition.

Effect of GDP_pc on whether articles receive any attention.

We first investigate how wealth might affect whether an event article receives any attention at all via our trained classification model. Fig 4 visualizes SHAP values for GDP_pc, demonstrating the impact of GDP_pc on the XGBoost classifier’s prediction for all events across separate geographic regions. We observe an upward linear trend for GDP_pc and its SHAP values, which indicates that events in richer countries are more likely to be viewed within one week of the event occurring. Notably, this positive correlation is stronger for the local German and Italian editions (Pearson’s r, r = 0.30 and r = 0.20, resp., p < 0.0125) and slightly lower for the global English and Spanish editions (Pearson’s r, r = 0.14 and r = 0.17, resp., p < 0.0125). According to our XGBoost classifier, the probability of events receiving at least some attention steadily increases with rising GDP_pc.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. SHAP values for GDP_pc for predicting whether events receive attention.

We show SHAP Values for GDP_pc for all articles by language and geographical region for our XGBoost classifier predicting whether articles receive more than ten views. For this classifier, SHAP values for GDP_pc (y-axis) measure by how much corresponding values for GDP_pc (x-axis) change the base prediction, i.e., the average log-odds of 0.36. We find a positive upward trend for SHAP values illustrated by linear regression lines, indicating that higher GDP_pc correlates with higher likelihood of events receiving any attention.

https://doi.org/10.1371/journal.pone.0289325.g004

Moreover, we can identify and characterize outliers in our prediction task through the local explanations provided by SHAP values. For example, in English we observe that a number of events in South Asia, which by closer investigation are Badminton tournaments in India, have the lowest SHAP values for GDP_pc out of all observed events (−0.23). Furthermore, the effect of GDP_pc on prediction outcome differs across events in Sub-Saharan Africa, as for example in German values for events in some lower-income countries in Africa (e.g., Somalia) are higher than for other richer countries in the region (e.g., South Africa). Additionally, lower values for events in certain high-income countries in Europe and Central Asia, observable for example in English or Italian, can be explained by these articles containing recaps of non-prominent sports events (e.g., bicycle races) that were created well after the event took place.

Effect of GDP_pc on the level of attention.

To analyze the potential influence of GDP_pc on the level of attention in views for the articles that received at least some attention (>10 views), we now consider the trained regression model. For this XGBoost regressor, we observe no clear visual pattern for SHAP values for GDP_pc across all language editions (Fig 5; see S1 Table for pairwise significance tests). Rather, we find an overall negative correlation between GDP_pc and its SHAP values for all four language editions (Pearson’s r: −0.55 for English, −0.60 for Spanish, −0.50 for German, −0.51 for Italian; p < 0.0125). Even though these findings suggest higher attention for lower-income regions, they may be counterintuitive in that only the most prolific events even surpass the threshold to receive attention or be covered on Wikipedia, while less consequential events are left out completely. While non-native editors might simply not show interest or be aware of such events, local editors may not have the time and resources to edit their native language Wikipedia [11], let alone work on larger language editions (e.g., English) to increase their representation in these vast knowledge bases. Consequently, certain regions in the Global South receive less attention from both non-native and native Wikipedia visitors in the corresponding language editions. This is one reason Wikipedia suffers from regional differences in user-generated information that favor regions in the Global North over regions with less economic power in the Global South [12]. These patterns are consistent with the findings previously discussed in Fig 2, which suggested that higher median attention to articles about events in a particular country might be inversely related to the amount of articles about events in that country.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. SHAP values for GDP_pc for predicting the level of attention.

We show SHAP Values for GDP_pc for all articles by language and geographical region for the XGBoost regressor predicting the number of views (logarithmic scale). For this regressor, SHAP values for GDP_pc (y-axis) measure by how much corresponding values for GDP_pc (x-axis) change the average prediction (i.e., average views in logarithmic scale of 7.721 for this regression task). We observe non-linearity in the relationship between SHAP values and GDP_pc, as illustrated by polynomial regression lines of third degree, suggesting that other factors besides GDP_pc influence prediction results.

https://doi.org/10.1371/journal.pone.0289325.g005

On a separate note, events in certain countries receive more or less attention than would be expected from their GDP_pc or region alone. For example, while we observe lower SHAP values for events in countries with lower GDP_pc (i.e., in Sub-Saharan Africa), especially for the English Wikipedia a considerate amount of events occurring in certain lower income countries, such as South Africa or Nigeria, exhibit larger positive SHAP values. Next, events in South Asia (e.g., India or Pakistan) receive more attention than most other events in all Wikipedia editions, a pattern that is again more profound in English and underscores this language edition’s global presence.GDP_pc SHAP values of events in Latin America and the Caribbean are relatively high in the Spanish Wikipedia, especially when compared to German and Italian. This is even more pronounced for the XGBoost model predicting coverage (see GitHub), suggesting that the Spanish Wikipedia focuses more on this region. Furthermore, events in richer countries in Europe and Central Asia generally do not have positive values associated with them, indicating that GDP_pc alone does not determine much about which events receive more attention, at least for this region. For example, local elections or sports events in Switzerland, such as competitions in the 2020 Winter Youth Olympics, generally receive very low attention, even though Switzerland’s GDP_pc is very high. In the case of North America, analyzing the effect of GDP_pc on predictions produced by our models indicates a split picture: On the one hand, certain events, such as motor races or mayoral elections receive less attention, even when located in high-income countries such as the USA. On the other hand, presidential elections and popular award shows such as the Oscars increase attention in all language editions. Multiple of these examples suggest that other factors, such as geographic proximity to a language edition’s main user demographic or article category have a stronger influence on the attention events receive. These findings are akin to social impact theory [57], in which social, cultural, or spatial proximity between the source and targets is one of the defining factors for the relevance of an event to an audience. Overall, while large sporting events or reoccurring local elections generate articles that receive at least some attention or coverage, they do however not receive as many views or edits as rarer high-attention events such as award shows or articles in regions where an edition’s language is widely prevalent.

Effects of event-specific features on level of attention.

Past research has suggested that certain event-specific factors, e.g., popular teams participating in a sports game [44], can influence attention patterns of an event. We thus now consider a model with features specific to one of our event categories: The effect of death toll on disasters in combination with the wealth of the event country.

Consequently, we fit an XGBoost regressor predicting views using only deaths and GDP_pc as independent variables (all variables in logarithmic scale). For this regressor, we compute SHAP values and find that attention (i.e., log(views)) increases strongly with higher GDP_pc and deaths (Pearson’s r of 0.81 and 0.93, resp.; see Fig 7). Overall, economic power amplifies the effect of event-specific features: the richer the country, the larger the effect of deaths on article attention. For example, even though over 160 people died in Mali during the 2019 Ogossagou massacre (https://w.wiki/Toq), SHAP values for deaths as well as SHAP interaction values for deaths and GDP_pc are lower (1.5 and −0.09, resp.) than for disasters in higher-income countries with fewer deaths. Finally, when considering just disaster articles this simple model using only deaths and GDP_pc performs similarly to the model utilizing the general features described in the previous experiments (mean squared error of 4.85 versus 5.32, no significant difference according to 95% bootstrapped CI). Although we understand that focusing only on a single category might be limiting, for the purpose of this work we use it as a demonstration that more sophisticated, hierarchical models might be necessary to fully explain and determine the exact attention certain events might attract. Given these results, future work should consider both general and event-specific features to predict the load in attention certain events bring to Wikipedia.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. SHAP values and SHAP interaction values for disaster articles.

We find that views to disaster articles increase for rising deaths and that this effect is stronger when GDP_pc is high. As we visualize higher GDP_pc with lighter (yellow) and lower values with darker (blue) colors, this finding is observable by the lighter points generally being positioned higher in both plots. Note that log1p(x) = log(x + 1).

https://doi.org/10.1371/journal.pone.0289325.g007