Do Different Map Types Support Map Reading Equally? Comparing Choropleth, Graduated Symbols, and Isoline Maps for Map Use Tasks

Abstract

It is acknowledged that various types of thematic maps emphasize different aspects of mapped phenomena and thus support different map users’ tasks. To provide empirical evidence, a user study with 366 participants was carried out comparing three map types showing the same input data. The aim of the study is to compare the effect of using choropleth, graduated symbols, and isoline maps to solve basic map user tasks. Three metrics were examined: two performance metrics (answer accuracy and time) and one subjective metric (difficulty). The results showed that the performance metrics differed between the analyzed map types, and better performances were recorded using the choropleth map. It was also proven that map users find the most commonly applied type of the map, choropleth map, as the easiest. In addition, the subjective metric matched the performance metrics. We conclude with the statement that the choropleth map can be a sufficient solution for solving various tasks. However, it should be remembered that making this type of map correctly may seem easy, but it is not. Moreover, we believe that the richness of thematic cartography should not be abandoned, and work should not be limited to one favorable map type only.

1. Introduction

A map is a useful tool for visualizing data and is used for a wide range of purposes. However, depending on the way a map is designed, it can be recommended for different types of tasks. In thematic cartography there are distinguished several map types, namely choropleth map, dot map, isoline, proportional symbols, or graduated symbols [1,2,3,4]. Each of these can show the same input quantitative data but use different visual variables [5], usually size, lightness, and hue. This results in emphasizing different aspects of mapped phenomena and therefore focusing users’ attention on different issues. MacEachren [6], when comparing map types, indicated that forms can vary from abrupt (e.g., proportional and graduated symbols, choropleth maps) to smooth (e.g., dot maps, heatmaps, isolines), and from continuous (e.g., choropleth maps, dasymetric, isoline) to discrete (e.g., proportional symbols, dot maps). These differences in the final forms of map types suggest that they may also differ in terms of supporting various map use tasks. In fact, many authors [1,3,4] emphasize that map type should be selected depending on how the final map is to be used. For example, a choropleth map is recommended for showing the overall geographical patterns of the mapped variable, whereas proportional and graduated symbols are considered to be useful for comparing values, especially in neighboring fields, but make it hard to see the general pattern and densities. Isolines are claimed to be a good choice for presenting the arrangement of the magnitude, as well as the steepness and orientation, of a surface gradient, and a dot map presents the geographic character of distribution more clearly than any other map type [1,2,3,4].

This in turn, leads to the conclusion that a map type selection should not be conducted at random, or should not be driven by custom or habit. It should be selected to best suit the task the map is aimed at. However, while formulating rules for map type selection according to the recommendations from handbooks, it is equally important to provide empirical evidence for those rules. As highlighted by Hegarty [7], empirically testing visual-display design principles is of high importance. Using the collected results, a broader education in visual literacy can be introduced.

In the study presented, we compared three map types, namely choropleth, graduated symbols, and isoline, to provide empirical evidence of how well they facilitate map use tasks of different kinds. We selected three commonly used map types that present quantitative data [8]. Because of the fact that all of them can present the same input data, they may be considered equal in terms of being of assistance when used for similar tasks. As mentioned above, according to many authors [1,3,4], different map types are recommended for different tasks and purposes; we thus wanted to provide empirical evidence for these claims. We also wanted to investigate how users’ opinions on the difficulty in solving tasks using different map types match other usability performance metrics, and the frequency of map type usage. We thus formulated three research questions:

• RQ1: Do different map types, presenting the same input data, facilitate users’ performance of various map use tasks equally?
• RQ2: Do users perceive more commonly applied map types as easier to use than map types encountered less frequently?
• RQ3: Does the subjective rating of the difficulty of tasks provided by users of different map types match the other performance metrics?

To answer the RQs we conducted a user study with 366 participants, high school students (see details in Section 3). We wanted to get an insight into the consequences of selecting different map types to present the same quantitative data.

2. Background

2.1. Informationally Equivalent Visualizations

The notion of informationally equivalent visualizations is taken from cartographic and geographic information science disciplines [9], yet its usefulness for map comparisons has been proven by, for example, Çöltekin et al. [10]. Two representations are understood as informationally equivalent if all the information in one is also inferable from the other, and vice versa [9,10]. This approach has been applied in relation to map interfaces [10]. Authors compared two informationally equivalent but differently designed interactive maps (namely Carto.net and Natlas) applying usability performance metrics and eye tracking. The collected results did not clearly favor any of the designs: participants solved the given tasks faster with the Carto.net interface than with Natlas, they also preferred the Carto.net tool. However, the Natlas interface resulted in more accurate answers.

This approach has been also applied using the same map types with different designs. Fabrikant et al. [11] compared not only the usability performance metrics but also eye movements of participants using static weather maps. The isoline weather maps tested displayed the same information, but they made either the task-relevant pressure information or the task-irrelevant temperature information salient. The results showed that improving visual hierarchy affected viewing behavior and response time. Studies of similar design were conducted with other types of materials, for example 2D and 3D pie charts [12], cartograms [13], maps of spatial accessibility [14], graduated symbols in relation to spatial distance between them [15] or flow maps [16]. All of these provided empirical evidence valuable for formulating the optimal solution for a particular map type, and guidance on how to refine a selected map type.

2.2. Comparing Map Types

In addition to the comparison of various, informationally equivalent options of the same map type, different map types have also been frequently compared. Depending on the purpose of the study, there are two possible solutions. One involves comparing map types that present different input data: for example, when evaluating students’ work with both quantitative and qualitative map types [8] and when comparing thematic map readings by students and their geography teacher [17].

However, in this approach, though it provides valuable results, the tested visualizations are not informationally equivalent. Maps that present the same input data but are designed quite differently, using different map types, were also compared. However, we believe that different map types can be informationally equivalent in relation to some of the tasks only, due to the fact that map types are not equal in terms of presenting all of the characteristics; for example, a choropleth map does not show variability within an enumeration area, unlike the dot maps. Therefore, we believe that the tasks that can be used to compare map types have to be selected carefully. This in turn provides support when selecting the most suitable map types in a given condition. For example, with regards to interactive maps, comparison of choropleth, dot density, proportional symbols, and isoline interactive maps, presenting the same data set at different scales, resulted in differences in the percentage of correct answers while completing elementary tasks [18,19]. The best results were obtained while using an isoline map, and the worst—a dot density map.

Empirical proof supporting different types of visualization selection (including map types) became even more relevant in the context of coordinated visualization tools (also called coordinated and multiple views, CMV). These tools integrate several visualization methods in separate but dynamically linked views and display data simultaneously in these views by means of interaction techniques [20]. As shown by Golebiowska et al. [21], when using CMV, participants choose to refer to different views depending on the task type. In order to select the most suitable visualization methods when designing CMV and other visualization tools (including maps), evaluations of different visualization types have been conducted. For instance, Koua et al. [22] compared accuracy of answers when using a choropleth map and other types of data visualization: parallel coordinate plot and self-organizing maps (SOM) of different kinds (SOM distance matrix representation, SOM 2D/3D surface, SOM component plans and SOM projection). The authors showed that the methods of data presentation tested differ in terms of performance for different kinds of tasks. For tasks involving visual attention and sequencing (locate, distinguish, rank), choropleth maps returned results with the best performance by participants, whereas for visual grouping and clustering, the SOM-based representations performed better than the parallel coordinate plot. In detailed exploration of the attributes of the data set, correlations, and relationships, the SOM was more effective than the map. Similarly, in the context of CMV, Edsall [23] compared the accuracy between users of parallel coordinate plot and scatter plot. The author proved that showing the same input data in various ways may result in different performance metrics.

Independent of direct motivation, comparing different map types are conducted with regard to the defined map tasks. As mentioned in the introduction, one map type can be a good support for one task, and at the same time it may fail when considering another task. According to many authors [1,2,3,4] isolines present magnitude more clearly. Choropleth maps serve well for pattern identification. However, for both of these tasks it is better not to use graduated or proportional symbols, which in turn are favorable for making comparisons. Therefore, map types should be carefully selected and the decision should be an informed one.

2.3. Importance of Subjective Metrics

Apart from the objective performance metrics—answer correctness and answer time—that are often applied in empirical user studies (and reported in all of the studies mentioned above), there is also an important subjective measure: participants’ preferences. Previous works on this issue suggest that users do not always prefer and choose the solutions and designs that best work for them [24,25]. In the study of Pickle et al. [26], the participants clearly preferred one of the choropleth map legend designs even though significant differences were not observed in the accuracy of their answers. Similarly, in another study of thematic map legend design [27], a clear preference for one of the solutions was noted, even though it did not match the time and accuracy of answers using the preferred solution. Moreover, it was suggested that users frequently justified the preference for one of the designs based on their familiarity with a given solution. This supports the opinion of Petchenik [28], who claimed that users prefer familiar and previously known solutions. The preference of participants for thematic maps that were not the most effective was tested with regard to experienced and naive users [7,29]. The authors showed that both inexperienced college students and experienced weather forecasters alike have a tendency to choose maps that are less efficient: more realistic and complex maps over less realistic and simple ones. Also when evaluating the choropleth maps, dot maps, and graduated symbols in the studies of Mendonça and Delazari [30,31], users preferred the choropleth map even though it turned out that they did not give better answers when using this map type. The same results for general tasks completed with choropleth were obtained by Roth et al. [18,19].

These kinds of characteristics are covered by many metrics: preference of choice [30,31], perceived level of difficulty [18,19,32,33], perceived level of confidence [18,19,34,35], rating of frustration/comfort when using a tested tool, and perceived speed of own performance [32].

To sum up, subjective ratings do not necessarily replicate the efficiency and effectiveness of work with a particular map. Therefore, when evaluating map types it is worth including subjective metrics.

3. User Study

The aim of this study is to fill the gap in the comparison of the effectiveness of map types for different analytical user tasks. To the best of our knowledge, systematic comparisons have not been conducted between informationally equivalent map types for a selected set of tasks.

We chose to cover the topic of quantitative mapping methods because, with the greater accessibility of geoportals, statistical data, and software for geoprocessing data, not only experts but also novice users often handle this kind of data. In the study, we analyze three commonly applied [10] usability performance metrics: effectiveness (correctness), efficiency (time), and the subjective rating of the task’s difficulty.

We formulated three hypotheses addressing the research questions presented in the introduction:

Hypothesis 1 (H1).

Users differ in terms of performance when using informationally equivalent but differently designed thematic maps for various tasks.

Hypothesis 2 (H2).

Map users perceive the most commonly applied map types as the easiest ones.

Hypothesis 3 (H3).

The subjective metric of difficulty mismatch the usability performance metrics of answer accuracy and answer time.

As recommended and justified by many authors, we believe that different map types emphasize various aspects of the phenomenon. Therefore map type has an impact on the effort a map user needs to solve different task types. Moreover, we expect that more commonly applied map types, especially popular in education and school atlases, may result from a higher level of users’ training and literacy. This in turn may result in a perceived lower level of difficulty than map types that are not often used. Finally, similar to the results of other authors, we believe that the applied subjective rating of difficulty does not necessarily match the objective metrics of answer time and accuracy.

3.1. Study Material

We decided to compare three quantitative map types: choropleth map (below abbreviated to CH), graduated symbols map (GS), and isoline map (IS), which are most often used in school cartography designed for teenagers [8]. To use these map types, we presented four sets of input data related to commerce. The map presented: (1) the number of supermarkets, (2) discount stores, (3) grocery stores, and (4) stalls. All these data were presented per 100,000 people.

In total 12 maps were created and served as the stimuli to be used when solving different map use tasks (Figure 1). For reproducibility of study, all maps are presented in Supplementary Materials (Figure S1).

Thematic data were obtained from Open Street Map [36]. Reference data, namely administrative units, were obtained from the official Polish State database [37], and data on population were taken from the Polish Local Data Bank [38].

3.2. Participants

In total 366 students from high schools located in 11 cities in Poland participated in the study, voluntarily. Study participants were aged between 15 and 20 years. The average age of the respondents was 18 years. 59% of the respondents were women and 41% were men. Most respondents used maps (paper and interactive) once every few months (27%) or once a month (26%). Only 57 people (16%) replied that they use maps more than once a week. Only 10% of the participants declared that they do not use maps.

3.3. Tasks and Procedures

Even complicated map use tasks can be broken into a series of simple steps that comprise analytical tasks. We used a compilation of objective-based taxonomies of user tasks, as presented by Roth [39]. We chose from this compilation only those tasks that could be conducted with a non-interactive static map. In total, participants solved 11 tasks. All but task 9 were closed-ended (

Table 1. Tasks and available answers (translated from Polish).

Group	Task Type	Task	Available Answers
1	T1 identify	Select the sentence that correctly describes area A.	a. It is located in the east of the presented area. b. It is one of the two areas with the lowest index value. c. There are over 6 discount stores per 100,000 people. d. There are 3 discount stores per 100,000 people.
	T2 find extremum	From among the areas labelled with letters, indicate the one with the smallest number of supermarkets per 100,000 people.	A B C D
	T3 distinguish	Select the area that does not match the others in terms of the index value.	A B C D
	T4 retrieve value	How many discount stores per 100,000 people are located in area A?	a. 0.6–1.0 b. 1.1–2.0 c. 4.1–6.0 d. 6.1–8.0
	T5 compare	Pick the sentence that correctly describes the areas where there are from 6.1 to 9.0 stalls per 100,000 people.	a. All these areas are north of the black line. b. More than half of these areas are south of the black line. c. All these areas are south of the black line. d. More than half of these areas lie to the north of the black line.
2	T6 interpret	Cities are marked with black squares on the map. Select the correct sentence.	a. Cities are located in the three areas where the index has the highest values. b. Areas with cities are adjacent only to areas with lower index values. c. In areas with cities, the index ranges from 12.1 to 14.0. d. All areas with cities are located in the east of the analyzed area.
	T7 categorize	In which of the marked areas are there 6 grocery stores per 100,000 people?	A B C D
	T8 cluster	From among the marked areas, select two with an index value included in the same class.	a. C and B b. B and D c. A and C d. A and B
	T9 sort	Order the areas from the lowest to the highest value of the index.	(open-ended)
	T10 correlate	Indicate the sentence that correctly describes the relationship between the selected areas.	a. In areas A and B the index belongs to other classes from the legend. b. Areas A and B are located in the central part of the analyzed area. c. In one of the areas, A or B, the index shows the lowest value. d. Areas A and B are located in the south-western part of the analyzed area.
	T11 locate	Identify the sentence that correctly describes the location of the areas where the index has the highest value.	a. They are located on the border of the analyzed area. b. They are adjacent to each other. c. They are located in the central part of the analyzed area. d. They are located in the south of the analyzed area.

). In four tasks (T2, T3, T7, T8), the respondents only had to indicate the correct area from those labelled on the map with the letters A, B, C or D. In five tasks (T1, T5, T6, T10, T11) participants were asked to choose a sentence that correctly describes a marked area. In one task (4), the respondents were expected to choose the value corresponding to the marked area. There was only one open-ended task (T9), in which the respondent had to sort enumeration units according to the index value.

In total, 11 different tasks were tested (see Table 1). However, since we wanted to avoid overloading a respondent with too many questions, we decided to divide participants into two groups (1: N = 184 and 2: N = 182). Each group was divided into three subgroups for each map type tested: choropleth, graduated symbols, isoline (

Table 2. Number of participants in groups.

Map Type	Group 1		Group 2
	Number of People	%	Number of People	%
choropleth map	61	33	64	35
graduated symbols map	61	33	57	31
isoline map	62	34	61	34
total	184	100.0	182	100.0

). Group 1 performed Tasks 1–5 and group 2 Tasks 6–11.

The study was conducted in Polish using a web application (Figure 2). Each of the respondents answered the questions individually.

The study began with an introduction explaining the purpose of the research (Figure 3). When starting the study, the application randomized the test. Each participant solved one of the six possible tests. The tests differed in the set of questions (Group 1 and Group 2) and map types (choropleth maps, graduated symbols map, isoline map). After displaying the question, it was possible to move to the next question only after selecting the answer and clicking “Next” (see Figure 2). After each task, the respondents rated the difficulty on a three-point scale (“easy,” “neither easy nor difficult,” “difficult”). In the end, the respondents completed a short questionnaire to identify their age, sex, and frequency of using maps.

3.4. Data Analysis

Data were statistically analyzed in SPSS software. A chi-square test, which allows the dependence between variables to be verified, was applied for correctness and difficulty metrics. Additionally, Cramér’s V was used to indicate the degree of association between the two variables. The time metrics did not data follow the normal distribution, therefore the Kruskal–Wallis test was performed.

4. Results

4.1. The Correctness of the Answer

The average accuracy of answers for all 11 tasks was 82%. The best results were obtained by users of the choropleth map (90%). In the group using the graduated symbols map, the share of correct answers was 81%, and in the case of the isoline map, 74%. The accuracy of the answers differed significantly between users of different map types: X² (2, N = 2379) = 67.026, p = 0.000, Cramér’s V = 0.168, p = 0.000. Pairwise comparisons showed that the differences in answer accuracy between each pair of the tested map types were statistically significant:

• CH-GS X² (1, N = 1579) = 25.791, p = 0.000, Cramér’s V = 0.128, p = 0.000 (with statistically significant better accuracy among choropleth map users by 9 percent points)
• CH-IS X² (1, N = 1611) = 67.634, p = 0.000, Cramér’s V = 0.205, p = 0.000 (with significantly better scores among choropleth map users by 16 percent points)
• GS-IS X² (1, N = 1568) = 10.110 p = 0.001, Cramér’s V = 0.080, p = 0.001 (with better results for participants working with the graduated symbols map by 7 percent points).

Accuracy of answers differed also between tasks. Three tasks were solved correctly by 95% or more of the respondents (Figure 4). The easiest tasks turned out to be: T4 retrieving value (97%), T2 identifying extreme values (96%), and placing objects into groups based on similar characteristics, namely T8 cluster (95%). The last of these tasks was correctly solved by the same percentage of respondents from each group. In the case of one task (T2 find extremum), the whole group using the choropleth map solved it correctly. The next highest percentage of correct answers occurred for T11 locate—81%. Five tasks resulted in between 70 and 80% correct answers: T6 interpret (77%), T9 sort (76%), T5 compare (75%), T1 identify (74%), T3 distinguish (74%). Only two tasks were solved correctly by less than 70% of the participants. In the case of T7 categorize this was 62%. The fewest correct answers were given in T10 correlate—61%.

When it comes to inferential analysis, statistical significance was found for 5 out of 11 tasks: T1 identify (p = 0.000), T3 distinguish (p = 0.000), T6 interpret (p = 0.001), T9 sort (p = 0.001), T10 correlate (p = 0.002). In each of these cases, the association between the mapping type and the correctness of the answers was moderate according to Cramér’s V (

Table 3. Inferential statistics for answer accuracy between participants using the three map types.

Task	Chi-Square	p	Cramér’s V	p	Pairwise Comparison	Chi-Square	p	Cramér’s V	p
T1 identify	16.628	0.000	0.301	0.000	CH-GS	0.957	0.328	0.089	0.328
					CH-IS	13.989	0.000	0.337	0.000
					GS-IS	8.095	0.004	0.257	0.004
T2 find extremum	5.685	0.058	0.176	0.058	-
T3 distinguish	97.855	0.000	0.729	0.000	CH-GS	1.034	0.309	0.092	0.309
					CH-IS	63.700	0.000	0.720	0.000
					GS-IS	58.814	0.000	0.680	0.000
T4 retrieve value	1.592	0.451	0.093	0.451	-
T5 compare	4.404	0.111	0.155	0.111	-
T6 interpret	14.124	0.001	0.279	0.001	CH-GS	4.109	0.043	0.184	0.043
					CH-IS	14.068	0.000	0.335	0.000
					GS-IS	3.085	0.079	0.162	0.079
T7 categorize	0.943	0.624	0.072	0.624	-
T8 cluster	0.021	0.989	0.011	0.989	-
T9 sort	13.168	0.001	0.269	0.001	CH-GS	12.272	0.000	0.318	0.000
					CH-IS	1.735	0.188	0.118	0.188
					GS-IS	5.038	0.025	0.207	0.025
T10 correlate	12.489	0.002	0.262	0.002	CH-GS	11.070	0.001	0.302	0.001
					CH-IS	8.081	0.004	0.254	0.004
					GS-IS	0.292	0.589	0.050	0.589
T11 locate	4.431	0.109	0.156	0.109	-

).

In tasks T1 identity and T3 distinguish, the highest number of correct answers were provided by participants using the choropleth map and graduated symbols map (T1 CH: 87%, GS: 80%; T3 CH: 98%, GS: 95%), and the percentage of correct answers from the group using the isoline map was much lower (T1 IS: 57%; T3 IS: 29%). In both tasks, in the case of pairwise comparisons, when the correctness of the answers were compared for the choropleth map and isoline map and the graduated symbols map and isoline map, the result of the statistical tests was significant; that is, the correctness of the answers and the map type used were moderately related according to Cramér’s V test (T1 CH-IS p = 0.000, GS-IS p = 0.004; T3 CH-IS p = 0.000, GS-IS p = 0.000).

In the interpreting task (T6), more participants using the choropleth map chose the correct answers (91%); slightly worse was the group using graduated symbols (77%), and the worst results were obtained by participants using the isoline map (62%). The dependence of the correctness of the answer on the map type was significant only when comparing the choropleth map to the other two map types (CH-GS p = 0.043, CH-IS p = 0.000). For the same pairs, significant results were obtained in the last task (T10 CH-GS p = 0.001, CH-IS p = 0.004), where about half of the participants in the groups using the graduated symbols map and the isoline map answered the question correctly (GS: 50%, IS 54%). In the group using the choropleth map, this was as high as 78% of participants.

The situation was different in the case of T9 sort, in which the group using the choropleth map again gave the greatest number of correct answers (88%). However, in this case, participants using the isoline map gave a higher number of correct answers (79%) than those using the graduated symbols map (60%). In this case, the pairwise comparison showed significant dependence between the correct answers and the map type when the graduated symbols map was included in the pair (CH-GS p = 0.000, GS-IS p = 0.025).

4.2. Answer Time

The lowest average time occurred in the group using the choropleth map (M = 26,1 s, SD = 0.62). The average result of the group using graduated symbols was slower (M = 28.0 s, SD = 0.61). On average, people using isolines needed the most time (M = 30.7 s, SD = 0.68). The groups differed significantly within this variable: H (2) = 40.253, p = 0.000. Post hoc comparisons indicated that all groups differed significantly from one another (CH-GS: p = 0.004, CH-IS: p = 0.000, GS-IS: p = 0.008).

The task that was answered the fastest on average was T5 retrieve value (M = 13.8 s, SD = 0.42). A bit more time was needed to complete T3 distinguish (M = 19.2 s, SD = 0.87). For five tasks the completion time ranged from 20 to 30 s: T2 find extremum (M = 20.5 s, SD = 0.87), T8 cluster (M = 24.5 s, SD = 0.94), T9 sort (M = 25.1 s, SD = 0.61), T11 locate (M = 26.2 s, SD = 0.86), T7 categorize (M = 29,1 s, SD = 0.86). For two tasks the mean completion time was nearly the same—T5 compare (M = 39.3 s, SD = 1.23), T10 correlate (M = 39.2 s, SD = 1.24). The mean completion time for task T1 identify was M = 47.0 s, SD = 1.54. The participants needed the most time to answer T6 interpret (M = 51.3 s, SD = 1.68).

For a detailed comparison of groups within each task, in most cases, the people using the isoline map needed more time to answer the question, and the group with the choropleth map the least time (Figure 5). Sometimes this difference in the average response time to a question was only one or two seconds (e.g., task T9 sort), but for some tasks, it was even 13 s (T6 interpret).

Significant differences in response time between the groups were found for 4 out of 11 tasks: T3 distinguish (p = 0.000), T4 retrieve value (p = 0.000), T6 interpret (p = 0.002), and T8 cluster (p = 0.007) (

Table 4. Inferential statistics for answer time between participants using the three map types.

Task	Kruskal–Wallis H	p	Map Type	M (s)	SD	Post Hoc Groups	Bonferroni Post Hoc	p
T1 identify	3.796	0.150	CH	45.5	2.95	-
			GS	44.7	2.25
			IS	50.8	2.71
T2 find extremum	1.361	0.506	CH	18.6	1.10	-
			GS	21.4	1.58
			IS	21.6	1.73
T3 distinguish	34.241	0.000	CH	14.6	1.36	CH-GS	35.754	0.001
			GS	18.9	1.28	CH-IS	55.485	0.000
			IS	23.9	1.63	GS-IS	19.731	0.120
T4 retrieve value	34.047	0.000	CH	10.9	0.42	CH-GS	33.115	0.002
			GS	13.7	0.61	CH-IS	55.764	0.000
			IS	16.6	0.86	GS-IS	22.631	0.055
T5 compare	4.194	0.123	CH	37.8	2.14	-
			GS	40.9	1.74
			IS	39.1	2.47
T6 interpret	12.512	0.002	CH	45.0	2.51	CH-GS	19.492	0.127
			GS	51.6	2.74	CH-IS	33.137	0.001
			IS	57.7	3.23	GS-IS	13.646	0.479
T7 categorize	0.804	0.669	CH	29.0	1.40	-
			GS	29.0	1.78
			IS	29.3	1.32
T8 cluster	9.975	0.007	CH	22.0	1.47	CH-GS	10.061	0.883
			GS	23.0	1.23	CH-IS	29.385	0.005
			IS	28.6	1.96	GS-IS	19.324	0.139
T9 sort	1.455	0.483	CH	24.5	1.09	-
			GS	25.7	1.23
			IS	25.2	0.87
T10 correlate	1.338	0.512	CH	39.3	2.23	-
			GS	37.3	1.99
			IS	41.0	2.21
T11 locate	3.176	0.204	CH	24.6	1.31	-
			GS	26.2	1.63
			IS	28.0	1.51

). In order to identify significant intergroup differences, post-hoc tests (Bonferroni) were carried out.

In task T3 distinguish and T4 retrieve value, the average time of the group using the choropleth map was significantly different from that of both the other groups (T3: CH-GS p = 0.001, CH-IS p = 0.000; T4: CH-GS p = 0.002, CH-IS p = 0.000). In both cases the group with the choropleth map was the fastest one, participants using graduated symbols needed a little more time, and people using isolines needed the most time (T3: CH M = 14.6 s, SD = 1.36, GS M = 18.9 s, SD = 1.28, IS M = 23.9 s, SD = 1.63; T4: CH M = 10.9 s, SD = 0.42, GS M = 13.7 s, SD = 0.61, IS M = 16.6 s, SD = 0.86).

In the two remaining questions (T6 interpret, T8 cluster), in which statistically significant results were obtained, differences occurred between the group using the choropleth map and the group using the isoline map (T6 CH-IS p = 0.001; T8 CH-IS p = 0.005). In each task participants using the choropleth map needed less time than those using the isoline map (T6 CH M = 45.0 s, SD = 2.51, IS M = 57.7 s, SD = 3.23; T8 CH M = 22.0 s, SD = 1.47, IS M = 28.6 s, SD = 1.96).

4.3. The Difficulty of the Task

Study participants most often assessed choropleth maps as “easy” (80%). The group using the graduated symbols map assessed tasks as easy in 69% of cases, and the group using the isoline map in 61% of cases. Participants from these two groups equally often rated the task as difficult (9% of answers). In the case of the choropleth map, the answer “difficult” was indicated in only 4% of cases. The map type was related to the difficulty of the tasks: X² (4, N = 2379) = 77.305, p = 0.000, Cramér’s V = 0.127, p = 0.000.

Pairwise comparisons:

• CH-GS X² (2, N = 1579) = 30.420, p = 0.000, Cramér’s V = 0.139, p = 0.000 (with statistically significant lower difficulty of the choropleth map)
• CH-IS X² (2, N = 1611) = 73.283, p = 0.000, Cramér’s V = 0.213, p = 0.000 (with statistically significant lower difficulty of the choropleth map)
• GS-IS X² (2, N = 1568) = 13.595 p = 0.001, Cramér’s V = 0.093, p = 0.001 (with statistically significant lower difficulty of the graduated symbols map).

The task that was assessed as being the easiest was T2 find extremum (90%). In the choropleth group, this percentage was as high as 95% (Figure 6). The “easy” assessment was given least frequently for T10 correlate (40%). For this task, there was also the lowest percentage of “easy” answers when considering individual groups, as only 26% of the participants using the isoline map assessed the task in this way. What is more, for T10 correlate, the highest overall percentage (20%) of all grades—“difficult”—was given. Furthermore, in the case of the graduated symbols group, this percentage was as high as 25% (Figure 6).

In the case of the task difficulty, significant relationships between this variable and the type of map were found in 7 out of 11 tasks: T1 identify (p = 0.013), T3 distinguish (p = 0.000), T4 retrieve value (p = 0.035), T5 compare (p = 0.000), T6 interpret (p = 0.002), T10 correlate (p = 0.048), T11 locate (p = 0.028) (

Table 5. Inferential statistics among participants using the three map types.

Task	chi-Square	p	CRAMÉR’S V	p	Pairwise Comparison	Chi-Square	p	Cramér’s V	p
T1 identify	12.629	0.013	0.186	0.013	CH-GS	1.400	0.497	0.107	0.497
					CH-IS	11.039	0.004	0.300	0.004
					GS-IS	6.846	0.033	0.236	0.033
T2 find extremum	3.053	0.549	0.091	0.549	-
T3 distinguish	41.146	0.000	0.334	0.000	CH-GS	1.981	0.371	0.127	0.371
					CH-IS	31.680	0.000	0.508	0.000
					GS-IS	20.708	0.000	0.410	0.000
T4 retrieve value	10.373	0.035	0.168	0.035	CH-GS	5.865	0.053	0.219	0.053
					CH-IS	4.089	0.043	0.182	0.043
					GS-IS	3.254	0.197	0.163	0.197
T5 compare	23.575	0.000	0.253	0.000	CH-GS	11.850	0.003	0.312	0.003
					CH-IS	22.491	0.000	0.482	0.000
					GS-IS	2.170	0.338	0.133	0.338
T6 interpret	17.472	0.002	0.219	0.002	CH-GS	12.523	0.002	0.322	0.002
					CH-IS	14.137	0.001	0.336	0.001
					GS-IS	0.128	0.938	0.033	0.938
T7 categorize	3.065	0.547	0.092	0.547	-
T8 cluster	5.600	0.231	0.124	0.231	-
T9 sort	2.305	0.680	0.080	0.680	-
T10 correlate	9.578	0.048	0.162	0.048	CH-GS	2.213	0.331	0.135	0.331
					CH-IS	7.466	0.024	0.244	0.024
					GS-IS	4.821	0.090	0.202	0.090
T11 locate	10.875	0.028	0.173	0.028	CH-GS	5.959	0.051	0.222	0.051
					CH-IS	10.222	0.006	0.286	0.006
					GS-IS	0.555	0.758	0.069	0.758

).

In T1 identify and T3 distinguish, the groups using choropleth and graduated symbols maps rated tasks similarly; however, the second group rated them as slightly harder (T1 CH “easy”: 67% and “difficult”: 3%, GS “easy”: 62% and “difficult”: 8%; T3 CH “easy”: 90% and “difficult”: 5%, GS “easy”: 82% and “difficult”: 7%). The assessments of both of these groups were much higher than the assessments of the group using isolines (T1 IS “easy”: 39% and “difficult”: 13%; T3 IS “easy”: 44% and “difficult”: 10%), and pairwise comparisons showed that it is between these that the dependence of the variable difficulty on the map type was significant (T1 CH-IS p = 0.004, GS-IS p = 0.033; T3 CH-IS p = 0.000, GS-IS p = 0.000).

For T5 compare and T6 interpret significant dependence between the difficulty of the task and the map types occurred when comparing the choropleth group to the other two groups: graduated symbols and isolines (T5 CH-GS p = 0.003, CH-IS p = 0.000; T6 CH-GS p = 0.002, CH-IS p = 0.001). Again, the participants who used the choropleth map assessed the tasks as the easiest (T5 CH “easy”: 80% and “difficult”: 5%; T6 CH “easy”: 67% and “difficult”: 3%). However, the assessments of groups using graduated symbols and isoline maps were more similar to each other (T5 GS “easy”: 51% and “difficult”: 15%, IS “easy”: 39% and “difficult”: 23%; T6 GS “easy”: 37% and “difficult”: 14%, IS “easy”: 36% and “difficult”: 16%).

In the case of three tasks (T4 retrieve value, T10 correlate, T11 locate), significant dependence between the difficulty of the task and the map type was found when analyzing pairs of the choropleth group and the isolines group (T4 CH-IS p = 0.043; T10 CH-IS p = 0.024; T11 CH-IS p = 0.006). In each of these cases, the group using the choropleth map assessed the tasks as easier (T4 CH “easy”: 95% and “difficult”: 0%, IS “easy”: 84% and “difficult”: 0%; T10 CH “easy”: 50% and “difficult”: 14%, IS “easy”: 26% and “difficult”: 21%; T11 CH “easy”: 75% and “difficult”: 3%, IS “easy”: 48% and “difficult”: 10%).

5. Discussion

The aim of the study was to compare three types of maps presenting the same quantitative data—choropleth map, graduated symbols map, isoline map—with regard to basic performance usability metrics—accuracy, time, and difficulty of tasks. Based on this we wanted to explore the issues related to differences in users’ performance, their subjective rating of map difficulty with regard to the frequency of occurrence of the map, and the relation between performance metrics and the subjective level of difficulty.

• RQ1: Do different map types, presenting the same input data, facilitate users’ performance of various map use tasks equally?
• H1: Users differ in terms of performance when using informationally equivalent but differently designed thematic maps for various tasks.

When it comes to the overall results, the best metrics of performance (answer accuracy and time) for all tasks were obtained when working with the choropleth map. The group using the graduated symbols maps came second and the group using the isoline map had the worst results. Thus, the results obtained for non-interactive maps in this study were the opposite of those for interactive maps from the study by Roth et al. [18,19] in which the isoline map guaranteed the highest accuracy. The differences in the results may be the effect of interactivity.

However, many authors stress that particular thematic map types support the solution of specific tasks [1,3,4]. When task-relevant information is salient, performances should be enhanced [11]. To verify this statement, our study considered a wide range of tasks. Should this be true, depending on the task, the best results should have been obtained by groups using different maps. For example, T1 interpret, relating to the overall pattern, should result in the best performance with a choropleth map. Whereas, in the task of comparing values (e.g., T9 cluster), the group using graduated symbols should achieve the best results. In the case of isolines, the characteristic task consisting of the arrangement of magnitudes in which this map could work best should be T10 correlate. However, according to the results obtained, the best metrics of performance (answer accuracy and time) for all the tasks tested were obtained using the choropleth map. We need to emphasize that we tested only analytical, basic types of tasks; therefore, these results should be verified using more complex tasks.

When it comes to the correctness of the answer, statistically significant results were obtained in 5 out of 11 tasks. In three tasks the group using the choropleth map was better than the group using the graduated symbols map (T6 interpret, T9 sort, T10 correlate), and in four tasks they were better than users of the isoline map (T1 identify, T3 distinguish, T6 interpret, T10 correlate). In the case of three tasks, when a pair of graduated symbols and isoline maps was considered, there were statistically significant relationships between the type of map and the correctness of the answer. In two cases (T1 identify and T3 distinguish), the group using the graduated symbol map scored better, and in one case (T9 sort) the group using the isoline map was better.

In terms of response time, statistically significant differences between the groups appeared in only 4 out of 11 tasks. In each of these cases, the group using the choropleth map was faster than the group using the isoline map (T3 distinguish, T4 retrieve value, T6 interpret, T8 cluster) and, in two cases, faster than the group using the graduated symbols map (T3 distinguish, T4 retrieve value).

In conclusion, users performance differed when using informationally equivalent but differently designed thematic maps for a particular task type. We thus accept Hypothesis 1 stating that users differ in terms of performance when using informationally equivalent but differently designed thematic maps for various analytical tasks. However, unlike in many studies [11,22,23], there were hardly any differences between the tasks. In most tested tasks choropleth map users achieved the best results in terms of accuracy and answer time.

• RQ2: Do users perceive more commonly applied map types as easier to use than map types encountered less frequently?
• H2: Map users perceive the most commonly applied map types as the easiest ones.

According to Havelková and Hanus [8], choropleth maps are most commonly used in school atlases and textbooks among quantitative map types. Graduated or proportional symbols are used somewhat less frequently, and isolines are used the least frequently among those considered. Overall, in our study, the group using the choropleth map assessed the task as the easiest, and the group using the isoline map as the most difficult. In terms of the subjective metric of task difficulty, there was the highest number of statistically significant cases (7 of 11 tasks: T1 identify, T3 distinguish, T4 retrieve value, T5 compare, T6 interpret, T10 correlate, T11 locate). In all of these, the choropleth map was assessed as easier than the isoline map. Moreover, in two cases the choropleth map was assessed as easier than the graduated symbols maps (T5 compare, T6 interpret) and in the other two tasks, the graduated symbols map was easier than the isoline map (T3 distinguish, T4 retrieve value).

In conclusion, the results obtained are consistent with the statement of Petchenik [28] that users prefer familiar and previously known solutions. We also believe that due to the common use of choropleth maps in school atlases, students can be trained better in reading information from this type of thematic map. They have more opportunities to refer to the choropleth map, to get an understanding of the mapped phenomenon, and finally they can be more “fluent” and experienced in using choropleth maps. Therefore, we accept Hypothesis 2, stating that the more commonly applied map type, namely the choropleth map, is rated as easier than other map types.

• RQ3: Does the subjective rating of the difficulty of tasks provided by users of different map types match the other performance metrics?
• H3: The subjective metric of difficulty mismatch the usability performance metrics of answer accuracy and answer time.

In the study presented in this article, for each of the analyzed metrics (answer accuracy, answer time and rated difficulty), the best results were obtained with regard to the choropleth map; worse results were found for the graduated symbols map, and the worst for the isoline map. The highest number of statistically significant cases occurred for the task difficulty metric (in 7 tasks out of 11), and the least for the answer time metric (in 4 tasks out of 11). In contrast with studies comparing choropleth maps, dot maps, and graduated symbols by Mendonça and Delazari [30,31] and Roth et al. [18,19], in which participants did not give better answers when using the choropleth map, yet they preferred it or assessed it as easy, in our study users of the choropleth map obtained the best performance metrics and assessed it as the easiest. Additionally, the results of the presented study are not coherent with the results of studies where there was no consistency in terms of performance metrics and subjective metrics [7,29]. To sum up, we reject Hypothesis 3, since the collected data suggest that both performance metrics and subjective rating of difficulty favor choropleth maps over graduated symbols and isoline map types. We believe that this match, unlike the previous studies [7,18,19,29,30,31], can be connected with the high level of training in school. However, we have to emphasize that in the reported study we focused only on the analytical tasks and did not cover more sophisticated and complex tasks like problem-solving or decision-making.

6. Conclusions

The conducted user study showed that choropleth maps can be an effective, efficient, and preferred tool for extracting specific information and conducting simple tasks. The positive opinion, in terms of rated difficulty, of the choropleth map showed that this map type is also perceived as the easiest solution for mapping spatial phenomena. The training in reading choropleth maps in school education, as shown by the analysis of school atlases [8], seems to result in a high level of literacy with regard to reading this map type. We believe that this may have both positive and negative consequences. On the one hand, a choropleth map is a powerful way of presenting spatial data, and currently it can be perceived as not challenging, because of the availability of data that can be mapped using choropleth maps, and that it is not a complicated process to create maps in GIS software. On the other hand, one has to be aware that seemingly simple choropleth map making has to be conducted carefully and with awareness regarding the mapped data, since the process of data classification, as well as of selecting the number of enumeration units, has an important impact on the resulting map [40]. Moreover, map users cannot limit their tools to one map type only and should not abandon other map types that can communicate information that is missing in choropleth maps; for example, variability within an enumeration unit, or data that are not classified. We believe that the use of every map type is valuable, depending on the purpose and problem to be solved. Therefore, even though the collected data favor one map type, the choropleth map, we believe that it is worth educating users using a much wider scope of thematic mapping solutions.