An anthropomorphic method for progressive matrix problems

https://doi.org/10.1016/j.cogsys.2012.08.002Get rights and content

Abstract

Progressive matrix problems are frequently used in modern IQ tests. In a progressive matrix problem, the task is to identify the missing element that completes the pattern of a pictorial matrix. We present a method for solving progressive matrix problems. The method is not limited to problems that are on the multiple choice format, which makes it potentially useful for solving real-world pattern discovery problems that do not come with predefined answer alternatives. The method is anthropomorphic in the sense that it uses certain problem solving strategies that were reported by high-achieving human solvers. We also describe a computer program implementing this method. The computer program was tested on the sets C, D, and E of Raven’s Standard Progressive Matrices test and it produced correct solutions for 28 of the 36 problems considered. This score corresponds roughly to an IQ of 100. Finally, we conclude that it is possible to solve progressive matrix problems without analyzing potential answer alternatives and discuss some implications of this finding.

Introduction

In this paper we describe a method and a computer program for solving progressive matrix problems. Our method is not limited to progressive matrix problems that are on the multiple choice format. Thus it can be used for solving progressive matrix problems both with and without predefined answer alternatives. Problems that are formulated without multiple choice alternatives are generally strictly harder to solve. They are also more relevant to cognitive modeling and more useful for solving real-world problems that do not come with predefined answer alternatives. Our method is anthropomorphic in the sense that we implemented certain problem solving strategies that have been reported by high-achieving human problem solvers.

In 1912, the intelligence quotient (IQ) was proposed by William Stern as a normally distributed measure of intelligence with median 100 and standard deviation 15 (Stern & Whipple, 1914). Today, the IQ is a widely used and commonly computed directly from the results obtained on standardized psychological tests such as the Wechsler tests and Raven’s Progressive Matrices (Groth-Marnat, 2009, Raven and Court, 2003). In using performance on such standardized tests to define intelligence, implicitly intelligence can be measured on any cognitive agent; either human or machine (Bringsjord & Schimanski, 2003).

The view that intelligence can be defined in terms of performance on standardized psychological tests has been challenged by several authors. Legg and Hutter (2007) argues that an ad hoc computer program that performs well on some specific intelligence tests would not qualify as intelligent: according to their paradigm, passing such test is a necessary, but not sufficient condition for intelligence.

The term artificial intelligence (AI) was coined by John McCarthy, who defined it as “the science and engineering of making intelligent machines” (McCarthy, 2007). Whatever the exact relation between performance on IQ tests and human intelligence might be, constructing computer programs for solving standardized intelligence tests is a fundamental challenge to AI. As we shall shortly see, this challenge has been met only in part.

Raven’s Progressive Matrices (RPM) is a standardized intelligence test that was introduced by Raven (1936). Each RPM problem is presented as a 2 × 2 or 3 × 3 matrix of images following some pattern. The bottom right position, or cell, is left blank and the solver’s task is to choose the missing image from a set of eight solution candidates. Fig. 1 shows an example of this type of problem. For copyright reasons, the illustrations in this paper do not depict original RPM problems, but constructed equivalents.

Extensive studies indicate high levels of correlation between the RPM scores and the scores on a range of other tests of (general) intelligence (Raven and Court, 2003, Snow et al., 1984). This, together with the fact that the RPM tests are entirely picture-based and require no language proficiency, has helped build the popularity of the tests.

On an abstract level, RPM problems are similar to pattern recognition problems since they are both problems of inductive reasoning, alike to number progression problems (Varzi, 2006). Moreover, pattern recognition is closely related to perception, particularly to how objects and groups of objects are perceived. Such phenomena have been studied extensively in Gestalt theory (Wertheimer, 1939) and structural information theory (SIT) (Leeuwenberg, 1968, Leeuwenberg, 1971).

The RPM set includes the Standard Progressive Matrices (SPM) and the Advanced Progressive Matrices (APM). The SPM, which is the target of this work, consists of five (increasingly difficult) sets of questions labeled A–E (Raven & Court, 2003). Each set contains 12 matrices. The sets A and B consist of 2 × 2 matrices, while C, D and E consist of 3 × 3 matrices. Another widely used set of RPM is the Colored Progressive Matrices (CPM), aimed at children, the elderly and people with learning disabilities.

The computational aspects of RPM have been subject to considerable investigation. Carpenter, Just, and Shell (1990) created two RPM solvers aimed at modeling the RPM solving performance of average and high scoring human subjects tested earlier in their study. Both programs achieved results correlating well with experimental data, however the input to the solvers were textual descriptions of the RPM problems, meaning that none of them solves RPM problems strictly speaking (Meo, Roberts, & Marucci, 2007).

Bringsjord and Schimanski (2003) implemented an RPM solver based on an automatic theorem prover, but reported no results on any of the sets.

Lovett et al., 2007, Lovett et al., 2010 developed an RPM solver using the SME/CogSketch sketch understanding architecture (Forbus et al., 2004, Forbus et al., 2008). They used an analogical reasoning strategy and addressed sections B-E of SPM with a score of 44/48 correct solutions.

Kunda, McGreggor, and Goel (2010) implemented two different visual solution strategies, of which the best performing one solves 58% of the SPM set.

Rasmussen and Eliasmith (2011) proposed a neural model for inductive rule generation and successfully applied it to RPM but did not report detailed results on the problem set.

In order to produce an answer to each matrix, all of solvers mentioned above rely on, or at least make use of, the eight provided solution candidates. Such practice is perfectly legitimate, as access to the alternatives is part of the format of the RPM test. However, evidence indicates that high-achieving human solvers employ strategies that take the solution candidates into less account, or none at all except from finally comparing “their” solution to the answer alternatives, than those used by lower scoring subjects (Bethell-Fox et al., 1984, Carpenter et al., 1990, Vigneau et al., 2006). To what extent are then the answer alternatives needed in order to give a correct answer?

In this work we will present an architecture capable of producing and drawing RPM solutions without considering the answer alternatives.

Section snippets

Method

We will now detail our approach to solving RPM computationally, after specifying the problem at hand.

Results

Given that the program produces outputs in form of on-screen images and it is built on the premise of not having access to the solution alternatives, the computed and displayed solutions are evaluated by visually comparing them to the answer alternatives of the original RPM test items.

It is also trivial to, after representing the correct solution candidate in the same format as each cell in the problem set, compare the given solution to the correct one, one vector graphics element at the time,

Discussion and conclusion

In this section we discuss why our program succeeds on certain problems and fails on others. We also discuss some aspects of the SPM test.

Acknowledgment

The authors thank Professor Carl Martin Allwood at the Department of Psychology, University of Gothenburg and Sara Henrysson Eidvall at Mensa International for valuable input.

References (30)

  • G. Groth-Marnat

    Handbook of psychological assessment

    (2009)
  • Kunda, M., McGreggor, K., & Goel, A. (2010). Taking a look (literally!) at the Ravens intelligence test: Two visual...
  • E. Leeuwenberg

    Structural information of visual patterns: An efficient coding system in perception

    (1968)
  • E. Leeuwenberg

    A perceptual coding language for visual and auditory patterns

    The American Journal of Psychology

    (1971)
  • S. Legg et al.

    Tests of machine intelligence

  • Cited by (23)

    • Visual mental imagery: A view from artificial intelligence

      2018, Cortex
      Citation Excerpt :

      P1 is 1.2 times larger than P2, the relative rotation between P1 and P2 is .0°, and there are no reflection relationships between P1 and P2.” Many subsequent AI systems have used similar formats of propositional representations to solve both geometric analogy and matrix reasoning problems, investigating many interesting aspects of this task domain including maintaining goals and subgoals in working memory (Carpenter, Just, & Shell, 1990), logical reasoning techniques (Bringsjord & Schimanski, 2003) techniques for analogical mapping between problem elements (Lovett et al., 2009), representing hierarchical patterns in problem information (Strannegård, Cirillo, & Ström, 2013), and the induction of solution rules (Rasmussen & Eliasmith, 2011). However, these propositional AI systems do not explain a different type of solution strategy that humans can and do use, which is to recruit visual mental imagery instead of relying purely on propositional (e.g., verbal or linguistic) mental representations.

    • Computer models solving intelligence test problems: Progress and implications

      2016, Artificial Intelligence
      Citation Excerpt :

      Notwithstanding, many modern computational systems keep assuming that hand-coded representations are available as inputs. Systems such as those of Ragni and Neubert [122,123] and Strannegård et al. [133] continue addressing Raven's progressive matrices by generating hand-coded propositional attribute-value vectors with, respectively, a fixed set of attributes (shape, size, number of sides, width, height, colour, rotation, position, and quantity); and vector graphics where each matrix is encoded as vectors of attribute-value pairs using different abstraction levels (features, elements, groups, and cells) by using the XML-based specification language from Microsoft, XAML. Prade, Richard, and Correa's approach [125–127] is a special case, applicable to different levels of representation (resolution).

    • Probabilistic Neural-Symbolic Models With Inductive Posterior Constraints

      2024, IEEE Transactions on Neural Networks and Learning Systems
    View all citing articles on Scopus
    View full text