When to stop — A cardinal secretary search experiment,☆☆

https://doi.org/10.1016/j.jmp.2020.102425Get rights and content

Highlights

  • Participants substantially deviate from risk neutral-optimal search by stopping early.

  • The deviation increases with number of candidates remaining.

  • The elicitation method has a strong influence on stopping behavior.

  • Decreasing the amount of candidates between rounds further fosters stopping early.

Abstract

The cardinal secretary search problem confronts the decision maker with more or less candidates who have identically and independently distributed values and appear successively in a random order without recall of earlier candidates. Its benchmark solution implies monotonically decreasing sequences of optimal value aspirations (acceptance thresholds) for any number of remaining candidates. We compare experimentally observed aspirations with optimal ones for different numbers of (remaining) candidates and methods of experimental choice elicitation: “hot” collects play data, “warm” asks for an acceptance threshold before confronting the next candidate, and “cold” for a complete profile of trial-specific acceptance thresholds. The initially available number of candidates varies across elicitation methods to obtain more balanced data. We find that actual search differs from benchmark behavior, in average search length and success, but also in some puzzling qualitative aspects.

Introduction

In commercial and private life, one is often individually responsible to search for a qualified candidate to fill a certain need. Since one usually interviews candidates successively, due to time and monetary constraints, one may not be able to assess the quality – in the following we refer to individual qualities as values – of all potential candidates. On the other hand the impossibility to keep candidates waiting for long questions recall. Without recall, past candidates are lost. Thus, waiting to check the last candidate is very risky. Consider the example of trying to find an apartment, an employee, or a life partner; one must often act immediately or otherwise risk to lose an attractive option. This highlights the relevance of asking and answering “When to stop searching?”.

The modified version of the secretary search paradigm, based on successively appearing and a priori identical candidates and no recall, has an elegant benchmark solution. Its optimal stopping rule does not maximize the probability to hire the best possible candidate2 but maximizes expected quality which requires cardinal values of candidates. The decision maker confronts potential candidates sequentially, each with quantifiable quality (an unambiguously recognizable monetary value). Candidates become unavailable when not accepted immediately. While stylized, this captures rather realistically the example of searching for a house in a popular area with many competitors, provided that the values of the various properties can be assessed unambiguously and quickly.

One important stylized aspect of this search type is that having already met more or less candidates is completely uninformative about the random values of later ones. This avoids, at least theoretically, engaging in Bayesian updating and allows to focus instead on how the number of remaining candidates affects search behavior. Behaviorally, qualities of past candidates may nevertheless matter, e.g. due to the Gambler’s fallacy, anxiety or regret when running out of candidates, especially if more recent candidates turn out worse than prior ones.

In field situations, previous experiences with search episodes could account for heterogeneity in aspiration formation and adaptation. A controlled stylized lab experiment helps to limit theoretically confounding effects of past experiences.3 Participants confront the various search tasks more than once which vary in the known initial number of a priori identical candidates. Although, theoretically, the number of remaining candidates is the only state variable, behaviorally such pure dependency of behavior on the state variable seemed questionable. Instead, we expected the quality of past rejected candidates and the difference between the initial and remaining number of candidates to matter and that larger differences will let one think that it is time to stop.

Optimality in dynamic decision tasks with finite horizon relies on backward induction or dynamic programming (see Bellman, 2013). Since the remaining number of candidates is the only state variable, the optimal strategy is a complete profile of first more and later less ambitious acceptance thresholds, derived in Appendix A.4 Although one could have experimentally induced risk neutrality via binary lottery incentives (the value of the accepted candidate determines linearly the probability of earning the larger rather than the smaller monetary amount),5 we let participants successively confront several search tasks, each with possibly many chance moves, to reduce the variance of earnings across all tasks.

In view of the stochastic complexity of the search tasks, optimal behavior of participants would be explanandum rather than explanans — but we do not confirm optimality. So optimal aspiration profiles are just benchmarks for analyzing actual search behavior of, at best, boundedly rational participants. The choice data, in part, directly reveal the success aspirations of participants and how they are adapted. This sheds light on the core concepts of bounded rationality theory like aspiration formation and adaptation (see originally Lewin, 1926; Hoppe, 1930; Lewin and Denbo, 1931; Lewin et al. 1944; Heckhausen, 1955, Simon, 1995, and specifically for aspiration adaptation Sauermann & Selten, 1962). If at all, individual differences in search behavior can be attributed to idiosyncratic characteristics of participants,6 like regret inclinations, analytic capability, etc.7

In the experiment participants accept or reject the successively revealed value vt in trial t directly after seeing it without recall (when rejecting vt in trial t this realization is lost and cannot be retrieved). If the last value is reached, it is automatically accepted. Since all values vt for t=1,,n (with n(2) denoting the initial number of candidates) are randomly and independently generated according to the uniform density distribution, concentrated on the interval [0,1], rejected values do not inform about future ones. So the number of remaining candidates nt is all what theoretically matters8 when deciding whether to accept or reject vt in trial t.

The experiment relies on many integer realizations. We have shifted up and enlarged the interval from 0 to 1 by allowing for all integer values v from 24 to 123 which are all equally probable. The number n of candidates is either 5, 10, or 15. Fig. 1 illustrates the optimal profile of aspirations or acceptance thresholds which decrease with the remaining number of trials, i.e. the number of remaining candidates (since the last candidate must be accepted).

We vary the initial number n(>1) of candidates within subjects (participants confront all three n-tasks) and between subjects only whether n increases or decreases. The other between subjects variation of choice elicitation is more psychologically grounded. In “cold”, participants are asked for complete strategies, i.e. a complete pattern of acceptance thresholds, before the first trial.9 For boundedly rational participants this presupposes an awareness that nothing can be inferred from the values of past rejected candidates and that more remaining candidates are better than less. Data of this condition allows to assess how actual and optimal aspirations differ. For example, whether actual aspirations are less often adapted than optimal ones when many candidates still remain to be seen.

The choice data in “hot” provide only the sequences of so far rejected values and the finally accepted one. When confronting early on a rather high value in “hot”, one may feel compelled to accept it in ways resembling the well known endowment effects. This likely differs from just imagining such a high early value in “cold”. In the hot “marshmallow delay” task10 children may have predominantly waited when choosing “coldly” but often failed to wait in “hot”. In our view, this suggests, on average, earlier stopping in “hot”.

The intermediate “warm” condition asks for trial-specific acceptance thresholds as in “cold” but only before encountering that trial, respectively its candidate. Thus one successively states acceptance thresholds in “warm” being aware of the rejected values so far. This allows for regret in “warm”, possibly measured by how many candidates have been lost and how far the present value is below the best former rejected one. Both aspects one can only anticipate when deciding in “cold”. We expected sharper declines of acceptance thresholds in “warm” due to an acute awareness of lost options. Behaviorally, post-decisional regret and how many opportunities have been lost could affect the next stated acceptance threshold in “warm”, similar to how it may affect it in “hot”.

In view of the crucial stochastic uncertainty of the iid-cardinal secretary search tasks we have abstained from adding another random event via experimentally inducing risk neutrality. Instead we promote risk neutrality via “cumulative pay”, i.e. participants are paid for all successive tasks to reduce their variance of earnings when viewing the entire experiment holistically. Using (risk neutral) optimality (RN-optimality from now on) as the benchmark, we partly focus on deviations from RN-optimality and how elicitation method, the nsequence, the number of remaining candidates and past experiences shape them.

Participants substantially deviate from RN-optimal search, especially when there are still many candidates to be seen. These deviations overwhelmingly let them stop too early although the opposite can also be observed. Not only the elicitation method matters but also whether the number n of initially available candidates increases or decreases. It seems that participants perceive the six successive rounds rather holistically, i.e. as a single comprehensive task what seems to justify that cumulative pay for several successive tasks reduces risk sensitivity.

In view of experimental methodology, cardinal secretarysearch tasks are interesting as they directly reveal success aspirations and their dependence on the remaining number of candidates but also the considerable heterogeneity in human psychology and cognition. Furthermore, they are suitable paradigms to shed new light on the debate among experimentalists whether to use the “cold” strategy method or to elicit “hot” play data which so far has largely neglected individual sequential choice making. One wonders how the debate so far could concentrate (Sonnemans, 2000, is an exception) on comparing elicitation methods for social and strategic interaction experiments without a profound decision theoretic foundation. For the latter we convincingly confirm that elicitation method and task sequencing matter crucially.

Regarding field relevance, one restriction is the known number of (remaining) attempts. In the field this may arise due to idiosyncratic characteristics of decision makers, for example, due to them being seriously time constrained. In the animal kingdom, an already starving predator has fewer attempts to hunt, much like somebody urgently searching for an apartment. Financial markets with stationary random-walk assets whose traders have to invest immediately could also be similar to our setup. Unlike Güth and Weiland (2011) we have neglected competition in search.

Section 2 informs about the related literature. Section 3 describes the experimental protocols. The data and main findings are described and statistically validated in Section 4 before the final discussion in Section 5.

Section snippets

On the literature

Our setup is rather specific in multiple ways: rather than trying to hire the best candidate, as in the classic secretary search task, we rely on the familiar expected profit motive in neo-classical economics; rather than inferring aspirations from sequential search data, we directly observe in “cold” and “warm” the stated aspiration levels which we can compare with the optimal ones. We will diagnose “anti-monotonicity” as one crucial aspect which clearly signals a much richer motivation of

The choice tasks and (risk neutral) optimality

The setup features a situation where DM confronts a known number n(2) of a priori identical candidates of whom DM has to hire one. What renders hiring difficult is that the quality or value for all candidates is randomly determined by the same independent and identical random move. Furthermore, candidates show-up sequentially and can be hired only when revealing their randomly selected value at their trial without recall (one cannot go back to former candidates in the sequence).

A strategy in

Data analysis

Due to less informative data in “hot”, we compare between-subjects conditions mainly via outcome data like average stopping times, payoffs, and standard deviations for n=15 when distinguishing only whether n=15 has been experienced first, respectively last (see Table 1). For n=15 one can also compare across “cold-increasing”, “warm”, and “hot” the frequencies of accepted but RN-unacceptable and rejected but RN-acceptable values till acceptance, i.e. omitting later evidence of such deviations

Conclusions

Let us begin by what can be learned from our analysis, especially in view of the different results due to variations in experimental choice elicitation (see first paragraph of Section 2). Clearly the familiar motive of neo-classical economics, expected profit, cannot alone account for how participants form value aspirations and adjust them across trials. This is obviously true quantitatively (see the striking differences between various conditions) but also partly qualitatively as revealed by

Acknowledgments

This research was funded by the Max Planck Institute for Collective Goods , Bonn.

References (34)

  • FerensteinE.Z. et al.

    A version of the Elfving optimal stopping time problem with random horizon

    Game Theory and Applications

    (2009)
  • FergusonT.S.

    Who solved the secretary problem?

    Statistical Science

    (1989)
  • FreemanP.R.

    The secretary problem and its extensions: A review

    International Statistical Review/Revue Internationale de Statistique

    (1983)
  • GüthW. et al.

    Aspiration formation and satisficing in search with (out) competition

    New Zealand Economic Papers

    (2011)
  • HeckhausenH.
  • KahnemanD. et al.

    Choices, values, and frames

    American Psychologist

    (1984)
  • KarlinS.

    Stochastic models and optimal policy for selling an asset

  • Cited by (0)

    Financing: This research project was financed by the Max Planck Institute for Research on Collective Goods.

    ☆☆

    The authors gratefully acknowledge the constructive and helpful advice of two anonymous referees on how to revise of the manuscript.

    1

    Department of Economics and Finance, LUISS Guido Carli, Viale Romania 32, Rome 00198, Italy.

    View full text