With an eye towards design of human-in-the-loop systems, we investigate human decision making in a social context for tasks that require the human to make repeated choices among finite alternatives. We consider a human decision maker who receives feedback on his/her own performance as well as on the choices of others performing the same task. We use a drift-diffusion, decision-making model that has been fitted to human neural and behavioral data in sequential, two-alternative, forced-choice tasks and recently extended to the social context with an empirically derived feedback term that depends on choices of other decision makers.

We show conditions for this model to be a Markov process, and we derive the steady-state probability distribution for choice sequences and individual performance as a function of the strength of the social feedback. It has recently been shown in behavioral experiments that human decision-making performance for a relatively easy task is decreased with this social feedback; we show that our analytic predictions agree with this finding.

In human-in-the-loop systems, humans are often faced with making repeated choices among finite alternatives in response to observations of the evolving system performance. In order to design humans into such systems, it is important to develop a systematic description of human decision making in this context. We examine a commonly used, drift-diffusion, decision-making model that has been fit to human neural and behavioral data in sequential, two-alternative, forced-choice tasks.

We show how this model and type of task together can be regarded as a Markov process, and we derive the steady-state probability distribution for choice sequences. Using the analytic expression for this distribution, we prove matching behavior for tasks that exhibit a matching point and we compute the sensitivity of steady-state choices to a model parameter that measures the decision maker's "exploratory" tendency.

A class of binary decision-making tasks called the two-alternative forced-choice task has been used extensively in psychology and behavioral economics experiments to investigate human decision making. The human subject makes a choice between two options at regular time intervals and receives a reward after each choice; for a variety of reward structures, these experiments show convergence of the aggregate behavior to rewards that are often suboptimal.

In this paper we present two models of human decision making: one is the Win-Stay, Lose-Switch (WSLS) model and the other is a deterministic limit of the popular Drift Diffusion (DD) model. With these models we prove the convergence of human behavior to the observed aggregate decision making for reward structures with matching points. The analysis is motivated by human-in-the-loop systems, where humans are often required to make repeated choices among finite alternatives in response to evolving system performance measures. We discuss application of the convergence result to the design of human-in-the-loop systems using a map from the human subject to a human supervisor.

Leveraging research by psychologists on human decision-making, we present a human-robot decision-making problem associated with a complex task and study the corresponding joint decision-making dynamics. The collaborative task is designed so that the human makes decisions just as human subjects make decisions in the two-alternative, forced-choice task, a well-studied decision-making task in behavioral experiments. The human subject chooses between two options at regular time intervals and receives a reward after each choice; for a variety of reward structures, the behavioral experiments show convergence to suboptimal choices.

We propose a human-supervised robot foraging problem in which the human supervisor makes a sequence of binary decisions to assign the role of each robot in a group in response to a report from the robots on their resource return. We discuss conditions under which the decision dynamics of this human-robot task is reasonably well approximated by the kinds of reward structures studied in the psychology experiments. Using the win-stay, lose-switch human decision-making model, we prove convergence to the experimentally observed aggregate human decision-making behavior for reward structures with matching points. Finally, we propose an adaptive law for robot reward feedback designed to help the human make optimal decisions.