As the variety and complexity of tasks carried about by telerobotic systems underwater increases, so does the demand for technology that increases efficiency and safety of operations. We present a novel approach to subsea manipulation that leverages the natural perceptive capability of human pilots. We present a method and framework for generation and rendering of six degree-of-freedom haptic virtual fixtures based on streaming point cloud data captured by an RGB-D sensor. Sensor data is captured at 30 Hz and virtual fixtures are constructed in real-time in parallel on a GPU. The method is applied to two typical underwater telerobotic valve operations by implementing a forbidden-region virtual fixture and a guidance virtual fixture. During operation, real-time haptic feedback is sent to the operator at 1000 Hz. This haptic feedback allows pilots of remotely-operated vehicles to receive sense of touch feedback through the use of non-contact sensors.

The Adaptable Monitoring Package (AMP) along with a Remotely Operated Vehicle (ROV) and custom tool skid, is being developed to support near-field (󖼚 meters) and long-range monitoring of hydrokinetic energy converters. The goal for the AMP is to develop a system capable of supporting a wide range of environmental monitoring in harsh oceanographic conditions, at a cost in line with other aspects of technology demonstrations. This paper presents a system description of all related infrastructure for the AMP, including supported instrumentation, deployment ROV and tool skid, launch platform, and docking station. Design requirements are driven by the monitoring instrumentation and the strong waves and currents that typify marine renewable energy sites. Hydrodynamic conditions from the Pacific Marine Energy Centers wave test sites and Admiralty Inlet, Puget Sound, Washington are considered in the design as early adoption case studies. A methodology is presented to increase the capabilities to deploy and operate the AMP in strong currents by augmenting thrust and optimizing the system drag profile through computational fluid dynamic modeling. Preliminary results suggest that the AMP should be deployable in turbulent environments with mean flow velocities up to 1 m/s.

With a principled methodology for systematic design of human–robot decision-making teams as a motivating goal, we seek an analytic, model-based description of the influence of team and network design parameters on decision-making performance. Given that there are few reliably predictive models of human decision making, we consider the relatively well-understood two-alternative choice tasks from cognitive psychology, where individuals make sequential decisions with limited information, and we study a stochastic decision-making model, which has been successfully fitted to human behavioral and neural data for a range of such tasks. We use an extension of the model, fitted to experimental data from groups of humans performing the same task simultaneously and receiving feedback on the choices of others in the group. First, we show how the task and model can be regarded as a Markov process. Then, we derive analytically the steady-state probability distributions for decisions and performance as a function of model and design parameters such as the strength and path of the social feedback. Finally, we discuss application to human–robot team and network design and next steps with a multirobot testbed.

With an eye towards design of human-in-the-loop systems, we investigate human decision making in a social context for tasks that require the human to make repeated choices among finite alternatives. We consider a human decision maker who receives feedback on his/her own performance as well as on the choices of others performing the same task. We use a drift-diffusion, decision-making model that has been fitted to human neural and behavioral data in sequential, two-alternative, forced-choice tasks and recently extended to the social context with an empirically derived feedback term that depends on choices of other decision makers.

We show conditions for this model to be a Markov process, and we derive the steady-state probability distribution for choice sequences and individual performance as a function of the strength of the social feedback. It has recently been shown in behavioral experiments that human decision-making performance for a relatively easy task is decreased with this social feedback; we show that our analytic predictions agree with this finding.

In human-in-the-loop systems, humans are often faced with making repeated choices among finite alternatives in response to observations of the evolving system performance. In order to design humans into such systems, it is important to develop a systematic description of human decision making in this context. We examine a commonly used, drift-diffusion, decision-making model that has been fit to human neural and behavioral data in sequential, two-alternative, forced-choice tasks.

We show how this model and type of task together can be regarded as a Markov process, and we derive the steady-state probability distribution for choice sequences. Using the analytic expression for this distribution, we prove matching behavior for tasks that exhibit a matching point and we compute the sensitivity of steady-state choices to a model parameter that measures the decision maker's "exploratory" tendency.

A class of binary decision-making tasks called the two-alternative forced-choice task has been used extensively in psychology and behavioral economics experiments to investigate human decision making. The human subject makes a choice between two options at regular time intervals and receives a reward after each choice; for a variety of reward structures, these experiments show convergence of the aggregate behavior to rewards that are often suboptimal.

In this paper we present two models of human decision making: one is the Win-Stay, Lose-Switch (WSLS) model and the other is a deterministic limit of the popular Drift Diffusion (DD) model. With these models we prove the convergence of human behavior to the observed aggregate decision making for reward structures with matching points. The analysis is motivated by human-in-the-loop systems, where humans are often required to make repeated choices among finite alternatives in response to evolving system performance measures. We discuss application of the convergence result to the design of human-in-the-loop systems using a map from the human subject to a human supervisor.

Leveraging research by psychologists on human decision-making, we present a human-robot decision-making problem associated with a complex task and study the corresponding joint decision-making dynamics. The collaborative task is designed so that the human makes decisions just as human subjects make decisions in the two-alternative, forced-choice task, a well-studied decision-making task in behavioral experiments. The human subject chooses between two options at regular time intervals and receives a reward after each choice; for a variety of reward structures, the behavioral experiments show convergence to suboptimal choices.

We propose a human-supervised robot foraging problem in which the human supervisor makes a sequence of binary decisions to assign the role of each robot in a group in response to a report from the robots on their resource return. We discuss conditions under which the decision dynamics of this human-robot task is reasonably well approximated by the kinds of reward structures studied in the psychology experiments. Using the win-stay, lose-switch human decision-making model, we prove convergence to the experimentally observed aggregate human decision-making behavior for reward structures with matching points. Finally, we propose an adaptive law for robot reward feedback designed to help the human make optimal decisions.