Human-Robot Interaction in Unstructured Environments

Progress Report
May 12, 2014
Kevin DeMarco
1 Paper Ideas
After collecting the data of the diver in the ME Dive Well last week and culling through the data, I have
identied four main topics I wish to address with the data set.
1.1 Diver Detection and Tracking with Sonar and Video
Investigate algorithms for fusing the sonar and video data to improve diver tracking. Also, investigate search
algorithms for locating the diver using the sonar, prioritizing each cluster as possible divers, using autonomy
to visit each cluster, and then verify the existence of the diver with the optical camera. This paper will use
the data that was collected to show that is possible to detect a diver with fused sonar and video data, but
then it will use the simulator to verify the search / high-level autonomy functions. These functions could be
tested in the eld for my nal thesis.
1.2 Using High-Level Autonomy for Safe Leader-Follower Behaviors in Under-
water Robots
When two humans perform leader-follower behaviors, they have to ensure that their buddys attention can
be acquired at any time. Thus, the follower cannot be positioned directly behind the leader, otherwise, if
the follower has a problem, the follower would not be able to notify the leader of the problem. Due to the
robots limited eld of view, it cannot maintain constant contact with the following diver. Thus, high-level
autonomy functions have to be built into the robot assistant that help to resolve the conicting behaviors of
moving to a waypoint and checking on the status of the following diver. Examples from the data collected
will be used to motivate the need for this behavior and the behavior will be veried in simulation for this
paper. For my thesis, I hope to verify the behavior with an actual diver.
1.3 Using Locally Adaptive Regression Kernels (LARK) in Field Robotics Ap-
plications
I have had some initial success in detecting the divers face using LARKs. This work should be extended
to detecting the divers hands, tools, workspace, etc. I think using LARKs in a eld robotics setting is very
interesting because the human team could merely upload a CAD or image le of a valve or item of interest
for the robot to nd and/or operate. Thus, this paper could deal with improving eld robotics operations
when humans are in-the-loop, but not teleoperating the robot.
1.4 Underwater Autonomous Tool Exchange with an Optical Camera
I collected video and sonar data of the diver exchanging a wrench several times with the robot. Part of the
operation requires the camera in the ROV to tilt, so that the manipulator is visible. If the hand detector
works correctly, and if there is a way to detect the divers hand extending towards the ROV, the ROV could
enter an exchange protocol that consists of autonomously tilting the camera, and operating the manipulator
to grab the tool from the diver. Unfortunately, at close-ranges, the sonar is useless at detecting the divers
limbs, this procedure would be focused on using the single optical camera to detect the divers intentions.
The protocol could be tested in a small pool or tank without too much diculty.
1