Online Replanning With Human-in-the-Loop for Non-Prehensile Manipulation in Clutter — A Trajectory Optimization Based Approach

In this work, we propose a new trajectory optimisation-based motion planning algorithm for the problem of non-prehensile manipulation in clutter.

We are interested in the problem where a number of robots, in parallel, are trying to solve reaching through clutter problems in a simulated warehouse setting. In such a setting, we investigate the performance increase that can be achieved by using a human-in-the-loop providing guidance to robot planners. These manipulation problems are challenging for autonomous planners, as they have to search for a solution in a high-dimensional space. In addition, physics simulators suffer from the uncertainty problem where a valid trajectory in simulation can be invalid when executed in the real-world. To tackle these problems, we propose an online-replanning method with a human-in-the-loop. This system enables a robot to plan and execute a trajectory autonomously, but also to seek high-level suggestions from a human operator if required at any point during execution. This method aims to minimize the human effort required, thereby increasing the number of robots that can be guided in parallel by a single human operator.

Brief overview

We use an optimization-based approach that integrates human input to solve the problem of reaching through clutter. Our system starts tackling the problems fully autonomously and decides to ask for human help only when needed. In this way, our system is capable of solving trivial problems fully autonomously without any human intervention. Our system integrates optimization and execution in a unified online-replanning framework that constantly optimizes and executes the solution in the real-world robustly.

One property of trajectory optimization is that the convergence rate of the cost of a problem can indicate whether the solver can explore new solutions (i.e., more time is needed) or if the solver is stuck at a local minimum (i.e., immediate human input could be beneficial). We leverage this property to adaptively decide when to ask for human help based on the problem at hand.

If you are interested to learn more about this work, please read the manuscript using the link above.

Citing

If you have any questions, please feel free to drop a line. Finally, if you want to cite this work, please use the following:

@article{papallas2020ral,
  title={Online replanning with human-in-the-loop for non-prehensile manipulation in clutter—a trajectory optimization based approach},
  author={Papallas, Rafael and Cohn, Anthony G and Dogar, Mehmet R},
  journal={IEEE Robotics and Automation Letters},
  volume={5},
  number={4},
  pages={5377--5384},
  year={2020},
  publisher={IEEE}
}