Action-Informed Estimation and Planning: Clearing Clutter on Staircases via Quadrupedal Pedipulation

Abstract

For robots to operate autonomously in densely cluttered environments, they must reason about and potentially physically interact with obstacles to clear a path. Safely clearing a path on challenging terrain, such as a cluttered staircase, requires controlled interaction. However, relying solely on exteroceptive (e.g., visual) feedback for interaction is challenging, as the robot's own body may occlude the object, and sensor measurements may be noisy. In this work, we propose an "action-informed" approach that tightly couples estimation and planning for quadrupedal robots to clear clutter on staircases via "pedipulation" (i.e., manipulation with a leg). Our action-informed estimation module uses proprioceptive (e.g., foot contact) feedback during an interaction to predict the object's displacement. This prediction serves as an action-informed prior to guide the perception system, enabling robust tracking even after partial pushes or brief occlusions. This interaction-aware state estimate is used by our motion planner to decide on subsequent actions, such as re-pushing, climbing, or treating an object as immovable. We show in hardware experiments that our interaction-aware system significantly outperforms an open-loop baseline in task success rate (e.g., 85% vs 35% on one task) and tracking accuracy, generalizing to various objects and environments.

Our Approach

Our system is built on a tight coupling between planning, estimation, and action. We propose a framework that robustly segments and localizes clutter on staircases by leveraging geometric priors of the environment

When the robot is commanded to clear an object, a planner computes collision-free foot paths combined with the 'movable primitives' for pushing, which are then executed by a pedipulation policy learned via reinforcement learning. We then fuse vision with proprioceptive contact feedback to track an object's state through a push, even when it's occluded or when the push is partial.

Results

Here are some results of our proposed work. Our algorithm generalizes to different types of objects and staircases.

Our perception-action coupling allows the robot to successfully track and re-plan after partial pushes, where the object doesn't fully clear the path.

The system can also detect immovable objects based on proprioception feedback during a push. It then updates its model and can re-plan a new path to continue its mission.