Continuous Wrist Control on the Hannes Prosthesis: a Vision-based Shared Autonomy Framework

Federico Vasile, Elisa Maiettini, Giulia Pasquale, Nicolò Boccardo, Lorenzo Natale
Italian Institute of Technology, University of Genoa
ICRA 2025
arXiv Code

TL;DR: We propose a shared autonomy framework for continuos wrist control,
aiming to improve the grasping motion of prosthetic hands
and reduce the cognitive load on the user.

Abstract

Most control techniques for prosthetic grasping focus on dexterous fingers control, but overlook the wrist motion. This forces the user to perform compensatory movements with the elbow, shoulder and hip to adapt the wrist for grasping. We propose a computer vision-based system that leverages the collaboration between the user and an automatic system in a shared autonomy framework, to perform continuous control of the wrist degrees of freedom in a prosthetic arm, promoting a more natural approach-to-grasp motion. Our pipeline allows to seamlessly control the prosthetic wrist to follow the target object and finally orient it for grasping according to the user intent. We assess the effectiveness of each system component through quantitative analysis and finally deploy our method on the Hannes prosthetic arm.

Visual Servoing Demonstration

We propose a visual servoing control to continuously orient the wrist towards the target object,
promoting a more natural approach-to-grasp motion.

(hover to play until the end)

Object Parts Segmentation

Our DINOv2Det model segments the object parts into top and side grasps.

Framework Overview

We propose a shared autonomy framework to closely mimic the kinematics of a reach-to-grasp task. During the Transport phase, we segment the object into parts using our DINOv2Det model and use a custom visual servoing (pp-IBVS) to orient the wrist towards the target object. Then, the user triggers the Rotation phase via EMG signals, and the wrist is engaged into Top or Side grasp based on the predicted object part. Finally, the user controls the closing of the fingers to grasp the object.

Framework Overview

EMG Baseline

We provide a qualitative comparison with a baseline using electromyography (EMG) signals for the control of the wrist and fingers. It adopts the Sequential Switching and Control paradigm, where the user drives each joint sequentially.
This method hinders a natural approach-to-grasp motion and increases the cognitive load on the user.

Visual Servoing Analysis

We show two experiments to highlight the differences between the standard Image Based Visual Servoing (s-IBVS)
and our proportional and partitioned Visual Servoing (pp-IBVS).
See Sec. V.B in the paper for more details.

BibTeX

@inproceedings{vasile2025continuous,
  title={Continuous Wrist Control on the Hannes Prosthesis: a Vision-based Shared Autonomy Framework},
  author={Vasile, Federico and Maiettini, Elisa and Pasquale, Giulia and Boccardo, Nicol{\`o} and Natale, Lorenzo},
  booktitle={2025 IEEE International Conference on Robotics and Automation (ICRA)},
  pages={},
  year={2025},
}