A Baseline Study and Benchmark for Few-Shot Open-Set Action Recognition with Feature Residual Discrimination

Choose a dataset between SSv2 and NTURGBD and check some qualitative comparison between the baseline Softmax and its improvement FR-Disc for open-set tasks.

The accept score is a percentage value on the confidence of the prediction, that is \( \hat{u}_i \) in the figure below.

The baseline tends to assign higher confidence scores to unknown queries, while FR-Disc produces lower scores, indicating better discrimination.

Few-Shot Action Recognition (FS-AR) has shown promising results but is often limited by a closed-set assumption that fails in real-world open-set scenarios.

While Few-Shot Open-Set (FSOS) recognition is well-established for images, its extension to spatio-temporal video data remains underexplored.

To address this, we propose an architectural extension based on a Feature-Residual Discriminator (FR-Disc), adapting previous work on skeletal data to the more complex video domain.

Extensive experiments on five datasets demonstrate that while common open-set techniques provide only marginal gains, our FR-Disc significantly enhances unknown rejection capabilities without compromising closed-set accuracy, setting a new state-of-the-art for FSOS-AR.