2026-04-13T01:36:38Zhttps://recercat.cat/oai/request

oai:recercat.cat:2117/4302652025-07-23T05:44:48Zcom_2072_1033col_2072_452951

00925njm 22002777a 4500 dc Rabadessa Alcaide, Oriol author 2025-05-14 Multimodal Large Language Models (MLLMs) excel as zero-shot reasoners across diverse domains. However, their application to real-world classification tasks, particularly in direct comparison with specialized models, remains underexplored. This work explores how MLLMs can be leveraged for zero-shot Human-Object Interaction (HOI) recognition and detection using token probability outputs. We first benchmark lightweight MLLMs, identifying Qwen2-VL and MiniCPM-V as the most effective families for HOI. We perform a comprehensive comparison of zero-shot strategies applicable to this task. A taxonomy of zero-shot approaches is proposed, integrating textual and visual prompting strategies. Our analysis over the HICO dataset reveals that Objects as Context boosts performance for multi-image-capable MLLMs, while ensembling text prompts enhances robustness. On the HICO-DET and V-COCO datasets, Objects as Context, Black Other Objects, and Blur the Background emerge as superior visual prompting methods for localization. Our approach achieves 53.50 mAP on HICO and 23.69 mAP on HICO-DET, outperforming prior zero-shot methods and being competitive with the current state-of-the-art supervised models. Our code is made publicly available Àrees temàtiques de la UPC::Informàtica::Intel·ligència artificial::Llenguatge natural Natural language processing (Computer science) Benchmarking (Management) Computer software -- Verification Models Massius de Llenguatge Multimodals Detecció de Interaccions Persona-Objecte Reconeixement de Interaccions Persona-Objecte Enginyeria d'Instruccions Aprenentatge Zero-shot Zero-shot Learning Multimodal Large Language Models Human-Object Interaction Recognition Human-Object Interaction Detection Prompt Engineering Tractament del llenguatge natural (Informàtica) Referenciació (Economia) Programari--Verificació Multimodal large language models for zero-shot real-world classification tasks: benchmark, taxonomy of prompting methods, and application to human-object interaction recognition and detection