pipette
ENEnglish

AdaFrame: Hierarchical Multimodal Deduplication with Adaptive Information Budgeting for Cost-Efficient Video Advertisement Analysis

Abdul Basit Tonmoy, Arman Luthra, Guang Lin, Qixin Deng

PreprintUso en el mundo real

En palabras de los autores

Uniform frame sampling for vision-language model (VLM)-based video analysis often floods multimodal backbones with redundant content, while existing keyframe selection methods are typically optimized for human viewing rather than downstream extraction accuracy. We propose AdaFrame, a content-adaptive preprocessing cascade that reduces visual redundancy without sacrificing extraction fidelity. Three progressively finer filters, perceptual hashing, learned perceptual image patch similarity (LPIPS), and CLIP-based semantic clustering, efficiently eliminate redundant frames before costly cross-modal inference. Per-video frame budgets are determined using Intrinsic Semantic Dimensionality (ISD), which estimates each video's semantic complexity through singular value decomposition of its frame embeddings. Experiments on 500 video advertisements from the Pitt Ads Dataset show that AdaFrame reduces the frame count by 48 percent and VLM inference cost by 75 percent compared with uniform sampling, while achieving 86.0 percent topic accuracy, an improvement of 1.6 percentage points, and 79.4 percent sentiment accuracy, an improvement of 7.5 percentage points.

Resultado principalEl resumen no menciona limitaciones.

Apareció: jueves, 15 de octubre. arXiv. Preprint, todavía sin revisión por pares.