SelectSep: Target-Selective Video Queries for Audio Source Separation
En palabras de los autores
Video-queried sound separation (VQSS) aims to isolate a target sound from an audio mixture using its corresponding video as a query. However, scene-level video queries can become ambiguous when multiple sounding objects are visible, particularly when the target and interferer are semantically similar. We propose SelectSep, a target-selective VQSS framework that jointly uses text and a single click coordinate to specify the target source and instance without requiring segmentation at inference time. We adapt the visual encoder through target-feature distillation with text and click conditioning, and then fine-tune the separator using the adapted visual features. We also introduce an on-screen interference setting, where both the target and interferer are visible, to evaluate target selectivity. Experiments on standard benchmarks and the on-screen interference setting demonstrate consistent improvements in separation quality and target selectivity, particularly for mixtures containing semantically similar sources.
Apareció: viernes, 30 de octubre. arXiv. Preprint, todavía sin revisión por pares.
Comentario de los autores: 5 pages, 2 figures, 2 tables. Submitted to ICASSP 2027