pipette
ENEnglish

MoCKEFA: Model-Conditioned Keyframe Extraction and Fair Attribution for Video Model Explanation

Lahiru Wijayasingha

PreprintAfirmaciones fuertes, leer con cuidado

En palabras de los autores

We show that a video can be represented with just a few keyframes without sacrificing a given model's belief about the video. Our method is the first of its kind that extracts these keyframes conditioned on the model's prediction, which is massively helpful for building explainable video models. We then fairly distribute the importance of the video to the model among the keyframes with game theoretically fair Shapley values. Through qualitative and quantitative experiments, we show that keyframe attribution is superior to the traditional frame attribution. Further, we conduct preliminary exploration on highlighting the important changes among the frames by looking at the differences between adjacent keyframes.

Resultado principalLimitación que admiten los autores

Apareció: martes, 20 de octubre. arXiv. Preprint, todavía sin revisión por pares.