The Information Bottleneck is NP-hard
In the authors' words
Given a joint distribution on a finite alphabet, the information bottleneck method seeks a representation of that is as informative as possible about while compressing , where and are conditionally independent given . We show that computing the optimal value of the information bottleneck problem, as well as its Lagrangian relaxation , is NP-hard. Moreover, unless , neither nor can be approximated in polynomial time within an additive error of order .
Main resultThe abstract does not state a limitation.
Appeared: Friday, October 16. arXiv. Preprint, not yet peer-reviewed.
Authors' comment: Preliminary version