A Novel Approach to the Partial Information Decomposition
Abstract
We consider the "partial information decomposition" (PID) problem, which aims to decompose the information that a set of source random variables provide about a target random variable into separate redundant, synergistic, union, and unique components. In the first part of this paper, we propose a general framework for constructing a multivariate PID. Our framework is defined in terms of a formal analogy with intersection and union from set theory, along with an ordering relation which specifies when one information source is more informative than another. Our definitions are algebraically and axiomatically motivated, and can be generalized to domains beyond Shannon information theory (such as algorithmic information theory and quantum information theory). In the second part of this paper, we use our general framework to define a PID in terms of the well-known Blackwell order, which has a fundamental operational interpretation. We demonstrate our approach on numerous examples and show that it overcomes many drawbacks associated with previous proposals.
- Publication:
-
Entropy
- Pub Date:
- March 2022
- DOI:
- 10.3390/e24030403
- arXiv:
- arXiv:1908.08642
- Bibcode:
- 2022Entrp..24..403K
- Keywords:
-
- partial information decomposition;
- redundancy;
- synergy;
- Computer Science - Information Theory;
- Quantitative Biology - Neurons and Cognition;
- Statistics - Machine Learning
- E-Print:
- Entropy, 2022