Article
Full entry |
PDF
(0.5 MB)
Feedback
PDF
(0.5 MB)
Feedback
Keywords:
information decomposition; unique information
information decomposition; unique information
Summary:
We study the unique information function $UI(T:X\setminus Y)$ defined by Bertschinger et al. within the framework of information decompositions. In particular, we study uniqueness and support of the solutions to the convex optimization problem underlying the definition of $UI$. We identify sufficient conditions for non-uniqueness of solutions with full support in terms of conditional independence constraints and in terms of the cardinalities of $T$, $X$ and $Y$. Our results are based on a reformulation of the first order conditions on the objective function as rank constraints on a matrix of conditional probabilities. These results help to speed up the computation of $UI(T:X\setminus Y)$, most notably when $T$ is binary. Optima in the relative interior of the optimization domain are solutions of linear equations if $T$ is binary. In the all binary case, we obtain a complete picture of where the optimizing probability distributions lie.
References:
[1] Amari, Shun-ichi, Nagaoka, Hiroshi: Methods of Information Geometry. American Mathematical Society 2000. DOI
[2] Banerjee, P. Kr., Olbrich, E., Jost, J., Rauh, J.: Unique informations and deficiencies. In: Proc. Allerton, 2018. DOI
[3] Banerjee, P. Kr., Rauh, J., Montúfar, G.: Computing the unique information. In: Proc. IEEE ISIT 2018, pp. 141-145 DOI
[4] Bertschinger, N., Rauh, J., Olbrich, E., Jost, J., Ay, N.: Quantifying unique information. Entropy 16 (2014), 4, 2161-2183. DOI
[5] Diaconis, Persi, Sturmfels, Bernd: Algebraic algorithms for sampling from conditional distributions. Ann. Statist. 26 (1998), 363-397.
[6] Fink, A.: The binomial ideal of the intersection axiom for conditional probabilities. J. Algebr. Combin. 33 (2011), 3, 455-463. DOI
[7] Finn, C., Lizier, J. T.: Pointwise partial information decomposition using the specificity and ambiguity lattices. Entropy 20 (2018), 4, 297. DOI
[8] Harder, M., Salge, Ch., Polani, D.: A bivariate measure of redundant information. Phys. Rev. E 87 (2013), 012130. DOI
[9] Ince, R.: Measuring multivariate redundant information with pointwise common change in surprisal. Entropy 19 (2017), 7, 318. DOI
[10] James, R., Emenheiser, J., Crutchfield, J.: Unique information via dependency constraints. J. Physics A 52 (2018), 1, 014002. DOI
[11] Makkeh, A., Theis, D. O., Vicente, R.: Bivariate partial information decomposition: The optimization perspective. Entropy 19 (2017), 10, 530. DOI
[12] Niu, X., Quinn, Ch.: A measure of synergy, redundancy, and unique information using information geometry. In: Proc. IEEE ISIT 2019. DOI
[13] Rauh, J., Banerjee, P. Kr., Olbrich, E., Jost, J.: Unique information and secret key decompositions. In: 2019 IEEE International Symposium on Information Theory (ISIT), pp. 3042-3046. DOI
[14] Rauh, J., Ay, N.: Robustness, canalyzing functions and systems design. Theory Biosciences 133 (2014), 2, 63-78. DOI
[15] Rauh, J., Bertschinger, N., Olbrich, E., Jost, J.: Reconsidering unique information: Towards a multivariate information decomposition. In: Proc. IEEE ISIT 2014, pp. 2232-2236. DOI
[16] Smith, N. A., R, Tromble, W.: Sampling Uniformly from the Unit Simplex. Technical Report 29, Johns Hopkins University 29, 2004.
[17] Williams, P., Beer, R.: Nonnegative decomposition of multivariate information. arXiv:1004.2515v1.
Search
Partner of
