Article
Full entry |
PDF
(1.2 MB)
Feedback
PDF
(1.2 MB)
Feedback
Keywords:
marginal problem; relative entropy; dependence structure simplification; explicit expression approximation; multiinformation; decomposable model; asteroid
marginal problem; relative entropy; dependence structure simplification; explicit expression approximation; multiinformation; decomposable model; asteroid
Summary:
Let $P$ be a discrete multidimensional probability distribution over a finite set of variables $N$ which is only partially specified by the requirement that it has prescribed given marginals $\lbrace P_{A};\ A\in {\cal S} \rbrace $, where ${\cal S}$ is a class of subsets of $N$ with $\bigcup {\cal S} = N$. The paper deals with the problem of approximating $P$ on the basis of those given marginals. The divergence of an approximation $\hat{P}$ from $P$ is measured by the relative entropy $H(P|\hat{P})$. Two methods for approximating $P$ are compared. One of them uses formerly introduced concept of dependence structure simplification (see Perez [Per79]). The other one is based on an explicit expression, which has to be normalized. We give examples showing that neither of these two methods is universally better than the other. If one of the considered approximations $\hat{P}$ really has the prescribed marginals then it appears to be the distribution $P$ with minimal possible multiinformation. A simple condition on the class ${\cal S}$ implying the existence of an approximation $\hat{P}$ with prescribed marginals is recalled. If the condition holds then both methods for approximating $P$ give the same result.
References:
[1] Csiszár I., Matúš F.: Information projections revisited. IEEE Trans. Inform. Theory 49 (2003), 1474–1490 MR 1984936 | Zbl 1063.94016
[2] Kellerer H. G.: Verteilungsfunktionen mit gegebenem Marginalverteilungen (in German, translation: Distribution functions with given marginal distributions). Z. Wahrsch. verw. Gerbiete 3 (1964), 247–270 MR 0175158
[3] Lauritzen S. L.: Graphical Models. Clarendon Press, Oxford 1996 MR 1419991 | Zbl 1055.62126
[4] Perez A.: $\varepsilon $-admissible simplifications of the dependence structure of random variables. Kybernetika 13 (1979), 439–449 MR 0472224
[5] Perez A.: The barycenter concept of a set of probability measures as a tool in statistical decision. In: The book of abstracts of the 4th Internat. Vilnius Conference on Probability Theory and Mathematical Statistics 1985, pp. 226–228
[6] Perez A.: Princip maxima entropie a princip barycentra při integraci dílčích znalostí v expertních systémech (in Czech, translation: The maximum entropy principle and the barycenter principle in partial knowledge integration in expert systems). In: Metody umělé inteligence a expertní systémy III (V. Mařík and Z. Zdráhal, eds.), ČSVT – FEL ČVUT, Prague 1987, pp. 62–74
[7] Perez A.: Explicit expression Exe – containing the same multiinformation as that in the given marginal set – for approximating probability distributions. A manuscript in Word, 2003
[8] Studený M.: Pojem multiinformace v pravděpodobnostním rozhodování (in Czech, translation: The notion of multiinformation in probabilistic decision-making). CSc Thesis, Czechoslovak Academy of Sciences, Institute of Information Theory and Automation, Prague 1987
[9] Studený M.: Probabilistic Conditional Independence Structures. Springer–Verlag, London 2005
Search
Partner of
