Entropy of a Mixture
Entropy of a Mixture Given a pair ( p 0 , p 1 ) (p_0, p_1) (p0,p1) of probability density functions and an interpolation factor λ ∈ [ 0 , 1 ] , \lambda \in [0, 1], λ∈[0,1], consider the mixture p λ = ( 1 − λ ) p 0 + λ p 1 . p_\lambda = (1 - \lambda) p_0 + \lambda p_1. pλ=(1−λ)p0+λp1. How does the entropy H ( p λ ) = − E X ∼ p λ ln p λ ( X ) H(p_\lambda) = -\E_{X \sim p_\lambda} \ln p_\lambda(X) H(pλ)=−EX∼pλlnpλ(X) of this mixture vary as a function of λ \lambda λ? The widget below