Divergence, Scattering Ellipses, Correlation- Animation



With Mathematical Expectation, and Variance of the two-dimensional distribution set at fixed values, and expressing correlation between features as $\rho_{ij}=\frac{\sigma_{ij}}{\sigma_i\sigma_j}$, we have the fixed projections lengths as well. Thus, our class distributions are bounded with the rectangles with sides parallel to the feature axes, whose centers of gravities coincide with class means, and rectangles' side lengths are equal to the variance of ''parallel'' feature. With covariance matrices equal in both of the classes, both rectangle bounding boxes are equal. The only variable now is correlation coefficient $\rho$, and it doesn't influence the position, or size of bounding boxes. But, its value does influence the position of major axes of ellipses inscribed inside rectangles, namely, eccentricity and slope of the axes of the ellipses vary. With the increase of $\rho$ the major axes of the ellipses change the direction. Correlation coefficient, $\rho$ determines the slope of the linearity between the two features. Hence, for $\rho=0, slope\to\infty$, while for $\rho\to1, slope\to\frac{\pi}{4}$.

For different values of $\rho$, the degree of intersection of the scattering ellipses varies and so does the level of informativeness of the pair of features that constitute the feature vector, where it is assumed that the quantity of informativeness of each feature individually is invariant.

Area subtended by the intersection curve does not grow monotonically. It is getting larger until we reach the peak of redundancy, at $\rho_{min}$, and then it decreases towards the region of synergism, which starts at some $\rho_m=\frac{2\sqrt{J(x_1)J(x_2)}}{J(x_1)+J(x_2)}]$, and goes towards unity (regarding the value of correlation coefficient). Generally, for $\vert\rho\vert=1$ we have the (aforementioned) case of error free recognition. This is rather 'advantageous' situation, since we could place our linear boundary anywhere between the scattering ellipses (which are turned into lines for such a perfect value of $\rho$), parallel to the rectangles diagonals that actually represent degenerate ellipses. For this case ( $\rho=1, or\rho=-1$), features are so strongly connected that even if the knowledge of each of them individually has no informativeness at all this powerful relationship yields fantastic results!!. But, BOTH features are needed (check egg-plants and socks again).

Animated gifs (that are 'jumping' above) show you how does the area of intersection of scattering ellipses change with respect to the value of correlation coefficient. In this example, due to the symmetry inside the range of redundancy, correlation coefficient varies inside the interval for which divergence yields values from Jmin to infinity.




<< >>