Effective Rank and Continuous Spectrum

# Effective Rank and Continuous Spectrum

written by Ge Yang

# Effective Rank

Roy et al1 provides the following definition for the effective rank:

Consider a matrix $A$ with a singular value decomposition (SVD)

$A = U D V$

where D is a $M \times N$ diagonal matrix with the singular values

$\sigma_1 \geq \sigma_2 \geq \cdots \geq \sigma_Q \geq 0$

where $Q = \min\{M, N\}$.

Definition 1 (Nuclear norm) The nuclear norm of the matrix $A$ is defined as

$\Vert \sigma \Vert_1=\sum_{i=1}^Q \sigma_i.$

Then the sequence of singular values gives rise to a nominal distribution

$p_i = \frac{\sigma_i}{\Vert \sigma \Vert_1} \text{ for } i \in \{1, 2, \dots, Q\}$

Definition 2 (Effective rank) The effective rank of a matrix $A$ is defined as the volume

$\mathrm{erank}(A) = \exp{H\left(\frac{\sigma_i}{\Vert \sigma \Vert_1}\right)}.$

where $H$ is the Shannon entropy of the nominal distribution given by the matrix spectrum.

# Entropy of A Continuous Spectrum

The effective rank given by Roy et al is only defined on discrete spectrum. We know that computing the entropy of a continuous distribution require special care2.

# Spectral Entropy in Finite Sampling Window

An additional consideration when computing the spectral entropy on a discrete spectrum, is that the windowing function produced by applying discrete Fourier transform in a finite data window imprints a spectral floor that contributes to the spectral distribution. We can subtract the spectral entropy of this windowing function from the spectral entropy of the discrete Fourier spectrum, s.t. the entropy of the discrete spectrum is $0$ when there is just a single component.

$H(S) = H(S) - H(w)$

Acknowledgements:

Leave the Acknowledgements here.

# References

1. Roy, O. and Vetterli, M. (2007) ‘The effective rank: A measure of effective dimensionality’, in 2007 15th European Signal Processing Conference, pp. 606–610. Available at: http://ieeexplore.ieee.org/document/7098875/
2. Ge Yang, 'Differential Entropy and Invariant Entropy Measure', Available at: https://www.episodeyang.com/blog/2022/10-23/entropy_continuous/