Roy et al1 provides the following definition for the effective rank:
Consider a matrix with a singular value decomposition (SVD)
where D is a diagonal matrix with the singular values
Definition 1 (Nuclear norm) The nuclear norm of the matrix is defined as
Then the sequence of singular values gives rise to a nominal distribution
Definition 2 (Effective rank) The effective rank of a matrix is defined as the volume
where is the Shannon entropy of the nominal distribution given by the matrix spectrum.
Entropy of A Continuous Spectrum
The effective rank given by Roy et al is only defined on discrete spectrum. We know that computing the entropy of a continuous distribution require special care2.
Spectral Entropy in Finite Sampling Window
An additional consideration when computing the spectral entropy on a discrete spectrum, is that the windowing function produced by applying discrete Fourier transform in a finite data window imprints a spectral floor that contributes to the spectral distribution. We can subtract the spectral entropy of this windowing function from the spectral entropy of the discrete Fourier spectrum, s.t. the entropy of the discrete spectrum is when there is just a single component.
Leave the Acknowledgements here.
- Roy, O. and Vetterli, M. (2007) ‘The effective rank: A measure of effective dimensionality’, in 2007 15th European Signal Processing Conference, pp. 606–610. Available at: http://ieeexplore.ieee.org/document/7098875/↩
- Ge Yang, 'Differential Entropy and Invariant Entropy Measure', Available at: https://www.episodeyang.com/blog/2022/10-23/entropy_continuous/↩