Penalized M-estimation of covariance matrices: geodesic convexity and non-smooth penalties
Description
TitlePenalized M-estimation of covariance matrices: geodesic convexity and non-smooth penalties
Date Created2019
Other Date2019-05 (degree)
Extent1 online resource (x, 119 pages) : illustrations
DescriptionMany multivariate statistical methods are fundamentally related to the estimation of covariance matrices. The usual sample estimate of the covariance matrix $Sigma$ is not well-conditioned, when the sample size $n$ is small, and can be highly variable when the dimension $q$ is of the same order as $n$. A common approach to deal with an insufficient data scenario is to use regularization or penalization. Previous authors have proposed a range of regularization methods based on the knowledge that the normal likelihood is convex in $Sigma^{-1}$, e.g. the Graphical lasso. However, a lesser known property of the normal likelihood function is that it is geodesicially convex both in $Sigma$ and in $Sigma^{-1}$. Based on the notion of geodesic convexity, or g-convexity for short, we study a family of orthogonally invariant and scale invariant penalty functions that yield unique and well-behaved solutions to the penalized sample covariance problem. The concept of g-convexity can be mathematically challenging, and in practice, it can be difficult to prove a function is g-convex. We present a theorem that simplifies the proof of g-convexity for orthogonally invariant functions.
When eigenvalues are not well separated, one may not wish to consider the individual eigenvector, but rather their joint eigenspace. That is, the subspace spanned by the eigenvectors associated with a group of eigenvalues. In this dissertation, we propose a class of non-smooth g-convex penalty functions that can automatically partition the eigenvalues into groups. By using model cross validation, our method simultaneously selects the correct multiplicities of the eigenvalues, as well as gives a good estimate of the true covariance matrix. We also show some model consistency results in both the classical asymptotic setting, i.e. $q ext{ fixed }, noinfty$ and the general asymptotic setting, i.e. $n,q oinfty$.
To solve the non-robustness problem of the sample covariance matrix, we extend the penalization method to the M-estimators of a scatter matrix. In particular, we consider a class of g-convex loss functions that gives rise to the monotone M-estimators. Thus, when adding a g-convex penalty term, we are able to study the existence and uniqueness of the penalized solution. For computation, we propose a simple and proven convergent iteratively reweighting algorithm for any g-convex penalized or constrained M-estimators of scatter. We also show that the penalized M-estimators of scatter can have a high breakdown point. Finally, we propose a new class of M-estimator of scatter, and show its limiting version yields the general spatial sign covariance matrices, as the tuning parameter goes to infinity.
NotePh.D.
NoteIncludes bibliographical references
Genretheses, ETD doctoral
LanguageEnglish
CollectionSchool of Graduate Studies Electronic Theses and Dissertations
Organization NameRutgers, The State University of New Jersey
RightsThe author owns the copyright to this work.
Statistical Profile