DescriptionThe ordinary linear model has been the bedrock of signal processing, statistics, and machine learning for decades. The last decade, however, has witnessed a marked transformation of this model: instead of the classical low-dimensional setting in which the sample size exceeds the number of features/predictors/variables, we are increasingly having to operate in the high-dimensional setting in which the number of variables far exceeds the sample size. Although such high-dimensional settings would ordinarily lead to ill-posed problems, the inference task has been studied under the rubric of high-dimensional statistical inference, where various notions of structure have been imposed on the model parameters to obtain unique solutions to the inference problem. While there are many statistical methods that guarantee unique solutions, these methods can easily become computationally prohibitive in ultrahigh-dimensional settings, in which the number of variables can scale exponentially with the sample size. In other cases, the traditional notions of structure on model parameters can be rather restrictive, especially when the variables naturally appear in the form of a multi-way array (tensor), as in the case of neuroimaging data analysis.
The purpose of this dissertation is to study inference using high-dimensional linear models for the cases when (i) the number of variables can scale exponentially with the number of samples, and (ii) the variables naturally form a tensor structure. Specifically, for each of these respective cases, the dissertation (i) proposes an efficient inference approach, (ii) provides high-probability performance guarantees for the proposed approach, and (iii) demonstrates efficacy of the inference approach in statistical analysis of real-world datasets.