DescriptionData representation is an important information processing task which finds use in diverse engineering applications like signal processing, machine learning, medical imaging, and geophysical data analysis, to name a few. Once a good representation model is known for a given application then the next question is learning that model under practical constraints imposed by the application. Two such constraints are i) data is available at geographically distributed sites, and ii) streaming data. In such scenarios distributed, decentralized, and online algorithms can be deployed for solving data representation problems, which are the focus of this dissertation. Specifically, this thesis focuses on solving following three problems: i) Solution for PCA for high-rate streaming data, ii) collaborative dictionary learning for big, distributed data, and iii) through the wall radar imaging (TWRI) in distributed settings. This thesis proposes new methods to tackle challenges arising due to distributed and steaming nature of data, providing a theoretical analysis of the proposed methodologies (except TWRI), and finally using simulations to demonstrate the efficacy of the proposed methods