Text

theses

The unprecedented rate at which data is being created and stored calls for scalable optimization techniques that allow e cient Big Data" analysis. In this work where there is only one computing node, that modi es the coordinate-sampling distribution for stochastic coordinate descent: we call this proportional stochastic coordinate descent (PSCD). This method treats the gradient of the function as a probability distribution to sample the coordinates, and may be useful in so-called lock-free decentralized optimization schemes. Although stochastic coordinate descent methods seem attractive due to their small per-iteration complexity, they show high variance in performance compared to full gradient descent algorithms. In order to address this issue we propose stochastic variance reduced coordinate descent that takes information from the previous gradient estimates into account. Lastly, we consider stochastic message passing algorithms that limit the communication required for decentralized and distributed convex optimization and provide convergence guarantees on the objective value. For general distributed optimization in which agents jointly minimize the sum of local objectives we propose treating the iterates as gradients and propose a stochastic coordinate-wise primal averaging algorithm for optimization.

ETD_7301

M.S.

Includes bibliographical references

by Mohsen Ghassemi

doi:10.7282/T3M90BTP

ETD graduate

The author owns the copyright to this work.