TY - JOUR TI - Improving application infrastructure provisioning using resource usage predictions from cloud metric data analysis DO - https://doi.org/doi:10.7282/t3-y8e4-5v69 PY - 2018 AB - There has been a huge interest by companies to utilize the cloud for their day-to-day operations. Cloud providers like AWS, Microsoft Azure, Google have been quite successful in serving its ever-increasing customer base. It is interesting to study how these companies use the cloud metrics to efficiently schedule their customers’ jobs and thereby utilize the shared infrastructure effectively. A lot of research has been done with the Google cloud cluster data released publicly in 2011 to analyze the task and job failure rates and predict failures thereby optimizing the resource utilization by smart scheduling techniques. 6 years from then, Microsoft Azure has also released their VM CPU utilization data publicly in October 2017 along with the SOSP 2017 paper called “Resource Central”. We will be one of the first to analyze this data set. In this work, we analyze this data and try to answer the following questions: 1. What are the VM CPU usage patterns by Azure subscribers? 2. Can we predict the future usage if yes, how and who all can benefit from this data? 3. Which techniques among statistical machine learning and deep learning are most suited to the Microsoft Azure data? 4. Can the learning models so formed be generalized for other similar data sets and problems like anomaly detection using log analysis at the application level? 5. How can these models augment the performance of existing VM scheduling algorithms? KW - Electrical and Computer Engineering KW - Web services KW - Cloud computing LA - eng ER -