I am enrolled in the Ph.D. program provided by the Ritchie School of Engineering and Computer Science. My Advisor will be Professor Zhihui Zhu. He is an assistant professor in the Department of Electrical and Computer Engineering. He has received a grant from the National Science Foundation (NSF) for "Collaborative Research: CIF: Small: Deep Sparse Models: Algorithms and Analysis". My Ph.D. programme is not coursework only. I have extra duties. Basically, these duties are developing, testing, and setting up research tasks as outlined by my advisor. During one academic year, I have to complete my duties and coursework for about 20 hours per week. The name of my project is “Collaborative Research: CIF: Small: Deep Sparse Models: Algorithms and Analysis”. I have received a full-pride GRA scholarship. GRA stands for the Global Research Alliance. The research fund will be financially supported by the NSF Division of Computing and Communication Foundations (Collaborative with J. Sulam at Johns Hopkins University). The following sections briefly describes my future research project.
Research Abstract:
Deep convolutional networks have significantly promoted the development of deep learning tools. Deep learning often includes multiple layers of processing units and supervised or unsupervised learning of feature representations in each layer. In the traditional approach, only fixed extractors will be used to gain expected outcomes. Deep learning approaches can use trainable feature extractors to develop the trainable classifier. In this sense, machines can perform very complicated tasks based on the self-learning process. At present, much evidence proves that machine-learning tools are extremely successful in obtaining state-of-the-art performance. But the research gap is that the theoretical understanding of the fundamental ideas behind the deep convolutional networks are still not available despite the fact that some machine learning algorithm are very impressive. Thus, this project aims to apply multi-layer sparse models to theoretically understand or optimize deep learning approaches.
Research Proposal:
The project is proposed to broaden researchers’ understanding about the appplication of the multi-layer sparse model on the deep convolutional neural networks. It can potentially advance the state-of-the-art in generalized sparse models of different numbers of layers. Provable and efficient optimization models can also be derived for the inverse problems associated with multi-layer sparse models. Models play a very critical role in signal and image processing and machine learning.
My research focuses on the interaction among the fields of signal processing, data analysis, and machine learning, using tools from optimization, approximation theory, and harmonic analysis. I am especially interested in efficient and reliable methods for extracting useful information in large-scale and high-dimensional signals and data. A few current projects include deep neural networks for unsupervised learning and inverse problems; landscape analysis of (nonsmooth) nonconvex optimizations; efficient and provably correct algorithms in the offline, stochastic, or distributed settings by exploiting the geometric properties; theoretically understanding of deep learning.
Research Significance and Anticipated Outcome:
One anticipated outcome is that after conducting the research, I should be able to theoretically describe the convolutional-sparse-coding model and under the deep learning process by presenting a multi-layer extension. Besides, I could design and analyze high-dimensional and large-scale optimization problems. Hopefully, the research team can identify the expressivity and generalization of a deep neural network, and find its application for inverse problems