My journey in machine learning started with learning linear transforms from data, and I further developed the concept by learning nonlinear transforms via deep neural networks.

The data-driven tight frame construction for image denoising is an unsupervised learning approach for image restoration, and dictionary learning for sparse coding is a supervised learning approach for classification. The key feature of these algorithms is to learn a linear transform that provides a good sparse approximation of the underlying images. Both algorithms have been proven to converge.

A deep learning framework for nonlinear transforms and their inverses is a fully connected neural network between the encoder and decoder. This framework has applications in signal/image processing, predictive data science, and forward and inverse problems.

Batch normalization is one of the most important techniques for training deep neural networks. It has proven to be extremely effective in avoiding gradient blow-ups during back-propagation and speeding up convergence. When considering batch normalization as a simple transform, my curiosity about why it works motivated me to further understand deep learning.

Image Restoration

1. 1. Chenglong Bao, Hui Ji, Zuowei Shen, Convergence analysis for iterative data-driven tight frame construction scheme, Applied and Computational Harmonic Analysis, 38(3), (2015), 510-523. PDF
   2. Jianfeng Cai, Hui Ji, Zuowei Shen, Guibo Ye, Data-driven tight frame construction and image denoising, Applied and Computational Harmonic Analysis, 37(1), (2014), 89-105. PDF

Classification

1. 1. Chenglong Bao, Hui Ji, Yuhui Quan, Zuowei Shen, Dictionary learning for sparse coding: Algorithms and convergence analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(7), (2016), 1356-1369. PDF
   2. Chenglong Bao, Hui Ji, Yuhui Quan, Zuowei Shen, L0 norm based dictionary learning by proximal methods with global convergence, IEEE Conference Computer Vision and Pattern Recognition (CVPR), Columbus, (2014). PDF

Learning Nonlinear Transform

1. 1. Yong Zheng Ong, Zuowei Shen, Haizhao Yang, Integral Autoencoder Network for Discretization-Invariant Learning, Journal of Machine Learning Research, 23, (2022), PDF

Batach Normalization

1. 1. Yongqiang Cai, Qianxiao Li, Zuowei Shen, Quantitative analysis of the effect of batch normalization on gradient descent, Thirty-sixth International conference on machine learning (ICML), (2019) PDF