npj Comp Mat

Here are some recent published papers grouped roughly in three directions

Approximation Theory for deep learning

A Brief Survey on the Approximation Theory for Sequence Modelling: In this recent review paper, we survey past and present works on the approximation theory of modelling sequential relationships, including those using recurrent networks, convolutional networks, encoder-decoder networks, and the transformers family of architectures. This should serve as a starting point for researchers who are interested in problems on the intersection of approximation theory, nonlinear dynamics and sequence modelling (chatGPT!). This is published in the newly established journal, Journal of Machine Learning.
Forward and Inverse Approximation Theory for Linear Temporal Convolutional Networks: In this paper, we further develop the approximation theory of temporal convolutional architectures, including a refined Jackson-type estimate for the approximation error, as well as a new Bernstein-estimate. This furthers the understanding of the structural bias of temporal (dilated) convolutional networks for sequence modelling. This will be presented as an oral talk at GSI 2023.
Approximation Analysis of Convolutional Neural Networks: In this work, we establish some approximation results for convolutional networks, paying particular attention to the compositional structure and how it can lead to (efficient) approximation. This is published in the East Asian Journal on Applied Mathematics.

Machine learning + scientific computing + control

Principled Acceleration of Iterative Numerical Methods Using Machine Learning: In this paper, we investigate the application of meta-learning type approaches to speed up iterative algorithms in scientific computing. An example is guessing the initial condition for a Jacobi solver (say for the Poisson equation as a sub-step in a Navier-Stokes equation solver). We show that a naive application of meta-learning (MAML algorithm) does not necessarily lead to gains in performance – contrary to what has been suggested in many recent empirical works. We concretely investigate this phenomena through analytical examples, and propose principled solutions to this dilemma. This work is published in ICML 2023.
Fairness In a Non-Stationary Environment From an Optimal Control Perspective: In this work, we connect the fairness problem in machine learning with control theory: in particular, ensuring that a machine learning model is fair to all demographic groups in a changing environment can be understood as an optimal control problem – in particular some sort of stabilisation problem. This allows us to design control strategies to promote fairness dynamically. This work is presented as a workshop paper in ICML 2023.

Machine learning + materials science

Knowledge-integrated machine learning for materials: lessons from gameplaying and robotics: In this perspective paper, we review and envision the fusion of machine learning and materials science research, making parallels to the development of game playing and robotics. This is published in Nature Reviews Materials.
Fast Bayesian optimization of Needle-in-a-Haystack problems using zooming memory-based initialization (ZoMBI): In this paper, we propose a modified Bayesian optimisation method that are shown to be effective for a class of optimisation problems involving the inverse design for materials discovery. This is the full publication corresponding to our earlier neurIPS workshop paper, and is now published in npj Computational Materials.
Tackling data scarcity with transfer learning: a case study of thickness characterization from optical spectra of perovskite thin films: In this paper, we tackle the data-scarcity issue in the application of machine learning in experimental sciences. In particular, we investigate the characterisation of thickness in perovskite thin films using data-driven techniques, yielding a transfer-learning framework that can be used for more general problems. This work is published in Digital Discovery.

Here are a number of recent accepted/published papers in machine learning and computer vision:

Approximation Theory of Convolutional Architectures for Time Series Modelling: In this paper, we develop some approximation theory for convolutional based architectures for time series analysis – with WaveNet as a prime example. This can be seen as a parallel for the approach taken in our previous paper, but this time for convolutional networks instead of recurrent networks. Our key finding here is that convolutional structures exploits certain “effective low rank” structures for efficient approximation, which can be very different from the “exponentially decaying memory structures” that RNN brings. This paper will appear at ICML 2021.
Adversarial Invariant Learning: In this paper, we develop methods to use adversarially chosen data splits to tackle the out-of-distribution generalization problems. This paper is published at CVPR 2021.

We also have a number of recent papers on the application of machine learning to science and engineering:

A Data Driven Method for Computing Quasipotentials: In this paper, we develop a efficient method for computing the Freidlin-Wentzell quasipotential for rare event analysis, especially in high dimensions. The idea is based on learning a decomposition of the force field, from which the quasipotential can be identified. This will appear in MSML 2021.
Two-step machine learning enables optimized nanoparticle synthesis: In this paper, we develop inverse design methodologies for nanoparticles of desired properties, through a combination of Bayesian optimization and neural network based methods. This paper is published at Npj Computational Materials.
Machine learning and high-throughput robust design of P3HT-CNT composite thin films for high electrical conductivity: In this paper, we investigate how to design carbon nanotubes with desired conductive properties through machine learning. The key difficulty here is the large amount of experimental noise, and we overcome this issue by developing a graph-based regression approach that takes more experimental information into account. This paper will appear in Advanced Functional Materials.

Qianxiao Li (李千骁)