Research

Research Vision

My research studies representation, optimization, and generalization in machine learning through the lens of scaling—how their interaction evolves as models, data, and compute grow.

• Representation (function spaces): Characterize neural networks via their induced function spaces, including expressivity, approximation, and inductive bias.

• Optimization (stochastic dynamics): Analyze training algorithms as high-dimensional stochastic dynamical systems, focusing on stability, implicit bias, and convergence.

• Generalization (statistical behavior): Quantify how generalization depends on representation and optimization, particularly in large-scale regimes.

Across these directions, I aim to develop a predictive science of learning that connects tractable models with real-world training, ranging from kernel methods and neural networks to modern LLM pretraining.

Research Highlights

(See also: Full publication list)

• Scaling laws:

  • Functional scaling laws (FSL) for kernel regression and LLM pretraining (NeurIPS 2025, Spotlight)

  • Understanding batch-size scheduling under FSL (ICLR 2026)

  • Optimal learning-rate scheduling under FSL (COLT 2026)

• Optimization:

  • A dynamical stability perspective of implicit regularization (NeurIPS 2018)

  • Stable minima must be flat due to anisotropic noise geometry (NeurIPS 2022)

  • Flat minima generalize well (ICML 2023)

• Representation:

  • Barron spaces and flow-induced function spaces (Constructive Approximation 2021)

  • Inductive biases in deep convolutional networks (NeurIPS 2023)

  • A duality framework for random features and two-layer networks (AoS 2025)

• Applications (theory to practice):

  • Accelerating transformer training via sharpness disparity (ICML 2025)