ResearchI work toward a predictive science of deep learning: explaining learning mechanisms, predicting training dynamics, and guiding algorithm design. My research moves between mathematical analysis of tractable models and large-scale experiments, with each informing and challenging the other. LLM Pretraining: Modeling, Prediction, and DesignWe seek a macroscopic description of LLM pretraining through effective equations that capture the mechanisms governing loss dynamics. This approach helps explain training behavior and improve training efficiency.
For more on FSL, see my talk slides. SGD Dynamics: Stability, Sharpness, and GeneralizationWhy does SGD always find minima that generalize well, even when the model is overparameterized? We introduced dynamical stability as a mechanism for SGD's implicit regularization. Our work links stability constraints and SGD's noise geometry to minima selection, sharpness, and generalization.
For an overview of this line of work, see my talk slides. Neural Network Theory: Approximation and EstimationWe introduced and named Barron spaces for two-layer neural networks, establishing approximation and generalization results within this function-space framework. Our broader work studies how neural networks efficiently learn high-dimensional functions and when and why they outperform linear methods.
For more on approximation–estimation duality, see my talk slides. |