Lei Wu (吴磊)
 |
Office: 智华楼372 |
About Me
I am an Assistant Professor in the School of Mathematical Sciences and Center for Machine Learning Research at Peking University.
Previously, I was a postdoctoral researcher at Princeton University and the University of Pennsylvania. I received my Ph.D. in Computational Mathematics from Peking University in 2018, advised by Prof. Weinan E, and my B.S. in Mathematics from Nankai University in 2012.
I aim to develop a predictive science of deep learning, connecting mathematical analysis of tractable models with empirical studies of large-scale training. My current focus is on understanding and improving LLM pretraining. See Research for details.
Recruiting
We welcome postdocs, PhD students, and undergraduates interested in the scientific foundations of AI. Please read my talk slides before contacting me. In your email, include your CV and a brief description of your research interests.
Recent Updates
2026-09: When and How signSGD Outperform SGD: A Functional Scaling Law Perspective accepted to NeurIPS 2026 (Spotlight).
2026-08: New preprints on effective learning rate (ELR) collapse and momentum in large-batch training are now available on arXiv.
2026-07: I taught a short course on Foundations of Deep Learning: Approximation, Generalization, and Scaling at 2026 Fudan University International Summer School on AI in Math.
2026-07: Co-organized the 2026 PKU Workshop on Mathematical Foundations of Modern Machine Learning with Fanghui Liu and Kun Yuan.
2026-06: Optimal Learning-Rate Schedules under Functional Scaling Laws: Power Decay and Warmup-Stable-Decay accepted to COLT 2026.
2026-06: GradPower: Powering Gradients for Faster Language Model Pre-Training accepted to ICML 2026.
2026-02: Smoothness Adaptivity in Constant-Depth Neural Networks: Optimal Rates via Smooth Activations released on arXiv.
2026-02: Fast Catch-Up, Late Switching: Optimal Batch Size Scheduling via Functional Scaling Laws accepted to ICLR 2026.
2025-09: Functional Scaling Laws in Kernel Regression: Loss Dynamics and Learning Rate Schedules accepted to NeurIPS 2025 (Spotlight).
2025-06: The Sharpness Disparity Principle in Transformers for Accelerating Language Model Pre-Training accepted to ICML 2025.
2025-02: A duality framework for analyzing random feature and two-layer neural networks accepted to Annals of Statistics.