Recent Preprints
-
Learning the identity: a case study of how SGD selects among functional decompositions
Andy Arditi, Weian Xie, David Bau, Liu Ziyin
Preprint 2026
[arXiv] -
Connectome-based biophysical modeling of a figure-ground discrimination circuit
Liu Ziyin, Yifei Lu, Alexander Borst, Sven Dorkenwald, Tomaso Poggio
Preprint 2026
[bioRxiv] -
Width-Independent Compressibility of Deep Neural Networks
Hong-Yi Wang, Mingze Wang, Liu Ziyin
Preprint 2026
[arXiv] -
Neural Quadratic Forms: A Unified Minimal Model for Sudden Learning and Scaling Laws
Liu Ziyin, Yizhou Xu, Tomaso Poggio, Isaac Chuang
Preprint 2026
[arXiv] -
Thermodynamic Irreversibility of Training Algorithms
Liu Ziyin, Yuanjie Ren, Adam Levine, Isaac Chuang
Preprint 2026
[arXiv] -
Does SGD Seek Flatness or Sharpness? An Exactly Solvable Model
Yizhou Xu, Pierfrancesco Beneventano, Isaac Chuang, Liu Ziyin
Preprint 2026
[arXiv] -
An Equivariance Toolbox for Learning Dynamics
Yongyi Yang, Liu Ziyin
Preprint 2025
[arXiv] -
Does Feedback Alignment Work at Biological Timescales?
Marc Gong Bacvanski, Liu Ziyin, Tomaso Poggio
Preprint 2025
[arXiv] -
Heterosynaptic Circuits Are Universal Gradient Machines
Liu Ziyin, Isaac Chuang, Tomaso Poggio
Preprint 2025
[arXiv]
Tutorial / Notes
Proof of a perfect platonic representation hypothesis (2025)
Parameter Symmetry Potentially Unifies Deep Learning Theory (2025)
Publications
-
How Biological Synapses Self-Assemble Gradient Learning
Qianli Liao*, Liu Ziyin*, Yulu Gan*, Brian Cheung, Mark T. Harnett, Tomaso Poggio
Proceedings of the National Academy of Sciences 123 (41), e2606245123 (2026)
[paper] [arXiv] -
Topological Invariance and Breakdown in Learning Dynamics
Yongyi Yang, Tomaso Poggio, Isaac Chuang, Liu Ziyin*
NeurIPS 2026
[arXiv] -
On the Invariance and Generality of Neural Scaling Laws
Xing Han, Liu Ziyin, Suchi Saria, Paul Pu Liang
NeurIPS 2026
[arXiv] -
When Reasoning Meets Its Laws
Junyu Zhang, Yifan Sun, Tianang Leng, Jingyan Shen, Liu Ziyin, Paul Pu Liang, Huan Zhang
NeurIPS 2026
[arXiv] -
Machine Learning Decoding of Circuit-Level Noise for Bivariate Bicycle Codes
John Blue, Harshil Avlani, Zhiyang He, Liu Ziyin, Isaac L. Chuang
Quantum 10, 2149 (2026)
[arXiv] -
Ubiquity of Emergent Hebbian Dynamics in Regularized Learning
David Koplow, Tomaso Poggio, Liu Ziyin
ICML 2026
[arXiv] -
A universal compression theory for lottery ticket hypothesis and neural scaling laws
Hong-Yi Wang, Di Luo, Tomaso Poggio, Isaac L. Chuang, Liu Ziyin*
ICLR 2026
[arXiv] -
Neural Thermodynamics: Entropic Forces in Deep and Universal Representation Learning
Liu Ziyin*, Yizhou Xu*, Isaac Chuang
NeurIPS 2025
[arXiv] -
Noise Balance and Stationary Distribution of Stochastic Gradient Descent
Liu Ziyin*, Hongchao Li*, Masahito Ueda
Physical Review E 111, 065303 (2025)
[paper] [arXiv] -
Compositional Generalization via Forced Rendering of Disentangled Latents
Qiyao Liang, Daoyuan Qian, Liu Ziyin, Ila Fiete
ICML 2025
[arXiv] -
Understanding the Emergence of Multimodal Representation Alignment
Megan Tjandrasuwita, Chanakya Ekbote, Liu Ziyin, Paul Pu Liang
ICML 2025
[arXiv] -
Formation of Representations in Neural Networks
Liu Ziyin, Isaac Chuang, Tomer Galanti, Tomaso Poggio
ICLR 2025 (spotlight: 5% of all submissions)
[paper] -
Remove Symmetries to Control Model Expressivity and Improve Optimization
Liu Ziyin*, Yizhou Xu*, Isaac Chuang
ICLR 2025
[paper] -
Three Mechanisms of Feature Learning in a Linear Network
Yizhou Xu*, Liu Ziyin*
ICLR 2025
[paper] -
Parameter Symmetry and Noise Equilibrium of Stochastic Gradient Descent
Liu Ziyin, Mingze Wang, Hongchao Li, Lei Wu
NeurIPS 2024
[arXiv] -
Symmetry Induces Structure and Constraint of Learning
Liu Ziyin
ICML 2024
[arXiv] -
Zeroth, first, and second-order phase transitions in deep neural networks
Liu Ziyin, Masahito Ueda
Physical Review Research 2023
[arXiv] -
Exact Solutions of a Deep Linear Network
Liu Ziyin, Botao Li, Xiangming Meng
Journal of Statistical Mechanics: Theory and Experiment, 2023
[arXiv] -
On the stepwise nature of self-supervised learning
James B. Simon, Maksis Knutins, Liu Ziyin, Daniel Geisz, Abraham J. Fetterman, Joshua Albrecht
ICML 2023
[arXiv] -
Sparsity by Redundancy: Solving L1 with SGD
Liu Ziyin*, Zihao Wang*
ICML 2023
[arXiv] -
What shapes the loss landscape of self-supervised learning?
Liu Ziyin, Ekdeep Singh Lubana, Masahito Ueda, Hidenori Tanaka
ICLR 2023
[arXiv] -
Exact Solutions of a Deep Linear Network
Liu Ziyin, Botao Li, Xiangming Meng
NeurIPS 2022
[arXiv] -
Posterior Collapse of a Linear Latent Variable Model
Zihao Wang*, Liu Ziyin*
NeurIPS 2022 (oral: 1% of all submissions)
[arXiv] -
Universal Thermodynamic Uncertainty Relation in Non-Equilibrium Dynamics
Liu Ziyin, Masahito Ueda
Physical Review Research (2022)
[arXiv] -
Theoretically Motivated Data Augmentation and Regularization for Portfolio Construction
Liu Ziyin, Kentaro Minami, Kentaro Imajo
ICAIF 2022 (3rd ACM International Conference on AI in Finance)
[arXiv] -
Power Laws and Symmetries in a Minimal Model of Financial Market Economy
Liu Ziyin, Katsuya Ito, Kentaro Imajo, Kentaro Minami
Physical Review Research (2022)
[arXiv] -
Logarithmic landscape and power-law escape rate of SGD
Takashi Mori, Liu Ziyin, Kangqiao Liu, Masahito Ueda
ICML 2022
[arXiv] -
SGD with a Constant Large Learning Rate Can Converge to Local Maxima
Liu Ziyin, Botao Li, James B. Simon, Masahito Ueda
ICLR 2022 (spotlight: 5% of all submissions)
[arXiv] -
Strength of Minibatch Noise in SGD
Liu Ziyin*, Kangqiao Liu*, Takashi Mori, Masahito Ueda
ICLR 2022 (spotlight: 5% of all submissions)
[arXiv] -
On the Distributional Properties of Adaptive Gradients
Zhang Zhiyi*, Liu Ziyin*
UAI 2021
[arXiv] -
Noise and Fluctuation of Finite Learning Rate Stochastic Gradient Descent
Kangqiao Liu*, Liu Ziyin*, Masahito Ueda
ICML 2021
[arXiv] -
Cross-Modal Generalization: Learning in Low Resource Modalities via Meta-Alignment
Paul Pu Liang*, Peter Wu*, Liu Ziyin, Louis-Philippe Morency, Ruslan Salakhutdinov
ACM Multimedia 2021
NeurIPS 2020 Workshop on Meta Learning
[arXiv] [code] -
Neural Networks Fail to Learn Periodic Functions and How to Fix It
Liu Ziyin, Tilman Hartwig, Masahito Ueda
NeurIPS 2020
[arXiv] -
Deep Gamblers: Learning to Abstain with Portfolio Theory
Liu Ziyin, Zhikang Wang, Paul Pu Liang, Ruslan Salakhutdinov, Louis-Philippe Morency, Masahito Ueda
NeurIPS 2019
[paper] [arXiv] [code] -
Think Locally, Act Globally: Federated Learning with Local and Global Representations
Paul Pu Liang*, Terrance Liu*, Liu Ziyin, Ruslan Salakhutdinov, Louis-Philippe Morency
NeurIPS 2019 Workshop on Federated Learning (oral, distinguished student paper award)
[workshop] [arXiv] [code] -
Multimodal Language Analysis with Recurrent Multistage Fusion
Paul Pu Liang, Ziyin Liu, Amir Zadeh, Louis-Philippe Morency
EMNLP 2018 (oral presentation)
[paper] [arXiv]