Liu Ziyin

Back

Recent Preprints

  1. Heterosynaptic Circuits Are Universal Gradient Machines
    2. Liu Ziyin, Isaac Chuang, Tomaso Poggio
    Preprint 2025
    [arXiv]
  2. Parameter Symmetry Breaking and Restoration Determines the Hierarchical Learning in AI Systems
    3. Liu Ziyin, Yizhou Xu, Tomaso Poggio, Isaac Chuang
    Preprint 2025
    [arXiv]
  3. Self-Assembly of a Biologically Plausible Learning Circuit
    4. Qianli Liao*, Liu Ziyin*, Yulu Gan*, Brian Cheung, Mark Harnett, Tomaso Poggio
    Preprint 2024
    [arXiv]

Tutorial / Notes

Proof of a perfect platonic representation hypothesis (2025)

Publications

  1. Emergence of Hebbian Dynamics in Regularized Non-Local Learners
    2. David Koplow, Tomaso Poggio, Liu Ziyin
    ICML 2026
    [arXiv]
  2. A universal compression theory: Lottery ticket hypothesis and superpolynomial scaling laws
    3. Hong-Yi Wang, Di Luo, Tomaso Poggio, Isaac L. Chuang, Liu Ziyin*
    ICLR 2026
    [arXiv]
  3. Neural Thermodynamics I: Entropic Forces in Deep and Universal Representation Learning
    4. Liu Ziyin*, Yizhou Xu*, Isaac Chuang
    NeurIPS 2025
    [arXiv]
  4. Law of Balance and Stationary Distribution of Stochastic Gradient Descent
    5. Liu Ziyin*, Hongchao Li*, Masahito Ueda
    Physical Review E
    [arXiv]
  5. Compositional Generalization Requires More Than Disentangled Representations
    6. Qiyao Liang, Daoyuan Qian, Liu Ziyin, Ila Fiete
    ICML 2025
    [arXiv]
  6. Understanding the Emergence of Multimodal Representation Alignment
    7. Megan Tjandrasuwita, Chanakya Ekbote, Liu Ziyin, Paul Pu Liang
    ICML 2025
    [arXiv]
  7. Formation of Representations in Neural Networks
    8. Liu Ziyin, Isaac Chuang, Tomer Galanti, Tomaso Poggio
    ICLR 2025 (spotlight: 5% of all submissions)
    [paper]
  8. Remove Symmetries to Control Model Expressivity
    9. Liu Ziyin*, Yizhou Xu*, Isaac Chuang
    ICLR 2025
    [paper]
  9. When Does Feature Learning Happen? Perspective from an Analytically Solvable Model
    10. Yizhou Xu*, Liu Ziyin*
    ICLR 2025
    [paper]
  10. Parameter Symmetry and Noise Equilibrium of Stochastic Gradient Descent
    11. Liu Ziyin, Mingze Wang, Hongchao Li, Lei Wu
    NeurIPS 2024
    [paper] [arXiv]
  11. Symmetry Induces Structure and Constraint of Learning
    12. Liu Ziyin
    ICML 2024
    [arXiv]
  12. Zeroth, first, and second-order phase transitions in deep neural networks
    13. Liu Ziyin, Masahito Ueda
    Physical Review Research 2023
    [arXiv]
  13. Exact Solutions of a Deep Linear Network
    14. Liu Ziyin, Botao Li, Xiangming Meng
    Journal of Statistical Mechanics: Theory and Experiment, 2023
    [paper] [arXiv]
  14. On the stepwise nature of self-supervised learning
    15. James B. Simon, Maksis Knutins, Liu Ziyin, Daniel Geisz, Abraham J. Fetterman, Joshua Albrecht
    ICML 2023
    [paper] [arXiv]
  15. Sparsity by Redundancy: Solving L1 with SGD
    16. Liu Ziyin*, Zihao Wang*
    ICML 2023
    [paper] [arXiv]
  16. What shapes the loss landscape of self-supervised learning?
    17. Liu Ziyin, Ekdeep Singh Lubana, Masahito Ueda, Hidenori Tanaka
    ICLR 2023
    [paper] [arXiv]
  17. Exact Solutions of a Deep Linear Network
    18. Liu Ziyin, Botao Li, Xiangming Meng
    NeurIPS 2022
    [paper] [arXiv]
  18. Posterior Collapse of a Linear Latent Variable Model
    19. Zihao Wang*, Liu Ziyin*
    NeurIPS 2022 (oral: 1% of all submissions)
    [paper] [arXiv]
  19. Universal Thermodynamic Uncertainty Relation in Non-Equilibrium Dynamics
    20. Liu Ziyin, Masahito Ueda
    Physical Review Research (2022)
    [paper] [arXiv]
  20. Theoretically Motivated Data Augmentation and Regularization for Portfolio Construction
    21. Liu Ziyin, Kentaro Minami, Kentaro Imajo
    ICAIF 2022 (3rd ACM International Conference on AI in Finance)
    [paper] [arXiv]
  21. Power Laws and Symmetries in a Minimal Model of Financial Market Economy
    22. Liu Ziyin, Katsuya Ito, Kentaro Imajo, Kentaro Minami
    Physical Review Research (2022)
    [paper] [arXiv]
  22. Logarithmic landscape and power-law escape rate of SGD
    23. Takashi Mori, Liu Ziyin, Kangqiao Liu, Masahito Ueda
    ICML 2022
    [paper] [arXiv]
  23. SGD with a Constant Large Learning Rate Can Converge to Local Maxima
    24. Liu Ziyin, Botao Li, James B. Simon, Masahito Ueda
    ICLR 2022 (spotlight: 5% of all submissions)
    [paper] [arXiv]
  24. Strength of Minibatch Noise in SGD
    25. Liu Ziyin*, Kangqiao Liu*, Takashi Mori, Masahito Ueda
    ICLR 2022 (spotlight: 5% of all submissions)
    [paper] [arXiv]
  25. On the Distributional Properties of Adaptive Gradients
    26. Zhang Zhiyi*, Liu Ziyin*
    UAI 2021
    [paper] [arXiv]
  26. Noise and Fluctuation of Finite Learning Rate Stochastic Gradient Descent
    27. Kangqiao Liu*, Liu Ziyin*, Masahito Ueda
    ICML 2021
    [paper] [arXiv]
  27. Cross-Modal Generalization: Learning in Low Resource Modalities via Meta-Alignment
    28. Paul Pu Liang*, Peter Wu*, Liu Ziyin, Louis-Philippe Morency, Ruslan Salakhutdinov
    ACM Multimedia 2021
    NeurIPS 2020 Workshop on Meta Learning
    [arXiv] [code]
  28. Neural Networks Fail to Learn Periodic Functions and How to Fix It
    29. Liu Ziyin, Tilman Hartwig, Masahito Ueda
    NeurIPS 2020
    [paper] [arXiv]
  29. Deep Gamblers: Learning to Abstain with Portfolio Theory
    30. Liu Ziyin, Zhikang Wang, Paul Pu Liang, Ruslan Salakhutdinov, Louis-Philippe Morency, Masahito Ueda
    NeuRIPS 2019
    [paper] [arXiv] [code]
  30. Think Locally, Act Globally: Federated Learning with Local and Global Representations
    31. Paul Pu Liang*, Terrance Liu*, Liu Ziyin, Ruslan Salakhutdinov, Louis-Philippe Morency
    NeurIPS 2019 Workshop on Federated Learning (oral, distinguished student paper award)
    [paper] [arXiv] [code]
  31. Multimodal Language Analysis with Recurrent Multistage Fusion
    32. Paul Pu Liang, Ziyin Liu, Amir Zadeh, Louis-Philippe Morency
    EMNLP 2018 (oral presentation)
    [paper] [supp] [arXiv] [slides]