Compositional Pattern-Producing Networks

Sources: code, paper, blog, TensorFlow.js

Li Ding | 丁立

I'm currently a Research Engineer at MIT, working on deep learning for autonomous vehicles. Our research (led by Dr. Lex Fridman) focuses on Human-Centered Artificial Intelligence (HCAI) that leverages both natural human intelligence and large-scale machine (deep) learning to explore the future of human-machine collaboration.

I also TA a few deep learning related courses at MIT, including 6.S094: Deep Learning for Self-Driving Cars, etc.. If you have questions regarding these courses, please email us via hcai [at]

Prior to joining MIT, I was a Research Associate at University of Rochester, doing research on human action recognition with Prof. Chenliang Xu.

I'm from Shanghai, China. On a side of fun, I'm a casual Kaggler who is interested in playing with various kinds of data. I like photography, modern art, electro-funk, and at the moment, traveling around the world with Pokémon Go.

liding [at]

Last updated on 2019/3/11


MIT Human-Centered Autonomous Vehicle

The interaction between human and machine with growing intelligence challenges our assumptions about the limitations of human beings at their worst and the capabilities of AI systems at their best. We explore Human-Centered Autonomous vehicle as an illustrative case study of concepts in shared autonomy.
More on project page.

Arguing Machines: Human Supervision of Black Box AI Systems

We consider the paradigm of a black box AI system that makes life-critical decisions. By proposing an “arguing machines” framework, we show that the disagreement between the two AI systems, without any knowledge of underlying system design or operation, is sufficient to arbitrarily improve the accuracy of the overall decision pipeline if given human supervision over disagreements.
More on project page.

Switch of Vehicle Control in Racing Simulation

A racing simulation environment with fully functioned end-to-end autonomous driving system is developed based on Forza Motorsport 7. We study the fundamental behavior in collaborative vehicle control between human players and autonomous driving algorithms.

Dynamic Scene and Optical Flow

Single-frame-based algorithms ignore the rich information contained in the dynamics of moving scene, which is usually too noisy or abstract to utilize. We study how motion representations such as optical flow can be used to help scene understanding.
More on project page.


Scene Segmentation and Large-Scale Annotation

We adopt the latest semantic segmentation techniques and modify the architecture to fit into the task of driving scene perception. We also explore ways to improve accuracy and robustness, such as edge case discovery and more efficient annotation methodology.

Fully / Weakly Supervised Action Localization

We propose a new action classification model and a novel iterative alignment approach to address weakly supervised action localization task that only require the transcript to learn the exact time of actions.
More in our CVPR '18 paper with code released on Github.


Human Interaction with Deep Reinforcement Learning Agents in Virtual Reality
Lex Fridman, Henri Schmidt, Jack Terwilliger, Li Ding
[NeurIPS 2018: Deep RL Workshop]

Arguing Machines: Human Supervision of Black Box AI Systems That Make Life-Critical Decisions
Lex Fridman, Li Ding, Benedikt Jenik, Bryan Reimer
[under review] [pdf] [arXiv]

MIT Autonomous Vehicle Technology Study: Large-Scale Deep Learning Based Analysis of Driver Behavior and Interaction with Automation
Lex Fridman, Daniel E. Brown, Michael Glazer, William Angell, Spencer Dodd, Benedikt Jenik, Jack Terwilliger, Julia Kindelsberger, Li Ding, Sean Seaman, et al.
[under review] [pdf] [arXiv]

Weakly-Supervised Action Segmentation with Iterative Soft Boundary Assignment
Li Ding, Chenliang Xu
[CVPR 2018] [pdf] [arXiv] [code] [poster]

TricorNet: A Hybrid Temporal Convolutional and Recurrent Network for Video Action Segmentation
Li Ding, Chenliang Xu
[arXiv Report] [pdf] [arXiv]


6.S094: Deep Learning for Self-Driving Cars (Winter IAP 2018 & 2019)
6.S091: Deep Reinforcement Learning (Winter IAP 2019).
6.S093: Human-Centered Artificial Intelligence (Winter IAP 2019).
6.S099: Artificial General Intelligence (Winter IAP 2018).


IEEE Transactions on Circuits and Systems for Video Technology (2018)
IEEE Access (2018)


Level: Competitions Expert (highest rank: 1169th | current rank)

Statoil/C-CORE Challenge
(Satellite Image Classification)

· 2018 · Top 6%

Data Science Bowl 2017
(Lung Cancer Detection)

· 2017 · Top 6%

Thanks for visiting!