Applications of AI techniques in science and engineering
We propose a method to use articial neural networks to approximate light scattering by multilayer nanoparticles. We find the network needs to be trained on only a small sampling of the data in order to approximate the simulation to high precision. Once the neural network is trained, it can simulate such optical processes orders of magnitude faster than conventional simulations. Furthermore, the trained neural network can be used solve nanophotonic inverse design problems by using back-propogation - where the gradient is analytical, not numerical. This method could be used in many other fields of computational physics; it would allow us to approximate physics simulations in fractions of the time. Furthermore, owing to the robustness of back-propogation, this method allows us to solve many inverse design problems without having to manually calculate the inverse equa- tions. Instead, we simply have to write a simulation for the forward calculation, and then train the model on it to easily solve the inverse design.
New Hardware for AI
Almost all computing nowdays is done using electrons. However, there are a few known algorithms that can be implemented in a superior way using light (photons). For example, matrix multiplication can be implemented with light essentially instantly (at light-speed), and theoretically with zero energy consumption. Since deep learning algorithms rely so heavily on matrix multiplication, there is a value proposition for implementing some of them with photonic (instead of electronic) hardware. Such systems could for certain applications be substantially faster, consume much less energy, and have much lower latency.
New algorithms for AI
Using unitary (instead of general) matrices in artificial neural networks (ANNs) is a promising way to solve the gradient explosion/vanishing problem, as well as to enable ANNs to learn long-term correlations in the data. We present an Efficient Unitary Neural Network (EUNN) architecture that parametrizes the entire space of unitary matrices in a complete and computationally efficient way, thereby eliminating the need for time-consuming unitary subspace-projections.
In another project, we present a novel recurrent neural network (RNN) based model that combines the remembering ability of unitary RNNs with the ability of gated RNNs to effectively forget redundant/irrelevant information in its memory. We achieve this by extending unitary RNNs with a gating mechanism.
The concepts of unitary evolution matrices and associative memory have boosted the field of Recurrent Neural Networks (RNN) to state-of-the-art performance in a variety of sequential tasks. However, RNN still have a limited capacity to manipulate long-term memory. To bypass this weakness the most successful applications of RNN use external techniques such as attention mechanisms. In yet another project, we propose a novel RNN model that unifies the state-of-the-art approaches: Rotational Unit of Memory (RUM). The core of RUM is its rotational operation, which is, naturally, a unitary matrix, providing architectures with the power to learn long-term dependencies by overcoming the vanishing and exploding gradients problem. Moreover, the rotational unit also serves as associative memory.