Artificial Intelligence techniques have progressed immensely over the past few years: many fields of science and engineering can now make use of these techniques. We are very interested in exploring how AI can help our research, and also how physics insights can lead to even more powerful AI algorithms. At this web page, you can learn more about this part of our research.

__ Applications of AI techniques in science and engineering __

We
propose
a method to use articial neural networks to approximate light scattering by multilayer
nanoparticles. We find the network needs to be trained on only a small sampling of the data in order
to approximate the simulation to high precision. Once the neural network is trained, it can simulate
such optical processes orders of magnitude faster than conventional simulations. Furthermore, the
trained neural network can be used solve nanophotonic
inverse design problems by using back-propogation - where the gradient is analytical, not numerical.
This method could be used in many other fields of
computational physics; it would allow us to approximate
physics simulations in fractions of the time. Furthermore,
owing to the robustness of back-propogation, this
method allows us to solve many inverse design problems
without having to manually calculate the inverse equa-
tions. Instead, we simply have to write a simulation for
the forward calculation, and then train the model on it
to easily solve the inverse design.

__ New Hardware for AI __

Almost all computing nowdays is done using electrons. However,
there are a few known algorithms that can be implemented in a
superior way using light (photons). For example, matrix
multiplication can be implemented with light essentially
instantly (at light-speed), and theoretically with zero
energy consumption. Since deep learning algorithms rely so
heavily on matrix multiplication, there is a value proposition
for implementing some of them with photonic (instead of
electronic) hardware. Such
systems
could for certain applications
be substantially faster, consume much less energy, and have
much lower latency.

__ New algorithms for AI __

Using unitary (instead of general) matrices in
artificial neural networks (ANNs) is a promising
way to solve the gradient explosion/vanishing
problem, as well as to enable ANNs to learn
long-term correlations in the data. We
present
an Efficient Unitary Neural
Network (EUNN) architecture that parametrizes the entire
space of unitary matrices in a complete and computationally
efficient way, thereby eliminating the need for
time-consuming unitary subspace-projections.

In another project, we
present
a novel recurrent neural network (RNN) based
model that combines the remembering ability of unitary
RNNs with the ability of gated RNNs to effectively forget
redundant/irrelevant information in its memory. We achieve
this by extending unitary RNNs with a gating mechanism.

The concepts of unitary evolution matrices and associative memory have boosted
the field of Recurrent Neural Networks (RNN) to state-of-the-art performance in
a variety of sequential tasks. However, RNN still have a limited capacity to manipulate
long-term memory. To bypass this weakness the most successful applications
of RNN use external techniques such as attention mechanisms.
In yet another project,
we propose a novel RNN model that unifies the state-of-the-art approaches:
Rotational Unit of Memory (RUM). The core of RUM is its rotational operation,
which is, naturally, a unitary matrix, providing architectures with the power to
learn long-term dependencies by overcoming the vanishing and exploding gradients
problem. Moreover, the rotational unit also serves as associative memory.