Teaching physics to AI yields insights into material properties
Despite their universal approximation capability, a drawback of deep neural networks (DNNs) is that they are “black boxes,” and it is unknown how or why they work, say the researchers. Thus, the physics discovered by the DNN remains hidden.
In their study, the researchers constructed a modern machine learning algorithm to determine the properties of a class of engineered materials known as metamaterials and to predict how they interact with electromagnetic fields. Because the algorithm first had to consider the metamaterial’s known physical constraints, the program was essentially forced to show its work.
Not only did the approach allow the algorithm to accurately predict the metamaterial’s properties, say the researchers, it did so more efficiently than previous methods while providing new insights.
“By incorporating known physics directly into the machine learning, the algorithm can find solutions with less training data and in less time,” says Willie Padilla, professor of electrical and computer engineering at Duke. “While this study was mainly a demonstration showing that the approach could recreate known solutions, it also revealed some insights into the inner workings of non-metallic metamaterials that nobody knew before.”
Metamaterials are synthetic materials composed of many individual engineered features, which together produce properties not found in nature through their structure rather than their chemistry. In this case, the metamaterial consists of a large grid of silicon cylinders that resemble a Lego baseplate.
Depending on the size and spacing of the cylinders, the metamaterial interacts with electromagnetic waves in various ways, such as absorbing, emitting or deflecting specific wavelengths. In their study, the researchers sought to build a type of machine learning model called a neural network to discover how a range of heights and widths of a single cylinder affects these interactions.
But, say the researchers, they also wanted its answers to make sense.
“Neural networks try to find patterns in the data, but sometimes the patterns they find don’t obey the laws of physics, making the model it creates unreliable,” says Jordan Malof, assistant research professor of electrical and computer engineering at Duke. “By forcing the neural network to obey the laws of physics, we prevented it from finding relationships that may fit the data but aren’t actually true.”
The physics that the researchers imposed upon the neural network is called a Lorentz model – a set of equations that describe how the intrinsic properties of a material resonates with an electromagnetic field. Rather than jumping straight to predicting a cylinder’s response, the model had to learn to predict the Lorentz parameters that it then used to calculate the cylinder’s response.
Incorporating that extra step, however, is much easier said than done, say the researchers.
“When you make a neural network more interpretable, which is in some sense what we’ve done here, it can be more challenging to fine tune,” says Omar Khatib, a postdoctoral researcher working in Padilla’s laboratory. “We definitely had a difficult time optimizing the training to learn the patterns.”
Once the model was working, however, it proved to be more efficient than previous neural networks the group had created for the same tasks. In particular, the researchers say they found this approach can dramatically reduce the number of parameters needed for the model to determine the metamaterial properties. They also found that this physics-based approach is capable of making discoveries all on its own.
As an electromagnetic wave travels through an object, it doesn’t necessarily interact with it in exactly the same way at the beginning of its journey as it does at its end – a phenomenon known as spatial dispersion. Because the researchers had to tweak the spatial dispersion parameters to get the model to work accurately, they discovered insights into the physics of the process that they hadn’t previously known.
“Now that we’ve demonstrated that this can be done,” says Padilla, “we want to apply this approach to systems where the physics is unknown.”
“Lots of people are using neural networks to predict material properties,” adds Malof, “but getting enough training data from simulations is a giant pain. This work also shows a path toward creating models that don’t need as much data, which is useful across the board.”