Their neuroinspired approach, say the researchers, could one day make it possible to train neural networks on low-power devices such as smartphones, laptops, and embedded devices. Currently, training neural networks to perform tasks like object recognition, autonomous navigation, or game playing requires large computers with hundreds to thousands of processors and weeks to months of training times.
The reason, say the researchers, is that such computations involve transferring data back and forth between two separate units — the memory and the processor — and this consumes most of the energy and time during neural network training. To address this, the researchers teamed up with ultralow-power embedded non-volatile memory technology company Adesto Technologies to develop hardware and algorithms that allow these computations to be performed directly in the memory unit - eliminating the need to repeatedly shuffle data.
"We are tackling this problem from two ends — the device and the algorithms — to maximize energy efficiency during neural network training," says Yuhan Shi, an electrical engineering Ph.D. student at UC San Diego and first author of a paper describing the research.
The hardware component of the approach consists of a super energy-efficient type of non-volatile memory technology — a 512-kilobit subquantum Conductive Bridging RAM (CBRAM) array. Based on Adesto's CBRAM memory technology, it consumes 10 to 100 times less energy than today's leading memory technologies.
However, instead of using it as a digital storage device that only has '0' and '1' states, the researchers demonstrated that it can be programmed to have multiple analog states to emulate biological synapses in the human brain. As a result, say the researchers, the so-called "synaptic device" can be used to do in-memory computing for neural network training.
"On-chip memory in conventional processors is very limited, so they don't have enough capacity to perform both computing and storage on the same chip," says Duygu Kuzum, a professor of electrical and computer engineering at the