"Within our toy scenario," say the researchers, "the process discovers several concepts known to have been useful to the research community. In the end, our approach manages to construct a model that outperforms hand-designs of comparable complexity."
The best evolved algorithm produced by the method includes techniques such as noise injection as data augmentation, bilinear model, gradient normalization, and weight averaging, and the improvement over the baseline also transfers to datasets that are not used during search. Through more experiments, say the researchers, they show that it is possible to guide the evolutionary search by controlling "the habitat" — i.e., the tasks on which the evolutionary process evaluates the fitness of the algorithms.
So far, say the researchers, they consider their work to be preliminary.
"We have yet to evolve fundamentally new algorithms, but it is encouraging that the evolved algorithm can surpass simple neural networks that exist within the search space," say the researchers. "Right now, the search process requires significant compute. As the coming years scale up available hardware and as the search methods become more efficient, it is likely that the search space will become more inclusive and the results will improve. We are excited at the prospects of discovering novel machine learning algorithms as we further our understanding of AutoML-Zero."
For more, see " AutoML-Zero: Evolving Machine Learning Algorithms From Scratch ."
Auto ML solution makes AI 'available to everyone'
Google AI tool decodes ancient Egyptian hieroglyphs
Google AI tool simplifies machine learning for businesses
Whitepaper: Building smart IoT devices with automated ML
Evolutionary algorithms promise more efficient plug-in hybrid EVs