With potential applications in areas including autonomous driving, vision-aided navigation and biomedical imaging, Cornell is leading a new $7.5 million research project to develop the scientific foundations for computing with light. The work is paving the way for hybrid optical-electronic systems that could reduce the energy and computational resources required for AI-driven image analysis and other visual inference tasks.
Francesco Monticone, associate professor in the School of Electrical and Computer Engineering, in the Cornell Duffield College of Engineering, leads the project, “Deep Nanophotonic Inference Systems.” It is supported by the U.S. Department of Defense Multidisciplinary University Research Initiative.
Turning light captured by cameras and other sensors into useful information – a task increasingly performed by AI – relies almost entirely on digital electronics. The project aims to change that by bringing together researchers from multiple institutions to explore how nanophotonic systems can perform some of those computations using the physics of light, working alongside electronic systems to reduce the computational burden on conventional processors.
“Optics can greatly add to electronic technologies in its ability to preprocess visual data quickly and efficiently, but many fundamental and practical questions remain,” Monticone said. “We need to understand what these hybrid systems can fundamentally achieve before we can design and optimize them for real-world applications such as autonomous driving, vision-aided navigation, adaptive sensing, biomedical imaging and many others.”
The research is organized around three interconnected aims. The first will identify the physical and information-theoretical limits of nanophotonic systems. The second will develop novel approaches for rapid simulation, inverse design and fabrication of 3D nanophotonic structures. The third will optimize how optical hardware and digital algorithms work together as complete end-to-end systems.
“What excites me most is having the opportunity to really get to the bottom of the foundational questions of this emerging field at the intersection of applied physics, electrical engineering and AI,” Monticone said. “We want to understand what is fundamentally possible, use that understanding to design optimized photonic inference systems, and work with our collaborators to actually fabricate and test them.”
Co-principal investigators for the project include researchers from Stanford University, the University of Pennsylvania, Vanderbilt University and the Massachusetts Institute of Technology. International collaborators include researchers at Sorbonne University and Lund University.
Some of the nanophotonic platforms the researchers will investigate are based on metamaterials – engineered structures that manipulate light or other physical waves in ways that natural materials cannot. Monticone is also advancing metamaterials as co-principal investigator of a second U.S. Department of Defense Multidisciplinary University Research Initiative, “N-Dimensional Space-Time Metamaterials,” led by the University of Pennsylvania.
The project will explore what becomes possible when metamaterials not only manipulate waves in space, but also change rapidly over time. By introducing time as an additional design dimension, researchers hope to unlock new ways of controlling the behavior of light, radio waves and even sound.
Monticone will investigate the fundamental limits of these dynamic metamaterials and develop new designs for collaborators to build and test, with potential applications ranging from reconfigurable communication and radar systems to quantum photonics and advanced sensing.
Syl Kacapyr is associate director of marketing and communications for Duffield Engineering.
