Optical computing has the potential to be faster and more energy-efficient than conventional digital-electronic computing for certain applications. This Perspective article surveys the differences between optics and electronics that could be exploited, and explores the physics and engineering challenges in realizing useful optical computers.