Published Paper Details:

Our proposed model provides an end-to-end autonomous vehicle simulation. We trained a Convolutional Neural Network (CNN) to directly map raw pixels to the steering commands from a single front-facing camera. This end-to-end approach proved to be surprisingly strong. The machine learns to drive in traffic on local roads with or without lane markings and highways with minimal training data from humans. It also works in places where the visual direction is not obvious, such as parking lots and on unpaved roads. The model learns internal representations of the required processing steps automatically, such as detecting useful road features with only the human steering angle as the training signal. We have never specifically conditioned it to detect the outline of paths, for example. Our end-to-end method optimizes all processing steps simultaneously as compared to the explicit decomposition of the problem, such as lane marking detection, route planning, and control. We claim that in the end, this will lead to better results and simpler systems. It will result in better performance because the internal components are self-optimizing to maximize overall system performance, rather than optimizing intermediate criteria chosen by humans e.g. Detecting roads. Such criteria are selected understandably for ease of human interpretation which does not automatically guarantee maximum performance of the system. Smaller networks are possible because with the minimal number of processing steps the machine learns to solve the problem.