Autonomous Vision, Night and Day
Pony.ai’s camera for traffic light detection
Driving through a traffic light control intersection under red light is incredibly dangerous. To reduce the probability of such an event to zero, Pony.ai has developed a camera trained by AI. We analyze traffic light data from millions of miles driven and all types of corner cases such as broken, partially covered, dirty, and incorrectly directed traffic lights. We address all these scenarios in the camera design and tuning process.
Camera shutter speed often called integration or the exposure time controls the time the camera sensor uses to capture the scene. Camera exposure control is autonomous and doesn’t depend on the car’s speed. It only decides the integration time based on the scene’s brightness, making it extremely hard for the conventional camera to capture LED lights with flickering.
The camera we develop at Pony.ai is designed to see traffic lights regardless of day or night, or whether it is a modern bright LED light with flickering or old-style traffic light using an Edison-type light bulb.
Pony.ai’s in-house designed camera (PiDC) is always using the same exposure time of 11ms to achieve that effect. To compensate for such a highly sensitive sensor setting, we use a neutral density filter, equivalent to multiple pairs of sunglasses stacked on top of each other, to avoid oversaturation. And, yes, we use this filter at night too!
Here is how our camera sees the world when we slightly adjusted its tuning for human vision:
The sun is in the center of our visual field. Usually, it would be very challenging for an autonomous vehicle camera to see traffic lights when the sun is behind it, due to the enormous difference between the sun and the traffic light. Still, Pony’s camera not only can clearly identify traffic light signals but also can reproduce the street sign backed by the sun disc, which is much darker than the traffic light.
Pony has added the PiDC camera to its AlphaX fleet and trained its traffic lights model, the number of monthly issues caused by traffic lights reduced significantly to zero in August 2021.
We have no doubt that the cameras we develop at Pony’s lab will make driving safer, more efficient, and more sustainable for future mobility.
Below you can read a technical article and watch the presentation
