What are the safety sensors & cameras on those new windshield packs? (like Volvo city safety)

Question

Many new models come with sensor & cameras on the windshield, like this Volvo:

• Although cars came from different manufactorers it seems that they all have pretty much the same pack - these 4 different types of sensors.
• The functions they enable are known: Different ADAS capabilities, sometimes almost partial selfdriving.

My question is what type exactly each one of them and for what he is responsible?

• Number 1 is mono camera, and Mobileye already almost demonstrated almost full autonomy with only mono camera. So why we need all others?
• We know the rule of LIDAR in the autonomous car tests performed now days, basically for HD mapping but for that they need 360 degrees rotation device like Velodyne. So does one of 2-3-4 in the picture is single direction small version of LIDAR?
• I guess the others are types of RADAR: So we know that the legacy parking sensors make use of ultrasonic sensors. Does one of them is here? Seems no.
• Again self driving tests, usually they use classical Radar & infra red. Are they here? and for what functions?

Thanks,

Answer 1

There are 3 basic aspects; cost, weather and redundancy.

Sensors like LIDAR are great at mapping the surrounding world and creating a point cloud. The problem is that high quality LIDARs are very expensive and the processing power to analyze that point cloud in real time is also very expansive. Another aspect is that LIDAR is not forgiving of weather like rain or snow.

Radar on the other hand has some different issues. Radar is really good at cutting through weather like rain or snow. Unfortunately radar setups that can actually map the 3D world are super expensive. The radar that is currently found on cars effectively returns the distance to an object in front and that's about it.

Vision systems try to make the best of both worlds. With reasonable hardware, video is far easier to process than a point cloud. Cameras are cheap. By using, for example Infrared light, things like weather is easier to deal with. The downfall of vision is that the camera needs a lot of support to make it dependable. Focusing video on the fly is problematic. Sensor 3 in you picture is probably used for focus or focus verification. The other two sensor probably asses lighting conditions. This helps deciding what gain to use and whether the camera should kick over to a night mode. There may even be an IR illumination to help the camera in night mode beyond what the headlights can do. The redundancy of the system then comes from that all the sensors must agree with the camera for the system to continue to functions. If one of the sensors the camera disagrees then the system shuts down. Finally by placing these sensor on the windshield in reach of the windshield wipers the area is kept clean.

Answer 2

City Safety is based on a combined camera and radar unit integrated at the top of the windscreen, in front of the interior rear-view mirror. The latest technology upgrade is a smarter and faster high-sensitive, megapixel image camera combined with advanced exposure control. This makes the detection and auto brake technology work effectively also when driving in darkness.

The radar’s task is to detect objects in front of the car and to determine their position and movement, and the distance to them. The camera identifies what type of object it is. The technology continuously monitors the object – and a central control unit uses the camera and radar data to evaluate the risk of a collision and to initiate the most efficient counteraction.

This is from:

Volvo's Press Release City Safety

That doesn't answer everything, but it demonstrates that radar is used for the distance, and the megapixel mono camera used for object recongnition.

It's possible #2-4 has one radar emitter and one receiver, but that still leaves no explanation for the extra one (or two) gizmos.

I'll continue to research. It's a great question, and there's precious little tech information that I can find ... so far

On edit:

Wikipedia claims the system can fail in heavy rain and snow. Which is probably due to the noise on the mono megapixel camera limiting object recognition. It's also clear from the Wiki article that the Volvo system is emergency auto-braking only; there's no self-drive or parallel parking aids [yet].

Answer 3

1. Camera.
2. Rain sensor.
3. LIDAR receiver (stereo).
4. LIDAR transmitter. Probably it shows a grid and the stereo receiver then can determine the distance and shape of the object. Radar is located in the front behind a rectangle thingie.

---

• First generation of City Safety didn't have a radar.
• Second generation models have radar in front.
• The third generation, now used, has no LIDAR but instead a multi-beam radar and a camera in windscreen (and of course rain sensor).

So now, LIDAR's problem with weather is erased since radar is better on that part. And since the radar is moved to the wind screen its not affected by heavy snow or ice build up like it was in the front before.

The camera and radar cooperate and gets two pictures of the world and if they are considered to be approximately the same, the system reacts. Can now detect pedestrian half way behind car (like kid running out in front of parked car), cyclist and cars.

I have the new system and is has NEVER indicated failure due to weather, like snow build up. My father has generation two and on some occasions the system triggers an alarm for the radar not working properly due to snow slush.

Works like a charm and together with rear radar and surrounding sodar you have a car that can detect obstacles, read signs, see the road, partly drive by itself, keep cruise control distance, warn rear driver if they don't see you are at standstill in a queue (the car then flashes the lights to wake them up) and to protect you (if they still doesn't react and my car thinks this will not end well, it pretensions seat belts, apply full brake if you are at stand still, to prevent you from bouncing further, and precharge the airbag safety system - if nothing happens, it releases the belts and so on), and it can find parking spot that fits and help you park there and of course help you with obstacles around the car when parking by yourself.

Oh, and blind spot too - even before the car comes to your side. It determines the speed of the car and then signal to me that soon, a car will be in your blind spot. Also using rear radar detectors. And if you reverse from a parking spot, they look at the sides of some one is crossing and alert you about that too.