How Vision and Image Processing Algorithms Propel Automotive ADAS Development

Advanced Driver Assistance Systems (ADAS) work with the support of software, hardware and firmware solutions developed on technologies like RADAR, LIDAR, vision and image processing, or AI to help the driver for a safe and smooth driving experience. ADAS development and adoption have gained enormous importance across OEM manufacturers in both automotive and IT industries since the ruling from the EU and US mandates the use of autonomous emergency-braking systems and forward-collision warning systems in all vehicles by 2020.

Out of all the technologies used in ADAS, vision and image processing is the predominant method used by ADAS providers for understanding the on-road environment, detection of objects, and taking corrective driving decisions. A set of full-HD cameras installed on the sides of the vehicle help in capturing the objects all around the vehicle for further analysis and processing by the ADAS system.

Let us find out some of the basic vision processing features, challenges, possible solutions, and application areas.

Road Signs Detection

Identification and classification of road signs are the major tasks of the ADAS systems for a controlled vehicle driving. Road signs such as speed limits, pedestrian or animal cross boards, roadwork indications, overhead warning boards, direction boards, railway tracks, etc., need to be detected in taking a decision on changing the direction and speed of the moving vehicle. Identification of traffic signs also comes under this task.

Along with the road signs detection, an ADAS system helps in detecting specific texts or signs as programmed by the user.

Few Challenges and Solutions

• False detection of signs: Employing edge-detection algorithms like Canny, Sobel, etc., to identify the signs in the captured video frame will be helpful in the proper identification of signs.
• Detection of inappropriate signs as road signs: Signs, which are present in the environment such as walls, houses, and posters should not be detected as road signs. By having a set of road sign images and using OpenCV algorithms, the detected signs can be classified whether they are road signs or not.
• False detection because of environmental conditions: By using the image pre-processing algorithms as smoothening, noise removal, and edge detection, the layers present on the image can be cleaned up for further analysis to avoid false detection due to environmental conditions such as night, fog, rain, etc.
• Frequent occurrences of road signs: If multiple road signs are in close proximity, leaving less time for ADAS systems to make a decision, the chances of missing the signs can become high and hence prone to accidents. By using a high-speed processor with multiple processing units like TI’s TDA2x, TDA2px, and TDA3x processor, this issue can be resolved.
• Presence of multiple signs in a frame: The ADAS system should have algorithms to separate the signs in a single frame, understand each one and take a decision as per requirement. Image processing algorithms such as image classifiers and dividers with sign-to-requirement mapping logic can be used to overcome this condition.

Pedestrian Detection

Pedestrian detection by ADAS systems is gaining a significant attention for safety control and accident prevention. When human behavior is unpredictable in the environment, pedestrian detection on road becomes a challenging task. It needs faster and safer methods to work with a high level of accuracy. This is applicable in backover prevention too.

Pedestrian tracking and prediction of orientation and intention are major parts of this task. Analysis of the visual information to distinguish other moving objects from humans is a necessary part of this activity.

This data, collected over a longer time period, can be used to predict the traffic condition and pedestrian count of the frequently traveling regions, which can help in controlling the speed of the vehicle.

Solution based on computer vision methods along with machine learning algorithms is the robust approach to handling this situation.

Few Challenges and Solutions

• False detection of pedestrian: A wrong detection of objects as a pedestrian can be avoided by identifying the unique features of people, which differentiates them from other objects. Face and body detection algorithms can be used to address this challenge.
• False detection of direction: Detection of the direction in which the pedestrians move is necessary to control the vehicle. To avoid the false detection of directions, algorithms that handle image data between successive frames as people counting can be used.
• False detection of speed: To refrain a vehicle from detecting false speed, the estimation of the speed in which the pedestrians move can be detected using deep learning modules on the successive image frames.
• Difficulties in distinguishing pedestrians: Difficulty in distinguishing pedestrians can lead to false detection and ultimately to an incorrect driving decision. It is useful to distinguish pedestrians based on different features such as size, gender, and height by applying sample datasets and human features to the algorithms.
• Distinguishing pedestrian movements: It is necessary to detect whether the pedestrians are in motion or not. Tracking algorithms can be used to handle this situation.

Lane Tracking (Lane Keep Assist) / Blind Spot Monitoring

A lane departure warning system warns the driver when the vehicle may depart from the current travel lane without lane changing signal. The ADAS system works on this by detecting traffic lane markings from the image data of a front or rear camera. This system also helps in left or right turn awareness.

The economic burden of traffic accidents amounts nearly 3% of the world GDP, according to the World Health Organization. It includes hospitalization expenses and loss due to property damages. ADAS systems help in reducing this by alarming the driver before critical situations or by preparing the vehicle to minimize the consequences if the crash is inevitable. LIDAR and cameras can be used for this purpose.

Few Challenges and Solutions

• Wrong lane detection: The divider present in the middle of the road should not be detected as a lane. The height and the other unique properties of the metal dividers should be used by object detection algorithms to differentiate the lanes and dividers.
• Proper lane transfer. Vision algorithms developed for vehicle localization with the proper datasets can be used to filter out non-relevant objects under various environmental conditions such as poor visibility and fog weather to help in a proper lane transfer. ADAS Image processing algorithms can be used for identifying the presence of other vehicles, their speed, and direction from the image frames data taken from the front, rear, and sides of the vehicle to help the driver in deciding for proper lane transfer.
• Incorrect object detection: The objects detected from the rear and side camera image data should be classified properly as objects which may block lane transfer. Object tracking algorithms can be used to avoid incorrect object detection.

Driver Authentication and Status Monitoring (DASM)

A DASM system in ADAS tracks the driver’s face direction, posture, and fatigue for assisting in the proper movement of the vehicle. The DASM feature uses the image data captured from the cameras installed inside the vehicle.

Few Challenges and Solutions

• Incorrect face detection: By using image-processing algorithms as face detection and face recognition with the help of AI support, the system can distinguish between the face of the driver and the passengers. This can be used to authenticate the correct driver.
• Incorrect condition detection: By using the facial feature detection in the image processing algorithms and mapping between them with the driver status, the condition of the driver can be estimated precisely.

Forward and Backward Collision Warnings (FBCW)

By using the image data from a stereo or monocular camera installed in front and rear sides of the vehicle, an FBCW system can detect other vehicles running on the path. This helps to warn the driver of a potential collision risk based on the distance between the vehicles and the speed of the own vehicle. This system can be useful in proper parking.

Few Challenges and Solutions

• False detection of vehicles: The objects other than vehicles need to be discarded while detecting the vehicles. The image processing algorithms with vehicle classifiers and AI can be used to avoid false object detection around the vehicle.
• Incorrect vehicle features detection: The speed, direction, and type of the vehicles running in front and rear need to be identified before the ADAS system takes the decision. If these properties are not estimated correctly, it will lead to a dangerous situation. Along with the image processing algorithms, RADAR support can be added to calculate these data precisely.

Parking Assistance

By analyzing the available parking space dimensions, an ADAS system helps the driver in deciding for parking the vehicle. This can be performed by processing the image data captured from the front, rear, and sides. Also, by using four fish-eye or wide-angle lens cameras, an ADAS system can capture images around the vehicle. These images will be processed as a bird’s eye view and will be used for top-view parking assistance.

Few Challenges and Solutions

• Improper detection of space: This can be avoided by combining the distance of the objects around the space and the own vehicle dimensions to calculate the appropriate space for parking.
• Wrong object detection: The objects around the space need to be detected properly. By using RADAR along with image processing algorithms, this issue can be resolved.

Vehicle Functions Control

Some of the vehicle functions controlled by the ADAS system are:

• Speed Control: Based on the nature of the region such as isolated, hilly or urban track, the maximum speed of the vehicle can be calculated and defined.
• Beam Control: An appropriate lighting range from the high and low beams can be selected by detecting the headlights of oncoming vehicles and the taillights of the leading vehicles.
• Engine control: Based on the signal duration detected from the traffic signals, the ADAS system can control the engine to switch it off or on. Similarly, it enables the coolants, if the duration of the STOP signal is detected more than a threshold count. This idea comes under traffic signal recognition and actuation process.

Few Challenges and Solutions

• Wrong region detection: This issue can be resolved with the help of AI and GPS information of the traveling territory. An ADAS system can control the speed of the vehicle along with proper image processing algorithms for detecting the ups and downs of the road.
• Wrong light intensity detection: By using LIDAR and the environmental conditions and by using vision-image processing tools such as edge detection, ADAS system can control the beam condition of the own vehicle to overcome the problem of wrong light intensity detection.
• Wrong sign detection: By applying OCR (Optical character recognition) concepts such as matrix matching, pattern matching or OpenCV algorithms, the traffic signs can be detected correctly.

Vehicle Classification

With the help of image data from the front-mounted camera, an ADAS system classifies the detected vehicle and controls the own vehicle based on that. For example, if the front-running vehicle is a fuel container or a container with huge cargo load, the own vehicle needs to maintain a normal speed and keep a distance from it of more than a threshold value.

If a learner board is detected on a nearby vehicle, an ADAS system adds more control of the own vehicle because the situation may become unpredictable.

If an ambulance is detected, an ADAS system controls the own vehicle to give space for the ambulance.

Few Challenges and Solutions

• Wrong vehicle detection: The speed, direction, and type of the vehicles running in front need to be identified before the ADAS system takes the decision. If these properties are not estimated correctly, it will lead to a dangerous situation. Along with the image processing algorithms, RADAR support can be added to calculate these data precisely.

By using effective and efficient ADAS systems in the vehicles, the traveling of drivers and passengers can be less accident-prone, risk-free, less manual and can move towards a fully automated driving control system.

eInfochips’ Automotive Engineering Services and Solutions assist automotive companies to design and develop ADAS systems through camera-based algorithm porting, sensor fusion for vision and RADAR-based ADAS  and feature enhancement through code testing. To know more about eInfochips capabilities in automotive and ADAS solutions, get in touch with us.