Vision-Based Road and Lane Detection: Challenges and Functional Modules

Vision-based road and lane detection systems face challenges in detecting lanes using road markings under various conditions. The system involves functional modules like image pre-processing, feature extraction, model fitting, and more to ensure accurate recognition. Techniques such as adjusting for illumination changes, obstacle removal, gradient-based lane detection, and structure-based road analysis are employed to enhance system performance.

• Vision-based
• Lane detection
• Road recognition
• Functional modules
• Challenges

tyb_we Follow

Uploaded on Oct 07, 2024 | 0 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

Presentation Transcript

1. VISION-BASED ROAD AND LANE DETECTION Menna El-Shaer CrIS UTC CENTER FOR AUTOMOTIVE RESEARCH

2. PROBLEM/ PERCEPTION CHALLENGES Monocular camera system or 3D stereo vision system Lane/Road Recognition System needs to Detect lanes using white or yellow solid or dashed road markings. Be flexible with different road structures. Be reliable, operate under different light and weather conditions. CENTER FOR AUTOMOTIVE RESEARCH

3. FUNCTIONAL MODULES OF A ROAD AND LANE RECOGNITION SYSTEM Image Pre-Processing Feature Extraction Temporal Integration Model Fitting Image to World Correspondence CENTER FOR AUTOMOTIVE RESEARCH

4. IMAGE PRE-PROCESSING Adjust for illumination changes usually done by adjusting the dynamic range of the camera Obstacle Removals could be done by defining ROIs (Pomerleau 1995, Broggi et al. 2006) for the relevant and irrelevant parts of the image and using tracking mechanisms to predict their locations in subsequent frames (Wu et al. 2008) upper image part (irrelevant part) after computing the vanishing line (horizon) is truncated (Zhang et al. 2009) CENTER FOR AUTOMOTIVE RESEARCH

5. FEATURE EXTRACTION Lanes Use gradients (Samadzadegan et al. 2006, Nieto et al. 2008) or steerable filters (McCall et al. 2006) using the assumption that the shape of the road is different from the lane Use box filters (Huang et al. 2009, Wu et al. 2008) using the assumption of narrow lanes and that they have brighter intensities than the road After filtering, either use the raw responses or create a binary map by thresholding (Shi et al. 2009, Lipski et al. 2008) Color features could also be used (Chin et al. 2005, Cheng et al. 2006, Sun et al. 2006) Line detectors e.g. Hough Transform could be used to find lane candidate markers (points) CENTER FOR AUTOMOTIVE RESEARCH

6. FEATURE EXTRACTION Roads Structure-based Methods Surface roughness or elevation variance (Yamaguchi et al. 2008, Huang et al. 2009) Appearance-based Methods Region Growing (Alvarez et al. 2007) Modeling Road Color Distribution (Nefian et al. 2006) Textural Features (Alon et al. 2006) and Gabor Filters (Rasmussen et al. 2005, Kong et al. 2009) Template Matching (McCall et al. 2006, Pomerleau 1995) Classification-based Algorithms using a-priori information (Cheng et al. 2006, Pomerleau 1995, Alon et al. 2006) CENTER FOR AUTOMOTIVE RESEARCH

7. MODEL FITTING Goal is to find a high-level representation of the lane path or road Input data to the model are lane or road boundary points Smoothness could be achieved by computing a fitting smooth score. A Hessian filter using the extracted boundary points could be computed for additional boundary direction information and computing the distance transform by measuring distances between points and their closest boundary points (Huang et al. 2009) RANSAC is usually used due to its ability to detect outliers (noisy points) and fit inliers CENTER FOR AUTOMOTIVE RESEARCH

8. TYPES OF MODELS Parametric Models Lines Hough Transform (Borkar et al. 2009, Jiang et al. 2010) Polynomial Curves Semi-Parametric Models Splines B-Splines (Wang et al. 2004) Active Contours (Sawano et al. 2006) Non-Parametric Models Ant Colony Optimization (Broggi et al. 2006) Hierarchical Bayesian Networks (Nefian et al. 2006) Minimum Path Estimation Problem (Beck & Stiller 2014) CENTER FOR AUTOMOTIVE RESEARCH

9. TEMPORAL INTEGRATION 2D Lane/Road Model Tracking in real world coordinates using a sequence of images by computing an ego-motion model (Yamaguchi et al. 2008) The transformed expected lane is used to find the best estimate for the road/lane model parameters using a Kalman filter (McCall et al. 2006, Kim 2008, Borkar et al. 2009) or a Particle filter (Danescu et al. 2009, Jiang et al. 2009) Probabilistic Graphical Models like Dynamic Bayesian Networks could be used for temporal reasoning (Kim 2008) Multiple Hypotheses Tracking Algorithm CENTER FOR AUTOMOTIVE RESEARCH

10. IMAGE TO WORLD CORRESPONDENCE Inverse Perspective Mapping Structure From Motion Algorithms to estimate the camera pose with respect to the ground plane (Yamaguchi et al. 2008) Dynamic Homography Matrix Estimation (Kang et al. 2014) CENTER FOR AUTOMOTIVE RESEARCH

11. PREVIOUS OSU WORK A Lane Tracking System for Intelligent Vehicle Applications Keith Redmill, Srinivasa Upadhya, Ashok Krishnamurthy, Umit Ozguner 2001 IEEE Intelligent Transportation Systems Conference Proceedings CENTER FOR AUTOMOTIVE RESEARCH

12. PREVIOUS OSU WORK Algorithm: 1. Use Inverse Perspective to map real world coordinates to a 2D Bird s eye view of the road scene. Estimate the position and location of lane boundaries with respect to the vehicle using the information in the first image frame. Define search regions as two bands centered on the left and right lane boundaries, extending up to a distance of 0.75 meters laterally. Extract two candidate points for lane markers from each search region using a match filter for sharp pixel intensity gradient. Extract the features which are the center-line candidates in this case. Features are chosen as those candidates with the normalized matched filter output greater than or equal to the mean of intensity values of the filter. 2. 3. 4. CENTER FOR AUTOMOTIVE RESEARCH

13. PREVIOUS OSU WORK Algorithm: 5. The best (optimal) center-line candidates is determined using a smoothness optimality criterion. 6. These optimal candidates will be used to calculate the road curve parameters. CENTER FOR AUTOMOTIVE RESEARCH

14. TO BE DONE Integrate the functionality of Multiple Lanes Splitting and Merging Lanes More Complex Road Structures Use other modalities with vision for a more reliable system CENTER FOR AUTOMOTIVE RESEARCH

15. MY RESEARCH PLAN First Task: Test different existing algorithms with video test data collected by Keith Redmill. CENTER FOR AUTOMOTIVE RESEARCH

16. REFERENCES Pomerleau D.: Rapidly adapting lateral position handler, IEEE Intelligent Vehicles Symposium (1995) Broggi, A., Cattani, S.: An agent based evolutionary approach to path detection for off-road vehicle guidance. Pattern Recognition Letters 27, 1164 1173 (2006) Wu, S., Chiang, H., Perng, J., Chen, C.,Wu, B., Lee, T.: The heterogeneous systems integration design and implementation for lane keeping on a vehicle. IEEE Trans. Intell. Transp. Syst. 9, 246 263 (2008) Zhang, G., Zheng, N., Cui, C., Yan, Y., Yuan, Z.: An efficient road detection method in noisy urban environment. In: IEEE Intelligent Vehicles Symposium (2009) Samadzadegan, F., Sarafraz, A., Tabibi, M.: Automatic lane detection in image sequences for vision based navigation purposes. In: ISPRS Image Engineering and Vision Metrology (2006) CENTER FOR AUTOMOTIVE RESEARCH

17. REFERENCES Nieto, M., Salgado, L., Jaureguizar, F., Arrospide, J.: Robust multiple lane road modeling based on perspective analysis. In: International Conference on Image Processing, pp. 2396 2399 (2008) McCall, J., Trivedi, M.: Video-based lane estimation and tracking for driver assistance: survey, system, and evaluation. IEEE Trans. Intell. Transp. Syst. 7, 20 37 (2006) Huang, A.S., Moore, D., Antone, M., Olson, E., Teller, S.: Finding multiple lanes in urban road networks with vision and LIDAR. Auton. Robots 26, 103 122 (2009) Shi, X., Kong, B., Zheng, F.: A new lane detection method based on feature pattern. In: International Congress on Image and Signal Processing, pp. 1 5 (2009) Lipski, C., Scholz, B., Berger, K., Linz, C., Stich, T., Magnor, M.: A fast and robust approach to lane marking detection and lane tracking. In: Southwest Symposium on Image Analysis and Interpretation, pp. 57 60 (2008) CENTER FOR AUTOMOTIVE RESEARCH

18. REFERENCES Cheng, H., Jeng, B., Tseng, P., Fan, K.: Lane detection with moving vehicles in the traffic scenes. IEEE Trans. Intell. Transp. Syst. 7, 571 582 (2006) Yamaguchi,K.,Watanabe,A., Naito, T., Ninomiya,Y.: Road region estimation using a sequence of monocular images. In: International Conference on Pattern Recognition, pp. 1 4 (2008) Alvarez, J., Lopez, A., Baldrich, R.: Shadow resistant road segmentation from a mobile monocular system. In: Iberian Conference on Pattern Recognition and Image Analysis, II, 9 16 (2007) Nefian, A., Bradski, G.: Detection of drivable corridors for off-road autonomous navigation. In: International Conference on Image Processing, pp. 3025 3028 (2006) Alon, Y., Ferencz, A., Shashua, A.: Off-road path following using region classification and geometric projection constraints. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. I, pp. 689 696 (2006) CENTER FOR AUTOMOTIVE RESEARCH

19. REFERENCES Rasmussen, C., Korah, T.: On-vehicle and aerial texture analysis for vision- based desert road following. In: CVPR Workshop on Machine Vision for Intelligent Vehicles, vol. III, p. 66 (2005) Kong, H., Audibert, J., Ponce, J.: Vanishing point detection for road detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 96 103 (2009) Borkar, A., Hayes, M., Smith, M.: Robust lane detection and tracking with RANSAC and Kalman filter. In: International Conference on Image Processing, pp. 3261 3264 (2009) Jiang, Y., Gao, F., Xu, G.: Computer vision-based multiple-lane detection on straight road and in a curve. In: Image Analysis and Signal Processing, pp. 114 117 (2010) Wang, Y., Teoh, E., Shen, D.: Lane detection and tracking using b-snake. Image Vis. Comput. 22, 269 280 (2004) CENTER FOR AUTOMOTIVE RESEARCH

20. REFERENCES Sawano, H., Okada, M.: A road extraction method by an active contour model with inertia and differential features. IEICE Trans. Inf. Syst. E 89D, 2257 2267 (2006) Kim, Z.: Robust lane detection and tracking in challenging scenarios. IEEE Trans. Intell. Transp. Syst. 9, 16 26 (2008) Danescu, R., Nedevschi, S.: Probabilistic lane tracking in difficult road scenarios using stereovision. IEEE Trans. Intell. Transp. Syst. 10, 272 282 (2009) Jiang, R., Klette, R., Vaudrey, T., Wang, S.: New lane model and distance transform for lane detection and tracking. In: Computer Analysis of Images and Patterns, pp. 1044 1052 (2009) Beck, J., Stiller, C.: Non-Parametric Lane Estimation in Urban Environments. IEEE Intelligent Vehicles Symp. (2014) Kang, S., Lee, S., Hur, J., Seo, S.: Multi-lane Detection based on Accurate Geometric Lane Estimation in Highway Scenarios. IEEE Intelligent Vehicles Symp. (2014) CENTER FOR AUTOMOTIVE RESEARCH