This blog post will discuss the effectiveness of some Machine Learning (ML) models in the context of training Autonomous Vehicles (AVs) as well as how to overcome some of their drawbacks.
For a long time, ML has been used to detect and extract meaningful patterns from data sets, with its increase in performance being directly correlated to the data set’s size. [1]
Its usefulness stems from the need to solve problems which the human development of algorithms for would be more costly that the actual efforts towards finding their solutions. [2]
There are many different models and approaches which may be more suitable than the next depending on a few different factors, mainly the size and quality of the training data, the accuracy and/or interpretability of the output, speed and/or training time, linearity, etc.. [3]
There are also approaches that can learn without using a data set, such as Q-Learning [4], Reinforcement Learning [5] and Neuroevolution of Augmenting Topologies [6], where the learning patterns emerge from rewarding accurately executed actions and punishing incorrectly executed actions.
Because there are many factors that go into considering how successful a fully autonomous vehicle is at fully executing driving operations all on its own [7], we will be looking at this using a more simplistic approach.
As such, let’s take the following example into consideration, using a generational approach to teach an F1 car how to drive around a racetrack. [8]
In this approach it is possible to observe how the algorithm overcomes obstacles such as the first right and left turn, as well as how many generations it takes to complete a full lap around the racetrack.
It is interesting to observe how, given the simplicity of this approach which solely focuses on completing a lap around the racetrack, it demonstrates the learning effectiveness of ML models regarding vehicle automation. As expected, this is nowhere near sufficient to advocate for its effectiveness in a real-world scenario, flooded by rules and stimulus which certainly take more than just five rays cast to measure obstacle distance, but it is certainly enough to demonstrate that it has the potential to be a worthy approach.
It is also interesting to note that throughout the ten generations observed in this approach, there was a demonstrable successful progression with each subsequent one. One can only wonder how else this model would’ve optimized its technique had it been given the time to run through more generations.
So, what does this bring us to? What are the next steps to look at before reaching a solution good enough to test in the real world? Perhaps the inclusion of pedestrian detection models such as YOLO [9], road signs and sensors like cameras, LiDARS/RADARS, Thermal Imagery and/or Ultrasound scanning which can detect and interpret real-world aspects such as atmospheric changes and other inputs which can be replicated in a simulation. Of course, this is not even enough to scratch the surface of the required needs, but perhaps a sufficiently good place to start.
Written by: Tiago Ferreira – Software Developer
Please get in touch if you have any questions or have got a topic in mind that you would like us to write about. You can submit your questions / topics via: Tech Blog Questions / Topic Suggestion
[1] Shai Shalev-Shwartz and Shai Ben-David (2014) “Understanding Machine Learning: From Theory to Algorithms”. Cambridge University Press pp. 3.
[2] Ethem Alpaydin (2014) “Introduction to Machine Learning (Third ed.)” MIT.
[3] Yogita Kinha (2022) “An Easy Guide to Choose the Right Machine Learning Algorithm”. Kdnuggets.com.
[4] Melo, Francisco S. “Convergence of Q-Learning: A Simple Proof”.
[5] databasecamp.de (2022) “Reinforcement Learning – Simply Explained!” databasecamp.de.
[6] Felix Chalumeau, Raphael Boige, Bryan Lim, Valentin Macé, Maxime Allard, Arthus Flajolet, Antoine Cully, Thomas Pierrot (2022) “Neuroevolution is a Competitive Alternative to Reinforcement Learning for Skill Discovery” Cornell University
[7] David Coffin, Sarah Oliver and John VerWey (2019) “Building Vehicle Autonomy: Sensors, Semiconductors, Software and U.S Competitiveness” Office of Industries, Working Paper ID-063.
[8] Skrelo (2022) “AI Learns to Drive an F1 Car” Youtube https://www.youtube.com/watch?v=YesQ_B9zu_Y
[9] Xun Zuo, Jiaojun Li, Jie Huang, Fan Yang, Tian Qiu and Yand Jiang (2021) “Pedestrian detection based on one-stage YOLO algorithm” Journal of Physics: Conference Series.
