Accelerating AI Adoption: From Design to Deployment in Mobility
Myrtle Binil Rajendrababu, MathWorks
Jayanth Balaji Avanashilingam, MathWorks
Koustubh Shirke, MathWorks
Nikita Pinto, MathWorks
AI is revolutionizing the mobility sector by enhancing vehicle capabilities, optimizing traffic systems, and supporting sustainable transportation solutions. The integration of AI technologies into mobility platforms enables advancements such as autonomous driving, real-time traffic management, and smart city infrastructure. AI-driven analytics and predictive modeling offer significant improvements including accelerated vehicle model development and sensor reductions. As AI continues to evolve, it reshapes the landscape of mobility, providing increased safety, efficiency, and adaptability.
In this session, we begin with an overview of AI’s impact on mobility, followed by an in-depth exploration of the following topics:
- Technological foundations: Examine the core technologies enabling AI in mobility, including machine learning algorithms, sensor integration, and data-driven architectures.
- Deployment challenges: Discuss the hurdles in deploying AI solutions, such as data privacy, ethical considerations, and regulatory compliance.
- Deployment strategies: Explore strategies for scalable and efficient AI deployment, focusing on infrastructure, iterative testing, and cross-industry collaboration.
- Legacy system integration: Address the transition from traditional systems to AI-enhanced architectures, highlighting modular and scalable design principles.
- Industry and academia collaborations: Learn how industry-academia collaboration is reshaping the adoptability of AI in automotive applications.
This session provides insights into AI’s transformative potential in mobility, emphasizing design, deployment, and future innovations.
Published: 22 Dec 2024
Welcome back, everyone. I hope you all had a refresher and charge up for our Tech Talk. So as all we know, the AI traction in automotive industry has grown significantly over the years, transforming the field by enhancing vehicle safety, efficiency, and the overall driving experience. These advancements are already visible in our latest on-road vehicles across various applications.
We got it here from our customer talks too. We actively support our customers developing and deploying these AI applications in production environments, and we have numerous use cases from industry. As you see in the screen, we have Continental using the data-driven engine temperature model for ECU development, Renault using deep learning networks to estimate the NOx estimation, fleet analysis to meet BS6 emission at Honda, reduced order model to predict engine performance by Cummins, et cetera.
However, the rapid adoption in the AI also presents challenges to engineers. Key questions arise are, What are the core technology foundation for developing AI, and how do we integrate with the existing system since we all know that in the automotive industry it is very, very rarely we start from the scratch and whether the deployment is straightforward or do we have challenges in deployment.
And as application developers and engineers, we constantly evaluate this particular space and find answers for them. Finally, it is also essential to see how academia is also evolving to address and support the AI-driven development trends in automotive industry. To discuss more on this, let me welcome Jayanth, Senior Application Engineer at MathWorks.
Jayanth specializes in artificial intelligence, focusing on data analysis applications with time series data. He has eight years of experience in research and industry, developing AI, machine learning, and deep learning solutions for customers.
With that, let me welcome Koustubh, who is also a Senior Application Engineer at MathWorks. Koustubh specializes in helping customers to build efficient data analytic and AI workflow. He works closely with customers on projects in data-driven modeling, predictive maintenance, reduced order modeling, Edge, and cloud deployments. Welcome, Koustubh.
We will also welcome Nikita. Nikita is the AI Academic Liaison for Asia-Pacific methods. She focuses on AI techniques for scientific and engineering applications, collaborating with educators, researchers, and students across India, China, Japan, Korea, Singapore, and Australia. She has also worked on interdisciplinary projects with international collaborators, government labs, and independent researchers.
So let me take this question from Koustubh. So starting from you, Koustubh, can you give some initial thoughts on AI challenges and the core technology foundations that can help building AI models?
Yeah, definitely. So based on our experience, most of the engineers, whether they are from mechanical side, electronic side, they are trying to integrate the AI in different projects, from different domains. But from the Gartner survey, what we have got, there are some barriers which normally block the roads when it comes to developing the AI models and deploying those AI models into the production.
So maybe let me list down maybe top three barriers. So first is nothing but the maybe integration of the existing technology. So you have got the right data in the right quantity. You have developed the model. And it's performing very well at desktop level. Now, issue is how you are going to take that model into the production.
So for example, when we talk about the automotive, how we are putting that model into the hardware, whether there is a compatibility between the hardware on that model is not, or if you want to put that model on the cloud, whether you are able to integrate that very well so that it will perform well in the same way in which it performs at the desktop level. So this is the first one.
Second one is the data complexity. First is you should be having the data. And that data should represent the actual physical system. Suppose I am training some AI model for the transmission. I should have the data which represent the physical behavior of the transmission. Then the data is raw. So you should be able to clean the data by integrating some your domain knowledge. For example, suppose you are cleaning the outliers. You should be aware of whether those outliers are really outliers or those are telling something about my system.
So that's why this is the main point when it comes to data complexity, as well as the data should be in right quantity to train the robust models. And third is very important. That is the lack of skills. Since we are engineers, we spend a lot of time in learning the domains, someone from mechanical side, someone from electronic side. So maybe the four years plus two years, maybe seven, eight years, we normally spend only in the learning domain.
But now, when it comes to the AI domain, you need to do a lot of additional things normally we didn't learn in our maybe engineering or maybe the B tech. So for example, data preprocessing, which of the techniques are more effective, how to choose the right model, how to tune those models, how to deploy the models. So in case there is a lack of the skills, it will take time to learn those techniques as well as to implement those techniques. It can delay the complete projects. So these are maybe the top three barriers which normally we face while establishing any kind of AI-based project.
Can you also give some insights on the core technology foundations for the CA model?
Yeah, so maybe when it comes to the core technology in AI in automotive world, let me put it into the two buckets. One is there are some AI technologies which are being used for the development phase. And second bucket, the AI technologies which is being used for the production. So first, let me talk about first bucket.
When I say development, like we have seen that simulation is invented or being used to reduce as well as to skip the prototype, sometimes to skip, sometimes to reduce. Now AI has gone one step ahead of it. Suppose you are working on the high-fidelity simulations. In case you want to speed up those simulations, you replace some part of your model with the AI-based model so that your simulation will be faster. We normally call it as reduced order models.
Exactly. So that is what also we heard from the customer talks from MBRDI. They are replacing some of their sensors. Coming to Jayanth. So when you go for the integration with the legacy systems and also for the deployment, can you share some initial thoughts on the scene?
Definitely. So just to reiterate, we are talking about integration of AI, which means AI cannot go all alone. That's why it need to go together with the engineering systems, which means there will be legacy system or legacy controller where AI can be additional add-on or ad hoc component to improvise the existing workflows. That's where the system simulation or integration comes into play. And one of the important step is no one want just the simulation to be residing just on laptop or desktop or just as a research problem.
Eventually, they want to take it to the final production level. That's where deployment comes into play. So whatever you develop, you test it, validate it together with other components or integrate with other components, and eventually you take it to the system or production-ready workflows where you can able to see the impact of AI. So that forms the remaining workflows of AI.
Great, Jayanth. So we can clearly see there are mainly four buckets. That is the data preparation, AI modeling, simulation and test, and deployment. Before you get into-- before we get into deeper on seeing each of these particular buckets and see the technicality of that, I would like to hear from Nikita. What does that academic collaboration mean for industry, and how the skills are being cultivated for these specialized talents?
Yeah, thanks. So we can think of it from the three main pillars that exist between industry and academia, which is industry, education, and research. So one of the things that we consistently hear from companies in the automotive space who are interested in developing software and AI-based models is that there is a consistent lack of talent. So students are graduating, but they're not ready to immediately get started with working on these AI or software workflows.
So in that case, there is this gap that exists. And how is it that industry can work with academia so that both needs are met in terms of a workforce as well as industry-ready courses? And on the other hand, we have research, academic research, which is very focused on theoretical knowledge and on the algorithms part.
And while this is very important and it takes the field forward in areas like predictive maintenance, ROMs, computer vision, autonomous systems and so on, we noticed that academia and research, they have these very specific challenges in the practical application. So for example, the lack of industry standard data sets or the lack of testing and validation facilities, how do I know that the algorithm I've developed can go onto a hardware target and run for a practical application?
So what we noticed now is that there are these gaps that exist on every side. And the question is, are we able to bridge this gap so that we can ensure a steady flow of both talent and innovative technological ideas from academia into industry?
Thanks, Nikita. That's a very good overview that we have made three things. One is the education, industry, and the research. And we as MathWorks are bridging this gap to get the specialized talents for the AI development. Now, we can get into a little bit deeper on what exactly is this core technology foundation. So we heard from Koustubh that there is data-driven workflow in the core technology foundation. Jayanth, can you take some examples and discuss on how this core technology foundation works?
Sure. But before getting over there, I have a small counter question back to you. So I believe you are two decades in the industry. Do you think a single team or a single individual take the complete ownership of end-to-end product development or project development? Because that is a segue towards my discussion as well.
Good question, yeah. I didn't expect that. We all heard from the customers. We have many user stories. And today, we heard from Tata Motors as well as from KPIT. And it is definitely not a single point of work. There are multiple challenges involved, multiple teams involved into that. So there will be embedded engineers, data engineers coming in picture. And with all of them together, we should bridge this gap.
Exactly. The point for AI is also same. Irrespective of whether you are developing software for AI, multiple team members are involved. So in this case also, from our experience working with several automotive OEMs and tier ones, what we came to know that there will be a group of engineers who are domain experts who define the problem statements and things like that, and there will be a certain set of engineers who take care of the deployment side of it, who generate the code from the problem statements they solve and take it to the integration workflows.
That's where a streamlined workflow is necessary. And we will take a couple of examples, what we are hearing from the end users, and we'll discuss in detail. Reiterating back to what Koustubh was mentioning, so he talked about AI for reduced order modeling and sensor modeling and things like that. In similar context, when we hear about automotive OEMs, we hear two workflows.
One, AI is used, whether they can leverage AI for accelerating their overall simulation efforts. And the second problem statement is primarily focused around AI for algorithm development, where the system generates a lot of data and they want to get value adds from the data that is being generated from your assets. In this case, it is automotive assets. It can be battery, or it can be engine and so on and so forth.
So for this particular problem statement our focus today, let's focus on AI for algorithm development bucket. And let's try to get into details and intricacies what we are talking about. And one of the features we are commonly hearing is, can I leverage my data to solve the predictive maintenance use case of my asset?
So it can be a multiple problem statement, like I want to estimate my maintenance cycle so that I can able to maintain my inventory or I can be able to reduce my downtime. So for this context, and we are hearing so much from morning about electrification and trends in electrification and AI, let's take a simple example from the electrification standpoint. So let's consider that you are working in a power converter, and you want to do the predictive maintenance of the power converter.
So the way we are considering this application is a power converter is used both in powertrain as well as charging infrastructure as well as multiple components inside. Now let us assume you want to add value adds to this data that you capture from your system. What you can do, you can able to ask the anomaly detection workflow. Like, you can estimate whether my machinery is behaving perfectly or not, or you can able to perform fault classification or different types of fault detection, or eventually whether I can able to estimate the remaining useful life of my systems.
So these are common questions that you can able to ask from your data. When you're coming back to the point of integrating AI to the system, it cannot go all alone or it cannot go independent. Let's take a quick look how it is integrated to the existing system. For instance, you have the complete power inverter or power converter workflow. On top of that, you want to validate your algorithm and test it before taking it to the production phase. That's where the integration comes into play, where you develop any algorithms and you can able to bring it to the simulation and validation platform. You can validate the correctness of your algorithm before taking it to production.
This forms a crucial integration between the legacy software system plus AI-based workflows because AI cannot go all alone. And this is not new to industry. For example, we have multiple customer use cases. In this case, I'm just referring to a case from Daihatsu, where they used AI workflows to estimate or to classify the engine sounds.
So in this case also, they want to map with the legacy workflows, and they integrate AI along with that so that they could able to quickly perform analysis of their workflows by integrating AI with their legacy systems. So that's a quick answer.
Thanks, Jayanth. So that gives a very good overview of the core technology foundation. That is mainly the first two buckets, the data preparation and the AI modeling, how it can actually work. So I will just put a small poll to the audience. So do you believe that the AI modeling can solve some of your problems, existing problems in your current workflow? How many of you have identified some of these problems in your current workflow? Maybe a hands up or something.
OK, they are all learning. Yeah. So we will see more traction in this. Now, when we talk about these core technology foundation, these technology foundation also doesn't come very easily, which means that we will have some challenges on that. So let me hand over to Koustubh to talk some of the challenges which typically an engineer faces when they are developing these AI models.
All right, so let me map to those maybe four buckets which we talked about, the data preprocessing, model training, then integration, then the deployment. So these are maybe some listed challenges which engineers are facing while establishing the AI-driven workflow. First, the accuracy. So accuracy is the first priority. So you should be having the right diversified data as well as you also need to choose the right model, fine tune it in order to get the accurate model.
And that should be validated very well with your data from your actual system. So that is the first priority. That is the accuracy. Second is the development time. So you need to get the data. You need to clean the data which takes around 70% of your whole time. Then you need to train the model. You need to check out the multiple models which is fitting well for your data. And then the integration with the existing system. It takes a lot of time. So again, it can delay the process. This is effective testing.
Let me share one example with you that is-- so we did one AI modeling for the engine emission analysis predictions. And we used the data only for maybe the XYZ altitude. And then we deploy the model to one location, which is further than that XYZ altitude. So both the altitude levels are different. That's why we are getting a lot of errors after deployment. The reason is that we didn't take care of much testing with the diversified altitude levels.
So in this way, normally, we should be having a right workflow to test the models with diversified data or the data which we didn't use for training those models. This is the compatibility. Like I mentioned earlier also, whatever the models you are training, those should be compatible with the deployment workflow. That is one way. Let me take again one more example. We are working with one hill team which want to establish the wrong workflow. So they want to replace one part of maybe vehicle with the AI-based models. And the issue was that that model is already retained by the data science team by using the open source that is Python.
Now, the question is that how we are going to integrate that prebuilt model into the plant model and generate the code for the hill. So this is the issue from different third party tool maybe to your tool or from your tool to the third party production environment.
Next is Suppose you have trained the data with maybe in vehicle variant one. You deploy the model, and suppose you make some changes, for example, you are changing some maybe turbocharger, you are changing some calibration. And if you are giving-- if you are using the same model with the new engine variants or new vehicle variants, it won't work effectively. So that deployed model should be refined robust over time. That's how you update the model, how we are going to update that, how we are going to maintain that model.
Next, the robustness. Your model should be consistent with respect to performance, irrespective of where it is deployed, irrespective of which kind of ambient conditions, operating conditions those models are suffering from. So this is one.
Last main point is the latency. So whatever the estimation speed, prediction speeds you are getting at the desktop level, ideally we should expect the same after deployment, maybe at the hardware level. But ideally, it won't happen because there are some-- maybe there is a difference in a static dynamic memory. So again, let me take one example that-- there is one use case where we train the model which can estimate the temperature of the subsystem.
Based on that, once we get the output from the temperature, then those will be feeding back to the cooling actuation system. It's working well in the system simulation. But to deploy it, if there is a latency, it can delay that cooling system. So these are maybe some challenges. We can maybe talk about that further, how we can overcome those challenges.
Sure, Koustubh, that gives a very clear indication that it's not simple, but it's not achievable also. But I like the maintainability part very much. So we are talking a lot about the SDV. Or you heard a lot about the SDV keynotes from the keynote speaker from the morning sessions. So when it comes to the maintainability, effectively updating that model is very much required. And we can actually hear from these particular talks how this can also be achieved going forward.
So before we go into the next bucket, I would like to understand from Nikita-- so we discussed initially that the AI model also requires-- or the engineers who build the AI model request some specialized skills. So along with the classic embedded development knowledge, it is also necessary that they have to upskill with the data engineers and the data processing methods. So let us hear from Nikita how this academia is shaping out and how we are supporting as MathWorks to nourish these talents with some examples.
Yeah, so I think that many of us here today, we've gone through some university traditional courses, through vehicle dynamics, control systems, embedded programming. And newer generations of engineers have also gone through AI courses. And when you look at the AI courses, the focus there is really, like I said, on the theoretical concept. But that does not guarantee that an engineer who graduates will be ready to enter the workforce and work on these real practical applications.
So in that case, the first step is to supplement the theoretical knowledge with numerical computational way of thinking. So for example, you can see a vehicle dynamics course from a professor at IIT Madras who uses simulation in addition to classroom experience to show students how things are working.
Another example is the HAW Hamburg that actually has a remote control programmable car. So when the students learn about vehicle dynamics, when they learn about controls and actuating things like the brakes, computer vision modules, and so on, they're able to try these out in a real way on the RC.
The next step after supplementing classroom would be capstone projects. So we've done our final year projects, but MathWorks has the MATLAB and Simulink challenge projects, which are a collection of projects based on real industry open questions. And this is a university from India. They have formed a multidisciplinary team, and they've built out this project that looks at sensing and perception for autonomous vehicles.
And the third option I would say that really takes technological innovation to the next level would be the automotive student competitions. So there is a nice picture here of this group based out of Munich. And this is for the Indy Autonomous Challenge. And it's a group of around 40 undergrads and a couple of PhD students. And what they've done is they've-- this car that you can see speeding along the track, all of this is done autonomously.
So the students actually had to go from designing the car-- designing the software for the car to looking at, how do you sense the environment? Once you've sensed the environment, how do you navigate and plan your path? And once that is done, how do you develop the control systems needed to actuate the parts of your car, so the throttle, the brake, the steering?
And doing all of this, they used hardware-in-the-loop simulations, and they used automated code generation. And they were finally able to participate and win in this challenge, reaching an average speed of around 218 kilometers per hour. And that's really something impressive when you see it coming out of students.
In India, we have automotive competitions as well. There is the SAE India Baja competition, which MathWorks has been supporting for the last 12 years. And as this competition looks towards the autonomous track as well, MathWorks is playing a huge role as a key contributor in making this process happen.
Thanks, Nikita. So this gives a very clear picture that we as MathWorks are collaborating with industry as well as with academia to actually bridge this gap and bring the students to have that experience on road and in the field so that when they come to industry, their gaps in the data modeling or in the AI will be actually covered.
So before we get into more details about the next four buckets, let us take some questions from audience.
So there is one question which has come is that data preparation takes time. So is there any techniques which MathWorks provide to ease this process?
Very good question. So that comes into the first bucket of the complete AI modeling. Koustubh, can you take it?
Yeah. So data preprocessing has different categories. First is maybe resampling the data. If you want data with some higher frequency, you want to upsample it or downsample it. Second is finding the outliers as well as removing or replacing those outliers. Then cleaning the data. So suppose the data is very noisy, how you are going to clean that data? And last thing is that maybe suppose you are getting the two data sets from the two different hardwares having the different frequencies, how we are going to merge it.
So you resample it, make it one sampling frequency, then merge it. So for all these kind of steps, normally there are some live tasks. So there are some-- we also call it as mini apps. So interactively, you can load the data. You can choose the methods, for example, data outlier, which kind of methods you want to invoke for outlier detection, which methods you want to do. You have to just-- by using some dropdown menus, you have to just define the methods, automatically data will be cleaned, and you can also able to see what is the raw data and what is the clean data.
And after that, once you do everything interactively, you can also automatically generate the MATLAB code. So maybe we call this a low-code, no-code methodology when it comes to data preprocessing. It saves a lot of time while choosing the algorithm and writing the code for those respective strategies.
I think one example which was showed by Tata Motors was aligning to almost in this--
Again, there is one diagnostic feature designer app. Suppose you want to do the feature extraction. So you have very less number of features which is not sufficient for training the right or robust model. You need to extract some more features in time frequency as well as the time frequency domain. So there is a diagnostic feature app. Again, it is interactive app where you load the data, choose the and all these things, choose the method, extract the features. Suppose I am extracting-- suppose from two features, I'm extracting 50 features. Then you can also rank the features based on their predictive power. And after all the interactions, again, at the back end, automatically MATLAB code will get generated.
To summarize that, yes, we have solutions, and there are multiple apps or sub apps in our toolboxes to do this. And you can utilize this to have this data preparation.
Yeah, one more on the modeling side. So is there any prebuilt model to support the automotive application development when it comes to AI?
May I take the question? Do we have any prebuilt models?
Yes.
So ideally, like in morning, we also discussed about multiple workflows in his keynote. Primarily, if you want to build a complete white box model from scratch, you can able to quickly leverage the inbuilt apps to build it. And we also have something called MATLAB Deep Learning Model Hub, where the research model will be published over there, and we'll update every month ideally, so people can use that. Or if your data science team has some prebuilt model using some open source platform, even you can able to bring that.
So to summarize, we have workflows by which you can able to bring domain-specific models. And also, we have certain models which are based on shipping examples we have, which you can readily use to quickly accelerate your modeling workflows.
Thanks, Jayanth. OK, moving forward. So again, maybe to Jayanth. So once we have this data as well as the AI model is ready there, then the next big challenge is to how we integrate with the existing workflow. So we clearly heard from MBRDI it is not from the scratch. So there is an example. But can you get into a little bit deeper how it can be integrated with the existing workflow?
Definitely. But a quick question to the audience-- are you using model-based design in your day-to-day work to develop the application software? Quick hand raises. I could see a few of them raise their hand. Great.
So how do you start your application development for the interest of other people? Ideally, we start with requirements for application, what we want to do. Then we'll get into the design phase, where we develop some algorithms, components, and things like that. We do the implementation by generating the target-specific codes depending upon what hardware you have. And finally, we'll integrate or stitch it together.
In course of time, we will do the test and verification process because we always-- it's good to fail early rather than waiting towards the end. So that's where the workflows will come into play, where we verify the workflows in between to understand whether we are having any bugs or flaws and things like that. And typically, we also call this as a V workflow or V cycle workflows.
Considering this as a blueprint, let's try to attach the data-driven techniques, which Koustubh explained about data preparation. So ideally, whatever we discussed about the requirements in the traditional software development, especially when it comes to a workflow, it starts with data because we capture the data. From the data, what we'll get to do the workflow is we try to develop the models. So that's where AI modeling comes into play, like a designing phase, when you consider your previous blueprint, which we discussed now.
And similar to the previous use case, we cannot directly take the AI model all the way, go to the deployment in the critical products. So that's where simulation and testing comes into play. Similar to the example, what we discussed about predictive maintenance use case, where we simulate, test all the possible scenarios so that AI model is very well validated. Once it is validated, we'll get into the phase of code generation, which means we do not want our model just sitting on our laptops or desktops. Eventually, we want to target the end-to-end hardware.
It can be embedded system or it can be enterprise system. Eventually, we want to integrate with the existing workflow and deploy it. That's where, when you consider about the four bucket workflow, which we showcased earlier, it is tightly aligning with your legacy MBD workflow, which means there is no difference between the MBD workflow and AI workflow. So even though we understand this is different and different teams develop and things like that, so you can able to think that even if you are MBD engineers, you can able to come and your AI developer or a data science engineer so that you can able to easily bring those two verticals closely and easily, and you can able to achieve the task that you are trying to solve a problem.
So this is not new again to the industry. For example, here is a case from Denso. So what they did is they built the AI models primarily for the control piece of it, to do the engine control kind of things. And eventually, what they did is they integrated with their existing model-based development workflow, where AI model is also integrated together. And they could able to verify, validate, and directly take it to the ECU implementation.
Why we are reiterating it? It's not new to automotive industry. People are already using MBD, and people are using AI, and it's easy to integrate with the help of MathWorks tools. I believe that answers.
Thanks, Jayanth. So the same messaging actually in the morning keynote also, where Avik actually told about the different workflows where, taking an example of virtual senses, how from the data preparation to the deployment can be possible and how it can be stitched to your model-based design workflow.
So the idea here is-- or the message here is that it is not completely a different workflow. You can stitch it together to your existing model-based design workflow. Now, coming to the next step, when you design everything, the next important step is to verify and validate. So Koustubh, from your experience, the verification of validation remains the same in the traditional model-based design or in the AI workflow? Is there some differences?
So let me first maybe talk about the MBD. So MBD, Model-Based Design, has established the verification and validation workflow. So it is nothing but the MBD has some plant models or some models which are based on some mathematical equation, logic, or some physics-based formula. So for that, the workflow is already established. So now, when we add the AI flavor or AI component into the MBD workflow, we need to do some additional tasks because the AI models are a little bit gray or maybe the black box models. So they are not much explainable.
So also, we need to do some additional tasks, for example explanatory. So that is one more task we need to do where that model should explain itself, how it is going to estimate the value or class based on which logic. That should be explainable.
Second, also, we call that-- there are some tests available. Normally, we call it as maybe the text or maybe some more test available which can normally check whether that integrated part of AI in MBD is performing or verified well or not, whether those are meeting our requirement which we want after the deployment.
And this is for maybe the verification, validation also. Again, same, we need to check whether that AI model is very much robust to perform in diverse road conditions, different ambient conditions, different driving conditions. So all these different kind of things we need to check before we are putting those AI plus MBD model into the production. So those are maybe three or four techniques. Normally, I have just talked about that is the explainability, rigor and test, then robustness, as well as the checking with respect to diverse maybe operating as well as driving conditions.
Now, let me also give you one example. So we work with for the AI in electrification workflow, where they developed some machine learning or deep-learning-based model which can maybe estimate the SOH, State Of Health, which is nothing but the remaining useful life of the battery. They did it in MATLAB, then integrated that into the simulation plant model. And they did the verification and validation for maybe the ISO 26262 certification. This is the good example or the good actual use case, which is actually implemented where that is V and V, Verification/Validation for the model-based design.
Thanks, Koustubh. So in summary, AI models, when you do verification and validation, the important things which need to be considered is explainability, rigor and trust, robustness, and the real-world scenarios. So there is a difference from the traditional model-based design when you come to an AI-based software development.
In the same-- so Koustubh, also from this user story, talked about the certification. So when an auto-- when you consider an automotive embedded system, certification plays an important role because the safety and the security is very important for any applications. So Nikita, to you, how the industry is shaping in the area of certifications? Is there new certifications evolving in the industries? Can you give some insights on that?
Sure. Sure. So the idea of AI governance is very important for us. This is because there is an increased focus from governments on regulation and certification. In the automotive space, for example, there is the ISO 8800. And this looks at AI safety for road vehicles. This is something that a draft is currently under process of development.
And similarly, in other areas or other fields you have, such as in aerospace and in medical devices as well, there are ongoing efforts. But while these are vertical-specific, there is also overarching efforts. And a great example of this is the EU AI Act, which came out earlier this year.
So keeping in mind all of these developments, we also need to understand how workflows are to evolve and to keep an eye on these developments and make sure that we are ready for them.
Thanks, Nikita. So the automotive industry is evolving with this particular certificate, and we are also, as MathWorks, as a tooling company, we are also watching this and seeing how these certifications can also be incorporated in our workflows.
So when Koustubh talked about the verification and validation workflow, there are two important aspects. One is the explainability, and the second one is the robustness. So taking that question back again to Nikita, can you get into a little bit deeper what exactly the explainability means and the robustness in the verification and validation workflow?
So I think if we want to work with AI models, the first thing that we need to address is this. Often a criticism of AI models is that they're very black box in nature. And if I'm a data scientist, an engineer, or an AI developer, then I'm going to, let's say, train this AI model, and I'm going to get it up to some level of accuracy on the data. And this will be integrated into a larger system that is going to go ahead and make decisions or recommendations.
And this will bring out very important questions from key stakeholders, which may be business owners, other companies I'm working with, customers as well as regulatory bodies. And these questions will be around, How much can I trust your model? When will your model fail? Why is it making a particular decision?
So if I as the engineer or the scientist am able to bring in some idea of explainability to give some insight into either what is the working of this model or what is the relationship between the inputs and the outputs that the model is predicting-- if I'm able to do that and therefore introduce explainability, I'm in a better position to start answering some of these crucial questions.
And once we [AUDIO OUT] also want to look at the robustness and the trustworthiness of these models. Let's start with robustness. You would have seen in the talk this morning where we made this-- the idea that an AI model can change its output and make a misclassification or an error based on a very small or imperceptible change in the input. This is called a perturbation or an adversarial attack.
And because this is possible now, we want to ensure that our model can perform across a range of inputs. But it's not possible to go and test all of these hundreds of thousands of millions of inputs. So in that case, what we can do is turn towards formal methods of verification, where we introduce some kind of perturbation, which is parametrized by some value. And what that gives us is now a volume, in this case, of images over which we are going to verify the model's performance.
So from that entire volume of images there in the box, the model-- a verification method will tell us that the model performs well. That means its output does not change for a certain proportion of the images. It changes for another proportion of the images, and we cannot say anything conclusively for a certain proportion. So this information becomes very important because we now when it performs robustly and when it does not. And we can take further steps that could include additional advanced techniques like adversarial training.
So that is about the robustness. But the next idea is about the trustworthiness. So if you see the previous slide, there's this idea that we want our model to be able to perform accurately on data that is within context. And when data is out of context or it's out of the distribution on which we've trained on, we want the model to be able to identify that, so it can flag it, reject it, and eventually pass it on to a human being for further consideration.
The way to do this, again, is through verification methods. And here we are passing the model data that is in-sample as well as out of sample with some kind of augmentation. And along with its final decision, we are asking it to assign a confidence, a distribution score. And that's what you can see here. There are these two histograms that looked very distinct from each other. One is from the in-sample, and one is from out-of-sample data.
And having an idea of these histograms can help us set a threshold so that when our final model makes a prediction, we can refer to this detector, and it can help us understand if this is likely from in distribution or out of distribution.
So I'll wrap up about these methods with a very nice user story. So this is a company, Musashi Seimitsu, that manufactures automotive parts. And they are manufacturing these parts, and they want to visually inspect for defects. And this is to the order of 1.3 million parts per month. What they wanted to do is come up with an automated way to perform visual inspection that involves setting up a camera, using some target hardware, and analyzing things on the shop floor itself.
And in addition to making a final decision as to whether this is a defective product or not, they also implemented explainable AI here in the form of Class Activation Mapping or CAM. And what CAM does is it maps the relationship between your class that you've predicted and the part of the image that corresponded to that prediction. So here you can see not only is it telling you OK or defective, it's also overlaying a heat map that is showing you which part of the image really significantly impacted this final decision.
Thanks, Nikita. So the example clearly states about the explainability and the robustness. There are techniques to cover this. And that is very important when it comes to the verification and validation of the AI models. So coming to the last bucket of the complete development is deployment. And as an MBD engineer, we are always concerned about this deployment.
So I will pass this question to Koustubh. So Koustubh, with this e-architecture evolving, we have HPCs. We have multiple platforms which are coming up in an automotive industry where the deployments also need to be done. So it is not only the MBD deployment. It goes for the microprocessor-based or cloud-based deployment and many other things. So can you give a given brief information on what are the different deployment possibilities in developing an AI model?
Yeah, definitely. So what I feel that if you are not putting our trained model into production, whether it is on simulation or whether it is on cloud or embedded hardware, we can't say that we did AI project. So it is incomplete. So that's why-- but just to enable that, our model should be compatible to other platforms, other third-party tools, and so on. So that's why let me recall what we discussed in the core technologies. It is divided into two buckets, development phase as well as the production phase.
Let me again bring that flavor here when I talk about the deployment portfolio. On the maybe right-hand side, you can talk about the production. Suppose you want to put your AI model into the hardware microcontrollers, then you can go with the coder products, where you can generate the C+ code, Cuda code, and really put that into the hardware.
So you generate the automatic code automatically without any human interference as it can reduce the human error as well as save a lot of time. And second is nothing but the compiler. Through compilation, you can deploy the model on the cloud, whether on-premises cloud, private or public cloud. You can also go ahead with the docker workflow. So these are some options through compiler where we are putting the model on the cloud.
On the left-hand side, we can talk about development. So the right-hand side is production. Left-hand is development. First option is that in case you are going to integrate the model into the hill environment. Again, generate the code, sometimes generate the DLL files and put that into the hill setup so that model can run in real-time domain.
Second option is that suppose mostly it is being used for the wrong workflow. Suppose you have developed something in MATLAB. It might be deep learning model or machine learning model. You want to take that into the third-party simulation tool or any other tool. There are two ways to convert that into the ONNX. Then take it to third-party tool.
Second option. We have just maybe introduced that TensorFlow exporter, where you can export your trained model through TensorFlow through the third-party tool. This is how we can cover the complete region when it comes to deploying the model into the production. Let me again talk about the main, which is very trending one, that is, the deploying the model into the hardware.
The issue is that, again-- the same issue also I mentioned as a part of the challenges, maintainability. Suppose I use the data from the vehicle number one. Suppose I am changing the vehicle, how that model will be performing well. So that's why there are two options. One is the incremental learning block. Suppose you have integrated model into the simulation environment. With the new data set, you can retrain the model in the simulation environment itself. That's why normally there are some workflow we call as a incremental learning blocks.
Secondly, suppose you have trained the model, you have deployed the model, and you want to retrain the model on the hardware itself. That's why we normally have workflow we call as a on-device learning. What it does, when you deploy the model on the hardware, you also deploy the additional functions from the MATLAB, which will help to retrain the deployed model again with the new data set. So that model becomes more robust, more refined over the time. So these are some maybe different options for the deployment as well as when it comes to the on-device learning or on-board learning.
Thanks, Koustubh. So that clearly explains how the maintainability can also be addressed with the deployment solutions which we actually have. It looks simple, the deployment, but it has its own challenges. So Jayanth, can you talk about these challenges on deployment?
Definitely. That always brings an interesting point. Like, for example, most of the engineers, we focus on accuracy of the models. So we keep on building the complex models, which gives more accurate. And that is going to be quite challenging when you take it to the actual production. So how many of them attended last year MAC, MATLAB Automotive Conference? Maybe it is worth going and watching a video where a OEM presented. Their data science team developed a model. When they try to generate a code, it was more than their hardware room budget.
So just to give a context, the model was like a decision tree or a forest kind of a model where they developed model sizes more than several MBs. So ideally, for the overall budget, they won't get that much huge room space to store the model or perform the inference on the hardware. That's where, as suggested-- said even though deployment appears to be straightforward, but understanding the hardware is one of the crucial step when it comes to the production-ready project.
So keeping that in mind, there are workflows by which you can able to make the model which consumes less memory footprint by keeping the accuracy intact. So we always develop a model primary focus as accuracy. But if you are going for production, we also want to understand, What is my hardware? Do I have enough memory budget to perform inference learning or deploy the model on the hardware? So that is why the newer workflow, the embedding workflow, compression workflow, where you can able to perform compression on the model-- when I say compression, we are not simply compressing the model and losing more than 90% of accuracy, which means it doesn't make any sense to perform AI workflows on the hardware.
So that's why the compression is an interesting technique, where you can able to perform analysis on the trained model and find which are the unnecessary component in the trained model and how to chop them off such that your memory constraint is matched. So in MathWorks tools, we support two types of compression techniques, one primarily focused on structural compressions, which means you have a structure of a model where you can able to compress the structure of the network in such a way that you reach the required memory footprint on your hardware.
So there are techniques like pruning, projections, and so on and so forth, which changes the structure of your trained model. The other side of it, which we often hear from the OEMs is I have hardware which is specific to certain data types. So my hardware support only fixed-point arithmetic or floating point arithmetic and so on and so forth. That's where you can also do the compression on the data-level compression.
So by effectively choosing the compression techniques, you can able to achieve the goal in such a way that you can able to fit a trained model inside your available hardware without compromising your accuracy, as well as solving your problem statement. So these are some interesting flavors by which you can able to achieve the actual production for AI-based workflows by compressing a model and targeting your embedded hardwares.
So what, Jayanth, you told is, like, in between the last two buckets, you need to introduce one extra bucket, which is actually the compression techniques, which actually solves the problem of deployment into a small memory footprint. So is this the same problem for cloud deployment also?
No, ideally, I would say. Because, for example, let's take an example of ChatGPT models, which is in Zigabyte. And there, we will deploy on the cloud inference, where it has elastic storage and so on and so forth, so that it can able to run seamlessly on cloud. So eventually, it depends on the end targets that you are targeting. If you are working on enterprise system, maybe the compression may not be useful, where your accuracy makes more sense. That's where the compression will be helpful. That's where the compression is not required. But whereas if you're going for embedded targets, where you want to play a trade-off between the model complexity as well as accuracy, such a way that you are solving a problem statement.
Great. So it is up to the engineers to decide where the final deployment is. And based upon that, the compression techniques need to be taken care or not need to be taken care should be decided.
Exactly.
Great. Thanks, Jayanth, for that. So we heard from Koustubh initially, when you develop an AI model, there are quite a lot of challenges. So I want to bring the challenges back again, that is, from accuracy to latency. And let us hear from Koustubh himself. By all these techniques, are we overcoming these challenges?
Yeah. Yeah, definitely. So Jayanth talked about the four bucket workflow, some compression techniques, pruning projection. Nikita also talked about the verification and validation, explainability, how it works with some use case. Let me summarize and map all these solutions which we talked with respect to the challenges which we discussed in earlier slides.
So first is the accuracy. Again, so this is a part of maybe the learning process. There are some techniques. Maybe it starts from the data cleaning. So accuracy first depends on the data cleaning, how your data is, whether it is cleaned or whether it is messy. Then it comes to the model training, choosing the right models, as well as the hyperparameter tuning. So these are maybe some two or three key points which can be used to overcome the accuracy part so that model will be more robust-- first choosing the right model, then tuning the model in the right way.
Second is nothing but the development time. As we talk about the four bucket workflow, we also talked about the lot of mini apps, live tasks, as well as the dedicated apps for the feature extraction, data cleaning, model cleaning. This helps you to easily clean the data, train the model, as well as deploy the model right. So it can save a lot of time so that it can-- there will be no delay with respect to the delivery.
Effective testing, we discussed the verification and validation of the AI model along with the MBD. There are some techniques available, like Nikita explained, which can help you to do the effective testing with the data sets which is not being used for the model training.
Next is compatibility. On the last previous slides, we talked about that development portfolio, production portfolio, where you can put the model into the hardware, maybe into the cloud by using compiler. Then for the third-party integration, ONNX and all these things as well as with the hill. So we are covering embedded hardware, cloud, as well as the third-party simulation environment, which can normally talks about how compatible our MATLAB-based AI model will be.
Then maintainability, we just talked about the incremental learning blocks in simulation domain as well as the on-board learning, which can help you to refine your trained deployed models with the new data sets so that this incremental learning or on-device learning will be taking care of the maintainability issue.
Then robustness. Again, this four bucket workflow, validation will be taking care of the robustness. Last is nothing but the latency. Jayanth talked about the different compression techniques, which normally help to reduce the static as well as dynamic memory space post-deployment by using some techniques like maybe pruning projection, which normally helps you to avoid the latency issues. So these are some challenges we discussed along with those solutions. And so we just talked about that.
OK, Koustubh, so we talked about the challenges. And now, from the different techniques, the four bucket workflow, as well as the compression techniques, we can clearly see how these challenges can be addressed with our tools and our workflow, where the engineers can develop a better AI model considering all the parameters and the data sets what they have. Thanks, Koustubh.
So coming to the last, I would like to hear from Nikita and how MathWorks is collaborating with the industry and academia with some examples to provide what are the different courses and offerings which we are providing to bridge the gap for the engineers or from the students when they come from the academia to the industry.
Yeah. So I know we've talked a lot about the potential for industry-academia collaboration, and I'd like to leave you with some concrete examples of this. So here on the first one you can see is a partnership between Continental and Amrita University.
The courses, this is an automotive course on layered architectures and touches upon industry-standard technologies such as AUTOSAR. Another course that we help facilitate is between Bosch and NIT Calicut. This is an interesting story. So the Bosch EV program were hiring new graduates. And they identified that they spent a lot of time in training these new graduates who had joined. So the graduates who came in were not ready as soon as they graduated to work on these automotive workflows.
So to bridge that gap, what they did was they worked with some of these professors from NIT Calicut, also with support of MathWorks, and they came up with an EV curriculum. And this curriculum was then floated. It led to engineers being more ready to enter the industry. So there was a faster uptake. And as soon as they got in, they were ready to go. It also led to an increased interest from the students, who now saw that their university is floating an industry-ready course. So there were gains that happened on both sides for industry as well as academia.
And currently at MathWorks, we've got four of these ongoing industry-academia partnerships that we have facilitated in India. And I'd like to leave you with the last one, which is an example of a course that is coming up in this year. And this course is on software-defined vehicles. This is a topic that we've been talking about since the morning.
This is a course that starts off with the basics of SDV. So students will come to learn what is SDV all about, what are ECUs, what is the software development life cycle. Then they will go ahead and learn things like the communication protocols, V to X, V to V. They'll also look at aspects of ADAS and navigation sensing and perception.
And finally, towards the end, this is also a very practical module where the students will learn about standards, so the ISO 26262, where they are going to not only look at case studies as to how this is applied, but they'll also look at practical applications and implementations of it. So if you're looking at this slide and you notice that this is something you would like to talk about to us or you notice that this is a gap that you want to bridge, please feel free to reach out to us. And we're happy to facilitate you partnering with a university to meet your employment and innovation needs.
Thanks, Nikita. That gives an insight of how MathWorks is actually collaborating with industries. We come to the last part of the session, so I will quickly summarize what we have actually learned.
So we know the automotive software development is steadily incorporating AI. We heard multiple use cases from this particular session and also from our customers, which is mainly focusing on-- all these developments are focusing on the security, safety resources, and the business needs.
When coming to integrate with the legacy systems, the AI-based development can seamlessly integrate into the MBD system, where Jayanth showed the complete workflow and how the four buckets can fit into the MBD workflow.
The AI model verification is not very similar to that of the traditional MBD workflow. There are changes, which we actually saw from-- or which we heard from Nikita that explainability, rigor, trust, and the robustness are some of the things which we need to take care.
When coming to the deployment challenges and strategies, there are multiple possibilities of deployment, and it is the engineer who has to see where they are deploying their models. And based upon this, the different deployment techniques need to be taken care.
Coming to the last, the collaboration between the academia and industry is evolving to address the existing challenges and bridge the gaps. And you can utilize multiple resource which are available with MathWorks to learn and to collaborate.
With this, we come to the end of this particular session. Thanks for listening. And why not, you can start building your automotive software with AI models. And we are here to support. Thank you.
[APPLAUSE]
So we have two questions from audience. Is there an example of federated learning model deployed which was developed using MATLAB? Yes?
This is different.
No, it's a different one. Is there an example of federated learning model deployed which was developed using MATLAB?
Yeah, we do have an example. And we do support federated learning in MathWorks toolchain. And ideally, we are also very much part of understanding the confidentiality of the data and how to bring the data together for training and all those aspects of federated learning. We do support federated learning workflows. And we have a shipping example with federated learning workflow in MATLAB, by which you can able to quickly understand how we are supporting and how we can able to take it forward.
So we are working with several OEMs and tier ones on this topic, but due to the confidentiality, we cannot share their names in public. But we are actively working with the teams who are interested in federated learning.
Thanks. Thanks, Jayanth. So we also have one more question. How AI-based prediction is different from response surface?
Yeah, so response surface will be based on some mathematical modeling. So suppose you are having maybe xy is equal to f of x. You try to estimate the f. f is nothing but maybe the y is equal to mx plus c. So we try to estimate the values of maybe the different coefficients of that line. So this is very maybe primary stage of the estimation of any function.
And this is for the curve fitting. Normally, we call it as a surface-fitting, normal curve-fitting toolbox, which normally provide the functions to fit the curve as well as to fit the surface. If it is a maybe xy is equal to f of x, it is curved. If y is equal to f of x1x2, it is a surface. So we try the different polynomial techniques with different orders to estimate the right curve, which can give the very less error.
Machine learning is a little bit advanced to that. Suppose we are not finding the right-- suppose data is very multi-dimensional. Suppose I am having maybe f of y is equal to f of x1, x2, x3, x4, x5. In that way, we cannot really fit one single surface. In that case, the machine learning comes into picture. We will try to estimate the different coefficients, sometimes in a white box way, sometimes in a gray box way. And the overall goal is to estimate the f, again, by using some linear regression, decision tree techniques, SVM techniques. Each method have their own algorithms, functions, as well as coefficients.
So those gather the different requirements. One is for maybe suppose you are having maybe one or two inputs. Machine learning, suppose you are having some higher dimensional data set.
Great. So I hope that answers the question that are both different problem statements. Based upon the requirement, we need to see which method need to be employed in that.
So just to add, we support both of these workflow in no code or app-based workflows, both, response fitting as well as the machine learning workflows. We have a couple of apps like Curve Fitter as well as the Regression Learner app, which you can able to try out both the apps to test it.
Thanks, Jayanth, for giving some clarity. So as a token of appreciation, we like to call CEO Patel and Ravi Kanth, Ravi, to the stage for having these wonderful questions.
[APPLAUSE]
I would call upon Avik to hand over the token.
Yes. Thanks, Avik.
Thanks.
So we would also like to announce the winner for the social world. That is Sudarshan Kanse from Sphinx Worldbiz Limited. So we request you to come to the stage and receive the 1K Amazon voucher and MathWorks goodies. Congratulations.
[APPLAUSE]
This brings us to the end of our session. Thank you all our speakers today from Tata Motors and MBRDI for sharing the insightful presentation on AI. Our Tech Talk speakers, Jayanth, Koustubh, and Nikita, thanks you as well for the wonderful session. If you have any questions or problem statements that you would like to discuss further on us, we are available after the session, and you can share the details via the feedback form as well. Once again, thank you, everyone, for making this conference a success. We look forward to hosting you all again. Thank you.