What’s New in MATLAB, Simulink, and RoadRunner for Automated Driving Development - MATLAB
Video Player is loading.
Current Time 0:00
Duration 18:27
Loaded: 0.90%
Stream Type LIVE
Remaining Time 18:27
 
1x
  • Chapters
  • descriptions off, selected
  • en (Main), selected
    Video length is 18:27

    What’s New in MATLAB, Simulink, and RoadRunner for Automated Driving Development

    MATLAB®, Simulink®, and RoadRunner help engineers to build automated driving systems with increasing levels of automation. Discover new features and examples in R2022b and R2023a that will allow you to:

    • Interactively design scenes and scenarios for driving simulation
    • Generate 3D scenes from HD map data
    • Generate driving scenarios from recorded sensor data
    • Simulate driving applications like emergency braking, lane following, lane change, platooning, and valet parking

    Published: 7 May 2023

    [MUSIC PLAYING]

    Hello, and welcome to joining the session. This session will introduce you to recent updates in Matlab, Simulink and Roadrunner that you can utilize for developing automated driving applications. I'm Linghui Zhang, leading the application engineering for automated driving at MathWorks.

    And I am Div Tiwari, product manager for automated driving. I encourage you to use the hashtag #MATLAB EXPO to share your experience with your online networks.

    Let us begin with a broad summary of common workflows for developing automated driving applications. Our customer often get started by getting values from existing date they have recorded from sensors like cameras, GPS, and LiDAR. This includes labeling that data or drawing inferences from that.

    The industry continues to utilize simulation to reduce vehicle testing and to explore scenarios that may be difficult or unsafe to reproduce in the real world. MathWorks provide tools that allow you to design virtual worlds that includes scenes, scenarios, and sensors. This virtual world aid in designing algorithms and systems for detection, localization, tracking, planning, and controls. And we provided two boxes that can help you to get started in each of these disciplines.

    Developing quality software includes design, test, deployment, as well as integration with common architectures like AUTOSAR, ROS, and DDS. This workflows are enforced throughout by verification and validation.

    It's also important to be able to integrate with external tools that you may be using. Within this view, four trends standout in industry that are closely related to design virtual worlds.

    First, representative scenes and scenarios are required to maximize simulation value. Second, companies want to increase automation of building scenes and scenarios from their recorded data. Third, engineers need interactive tools to design new and refine automatically viewed scenes and scenarios. Finally, companies use multiple driving simulation tools, and industry invests in standards for true interoperability. Now, my colleague Div will show how these trends are reflected in MathWork's recent investments.

    Thank you, Linghui. We will dive into four areas that embody these trends. Firstly, designing 3D scenes with RoadRunner. Second, designing scenarios to populate the scenes with dynamic actors. This uses RoadRunner scenario. Third, to design, simulate, and test driving applications with Automated Driving Toolbox. And fourth, to build virtual scenarios from recorded real world sensor data using the scenario builder support package. Let's start with scene design.

    RoadRunner is an interactive and intuitive tool for designing scenes. You can create scenes from scratch using the graphical editor and the Roadrunner asset library, which contains several 3D models and assets; import data from a variety of formats, including OpenDRIVE, Zenrin SD maps and more.

    Now, some of you may be working with HD map formats, like Here or TomTom. You can import these into RoadRunner using the Roadrunner scene-builder product. You can then export to file formats like OpenDRIVE and Filmbox, or directly export to a variety of third-party simulators.

    Here's a look at the Roadrunner interface in action. It has been used very frequently to author scenes, including roads and roadside objects, such as trees, traffic signs, and buildings. These static scenes are the foundation for scenario simulation. I'll highlight a few of the latest features of RoadRunner for scene design.

    R2023a includes enhanced ASAM OpenCRG support, such as, for specifying global mode for CRG data, creating synthetic CRG assets for selected road, and adding rumble strips. The traffic island tool lets you create freeform traffic islands in RoadRunner scenes and customize them extensively. Tools like these show the detail and precision with which you can create and customize scenes in RoadRunner.

    On the theme of interoperability, RoadRunner now supports import and export of ASAM OpenDRIVE 1.7 files. You can also import from Baidu Apollo 3.0 and 5.0 XML formats, and use the data to build scenes using the scene-builder tool. Working on large scenes single-handedly can become cumbersome and time-consuming. Multiple users can share work on scenes, so that you can merge them into a single scene later.

    RoadRunner API lets you import, save, and export scenes from MATLAB command line, or from other languages, such as Python or C++. R2023a adds the ability to import HD map data programmatically.

    You can launch RoadRunner in console mode. This enables distributed workflows. For example, running Roadrunner from a remote server, and empowers you to use RoadRunner at a much larger scale.

    Roadrunner HD map is a road data model for representing HD map data introduced in R2022b. Here's an example of what you can accomplish using RoadRunner HD map.

    Let's say you want to build a 3D scene of Pikes Peak, Colorado. We start by reading a KML file containing its latitude-longitude coordinates, and a GeoTIFF containing its elevation data. The data points in the KML file were sparse, and the road contains sharp curves. So we upsample the data. Mapping Toolbox makes it easy to work with geographic data and handle the projection calculations.

    Now, we create a RoadRunner HD map object and specify its lane markings before exporting to a RRHD file that can be imported into Roadrunner. That was Roadrunner for scene design. Scenario design uses those scenes to help simulate and test automated driving systems.

    RoadRunner Scenario, released last year in our R2022a, is an interactive editor that lets you design scenarios. You can import trajectories from CSV files or from OpenSCENARIO and edit them. If you have a scenario built in RoadRunner Scenario, you can export that to either OpenSCENARIO 1.x or 2.0 formats. You can simulate actor behavior from other tools, like Simulink and CARLA, and perform automation using MATLAB, Python, or any gRPC supported language.

    With RoadRunner Scenario, users can add different kinds of vehicles and pedestrians to a scenario. Vehicles have various types of behaviors, such as follow a road or lane, change speed, change lane, or offset with respect to lane centers. Almost all related variables can be parameterized to generate scenario variations programmatically. A user can then simulate the scenario using in-editor simulation.

    Now, we've been adding many new features to RoadRunner Scenario. Here are some that will help you create a variety of representative scenarios. You can add pedestrian actors and assign them paths and behaviors, use built-in models, or import custom character meshes. You can combine multiple actors into a single actor group, such as a truck with a trailer. You can simulate reverse motion of an actor following a path or a lane.

    To get acquainted with the different kinds of scenario logic and behaviors in RoadRunner Scenario, you can make use of a variety of pre-built sample scenario files introduced in R2022b.

    The first example shows a driver cutting into the lane and breaking. In the second one, the yellow car maintains a space gap. Notice how the graphical logic editor simplifies the process of defining scenarios. Here are a few more of the sample scenarios available. Notably, the bottom right one shows the control of scenario logic through user-defined action, behavior, and parameter assets.

    As mentioned earlier, you can export a scenario to ASAM's OpenSCENARIO 1.x format. Here, we see it visualized in esmini, which is a popular open source tool for OpenSCENARIO, or export to OpenSCENARIO 2.0. MathWorks is an ASAM member and an active participant in the implementer's forum for this standard.

    We have looked at actor behavior described by the Scenario logic editor. These are built-in agents. For specific needs, a user might want more diverse behavior than a built-in agent. With co-simulation capabilities, any of the vehicles can be driven by a different type of agent. You can author those agents using MATLAB, Simulink, and CARLA to achieve your specific needs.

    You can save a RoadRunner scenario simulation log into a file. Starting with R2023a, you can replay this saved scenario without any computation from an associated co-simulation client, like MATLAB or Simulink. Let's now talk about using virtual worlds to simulate driving applications.

    Automated Driving Toolbox includes patterns of applications for various driving simulations. These application examples demonstrate workflows across products. They use Roadrunner and Roadrunner Scenario for environment creation, Vehicle Dynamics Blockset for high-fidelity dynamics, for algorithm development, using toolboxes for LiDAR, computer vision, sensor fusion and tracking, navigation, and model-predictive controls. Some of these examples also showcase co-generation and test automation capabilities.

    There are multiple application examples representing each of these categories. Each example implements features like adaptive cruise control and emergency braking, or higher level functions, such as lane change and parking. Use the examples as a basis for designing and testing automated driving applications.

    An essential part of developing such applications is simulating sensors. MathWorks has been investing and supporting a cuboid simulation environment, where vehicles and other actors are represented by simple box shapes or polygon meshes, and the Unreal Engine simulation environment, with more realistic graphics and to generate high-fidelity sensor data.

    Some of these sensors, such as LiDAR, radar, ultrasonic, vision, and lane detections work in both these environments. Some, like the monocular camera and fisheye camera, work in the unreal world. Similarly, radar IQ signals work only in the cuboid world. These sensors I've mentioned so far helped detect objects.

    There are also sensors that detect position, such as Wheel Encoder, GPS, IMU, and INS. And because these sensors detect position, they can be used in either of the two worlds. Let's now look at an application example that integrates sensor models with RoadRunner Scenario.

    This example, introduced in R2023a, shows how to an autonomous emergency braking system in Simulink, and co-simulate the system with RoadRunner Scenario. It uses Unreal Engine sensor models and a scene that contains elevated roads. The example model's high-fidelity vehicle dynamics with 14 degrees of freedom and uses camera, radar, and terrain sensors in the 3D simulation environment.

    This example, also added in R2023a, shows how you can co-simulate a platooning application with RoadRunner Scenario and Simulink. Platooning is the coordinated travel by two or more autonomous vehicles, with each vehicle maintaining a consistent distance from the preceding vehicle. Vehicles traveling in tightly spaced platoons can improve traffic flow, safety, and fuel economy. This platooning system example demonstrates vehicle-to-vehicle communication, tractor-trailer dynamics, longitudinal controller, and lateral controller components.

    Building scenarios from recorded vehicle data enables you to mimic real world driving scenarios and improve the test coverage of automated driving systems. Scenario Builder is a support package for Automated Driving Toolbox released with R2022b.

    Many of you have access to map data and recorded data from sensors like camera, LiDAR, GPS, and IMU. Scenario-builder uses these to generate vehicle trajectories for creating driving scenarios. In doing so, it makes use of perception, navigation, and sensor fusion workflows.

    In this example, we generate a RoadRunner scenario using Scenario Builder. GPS gives us the ego vehicle's trajectory And preprocessed track lists identify surrounding actors. The Roadrunner API allows us to combine this data into a Roadrunner scenario.

    There are related examples that show how you can generate such track lists from camera or LiDAR or detections, or how to generate a RoadRunner scene. Let's look at some recently added examples of that.

    Raw GPS data is typically noisy. Smoothing it can help better localize the ego vehicle. This example also incorporates lane detections and HD map data for lane level localization.

    You can fuse prerecorded LiDAR and camera object detections to create a smooth vehicle track list used to populate the generated scenario. Track lists are crucial for interpreting the position and trajectories of the vehicles in a scenario. And this example shows how to generate a road scene using lanes from labeled camera images and raw LiDAR data.

    In summary, design 3D scenes with RoadRunner from scratch or by importing data and export to a variety of formats. Design rich scenarios easily in RoadRunner Scenario. Create variations, co-simulate, and export to OpenSCENARIO. Refer to application examples in Automated Driving Toolbox to simulate automated driving applications. And utilize automation to accelerate the process of building scenarios with the Scenario Builder support package.

    If you want to learn more about what we've shared today or explore these tools for your specific use cases, we would love to talk. Please email us at automated-driving@mathworks.com. Thank you so much for your time, and enjoy MATLAB EXPO.

    [MUSIC PLAYING]