Automate Real-Time Testing of Highway Lane Following Controller Using ASAM XIL

This example uses:

This example shows how to deploy a highway lane following controller to a Simulink® Real-Time™ target using ASAM® XIL. It also shows how to automate testing of the deployed controller model using Simulink Test™.

Introduction

The lane following controller is a fundamental component in highway driving applications, as it combines lateral and longitudinal controls for an ego vehicle. This critical component is most commonly deployed to a real-time processor. This example shows how you can test a lane following controller on a Simulink Real-Time target using the ASAM XIL framework. ASAM XIL is an API standard for communication between test automation tools and target hardware (test benches). Implement the ASAM XIL APIs using the Simulink Test Support Package for ASAM XIL Standard. The Simulink Real-Time Support Package for ASAM XIL Standard enables you to use these APIs with Simulink Real-Time targets. The ASAM XIL APIs enable you to run test cases, created in Simulink Test, to run and validate an algorithm on real-time hardware.

This example uses ASAM XIL APIs to deploy and test a lane following controller on a target. You can use the process shown in this example to deploy the lane following controller to any other ASAM XIL compliant hardware. The example also shows how you can reuse desktop simulation test cases and automate regression testing for the deployed controller. This example builds on the Automate Real-Time Testing for Highway Lane Following Controller example.

System Configuration

This example uses a hardware setup that primarily consists of two machines, a host and a target, connected by Ethernet.

The hardware configuration for the example consists of these parts:

Target — Speedgoat® Performance Real-Time machine with an Intel® Core™ i7 @4.2 GHz CPU, running the Simulink Real-Time operating system. For more information, see Speedgoat Performance Real-Time Target Machine.
Host — Intel Xeon® @3.60 GHz CPU, running a Windows® 64-bit operating system.
Ethernet cable connecting the target to the host.

The host sets up the simulation environment, configures the test scenarios, and models the vehicle dynamics of the ego vehicle. It sends signals carrying information about lanes, tracks, ego velocity, and set velocity to the target using the user datagram protocol (UDP). It receives the steering angle and acceleration to generate the ego pose from the target using UDP.

The host uses ASAM XIL APIs to connect and deploy the target model to the target machine, control the model running on the target, log the signals on the target, and get the signals to the host for analysis of the performance of the target model.

The target runs the lane following decision logic and lane following controller, and it sends steering angle and acceleration signals to the host using UDP over the Ethernet connection.

Using this setup, you can deploy the lane following controller to the target, run the host model for a test scenario, and log and visualize simulation results.

In this example, you:

Review Models and Build Executable — Review the host and target models, and build an executable for the target model.
Configure XIL Framework — Configure and initialize an XIL framework object to deploy the target executable to the Simulink Real-Time target.
Set Up Data Acquisition and Start Framework — Configure the XIL framework object to acquire data from specified signals. Start the framework to simulate the target executable. Then, begin acquisition to log the data.
Simulate Host Model — Configure the host model with a test scenario and simulate the model.
Stop Data Acquisition and Framework — Fetch the logged signal data from the target and stop the acquisition and framework.
Analyze Results — Plot and analyze performance metrics for the longitudinal and lateral controllers.
Explore Test Manager File — Configure the Test Manager to automate the testing of the highway lane following controller using multiple test scenarios through the XIL framework. Evaluate the results, and export them for external review.

Review Models and Build Executable

This example reuses the host and target models from the Automate Real-Time Testing for Highway Lane Following Controller example.

Open the host model.

open_system("RTHLFControllerHost")

The host model contains these subsystems:

Simulation 3D Scenario — Contains the road, vehicles, and sensors used for 3D simulation.
UDP Data Send — Sends scenario data to the target model. The lane following controller, running on real-time hardware, receives this data, processes it, and sends back its output to the host model using the UDP interface.
UDP Data Receive — Receives lane following controller output data from the target using the UDP interface.
Vehicle Dynamics — Contains a 3DOF dynamic model for the ego vehicle.
Metrics Assessment — Assesses system-level behavior.

Open the target model.

open_system("RTHLFControllerTarget")

The target model contains these subsystems:

UDP Data Receive — Receives data, using the UDP interface, required for the lane following controller model to run.
Highway Lane Following Controller — Enabled subsystem that contains the lane following decision logic and lane following controller algorithms.
UDP Data Send — Sends the outputs of the controller to the host model, which the host model requires for the Vehicle Dynamics subsystem.

To deploy the RTHLFControllerTarget model to the Simulink Real-Time target using the ASAM XIL framework, generate the executable for the target model.

Before generating the executable, update SystemTargetFile configuration parameter of the target model by connecting to the target.

% Create slrealtime object
tg = slrealtime;

% Specify IP address for target machine
setipaddr(tg,"10.1.10.15")

% Connect to the target machine
connect(tg);

% Read the system target filename of the target machine
modelSTF = getSTFName(tg);

% Set SystemTargetFile parameter in the target model
set_param("RTHLFControllerTarget",SystemTargetFile=modelSTF);
save_system("RTHLFControllerTarget");

% Disconnect the target machine
disconnect(tg);

Build the executable for the target model.

slbuild("RTHLFControllerTarget")

The slbuild command generates an executable file, RTHLFControllerTarget.mldatx.

Configure XIL Framework

Follow these steps to configure the XIL framework using the Simulink Real-Time target.

1. Create Port Configuration File

Generate an XML file that configures XIL ports for the target by using the createPortConfigureFile (Simulink Real-Time) function.

slrealtime.createPortConfigureFile('HLFController.xml','10.1.10.15','RTHLFControllerTarget')

This image shows the generated XML file, HLFController.xml, which specifies the IP address and filename of the target model.

2. Create Framework Object

Add the sltest.xil.framework.Framework (Simulink Test) class to the import list, and then use it to create a Framework object. You must use only one Framework object at a time.

import sltest.xil.framework.*;
framework = Framework;

3. Configure Framework Object

To communicate with the target model, RTHLFControllerTarget, configure the Framework object with a Model Access Port (MAPort), which enables you to access the RTHLFControllerTarget model on the target hardware.

framework.Configuration.addModelAccessPort(...
   'MAPort', ...
   'asamxil.v2_1', ...
   VendorName='MathWorks', ...
   ProductName='XIL API', ...
   ProductVersion=1.0', ...
   PortConfigFile=fullfile(pwd,'HLFController.xml'))

The framework configuration options depend on the target hardware. For more information about port configurations, see Create and Run Tests Using the ASAM XIL Standard (Simulink Test).

4. Map Test Variables

This example uses these signals to test the performance of the lane following controller:

enable_signal — Use this signal to synchronize the logged data on the host and target models.
relative_distance — Specifies the relative longitudinal distance between the ego vehicle and the most important object (MIO). Units are in meters.
relative_velocity — Specifies the relative longitudinal velocity between the ego vehicle and the MIO. Units are in meters per second.

To enable logging of these signals using the XIL framework, specify their paths relative to the target model.

pathToEnableSignalInModel     = 'RTHLFControllerTarget/UDP Data Receive:1';
pathToRelativeDistanceInModel = 'RTHLFControllerTarget/Highway Lane Following Controller/Resettable Highway Lane Following Controller/Lane Following Decision Logic/Find Lead Car:1';
pathToRelativeVelocityInModel = 'RTHLFControllerTarget/Highway Lane Following Controller/Resettable Highway Lane Following Controller/Lane Following Decision Logic/Find Lead Car:2';

% Test variable names
testVariables = {'relative_distance', 'relative_velocity', 'enable_signal'};

% Map the model relative paths to the test variables in the framework
framework.Configuration.addTestVariableMapping(testVariables{3}, 'MAPort', pathToEnableSignalInModel)
framework.Configuration.addTestVariableMapping(testVariables{1}, 'MAPort', pathToRelativeDistanceInModel)
framework.Configuration.addTestVariableMapping(testVariables{2}, 'MAPort', pathToRelativeVelocityInModel)

5. Initialize Framework

To establish the connection to the target machine, use the init method of the XIL framework. Using the IP address specified in the port configuration file, HLFController.xml, the method establishes a connection between the host and target, and loads the executable file, RTHLFControllerTarget.mldatx, into the target. If the target is already running an executable, the method connects to the target and does not load an executable file to it. Initialize the framework by using the init method.

framework.init

Set Up Data Acquisition and Start Framework

To acquire data from the mapped test variables, you must instantiate them by using the createVariable method. Set up data acquisition to log test variables.

for i=1:length(testVariables)
    variable = framework.createVariable(testVariables{i});
    framework.Acquisition.setupWithVariables(variable)
end

To run the lane following controller executable on the target machine, use the start method of the XIL framework.

framework.start

You can start logging data for test variables at any time during simulation. The time instance when you trigger data acquisition is considered as t = 0. Use the Acquisition.start method to start logging data.

framework.Acquisition.start

Simulate Host Model

Once you start running the lane following controller application on the target hardware, start simulating a test scenario using the host model, RTHLFControllerHost. The host model provides input data to the lane following controller application running on the target hardware. The host model also receives the output data from the target by using the UDP interface.

Configure the host model to simulate the scenario_LFACC_03_Curve_StopnGo scenario. This scenario contains six vehicles, including the ego vehicle, and defines their trajectories.

helperSLHLFControllerHostSetup(scenarioFcnName="scenario_LFACC_03_Curve_StopnGo")

Simulate the host model.

sim("RTHLFControllerHost")

To enable you to analyze the performance of the highway lane following controller, the host model logs some signals to the base MATLAB® workspace variable logsout during simulation.

Stop Data Acquisition and Framework

Once the host model completes the simulation, log the test variable data to a workspace variable, logsoutXIL, by using the Acquisition.fetch method.

logsoutXIL = framework.Acquisition.fetch;
relative_distance = logsoutXIL.getElement(testVariables{1});
relative_velocity = logsoutXIL.getElement(testVariables{2});
enable_signal = logsoutXIL.getElement(testVariables{3});

Stop data acquisition, and stop running the executable on the target.

framework.Acquisition.stop
framework.stop

Analyze Results

This example provides these helper functions to plot performance metrics for the longitudinal and lateral controllers.

helperPlotSpacingResultsFromXIL — Plots performance metrics for the longitudinal controller.
helperPlotLFLateralResults — Plots performance metrics for the lateral controller.

Plot the performance metrics for the longitudinal controller.

hFigSpacingResults = helperPlotSpacingResultsFromXIL(logsout,time_gap,default_spacing,relative_distance,relative_velocity,enable_signal);

Examine the results.

The Relative longitudinal distance plot shows the distance between the ego vehicle and the MIO. In this case, the ego vehicle approaches the MIO and even exceeds the safe distance, in some cases. The host input this signal to the controller.
The Relative longitudinal velocity plot shows the relative velocity between the ego vehicle and the MIO. Because this example contains no tracker, and tracks data bus filled using ground truth, the estimated velocity is almost identical to the ground truth. The host input this signal to the controller.
The Absolute acceleration plot shows that the controller commands the vehicle to decelerate when it approaches the MIO. The controller output this signal.
The Absolute velocity plot shows that the ego vehicle initially follows the set velocity, but avoids a collision by slowing down when the MIO slows down. The host input this signal to the controller.

Close the figure.

close(hFigSpacingResults)

Plot the performance metrics for the lateral controller.

hFigLatResults = helperPlotLFLateralResults(logsout);

Examine the results.

The Detected lane boundary lateral offsets plot shows the lateral offsets of the detected left-lane and right-lane boundaries from the center of the lane. The host input this signal to the controller.
The Lateral deviation plot shows the lateral deviation of the ego vehicle from the centerline of the lane. Ideally, lateral deviation is zero meters, which indicates that the ego vehicle follows the centerline exactly. Small deviations occur when the vehicle changes velocity to avoid collision with another vehicle. The host input this signal to the controller.
The Relative yaw angle plot shows the relative yaw angle between the ego vehicle and the centerline of the lane. The relative yaw angle is close to zero radians, which indicates that the heading angle of the ego vehicle closely matches the yaw angle of the centerline. The host input this signal to the controller.
The Steering angle plot shows the steering angle of the ego vehicle. Note that the plot shows little deviation, indicating that the steering angle trajectory is smooth. The controller output this signal.

Close the figure.

close(hFigLatResults)

Explore Test Manager File

Simulink Test includes the Test Manager, which you can use to author test cases for Simulink models. After authoring your test cases, you can group them and execute them individually or in a batch.

This example reuses the test scenarios from the Automate Real-Time Testing for Highway Lane Following Controller example.

The example uses these test file callbacks:

The SETUP callback calls the helperSetupXILFrameworkObject function, which creates instance of an XIL Framework object, associates it with a Simulink Real-Time target, and maps test variables to log signals of interest on the target model.
The CLEANUP callback clears the XIL Framework object instance from the workspace.

Open the test file HLFControllerXILTestsAssessment.mldatx in the Test Manager and explore the configuration.

sltestmgr
sltest.testmanager.load("HLFControllerXILTestsAssessment.mldatx");

The file contains these test case callbacks:

The POST-LOAD callback calls the helperStartXILFramework function, which creates the XIL Framework object, instantiates mapped test variables, starts data acquisition, and starts the framework. It also configures the host model for a test scenario.
The CLEANUP callback calls the helperStopXILFramework function, which stops data acquisition and the framework. It plots the results of the longitudinal and lateral controller metrics, for analysis of the performance of the model on the target.

Automate Testing

Using the created Test Manager file, run a single test case to test the application on a real-time machine.

Use this code to test the lane following controller model with the scenario_LFACC_03_Curve_StopnGo test scenario from Simulink Test.

testFile = sltest.testmanager.load("HLFControllerXILTestsAssessment.mldatx");
testSuite = getTestSuiteByName(testFile,"Test Scenarios");
testCase = getTestCaseByName(testSuite,"scenario_LFACC_03_Curve_StopnGo");
resultObj = run(testCase);

To generate a report after the simulation, use this code:

sltest.testmanager.report(resultObj,"Report.pdf", ...
Title="Real-Time Highway Lane Following Controller XIL Test Assessment", ...
IncludeMATLABFigures=true, ...
IncludeErrorMessages=true, ...
IncludeTestResults=false, ...
LaunchReport=true);

Examine Report.pdf. Note that the Test Environment section shows the platform on which you have run the test, and the MATLAB version used for testing. The Summary section shows the outcome of the test and the duration of the simulation in seconds. The Results section shows pass or fail results based on the assessment criteria. This section also shows the logged plots from the CLEANUP callback commands.

Run and Explore Results for All Test Scenarios

You can simulate the system for all the tests by using the run(testFile) command. Alternatively, you can simulate the system by clicking Play in the Test Manager.

When the test simulations are complete, you can view the results for all the tests in the Results and Artifacts tab of the Test Manager. For each test case, the Check Static Range (Simulink) blocks in the model enable the Test Manager to visualize overall pass or fail results.

You can find the generated report in the current working directory. This report contains a detailed summary of the pass or fail statuses and plots for each test case.