This example shows you how to replace a particular color in a live video stream with an image on Raspberry Pi™ hardware by using a MATLAB® Function block.
The Simulink® model in this example compares each pixel in the acquired image with the specified color. If the pixel of the acquired image is close enough to the specified color, the pixel is replaced with the corresponding pixel of the image. This process is known as chroma keying.
In this example, you will learn how to:
Acquire live video stream from a webcam or a camera module connected to the Raspberry Pi hardware.
Implement the chroma keying algorithm on the acquired video using a MATLAB Function block.
Display the processed video on the Raspberry Pi hardware.
Virtual network computing (VNC) viewer (optional)
Image Processing Toolbox™
Tip: Before you start this example, we recommend you complete the Getting Started with Simulink Support Package for Raspberry Pi Hardware example.
1. Connect the micro-USB end of the USB cable to the Raspberry Pi and the regular USB end of the USB cable to the computer. Wait until the PWR LED on the hardware starts blinking. 2. To connect a video device to the hardware, do any one of the following:
If you have a webcam, connect the webcam to one of the USB ports on the hardware. Note that some webcams draw too much power and may require a powered USB hub for proper operation.
If you have a Raspberry Pi camera module, connect the camera module to the hardware by using a CSI cable. For information on how to connect the camera module to the hardware, follow the instructions in the manual provided by the manufacturer.
3. Using the Hardware Setup screen, configure the Raspberry Pi network.
Open the Chroma Key model.
The model consists of these blocks:
Red, Green, and Blue References: The Constant value in these blocks represent the red (R), green (G), and blue (B) components of the color to be replaced. For example, if the color to be replaced is green, specify the values as
0. You can specify any other color by changing the values in the Constant value parameter. To find the RGB values of a color, perform these steps:
1. In the MATLAB Command Window, create a connection to the Raspberry Pi hardware using
mypi = raspi;
2. Create a connection to the
cameraboard and assign the connection to a handle,
mycam = cameraboard(mypi);
3. Plot and display the image captured in the MATLAB Figure Window.
img = snapshot(mycam); imshow(img);
4. Place the cursor on the color to be replaced. The MATLAB Data Cursor tool displays the RGB values of the colour. Copy these values and paste them in the Constant value parameter of the corresponding block parameters dialog box.
V4L2 Video Capture: This block captures live video from the video device connected to the hardware. The block outputs RGB values of the acquired pixel as an 8-bit value.
Double-click the block to specify the name of the video device in the Device name parameter. To find the name of the device:
1. Open the hardware terminal and use the
ls command. This command lists all the video devices connected to the hardware.
ls -al /dev/video*
2. Copy the name of your video acquiring device and paste the name in the Device name parameter.
MATLAB Function: This block implements the chroma keying algorithm on the acquired video. Chroma keying converts the RGB values of the acquired color to its equivalent Y, Cb, and Cr components. The Y component represents the luminance component and determines how light or dark the video is. Cb and Cr components represent the chroma components to measure similarity to a source color. Measuring color similarity using only the Cb and Cr components makes chroma keying robust to variations in luminance values in light and dark areas of the solid background color. If the measured similarity is within the threshold limit, each pixel in the acquired image is replaced with the corresponding pixel of the image. For information on how chroma keying is implemented in MATLAB, double-click the block.
SDL Video Display: This block displays the processed video on the hardware.
1. To open the Model Configuration Parameters dialog box, click the gear icon on the Simulink model toolbar.
2. In the Configuration Parameters dialog box, select Hardware Implementation.
3. Set the Hardware board parameter to
Raspberry Pi. This selection automatically populates the parameters in the Hardware board settings with the default values for Raspberry Pi.
4. In the Hardware board settings pane, expand Target hardware resources and select Board Parameters. Specify these parameter values:
Device Address: The IP address or host name of the hardware.
Username: Specify the root user name of the Linux system running on the hardware. The default user name of the Raspbian Linux distribution is
Password: Specify the root password of the Linux system running on the hardware. The default password of the Raspbian Linux distribution is
5. Click Apply. Click OK to save your changes.
On the Hardware tab of the Simulink model, in the Mode section, select Run on board and then click Monitor & Tune. The lower-left corner of the model window displays status while the support package generates the code. After successfully generating the code, the support package loads and runs the code on the computer. The executable starts acquiring live video stream from the video device, replaces the green color in the acquired video with the sample image, and then displays the result on the computer.
On the Hardware tab of the Simulink model, in the Mode section, select Run on board and then click Build, Deploy & Start. The lower-left corner of the model window displays status while the support package generates the code. After successfully generating the code, the support package loads and runs the code on the hardware. The executable starts acquiring live images from the video device, replaces the green color in the acquired video with the sample image, and then displays the result on the hardware screen. To view the Raspberry Pi screen, use a VNC viewer and perform a remote session on the hardware to get the display. You can alternatively connect an HDMI cable between a monitor and the hardware.
Modify the color to replace and deploy the model on the Raspberry Pi hardware again.