Visualize Sensor Data from Android Device in RViz

This example shows how to generate an Android™ application to visualize live sensor data from an Android device on the ROS visualization (RViz) tool using Simulink® Support Package for Android Devices. In this example, sensor data such as orientation, angular velocity, and acceleration of the device are visualized. Your computer is configured as ROS master, and the Android device is configured as ROS client. The device connects to the computer through the Robot Operating System (ROS) interface. The sensor data from the device is visualized using the RViz tool running on the computer.

Requirements

Hardware Requirements

  • Android device

  • USB cable

Software Requirements

Install these products in your computer:

Pre-Requisites

Introduction

Robot Operating System (ROS) is a communication interface that enables different parts of a robot system to discover, send, and receive data. Using the ToApp block from the Simulink Support Package for Android Devices, you can exchange data with ROS-enabled physical robots or robot simulators such as Gazebo. For more information, see ROS and the Concepts section on the ROS website.

Step 1: Run the Gazebo Simulator on the Computer

1. Install the MathWorks-provided VM on your computer. If you have already installed this VM or if you choose to use your own VM, skip this step.

2. Open a new terminal on the VM installed on your computer.

3. In the terminal window, type ifconfig. This command displays the configuration of the network interfaces of the VM. Under eth0, the inet address displays the IP address of the VM. Note the IP address for later use.

4. Set the values of the standard ROS environment variables, ROS_MASTER_URI and ROS_HOSTNAME, by executing these commands in the terminal. Replace IP_OF_VM with the IP address of the VM noted in the previous step.

echo export ROS_MASTER_URI=http://IP_OF_VM:11311 >> ~/.bashrc
echo export ROS_HOSTNAME=IP_OF_VM >> ~/.bashrc

5. To enable communication between ROS nodes, execute this command.

roscore

Step 2: Generate an Android Project Using Simulink

This section explains how to generate an Android project from a Simulink model.

Open the Simulink Model

1. Connect the Android device to your computer with a USB cable.

2. Install the Simulink Support Package for Android Devices. If you have already installed the support package, skip this step.

3. In the Simulink editor, open the IMUControl model. This model is preconfigured to run on the Android device. The model has these blocks:

  • Orientation: The block reads the rotation of the device using three angular quantities: pitch (around x- axis), roll (around y- axis), and azimuth (around z- axis).

  • Accelerometer: The block reads acceleration along the x-, y-, and z- directions of the device.

  • Gyroscope: The block reads the rate of rotation along the x-, y-, and z- directions of the device.

  • MATLAB Function (Simulink): The MATLAB function block converts the output of the Orientation block to quaternion angles.

  • ToApp: The block accepts the sensor data from the device and sends the data to the ROS Publish method. The ROS Publish method publishes the data to RViz.

  • Scope: The block displays signals generated during simulation.

Step 3: Import the Project and Add ROS Dependencies in Android Studio

1. Import the generated project to Android Studio by following the steps as described in Import Project into Google Android Studio.

2. Select the Project option from the top-left drop-down menu above the File Explorer.

3. Open the build.gradle file of your project and update the file by changing the code at the positions labeled in the table.

Position New Code
1 Add this line in the buildscript section.
apply from: "https://github.com/rosjava/android_core/raw/kinetic/buildscript.gradle"
2 Add these lines in the allprojects section.
apply plugin: 'ros-android'
afterEvaluate { project ->
       android {
           // Exclude a few files that are duplicated across our dependencies and
           // prevent packaging Android applications.
           packagingOptions {
               exclude "META-INF/LICENSE.txt"
               exclude "META-INF/NOTICE.txt"
           }
       }
   }
3 Add these ROS dependencies below the listed dependencies.
compile 'org.ros.android_core:android_10:[0.3,0.4)'
compile "org.ros.rosjava_messages:diagnostic_msgs:[1.12,1.13)"
compile "org.ros.rosjava_messages:sensor_msgs:[1.12,1.13)"
compile "org.ros.rosjava_core:rosjava:[0.3.2,0.4)"
compile 'org.ros.rosjava_core:rosjava_geometry:[0.3,0.4)'
compile 'org.ros.rosjava_messages:visualization_msgs:[1.12,1.13)'

4. Open the AndroidManifest.xml file located in the src > main folder of your project. Update the file by changing the code at the positions labeled in the table.

Position New Code
1 Add this code after the application tag.
xmlns:tools="http://schemas.android.com/tools" tools:replace="android:icon"
2 Replace the action and category sections.
<action android:name="com.android.rosupport.START_MODEL"/>
<category android:name="android.intent.category.DEFAULT" />
3 Add the activity and service sections immediately after the activity section.
<activity android:label="@string/app_name" android:name=".MainActivity"
   android:theme="@style/MWAppTheme" android:configChanges="orientation|screenSize">
   <intent-filter>
       <action android:name="android.intent.action.MAIN"/>
       <category android:name="android.intent.category.LAUNCHER"/>
   </intent-filter>
</activity>
<service android:name=".NodeMainExecutorService"
   tools:ignore="ExportedService">
   <intent-filter>
       <action android:name="org.ros.android.NodeMainExecutorService"/>
   </intent-filter>
</service>
4 Add these ROS uses-permissions below the listed uses-permission.
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>
<uses-permission android:name="android.permission.ACCESS_WIFI_STATE"/>
<uses-permission android:name="android.permission.CHANGE_WIFI_STATE"/>
<uses-permission android:name="android.permission.WAKE_LOCK"/>
<uses-permission android:name="android.permission.SYSTEM_ALERT_WINDOW"/>

5. Change the MATLAB Current Folder by using this command in the MATLAB Command Window:

cd(fullfile(matlabshared.supportpkg.getSupportPackageRoot,'toolbox','target','supportpackages','android','androidexamples','ROSonAndroid','IMUControl'))

a. Copy these files and paste them in the src > main > java > com.example.IMUControl folder of your project. Here, IMUControl is the name of your project.

  • ROSMainActivity.java

  • PublishNode.java

  • MainActivity.java

  • MWNodeMainExecutorListener.java

  • MWNodeMainExecutorService.java

  • MWNodeMainExecutorServiceListener.java

b. Save the main.xml file in the src > main > res > layout folder of your project.

c. Save the mwstyle.xml file in the src > main > res > values folder of your project.

6. Open the IMUControl.java file located in the src > main > java > com.example.IMUControl folder of your project. Here, IMUControl is the name of your project. Update the file by changing the code at the positions labeled in the table.

Position New Code
1 Add the ROS import statements.
import org.ros.node.NodeConfiguration;
import org.ros.node.NodeMainExecutor;
2 Replace AppCompactActivity with ROSMainActivity.
3 Add the PublishNode statement before the definition of the CountDownTimer method.
PublishNode pn;
4 Add the ROSPublish and init method definitions before the definition of the naMain method.
public void ROSPublish(float[] data)
{
   if(pn != null)
       pn.publishData(data);
}
protected  void init(NodeMainExecutor nodeMainExecutor) {
   pn = new PublishNode("/android/imu","sensor_msgs/Imu");
   NodeConfiguration nodeConfiguration = NodeConfiguration.newPublic(getRosHostname());
   nodeConfiguration.setMasterUri(getMasterUri());
   nodeMainExecutor.execute(pn, nodeConfiguration);
}
The ROSPublish method accepts the data from the Orientation, Accelerometer, and Gyroscope blocks of the Simulink model. The method then publishes the data to RViz (ROS master). The data type of the published data depends on the message type of the ROS topic. In this example, the publish topic is /android/imu and the message type is sensor_msg/Imu. You can use a different topic name and message type based on the requirement of your application. The init method is called after the ROS client is created. In this function, you can specify the number of publishers and subscribers required for the application. In this example, only one publisher is required.

Step 4: Sync Project Files in Android Studio

In Android Studio, click the Sync Now button in the top-right corner of the Android Studio editor.

When syncing your project, you may see the following error message. To resolve this issue, click the link that appears in the error message.

Note: The build tools version number specified in the error message may vary depending on the version of the Android gradle plugin installed in the computer.

Step 5: Configure and Run the Android Device as ROS Client

Note: Before proceeding, ensure that your computer and the Android device are connected to the same Wi-Fi network.

1. In the Android Studio toolbar, select Run > Run 'IMUControl'. Android Studio installs the application on your device. The application starts running on the device and provides you the options to configure the device as ROS master or ROS client.

2. Select the ROS Client option and specify the IP address and the port number. In this example, the IP address is 192.255.88.128 and the port number is 11311.

3. To connect the device to the computer, click CONNECT. The device starts sending sensor data to computer. The application displays signals that represent the sensor data from the device over a certain amount of time. The y-axis of the application represents sensor data, whereas the x-axis represents time.

Step 6: Visualize Sensor Data in RViz

1. Open a terminal on the VM installed on your computer and execute this command. The RViz tool is launched.

rviz

2. To add a display, click the Add button under the Displays sidebar.

3. In the Create Visualization pop-up window, scroll to Imu under the rviz_imu_plugin category, and then click OK. An Imu display is added to the Display sidebar.

4. In the Displays sidebar, click the Imu display. A list of properties associated with Imu is listed.

5. From the properties list, set Topic to /android/Imu. In the Visualization window, you will now see an arrow representing the position and orientation of the device. Move the device and observe as the arrow changes its position.

This illustration shows the arrow in the Visualization window when the device is facing in the upward direction.

See Also

Set Position of Bebop Drone on Gazebo Simulator from Android Device Using ROS