Mobile Robot Teleoperation with the Jetson Nano and ROS

In previous articles we’ve looked at building a mobile robot and programming it using the Arduino IDE and Jupyter notebooks on a Jetson Nano. This approach is beginner-friendly because it requires minimal setup and helped us to implement teleoperation and a simple go-to-goal behavior.

Once we start adding additional sensors or functionality, however, this approach becomes cumbersome: We have to define custom messages and program interfaces and implement them from scratch. This can be a tricky task, distracting from the more fun part of writing actual code for the robot.

This is where ROS comes in. The Robot Operating System gives us a powerful toolkit of libraries and messages as well as a node-based publisher/subscriber (or server/client) architecture. ROS is open-source software used by researchers, roboticists and hobbyists alike. In fact, it’s so versatile that NASA uses it on the ISS. Unfortunately, it also has a tremendously steep learning curve. ROS components are well documented and come with tutorials, but it can be unclear how all of them might fit together to build a system. That’s why, in this and following articles, I want to provide practical walkthroughs of building robotic applications from scratch using ROS and implementing them for a physical robot (rather than a simulation), using my Cerus mobile platform as an example. Keep in mind that I am still a ROS beginner — I test my code and it generally works well but there may be better ways to do things. Please feel free to suggest changes!

Teleoperation Basics

Like in our previous teleoperation example, we will use a wireless Xbox controller to steer our robot. Additionally, because ROS gives us better tools, we’ll finally add the ability to drive sideways using Cerus’ Mecanum wheels.

System Setup

Our mobile robotic system looks like this:

The latest official version of Jetpack (the OS for the Jetson Nano) is still based on Ubuntu 18.04, but the latest long-term support version of ROS (ROS Noetic) requires Ubuntu 20.04. Luckily there’s a community-made version of Jetpack that is based on Xubuntu 20.04 — you can get it here. Just flash the .img to a MicroSD card (e.g. using Balena Etcher) and insert it into the Jetson Nano. Boot and follow the setup instructions. I’ve tested this on the Jetson Nano 4GB and it works well.

We’ll also need to connect the Xbox Controller to the Jetson Nano. To do that, you’ll need to add a Bluetooth Module to the Jetson Nano. I use the Intel 8265AC, which conveniently also adds WiFi to our system.

With the Bluetooth hardware installed, we’ll have to disable a Bluetooth DRM feature to pair the Xbox Controller with your Jetson Nano. This is easily done with the following steps (Thanks to Benjamin Kriss):

Step1: On the Jetson Nano, open a terminal (CTRL + ALT + T) and install sysfsutils:

$ sudo apt-get install sysfsutils

Step 2: Edit the config using nano:

$ sudo nano /etc/sysfs.conf

Step 3: Append this to the end of the config to disable Bluetooth ertm:

$ /module/bluetooth/parameters/disable_ertm=1

Step 4: Reboot the Jetson Nano:

$ sudo reboot

Step 5: Power up your Xbox controller and press the pair button (the Xbox logo will flash rapidly). Select “Set up new device…” from the Ubuntu Bluetooth menu (top right corner of the screen) and follow the instructions. The Xbox logo should now show a solid light.

Note: Sometimes you need to select “Devices” in the Bluetooth menu, click on the Xbox controller and select “connect” for it to fully connect.

Step 6: Make sure your gamepad is successfully connected by using this online gamepad tester.

ROS Installation

With Xubuntu 20.04 on our Jetson Nano, we can now install ROS. I recommend ROS Noetic, installation instructions can be found here: noetic/Installation — ROS Wiki

I also recommend reading the Getting Started Guide and following some of the Beginner Tutorials.

We will use two packages that are not part of the ROS base installation. Joy and rosserial.

Joy enables us to use the Xbox One controller inside of ROS to control our robot. You can install it with:

$ sudo apt-get install ros-noetic-joy

A full installation tutorial is here.

Next, we’ll install rosserial, which allows us to interface the Jetson Nano with the Arduino Mega:

$ sudo apt-get install ros-noetic-rosserial-arduino
$ sudo apt-get install ros-noetic-rosserial

A complete overview of rosserial including useful tutorials is here.

Ensure that you setup and compile your catkin workspace before continuing. Here is how.

Finally, we need to set up our Xbox controller to work with ROS. You can follow this simple guide to do so.

Creating a Package

Packages are used to organize code in ROS. We’ll create our own for this teleoperation application:

$ catkin_create_pkg cerusbot_teleop_joy std_msgs roscpp joy rosserial_arduino

With the above command we create a new package called “cerusbot_teleop_joy” (you can call this whatever you like) and we include dependencies to our previously installed packages joy and rosserial_arduino (the Arduino ROS client) as well as the basic dependencies roscpp and std_msgs.

Next, we compile our workspace to build this new package:

$ cd ~/catkin_ws/
$ catkin_make

Make sure that the workspace compiles without errors before proceeding.

Nodes, Publishers and Subscribers

The real magic of ROS is that it enables us to concurrently run multiple scripts or programs (nodes) written in C++ or Python, that can publish information or subscribe to it. The type of information is organized in topics and the data is standardized as messages. To give a concrete example: For our mobile base, we want the Jetson Nano to send velocity commands to the Arduino, which will then turn them into commands for the motor drivers. To do this, we can publish on the ROS topic cmd_vel, which has the message type geometry_msgs/Twist. The Twist message is defined by ROS as follows:

# This expresses velocity in free space broken into its linear and angular parts.
 Vector3 linear
 Vector3 angular

This means we can publish the Twist message containing linear speeds in X and Y direction as well as angular velocity around the Z axis from our Jetson Nano and have the Arduino Mega subscribe to it. ROS will take care of the data interface between the two devices, letting you set certain parameters like the message frequency, but abstracting away the complicated details of interfacing different devices!

Jetson Code (Publisher Node)

With our infrastructure ready, we can write some C++ code (this could also be written in Python) for the Jetson Nano. We will subscribe to the analog stick axes values coming from our Xbox Controller (these are published on the joy topic) and publish the relevant axis data as controls for the Arduino mega as geometry_msgs/Twist messages. Make sure this code is placed inside the package folder we created, in our specific case the path is ~/catkin_ws/src/cerusbot_teleop_joy

First we include the required headers for ROS and the Twist and Joy messages. We then create a class that contains a joyCallback to read the axes positions from the Xbox controller and a node handle. We also define variables for linear, angular and strafe (moving laterally) speed as well as a publisher vel_pub and a subscriber joy_sub that we’ll use later.

Next, we initialize linear, angular and strafe to 1, 2, and 3. (These are the axis names of the Xbox controller analog sticks — you can change these to match your own preferences.)

The parameters we set with nh.param allow us to easily customize this program later using a launch file.

We also define vel_pub to advertise/publish the Twist message and joy_sub to subscribe to incoming messages from the Xbox Controller.

Lastly, we construct the Twist message and publish it.

Finally, we instantiate our class TeleopCerus and call ros::spin(); to loop infinitely.

You can find the complete code on GitHub, here. This code is based on a ROS tutorial for the Joy package, you can take a look for further reference, here.

Arduino Code (Subscriber Node)

The Arduino program is very similar to the one we wrote in Part 2 of this series. The main change is that instead of implementing a custom serial interface on the Arduino, we can take advantage of rosserial and simply subscribe to the geometry_msgs/Twist message to get speed commands from the Jetson Nano. Make sure you set up your Arduino IDE to work with ROS, following this guide. You can then simply compile and upload this code to your board using the Arduino IDE.

The code below is shortened to highlight the key ROS elements, you can find the complete code on GitHub, here.

First we include the headers for ROS and the Twist message and define a node handle and Twist message.

We then create a Twist message callback, define the forward kinematics for the Mecanum wheels (derived from this great article) and then subscribe to the Twist message.

Finally, we initialize our node and subscriber and then loop forever.

Update CMakeLists and Compile

With our code completed, we now need to update the CMakeLists file of our package and re-compile our catkin workspace. Add the following lines of code to the CMakeLists.txt in your package folder (catkin_ws/src/cerusbot_teleop_joy):

add_executable(cerus_teleop_joy src/cerus_teleop_joy.cpp)
target_link_libraries(cerus_teleop_joy ${catkin_LIBRARIES})

Then compile using:

$ cd ~/catkin_ws
$ catkin_make

Launching the System

To launch the system and drive our robot around we’ll first make a launch file and place it in catkin_ws/src/cerusbot_teleop_joy/ launch/cerus_teleop.launch. This allows us to launch our required nodes all at once:

Be sure to change js0 to whatever the input name of the Xbox controller is on your system and /dev/ttyACM0 to whatever USB port the Arduino is connected to. You can also tweak the Axes names and gains based on your preferences.

Now we can start a roscore (the ROS kernel):

$ roscore

And launch our package:

$ roslaunch cerusbot_teleop_joy cerus_teleop.launch

Now we can drive our robot using the left (forward, reverse and rotate) and right analog stick (strafe left and right) of the Xbox controller (I’ll share a video soon, after completing a few hardware upgrades on my robot)!

Conclusion

We’ve covered a lot of ground in this article. We’ve set up ROS on the Jetson Nano, connected an Xbox controller, created our own ROS package, written a publisher and subscriber node, edited the CMakeLists file and created a launch file. These are some of the most important components for making applications with ROS. I hope this practical walkthrough was helpful — thanks for joining me on my robotics journey and see you in the next article.