4. Hello Robot¶
4.1. Learning Objectives¶
This tutorial details how to add and move a mobile robot in Omniverse Isaac Sim in an extension application. After this tutorial, you will understand how to add a robot to the simulation and apply actions to its wheels using python.
10-15 Minute Tutorial
4.2. Getting Started¶
Prerequisites
Please review Hello World prior to beginning this tutorial.
Begin with the source code of the Awesome Example developed in the previous tutorial, Hello World.
4.3. Adding a Robot¶
Let’s begin by adding a NVIDIA Jetbot to the scene. We can access the library of Omniverse Isaac Sim robots, sensors, and environments located on a Omniverse Nucleus Server using python as well as navigating through it using the Content window. Note: the server connected to is the one that will be shown. Make sure you went through Isaac Sim First Run before proceeding.
Add the assets by simply dragging them to the stage window or the viewport.
Now lets try to do the same thing through python in our Awesome Example
Create a new stage File > new > Don’t Save
Open the extension_examples/user_examples/hello_world.py file by clicking on the Open Source Code button in the Awesome Example window.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45
from omni.isaac.examples.base_sample import BaseSample from omni.isaac.core.utils.nucleus import find_nucleus_server from omni.isaac.core.utils.stage import add_reference_to_stage from omni.isaac.core.robots import Robot import carb class HelloWorld(BaseSample): def __init__(self) -> None: super().__init__() return def setup_scene(self): world = self.get_world() world.scene.add_default_ground_plane() # Use the find_nucleus_server instead of changing it every time # you configure a new server with /Isaac folder in it result, nucleus_server = find_nucleus_server() if result is False: # Use carb to log warnings, errors and infos in your application (shown on terminal) carb.log_error("Could not find nucleus server with /Isaac folder") asset_path = nucleus_server + "/Isaac/Robots/Jetbot/jetbot.usd" # This will create a new XFormPrim and point it to the usd file as a reference # Similar to how pointers work in memory add_reference_to_stage(usd_path=asset_path, prim_path="/World/Fancy_Robot") # Wrap the jetbot prim root under a Robot class and add it to the Scene # to use high level api to set/ get attributes as well as initializing # physics handles needed..etc. # Note: this call doesn't create the Jetbot in the stage window, it was already # created with the add_reference_to_stage jetbot_robot = world.scene.add(Robot(prim_path="/World/Fancy_Robot", name="fancy_robot")) # self.log_info is a convenience function to print info on the example window instead of # the terminal for debugging. # Note: before a reset is called, we can't access information related to an Articulation # because physics handles are not initialized yet. setup_post_load is called after # the first reset so we can do so there self.log_info("Num of degrees of freedom before first reset: " + str(jetbot_robot.num_dof)) # prints None return async def setup_post_load(self): self._world = self.get_world() self._jetbot = self._world.scene.get_object("fancy_robot") # Print info about the jetbot after the first reset is called self.log_info("Num of degrees of freedom after first reset: " + str(self._jetbot.num_dof)) # prints 2 self.log_info("Joint Positions after first reset: " + str(self._jetbot.get_joint_positions())) return
Press the PLAY
button to start simulating the robot. Although it is simulating, it is not moving. The next section walks through how to make the robot move.
4.4. Move the Robot¶
In Omniverse Isaac Sim, Robots are constructed of physically accurate articulated joints. Applying actions to these articulations make them move.
Let’s apply random velocities to the Jetbot’s articulation controller to get it moving.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 | from omni.isaac.examples.base_sample import BaseSample
from omni.isaac.core.utils.nucleus import find_nucleus_server
from omni.isaac.core.utils.stage import add_reference_to_stage
from omni.isaac.core.robots import Robot
from omni.isaac.core.utils.types import ArticulationAction
import carb
import numpy as np
class HelloWorld(BaseSample):
def __init__(self) -> None:
super().__init__()
return
def setup_scene(self):
world = self.get_world()
world.scene.add_default_ground_plane()
result, nucleus_server = find_nucleus_server()
if result is False:
carb.log_error("Could not find nucleus server with /Isaac folder")
asset_path = nucleus_server + "/Isaac/Robots/Jetbot/jetbot.usd"
add_reference_to_stage(usd_path=asset_path, prim_path="/World/Fancy_Robot")
jetbot_robot = world.scene.add(Robot(prim_path="/World/Fancy_Robot", name="fancy_robot"))
return
async def setup_post_load(self):
self._world = self.get_world()
self._jetbot = self._world.scene.get_object("fancy_robot")
# This is an implicit PD controller of the jetbot/ articulation
# setting PD gains, applying actions, switching control modes..etc.
# can be done through this controller.
# Note: should be only called after the first reset happens to the world
self._jetbot_articulation_controller = self._jetbot.get_articulation_controller()
# Adding a physics callback to send the actions to apply actions with every
# physics step executed.
self._world.add_physics_callback("sending_actions", callback_fn=self.send_robot_actions)
return
def send_robot_actions(self, step_size):
# Every articulation controller has apply_action method
# which takes in ArticulationAction with joint_positions, joint_efforts and joint_velocities
# as optional args. It accepts numpy arrays of floats OR lists of floats and None
# None means that nothing is applied to this dof index in this step
# ALTERNATIVELY, same method is called from self._jetbot.apply_action(...)
self._jetbot_articulation_controller.apply_action(ArticulationAction(joint_positions=None,
joint_efforts=None,
joint_velocities=5 * np.random.rand(2,)))
return
|
Press the PLAY
button to start simulating the robot.
Note: pressing STOP
then PLAY
in this workflow might not reset the world properly. To reset press the RESET
button instead.
Extra Practice This example applies random velocities to the Jetbot’s articulation controller. Try the following exercises:
Make the Jetbot move backwards.
Make the Jetbot turn right.
Make the Jetbot stop after 5 seconds.
4.5. Using the Jetbot Class¶
Omniverse Isaac Sim also has Robot specific extensions that provide further customized functions and access to other controllers and tasks (more on this later on). Let’s re-write the previous code using the Jetbot Class to make it simpler.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 from omni.isaac.examples.base_sample import BaseSample from omni.isaac.jetbot import Jetbot from omni.isaac.core.utils.types import ArticulationAction import numpy as np class HelloWorld(BaseSample): def __init__(self) -> None: super().__init__() return def setup_scene(self): world = self.get_world() world.scene.add_default_ground_plane() # This takes care of adding a reference to the appropriate asset # in case that the prim path doesn't exist in the stage yet jetbot_robot = world.scene.add(Jetbot(prim_path="/World/Fancy_Robot", name="fancy_robot")) return async def setup_post_load(self): self._world = self.get_world() self._jetbot = self._world.scene.get_object("fancy_robot") self._world.add_physics_callback("sending_actions", callback_fn=self.send_robot_actions) return def send_robot_actions(self, step_size): self._jetbot.apply_wheel_actions(ArticulationAction(joint_positions=None, joint_efforts=None, joint_velocities=5 * np.random.rand(2,))) return
4.6. Summary¶
This tutorial covered the following topics:
Adding Omniverse Isaac Sim library components from a Nucleus Server
Adding a
Robot
to theWorld
Using a Joint-Level controller to articulate a
Robot
Using Robot Classes
Using Robot specific extensions
4.6.1. Next Steps¶
Continue on to the next tutorial in our Essential Tutorials series, Adding A Controller, to learn how to add a controller to Jetbot.
4.6.2. Further Learning¶
Nucleus Server
For an overview of how to best leverage Nucleus Servers, see the tutorial Nucleus Overview in NVIDIA Omniverse
Robot Specific Extensions
Omniverse Isaac Sim provides several robot extensions such as omni.isaac.franka, omni.isaac.universal_robots and much more. Checkout the standalone examples located at standalone_examples/api/omni.isaac.franka and standalone_examples/api/omni.isaac.universal_robots/ if you are interested to know more.