Lula Trajectory Generator

Learning Objectives

This tutorial shows how the Lula Trajectory Generator in the Motion Generation extension can be used to create both task-space and c-space trajectories that can be easily applied to a simulated robot Articulation.

Getting Started

Prerequisites

This tutorial includes multiple variations of a standalone script that loads a UR10 robot and has it follow a trajectory. Each of these variations may be saved as a Python file and run as demonstrated in the Standalone Application tutorial.

Generating a C-Space Trajectory

The LulaCSpaceTrajectoryGenerator class is able to generate a trajectory that connects a provided set of c-space waypoints. The code snippet below demonstrates how, given appropriate config files, the LulaCSpaceTrajectoryGenerator class can be initialized and used to create a sequence of ArticulationAction that can be set on each frame to produce the desired trajectory.

 1from omni.isaac.kit import SimulationApp
 2
 3simulation_app = SimulationApp({"headless": False})
 4
 5from omni.isaac.motion_generation.lula import LulaCSpaceTrajectoryGenerator, LulaKinematicsSolver
 6from omni.isaac.motion_generation import ArticulationTrajectory
 7from omni.isaac.core import World
 8from omni.isaac.core.utils.nucleus import get_assets_root_path
 9from omni.isaac.core.utils.stage import add_reference_to_stage
10from omni.isaac.core.articulations import Articulation
11from omni.isaac.core.utils.extensions import get_extension_path_from_name
12from omni.isaac.core.utils.numpy.rotations import rot_matrices_to_quats
13from omni.isaac.core.prims import XFormPrim
14import os
15import numpy as np
16
17my_world = World(stage_units_in_meters=1.0)
18
19robot_prim_path = "/ur10"
20usd_path = get_assets_root_path() + "/Isaac/Robots/UniversalRobots/ur10/ur10.usd"
21end_effector_name = "ee_link"
22
23add_reference_to_stage(usd_path, robot_prim_path)
24my_robot = Articulation(robot_prim_path)
25
26my_world.reset()
27my_robot.initialize()
28
29# Lula config files for supported robots are stored in the motion_generation extension under "/motion_policy_configs"
30mg_extension_path = get_extension_path_from_name("omni.isaac.motion_generation")
31rmp_config_dir = os.path.join(mg_extension_path, "motion_policy_configs")
32
33#Initialize a LulaCSpaceTrajectoryGenerator object
34c_space_trajectory_generator = LulaCSpaceTrajectoryGenerator(
35    robot_description_path = rmp_config_dir + "/universal_robots/ur10/rmpflow/ur10_robot_description.yaml",
36    urdf_path = rmp_config_dir + "/universal_robots/ur10/ur10_robot.urdf"
37)
38
39# 6 DOF c_space points selected from within the UR10's joint limits
40c_space_points = np.array([
41    [-0.41 , 0.5 , -2.36 , -1.28 , 5.13 , -4.71 , ],
42    [-1.43 , 1.0 , -2.58 , -1.53 , 6.14 , -4.74 , ],
43    [-2.83 , 0.34 , -2.11 , -1.38 , 1.26 , -4.71 , ]
44    ])
45trajectory = c_space_trajectory_generator.compute_c_space_trajectory(c_space_points)
46if trajectory is None:
47    print("No trajectory could be generated!")
48    exit(0)
49
50# Use the ArticulationTrajectory wrapper to run Trajectory on UR10 robot Articulation
51physics_dt = 1/60. # Physics steps are fixed at 60 fps when running a standalone script
52articulation_trajectory = ArticulationTrajectory(my_robot,trajectory,physics_dt)
53
54# Returns a list of ArticulationAction meant to be set on subsequent physics steps
55action_sequence = articulation_trajectory.get_action_sequence()
56
57
58#Create frames to visualize the c_space targets of the UR10
59lula_kinematics_solver = LulaKinematicsSolver(
60    robot_description_path = rmp_config_dir + "/universal_robots/ur10/rmpflow/ur10_robot_description.yaml",
61    urdf_path = rmp_config_dir + "/universal_robots/ur10/ur10_robot.urdf"
62)
63for i,point in enumerate(c_space_points):
64    position,rotation = lula_kinematics_solver.compute_forward_kinematics(end_effector_name, point)
65    add_reference_to_stage(get_assets_root_path() + "/Isaac/Props/UIElements/frame_prim.usd", f"/target_{i}")
66    frame = XFormPrim(f"/target_{i}",scale=[.04,.04,.04])
67    frame.set_world_pose(position,rot_matrices_to_quats(rotation))
68
69
70
71#Run the action sequence on a loop
72articulation_controller = my_robot.get_articulation_controller()
73action_index = 0
74while simulation_app.is_running():
75    my_world.step(render=True)
76    if my_world.is_playing():
77        if action_index == 0:
78            #Teleport robot to the initial state of the trajectory
79            my_robot.set_joint_positions(c_space_points[0])
80            my_robot.set_joint_velocities(np.zeros_like(c_space_points[0]))
81
82        articulation_controller.apply_action(action_sequence[action_index])
83
84        action_index += 1
85        action_index %= len(action_sequence)
86
87simulation_app.close()

On lines 33-37, the LulaCSpaceTrajectoryGenerator class is initialized. It is then used to generate an instance of the Trajectory Interface that connects a series of c_space waypoints (see lines 40-48). This Trajectory object stores the desired trajectory, but it needs to be wrapped in an Articulation Trajectory to interface with the robot Articulation (lines 50-52). The ArticulationTrajectory is used on line 52 to generate a sequence of ArticulationAction that can be directly passed to the robot Articulation on subsequent frames.

If no trajectory can be computed that connects the c-space waypoints, the trajectory returned by LulaCSpaceTrajectoryGenerator.compute_c_space_trajectory will be None. This can occur when one of the specified c-space waypoints is not reachable or is very close to a joint limit. On lines 46-48, the output trajectory is checked, and if it does not exist, the simulation is closed.

One lines 59-67, a visualization of the original c_space_points is created by converting them to task-space points. This code is not functional, but it helps to verify that the robot is hitting every target.

In the main loop on lines 71-85, the trajectory is rolled out on loop forever. Each action in the action_sequence is commanded for exactly one frame, and when the trajectory is on its first frame, the robot Articulation is teleported to the initial state of the trajectory.

../_images/isaac_sim_c_space_trajectory.gif

Generating a Task-Space Trajectory

Simple Case: Linearly Connecting Waypoints

Generating a task-space trajectory is similar to generating a c-space trajectory. In the simplest use-case, the user may pass in a set of task-space position and quaternion orientation targets, which will be linearly interpolated in task-space to produce the resulting trajectory. An example is provided in the code snippet below.

 1from omni.isaac.kit import SimulationApp
 2
 3simulation_app = SimulationApp({"headless": False})
 4
 5from omni.isaac.motion_generation.lula import LulaTaskSpaceTrajectoryGenerator
 6from omni.isaac.motion_generation import ArticulationTrajectory
 7from omni.isaac.core import World
 8from omni.isaac.core.utils.nucleus import get_assets_root_path
 9from omni.isaac.core.utils.stage import add_reference_to_stage
10from omni.isaac.core.articulations import Articulation
11from omni.isaac.core.utils.extensions import get_extension_path_from_name
12from omni.isaac.core.prims import XFormPrim
13import os
14import numpy as np
15
16my_world = World(stage_units_in_meters=1.0)
17
18robot_prim_path = "/ur10"
19usd_path = get_assets_root_path() + "/Isaac/Robots/UniversalRobots/ur10/ur10.usd"
20end_effector_name = "ee_link"
21
22add_reference_to_stage(usd_path, robot_prim_path)
23my_robot = Articulation(robot_prim_path)
24
25my_world.reset()
26my_robot.initialize()
27
28# Lula config files for supported robots are stored in the motion_generation extension under "/motion_policy_configs"
29mg_extension_path = get_extension_path_from_name("omni.isaac.motion_generation")
30rmp_config_dir = os.path.join(mg_extension_path, "motion_policy_configs")
31
32#Initialize a LulaCSpaceTrajectoryGenerator object
33task_space_trajectory_generator = LulaTaskSpaceTrajectoryGenerator(
34    robot_description_path = rmp_config_dir + "/universal_robots/ur10/rmpflow/ur10_robot_description.yaml",
35    urdf_path = rmp_config_dir + "/universal_robots/ur10/ur10_robot.urdf"
36)
37
38# Choose reachable position and orientation targets
39task_space_position_targets = np.array([
40    [0.3, -0.3, 0.1],
41    [0.3, 0.3, 0.1],
42    [0.3, 0.3, 0.5],
43    [0.3, -0.3, 0.5],
44    [0.3, -0.3, 0.1]
45    ])
46task_space_orientation_targets = np.tile(np.array([0,1,0,0]),(5,1))
47trajectory = task_space_trajectory_generator.compute_task_space_trajectory_from_points(
48    task_space_position_targets, task_space_orientation_targets, end_effector_name
49)
50if trajectory is None:
51    print("No trajectory could be generated!")
52    exit(0)
53
54# Use the ArticulationTrajectory wrapper to run Trajectory on UR10 robot Articulation
55physics_dt = 1/60.
56articulation_trajectory = ArticulationTrajectory(my_robot,trajectory,physics_dt)
57
58# Returns a list of ArticulationAction meant to be set on subsequent physics steps
59action_sequence = articulation_trajectory.get_action_sequence()
60
61
62#Create frames to visualize the task_space targets of the UR10
63for i in range(len(task_space_position_targets)):
64    add_reference_to_stage(get_assets_root_path() + "/Isaac/Props/UIElements/frame_prim.usd", f"/target_{i}")
65    frame = XFormPrim(f"/target_{i}",scale=[.04,.04,.04])
66    position = task_space_position_targets[i]
67    orientation = task_space_orientation_targets[i]
68    frame.set_world_pose(position,orientation)
69
70
71#Run the action sequence on a loop
72articulation_controller = my_robot.get_articulation_controller()
73action_index = 0
74while simulation_app.is_running():
75    my_world.step(render=True)
76    if my_world.is_playing():
77        if action_index == 0:
78            #Teleport robot to the initial state of the trajectory
79            initial_state = action_sequence[0]
80            my_robot.set_joint_positions(initial_state.joint_positions)
81            my_robot.set_joint_velocities(initial_state.joint_velocities)
82
83        articulation_controller.apply_action(action_sequence[action_index])
84
85        action_index += 1
86        action_index %= len(action_sequence)
87
88simulation_app.close()

In moving to the task-space trajectory generator, there are few code changes required. The initialization is nearly the same on line 33 as for the c-space trajectory generator. The main changes are on lines 38-49 where a task-space trajectory is specified. When using the function LulaTaskSpaceTrajectoryGenerator.compute_task_space_trajectory_from_points, a position and orientation target must be specified for each task-space waypoint. Additionally, a frame from the robot URDF must be specified as the end effector frame. If the waypoints cannot be connected to form a trajectory, the compute_task_space_trajectory_from_points function will return None. This case is checked on lines 50-52.

There are minor changes in the rest of the code snippet from the c-space trajectory case, but they are not significant enough to discuss in this tutorial.

../_images/isaac_sim_task_space_trajectory.gif

Defining Complicated Trajectories

The LulaTaskSpaceTrajectoryGenerator can be used to create paths with more complicated specifications than to connect a set of task-space targets linearly. Using the class lula.TaskSpacePathSpec, the user may define paths with arcs and circles with multiple options for orientation targets. The code snippet below demonstrates creating a lula.TaskSpacePathSpec and gives an example of each available function for adding to a task-space path.

  1from omni.isaac.kit import SimulationApp
  2
  3simulation_app = SimulationApp({"headless": False})
  4
  5from omni.isaac.motion_generation.lula import LulaTaskSpaceTrajectoryGenerator
  6from omni.isaac.motion_generation import ArticulationTrajectory
  7from omni.isaac.core import World
  8from omni.isaac.core.utils.nucleus import get_assets_root_path
  9from omni.isaac.core.utils.stage import add_reference_to_stage
 10from omni.isaac.core.articulations import Articulation
 11from omni.isaac.core.utils.extensions import get_extension_path_from_name
 12from omni.isaac.core.prims import XFormPrim
 13import os
 14import numpy as np
 15import lula
 16
 17my_world = World(stage_units_in_meters=1.0)
 18
 19robot_prim_path = "/ur10"
 20usd_path = get_assets_root_path() + "/Isaac/Robots/UniversalRobots/ur10/ur10.usd"
 21end_effector_name = "ee_link"
 22
 23add_reference_to_stage(usd_path, robot_prim_path)
 24my_robot = Articulation(robot_prim_path)
 25
 26my_world.reset()
 27my_robot.initialize()
 28
 29# Lula config files for supported robots are stored in the motion_generation extension under "/motion_policy_configs"
 30mg_extension_path = get_extension_path_from_name("omni.isaac.motion_generation")
 31rmp_config_dir = os.path.join(mg_extension_path, "motion_policy_configs")
 32
 33#Initialize a LulaCSpaceTrajectoryGenerator object
 34task_space_trajectory_generator = LulaTaskSpaceTrajectoryGenerator(
 35    robot_description_path = rmp_config_dir + "/universal_robots/ur10/rmpflow/ur10_robot_description.yaml",
 36    urdf_path = rmp_config_dir + "/universal_robots/ur10/ur10_robot.urdf"
 37)
 38
 39
 40# The following code demonstrates how to specify a complicated task-space path
 41# using the lula.TaskSpacePathSpec object
 42
 43#Lula has its own classes for Rotations and 6 DOF poses: Rotation3 and Pose3
 44r0 = lula.Rotation3(np.pi, np.array([1.0, 0.0, 0.0]))
 45t0 = np.array([.3,-.3,.3])
 46spec = lula.create_task_space_path_spec(lula.Pose3(r0,t0))
 47
 48# Add path linearly interpolating between r0,r1 and t0,t1
 49t1 = np.array([.3,-.3,.5])
 50r1 = lula.Rotation3(np.pi/2,np.array([1,0,0]))
 51spec.add_linear_path(lula.Pose3(r1, t1))
 52
 53# Add pure translation.  Constant rotation is assumed
 54spec.add_translation(t0)
 55
 56# Add pure rotation.
 57spec.add_rotation(r0)
 58
 59# Add three-point arc with constant orientation.
 60t2 = np.array([.3,.3,.3,])
 61midpoint = np.array([.3,0,.5])
 62spec.add_three_point_arc(t2, midpoint, constant_orientation = True)
 63
 64# Add three-point arc with tangent orientation.
 65spec.add_three_point_arc(t0, midpoint, constant_orientation = False)
 66
 67# Add three-point arc with orientation target.
 68spec.add_three_point_arc_with_orientation_target(lula.Pose3(r1, t2), midpoint)
 69
 70# Add tangent arc with constant orientation. Tangent arcs are circles that connect two points
 71spec.add_tangent_arc(t0, constant_orientation = True)
 72
 73# Add tangent arc with tangent orientation.
 74spec.add_tangent_arc(t2, constant_orientation=False)
 75
 76# Add tangent arc with orientation target.
 77spec.add_tangent_arc_with_orientation_target(
 78    lula.Pose3(r0, t0))
 79
 80trajectory = task_space_trajectory_generator.compute_task_space_trajectory_from_path_spec(
 81    spec, end_effector_name
 82)
 83if trajectory is None:
 84    print("No trajectory could be generated!")
 85    exit(0)
 86
 87# Use the ArticulationTrajectory wrapper to run Trajectory on UR10 robot Articulation
 88physics_dt = 1/60.
 89articulation_trajectory = ArticulationTrajectory(my_robot,trajectory,physics_dt)
 90
 91# Returns a list of ArticulationAction meant to be set on subsequent physics steps
 92action_sequence = articulation_trajectory.get_action_sequence()
 93
 94
 95#Run the action sequence on a loop
 96articulation_controller = my_robot.get_articulation_controller()
 97action_index = 0
 98while simulation_app.is_running():
 99    my_world.step(render=True)
100    if my_world.is_playing():
101        if action_index == 0:
102            #Teleport robot to the initial state of the trajectory
103            initial_state = action_sequence[0]
104            my_robot.set_joint_positions(initial_state.joint_positions)
105            my_robot.set_joint_velocities(initial_state.joint_velocities)
106
107        articulation_controller.apply_action(action_sequence[action_index])
108
109        action_index += 1
110        action_index %= len(action_sequence)
111
112simulation_app.close()

The code snippet above moves mainly between three translations: t0, t1, t2 with possible rotations r0, r1. The lula.TaskSpacePathSpec object is created with an initial position on line 46. Each following add function that is called adds a path between the last position in the path_spec so far and a new position.

The basic possibilities are:

Linearly interpolate translation to a new point while keeping rotation fixed (line 54)

Linearly interpolate rotation to a new point while keeping translation fixed (line 57)

Linearly interpolate both rotation and translation to a new 6 DOF point (line 51)

The lula.TaskSpacePathSpec also makes it easy to define various arcs and circular paths that connect points in space. A three-point arc can be defined that moves through a midpoint to a translation target. There are three options for the orientation of the robot while moving along the path:

Keep rotation constant (line 62)

Always stay oriented tangent to the arc (line 65)

Linearly interpolate rotation to a rotation target (line 68)

Finally, a circular path may be specified without defining a midpoint as on lines 71, 74, and 77. The same three options for specifying orientation are available.

../_images/isaac_sim_task_space_path_spec.gif

Summary

This tutorial shows how to use the Lula Trajectory Generator to generate c-space and task-space trajectories for a robot. Task-space trajectories may be specified simply using a series of task-space waypoints that will be connected linearly, or they may be defined piecewise with many different options for connecting each pair of points in space.

Further Learning

Reference the Motion Generation page for a complete description of trajectories in Omniverse Isaac Sim.