4. Using Sensors: Generic Range Sensor

The generic LiDAR sensor lets users have full control of the scanning pattern and frequency of the sensors. This is ideal for sensors that have scanning patterns that are non-rotational, asymmetrical, or non-repetitive.

4.1. Learning Objectives

This tutorial introduces how to customize scanning patterns for a generic range sensor in Omniverse Isaac Sim. After this tutorial, you will know how to add a range sensor and customize its scanning pattern.

10-15 Minute Tutorial

4.2. Getting Started

Prerequisites

  • Complete Isaac Sim Interface and the GUI tutorials to learn the basics of constructing objects and adding sensors using the GUI.

  • The generic sensor can only be customized using Python APIs, please complete Isaac Sim Workflows, and Hello World to familiarize yourself with using python APIs both in the standalone and extension workflow.

  • The generic sensor is an extension of the rotating LiDAR sensor, so please complete the Advanced Tutorial Using Sensors: LIDAR to learn how to add a LiDAR sensor to scene, use it to detect objects, and attach it to geometries.

To start, let’s run the generic range sensor example:

  1. Go to the top menu bar and click Isaac Examples > Sensors > Generic Range Sensor.

  2. Press the Load Sensor button.

  3. Press the Load Scene button.

  4. Press the Set Sensor Pattern button to load the example sensor pattern.

  5. Press the PLAY button to begin simulating.

You will see in the viewport a LiDAR sensor originated from the origin, a rectangular wall sitting slightly off the origin in the +x direction, and the rays turning red when they hit the block. You’ll also notice that the LiDAR rays are not rotating but are zigzagging within a confined region.

../_images/isaac_tutorial_advanced_generic_sensor_example.gif

To visualize the pattern, you can save the image imprinted on the wall from the rays that hit it. To do so, select or type out the desire output directory and press Save Pattern Image. Open the saved image file and you should see a zigzag pattern.

../_images/isaac_tutorial_advanced_generic_sensor_pattern.png

4.3. Generating Scanning Pattern

To customize scanning patterns, these are the parameters that needs to be filled or modified.

  • sampling_rate: Number of scans per second.

  • batch_size: The number of scans each batch of data contains. The size should be at least large enough to run a few rendering frames without running out. For example, if you wish to scan at a sampling rate of 2400 scans per second, and your frame rendering rate is at 120 fps, then each frame will render 20 scans. If you send a batch size 12000, you should be able to render 600 frames, or 5 seconds at 120 fps before you run out of data. If batch_size is less than what is needed to satisfy the desired sampling rate (i.e. batch_size < sampling_rate/fps), then the sensor will scan at a rate that equals the batch_size per frame, which likely means you will be scanning slower than desired.

  • sensor_pattern: a Nx2 size numpy array. N is batch_size, and the columns are [azimuth, zenith] angles of each scanning ray. Azimuth is the ray’s horizontal angle measured from x-axis, and zenith angle is the vertical angle measured from z-axis.

  • origin_offsets: (Optional) a Nx3 size numpy array, N is the batch size, and each row is the individual ray’s offset from origin in [x,y,z] coordinates.

4.3.1. Online Pattern Generation

Let’s take a closer look at our example code to see how to produce the zigzag scanning pattern. The pattern in the example is generated programmatically inside the same script that runs the example. Click on the Open Source Code icon in the upper right hand corner of the example window and open the python source code for this example.

The pattern generation happens inside _set_sensor_pattern() function, copied below. The pattern is sweeping horizontally 10 times for each round of up and down, resulting in the zigzag.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
def _set_sensor_pattern(self):
    # custom pattern generation

    # send data in batch that are at least large enough to run a few rendering frames without running out of data.
    # if batch_size > (sampling rate/rendering rate), the sensor will process all of the batch and ask for the next batch right before it runs out.
    # if batch_size < (sampling rate/rendering_rate), the sensor will scan only the provided rays in a given frame, which means it will be scanning slower than intended
    self._batch_size = int(1e6)  # size of each batch of data being processed
    self._half_batch = int(self._batch_size / 2)
    # example scanning pattern is a zigzag
    # each ray specified by an azimuth (horizontal angle measured from x-axis) and a zenith angle (vertical angle measured from z-axis)
    frequency = 10
    N_pts = int(self._batch_size / frequency / 2)
    # azimuth angle zigzag between the limits (frequency) times every batch
    self._azimuth = np.tile(
        np.append(np.linspace(-np.pi / 4, np.pi / 4, N_pts), np.linspace(np.pi / 4, -np.pi / 4, N_pts)), frequency
    )
    # zenith angle goes up and down once every batch
    self._zenith = np.append(
        np.linspace(-np.pi / 4, np.pi / 4, self._half_batch), np.linspace(np.pi / 4, -np.pi / 4, self._half_batch)
    )

    # custom pattern must be sent as an array of [azimuth, zenith] angles.
    self.sensor_pattern = np.stack((self._azimuth, self._zenith))

Origin offset is optional. For the example, a small random offset was added as seen below. For no offsets, you can either use an array of zeros, or simply skip setting the origin_offsets parameter.

1
2
3
4
    # individual rays can have an offset at the origin
    # adding random offsets to the origin for the example pattern
    self.origin_offsets = 5 * np.random.random((self._batch_size, 3))
    # self.origin_offsets = np.zeros((self._batch_size,3))                  # no offsets

4.3.2. Streaming Pattern Through File

If you do not have a programmatic way to generate the scanning pattern from scratch, or if you do not wish to disclose the generation method of the scanning pattern, you can also import data from file. The example below shows importing data from a .csv file and converting it to match the format of sensor_pattern parameter.

1
2
3
4
    ## import data from file
    self._sensor_pattern = np.loadtxt("filename.csv", delimiter=",")
    self._batch_size = np.shape(self._sensor_pattern)[0]
    self.sensor_pattern = np.deg2rad(self._sensor_pattern).T.copy()        ##  MUST USE .copy()

4.4. Setting Scanning Pattern

When the sensor processes each batch of [azimuth, zenith] pairs, just before it is about to run out of data, it will set the variable send_next_batch() to True, at which point, you can send the next batch via set_next_batch_rays(prim_path, sensor_pattern), plus set_next_batch_offsets(prim_path, sensor_pattern) if there are any origin offsets. Like shown below.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
def _on_editor_step(self, step):
    if not self._timeline.is_playing():
        return

    if self._timeline.is_playing():
        if self._generic:
            if self._pattern_set:
                if self._sensor.send_next_batch(
                    self._genericPath
                ):  # send_next_batch will turn True if the sensor is running out data and needs more
                    self._sensor.set_next_batch_rays(
                        self._genericPath, self.sensor_pattern
                    )  # set the next batch data using set_next_batch_rays()
                    self._sensor.set_next_batch_offsets(
                        self._genericPath, self.origin_offsets
                    )  # (Optional) add individual ray offsets if there are any

4.5. Summary

This tutorial covered the following topics:

  1. Using the generic range sensor.

  2. Generate or stream custom scanning patterns for the generic range sensor.