What Is Isaac Sim?

NVIDIA Omniverse™ Isaac Sim is a robotics simulation toolkit for the NVIDIA Omniverse™ platform. Isaac Sim has essential features for building virtual robotic worlds and experiments. It provides researchers and practitioners with the tools and workflows they need to create robust, physically accurate simulations and synthetic datasets. Isaac Sim supports navigation and manipulation applications through Isaac SDK and ROS/ROS2. It simulates sensor data, such as RGB-D, Lidar and IMU sensors, and various computer vision techniques, such as Domain Randomization, ground truth labeling, segmentation, and bounding boxes.

../_images/isaac_main_intro.png

System Architecture

../_images/isaac_overview_system_diagram_0.png

Development Workflows

../_images/isaac_overview_workflows_diagram_1.png

Omniverse Kit

Omniverse Isaac Sim uses the NVIDIA Omniverse™ Kit SDK, a toolkit for building native Omniverse applications and microservices. Omniverse Kit provides a wide variety of functionality through a set of light-weight plugins. Plugins are all authored with C interfaces for persistent ABI compatibility, however a Python interpreter is also provided for accessible scripting and customization.

The Python API that can be used to write new extensions to Omniverse Kit or new experiences for Omniverse.

Omniverse Nucleus

Omniverse Isaac Sim uses NVIDIA Omniverse™ Nucleus to access content such as USD files for environments and robots. Omniverse Nucleus services allow a variety of client applications, renderers, and microservices to share and modify representations of virtual worlds in Omniverse Isaac Sim.

Nucleus operates under a publish/subscribe model. Subject to access controls, Omniverse clients can publish modifications to digital assets and virtual worlds to the Nucleus Database (DB) or subscribe to their changes. Changes are transmitted in real-time between connected applications. Digital assets can include geometry, lights, materials, textures and other data that describe virtual worlds and their evolution through time.

This allows a variety of Omniverse-enabled client applications (Apps, Connectors, and others) to share and modify authoritative representations of virtual worlds.

  • See Nucleus Overview for a more in-depth look at Nucleus’s data model, architecture, and distribution platforms.

USD

Omniverse Isaac Sim uses USD interchange file format to represent scenes. Universal Scene Description (USD) is an easily extensible, open-source 3D scene description and file format developed by Pixar for content creation and interchange among different tools. As a result of its power and versatility, it’s being widely adopted, not only in the visual effects community, but also in architecture, design, robotics, manufacturing, and other disciplines.

Documentation Overview

The documentation is split into the following sections: Installation Guide, API Documentation, Tutorials, Manuals, and Reference Materials.

Installation Guide

The Basic Isaac Sim Installation is the recommended installation process for all users.

Omniverse Isaac Sim also supports more advanced, headless, and remote installation options. Docker, Cloud, and Streaming related setups are covered in the Advanced Installation guide.

API Documentation

Python API Documentation for both Omniverse and Isaac Sim Extensions can be found here.

Tutorials

We strongly recommend beginning with the Required Tutorials. This series of short 10-15 minute tutorials are designed to teach the foundations of how to effectively develop your own simulations and tools in Omniverse Isaac Sim.

  • Required Tutorials: Essential tutorials which start at the fundamentals and build up to more complex scenarios with multiple robots interacting with each other

  • GUI Tutorials: Tutorials that covers world building via GUI and the fundamentals of working in Omniverse Kit.

  • Advanced Tutorials: Tutorials which cover specific individual topics

  • Replicator Tutorials: Tutorials that cover synthetic data generation

  • ROS Tutorials: Tutorials walking though different ROS workflows building up in complexity

  • ROS2 tutorials: Tutorials showing how to use the ROS2 bridge

Examples

We provide a suite of useful examples for users to quickly try intermediate and advanced applications in Omniverse Isaac Sim. Each example brings well commented source code, so users can learn, grab, and modify for their own use cases.

See the complete list of examples in the Isaac Examples Menu.

Manuals

The Manuals section provides documentation for various extensions, additional examples, and features provided in Isaac Sim. These generally cover APIs or details about a specific topic and are less tutorialized.

Reference Material

The Reference Material section provides references, links, and resources to assist developers.