Using the Public Streaming API

The following guide contains information pertaining to publishing custom Omniverse applications to the public streaming API in order to support hosting custom-built applications for use by Users subscribed to the Omniverse public streaming API. This guide assumes your organization is already registered to the Omniverse public streaming API, allowing your Users to access collections of Omniverse workflows from their web browsers, streaming directly their Nucleus scenes from their storage locations.

To get started with the code samples used for illustrative purposes in this guide, you may wish to create a custom Omniverse extension which will be used for forwarding communications between your local installation and the remote server hosting the streaming sessions:

exts/omni.services.streaming.application.example/config/extension.toml
 1[package]
 2authors = ["Name <name@email.com>"]
 3category = "Services"
 4changelog = "docs/CHANGELOG.md"
 5description = "Application streaming integration sample"
 6icon = "data/icon.png"
 7keywords = ["kit", "services", "application", "streaming", "example"]
 8readme  = "docs/README.md"
 9repository = "https://gitlab-master.nvidia.com/kit-extensions/kit-services"
10title = "Application streaming integration sample"
11version = "0.1.0"
12
13[dependencies]
14"omni.services.core" = {}
15"omni.services.client" = {}
16"omni.services.transport.server.http" = {}
17"omni.services.transport.client.http_async" = {}
18
19[[python.module]]
20name = "omni.services.streaming.application.example"
21
22[settings]
23exts."omni.services.transport.server.http".http.enabled = true
24exts."omni.services.transport.server.http".host = "0.0.0.0"
25exts."omni.services.transport.server.http".port = 8011
exts/omni.services.streaming.application.example/models/streaming.py
 1# Copyright (c) 2023, NVIDIA CORPORATION.  All rights reserved.
 2#
 3# NVIDIA CORPORATION and its licensors retain all intellectual property
 4# and proprietary rights in and to this software, related documentation
 5# and any modifications thereto.  Any use, reproduction, disclosure or
 6# distribution of this software and related documentation without an express
 7# license agreement from NVIDIA CORPORATION is strictly prohibited.
 8
 9from enum import Enum
10
11from fastapi import BaseModel, Field
12
13
14class PortType(str, Enum):
15   """Application port types."""
16   tcp = "TCP"
17   udp = "UDP"
18
19class Port(BaseModel):
20   """Application port specifications."""
21   name: str = Field(..., description="Port name.")
22   number: int = Field(..., ge=0, le=65535, description="Port number.")
23   type: PortType = Field(..., description="Port type.")
24
25class StreamingMode(str, Enum):
26   """Streaming mode."""
27   secure = "secure"
28   insecure = "insecure"
29
30class StreamRequestSpec(BaseModel):
31   """Extensible specification that is passed to the streaming controller endpoint."""
32   kit_args: List[str] = Field(
33      default=[], description="Array of strings to be appended to the Omniverse application arguments."
34   )
35   kit_env: Dict[str, str] = Field(
36      default={}, description="Dictionary containing environment variables for the Omniverse application container."
37   )
38   streaming_mode: StreamingMode = Field(
39      default=StreamingMode.secure, description="Streaming mode for the application."
40   )
41   image: Optional[str] = Field(
42      default=None, description="Full path to Omniverse application image with tag."
43   )
44   user_ports: List[Port] = Field(
45      default=[], description="List of extra container ports to expose."
46   )
47   user_labels: Dict[str, str] = Field(
48      default={}, description="Optional list of labels to assign to the Omniverse application pod."
49   )
50   node_labels: Dict[str, str] = Field(
51      default={}, description="Optional set of node labels that user wants to use for scheduling purposes."
52   )
53
54class CreateStreamRequest(BaseModel):
55   """Model of a request to create a new streaming instance."""
56   url: str = Field(..., description="Stage URL (e.g: `omniverse://<server_fqdn>/<file_path_to_usd?>`)")
57   access_token: str = Field(..., description="Access token that the Omniverse application will use to connect to Nucleus.")
58   username: str = Field(..., description="Authenticated username to use for the session.")
59   spec: Optional[StreamRequestSpec] = Field(default=StreamRequestSpec(), description="Extensible stream specification.")
exts/omni.services.streaming.application.example/python_ext.py
 1# Copyright (c) 2023, NVIDIA CORPORATION.  All rights reserved.
 2#
 3# NVIDIA CORPORATION and its licensors retain all intellectual property
 4# and proprietary rights in and to this software, related documentation
 5# and any modifications thereto.  Any use, reproduction, disclosure or
 6# distribution of this software and related documentation without an express
 7# license agreement from NVIDIA CORPORATION is strictly prohibited.
 8
 9import omni.ext
10
11from omni.services.core.main import deregister_router, register_router
12
13from .services.streaming import router
14
15
16class StreamingApplicationExampleExtension(omni.ext.IExt):
17    """Streaming application example extension."""
18
19    def on_startup(self, ext_id: str) -> None:
20        register_router(router=router, tags=["examples"])
21
22    def on_shutdown(self) -> None:
23        deregister_router(router=router)

Healthcheck

Before attempting to communicate with remote endpoints in order to schedule Omniverse application streams, you may wish to first reach the healthcheck API available on the service in order to ensure resources are in a state allowing them to serve incoming requests. Proceeding by first establishing that components are exhibiting standard operational behavior can greatly help reduce potential investigation times in cases where resources may be limited on remote clusters, or if network connectivity issues may be temporarily affecting provisioning due to maintenance windows.

To query the /healthcheck API for the status of the streaming service APIs, navigate to the OpenAPI specification page of a production cluster (e.g. https://store-portal.ovx-prd31-sjc11.proxy.ace.ngc.nvidia.com/backend/api-docs), expanding the /healthcheck endpoint of the Swagger documentation page before clicking the _Execute_ button and confirming that the endpoint responds with the successful application/json Content-Type of:

1{
2   "ok": true
3}

Listing Streaming Sessions

In order to list existing streaming sessions started by Users subscribed to the service, an HTTP GET request can be issued against a service endpoint responsible for returning the list of existing streaming sessions. The most straightforward example illustrating the results of this operation can be implemented using the following code sample:

exts/omni.services.streaming.application.example/services/streaming.py
 1# Copyright (c) 2023, NVIDIA CORPORATION.  All rights reserved.
 2#
 3# NVIDIA CORPORATION and its licensors retain all intellectual property
 4# and proprietary rights in and to this software, related documentation
 5# and any modifications thereto.  Any use, reproduction, disclosure or
 6# distribution of this software and related documentation without an express
 7# license agreement from NVIDIA CORPORATION is strictly prohibited.
 8
 9from typing import Any, Dict
10
11from omni.services.client import AsyncClient
12from omni.services.core import routers
13
14router = routers.ServiceAPIRouter()
15
16
17@router.get(
18   "/list-streams",
19   summary="Return the list of streaming sessions.",
20   description="Returns the list of streaming sessions currently active.",
21)
22async def list_streams() -> Dict[str, Any]:
23   """
24   Return metadata information about application streaming sessions.
25
26   Args:
27      None
28
29   Returns:
30      Dict[str, Any]: Metadata information about application streaming sessions.
31
32   """
33    client = AsyncClient(uri="https://cockpit.devtest.az.cloud.omniverse.nvidia.com")
34    response = await client.session_service.session.get()
35    return response

In the case of a successful response, the remote endpoint will respond with an HTTP 200 and an application/json Content-Type containing details about the streaming session that was created on behalf of the User:

 1{
 2   "results": [
 3      {
 4         "session_record": {
 5            "session_id": "99a13993-77d4-402b-81a8-a563858908d9",
 6            "username": "user@email.com",
 7            "source_address": "0.0.0.0",
 8            "destination_address": "12.34.56.78",
 9            "session_routes": "16249:TCP:NPORT:30136,15426:UDP:MEDIA:31306",
10            "nucleus": "external.devtest.az.cloud.omniverse.nvidia.com",
11            "stream_type": "app",
12            "pod_name": "ovc-v1-kit-99a13993-77d4-402b-81a8-a563858908d9",
13            "service_name": "ovc-v1-kit-99a13993-77d4-402b-81a8-a563858908d9",
14            "configmap_name": "ovc-v1-kit-99a13993-77d4-402b-81a8-a563858908d9",
15            "session_link": "https://devtest.cloud.omniverse.nvidia.com/webclient/index.html?signalingserver=customer.devtest.az.cloud.omniverse.nvidia.com&signalingport=48322&mediaserver=customer.devtest.az.cloud.omniverse.nvidia.com&mediaport=15426&sessionid=99a13993-77d4-402b-81a8-a563858908d9&mic=0&cursor=free&server=&nucleus=external.devtest.az.cloud.omniverse.nvidia.com&resolution=1920x1080&fps=60&autolaunch=true&backendurl=public-api.devtest.az.cloud.omniverse.nvidia.com&accessToken=<Nucleus Access Token>&terminateVerb=DELETE",
16            "created_at": "2023-08-04T14:53:17.273069+00:00",
17            "updated_at": "2023-08-04T14:53:17.273078+00:00"
18         },
19         "status": {
20            "phase": "Running",
21            "node_name": "cbg1-f06-ovx-05",
22            "host_ip": "12.34.56.78",
23            "pod_ip": "100.96.18.25",
24            "start_time": "2023-08-04T14:53:11+00:00"
25         }
26      },
27      // Include additional session metadata, for any available on the service.
28      // [...]
29   ]
30}

Creating Streaming Sessions

In order to create new streaming sessions, queries can be issued against the create-stream API, which accepts POST queries containing information such as the access token to use to authorize requests for the given username, along with data such as the URL of the NGC Docker container hosting the Omniverse application to stream:

exts/omni.services.streaming.application.example/services/streaming.py
 1# Copyright (c) 2023, NVIDIA CORPORATION.  All rights reserved.
 2#
 3# NVIDIA CORPORATION and its licensors retain all intellectual property
 4# and proprietary rights in and to this software, related documentation
 5# and any modifications thereto.  Any use, reproduction, disclosure or
 6# distribution of this software and related documentation without an express
 7# license agreement from NVIDIA CORPORATION is strictly prohibited.
 8
 9from typing import Any, Dict
10
11from omni.services.client import AsyncClient
12from omni.services.core import routers
13
14from .models.streaming import CreateStreamRequest, StreamingMode, StreamRequestSpec
15
16router = routers.ServiceAPIRouter()
17
18
19@router.post(
20   "/create-stream",
21   summary="Create a new streaming session",
22   description="Returns metadata information about a newly-created streaming session.",
23)
24async def create_stream() -> Dict[str, Any]:
25   """
26   Send a request to create a streaming session using the given NGC Docker container on behalf of the User.
27
28   Args:
29      None
30
31   Returns:
32      Dict[str, Any]: Metadata information about the streaming session that was just created.
33
34   """
35   client = AsyncClient(uri="https://cockpit.devtest.az.cloud.omniverse.nvidia.com")
36   create_stream_options = CreateStreamRequest(
37      url="omniverse://store.devtest.az.cloud.omniverse.nvidia.com",
38      access_token="<Nucleus Access Token>",
39      username="user@email.com",
40      spec=StreamRequestSpec(
41          kit_args=[],
42          kit_env={},
43          streaming_mode=StreamingMode.secure,
44          image="nvcr.io/nvidian/omniverse/ov-composer-streaming-webrtc-ovc:2023.1.0-beta.28",
45          user_ports=[],
46          user_labels={},
47          node_labels={},
48      ),
49   )
50   response = await client.session_service.session.post(**create_stream_options.dict())
51   return response

Note

The code sample above provides a link to an Omniverse USD Composer application instance featuring WebRTC streaming capabilities in order to enable remote development workflows. To publish a custom version of an application’s Docker container, host the desired features to your organization’s private registry on NGC.

For additional information about using the NGC private registry, consult the User Guide available online.

In the case of a successful stream creation, the remote endpoint will respond with an HTTP 200 an a application/json Content-Type containing the URL of the stream that was created on behalf of the User:

1{
2   "redirect": "https://devtest.cloud.omniverse.nvidia.com/webclient/index.html?signalingserver=customer.devtest.az.cloud.omniverse.nvidia.com&signalingport=48322&mediaserver=customer.devtest.az.cloud.omniverse.nvidia.com&mediaport=15426&sessionid=99a13993-77d4-402b-81a8-a563858908d9&mic=0&cursor=free&server=&nucleus=external.devtest.az.cloud.omniverse.nvidia.com&resolution=1920x1080&fps=60&autolaunch=true&backendurl=public-api.devtest.az.cloud.omniverse.nvidia.com&accessToken=<Nucleus Access Token>&terminateVerb=DELETE",
3   "session_id": "99a13993-77d4-402b-81a8-a563858908d9",
4   "session_routes": "16249:TCP:NPORT:30136,15426:UDP:MEDIA:31306"
5}

Deleting Streaming Sessions

Once interaction with a streaming session has been completed, Users can proceed to terminate the application instance they have initiated in order to free resources on remote host and increase hardware availability for Users wishing to make use of greater node capacity. To terminate sessions, Users can emit DELETE requests against the streaming API and supplying the unique identifier of the session to terminate:

exts/omni.services.streaming.application.example/services/streaming.py
 1# Copyright (c) 2023, NVIDIA CORPORATION.  All rights reserved.
 2#
 3# NVIDIA CORPORATION and its licensors retain all intellectual property
 4# and proprietary rights in and to this software, related documentation
 5# and any modifications thereto.  Any use, reproduction, disclosure or
 6# distribution of this software and related documentation without an express
 7# license agreement from NVIDIA CORPORATION is strictly prohibited.
 8
 9from typing import Any, Dict
10
11from omni.services.client import AsyncClient
12from omni.services.core import routers
13
14router = routers.ServiceAPIRouter()
15
16
17@router.delete(
18   "/{session_id}",
19   summary="Terminate a session",
20   description="Terminate the given unique session identifier.",
21)
22async def delete_stream(session_id: str,) -> Dict[str, Any]:
23   """
24   Send a request to terminate the given session unique identifier.
25
26   Args:
27      session_id (str): Unique identifier of the session to terminate.
28
29   Returns:
30      Dict[str, Any]: Metadata information about the session that was just terminated.
31
32   """
33   client = AsyncClient(uri="https://cockpit.devtest.az.cloud.omniverse.nvidia.com")
34   response = await getattr(client.session_service.session, session_id).delete()
35   return response

In the case of a successful request, the remote endpoint will respond with an HTTP 200 and the content "OK".