Run and Deploy the Toybox Channel

Synthetic data engineering on the Rendered.ai platform begins with creating a channel. Getting started can be done by cloning Rendered.ai’s toybox channel and deploying it to the platform. At the end of this tutorial, you will have a channel that is owned by your Organization and available to your Workspaces at https://deckard.rendered.ai. You can use this channel to create new datasets, but more importantly you’ll be able to make changes to the channel to customize it to your liking so that you can generate the synthetic datasets you need.

The following sections will take you through cloning the channel, running the channel locally and finally deploying the channel to the Rendered.ai platform.

• Running the Toybox Channel

  • Cloning the Toybox Channel

  • Opening the Development Container in VSCode

  • Running the Toybox Channel

• Deploying the Toybox Channel

• Using the new Toybox Channel

  • Adding Channel to Workspace

  • Creating a new Graph

  • Set the Channel’s Default Graph

  • Summary

Running the Toybox Channel

To run the toybox channel in a local or cloud development environment, we need to first fetch the toybox channel codebase from Github.

Cloning the Toybox Channel

The toybox channel code is shared as open source on Github.

Github Repository: Rendered-ai/toybox

The repository includes a channel file that describes the channel, the toybox package that defines generator and modifier nodes used by the channel, and a .devcontainer folder that is used for the VSCode Remote Containers development environment. To learn more about the structure and architecture of this codebase, visit Ana Software Architecture.

To clone the channel repository, navigate to a directory of your choosing and call the following command in the terminal:

git clone https://github.com/Rendered-ai/toybox.git

Opening the Development Container in VSCode

After cloning the codebase, we’ll use VSCode’s Remote Containers extension to build and run an interactive Docker container that has all the system libraries required to run the channel. If you have not setup VSCode Remote Containers before, refer to Setting Up the Development Environment.

Open a new Window in VSCode and select Open Folder, navigate to the local directory that you had cloned the repository to and select Open. You should be greeted with a similar screen to the one shown below:

VSCode Folder Window

Once in VSCode, we can start our Development Container by pressing F1 (also CTRL+SHIFT+P or navigate to View > Command Palette…) then selecting Remote-Containers: Rebuild and Reopen in Container.

Note: the first time this is called can take awhile due to the size of the base docker image.

Once the remote container is built, the VSCode window should look similar to below. Notice the green Dev Container indicator in the bottom-left-hand corner of the window.

VSCode Development Container

Congratulations, you are now in the development container.

Running the Toybox Channel

To run the toybox channel we first need to mount the necessary volumes, cloud-hosted data repositories for content that will be used when running the channel. Rendered.ai provides the anamount utility to make this easy.

The anamount command requires Rendered.ai platform credentials. If you do not have an account, register for an account at rendered.ai . If you are still having issues accessing the volume, contact [e-mail] for help.

The code block below shows how to mount the toybox volume using the terminal:

(anatools) anadev@test:/workspaces/toybox$ anamount
Using channelfile found at ./toybox.yml.
If this is the wrong channel, specify a channelfile using the --channel argument.
Enter your credentials for the Rendered.ai Platform.
Email: [e-mail]
Password: ************
Mounting volume e66b164e-8796-48aa-8597-636d85bec240...complete!
Remounting volumes in 3420s...

You may notice that a new directory called data has been added to your channel directory. In this directory we find a volumes/e66b164e-8796-48aa-8597-636d85bec240 directory that has a few files in it.

Mounted Volume

These files are static assets that the toybox package uses when generating the scenes and are necessary run the toybox channel successfully. Once we have successfully mounted the necessary volumes we can run the channel.

Note that the anamount process is required to stay open to keep the volume mounted. You’ll need to start a new terminal for executing the ana command.

To run the channel we use the ana command.

(anatools) anadev@test:/workspaces/toybox$ ana --channel toybox --graph graphs/default.yml

This command will generate a new run of the default graph using the toybox channel. The output of the new run can be found in the /output directory.

Running ana

The toybox channel will produce a new file in the annotations, images, masks and metadata directories on each run. Running a channel multiple times will produce a dataset.

Now that we have successfully run the toybox channel, let’s deploy it to the platform.

Deploying the Toybox Channel

To deploy the toybox channel to the platform we need to run the anadeploy command. This will create a new remote channel entry on the platform and build and deploy a docker image to the platform for the channel. After this process, we will be able to user our new channel in the Rendered.ai platform web interface to create graphs and datasets.

Follow the prompts in the code block below to create a new channel in your Organization named custom_channel.

(anatools) anadev@test:/workspaces/toybox$ anadeploy                    
Using channelfile found at ./toybox.yml.
If this is the wrong channel, specify a channelfile using the --channel argument.
Enter your credentials for the Rendered.ai Development Platform.
Email: [e-mail]
Password: ********
Please select one of the following options:
        [0]   Create a new managed channel.
Enter your choice: 0
Select an organization to create a new managed channel in:
        [0]   default
Enter your choice: 0
Creating a new channel using the ./toybox.yml channel file in default organization. Continue (y/n)? y
Please provide a name for the channel or leave blank for 'toybox':  new_channel
Step: 6/6
State: Channel Deployment Complete
Message: The channel has been fully deployed and is ready for use.

After the deployment is complete, we can navigate to the platform website to add our channel to a Workspace.

Using the new Toybox Channel

Navigate to https://deckard.rendered.ai and login. Once signed in, we can view the channels available to our Organization by navigating to the Organizations page in the top-right under the user icon. On the Organizations page, navigate to the Channels tab.

Organization Channels

We can see that we have a new channel deployed called custom_channel that is owned by our Organization. Now to use the channel we need to add it to a Workspace. Navigate to the Workspaces tab and select a Workspace.

Adding Channel to Workspace

In the three-button ellipsis, we can add channels to our Workspace by first clicking on the Resources tab.

Workspace Resources

Then in the Resources pop-up, click on the Channels tab, select the channel you want to add, click Add, and then click Save.

Adding Channels to a Workspace

Creating a new Graph

To use this channel we need to create a new graph using the nodes from the channel. Select New Graph on the Workspace page and fill in the Name and channel fields. In this case we are creating a new graph called test using the custom_channel channel.

Select Create, this will create the new graph and navigate you to it. Once inside you’ll notice that there are no nodes in the graph yet – this is because all new channels need to have their default graphs set. So first we’ll build the graph we want using the nodes from the expanded panel on the left then use Preview to make sure it works.

Channel Nodes

Toybox Graph

Preview

Set the Channel’s Default Graph

We can set this graph as the default graph by using the anatools SDK. If we go back into our Dev Container in VSCode, we can create a new terminal and use the interactive python to set the default graph for the channel.

We need three parameters for setting the default graph for a channel.

• channelId - this can be retrieved from get_managed_channels or the GUI.

• workspaceId - the workspace where the graph exists, can be retrieved from GUI url.

• graphId - the graph that you’d like to set as the default graph, can be retrieved from GUI url.

(anatools) anadev@test:/workspaces/toybox$ python
Python 3.7.7 (default, May  7 2020, 21:25:33) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import anatools
>>> client = anatools.client()
Enter your credentials for the Rendered.ai  Platform.
Email: [e-mail]
Password: ********
These are your organizations and workspaces:          
    default Organization                          fca0ef6b-1863-42f4-a2dc-ddd6c248dc95              
        Toybox                                    6cc85813-844c-4a98-9977-c74c047db3b4              
Signed into Rendered.ai Development Platform with [e-mail]
The current workspace is: 6cc85813-844c-4a98-9977-c74c047db3b4
>>> client.get_managed_channels()
[{'channelId': '517c1c59-ee31-4831-8fd0-d9a02f6baf80', 'organizationId': 'fca0ef6b-1863-42f4-a2dc-ddd6c248dc95', 'name': 'custom_channel', 'instanceType': 'p2.xlarge', 'volumes': ['3cabcc67-a398-4bee-aa1d-ecf67a72760f'], 'timeout': 120, 'interfaceVersion': 1, 'createdAt': '2021-12-16T00:13:17.027Z', 'updatedAt': '2022-05-19T22:57:40.279Z', 'organizations': [{'organizationId': 'fca0ef6b-1863-42f4-a2dc-ddd6c248dc95', 'name': 'default'}]}]
>>> client.set_default_graph(channelId='517c1c59-ee31-4831-8fd0-d9a02f6baf80',workspaceId='6cc85813-844c-4a98-9977-c74c047db3b4', graphId='8Hj5xIfo56FgZ6JVmuif')
True
>>> 

By creating a new graph from the Workspace page in the GUI, we’ll see that the nodes are now in place.

Summary

At this point, you have successfully cloned the toybox channel, deployed it to your organization as a new channel, and created a graph that can be used to run jobs and as a starting point for more graphs.