Synthetic data engineering on the Rendered.ai platform begins with creating a channel. Getting started can be done by cloning Rendered.ai’s example channel and deploying it to the platform. At the end of this tutorial, you will have a channel that is owned by your Organization and available to your Workspaces at https://deckard.rendered.ai. You can use this channel to create new datasets, but more importantly you’ll be able to make changes to the channel to customize it to your liking and redeploy it.

The following sections will take you through cloning the channel, running the channel locally and finally deploying the channel to the Rendered.ai platform.

Running the Example Channel

To run the example channel in a local or cloud development environment, we need to first fetch the example channel codebase from Github.

Cloning the Example Channel

The example channel code is shared as open source on Github.

The repository includes a channel file that describes the channel, the example package that defines generator and modifier nodes used by the channel, and a .devcontainer folder that is used for the VSCode Remote Containers development environment. To learn more about the structure and architecture of this codebase, visit https://support.rendered.ai/dg/Ana-Software-Architecture.1576796161.html.

To clone the channel repository, navigate to a directory of your choosing and call the following command in the terminal:

git clone https://github.com/Rendered-ai/example.git
CODE

Opening the Development Container in VSCode

After cloning the codebase, we’ll use VSCode’s Remote Containers extension to build and run an interactive Docker container that has all the system libraries required to run the channel. If you have not setup VSCode Remote Containers before, refer to https://support.rendered.ai/dg/Setting-Up-the-Development-Environment.1576501249.html.

Open a new Window in VSCode and select Open Folder, navigate to the local directory that you had cloned the repository to and select Open. You should be greeted with a similar screen to the one shown below:

VSCode Folder Window

Once in VSCode, we can start our Development Container by pressing F1 (also CTRL+SHIFT+P or navigate to View > Command Palette…) then selecting Remote-Containers: Rebuild and Reopen in Container.

Note: the first time this is called can take awhile due to the size of the base docker image.

Once the remote container is built, the VSCode window should look similar to below. Notice the green Dev Container indicator in the bottom-left-hand corner of the window.

VSCode Development Container

Congratulations, you are now in the development container.

Running the Example Channel

To run the example channel we first need to mount the necessary volumes, cloud-hosted data repositories for content that will be used when running the channel. Rendered.ai provides the anamount utility to make this easy.

The anamount command requires Rendered.ai platform credentials. If you do not have an account, register for an account at http://rendered.ai . If you are still having issues accessing the volume, contact admin@rendered.ai for help.

The code block below shows how to mount the example volume using the terminal:

(anatools) anadev@test:/workspaces/example$ anamount
Using channelfile found at ./example.yml.
If this is the wrong channel, specify a channelfile using the --channel argument.
Enter your credentials for the Rendered.ai Platform.
Email: test@rendered.ai
Password: ************
Mounting volume df8ad806-223b-4d56-a932-838da835ec62...complete!
Remounting volumes in 3420s...
CODE

You may notice that a new directory called data has been added to your channel directory. In this directory we find a volumes/df8ad806-223b-4d56-a932-838da835ec62 directory that has a few files in it.

Mounted volume

These files are static assets that the example package uses when generating the scenes and are necessary run the example channel successfully. Once we have successfully mounted the necessary volumes we can run the channel.

Note that the anamount process is required to stay open to keep the volume mounted. You’ll need to start a new terminal for executing the ana command.

To run the channel we use the ana command.

(anatools) anadev@test:/workspaces/example$ ana --channel example --graph graphs/default.yml
BASH


This command will generate a new run of the default graph using the example channel. The output of the new run can be found in the /output directory.

Running ana

The example channel will produce a new file in the annotations, images, masks and metadata directories on each run. Running a channel multiple times will produce a dataset.

Now that we have successfully run the Example channel, let’s deploy it to the platform.

Deploying the Example Channel

To deploy the example channel to the platform we need to run the anadeploy command. This will create a new remote channel entry on the platform and build and deploy a docker image to the platform for the channel. After this process, we will be able to user our new channel in the Rendered.ai platform web interface to create graphs and datasets.

Follow the prompts in the code block below to create a new channel in your Organization named custom_channel.

(anatools) anadev@test:/workspaces/example$ anadeploy                    
Using channelfile found at ./example.yml.
If this is the wrong channel, specify a channelfile using the --channel argument.
Enter your credentials for the Rendered.ai Development Platform.
Email: test@rendered.ai
Password: ********
Please select one of the following options:
        [0]   Create a new managed channel.
Enter your choice: 0
Select an organization to create a new managed channel in:
        [0]   default
Enter your choice: 0
Creating a new channel using the ./example.yml channel file in default organization. Continue (y/n)? y
Please provide a name for the channel or leave blank for 'example':  new_channel
Step: 6/6
State: Channel Deployment Complete
Message: The channel has been fully deployed and is ready for use.
BASH

After the deployment is complete, we can navigate to the platform website to add our channel to a Workspace.

Using the new Example Channel

Navigate to https://deckard.rendered.ai and login. Once signed in, we can view the channels available to our Organization by navigating to the Organizations page in the top-right under the user icon. On the Organizations page, navigate to the Channels tab.

Default Organization Channels

We can see that we have a new channel deployed called custom_channel that is owned by our Organization. Now to use the channel we need to add it to a Workspace. Navigate to the Workspaces tab and select a Workspace.

Adding Channel to Workspace

In the three-button ellipsis, we can add channels to our Workspace by selecting Channels and checking the box next to the channel name.

Example Workspace

Adding Channels to workspace

Creating a new Graph

To use this channel we need to create a new graph using the nodes from the channel. Select New Graph on the Workspace page and fill in the Name and channel fields. In this case we are creating a new graph called test using the custom_channel channel.

Select Create, this will create the new graph and navigate you to it. Once inside you’ll notice that there are no nodes in the graph yet – this is because all new channels need to have their default graphs set. So first we’ll build the graph we want using the nodes from the expanded panel on the left then use Preview to make sure it works.

Channel Nodes

Graph

Preview

Set the Channel’s Default Graph

We can set this graph as the default graph by using the anatools SDK. If we go back into our Dev Container in VSCode, we can create a new terminal and use the interactive python to set the default graph for the channel.

We need three parameters for setting the default graph for a channel.

  • channelId - this can be retrieved from get_managed_channels or the GUI.

  • workspaceId - the workspace where the graph exists, can be retrieved from GUI url.

  • graphId - the graph that you’d like to set as the default graph, can be retrieved from GUI url.

(anatools) anadev@test:/workspaces/example$ python
Python 3.7.7 (default, May  7 2020, 21:25:33) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import anatools
>>> client = anatools.client()
Enter your credentials for the Rendered.ai  Platform.
Email: test@rendered.ai
Password: ********
These are your organizations and workspaces:          
    default Organization                          fca0ef6b-1863-42f4-a2dc-ddd6c248dc95              
        Example                                   6cc85813-844c-4a98-9977-c74c047db3b4              
Signed into Rendered.ai Development Platform with ethan@rendered.ai
The current workspace is: 6cc85813-844c-4a98-9977-c74c047db3b4
>>> client.get_managed_channels()
[{'channelId': '517c1c59-ee31-4831-8fd0-d9a02f6baf80', 'organizationId': 'fca0ef6b-1863-42f4-a2dc-ddd6c248dc95', 'name': 'custom_channel', 'instanceType': 'p2.xlarge', 'volumes': ['3cabcc67-a398-4bee-aa1d-ecf67a72760f'], 'timeout': 120, 'interfaceVersion': 1, 'createdAt': '2021-12-16T00:13:17.027Z', 'updatedAt': '2022-05-19T22:57:40.279Z', 'organizations': [{'organizationId': 'fca0ef6b-1863-42f4-a2dc-ddd6c248dc95', 'name': 'default'}]}]
>>> client.set_default_graph(channelId='517c1c59-ee31-4831-8fd0-d9a02f6baf80',workspaceId='6cc85813-844c-4a98-9977-c74c047db3b4', graphId='8Hj5xIfo56FgZ6JVmuif')
True
>>> 
CODE

By creating a new graph from the Workspace page in the GUI, we’ll see that the nodes are now in place.

Create new test graph

New graph with default nodes

Summary

At this point, you have successfully cloned the example channel, deployed it to your organization as a new channel, and created a graph that can be used to run jobs and as a starting point for more graphs.