3D and 2D assets can be added to a channel either as objects of interest or to create backgrounds and other scene context. This tutorial will walk through adding a muscle car as a toy to the Rendered.ai example channel. In particular, the requirements for working with 3D assets in Blender supported Rendered.ai channels, including material properties, rigid body simulation, and the rendering engine.

For Rendered.ai channels that generate synthetic imagery using Blender, such as the Rendered.ai example channel, the data assets are persisted as a .blend file type. Check your system for the Blender Requirements, and make sure you can run Blender from the command line. Blender is an open-source 3D modeling desktop application and the .blend file properties vary depending on the version of Blender.

Use Blender version 2.90.1 to match the latest build of the Rendered.ai dev container.

In this tutorial, a car is downloaded from TurboSquid, an online service that hosts paid and free 3D models. More great resources can be found at BlenderKit and SketchFab. Regardless of where you obtain 3rd party 3D models, be sure you are following the license and copyright requirements of the content you are using when you use it with Rendered.ai, as specified in our Terms of Use.

To follow this workflow, you will first want to follow the instructions for Creating a Copy of the Example Channel and it will be helpful to understand Adding Content to a Channel

Download a 3D Model File

For simplicity, the model used here will be provided as a Blender file (.blend). For details on supported file formats for Blender 2.90 see the documentation, https://docs.blender.org/manual/en/2.90/files/import_export.html.

A quick search for free cars on TurboSquid shows many to choose from. In the screenshot below, notice the search engine supports filtering for free and Low Poly assets. Limiting the number of polygons makes the model size smaller and will speed up the deployment and testing process when the Rendered.ai example channel loads the models into the channel container.

TurboSquid Cars in Blender Format

Download the following model by logging in to TurboSquid and downloading the link with ‘blender’ in the name: https://www.turbosquid.com/FullPreview/Index.cfm/ID/1573283

TurboSquid Account View

Open the .rar file in your Downloads directory and extract the file to ana/ana/data/volumes/example. The .rar file can now be deleted. Now open Blender, and load the .blend file you saved in the local data volumes directory.

All data assets are stored in data/volumes which will be either built into the channel container or pushed to your organization’s volume storage. For more details, see Volumes and Package Data

The Downloaded Object in Blender

Prepare the Object for Use in a Package

The file will have to be modified in Blender to be used in a Rendered.ai package. These modifications are as follows:

  • Remove everything except the object, including any cameras, lights, or plane background

  • Rename the collection with a meaningful name for use in the package schema

  • Ensure only one object in the collection

  • Save the file without any spaces in the filename

In Blender, you can change names in the Outliner panel under the View Layer option. Below is an example of how it might look: the white box labeled “Muscle Car” is the collection, the orange triangles are the objects. You can change the name by double-clicking it and typing in your change.

FIX This Collection, the Rendered.ai engine does not support multiple object collections!

In this file there are multiple objects (orange triangle icons) in the collection that collectively make up the car. For our purposes, we want to collect them into a single object by doing the following:

  1. In the outliner (on the right where all the objects are listed) - select all by typing ‘a’

  2. Parent all the objects to the first one (the “Body” object in this case) by typing ctrl-p when hovering in the 3D view (Object Mode); select Object.

  3. [Optional] Rename the object to “Parent Object”

Singe Object Collection

Scale / Rotation

The Parent Object needs to be the right scale for use with other objects in the channel. To change the scale in Blender, select the object in the outliner pane, press the S key and enter a scale factor.

It is important to “Apply” scaling and rotations: Ctrl-a when in the 3D view (Object Mode)

Different 3D model formats and applications will have a variety of default object rotation. To change rotation in Blender select the object, press the R key, then press the X or Y key to specify an axis of rotation, and move your mouse until the object is oriented correctly and click to apply the rotation.

To make sure your object is facing the correct direction, in Blender press the 1 key on your number pad or go to View → Viewpoint → Front to reorient your camera and then rotate your object until its front face is pointing towards the camera (along the green line).

Rotating the Object in Blender

Rigid Body

The object placement algorithm in the example channel drops objects in a gravitational field such that they collide and interact with each other and the box object. The dropped objects need rigid body simulation properties to behave as expected.

For a great overview of Rigid Body Physics in Blender visit https://www.youtube.com/watch?v=lctjzasiy64

For channels with rigid body requirements:

  • Add the collection to the rigid body world in the Scene Properties (Cone with Bubbles)

  • Make sure Rigid Body type is Active

    • You must leave Dynamic checked

    • Do not check Animated

  • Select the Parent Object and select the Physics Properties (Blue Circle Orbit)

    • Make sure under Physics, the Rigid body is turned on (Active)

    • Check that the value for Collision under the Rigid Body panel is set to Convex Hull (Mesh works too, but takes longer to compute as it models the surface with all the vertices)

    • When turning on Rigid Body, if the object appears in a different collection, right click the new daughter object and select “Unlink”

Add the Object to the Package

Once the file is ready to use, add it to a package. In this tutorial, the object class will be added to a new Python module named ‘alternative_objects’ in a package called ‘custom_package’. As shown below, the class has an accompanying schema. These should be stored in a nodes directory to keep the package organized. Follow the file structure laid out in the image below, creating empty files to be filled with the code snippets shown lower in this section.

New Package “custom_package”

The package config file, package.yml, is used to tell the Rendered.ai engine where to find the object data as follows

volumes:
  example: volumes/example

# Define objects. Note that filenames are relative to the value passed in from "--data"
objects:

  Muscle Car:
    filename: example:LowPolyMuscleCougarxr1970.blend
CODE

 

The object class is defined in alternative_objects.py as follows

from ana.packages.common.lib.node import Node
from ana.packages.common.lib.ana_object import AnaObject
from ana.packages.common.lib.generator import get_blendfile_generator
import logging

logger = logging.getLogger(__name__)

class MuscleCarNode(Node):
    """
    A class to represent the Muscle Car node, a node that instantiates a generator for the MuscleCar object.
    """

    def exec(self):
        logger.info("Executing {}".format(self.name))
        return {"Muscle Car Generator": get_blendfile_generator("custom_package", AnaObject, "Muscle Car")}
CODE

 

With the schema defined in alternative_objects.yml as follows:

schemas:
  MuscleCarNode:
    inputs: []
    outputs:
    - name: Muscle Car Generator
      description: Object generator
    tooltip: Muscle car factory
CODE

Add the Object to the Channel

The Rendered.ai engine finds the package assets via the channel.yml file located in ana\channels\example\config. Adding an entry to the channel.yml will add the object to the channel.

Adding Muscle Car to channel.yml

Next, add an entry to the deckard.yml configuration file in the same folder.

Adding Muscle Car to deckard.yml

Channel nodes that are not specified in deckard.yml will not be categorized and will appear at the bottom of the list in the web interface as shown below.

Rendered.ai Example Channel missing ‘Skateboard’ in deckard.yml

Add Object to a Mappings File (Optional)

To generate annotations used in training a computer vision model, the object will need to be added to a mappings file. This does not affect image generation, only future use of the annotations conversions.

Add 'Muscle Car' to the default.yml file in the mappings folder

Test the New Object

Now test the code and object file by adding a node to the test graph and running the channel. In the channel directory graphs/example_test.yml add the object node and link it to the ObjectPlacement node.

version: 2
nodes:
...
  MuscleCar-0:
    nodeClass: Muscle Car

  ObjectPlacement:
    nodeClass: Random Placement
    values: {Number of Objects: 20}
    links:
      Object Generators:
      - {sourceNode: MuscleCar-0, outputPort: Muscle Car Generator}
      - {sourceNode: ColorToys, outputPort: Generator}
      - {sourceNode: "Rubik's Cube", outputPort: "Rubik's Cube Generator"}
...
CODE

Run Ana from the command line as described in the README file and open the generated file ana/output/images/0000000000-250-RGBCamera.png

cd /workspaces/ana/ana;
blender --background --python ana.py -- --channel custom_channel --graph example_test
CODE

Muscle Cars in an Generated Image

Verify Metadata

Each image has an associated annotation file and metadata file. In the screenshot below, the contents of the metadata file verify the new object is annotated.

Metadata Result Contains 'Muscle Car' Entries