Domain Adaptation

Domain adaptation is a technique used to bridge the gap between real and synthetic domains when generating synthetic datasets. Because our imagery is computer generated, it may lack some of the characteristics that make-up real imagery. We use Generative Adversarial Networks or GANs as a tool to help us bridge that gap by training our GAN on real imagery then applying that GAN to the synthetic dataset. At Rendered.ai we use CycleGAN to accomplish this. CycleGAN is an image-to-image translation project shared at ICCV 2017. References for training a CycleGAN model can be found on the project site.

This tutorial assumes that the user has a PyTorch model file (.pth) from training a CycleGAN. The tutorial can be completed using the below example file. This model was trained using the XView dataset, it requires --netG resnet_3blocks --no_dropout flags to be set during upload.

xview.pth

Creating a GAN Model

To create a GAN Model on the platform, navigate to your organization’s GAN Models tab on the Landing page. Click on the New GAN Model button to create a new GAN Model.

Next a dialog will appear where it will ask you to fill in the following fields: name, file, flags and description. The name is required and the description is optional. The file is the PyTorch model file that is going to be uploaded.

The flags field is used to run inference with the GAN exactly as it is trained. For example, if you trained the GAN with the --no_dropout flag, you’d need to set the flags to “--no_dropout“ for the service to run appropriately. Our backend automatically sets the following flags based on the input dataset, so they are not required to pass in as part of the flags input: dataroot, results_dir, name, model, load_size, crop_size, num_test.

Clicking the Create button will upload the model file and create a new GAN Model entry in the organization.

Next we’ll need to add the GAN into a workspace before using it with the GAN service.

Using a GAN Model

The first thing we’ll need to do is go to our Workspace, click the three-dot icon and select Resources.

This will bring us to a dialog where we can add and remove different resources to the Workspace. We’ll want to add the xview GAN Model to the Included column of the dialog then click the Save button.

Finally, we can navigate to the Datasets Library page and select a dataset that we want to domain shift. Next to the GAN Datasets section of that dataset, we’ll click the + icon to create a new GAN Dataset.

Here we can give the dataset a new name and description, then select which GAN Model we want to run the dataset against. Clicking the create button will start a new job to domain shift the dataset using the CycleGAN model.

After kicking off the new GAN Dataset job we will see the time dial icon indicating that the job is running. As a reminder, the symbols used for these services are indicated below:

	No symbol means that the service job is complete and ready to use.
	The sand dial symbol means that the job is running. It will remain this way until the job has either completed or failed.
	The error symbol means that the job has an issue. You can click on the symbol to fetch a log of the service to help determine what caused the issue.

When the GAN Dataset job has finished, the symbol is removed and we can go-to, download or delete the GAN Dataset. The below screenshot is a look at our new GAN Dataset in the Datasets Library.