GAN stands for Generative Adversarial Network, which is essentially applying game theory and putting a couple of artificial neural networks to compete with each other while they are trained at the same time. One network tries to generate the image and the other tries to detect if it is real or fake. Actually, it is something very simple, but pretty effective too. This is clearer with an image:
But again, how can we use this to accomplish our goal? It turns out that there is a kind of GAN Network named pix2pix for Data Augmentation. This kind of GAN can be used as an input, a pre-defined sketch of the real one. For example, drawing a doodle and from there build a picture like a landscape or anything you want. An example of this is the application that Nvidia did to generate artificial landscapes. The Link for the video is given here.
Ok, so maybe this can work. At that moment the label team has already labeled some images, so if we use these labels to build some doodles, then we can use this to train a GAN to generate the images. It actually works!
Now we just need to find a way to generate random doodles to feed the pix2pix GAN. Here is another GAN to the rescue, a DCGAN in this case. In this case, the idea was to generate a random doodle from random noise. Getting something like this:
And finally putting all the pieces together, with the help of some Python and Opencv code, we end up with a script that generates a 100% random image from pure noise with the corresponding labels. At the moment we can generate thousands of synthetic images with their corresponding labels in a JSON file in coco format. For the labels, we use the doodle to get labels by masking the colors and then build the synthetic images from the doodle.
Conclusions
For now, the results look promising, but they are just preliminary results and can be enhanced, for example, the labels that we use, only had labels for trees or not trees, this can be enhanced by another label to make the model more specific and accurate, like for example also label roads, fields, buildings, lakes, rivers and so on, to make the model generate this stuff.
More About Omdena
Omdena is an innovation platform for building AI solutions to real-world problems through the power of bottom-up global collaboration.
