Is AI able to remove mosaic?

Hello, everyone.

Have you ever thought about using AI technology to remove mosaics?

It’s quite difficult to think about this problem carefully, because the AI ​​technology we used before, whether it is face recognition or OCR recognition, can at least be recognized manually. But if you are given a mosaic picture, can you restore it?

Obviously it’s difficult. If humans can't recover, how can we teach computers to recover?

Do you still remember the article "Using AI to Generate Avatars" I wrote a few days ago. In that article, we trained a DCGAN model that can generate an image from arbitrary random numbers.

Noise map generated with random numbers as pixels

Model generates normal avatar from random numbers

DCGAN​ consists of two models: a generator model and a discriminator model. The function of the generator model is to generate a picture from a set of random numbers. The closer the picture is to the training sample, the better, thereby fooling the discriminator model. The discriminator model needs to continuously improve its discrimination ability to prevent the pictures generated by the generator model from getting passed.

If we change the input of the above generator model from random numbers to pictures with mosaics, the output will be pictures without mosaics. Is it possible to train a model that removes mosaic?

Next, I will share with you how to train the mosaic removal model, and then share a ready-made tool that you can download and use directly to try the effect.

1. Pix2pix CycleGAN

Here we do not use the DCGAN introduced above, but use two more powerful models, Pix2pix and CycleGAN, to train separately.

Pix2pix​ is an image translation algorithm based on GAN. From mosaic pictures to normal pictures, it is essentially similar to the conversion from one language to another. It is a translation process.

Pix2pix model translation

The effect of CycleGAN’s implementation is simply to convert images between different domains while maintaining its shape. constant.

CycleGAN model

The article provides us with a data set and a complete training process, which can help us train the model at a lower cost.

First, download the data set

Data set

a total of 654M.

Then, download the Paddle pre-training model

Pre-training model

Finally, train the Pix2pix and CycleGAN models respectively.

Pix2pix model

python gan<span style="color: rgb(215, 58, 73); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">/</span>infer<span style="color: rgb(0, 92, 197); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">.py</span> <br><span style="color: rgb(106, 115, 125); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">--init_model output/pix2pix/checkpoints/110/ </span><br><span style="color: rgb(106, 115, 125); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">--dataset_dir /home/aistudio/ </span><br><span style="color: rgb(106, 115, 125); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">--image_size 256 </span><br><span style="color: rgb(106, 115, 125); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">--n_samples 1 </span><br><span style="color: rgb(106, 115, 125); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">--crop_size 256 </span><br><span style="color: rgb(106, 115, 125); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">--model_net Pix2pix </span><br><span style="color: rgb(106, 115, 125); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">--net_G unet_256 </span><br><span style="color: rgb(106, 115, 125); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">--test_list /home/aistudio/test_list.txt </span><br><span style="color: rgb(106, 115, 125); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">--output ./infer_result/pix2pix/</span>

CycleGAN model

python gan<span style="color: rgb(215, 58, 73); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">/</span>infer<span style="color: rgb(0, 92, 197); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">.py</span> <br><span style="color: rgb(106, 115, 125); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">--init_model output/cyclegan/checkpoints/48/ </span><br><span style="color: rgb(106, 115, 125); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">--dataset_dir /home/aistudio/ </span><br><span style="color: rgb(106, 115, 125); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">--image_size 256 </span><br><span style="color: rgb(106, 115, 125); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">--n_samples 1 </span><br><span style="color: rgb(106, 115, 125); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">--crop_size 256 </span><br><span style="color: rgb(106, 115, 125); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">--input_style A </span><br><span style="color: rgb(106, 115, 125); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">--test_list /home/aistudio/test_list.txt </span><br><span style="color: rgb(106, 115, 125); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">--model_net CycleGAN </span><br><span style="color: rgb(106, 115, 125); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">--net_G resnet_9block </span><br><span style="color: rgb(106, 115, 125); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">--g_base_dims 32 </span><br><span style="color: rgb(106, 115, 125); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">--output ./infer_result/cyclegan/</span>

After training, you can run the gan/infer.py file, Compare the mosaic removal effects of these two models.

2. Ready-made tools

If you don’t want to train the model yourself, here is a ready-made project to share with you. It is also based on semantic segmentation and image translation, with reference to Pix2pix and CycleGAN.

Project address: https://github.com/HypoX64/DeepMosaics/blob/master/README_CN.md

For Windows users, the author provides an installation-free software package containing a GUI interface.

UI interface

We said before that this technology is still relatively difficult, so don’t have too high expectations. Here is the actual removal effect.

Coding

Decoding

The effect is still good, but it is not as perfect as imagined. You can download and run it to try.

The above is the detailed content of Is AI able to remove mosaic?. For more information, please follow other related articles on the PHP Chinese website!