TransWikia.com

Improving the pix2pix Architecture for Sketch to Image Translation on a Dataset of Sketches of People to Photos of People

Data Science Asked on January 5, 2021

For a university project I need to create a neural network which translates sketches of people into images.
In order to implement such a neural network, I decided to implement a pix2pix GAN architecture.
The neural network is trained an evaluated on a modified version of the CUFS dataset provided by my professors.

While the neural network is able to perform image transalations, as shown in the image pair below, I wonder if there are ways to improve the quality of the results?
In particular I was wondering if there are ways make the image look sharper?

Sample sketch to image translation

I am already augmenting the relatively small training dataset of 70 sketch-photo pairs to 770 ones.
Furthermore, I tried to decrease the value of the $lambda$ parameter.
Regarding the epoch, I figured out that the validation loss does not drop significantly after 160 iterations anymore.

I implemented the network in a Jupyter notebook hosted on Google Colab.
As you can see in the notebook’s last cell, the currently achieved L1 loss on the testing dataset is about 0.191 on average.
Here is a link to the dataset used for training and testing.

Please feel free to take a look at the code and tell me ways on how to improve the code.
In addition, I would be most grateful if you could provide additional ways on how to improve the generated image quality.

One Answer

I managed to obtain better results with less artifacts by implementing the code of the paper "High-Quality Facial Photo-Sketch Synthesis Using Multi-Adversarial Networks".

I found this code to be quite nice working for my purposes. If you need, I can also provide a Colab notebook with my ported code.

Answered by Lukas on January 5, 2021

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP