-
Notifications
You must be signed in to change notification settings - Fork 2.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use GAN to calculate style loss #441
base: master
Are you sure you want to change the base?
Conversation
Hi, it's interesting. what are related papers of your code. I wanna take a look. |
I didn't see any paper describing this yet. However, you can look at this paper that proposed GAN. Here I use GAN to capture the distribution of gram matrices. |
I think it's better to calculate correct prediction percent of D for fakes and reals. |
The correct rates of discriminators fluctuate except the 4th style layer. the discriminator can always success on this layer. I guess this is because this layer is also used as the content layer. |
So what the limitations of your idea here? Is the image size limited to 256 like it is in CycleGAN? Do we need a thousand image dataset and pretrained CycleGAN model for each artists' style? |
Like the original style transfer, you can decide image size by using I'm not sure the lower limit of the images needed, the Shin style demonstrated there used 170 images. |
Learn from multiple styles with GAN
When using hundreds of pictures as style images, a discriminator could be used to calculate the style loss. The discriminator takes gram matrix as input and was trained to tell whether the generated image belongs to the target style.
The traditional way of calculating sytle loss:
The new way of calculating style loss:
Results
Imitate Shinkai Makoto Style
Transfered with ~160 high quality style images.
Imitate Monet(Comparing to CycleGAN)
Imitate Vangogh(Comparing to CycleGAN)
Usage
Download style image set(borrowed from CycleGAN):
bash ./datasets/download_dataset.sh <dataset name>
<dataset name>
could be monet2photo, vangogh2photo, ukiyoe2photo, cezanne2photoDo style transfer
-gan
command specifies using Discriminators to calculate style losses.d_learning_rate
is the learning rate for Discriminators.list_images.sh
helps to list all images in one directory, all files in that directory should not contain space andstyle_image_dir
should not contain~
. You need to play with parameters for different style and size.example
Transfer fj.jpg to vangogh style
bash ./datasets/download_dataset.sh vangogh2photo