Torch support (ResNet, Inceptions, etc.), make loadcaffe optional

[szagoruyko]

Please don't merge this yet, I will upload facebook resnets and inception models in compatible format and update readme.

would be nice if someone volunteer to find good parameters for ResNet-50 for example to put in readme.

[jcjohnson]

This looks awesome - thanks! I should have some time to play around with it next week.

[3DTOPO]

Sounds cool. I would be happy to with experiment some and report back when the models are available.

[htoyryla]

Sergey Zagoruyko [email protected] kirjoitti 13.3.2016 kello 2.12:

Please don't merge this yet, I will upload facebook resnets and inception models in compatible format and update readme.

would be nice if someone volunteer to find good parameters for ResNet-50 for example to put in readme.

I tried to run your version with CPU but got this error unknown Torch class <torch.CudaTensor> when using Resnet:

th neural_style2.lua -gpu -1 -print_iter 1 -save_iter 50 -num_iterations 2000 -model_file models/resnet-50.t7 -proto_file models/ResNet-50-deploy.prototxt -content_layers res2a -style_layers res1a,res2a,res3a,res4a

/home/hannu/torch/install/bin/luajit: /home/hannu/torch/install/share/lua/5.1/torch/File.lua:343: unknown Torch class <torch.CudaTensor>

Updating Torch does not help. Google finds Torch-related discussions about this error but no clear solution.

At the moment I don’t have a GPU installed.

Hannu

[htoyryla]

Hannu Töyrylä [email protected] kirjoitti 15.3.2016 kello 8.08:

Sergey Zagoruyko [email protected] kirjoitti 13.3.2016 kello 2.12:

Please don't merge this yet, I will upload facebook resnets and inception models in compatible format and update readme.

would be nice if someone volunteer to find good parameters for ResNet-50 for example to put in readme.

I tried to run your version with CPU but got this error unknown Torch class <torch.CudaTensor> when using Resnet:

th neural_style2.lua -gpu -1 -print_iter 1 -save_iter 50 -num_iterations 2000 -model_file models/resnet-50.t7 -proto_file models/ResNet-50-deploy.prototxt -content_layers res2a -style_layers res1a,res2a,res3a,res4a

/home/hannu/torch/install/bin/luajit: /home/hannu/torch/install/share/lua/5.1/torch/File.lua:343: unknown Torch class <torch.CudaTensor>

Updating Torch does not help. Google finds Torch-related discussions about this error but no clear solution.

Looking further into those discussions, it could depend on how the model has been saved. If that is the case, the question is where to find Resnet-models which work on CPU with torch.

I downloaded from here https://github.com/facebook/fb.resnet.torch/tree/master/pretrained

Hannu

[szagoruyko]

I converted ResNet-34 here https://gist.github.com/szagoruyko/8828e09cc4687afd324d, it should work with CPU, CUDA and OpenCL. The script to convert original fb models is also there.

[htoyryla]

Thanks. Now I had the time to figure out how to get this working. With ResNet-34, the layers names are integers, 3 to 7 being the most likely to be useful. Now I can make some tests. Appears to use less than 2GB for a 512px image.

[htoyryla]

Made a series of tests with the default images. With resnet34, I found that these parameters are a good starting point:

-content_layers 3,4
-style_layers 4,5,6
-content_weight 1e5
-style_weight 1e12

Reducing both weight proportionally results in images with reduced contrast and brightness (histogram concentrated in the middle), like here (cw = 1e1, sw = 1e7).

whereas with cw=1e6, sw=1e12

PS. While the above values worked well with the default images, trying out some different material indicates that the same values do not work with all kinds of materials. One has to adjust them depending on whether style or content dominates too much.

[htoyryla]

Looking more carefully at the images I made using resnet34, there is an obvious rasterization effect in all of them. Increasing tv_weight does not help.