Matcaffe2

[sguada]

Inspired by PyCaffe, a new and completely renovated MatCaffe interface (although backward compatible).

Now there is class CaffeNet that handle all the interactions with caffe.

New Features:

• net = CaffeNet.instance gets a instance of caffe network using default imagenet_deploy and caffe_reference_model
• net = CaffeNet.instance(model_def_file, model_file) gets an instance of caffe network using the parameters specified.

net = 
  CaffeNet with properties:

    model_def_file: '../../examples/imagenet/imagenet_deploy.prototxt'
        model_file: '../../examples/imagenet/caffe_reference_imagenet_model'
       layers_info: [23x1 struct]
        blobs_info: [15x1 struct]
              mode: 'CPU'
             phase: 'TRAIN'
         device_id: 0
       input_blobs: []
      output_blobs: []
           weights: [8x1 struct]

• Now one can set mode, phase or device_id, just by assigning the properties or using the corresponding methods.
  net.phase = 'TEST'; net.set_phase_test; net.set_phase('TEST');
net.mode = 'GPU'; net.set_mode_gpu; net_set_mode('GPU');
net.device_id = 1;net.set_device(1);
• One can access the weights of the network just by

net.weights   
8x1 struct array with fields:

    layer_name
    weights

• More amazingly can change the weights of the network by just assigning a new set of weights, for example changing the bias of the first layer conv1.

new_weights = net.weights;
new_weights(1)                    
    layer_name: 'conv1'
       weights: {2x1 cell}
new_weights(1).weights{2}(:) = 0; 
net.weights = new_weights;

• Then one can save the new network in a different model_file

net.save_net('path_to/new_model')

• And load it later

net.load_net('path_to/new_model')

More examples about new features to follow.

[sguada]

@rbgirshick Could take a look a give me your opinion?

[rbgirshick]

This sounds like a very welcome update! I'll review the code this weekend
and give you feedback.

On Fri, Jun 13, 2014 at 7:52 PM, Sergio Guadarrama <[email protected]

wrote:

@rbgirshick https://github.com/rbgirshick Could take a look a give me
your opinion?

—
Reply to this email directly or view it on GitHub
#501 (comment).

http://www.cs.berkeley.edu/~rbg/

[sguada]

This PR includes #502 to allow compute gradients during TEST phase, otherwise the gradients change every time they are computed since dropout changes during TRAIN.

[moskewcz]

[ i only skimmed the diff, so apologies if i misread it. ]

i can't really comment on if caffe wants/needs a classdef style interface wrapper as i'm not really a matlab user. i guess the syntax/usage is nicer than the 'raw' mex-file interface? i guess there's also still really only one global net object behind the mex interface? i dunno if matlab has a way/idiom to actually have c++ objects (i.e. a way to set/get a void * + ownership/lifetime hooks) encapsulated inside matlab objects such that multiple net objects could be supported.

while the current matlab interface (i.e. in dev) is octave compatible, this is not, since octave doesn't support classdef. while octave could still probably use the lower-level interface of the defined by matcaffe.cpp, any examples/code that used the classdef layer would become incompatible.

so ...

1. should anyone ever use octave? probably out of scope here, but a fair question: why not just use python if you don't like matlab? a couple possible reasons: 1a) existing code and 1b) some people like the matlab programming model / toolkits. i'm not too knowledgeable/opinionated on this topic.

2. will octave ever support classdef? matlab compatibility is a goal for octave, and they have done at least some work towards classdef support. on the whole, octave seems to have improving matlab compatibility (with serious, notable exceptions), but matlab is a moving target. but there's no timeline for support and classdef is a complex feature.

3. does it matter if caffe or other projects are octave-friendly? it does to me, but mainly as a developer rather than a user. there's also the general principle that i think it would be nice if (long term / globally) there was a free software option for [caffe+]matlab style development -- at least as a way to help transition away from using matlab if not as a final solution. independence from matlab can be important in both commercial and academic settings for various reasons: financial, legal, philosophical, for debugging/development, etc.

there are costs/benefit of various levels of octave compatibility, but classdef support/usage is currently a thorny issue ...

for the record, i think my status on the three questions above is:

yes/yes/yes

but, i'm expecting some no/no/no replies ... ;)

[rbgirshick]

Sorry, I still haven't gotten to reviewing this PR. I'm still swamped with
CVPR prep. However, I have some comments on the issues that @moskewcz
raised.

I would like to rewrite matcaffe to remove The One Global Net. That should
be easy. We just need to maintain a vector of nets which are referenced by
an integer handle. Each call to caffe(...) passes in the desired handle. We
could then remove the silly "init key" (that I added). However, this will
break the current matcaffe API. I think we'll likely want the changes in
the current PR even if we change the API more dramatically down the road.

Regarding the three questions:

I'm not personally concerned with Octave compatibility. I don't know any
vision researchers who use Octave on a regular basis. If I were to pick an
open source alternative to matlab part of the point, for me, would be to
use a better programming language. In that case octave would be at the
bottom of my list. I'd use Python or Julia.

Ross

On Wed, Jun 18, 2014 at 10:35 AM, moskewcz [email protected] wrote:

[ i only skimmed the diff, so apologies if i misread it. ]

i can't really comment on if caffe wants/needs a classdef style interface
wrapper as i'm not really a matlab user. i guess the syntax/usage is nicer
than the 'raw' mex-file interface? i guess there's also still really only
one global net object behind the mex interface? i dunno if matlab has a
way/idiom to actually have c++ objects (i.e. a way to set/get a void * +
ownership/lifetime hooks) encapsulated inside matlab objects such that
multiple net objects could be supported.

while the current matlab interface (i.e. in dev) is octave compatible,
this is not, since octave doesn't support classdef. while octave could
still probably use the lower-level interface of the defined by
matcaffe.cpp, any examples/code that used the classdef layer would become
incompatible.

so ...

1. should anyone ever use octave? probably out of scope here, but a fair
   question: why not just use python if you don't like matlab? a couple
   possible reasons: 1a) existing code and 1b) some people like the matlab
   programming model / toolkits. i'm not too knowledgeable/opinionated on this
   topic.

2. will octave ever support classdef? matlab compatibility is a goal for
   octave, and they have done at least some work towards classdef support. on
   the whole, octave seems to have improving matlab compatibility (with
   serious, notable exceptions), but matlab is a moving target. but there's no
   timeline for support and classdef is a complex feature.

3. does it matter if caffe or other projects are octave-friendly? it does
   to me, but mainly as a developer rather than a user. there's also the
   general principle that i think it would be nice if (long term / globally)
   there was a free software option for [caffe+]matlab style development -- at
   least as a way to help transition away from using matlab if not as a final
   solution. independence from matlab can be important in both commercial and
   academic settings for various reasons: financial, legal, philosophical, for
   debugging/development, etc.

there are costs/benefit of various levels of octave compatibility, but
classdef support/usage is currently a thorny issue ...

for the record, i think my status on the three questions above is:

yes/yes/yes

but, i'm expecting some no/no/no replies ... ;)

—
Reply to this email directly or view it on GitHub
#501 (comment).

http://www.cs.berkeley.edu/~rbg/

[sguada]

@moskewcz If you look at the code you would see that it is mostly syntactic sugar to have a clear interface and design, and a little bit of lazy evaluation. So any Octave user could easily write a OctaveCaffe.m file that simply encapsulate the calls to caffe.mex

See https://github.com/BVLC/caffe/pull/501/files#diff-074ccb45539e7635cc6a871374289d7aR112
If you need help creating that code I can also create a template for Octave.

Regarding having multiple nets, I thought about it and have a design to maintain backward compatibility, but put it on hold until implementing the Solver which needs multiple nets. As @rbgirshick said, just by using a std::vector<Net> where the default Net is stored in the first position, and the init_key is the position in that vector.

However I could include the multiple nets in this PR.

[moskewcz]

@ross: okay, i see how int handles-to-nets could work. i was hoping for something that would allow for lifetime management and/or ownership transfer (i.e. like shared_ptr+boost::python, PyCapusle, or Lua's userdata).
@sguada: right, that makes sense. i was figuring octave could/would just use continue to use the mex-function directly, but i guess there may be some alternative to classdef that yields a similar effect and works in octave and can handle whatever logic/functionality is part of classdef-based wrapper. i read that the 'old style' pre-2008a matlab class system does work in octave, but i don't know much about it. i've seen some people use subfunction references as a third method of semi-OOP in matlab, but i don't think that's well supported in octave either :/ ...

[ross]

Think you have the wrong ross

[moskewcz]

yeah, oops. sorry @ross, github noob error. thanks for dropping by? ;)

On Wed, Jun 18, 2014 at 1:55 PM, Ross McFarland [email protected]
wrote:

Think you have the wrong ross

—
Reply to this email directly or view it on GitHub
#501 (comment).

[rbgirshick]

Is this change something you want merged or did you just add it for your own debugging? If the later, please remove from the PR.

[rbgirshick]

@sguada here are some high level comments. More detailed comments are
inline in the PR.

-- I think that matlab/caffe/*.m files should be prefixed with either
caffe_ or matcaffe_ in order to avoid polluting people's environments
with generic sounding function names (e.g., displayData). Although
matcaffe_ was originally used for the demo code, I personally prefer
caffe_.

-- It would be great to add some documentation to CaffeNet.m

-- This might be for a separate PR, but it would be great to add unit tests
for the matcaffe wrapper.

On Wed, Jun 18, 2014 at 2:08 PM, moskewcz [email protected] wrote:

yeah, oops. sorry @ross, github noob error. thanks for dropping by? ;)

On Wed, Jun 18, 2014 at 1:55 PM, Ross McFarland [email protected]

wrote:

Think you have the wrong ross

—
Reply to this email directly or view it on GitHub
#501 (comment).

—
Reply to this email directly or view it on GitHub
#501 (comment).

http://www.cs.berkeley.edu/~rbg/

[shelhamer]

Suggestion #657 (comment) to check inputs and complain gracefully instead of crashing.

[shelhamer]

A new release is brewing soon. Is there any hope of finishing this up so that matcaffe is on even footing with pycaffe?

[sguada]

@jeffdonahue rebased and ready for review :)

[bharath272]

Was looking at this since I tend to use matcaffe pretty heavily. Thanks a lot for this!
I was testing out the matcaffe_demo_backward, and found out a couple of minor things:

1. The demo, and the default constructor for caffeNet, assumes the imagenet_deploy prototxt and imagenet model are already there. It might be good to mention this in the documentation in the demo...
2. Also, if the prototxt doesn't have force_backward: true, the backward pass doesn't happen and backward gives 0 gradients. Another thing to mention in the demo documentation perhaps?

[bharath272]

I think caffe_safe('save_net',...) shouldn't return anything, since the corresponding matcaffe function doesn't return anything.

[shelhamer]

Once @jeffdonahue's and @bharath272's comments are addressed let's squash this all and merge it. If we missed the summer of matcaffe 2 we should certainly have the fall.

[shelhamer]

@sguada this is so close!

[sharathchandra92]

@sguada @shelhamer Hi, Is there any way to extract the fc7,fc6 feature vectors from pretrained models using the current matcaffe in master? I see that the hack mentioned in #299 is no longer working with the current build. Could you suggest an alternative?
Thanks :)

[shelhamer]

New plan: @bharath272 and @s-gupta have volunteered to take the last steps to see Matcaffe 2 to completion. This'll be carried out by #1913.