How do I specify the number of top blobs allocated for a DataLayer?

[BlGene]

I am trying to implement a net which as the following structure:

I am working in DataLayer::InternalThreadEntry where a Datum is loaded from a DB, then instead of just getting one Datum from DataTransformer::Transform I transfrom the image in three different ways(a,b,c), my question is:

1. how I specify the DataLayer(i.e. data_layer.cpp) to have 3(+1)* top blobs instead of 1(+1)?
2. How do I specify the same convolutional layer(s) to run on all top blobs.

* Apparently the label is also provided as a Blob to top, so this is included in the number of top Blobs.

[shelhamer]

Why change the data layer? Define three data layers and configure the
transformation parameters of each as desired.

For convolution multiple inputs and outputs are supported in the special
case that each is the same size. If that's the case, then just list each
data layer top as a conv bottom. If each data layer top is a different
size, then declare a conv layer for each that take their respective data
layer tops as bottoms, and turn on weight sharing for the convolutions by
setting their param field to the same name like "conv_shared" or whatever
you like.

For the details of weight sharing see #546.

On Tue, Sep 9, 2014 at 4:18 PM, BlGene [email protected] wrote:

I am trying to implement a net which as the following structure:
[image: Image]
https://camo.githubusercontent.com/845f11973526c4c856f00a0793a58f0d70da020b/687474703a2f2f6935372e74696e797069632e636f6d2f6679306e65302e706e67

I am working in DataLayer::InternalThreadEntry where a Datum is loaded
from a DB, then instead of just getting one Datum from
DataTransformer::Transform I transfrom the image in three different
ways(a,b,c), my question is:

how I specify the DataLayer(i.e. data_layer.cpp) to have 3(+1)* top
blobs instead of 1(+1)?
2.

How do I specify the same convolutional layer(s) to run on all top
blobs.

• Apparently the label is also provided as a Blob to top, so this is
  included in the number of top Blobs.

—
Reply to this email directly or view it on GitHub
#1057.

[BlGene]

@shelhamer I am working with densely labeled images and I am sampling small patches out of these (see here). This is done using a DataSampler class that frequency balances categories and specifies a random patch from the training set. So I am modifying the data layer anyway. Using this design means that the position of the patch that I sample is chosen in the data layer, which means that I can't have three (independent) data layers.

I'm not sure if this is the best design. Alternativley I could move the patch location sampler out of the data layer but I am using the DataTransformer class as a sensible template to design the DataSampler.

[BlGene]

So from what I've looked at I would need to specify in the network description how many output blobs I want to have, then I would need to use this to add these blobs to the network, by modifying Net::AppendTop.

Since I load/transform the data in a prefetch thread (DataLayer::InternalThreadEntry) which writes to prefetch_data_ I need to modify this variable to hold more Blobs too.

Should I go ahead with this, or is there a better way to achieve what I want?

[shelhamer]

Seems reasonable given your needs. Please discuss further if you like on the caffe-users mailing list.

[BlGene]

Ty, should have closed issue myself.