Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conv layer: fix crash by setting weight pointer #1048

Merged
merged 2 commits into from
Sep 8, 2014

Conversation

jyegerlehner
Copy link
Contributor

The problem is that in ConvolutionLayer::Backward_[gpu,cpu], the pointer "weight" is set only if the this->param_propagate_down_[0] is true, otherwise it is left NULL. However "weight" is used if either this->param_propagate_down_[0] or propagate_down[i] is true. So in the case where the first bool is false but the second is true, we end up adding weight offset to a NULL "weight". This caused a seg fault running on a CPU and cuda error 13 running on GPU.

I hit this in a scenario where I was doing layerwise training of a convolutional autoencoder. The learning rate for the already-trained layers was set to zero, and the newly-added layers had non-zero learning rate.

@jeffdonahue
Copy link
Contributor

Eep, nice catch, and thanks for the fix!

jeffdonahue added a commit that referenced this pull request Sep 8, 2014
Conv layer: fix crash by setting weight pointer
@jeffdonahue jeffdonahue merged commit 5ab3d97 into BVLC:dev Sep 8, 2014
@jyegerlehner jyegerlehner deleted the conv_layer-init-weight branch September 8, 2014 06:28
mitmul pushed a commit to mitmul/caffe that referenced this pull request Sep 30, 2014
Conv layer: fix crash by setting weight pointer
shelhamer added a commit that referenced this pull request Oct 1, 2014
fix NULL pointer when back-propagating to the data for fixed i.e. not
back-propagated weights. same issue as #1048
@shelhamer
Copy link
Member

Fixed for CuDNNConvolutionLayer in a02ec65.

mitmul pushed a commit to mitmul/caffe that referenced this pull request Oct 11, 2014
fix NULL pointer when back-propagating to the data for fixed i.e. not
back-propagated weights. same issue as BVLC#1048
RazvanRanca pushed a commit to RazvanRanca/caffe that referenced this pull request Nov 4, 2014
Conv layer: fix crash by setting weight pointer
RazvanRanca pushed a commit to RazvanRanca/caffe that referenced this pull request Nov 4, 2014
fix NULL pointer when back-propagating to the data for fixed i.e. not
back-propagated weights. same issue as BVLC#1048
cbfinn pushed a commit to cbfinn/caffe that referenced this pull request Feb 26, 2015
fix NULL pointer when back-propagating to the data for fixed i.e. not
back-propagated weights. same issue as BVLC#1048
slayton58 pushed a commit to slayton58/caffe that referenced this pull request Mar 4, 2015
fix NULL pointer when back-propagating to the data for fixed i.e. not
back-propagated weights. same issue as BVLC#1048
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants