-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Some questions about Key.Net #6
Comments
Hello, Thank you for your interest in our work, and for bringing this error up with us. The keypoints on top do look strange, and this behaviour is indeed not intended. We are looking into it, we will update this issue when we find the problem. Thank you once again! |
Hi again, thank you for your answer and time :) In the meanwhile, I delved a little more into the code/model and gathered some findings. The TL;DR is that setting a threshold in the response map of ~1.124 solves the problem. Below I explain why I think so, hoping it is useful, Since we have a 0-threshold when extracting the keypoints from the response maps, this causes that even when an image completely lacks texture, the system will return keypoints. An example, where the colobar refers to the keypoint' scores is shown here: As a sidenote, in this case the features outputted by the handcrafted block are 0-valued since the gradients are also 0-valued. As can be seen, the scores have a constant value which approximately is 1.1235435, The origin of this value comes from the bias of the last conv. layer, as shown below: print(keynet_model.last_conv[0].bias)
# This gives:
# Parameter containing:
# tensor([1.1235], requires_grad=True) Since the network during inference behaves linearly, then inputs (or receptive fields) with 0-valued gradients will always be offset by this final bias. In fact, imposing this threshold we effectively remove the previous strange keypoints: As a final note, I have also experimented the system in its Tensorflow version and this is not present since the final bias is 0: print([v for v in tf.trainable_variables() if v.name == 'model_deep_detector_conv_last_layer/bias:0'][0].eval())
# This gives:
# [0.] Just in case it is deemed useful I'm preparing a pull request (for that I may use my main acc. 😅). Best |
Hi @axelBarroso , congrats for the great work and thanks for open-sourcing it :)
I have tried a little the system and the detections that I'm getting look like the following examples:
My gut feeling tells me that the keypoints detected at the top may not be as distinctive as the ones detected in other regions. On the one hand they are highly concentrated and on the other hand they belong regions without texture.
Is this a sign that I may have broken something while installing and playing with the system?
Otherwise, Key.Net should be tuned with the following hyperparameters?:
Thanks again and apologies for the long questions.
The text was updated successfully, but these errors were encountered: