-
Notifications
You must be signed in to change notification settings - Fork 19.4k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
PR #18105: [Add Feature] - Throw an error if softmax is used with 1 n…
…euron Imported from GitHub PR #18105 This is a utility function to check if the usage of softmax makes sense (new users make this mistake a lot). Applying softmax on a single neuron will make the model output ones everytime, there are too many Stackoverflow posts about this. In order to see this in action, please check [the gist](https://colab.research.google.com/gist/Frightera/fdcec020fff6ee9521ae2fd3eaed774d/checksoftmaxlastlayer.ipynb). This applies for any other layers (Conv2D etc.) where the applied axis (axis=-1 default) of softmax has only one unit. Copybara import of the project: -- 90c95b1 by Kaan Bıçakcı <[email protected]>: Add last layer activation check for softmax -- 1cedb20 by Kaan Bıçakcı <[email protected]>: Split logic for sequential and functional models -- 529f968 by Kaan Bıçakcı <[email protected]>: Add tests for _check_last_layer_activation -- d1acddb by Kaan Bıçakcı <[email protected]>: Update sequential check -- 8363016 by Kaan Bıçakcı <[email protected]>: Update tests, logic and reformatting -- ebf16c3 by Kaan Bıçakcı <[email protected]>: Update tests and the logic -- afc156a by Kaan Bıçakcı <[email protected]>: Make validate_softmax_activation experimental -- 3a228fb by Kaan Bıçakcı <[email protected]>: Fix edge case for _validate_softmax_output -- e9c950e by Kaan Bıçakcı <[email protected]>: Check the softmax axis and raise an error if relevant -- 6355b23 by Kaan Bıçakcı <[email protected]>: Update softmax check tests -- a6745ee by Kaan Bıçakcı <[email protected]>: Minor typo fix -- 92281f6 by Kaan Bıçakcı <[email protected]>: Fix test fails for _check_output_activation_softmax -- 72a035f by Kaan Bıçakcı <[email protected]>: Resolve conflict -- 0af059c by Kaan Bıçakcı <[email protected]>: Squashed commit master (merge) to resolve conflicts -- 065cdea by Kaan Bıçakcı <[email protected]>: Revert "Squashed commit master (merge) to resolve conflicts" This reverts commit 0af059c. -- 446f1dd by Kaan Bıçakcı <[email protected]>: Remove steps_per_execution_tuning from imports -- 1fbd931 by Kaan Bıçakcı <[email protected]>: Fix lint Merging this change closes #18105 FUTURE_COPYBARA_INTEGRATE_REVIEW=#18105 from Frightera:last_layer_softmax_warn 1fbd931 PiperOrigin-RevId: 542046193
- Loading branch information
1 parent
5f9d052
commit 4af7328
Showing
2 changed files
with
191 additions
and
5 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters