Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove additional softmax function if ClassificationNeuralNetwork uses CategoricalCrossEntropy #581

Closed
Marsmaennchen221 opened this issue Mar 25, 2024 · 0 comments · Fixed by #645
Closed

Remove additional softmax function if ClassificationNeuralNetwork uses CategoricalCrossEntropy #581

Marsmaennchen221 opened this issue Mar 25, 2024 · 0 comments · Fixed by #645
Assignees
@sibre28
Labels
bug 🪲 Something isn't working

Comments

@Marsmaennchen221
Copy link
Contributor

Marsmaennchen221 commented Mar 25, 2024 •
edited
Loading

Describe the bug

Currently, the last layer of a ClassificationNeuralNetwork module uses softmax as an activation function, but this is not required and can lead to problems in the learning because pytorch CategoricalCrossEntropy already includes a softmax function, as described in the documentation:

Note that this case is equivalent to applying LogSoftmax on an input, followed by NLLLoss.

To Reproduce

/

Expected behavior

Don't use an extra softmax function

Screenshots (optional)

No response

Additional Context (optional)

No response
@Marsmaennchen221 Marsmaennchen221 added the bug 🪲 Something isn't working label Mar 25, 2024
@github-project-automation github-project-automation bot moved this to Backlog in Library Mar 25, 2024
Marsmaennchen221 added a commit that referenced this issue Apr 23, 2024
@Marsmaennchen221
feat: added Convolutional2DLayer, FlattenLayer, `MaxPooling2DLaye…
312feec 
…r` and `AvgPooling2DLayer`

feat: added `InputConversionImage` and `OutputConversionImage`
feat: added class `ImageSize` and methods `ImageList.sizes` and `Image.size` to get the sizes of the respective images
feat: added ability to iterate over `SingleSizeImageList`
feat: added option `None` for no activation function in `ForwardLayer`
fix: fixed bug #581 in removing the Softmax function from the last layer in `NeuralNetworkClassifier`
@Marsmaennchen221 Marsmaennchen221 mentioned this issue Apr 23, 2024
Merged
@Marsmaennchen221 Marsmaennchen221 linked a pull request Apr 23, 2024 that will close this issue
feat: Add ImageDataset and Layer for ConvolutionalNeuralNetworks #645
Merged
lars-reimann pushed a commit that referenced this issue May 6, 2024
@Marsmaennchen221 @megalinter-bot
feat: Add ImageDataset and Layer for ConvolutionalNeuralNetworks (#645
5b6d219 
)

Closes #579, #580, #581 

### Summary of Changes

feat: added `Convolutional2DLayer`, `ConvolutionalTranspose2DLayer`,
`FlattenLayer`, `MaxPooling2DLayer` and `AvgPooling2DLayer`
feat: added `InputConversionImage`, `OutputConversionImageToColumn`,
`OutputConversionImageToTable` and `OutputConversionImageToImage`
feat: added generic `ImageDataset`
feat: added class `ImageSize` and methods `ImageList.sizes` and
`Image.size` to get the sizes of the respective images
feat: added ability to iterate over `SingleSizeImageList`
feat: added param to return filenames in `ImageList.from_files`
feat: added option `None` for no activation function in `ForwardLayer`
feat: added `Image.__array__` to convert a `Image` to a `numpy.ndarray`
feat: added equals check to `OneHotEncoder`
fix: fixed bug #581 in removing
the Softmax function from the last layer in `NeuralNetworkClassifier`
refactor: move `image.utils` to `image._utils`
refactor: extracted test devices from `test_image` to `helpers.devices`

---------

Co-authored-by: megalinter-bot <[email protected]>
@github-project-automation github-project-automation bot moved this from Backlog to ✔️ Done in Library May 6, 2024
lars-reimann pushed a commit that referenced this issue May 9, 2024
@semantic-release-bot
chore(release): 0.24.0 [skip ci]
22d6cb6 
## [0.24.0](v0.23.0...v0.24.0) (2024-05-09)

### Features

* `Column.plot_histogram()` using `Table.plot_histograms` for consistent results ([#726](#726)) ([576492c](576492c))
* `Regressor.summarize_metrics` and `Classifier.summarize_metrics` ([#729](#729)) ([1cc14b1](1cc14b1)), closes [#713](#713)
* `Table.keep_only_rows` ([#721](#721)) ([923a6c2](923a6c2))
* `Table.remove_rows` ([#720](#720)) ([a1cdaef](a1cdaef)), closes [#698](#698)
* Add `ImageDataset` and Layer for ConvolutionalNeuralNetworks ([#645](#645)) ([5b6d219](5b6d219)), closes [#579](#579) [#580](#580) [#581](#581)
* added load_percentage parameter to ImageList.from_files to load a subset of the given files ([#739](#739)) ([0564b52](0564b52)), closes [#736](#736)
* added rnn layer and TimeSeries conversion ([#615](#615)) ([6cad203](6cad203)), closes [#614](#614) [#648](#648) [#656](#656) [#601](#601)
* Basic implementation of cell with polars ([#734](#734)) ([004630b](004630b)), closes [#712](#712)
* deprecate `Table.add_column` and `Table.add_row` ([#723](#723)) ([5dd9d02](5dd9d02)), closes [#722](#722)
* deprecated `Table.from_excel_file` and `Table.to_excel_file` ([#728](#728)) ([c89e0bf](c89e0bf)), closes [#727](#727)
* Larger histogram plot if table only has one column ([#716](#716)) ([31ffd12](31ffd12))
* polars implementation of a column ([#738](#738)) ([732aa48](732aa48)), closes [#712](#712)
* polars implementation of a row ([#733](#733)) ([ff627f6](ff627f6)), closes [#712](#712)
* polars implementation of table ([#744](#744)) ([fc49895](fc49895)), closes [#638](#638) [#641](#641) [#649](#649) [#712](#712)
* regularization for decision trees and random forests ([#730](#730)) ([102de2d](102de2d)), closes [#700](#700)
* Remove device information in image class ([#735](#735)) ([d783caa](d783caa)), closes [#524](#524)
* return fitted transformer and transformed table from `fit_and_transform` ([#724](#724)) ([2960d35](2960d35)), closes [#613](#613)

### Bug Fixes

* make `Image.clone` internal ([#725](#725)) ([215a472](215a472)), closes [#626](#626)

### Performance Improvements

* improved performance of `TabularDataset.__eq__` by a factor of up to 2 ([#697](#697)) ([cd7f55b](cd7f55b))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sibre28 sibre28

Labels
bug 🪲 Something isn't working
Projects
Library
Status: ✔️ Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: Add ImageDataset and Layer for ConvolutionalNeuralNetworks Safe-DS/Library
2 participants
@Marsmaennchen221 @sibre28