diff --git a/development/_downloads/bc82bea3a5dd7bdba60b65220891d9e5/examples_python.zip b/development/_downloads/bc82bea3a5dd7bdba60b65220891d9e5/examples_python.zip index f31a820a7..e5c3e6635 100644 Binary files a/development/_downloads/bc82bea3a5dd7bdba60b65220891d9e5/examples_python.zip and b/development/_downloads/bc82bea3a5dd7bdba60b65220891d9e5/examples_python.zip differ diff --git a/development/_downloads/fb625db3c50d423b1b7881136ffdeec8/examples_jupyter.zip b/development/_downloads/fb625db3c50d423b1b7881136ffdeec8/examples_jupyter.zip index 55c623ec1..8bd6ed7ef 100644 Binary files a/development/_downloads/fb625db3c50d423b1b7881136ffdeec8/examples_jupyter.zip and b/development/_downloads/fb625db3c50d423b1b7881136ffdeec8/examples_jupyter.zip differ diff --git a/development/_images/sphx_glr_example_plot_over_time_001.png b/development/_images/sphx_glr_example_plot_over_time_001.png index b807275c4..147901012 100644 Binary files a/development/_images/sphx_glr_example_plot_over_time_001.png and b/development/_images/sphx_glr_example_plot_over_time_001.png differ diff --git a/development/_images/sphx_glr_example_plot_over_time_thumb.png b/development/_images/sphx_glr_example_plot_over_time_thumb.png index f4cad3d80..e39029ce0 100644 Binary files a/development/_images/sphx_glr_example_plot_over_time_thumb.png and b/development/_images/sphx_glr_example_plot_over_time_thumb.png differ diff --git a/development/_images/sphx_glr_example_visualization_001.png b/development/_images/sphx_glr_example_visualization_001.png index 3b58e0342..169bc5639 100644 Binary files a/development/_images/sphx_glr_example_visualization_001.png and b/development/_images/sphx_glr_example_visualization_001.png differ diff --git a/development/_images/sphx_glr_example_visualization_thumb.png b/development/_images/sphx_glr_example_visualization_thumb.png index 06c10358c..d46d77f01 100644 Binary files a/development/_images/sphx_glr_example_visualization_thumb.png and b/development/_images/sphx_glr_example_visualization_thumb.png differ diff --git a/development/_sources/examples/20_basics/example_image_classification.rst.txt b/development/_sources/examples/20_basics/example_image_classification.rst.txt index fb1816afc..927902cd3 100644 --- a/development/_sources/examples/20_basics/example_image_classification.rst.txt +++ b/development/_sources/examples/20_basics/example_image_classification.rst.txt @@ -35,22 +35,22 @@ Image Classification Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/train-images-idx3-ubyte.gz Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/train-images-idx3-ubyte.gz to ../datasets/FashionMNIST/raw/train-images-idx3-ubyte.gz - 0%| | 0/26421880 [00:00 + @@ -165,20 +165,20 @@ Print the final ensemble performance .. code-block:: none - {'accuracy': 0.8670520231213873} - | | Preprocessing | Estimator | Weight | - |---:|:--------------------------------------------------------------------------------------|:----------------------------------------------------------------|---------:| - | 0 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,MinMaxScaler,FastICA | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.56 | - | 1 | SimpleImputer,Variance Threshold,MinorityCoalescer,OneHotEncoder,Normalizer,KernelPCA | embedding,ShapedResNetBackbone,FullyConnectedHead,nn.Sequential | 0.38 | - | 2 | SimpleImputer,Variance Threshold,NoCoalescer,NoEncoder,StandardScaler,PCA | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.02 | - | 3 | None | CBLearner | 0.02 | - | 4 | None | SVMLearner | 0.02 | + {'accuracy': 0.8728323699421965} + | | Preprocessing | Estimator | Weight | + |---:|:-------------------------------------------------------------------------------------------|:-------------------------------------------------------------|---------:| + | 0 | None | CBLearner | 0.5 | + | 1 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,MinMaxScaler,FastICA | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.22 | + | 2 | SimpleImputer,Variance Threshold,MinorityCoalescer,OneHotEncoder,PowerTransformer,Nystroem | embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.14 | + | 3 | SimpleImputer,Variance Threshold,NoCoalescer,NoEncoder,StandardScaler,PCA | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.12 | + | 4 | None | RFLearner | 0.02 | autoPyTorch results: Dataset name: Australian Optimisation Metric: accuracy Best validation score: 0.8713450292397661 - Number of target algorithm runs: 27 - Number of successful target algorithm runs: 26 + Number of target algorithm runs: 23 + Number of successful target algorithm runs: 22 Number of crashed target algorithm runs: 0 Number of target algorithms that exceeded the time limit: 1 Number of target algorithms that exceeded the memory limit: 0 @@ -190,7 +190,7 @@ Print the final ensemble performance .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 5 minutes 24.577 seconds) + **Total running time of the script:** ( 5 minutes 27.372 seconds) .. _sphx_glr_download_examples_20_basics_example_tabular_classification.py: diff --git a/development/_sources/examples/20_basics/example_tabular_regression.rst.txt b/development/_sources/examples/20_basics/example_tabular_regression.rst.txt index ffde41825..8b521bc34 100644 --- a/development/_sources/examples/20_basics/example_tabular_regression.rst.txt +++ b/development/_sources/examples/20_basics/example_tabular_regression.rst.txt @@ -125,7 +125,7 @@ Search for an ensemble of machine learning algorithms .. code-block:: none - + @@ -159,19 +159,20 @@ Print the final ensemble performance .. code-block:: none - {'r2': 0.9407884171054208} + {'r2': 0.9412847640085195} | | Preprocessing | Estimator | Weight | |---:|:-------------------------------------------------------------------------------------------------|:----------------------------------------------------------------|---------:| - | 0 | None | CBLearner | 0.44 | - | 1 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.42 | + | 0 | None | CBLearner | 0.46 | + | 1 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.4 | | 2 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.1 | - | 3 | None | LGBMLearner | 0.04 | + | 3 | None | LGBMLearner | 0.02 | + | 4 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.02 | autoPyTorch results: - Dataset name: 59922def-0351-11ed-8824-d5cce4119db9 + Dataset name: ba73302f-0375-11ed-8828-9bcdcaaf1ae6 Optimisation Metric: r2 - Best validation score: 0.8670098636440993 - Number of target algorithm runs: 24 - Number of successful target algorithm runs: 22 + Best validation score: 0.8669094525651709 + Number of target algorithm runs: 22 + Number of successful target algorithm runs: 20 Number of crashed target algorithm runs: 0 Number of target algorithms that exceeded the time limit: 2 Number of target algorithms that exceeded the memory limit: 0 @@ -183,7 +184,7 @@ Print the final ensemble performance .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 5 minutes 36.793 seconds) + **Total running time of the script:** ( 6 minutes 2.422 seconds) .. _sphx_glr_download_examples_20_basics_example_tabular_regression.py: diff --git a/development/_sources/examples/20_basics/example_time_series_forecasting.rst.txt b/development/_sources/examples/20_basics/example_time_series_forecasting.rst.txt index 31b4b2a47..18dc19c07 100644 --- a/development/_sources/examples/20_basics/example_time_series_forecasting.rst.txt +++ b/development/_sources/examples/20_basics/example_time_series_forecasting.rst.txt @@ -150,7 +150,7 @@ Search for an ensemble of machine learning algorithms .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 1 minutes 3.199 seconds) + **Total running time of the script:** ( 1 minutes 6.152 seconds) .. _sphx_glr_download_examples_20_basics_example_time_series_forecasting.py: diff --git a/development/_sources/examples/20_basics/sg_execution_times.rst.txt b/development/_sources/examples/20_basics/sg_execution_times.rst.txt index f9a69c73a..13192a137 100644 --- a/development/_sources/examples/20_basics/sg_execution_times.rst.txt +++ b/development/_sources/examples/20_basics/sg_execution_times.rst.txt @@ -5,14 +5,14 @@ Computation times ================= -**12:11.890** total execution time for **examples_20_basics** files: +**12:43.941** total execution time for **examples_20_basics** files: +----------------------------------------------------------------------------------------------------------------+-----------+--------+ -| :ref:`sphx_glr_examples_20_basics_example_tabular_regression.py` (``example_tabular_regression.py``) | 05:36.793 | 0.0 MB | +| :ref:`sphx_glr_examples_20_basics_example_tabular_regression.py` (``example_tabular_regression.py``) | 06:02.422 | 0.0 MB | +----------------------------------------------------------------------------------------------------------------+-----------+--------+ -| :ref:`sphx_glr_examples_20_basics_example_tabular_classification.py` (``example_tabular_classification.py``) | 05:24.577 | 0.0 MB | +| :ref:`sphx_glr_examples_20_basics_example_tabular_classification.py` (``example_tabular_classification.py``) | 05:27.372 | 0.0 MB | +----------------------------------------------------------------------------------------------------------------+-----------+--------+ -| :ref:`sphx_glr_examples_20_basics_example_time_series_forecasting.py` (``example_time_series_forecasting.py``) | 01:03.199 | 0.0 MB | +| :ref:`sphx_glr_examples_20_basics_example_time_series_forecasting.py` (``example_time_series_forecasting.py``) | 01:06.152 | 0.0 MB | +----------------------------------------------------------------------------------------------------------------+-----------+--------+ -| :ref:`sphx_glr_examples_20_basics_example_image_classification.py` (``example_image_classification.py``) | 00:07.321 | 0.0 MB | +| :ref:`sphx_glr_examples_20_basics_example_image_classification.py` (``example_image_classification.py``) | 00:07.995 | 0.0 MB | +----------------------------------------------------------------------------------------------------------------+-----------+--------+ diff --git a/development/_sources/examples/40_advanced/example_custom_configuration_space.rst.txt b/development/_sources/examples/40_advanced/example_custom_configuration_space.rst.txt index eacc0b3b4..2ebd60980 100644 --- a/development/_sources/examples/40_advanced/example_custom_configuration_space.rst.txt +++ b/development/_sources/examples/40_advanced/example_custom_configuration_space.rst.txt @@ -163,7 +163,7 @@ Search for an ensemble of machine learning algorithms .. code-block:: none - + @@ -194,27 +194,22 @@ Print the final ensemble performance .. code-block:: none - {'accuracy': 0.8497109826589595} - | | Preprocessing | Estimator | Weight | - |---:|:-------------------------------------------------------------------------------------------------|:----------------------------------------------------------|---------:| - | 0 | None | LGBMLearner | 0.24 | - | 1 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,NoScaler,NoFeaturePreprocessing | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.18 | - | 2 | None | RFLearner | 0.16 | - | 3 | None | ETLearner | 0.12 | - | 4 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.1 | - | 5 | None | KNNLearner | 0.06 | - | 6 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,StandardScaler,LinearSVC Preprocessor | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.04 | - | 7 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.04 | - | 8 | SimpleImputer,Variance Threshold,MinorityCoalescer,OneHotEncoder,NoScaler,PCA | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.02 | - | 9 | None | SVMLearner | 0.02 | - | 10 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.02 | + {'accuracy': 0.8554913294797688} + | | Preprocessing | Estimator | Weight | + |---:|:-------------------------------------------------------------------------------------------|:----------------------------------------------------------|---------:| + | 0 | None | LGBMLearner | 0.24 | + | 1 | None | RFLearner | 0.24 | + | 2 | None | SVMLearner | 0.16 | + | 3 | None | ETLearner | 0.14 | + | 4 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,NoScaler,NoFeaturePreprocessing | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.12 | + | 5 | SimpleImputer,Variance Threshold,MinorityCoalescer,OneHotEncoder,NoScaler,PCA | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.1 | autoPyTorch results: - Dataset name: 3dfb9719-0355-11ed-8824-d5cce4119db9 + Dataset name: c5589c90-0379-11ed-8828-9bcdcaaf1ae6 Optimisation Metric: accuracy Best validation score: 0.8596491228070176 - Number of target algorithm runs: 18 - Number of successful target algorithm runs: 15 - Number of crashed target algorithm runs: 2 + Number of target algorithm runs: 17 + Number of successful target algorithm runs: 13 + Number of crashed target algorithm runs: 3 Number of target algorithms that exceeded the time limit: 1 Number of target algorithms that exceeded the memory limit: 0 @@ -274,7 +269,7 @@ Search for an ensemble of machine learning algorithms .. code-block:: none - + @@ -304,22 +299,24 @@ Print the final ensemble performance .. code-block:: none - {'accuracy': 0.8670520231213873} + {'accuracy': 0.8554913294797688} | | Preprocessing | Estimator | Weight | |---:|:---------------------------------------------------------------------------------------------|:----------------------------------------------------------------|---------:| - | 0 | None | LGBMLearner | 0.32 | + | 0 | None | RFLearner | 0.28 | | 1 | None | SVMLearner | 0.28 | - | 2 | None | RFLearner | 0.26 | - | 3 | None | ETLearner | 0.1 | - | 4 | SimpleImputer,Variance Threshold,NoCoalescer,NoEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.02 | - | 5 | SimpleImputer,Variance Threshold,NoCoalescer,NoEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.02 | + | 2 | None | KNNLearner | 0.14 | + | 3 | None | ETLearner | 0.12 | + | 4 | SimpleImputer,Variance Threshold,NoCoalescer,NoEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.06 | + | 5 | SimpleImputer,Variance Threshold,NoCoalescer,NoEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.06 | + | 6 | None | LGBMLearner | 0.04 | + | 7 | SimpleImputer,Variance Threshold,NoCoalescer,NoEncoder,Normalizer,KernelPCA | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.02 | autoPyTorch results: - Dataset name: a6de045b-0355-11ed-8824-d5cce4119db9 + Dataset name: 2b3b0f0c-037a-11ed-8828-9bcdcaaf1ae6 Optimisation Metric: accuracy - Best validation score: 0.8596491228070176 - Number of target algorithm runs: 17 - Number of successful target algorithm runs: 13 - Number of crashed target algorithm runs: 3 + Best validation score: 0.8654970760233918 + Number of target algorithm runs: 18 + Number of successful target algorithm runs: 12 + Number of crashed target algorithm runs: 5 Number of target algorithms that exceeded the time limit: 1 Number of target algorithms that exceeded the memory limit: 0 @@ -330,7 +327,7 @@ Print the final ensemble performance .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 5 minutes 47.668 seconds) + **Total running time of the script:** ( 5 minutes 39.400 seconds) .. _sphx_glr_download_examples_40_advanced_example_custom_configuration_space.py: diff --git a/development/_sources/examples/40_advanced/example_parallel_n_jobs.rst.txt b/development/_sources/examples/40_advanced/example_parallel_n_jobs.rst.txt index 9fbd937d0..13b680c71 100644 --- a/development/_sources/examples/40_advanced/example_parallel_n_jobs.rst.txt +++ b/development/_sources/examples/40_advanced/example_parallel_n_jobs.rst.txt @@ -36,7 +36,7 @@ with AutoPyTorch .. code-block:: none - [ERROR] [2022-07-14 09:12:42,959:asyncio.events] + [ERROR] [2022-07-14 13:33:48,799:asyncio.events] Traceback (most recent call last): File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/distributed/utils.py", line 778, in wrapper return await func(*args, **kwargs) @@ -49,7 +49,7 @@ with AutoPyTorch File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/asyncio/tasks.py", line 659, in sleep return await future asyncio.exceptions.CancelledError - [ERROR] [2022-07-14 09:12:42,971:asyncio.events] + [ERROR] [2022-07-14 13:33:48,829:asyncio.events] Traceback (most recent call last): File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/distributed/utils.py", line 778, in wrapper return await func(*args, **kwargs) @@ -68,12 +68,12 @@ with AutoPyTorch asyncio.exceptions.CancelledError {'accuracy': 0.8497109826589595} autoPyTorch results: - Dataset name: 5df1238d-0354-11ed-8824-d5cce4119db9 + Dataset name: cf74cfb3-0378-11ed-8828-9bcdcaaf1ae6 Optimisation Metric: accuracy Best validation score: 0.8713450292397661 - Number of target algorithm runs: 53 - Number of successful target algorithm runs: 37 - Number of crashed target algorithm runs: 13 + Number of target algorithm runs: 37 + Number of successful target algorithm runs: 29 + Number of crashed target algorithm runs: 5 Number of target algorithms that exceeded the time limit: 3 Number of target algorithms that exceeded the memory limit: 0 @@ -151,7 +151,7 @@ with AutoPyTorch .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 5 minutes 34.013 seconds) + **Total running time of the script:** ( 5 minutes 49.003 seconds) .. _sphx_glr_download_examples_40_advanced_example_parallel_n_jobs.py: diff --git a/development/_sources/examples/40_advanced/example_pass_feature_types.rst.txt b/development/_sources/examples/40_advanced/example_pass_feature_types.rst.txt index 12c254e24..71da06988 100644 --- a/development/_sources/examples/40_advanced/example_pass_feature_types.rst.txt +++ b/development/_sources/examples/40_advanced/example_pass_feature_types.rst.txt @@ -153,7 +153,7 @@ Search for an ensemble of machine learning algorithms .. code-block:: none - + @@ -184,18 +184,18 @@ Print the final ensemble performance .. code-block:: none - {'accuracy': 0.9490740740740741} + {'accuracy': 0.9467592592592593} | | Preprocessing | Estimator | Weight | |---:|:---------------------------------------------------------------------------------------------|:----------------------------------------------------------------|---------:| - | 0 | SimpleImputer,Variance Threshold,MinorityCoalescer,NoEncoder,NoScaler,NoFeaturePreprocessing | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.74 | - | 1 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,NoScaler,NoFeaturePreprocessing | embedding,ResNetBackbone,FullyConnectedHead,nn.Sequential | 0.14 | + | 0 | SimpleImputer,Variance Threshold,MinorityCoalescer,NoEncoder,NoScaler,NoFeaturePreprocessing | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.7 | + | 1 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,NoScaler,NoFeaturePreprocessing | embedding,ResNetBackbone,FullyConnectedHead,nn.Sequential | 0.18 | | 2 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,NoScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.12 | autoPyTorch results: Dataset name: Australian Optimisation Metric: accuracy - Best validation score: 0.9135514018691588 - Number of target algorithm runs: 8 - Number of successful target algorithm runs: 8 + Best validation score: 0.9088785046728972 + Number of target algorithm runs: 7 + Number of successful target algorithm runs: 7 Number of crashed target algorithm runs: 0 Number of target algorithms that exceeded the time limit: 0 Number of target algorithms that exceeded the memory limit: 0 @@ -207,7 +207,7 @@ Print the final ensemble performance .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 1 minutes 44.896 seconds) + **Total running time of the script:** ( 1 minutes 48.797 seconds) .. _sphx_glr_download_examples_40_advanced_example_pass_feature_types.py: diff --git a/development/_sources/examples/40_advanced/example_plot_over_time.rst.txt b/development/_sources/examples/40_advanced/example_plot_over_time.rst.txt index e892a4bc9..6ab32b064 100644 --- a/development/_sources/examples/40_advanced/example_plot_over_time.rst.txt +++ b/development/_sources/examples/40_advanced/example_plot_over_time.rst.txt @@ -89,9 +89,9 @@ Task Definition .. code-block:: none - [0 1 0 1 1 0 0 1 1 1 1 0 0 1 0 0 0 0 1 1 0 1 1 1 1 1 0 1 1 1 1 0 0 1 1 0 0 - 1 0 0 0 1 0 0 0 1 0 1 1 0 0 0 0 1 0 1 0 1 0 1 0 0 0 0 1 0 0 0 1 0 1 1 1 1 - 1 1 0 0 1 0 0 0 0 1 1 0 0 0 1 0 0 1 0 1 1 0 1 1 0 0] + [1 0 1 1 0 1 1 0 0 1 0 1 0 0 0 1 1 1 1 1 1 0 0 1 1 1 1 1 1 1 1 0 0 1 0 0 1 + 0 1 1 1 1 1 1 1 0 0 1 0 0 1 0 0 0 1 1 0 0 0 0 1 0 1 0 0 1 0 0 1 0 0 0 0 0 + 1 1 1 1 1 1 1 1 0 1 0 0 0 1 1 0 1 0 0 0 1 0 1 1 0 1] @@ -121,7 +121,7 @@ API Instantiation and Searching .. code-block:: none - + @@ -186,7 +186,7 @@ _, ax = plt.subplots() <=== You can feed it to post-process the figure. .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 2 minutes 21.398 seconds) + **Total running time of the script:** ( 2 minutes 22.121 seconds) .. _sphx_glr_download_examples_40_advanced_example_plot_over_time.py: diff --git a/development/_sources/examples/40_advanced/example_resampling_strategy.rst.txt b/development/_sources/examples/40_advanced/example_resampling_strategy.rst.txt index 861944ec6..1fb0bf42e 100644 --- a/development/_sources/examples/40_advanced/example_resampling_strategy.rst.txt +++ b/development/_sources/examples/40_advanced/example_resampling_strategy.rst.txt @@ -139,7 +139,7 @@ Search for an ensemble of machine learning algorithms .. code-block:: none - + @@ -171,22 +171,24 @@ Print the final ensemble performance .. code-block:: none - {'accuracy': 0.8554913294797688} - | | Preprocessing | Estimator | Weight | - |---:|:-------------------------------------------------------------------------------------------------|:----------------------------------------------------------------|---------:| - | 0 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,NoScaler,LinearSVC Preprocessor | embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.28 | - | 1 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.28 | - | 2 | SimpleImputer,Variance Threshold,NoCoalescer,NoEncoder,StandardScaler,SPC | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.24 | - | 3 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.18 | - | 4 | None | KNNLearner | 0.02 | + {'accuracy': 0.8728323699421965} + | | Preprocessing | Estimator | Weight | + |---:|:-------------------------------------------------------------------------------------------|:-------------------------------------------------------------|---------:| + | 0 | None | RFLearner | 0.34 | + | 1 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,NoScaler,LinearSVC Preprocessor | embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.28 | + | 2 | None | ETLearner | 0.16 | + | 3 | None | LGBMLearner | 0.08 | + | 4 | None | SVMLearner | 0.06 | + | 5 | None | KNNLearner | 0.06 | + | 6 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,QuantileTransformer,TruncSVD | no embedding,ResNetBackbone,FullyConnectedHead,nn.Sequential | 0.02 | autoPyTorch results: - Dataset name: 0d6aeb2d-0356-11ed-8824-d5cce4119db9 + Dataset name: 8fd61711-037a-11ed-8828-9bcdcaaf1ae6 Optimisation Metric: accuracy - Best validation score: 0.8713450292397661 + Best validation score: 0.8596491228070176 Number of target algorithm runs: 18 - Number of successful target algorithm runs: 14 - Number of crashed target algorithm runs: 2 - Number of target algorithms that exceeded the time limit: 2 + Number of successful target algorithm runs: 13 + Number of crashed target algorithm runs: 4 + Number of target algorithms that exceeded the time limit: 1 Number of target algorithms that exceeded the memory limit: 0 @@ -250,7 +252,7 @@ Search for an ensemble of machine learning algorithms .. code-block:: none - + @@ -282,22 +284,23 @@ Print the final ensemble performance .. code-block:: none - {'accuracy': 0.8728323699421965} + {'accuracy': 0.861271676300578} | | Preprocessing | Estimator | Weight | |---:|:-----------------------------------------------------------------------------------------------|:----------------------------------------------------------------|---------:| - | 0 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,NoScaler,LinearSVC Preprocessor | embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.56 | - | 1 | None | TabularTraditionalModel | 0.16 | - | 2 | None | TabularTraditionalModel | 0.12 | - | 3 | None | TabularTraditionalModel | 0.08 | - | 4 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,QuantileTransformer,TruncSVD | no embedding,ResNetBackbone,FullyConnectedHead,nn.Sequential | 0.04 | - | 5 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,MinMaxScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.04 | + | 0 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,NoScaler,LinearSVC Preprocessor | embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.36 | + | 1 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,MinMaxScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.22 | + | 2 | None | TabularTraditionalModel | 0.18 | + | 3 | None | TabularTraditionalModel | 0.16 | + | 4 | None | TabularTraditionalModel | 0.04 | + | 5 | None | TabularTraditionalModel | 0.02 | + | 6 | None | TabularTraditionalModel | 0.02 | autoPyTorch results: - Dataset name: 7094e4a0-0356-11ed-8824-d5cce4119db9 + Dataset name: f5c88fd4-037a-11ed-8828-9bcdcaaf1ae6 Optimisation Metric: accuracy Best validation score: 0.8626733083495604 - Number of target algorithm runs: 15 - Number of successful target algorithm runs: 10 - Number of crashed target algorithm runs: 4 + Number of target algorithm runs: 13 + Number of successful target algorithm runs: 9 + Number of crashed target algorithm runs: 3 Number of target algorithms that exceeded the time limit: 1 Number of target algorithms that exceeded the memory limit: 0 @@ -364,7 +367,7 @@ Search for an ensemble of machine learning algorithms .. code-block:: none - + @@ -395,23 +398,25 @@ Print the final ensemble performance .. code-block:: none - {'accuracy': 0.8670520231213873} + {'accuracy': 0.8554913294797688} | | Preprocessing | Estimator | Weight | |---:|:-------------------------------------------------------------------------------------------------|:----------------------------------------------------------------|---------:| - | 0 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,NoScaler,LinearSVC Preprocessor | embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.62 | - | 1 | None | RFLearner | 0.14 | - | 2 | None | KNNLearner | 0.12 | - | 3 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.06 | - | 4 | None | SVMLearner | 0.04 | - | 5 | None | LGBMLearner | 0.02 | + | 0 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,NoScaler,LinearSVC Preprocessor | embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.6 | + | 1 | None | RFLearner | 0.12 | + | 2 | None | SVMLearner | 0.08 | + | 3 | None | KNNLearner | 0.06 | + | 4 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.06 | + | 5 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,QuantileTransformer,TruncSVD | no embedding,ResNetBackbone,FullyConnectedHead,nn.Sequential | 0.04 | + | 6 | None | LGBMLearner | 0.02 | + | 7 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,MinMaxScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.02 | autoPyTorch results: - Dataset name: dd8aeda6-0356-11ed-8824-d5cce4119db9 + Dataset name: 676e8439-037b-11ed-8828-9bcdcaaf1ae6 Optimisation Metric: accuracy Best validation score: 0.8362573099415205 Number of target algorithm runs: 17 Number of successful target algorithm runs: 13 - Number of crashed target algorithm runs: 3 - Number of target algorithms that exceeded the time limit: 1 + Number of crashed target algorithm runs: 4 + Number of target algorithms that exceeded the time limit: 0 Number of target algorithms that exceeded the memory limit: 0 @@ -421,7 +426,7 @@ Print the final ensemble performance .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 8 minutes 34.416 seconds) + **Total running time of the script:** ( 8 minutes 57.141 seconds) .. _sphx_glr_download_examples_40_advanced_example_resampling_strategy.py: diff --git a/development/_sources/examples/40_advanced/example_run_with_portfolio.rst.txt b/development/_sources/examples/40_advanced/example_run_with_portfolio.rst.txt index 582cee59b..4f86daa5a 100644 --- a/development/_sources/examples/40_advanced/example_run_with_portfolio.rst.txt +++ b/development/_sources/examples/40_advanced/example_run_with_portfolio.rst.txt @@ -131,7 +131,7 @@ Search for an ensemble of machine learning algorithms .. code-block:: none - + @@ -162,30 +162,25 @@ Print the final ensemble performance .. code-block:: none - {'accuracy': 0.8670520231213873} + {'accuracy': 0.8786127167630058} | | Preprocessing | Estimator | Weight | |---:|:-------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------|---------:| - | 0 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,NoScaler,KernelPCA | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.16 | - | 1 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,MinMaxScaler,FastICA | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.14 | - | 2 | SimpleImputer,Variance Threshold,MinorityCoalescer,OneHotEncoder,MinMaxScaler,PolynomialFeatures | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.14 | - | 3 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedResNetBackbone,FullyConnectedHead,nn.Sequential | 0.08 | - | 4 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedResNetBackbone,FullyConnectedHead,nn.Sequential | 0.08 | - | 5 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,PowerTransformer,ETC | embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.08 | - | 6 | None | CBLearner | 0.06 | - | 7 | None | RFLearner | 0.06 | - | 8 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.04 | - | 9 | None | ETLearner | 0.04 | - | 10 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedResNetBackbone,FullyConnectedHead,nn.Sequential | 0.04 | - | 11 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedResNetBackbone,FullyConnectedHead,nn.Sequential | 0.02 | - | 12 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedResNetBackbone,FullyConnectedHead,nn.Sequential | 0.02 | - | 13 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedResNetBackbone,FullyConnectedHead,nn.Sequential | 0.02 | - | 14 | None | KNNLearner | 0.02 | + | 0 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,NoScaler,KernelPCA | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.48 | + | 1 | None | CBLearner | 0.16 | + | 2 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,MinMaxScaler,FastICA | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.12 | + | 3 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedResNetBackbone,FullyConnectedHead,nn.Sequential | 0.06 | + | 4 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedResNetBackbone,FullyConnectedHead,nn.Sequential | 0.04 | + | 5 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedResNetBackbone,FullyConnectedHead,nn.Sequential | 0.04 | + | 6 | None | ETLearner | 0.04 | + | 7 | None | RFLearner | 0.02 | + | 8 | None | KNNLearner | 0.02 | + | 9 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedResNetBackbone,FullyConnectedHead,nn.Sequential | 0.02 | autoPyTorch results: - Dataset name: 07882087-0353-11ed-8824-d5cce4119db9 + Dataset name: 796c85a4-0377-11ed-8828-9bcdcaaf1ae6 Optimisation Metric: accuracy - Best validation score: 0.8830409356725146 - Number of target algorithm runs: 27 - Number of successful target algorithm runs: 25 + Best validation score: 0.8771929824561403 + Number of target algorithm runs: 23 + Number of successful target algorithm runs: 21 Number of crashed target algorithm runs: 0 Number of target algorithms that exceeded the time limit: 2 Number of target algorithms that exceeded the memory limit: 0 @@ -197,7 +192,7 @@ Print the final ensemble performance .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 5 minutes 27.265 seconds) + **Total running time of the script:** ( 5 minutes 21.984 seconds) .. _sphx_glr_download_examples_40_advanced_example_run_with_portfolio.py: diff --git a/development/_sources/examples/40_advanced/example_single_configuration.rst.txt b/development/_sources/examples/40_advanced/example_single_configuration.rst.txt index d965790a5..6e7cc67c9 100644 --- a/development/_sources/examples/40_advanced/example_single_configuration.rst.txt +++ b/development/_sources/examples/40_advanced/example_single_configuration.rst.txt @@ -236,8 +236,8 @@ Fit the configuration 'trainer:__choice__': 'StandardTrainer', }) , instance=None, instance_specific=None, seed=1, cutoff=69, capped=False, budget=5, source_id=0) - RunValue(cost=0.03379224030037542, time=27.251476526260376, status=, starttime=1657789982.8888395, endtime=1657790011.193156, additional_info={'opt_loss': {'accuracy': 0.03379224030037542}, 'duration': 27.18387222290039, 'num_run': 2, 'train_loss': {'accuracy': 0.0012515644555695093}, 'test_loss': {'accuracy': 0.028785982478097605}, 'configuration_origin': None}) - {'imputer': SimpleImputer(random_state=RandomState(MT19937) at 0x7F2404596C40), 'variance_threshold': VarianceThreshold(random_state=RandomState(MT19937) at 0x7F2404596640), 'coalescer': , 'encoder': , 'scaler': , 'feature_preprocessor': , 'tabular_transformer': TabularColumnTransformer(random_state=RandomState(MT19937) at 0x7F2404596C40), 'preprocessing': EarlyPreprocessing(random_state=RandomState(MT19937) at 0x7F2404596C40), 'network_embedding': , 'network_backbone': , 'network_head': , 'network': NetworkComponent(network=Sequential( + RunValue(cost=0.03629536921151444, time=42.518319606781006, status=, starttime=1657805656.4641984, endtime=1657805700.0393107, additional_info={'opt_loss': {'accuracy': 0.03629536921151444}, 'duration': 42.437915325164795, 'num_run': 2, 'train_loss': {'accuracy': 0.006257822277847325}, 'test_loss': {'accuracy': 0.033166458072590776}, 'configuration_origin': None}) + {'imputer': SimpleImputer(random_state=RandomState(MT19937) at 0x7F83C8AEE040), 'variance_threshold': VarianceThreshold(random_state=RandomState(MT19937) at 0x7F83C835A940), 'coalescer': , 'encoder': , 'scaler': , 'feature_preprocessor': , 'tabular_transformer': TabularColumnTransformer(random_state=RandomState(MT19937) at 0x7F83C8AEE040), 'preprocessing': EarlyPreprocessing(random_state=RandomState(MT19937) at 0x7F83C8AEE040), 'network_embedding': , 'network_backbone': , 'network_head': , 'network': NetworkComponent(network=Sequential( (0): _NoEmbedding() (1): Sequential( (0): Linear(in_features=73, out_features=200, bias=True) @@ -257,7 +257,7 @@ Fit the configuration (3): Linear(in_features=128, out_features=2, bias=True) ) ), - random_state=RandomState(MT19937) at 0x7F2404596C40), 'network_init': , 'optimizer': , 'lr_scheduler': , 'data_loader': FeatureDataLoader(random_state=RandomState(MT19937) at 0x7F2404596C40), 'trainer': } + random_state=RandomState(MT19937) at 0x7F83C8AEE040), 'network_init': , 'optimizer': , 'lr_scheduler': , 'data_loader': FeatureDataLoader(random_state=RandomState(MT19937) at 0x7F83C8AEE040), 'trainer': } @@ -265,7 +265,7 @@ Fit the configuration .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 0 minutes 41.236 seconds) + **Total running time of the script:** ( 1 minutes 2.817 seconds) .. _sphx_glr_download_examples_40_advanced_example_single_configuration.py: diff --git a/development/_sources/examples/40_advanced/example_visualization.rst.txt b/development/_sources/examples/40_advanced/example_visualization.rst.txt index de5852da7..0fc780dd8 100644 --- a/development/_sources/examples/40_advanced/example_visualization.rst.txt +++ b/development/_sources/examples/40_advanced/example_visualization.rst.txt @@ -135,7 +135,7 @@ Build and fit a classifier .. code-block:: none - + @@ -245,7 +245,7 @@ Plotting the model performance .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 3 minutes 43.363 seconds) + **Total running time of the script:** ( 3 minutes 51.269 seconds) .. _sphx_glr_download_examples_40_advanced_example_visualization.py: diff --git a/development/_sources/examples/40_advanced/sg_execution_times.rst.txt b/development/_sources/examples/40_advanced/sg_execution_times.rst.txt index 0b3000fcf..88404a8a3 100644 --- a/development/_sources/examples/40_advanced/sg_execution_times.rst.txt +++ b/development/_sources/examples/40_advanced/sg_execution_times.rst.txt @@ -5,22 +5,22 @@ Computation times ================= -**33:54.256** total execution time for **examples_40_advanced** files: +**34:52.532** total execution time for **examples_40_advanced** files: +------------------------------------------------------------------------------------------------------------------------+-----------+--------+ -| :ref:`sphx_glr_examples_40_advanced_example_resampling_strategy.py` (``example_resampling_strategy.py``) | 08:34.416 | 0.0 MB | +| :ref:`sphx_glr_examples_40_advanced_example_resampling_strategy.py` (``example_resampling_strategy.py``) | 08:57.141 | 0.0 MB | +------------------------------------------------------------------------------------------------------------------------+-----------+--------+ -| :ref:`sphx_glr_examples_40_advanced_example_custom_configuration_space.py` (``example_custom_configuration_space.py``) | 05:47.668 | 0.0 MB | +| :ref:`sphx_glr_examples_40_advanced_example_parallel_n_jobs.py` (``example_parallel_n_jobs.py``) | 05:49.003 | 0.0 MB | +------------------------------------------------------------------------------------------------------------------------+-----------+--------+ -| :ref:`sphx_glr_examples_40_advanced_example_parallel_n_jobs.py` (``example_parallel_n_jobs.py``) | 05:34.013 | 0.0 MB | +| :ref:`sphx_glr_examples_40_advanced_example_custom_configuration_space.py` (``example_custom_configuration_space.py``) | 05:39.400 | 0.0 MB | +------------------------------------------------------------------------------------------------------------------------+-----------+--------+ -| :ref:`sphx_glr_examples_40_advanced_example_run_with_portfolio.py` (``example_run_with_portfolio.py``) | 05:27.265 | 0.0 MB | +| :ref:`sphx_glr_examples_40_advanced_example_run_with_portfolio.py` (``example_run_with_portfolio.py``) | 05:21.984 | 0.0 MB | +------------------------------------------------------------------------------------------------------------------------+-----------+--------+ -| :ref:`sphx_glr_examples_40_advanced_example_visualization.py` (``example_visualization.py``) | 03:43.363 | 0.0 MB | +| :ref:`sphx_glr_examples_40_advanced_example_visualization.py` (``example_visualization.py``) | 03:51.269 | 0.0 MB | +------------------------------------------------------------------------------------------------------------------------+-----------+--------+ -| :ref:`sphx_glr_examples_40_advanced_example_plot_over_time.py` (``example_plot_over_time.py``) | 02:21.398 | 0.0 MB | +| :ref:`sphx_glr_examples_40_advanced_example_plot_over_time.py` (``example_plot_over_time.py``) | 02:22.121 | 0.0 MB | +------------------------------------------------------------------------------------------------------------------------+-----------+--------+ -| :ref:`sphx_glr_examples_40_advanced_example_pass_feature_types.py` (``example_pass_feature_types.py``) | 01:44.896 | 0.0 MB | +| :ref:`sphx_glr_examples_40_advanced_example_pass_feature_types.py` (``example_pass_feature_types.py``) | 01:48.797 | 0.0 MB | +------------------------------------------------------------------------------------------------------------------------+-----------+--------+ -| :ref:`sphx_glr_examples_40_advanced_example_single_configuration.py` (``example_single_configuration.py``) | 00:41.236 | 0.0 MB | +| :ref:`sphx_glr_examples_40_advanced_example_single_configuration.py` (``example_single_configuration.py``) | 01:02.817 | 0.0 MB | +------------------------------------------------------------------------------------------------------------------------+-----------+--------+ diff --git a/development/_sources/releases.rst.txt b/development/_sources/releases.rst.txt index 287a5e8c7..4ce326bb8 100644 --- a/development/_sources/releases.rst.txt +++ b/development/_sources/releases.rst.txt @@ -12,6 +12,48 @@ Releases ======== +Version 0.2 +=========== +[FIX] Documentation and docker workflow file (#449) +[RELEASE] Changes for release v0.2 (#446) +[ADD] Allow users to pass feat types to tabular validator (#441) +[ADD] docs for forecasting task (#443) +[FIX] fit updates in gluonts (#445) +[ADD] Time series forecasting (#434) +[FIX] fix dist twine check for github (#439) +[ADD] Subsampling Dataset (#398) +[feat] Add __str__ to autoPyTorchEnum (#405) +[ADD] feature preprocessors from autosklearn (#378) +[refactor] Fix SparseMatrixType --> spmatrix and add ispandas (#397) +[ADD] dataset compression (#387) +[fix] Update the SMAC version (#388) +[feat] Add new task inference for APT (#386) +[FIX] Datamanager in memory (#382) +[FIX] Fix: keyword arguments to submit (#384) +[feat] Add coalescer (#376) +[FIX] Remove redundant categorical imputation (#375) +[ADD] scalers from autosklearn (#372) +[ADD] variance thresholding (#373) +[fix] Change int to np.int32 for the ndarray dtype specification (#371) +[fix] Hotfix debug no training in simple intensifier (#370) +[ADD] Test evaluator (#368) +[FIX] Fix 361 (#367) +[FIX] fix error after merge +[ADD] Docker publish workflow (#357) +[ADD] fit pipeline honoring API constraints with tests (#348) +[FIX] Update workflow files (#363) +[feat] Add the option to save a figure in plot setting params (#351) +[FIX] Cleanup of simple_imputer (#346) +[feat] Add an object that realizes the perf over time viz (#331) + +Contributors v0.2 +***************** + +* Ravin Kohli +* Shuhei Watanabe +* Eddie Bergman +* Difan Deng + Version 0.1.1 ============== [refactor] Completely refactored version with a new scikit-learn compatible API. diff --git a/development/examples/20_basics/example_image_classification.html b/development/examples/20_basics/example_image_classification.html index 55a49dd28..b1a7404db 100644 --- a/development/examples/20_basics/example_image_classification.html +++ b/development/examples/20_basics/example_image_classification.html @@ -127,52 +127,52 @@ Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/train-images-idx3-ubyte.gz to ../datasets/FashionMNIST/raw/train-images-idx3-ubyte.gz 0%| | 0/26421880 [00:00<?, ?it/s] - 0%| | 32768/26421880 [00:00<01:52, 235602.90it/s] - 0%| | 65536/26421880 [00:00<01:52, 234351.38it/s] - 0%| | 131072/26421880 [00:00<01:17, 340594.04it/s] - 1%| | 229376/26421880 [00:00<00:54, 483036.36it/s] - 2%|1 | 458752/26421880 [00:00<00:28, 898436.87it/s] - 3%|3 | 917504/26421880 [00:00<00:14, 1705017.51it/s] - 7%|7 | 1867776/26421880 [00:00<00:07, 3360695.16it/s] - 14%|#4 | 3702784/26421880 [00:01<00:03, 6451569.09it/s] - 26%|##5 | 6750208/26421880 [00:01<00:01, 11225603.32it/s] - 37%|###7 | 9863168/26421880 [00:01<00:01, 14590423.72it/s] - 49%|####9 | 13008896/26421880 [00:01<00:00, 16972192.00it/s] - 61%|###### | 16056320/26421880 [00:01<00:00, 18420560.72it/s] - 72%|#######1 | 19005440/26421880 [00:01<00:00, 19207823.64it/s] - 84%|########3 | 22085632/26421880 [00:01<00:00, 20031175.65it/s] - 95%|#########5| 25231360/26421880 [00:02<00:00, 20738665.14it/s] -100%|##########| 26421880/26421880 [00:02<00:00, 12457424.40it/s] + 0%| | 32768/26421880 [00:00<02:05, 209450.57it/s] + 0%| | 65536/26421880 [00:00<02:06, 207572.68it/s] + 0%| | 131072/26421880 [00:00<01:27, 300950.17it/s] + 1%| | 229376/26421880 [00:00<01:01, 427461.71it/s] + 2%|1 | 491520/26421880 [00:00<00:29, 869131.19it/s] + 4%|3 | 950272/26421880 [00:00<00:16, 1556559.36it/s] + 7%|7 | 1933312/26421880 [00:01<00:07, 3073356.65it/s] + 15%|#4 | 3833856/26421880 [00:01<00:03, 5905149.97it/s] + 26%|##6 | 6881280/26421880 [00:01<00:01, 10049278.31it/s] + 38%|###7 | 9928704/26421880 [00:01<00:01, 12847665.35it/s] + 48%|####8 | 12746752/26421880 [00:01<00:00, 14324144.88it/s] + 60%|#####9 | 15826944/26421880 [00:01<00:00, 15871843.82it/s] + 71%|#######1 | 18874368/26421880 [00:02<00:00, 16872598.82it/s] + 83%|########2 | 21856256/26421880 [00:02<00:00, 17403201.35it/s] + 94%|#########4| 24838144/26421880 [00:02<00:00, 17815867.41it/s] +100%|##########| 26421880/26421880 [00:02<00:00, 11043343.33it/s] Extracting ../datasets/FashionMNIST/raw/train-images-idx3-ubyte.gz to ../datasets/FashionMNIST/raw Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/train-labels-idx1-ubyte.gz Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/train-labels-idx1-ubyte.gz to ../datasets/FashionMNIST/raw/train-labels-idx1-ubyte.gz 0%| | 0/29515 [00:00<?, ?it/s] -100%|##########| 29515/29515 [00:00<00:00, 205829.78it/s] -100%|##########| 29515/29515 [00:00<00:00, 205327.60it/s] +100%|##########| 29515/29515 [00:00<00:00, 184548.40it/s] +100%|##########| 29515/29515 [00:00<00:00, 184079.42it/s] Extracting ../datasets/FashionMNIST/raw/train-labels-idx1-ubyte.gz to ../datasets/FashionMNIST/raw Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/t10k-images-idx3-ubyte.gz Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/t10k-images-idx3-ubyte.gz to ../datasets/FashionMNIST/raw/t10k-images-idx3-ubyte.gz 0%| | 0/4422102 [00:00<?, ?it/s] - 1%| | 32768/4422102 [00:00<00:18, 233477.25it/s] - 1%|1 | 65536/4422102 [00:00<00:18, 232894.04it/s] - 3%|2 | 131072/4422102 [00:00<00:12, 339241.27it/s] - 5%|5 | 229376/4422102 [00:00<00:08, 480773.14it/s] - 10%|9 | 425984/4422102 [00:00<00:04, 811302.76it/s] - 20%|## | 884736/4422102 [00:00<00:02, 1643749.67it/s] - 39%|###9 | 1736704/4422102 [00:00<00:00, 3085506.46it/s] - 79%|#######8 | 3473408/4422102 [00:01<00:00, 6031981.20it/s] -100%|##########| 4422102/4422102 [00:01<00:00, 3921463.52it/s] + 1%| | 32768/4422102 [00:00<00:20, 209311.82it/s] + 1%|1 | 65536/4422102 [00:00<00:20, 208198.88it/s] + 3%|2 | 131072/4422102 [00:00<00:14, 303322.26it/s] + 5%|5 | 229376/4422102 [00:00<00:09, 431002.39it/s] + 10%|9 | 425984/4422102 [00:00<00:05, 726735.34it/s] + 20%|## | 884736/4422102 [00:00<00:02, 1469897.89it/s] + 39%|###9 | 1736704/4422102 [00:01<00:00, 2761359.22it/s] + 79%|#######8 | 3473408/4422102 [00:01<00:00, 5399936.71it/s] +100%|##########| 4422102/4422102 [00:01<00:00, 3500570.56it/s] Extracting ../datasets/FashionMNIST/raw/t10k-images-idx3-ubyte.gz to ../datasets/FashionMNIST/raw Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/t10k-labels-idx1-ubyte.gz Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/t10k-labels-idx1-ubyte.gz to ../datasets/FashionMNIST/raw/t10k-labels-idx1-ubyte.gz 0%| | 0/5148 [00:00<?, ?it/s] -100%|##########| 5148/5148 [00:00<00:00, 37947762.73it/s] +100%|##########| 5148/5148 [00:00<00:00, 38284179.06it/s] Extracting ../datasets/FashionMNIST/raw/t10k-labels-idx1-ubyte.gz to ../datasets/FashionMNIST/raw Pipeline CS: @@ -207,15 +207,14 @@ Pipeline Random Config: ________________________________________ Configuration(values={ - 'image_augmenter:GaussianBlur:sigma_min': 1.800750044920493, - 'image_augmenter:GaussianBlur:sigma_offset': 0.0008507475449754942, - 'image_augmenter:GaussianBlur:use_augmenter': True, + 'image_augmenter:GaussianBlur:use_augmenter': False, 'image_augmenter:GaussianNoise:use_augmenter': False, 'image_augmenter:RandomAffine:use_augmenter': False, - 'image_augmenter:RandomCutout:use_augmenter': False, + 'image_augmenter:RandomCutout:p': 0.34114189827681496, + 'image_augmenter:RandomCutout:use_augmenter': True, 'image_augmenter:Resize:use_augmenter': False, - 'image_augmenter:ZeroPadAndCrop:percent': 0.3938396231176561, - 'normalizer:__choice__': 'ImageNormalizer', + 'image_augmenter:ZeroPadAndCrop:percent': 0.17619897373538618, + 'normalizer:__choice__': 'NoNormalizer', }) Fitting the pipeline... @@ -223,7 +222,7 @@ ImageClassificationPipeline ________________________________________ 0-) normalizer: - ImageNormalizer + NoNormalizer 1-) preprocessing: EarlyPreprocessing @@ -288,7 +287,7 @@ print(pipeline) -

Total running time of the script: ( 0 minutes 7.321 seconds)

+

Total running time of the script: ( 0 minutes 7.995 seconds)

Out:

-
<autoPyTorch.api.tabular_classification.TabularClassificationTask object at 0x7f2407c75af0>
+
<autoPyTorch.api.tabular_classification.TabularClassificationTask object at 0x7f83c94a1970>
 

 

 
@@ -205,26 +205,26 @@ 
Print the final ensemble performanceOut:

-
{'accuracy': 0.8670520231213873}
-|    | Preprocessing                                                                         | Estimator                                                       |   Weight |
-|---:|:--------------------------------------------------------------------------------------|:----------------------------------------------------------------|---------:|
-|  0 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,MinMaxScaler,FastICA       | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential       |     0.56 |
-|  1 | SimpleImputer,Variance Threshold,MinorityCoalescer,OneHotEncoder,Normalizer,KernelPCA | embedding,ShapedResNetBackbone,FullyConnectedHead,nn.Sequential |     0.38 |
-|  2 | SimpleImputer,Variance Threshold,NoCoalescer,NoEncoder,StandardScaler,PCA             | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential       |     0.02 |
-|  3 | None                                                                                  | CBLearner                                                       |     0.02 |
-|  4 | None                                                                                  | SVMLearner                                                      |     0.02 |
+
{'accuracy': 0.8728323699421965}
+|    | Preprocessing                                                                              | Estimator                                                    |   Weight |
+|---:|:-------------------------------------------------------------------------------------------|:-------------------------------------------------------------|---------:|
+|  0 | None                                                                                       | CBLearner                                                    |     0.5  |
+|  1 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,MinMaxScaler,FastICA            | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential    |     0.22 |
+|  2 | SimpleImputer,Variance Threshold,MinorityCoalescer,OneHotEncoder,PowerTransformer,Nystroem | embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential |     0.14 |
+|  3 | SimpleImputer,Variance Threshold,NoCoalescer,NoEncoder,StandardScaler,PCA                  | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential    |     0.12 |
+|  4 | None                                                                                       | RFLearner                                                    |     0.02 |
 autoPyTorch results:
         Dataset name: Australian
         Optimisation Metric: accuracy
         Best validation score: 0.8713450292397661
-        Number of target algorithm runs: 27
-        Number of successful target algorithm runs: 26
+        Number of target algorithm runs: 23
+        Number of successful target algorithm runs: 22
         Number of crashed target algorithm runs: 0
         Number of target algorithms that exceeded the time limit: 1
         Number of target algorithms that exceeded the memory limit: 0
 

 

-
Total running time of the script: ( 5 minutes  24.577 seconds)

+
Total running time of the script: ( 5 minutes  27.372 seconds)

 
 
Out:

-
<autoPyTorch.api.tabular_regression.TabularRegressionTask object at 0x7f248d0d5d90>
+
<autoPyTorch.api.tabular_regression.TabularRegressionTask object at 0x7f8459f30d90>
 

 

 
@@ -199,25 +199,26 @@ 
Print the final ensemble performanceOut:

-
{'r2': 0.9407884171054208}
+
{'r2': 0.9412847640085195}
 |    | Preprocessing                                                                                    | Estimator                                                       |   Weight |
 |---:|:-------------------------------------------------------------------------------------------------|:----------------------------------------------------------------|---------:|
-|  0 | None                                                                                             | CBLearner                                                       |     0.44 |
-|  1 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential |     0.42 |
+|  0 | None                                                                                             | CBLearner                                                       |     0.46 |
+|  1 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential |     0.4  |
 |  2 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential |     0.1  |
-|  3 | None                                                                                             | LGBMLearner                                                     |     0.04 |
+|  3 | None                                                                                             | LGBMLearner                                                     |     0.02 |
+|  4 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential |     0.02 |
 autoPyTorch results:
-        Dataset name: 59922def-0351-11ed-8824-d5cce4119db9
+        Dataset name: ba73302f-0375-11ed-8828-9bcdcaaf1ae6
         Optimisation Metric: r2
-        Best validation score: 0.8670098636440993
-        Number of target algorithm runs: 24
-        Number of successful target algorithm runs: 22
+        Best validation score: 0.8669094525651709
+        Number of target algorithm runs: 22
+        Number of successful target algorithm runs: 20
         Number of crashed target algorithm runs: 0
         Number of target algorithms that exceeded the time limit: 2
         Number of target algorithms that exceeded the memory limit: 0
 

 

-
Total running time of the script: ( 5 minutes  36.793 seconds)

+
Total running time of the script: ( 6 minutes  2.422 seconds)

 

 

-
Total running time of the script: ( 1 minutes  3.199 seconds)

+
Total running time of the script: ( 1 minutes  6.152 seconds)