Merge pull request #96 from HealthyPear/feature-add_RF_for_energy

Improve models generation
cta-observatory · Apr 15, 2021 · 4928ae4 · 4928ae4
2 parents 166fbe0 + d10dac3
commit 4928ae4
Show file tree

Hide file tree

Showing 24 changed files with 1,305 additions and 592 deletions.
diff --git a/docs/mva/index.rst b/docs/mva/index.rst
@@ -10,20 +10,30 @@ Introduction
 classification problems. It is based on machine learning methods available in
 scikit-learn_. Internally, the tables are dealt with the Pandas_ Python module.
 
-For each type of camera a regressor/classifier should be trained. For both type of models
-an average of the image estimates is later computed to determine a global
-output for the event (energy or score/gammaness).
+For each type of camera a regressor/classifier should be trained.
+For both type of models an average of the image estimates is later computed to
+determine a global output for the event (energy or score/gammaness).
 
 Details
 -------
 
-Data is split in train and test subsamples by images.
+Data is split in train and test subsamples by single telescope images.
 
-The class `TrainModel` uses a training sample composed of gamma-rays for a
+The class ```TrainModel``` uses a training sample composed of gamma-rays for a
 regression model. In addition of a gamma-ray sample, a sample of
-protons is also used to build a classifier. The training of a model is done via
-the GridSearchCV_ algorithm which allows to find the best hyper-parameters of
-the models.
+protons is also used to build a classifier.
+
+The training of a model can be done also via the GridSearchCV_ algorithm which 
+allows to find the best hyper-parameters of the models.
+
+Supported models:
+
+- ``sklearn.ensemble.RandomForestClassifier``
+- ``sklearn.ensemble.RandomForestRegressor``
+- ``sklearn.ensemble.AdaBoostRegressor``
+
+For details about the generation of each model type, please refer to 
+:ref:`model_building`.
 
 Reference/API
 -------------

diff --git a/docs/scripts/DL2.rst b/docs/scripts/DL2.rst
@@ -15,14 +15,11 @@ By invoking the help argument, you can get help about how the script works:
 
 .. code-block::
 
-  usage: protopipe-DL2 [-h] --config_file CONFIG_FILE -o OUTFILE [-m MAX_EVENTS]
-                       [-i INDIR] [-f [INFILE_LIST [INFILE_LIST ...]]]
-                       [--cam_ids [CAM_IDS [CAM_IDS ...]]] [--wave_dir WAVE_DIR]
-                       [--wave_temp_dir WAVE_TEMP_DIR] [--wave | --tail]
-                       [--debug] [--regressor_dir REGRESSOR_DIR]
-                       [--classifier_dir CLASSIFIER_DIR]
-                       [--force_tailcut_for_extended_cleaning FORCE_TAILCUT_FOR_EXTENDED_CLEANING]
-                       [--save_images]
+  usage: protopipe-DL2 [-h] --config_file CONFIG_FILE -o OUTFILE [-m MAX_EVENTS] [-i INDIR] [-f [INFILE_LIST [INFILE_LIST ...]]]
+                     [--cam_ids [CAM_IDS [CAM_IDS ...]]] [--wave_dir WAVE_DIR] [--wave_temp_dir WAVE_TEMP_DIR] [--wave | --tail] [--debug]
+                     [--regressor_dir REGRESSOR_DIR] [--classifier_dir CLASSIFIER_DIR]
+                     [--force_tailcut_for_extended_cleaning FORCE_TAILCUT_FOR_EXTENDED_CLEANING] [--save_images]
+                     [--regressor_config REGRESSOR_CONFIG] [--classifier_config CLASSIFIER_CONFIG]
 
   optional arguments:
     -h, --help            show this help message and exit
@@ -35,11 +32,9 @@ By invoking the help argument, you can get help about how the script works:
                           give a specific list of files to run on
     --cam_ids [CAM_IDS [CAM_IDS ...]]
                           give the specific list of camera types to run on
-    --wave_dir WAVE_DIR   directory where to find mr_filter. if not set look in
-                          $PATH
+    --wave_dir WAVE_DIR   directory where to find mr_filter. if not set look in $PATH
     --wave_temp_dir WAVE_TEMP_DIR
-                          directory where mr_filter to store the temporary fits
-                          files
+                          directory where mr_filter to store the temporary fits files
     --wave                if set, use wavelet cleaning -- default
     --tail                if set, use tail cleaning, otherwise wavelets
     --debug               Print debugging information
@@ -50,3 +45,7 @@ By invoking the help argument, you can get help about how the script works:
     --force_tailcut_for_extended_cleaning FORCE_TAILCUT_FOR_EXTENDED_CLEANING
                           For tailcut cleaning for energy/score estimation
     --save_images         Save images in images.h5 (one file testing)
+    --regressor_config REGRESSOR_CONFIG
+                          Configuration file used to produce regressor model
+    --classifier_config CLASSIFIER_CONFIG
+                          Configuration file used to produce classification model
diff --git a/docs/scripts/data_training.rst b/docs/scripts/data_training.rst
@@ -19,15 +19,10 @@ By invoking the help argument, you can get help about how the script works:
 
 .. code-block::
 
-  usage: protopipe-TRAINING [-h] --config_file CONFIG_FILE -o OUTFILE
-                            [-m MAX_EVENTS] [-i INDIR]
-                            [-f [INFILE_LIST [INFILE_LIST ...]]]
-                            [--cam_ids [CAM_IDS [CAM_IDS ...]]]
-                            [--wave_dir WAVE_DIR]
-                            [--wave_temp_dir WAVE_TEMP_DIR] [--wave | --tail]
-                            [--debug] [--save_images]
-                            [--estimate_energy ESTIMATE_ENERGY]
-                            [--regressor_dir REGRESSOR_DIR]
+  usage: protopipe-TRAINING [-h] --config_file CONFIG_FILE -o OUTFILE [-m MAX_EVENTS] [-i INDIR] [-f [INFILE_LIST [INFILE_LIST ...]]]
+                          [--cam_ids [CAM_IDS [CAM_IDS ...]]] [--wave_dir WAVE_DIR] [--wave_temp_dir WAVE_TEMP_DIR] [--wave | --tail]
+                          [--debug] [--save_images] [--estimate_energy ESTIMATE_ENERGY] [--regressor_dir REGRESSOR_DIR]
+                          [--regressor_config REGRESSOR_CONFIG]
 
   optional arguments:
     -h, --help            show this help message and exit
@@ -40,20 +35,19 @@ By invoking the help argument, you can get help about how the script works:
                           give a specific list of files to run on
     --cam_ids [CAM_IDS [CAM_IDS ...]]
                           give the specific list of camera types to run on
-    --wave_dir WAVE_DIR   directory where to find mr_filter. if not set look in
-                          $PATH
+    --wave_dir WAVE_DIR   directory where to find mr_filter. if not set look in $PATH
     --wave_temp_dir WAVE_TEMP_DIR
-                          directory where mr_filter to store the temporary fits
-                          files
+                          directory where mr_filter to store the temporary fits files
     --wave                if set, use wavelet cleaning -- default
     --tail                if set, use tail cleaning, otherwise wavelets
     --debug               Print debugging information
     --save_images         Save also all images
     --estimate_energy ESTIMATE_ENERGY
-                          Estimate the events' energy with a regressor from
-                          protopipe.scripts.build_model
+                          Estimate the events' energy with a regressor from protopipe.scripts.build_model
     --regressor_dir REGRESSOR_DIR
                           regressors directory
+    --regressor_config REGRESSOR_CONFIG
+                          Configuration file used to produce regressor model
 
 The configuration file used by this script is ``analysis.yaml``,