(v3.0.8) NormalizeAction Wrapper migration from Sinergym Environment Module

docs/source/pages/controllers.rst

@@ -13,9 +13,5 @@ an action to interact with the environment. You can develop your own
 controllers or modify current rules, for example. You can see an example of usage in 
 section :ref:`Rule Controller example`.
 
-Make sure that ``flag_normalization`` is ``False`` in the continuous environments. Otherwise, *Sinergym* will show an error
-due to the action space don't let you to work with the real values of building actuators (see :ref:`Normalization flag` 
-and :ref:`Rule Controller example`).
-
 .. warning:: You have to make sure that the variables used particularly for your controller 
              are part of the observation space of the configured environment.

docs/source/pages/environments.rst

@@ -374,7 +374,7 @@ space of the environment is being made. ``time_variables``, ``variables`` and ``
 This allows us to do a **dynamic definition** of spaces, *Sinergym* will adapt the building model.
 Observation space is created automatically, but action space must be defined in order to set up
 the range values supported by the Gymnasium interface in the actuators, or the number of discrete values if
-it is a discrete environment.
+it is a discrete environment (using the wrapper for discretization).
 
 Then, the argument called ``action_space`` defines this action space following the **gymnasium standard**.
 *EnergyPlus* simulator works only with continuous values, so *Sinergym* action space defined must be continuous
@@ -393,26 +393,6 @@ of all its benefits instead of using the *EnergyPlus* simulator directly, meanwh
 managed by **default building model schedulers** as mentioned. For more information, see the example of use 
 :ref:`Default building control setting up an empty action interface`.
 
-Normalization flag
-===================
-
-The argument called ``flag_normalization`` indicates whether action space specified will be normalized to
-``[-1,1]`` or not. Then, *Sinergym* will use the real space specified in **action_space** argument or this 
-normalized space depending on this flag value. This is done in order to make environments more generic 
-in DRL solutions.
-*Sinergym* **parse** these values to real action space defined in environment internally before to 
-send it to *EnergyPlus* Simulator by the API middleware.
-
-.. important:: The method in charge of parse this values from [-1,1] to real action space if it is required is 
-        called ``_action_transform(action)`` in *sinergym/sinergym/envs/eplus_env.py*.
-        We always recommend to use the normalization in action space for DRL solutions, since this space is 
-        compatible with all algorithms. However, if you are implementing your own rule-based controller 
-        and working with real action values, for example, you can deactivate normalization.
-
-.. note:: By default, all *Sinergym*'s environments will have normalization in action space.
-        It is possible to specify the **flag_normalization** to false in the constructor argument or
-        to change it during the execution using ``env.update_flag_normalization(False)``.
-
 Environment name
 ================
 

docs/source/pages/modules/sinergym.envs.eplus_env.EplusEnv.rst

@@ -22,7 +22,6 @@
       ~EplusEnv.render
       ~EplusEnv.reset
       ~EplusEnv.step
-      ~EplusEnv.update_flag_normalization
 
 
 

docs/source/pages/modules/sinergym.utils.wrappers.NormalizeAction.rst

@@ -0,0 +1,47 @@
+sinergym.utils.wrappers.NormalizeAction
+=======================================
+
+.. currentmodule:: sinergym.utils.wrappers
+
+.. autoclass:: NormalizeAction
+   :members:                                                           
+   :undoc-members:               
+
+
+   .. automethod:: __init__
+
+
+   .. rubric:: Methods
+
+   .. autosummary::
+
+      ~NormalizeAction.__init__
+      ~NormalizeAction.action
+      ~NormalizeAction.class_name
+      ~NormalizeAction.close
+      ~NormalizeAction.get_wrapper_attr
+      ~NormalizeAction.render
+      ~NormalizeAction.reset
+      ~NormalizeAction.reverting_action
+      ~NormalizeAction.step
+      ~NormalizeAction.wrapper_spec
+
+
+
+
+
+   .. rubric:: Attributes
+
+   .. autosummary::
+
+      ~NormalizeAction.action_space
+      ~NormalizeAction.logger
+      ~NormalizeAction.metadata
+      ~NormalizeAction.np_random
+      ~NormalizeAction.observation_space
+      ~NormalizeAction.render_mode
+      ~NormalizeAction.reward_range
+      ~NormalizeAction.spec
+      ~NormalizeAction.unwrapped
+
+

docs/source/pages/modules/sinergym.utils.wrappers.rst

@@ -25,6 +25,7 @@
       LoggerWrapper
       MultiObjectiveReward
       MultiObsWrapper
+      NormalizeAction
       NormalizeObservation
       OfficeGridStorageSmoothingActionConstraintsWrapper
       PreviousObservationWrapper

docs/source/pages/wrappers.rst

@@ -29,6 +29,30 @@ DatetimeWrapper
 Wrapper to substitute ``day_of_month`` value by ``is_weekend`` flag, and ``hour`` and ``month`` by sin and cos values. 
 Observation space is updated automatically.
 
+***********************
+NormalizeAction
+***********************
+
+Wrapper to apply normalization in action space. It is very useful in DRL algorithms such as some of them
+only works with normalized values correctly, making environments more generic in DRL solutions.
+
+By default, the normalization is applied in the range ``[-1,1]``. However, other **range** can be specified when wrapper
+is instantiated.
+
+*Sinergym* **parse** these values to real action space defined in original environment internally before to 
+send it to *EnergyPlus* Simulator by the API middleware.
+
+.. important:: The method in charge of parse this values from [-1,1] to real action space if it is required is 
+        called ``reverting_action(action)`` in the wrapper class.
+        We always recommend to use the normalization in action space for DRL solutions, since this space is 
+        compatible with all algorithms. However, if you are implementing your own rule-based controller 
+        and working with real action values, for example, you don't must to apply this wrapper.
+
+.. image:: /_static/normalize_action_wrapper.png
+  :scale: 50 %
+  :alt: Normalize action wrapper graph.
+  :align: center
+
 ***********************
 DiscretizeEnv
 ***********************

examples/change_environment.ipynb

@@ -421,7 +421,6 @@
             "source": [
                 "env = gym.make('Eplus-datacenter-cool-continuous-stochastic-v1', \n",
                 "                env_name='new_env_name',\n",
-                "                flag_normalization=False,\n",
                 "                max_ep_data_store_num=20\n",
                 "                )"
             ]

examples/drl.ipynb

@@ -265,8 +265,8 @@
                 }
             ],
             "source": [
-                "env = gym.make(environment, env_name=experiment_name, flag_normalization=False)\n",
-                "eval_env = gym.make(environment, env_name=experiment_name+'_EVALUATION', flag_normalization=False)"
+                "env = gym.make(environment, env_name=experiment_name)\n",
+                "eval_env = gym.make(environment, env_name=experiment_name+'_EVALUATION')"
             ]
         },
         {
@@ -276,7 +276,7 @@
                 "collapsed": false
             },
             "source": [
-                "We can also add a Wrapper to the environment, we are going to use a Logger (extension of `gym.Wrapper`) this is used to monitor and log the interactions with the environment and save the data into a CSV. Files generated will be stored as artifact in *wandb* too."
+                "We can also add a Wrapper to the environment, we are going to use an action normalization wrapper and a logger (extensions of `gym.Wrapper`). Normalization is very recommended in DRL algorithms with continuous action space and logger is used to monitor and log the interactions with the environment and save the data into a CSV. Files generated will be stored as artifact in *wandb* too."
             ]
         },
         {
@@ -299,7 +299,9 @@
                 }
             ],
             "source": [
+                "env = NormalizeAction(env)\n",
                 "env = LoggerWrapper(env)\n",
+                "eval_env = NormalizeAction(eval_env)\n",
                 "eval_env = LoggerWrapper(eval_env)"
             ]
         },
@@ -53804,7 +53806,7 @@
             "cell_type": "markdown",
             "metadata": {},
             "source": [
-                "We make the gymnasium environment and wrap with LoggerWrapper. We can use the evaluation experiment name to rename the environment."
+                "We make the gymnasium environment, it is **important to wrap the environment with the same wrappers as used in training**. We can use the evaluation experiment name to rename the environment."
             ]
         },
         {
@@ -53834,8 +53836,9 @@
                 }
             ],
             "source": [
-                "env=gym.make(environment, env_name=evaluation_name, flag_normalization=False)\n",
-                "env=LoggerWrapper(env)"
+                "env = gym.make(environment, env_name=evaluation_name)\n",
+                "env = NormalizeAction(env)\n",
+                "env = LoggerWrapper(env)"
             ]
         },
         {

examples/rule_controller_example.ipynb

@@ -44,9 +44,7 @@
                 "collapsed": false
             },
             "source": [
-                "Now, we can define the environment we want to use, in our case we are using the Eplus demo.\n",
-                "Don't forget to deactivate the `flag_normalization`. It is important due to the fact that *MyRuleBasedController*\n",
-                "works with real values, not `[-1,1]` space."
+                "Now, we can define the environment we want to use, in our case we are using the Eplus demo."
             ]
         },
         {
@@ -77,7 +75,7 @@
                 }
             ],
             "source": [
-                "env = gym.make('Eplus-5zone-hot-continuous-v1', flag_normalization=False)"
+                "env = gym.make('Eplus-5zone-hot-continuous-v1')"
             ]
         },
         {