(v3.0.0) - Energyplus Python API integrated as Sinergym simulator eng…

…ine (#351)

* Dockerfile: Deleted BCVTB dependency in container and added pyenergyplus package in PYTHONPATH

* Version of Sinergym updated from 2.5.2 to 3.0.0

* Modeling: Deleting and updating unusued methods in abstract class

* Modeling: ModelJSON class constructor update; Deleting variables XML tree and action definition functionality, checkert observation variables deleted too

* Modeling: adapt_variables_to_cfg_and_building, apply_action_definition and save_variables_cfg deleted. These functionalities will not be required anymore

* Modeling: Update _get_eplus_run_info to _get_eplus_runperiod and deleted get_current_time_info (it could be calculated with E+ API). _get_one_epi_len is now _get_runperiod_len

* Modeling: Set set_experiment_working_dir as private method (only called once in ModelJSON constructor)

* Modeling: Deleted action definition asserts in ModelJSON checker and updating class properties

* Simulator: Simulator PARADIGM change. Deleted BCVTB structure and integrating E+ API, threading structure to stop the simulation step by step with queues

* Common: Create_variable_weather deleted

* Common: Updated name export_actuators_to_excel to export_schedulers_to_excel

* Deleted sinergym/data/variables (RDD files is not required with E+ API)

* Environment: Updated import module dependencies

* Environment: Updating Sinergym constructor; observation and action variables subtituted by time_variables,variables,meters and actuators, observation_space is not neccesary to be specified, model and simulator are attributes of the environment, queues to simulator E+ API communication are defined

* Environment: Reset update. Adapting to new simulator structure.

* Environment: Step update. Adapting to new simulator structure.

* Environment: close update. Adapting to new simulator structure.

* Environment: Updating sinergym env checker and deleting get_schedulers method

* Environment: Updating Sinergym env properties, in order to extract all useful information from simulator and model layers directly

* Reward: Fix obs_dict key name (day is now day_of_month)

* utils.constants: Updating terminal logger format and stablishing a level for each layer

* Updating terminal logger in environment layer and added properties to the class

* Simulator: Configured Thread with daemon True value, added empty print in order to not overwrite progress bar

* Fixed simulation complete if statement when we early stop an episode from environment

* Simulator: Added code to write handles data available when simulator initialize handles (impossible to do it before)

* Simulator: Updated and configured the terminal logger in simulator layer

* Modeling: Updated and configured the terminal logger in modeling layer

* changing error message by warning message when action in step is not contained in environment action space.

* Added message colors in Sinergym terminal logger

* [Simulator] Deleted unused progress_value attribute

* Set up env, sim and modeling loggers as class attribute

* Simulator: Changed flags to system ready and reformulated in callbacks

* Simulator: Commentted debug simulator in queues states

* Simulator: Update start() and stop() functionality in order to initilize handles only once. Flags must be stablished correctly in stop method

* Environment: Updated constructor and reset method to adapt to new simulator structure

* [ci-skip] Fixed timestep and terminated flag issue

* Merging obs and info queue write in the same energyplus callback

* Simulator: Fixed assertion error bug for actuators

* Environemnt: Action sent to the simulator annotated in info dict

* Simulator: Improved progress bar

* Environment: Updated docstring, fixed some pytype and improved terminal logs

* Environment: Updated name energyplus_simulation to energyplus_simulator

* Simulator: docstring updated

* Modeling: docstring updated

* Added is_running property to environment and simulator (equivalent to episode existed)

* Wrappers: Adapted wrappers and improvement of terminal logs added

* Logger: Fixed CSVLogger new info key in time_elapsed

* Utils: Adapted callbacks attributes and fixed wrapper time key

* Utils: Adapted controllers to new environment attributes

* Utils: Fixed evaluation.py

* Fixed day of month key in reward time variable

* 5Zone: Deleted default variables and meters in epJSON file

* wrappers: Fixed datetimewrapper logger name

* Modeling: Added meters, variables and actuators in constructor and created adaptation for Output:variable and output:meter values method

* Env: Adapted modeling changes for adaptation of variables end meters

* Deleted pandas module in modeling module

* environments registration automatized

* Modeling: Fixed reference bug in logger

* data: Deleted all output:variable and output:meter elements from buildings epJSON baseline

* Environment: Fixed assert in env_checker

* register: Fixed some bugs and updated reward arguments

* Constants: Adapted constants default values to new structure

* Improved terminal log format

* Simulator: Deleted base.py and eplus_alpha.py (not used)

* Simulator: Fixed progress bar to 101% in some environments

* Rewards: Deleted dependency in rewards. Rewards doesn't require year as observation variables now

* tests: Deleted EPLUS and BCVTB path, it is not required anymore

* register: Fixed weather specification in Sinergym default environments

* Env: Added readapt building to config method call from modeling in reset

* Modeling: Deleted skeleton class in modeling

* Modeling: Updated order of methods and header names, adapt_building_to_conf update runperiod attribute, _get_runperiod_len now uses runperiod attribute instead of recalculate runperiod

* Tests: Updated tests of modeling part, improved some tests too

* Wrappers: Fixed day key (day_of_month) in DateTimeWrapper and setpoints initial values argument added in DiscreteIncrementalWrapper

* Tests: Updated common tests

* Tests: Updated conftest fixtures and simplified their definitions

* Tests: Updated fixtures names in common tests

* tests: Adapted reward tests

* Tests: Created simulator tests from scratch (it is not similar at all compared with old simulator)

* tests: Adapted all environment tests

* Env: Fixed observation space for officegrid (hardcoded a custom observation space, this will be fixed when we can read ranges of variables) and increased general observation space (shop failed)

* tests: Fixed fixture name bug

* Controller: Fixed controllers bugs

* Wrappers and tests: Fixed wrappers bugs and adapted tests

* Updated environments ranges and tests

* Fixed environment test bug

* Tests: Stable baselines 3 tests updated

* Added test simulator output to conftests autoremove

* Fixed pytype errors

* Deleted unusued import in Sinergym project and isort applied

* Env: Added timestep_per_episode property

* Scripts: Updated and fixed some bugs in train and load agents scripts

* Scripts: Fixed more bugs in train/load scripts and added an exception management in order to delete remote container if the script is executing there and an error happens

* Scripts: Fixed more bugs

* Scripts: Avoid Sinergym logger and sb3 logger same name

* Examples: Updated Code and text for the Sinergym examples notebooks

* Env: indicating in reset the environment name in terminal logger (usefull when a machine is running several envs at the same time)

* Simulator: Deleted year key in info dict in queue

* Env: indicating the environemnt name in close method for terminal logger (usefull when a machine is running several envs at the same time)

* Modeling: Improved some logger terminal messages

* Added name argument in simulator constructor and fixed queue get in environment

* Evaluation: Updated function to evaluate a policy

* Callbacks: Updated LoggerEvalCallback, deleted dummyvecenv wrap to avoid automatic resets in evaluations

* scripts: Fixed env name in JSON and deleted MOnitor wrap and fixed eval freq timesteps in train script

* Examples: Updated notebooks outputs

* JSON and pytype fixes

* Doc: Updated modules headers for API Reference

* Doc: Updated introduction

* Doc: Updated nav bar colors

* Doc: update installation section

* Doc: Updated some diagrams

* Doc: Added getting_env_information notebook to the documetation

* Doc: Updated building section, improved introduction and usage example sections

* Doc: Updated weather section

* Doc: Updated environments section

* Doc: Deleted normalize observation note in environments section and added in wrappers section

* Documentation: General improvements and fixed some mistakes

* Container: Added docs to the container in order to compile documentation in github workflows

* Github actions: Using Dockerfile in compilation of the documentation when we crate a PR

* Documentation: Fixed spelling errors

* Github actions: Using Dockerfile in compilation of the documentation when we merge a PR

* Documentation: Updated README.MD

* Updated docstrings in env, modeling and simulator

.github/workflows/create_pr.yml

@@ -59,23 +59,15 @@ jobs:
           filters: |
             doc:
               - 'docs/source/**'
-      - name: Upgrade pip
-        if: steps.verify-documentation-update.outputs.doc == 'true'
-        run: python -m pip install --upgrade pip
-      - name: Install pandoc dependencies
-        if: steps.verify-documentation-update.outputs.doc == 'true'
-        run: sudo apt install -y pandoc
-      - name: Install dependencies [sinergym extras]
+      - name: Build the latest Docker image
         if: steps.verify-documentation-update.outputs.doc == 'true'
-        run: |
-          pip install -e .[extras] &&
-          pip install --upgrade requests
+        run: docker build . --file Dockerfile --build-arg SINERGYM_EXTRAS=[extras] --tag pullrequest/sinergym:latest
       - name: Compile documentation
         if: steps.verify-documentation-update.outputs.doc == 'true'
-        run: cd docs && sphinx-build -M html source build
+        run: docker run -t pullrequest/sinergym:latest /bin/bash -c 'sphinx-build -M html docs/source docs/build'
       - name: Check sphinx spelling
         if: steps.verify-documentation-update.outputs.doc == 'true'
-        run: cd docs && sphinx-build -M spelling source build
+        run: docker run -t pullrequest/sinergym:latest /bin/bash -c 'sphinx-build -M spelling docs/source docs/build'
 
   tests:
     name: build container and execute pytest

.github/workflows/merge_pr.yml

@@ -61,23 +61,15 @@ jobs:
           filters: |
             doc:
               - 'docs/source/**'
-      - name: Upgrade pip
-        if: steps.verify-documentation-update.outputs.doc == 'true'
-        run: python -m pip install --upgrade pip
-      - name: Install pandoc dependencies
-        if: steps.verify-documentation-update.outputs.doc == 'true'
-        run: sudo apt install -y pandoc
-      - name: Install dependencies [sinergym extras]
+      - name: Build the latest Docker image
         if: steps.verify-documentation-update.outputs.doc == 'true'
-        run: |
-          pip install -e .[extras] &&
-          pip install --upgrade requests
+        run: docker build . --file Dockerfile --build-arg SINERGYM_EXTRAS=[extras] --tag pullrequest/sinergym:latest
       - name: Compile documentation
         if: steps.verify-documentation-update.outputs.doc == 'true'
-        run: cd docs && rm -R compilation/ && sphinx-multiversion source compilation
+        run: docker run -t pullrequest/sinergym:latest /bin/bash -c 'sphinx-multiversion docs/source docs/compilation'
       - name: Check sphinx spelling
         if: steps.verify-documentation-update.outputs.doc == 'true'
-        run: cd docs && sphinx-build -M spelling source compilation
+        run: docker run -t pullrequest/sinergym:latest /bin/bash -c 'sphinx-build -M spelling docs/source docs/compilation'
       - name: Pull local repository (needed if format commit has been done before)
         if: steps.verify-documentation-update.outputs.doc == 'true'
         run: git pull origin main

Dockerfile

@@ -35,13 +35,9 @@ ENV ENERGYPLUS_DOWNLOAD_BASE_URL https://github.com/NREL/EnergyPlus/releases/dow
 ENV ENERGYPLUS_DOWNLOAD_FILENAME EnergyPlus-$ENERGYPLUS_VERSION-$ENERGYPLUS_SHA-Linux-Ubuntu22.04-x86_64.sh 
 ENV ENERGYPLUS_DOWNLOAD_URL $ENERGYPLUS_DOWNLOAD_BASE_URL/$ENERGYPLUS_DOWNLOAD_FILENAME
 
-# Collapse the update of packages, download and installation into one command
-# to make the container smaller & remove a bunch of the auxiliary apps/files
-# that are not needed in the container
-ENV BCVTB_PATH=/usr/local/bcvtb
-
 RUN apt-get update && apt-get upgrade -y \
     && apt-get install -y ca-certificates curl libx11-6 libexpat1 \
+    && apt-get install -y git wget \
     #Energyplus installation
     && curl -SLO $ENERGYPLUS_DOWNLOAD_URL \
     && chmod +x $ENERGYPLUS_DOWNLOAD_FILENAME \
@@ -51,12 +47,6 @@ RUN apt-get update && apt-get upgrade -y \
     && rm -rf PostProcess/EP-Compare PreProcess/FMUParser PreProcess/ParametricPreProcessor PreProcess/IDFVersionUpdater \
     # Remove the broken symlinks
     && cd /usr/local/bin find -L . -type l -delete \
-    # BCVTB installation
-    && echo "Y\r" | apt-get install default-jre openjdk-8-jdk \ 
-    && apt-get install -y git wget iputils-ping \
-    && wget http://github.com/lbl-srg/bcvtb/releases/download/v1.6.0/bcvtb-install-linux64-v1.6.0.jar \
-    && yes "1" | java -jar bcvtb-install-linux64-v1.6.0.jar \
-    && cp -R 1/ $BCVTB_PATH && rm -R 1/ \
     # Install pip, and make python point to python3
     && apt install python3-pip -y \
     && ln -s /usr/bin/python3 /usr/bin/python \
@@ -67,6 +57,9 @@ RUN apt-get update && apt-get upgrade -y \
     && apt-get autoremove -y && apt-get autoclean -y \
     && rm -rf /var/lib/apt/lists/* 
 
+# Python add pyenergyplus path in order to detect API package
+ENV PYTHONPATH="/usr/local/EnergyPlus-${ENERGYPLUS_INSTALL_VERSION}"
+
 WORKDIR /sinergym
 COPY requirements.txt .
 COPY MANIFEST.in .
@@ -75,6 +68,7 @@ COPY scripts /sinergym/scripts
 COPY sinergym /sinergym/sinergym
 COPY tests /sinergym/tests
 COPY examples /sinergym/examples
+COPY docs/source /sinergym/docs/source
 RUN pip install -e .${SINERGYM_EXTRAS}
 
 #RUN pip install idna && pip install six

README.md

@@ -55,66 +55,67 @@ Please, help us to improve by **reporting your questions and issues** [here](htt
 
 The main functionalities of *Sinergym* are the following :
 
--  **Include different simulation engines**. Communication between
+-  **Compatibility with simulation engines**. Communication between
    Python and [EnergyPlus](https://energyplus.net/) is established
-   using [BCVTB](https://simulationresearch.lbl.gov/bcvtb/FrontPage) middleware.
-   Since this tool allows for interacting with several simulation
-   engines, more of them (e.g.
+   using [Energyplus Python API](https://energyplus.readthedocs.io/en/latest/api.html) as a middleware.
+   However, more of them (e.g.
    [OpenModelica](https://openmodelica.org/)) could be included in
-   the backend while maintaining the Gymnasium API.
+   the backend while maintaining the Gymnasium API in the future.
 
 -  **Benchmark environments**. Similarly to *Atari* or *Mujoco* environments
    for RL community, we are designing a set of environments for
    benchmarking and testing deep RL algorithms. These environments may
    include different buildings, weathers, action/observation spaces, function rewards, etc.
 
 -  **Customizable environments**. We aim to provide a
-   package that allows to modify experimental settings in an easy
-   manner. The user can create his own environments defining his own
-   building model, weather, reward, observation/action space and variables, environment name, etc.
+   package which allows modifying experimental settings in an easy
+   manner. The user can create his own environments, combining his own
+   building model, weather, reward, observation/action space, variables, actuators, environment name, etc.
    The user can also use these pre-configured environments available in *Sinergym* 
    and change some aspect of it (for example, the weather) in such 
    a way that he does not  have to make an entire definition of the 
    environment and can start from one pre-designed by us.
    Some parameters directly associated with the simulator can be set as **extra configuration** 
    as well, such as people occupant, time-steps per simulation hour, run-period, etc.
 
--  **Customizable components**: *Sinergym* is easily scalable by third parties.
+-  **Customizable components**. *Sinergym* is easily scalable by third parties.
    Following the structure of the implemented classes, new custom components 
    can be created for new environments such as function rewards, wrappers,
    controllers, etc.
 
--  **Automatic Building Model adaptation to user changes**: Building models (*IDF*) will be
-   adapted to specification of each simulation by the user. For example, ``Designdays`` and 
-   ``Location`` components from *IDF* files will be adapted to weather file (*EPW*) specified in
-   *Sinergym* simulator backend without any intervention by the user (only the environment definition).
-   *BCVTB middleware* external interface in *IDF* model and *variables.cfg* file is generated when 
-   simulation starts by *Sinergym*, this definition depends on action and observation space and variables defined.
-   In short, *Sinergym* automates the whole process of model adaptation so that the user 
+-  **Automatic Building Model adaptation to user changes**. Many of the updates to the environment definition require changes 
+   to the building model (*epJSON* file) to adapt it to these new features before the simulation starts, which *Sinergym* will 
+   perform automatically. For example, using another weather file requires building location and design days update, using new 
+   observation variables requires to update the `Output:Variable` and `Output:Meter` fields, the same occurs with extra 
+   configuration context concerned with simulation directly, if weather variability is set, then a weather with noise 
+   will be used. These new building and weather file versions, is saved in the *Sinergym* output folder, leaving the original 
+   intact. In short, *Sinergym* automates the whole process of model adaptation so that the user 
    only has to define what he wants for his environment.
 
--  **Automatic external interface integration for actions**. Sinergym provides functionality to obtain information 
-   about the environments such as the zones or the schedulers available in the environment model. Using that information,
-   which is possible to export in a excel, users can know which controllers are available in the building and, then, control 
-   them with an external interface from an agent. To do this, users will make an **action definition** in which it is
-   indicated which default controllers they want to replace in a specific format and *Sinergym* will take care of the relevant internal 
-   changes in the model.
+-  **Automatic actuators control**. Related to the above, it will only be necessary to specify the name of the actuators to be controlled 
+   through the actions of the Gymnasium interface, and *Sinergym* will take care of everything.
+
+-  **Extensive environment information**. It is important that users can get some information about *Sinergym* background components from environment interface easily.
+   From environment instance, it is possible to consult available schedulers, variables which compose an observation and action, whether simulator is running,
+   the building run period, episode length, timesteps per episode, available building zones... And much more.
 
 -  **Stable Baseline 3 Integration**. Some functionalities like callbacks
    have been customized by our team in order to test easily these environments
-   with deep reinforcement learning algorithms. 
-   This tool can be used with any other DRL library that supports the * Gymnasium* interface as well.
+   with deep reinforcement learning algorithms and logger specific information about 
+   *Sinergym* environments. 
+   However, *Sinergym* is completely agnostic to any DRL algorithm and can be used with any DRL 
+   library that works with gymnasium interface.
+
+-  **Google Cloud Integration**. Whether you have a Google Cloud account and you want to
+   use your infrastructure with *Sinergym*, we tell you some details about how to do it.
 
--  **Weights & Biases tracking and visualization**. One of Sinergym's objectives is to automate
+-  **Weights & Biases tracking and visualization compatibility**. One of *Sinergym*'s objectives is to automate
    and facilitate the training, reproducibility and comparison of agents in simulation-based 
-   building control problems, managing and monitoring model lifecycle from training to deployment. [WandB](https://wandb.ai/site) 
+   building control problems, managing and monitoring model lifecycle from training to deployment. [WandB](https://wandb.ai/site)
    is an open-source platform for the machine learning lifecycle helping us with this issue. 
    It lets us register experiments hyperparameters, visualize data recorded in real-time, 
    and store artifacts with experiment outputs and best obtained models. 
 
--  **Google Cloud Integration**. Whether you have a Google Cloud account and you want to
-   use your infrastructure with *Sinergym*, we tell you some details about how to do it.
-
 -  **Notebooks examples**. *Sinergym* develops code in notebook format with the purpose of offering use cases to 
    the users in order to help them become familiar with the tool. They are constantly updated, along with the updates 
    and improvements of the tool itself.

docs/source/_templates/layout.html

@@ -0,0 +1,16 @@
+{% extends "!layout.html" %}
+{% block footer %} {{ super() }}
+
+<style>
+    /* Sidebar header (and topbar for mobile) */
+    .wy-side-nav-search,
+    .wy-nav-top {
+        background: #a5beba;
+    }
+
+    /* Sidebar */
+    .wy-nav-side {
+        background: #2b3435;
+    }
+</style>
+{% endblock %}

docs/source/conf.py

@@ -97,7 +97,9 @@
 html_favicon = '_static/logo-sidebar.png'
 
 # Change sidebar-logo background
-html_theme_options = {'style_nav_header_background': '#a9c1be'}
+html_theme_options = {'style_nav_header_background': '#a5beba',
+                      'display_version': True,
+                      }
 
 # Enable global sidebar
 html_sidebars = {'**': ['globaltoc.html',

docs/source/index.rst

@@ -7,16 +7,16 @@ Welcome to **sinergym**!
 Contributing
 ############
 
-For all those interested in improving Sinergym, there are always improvements to be made. 
-You can check `issues <https://github.com/ugr-sail/sinergym/issues>`__ in the repo.
+For all those interested in improving *Sinergym*, there are always improvements to be made. 
+You can check `issues <https://github.com/ugr-sail/sinergym/issues>`__ in the repository.
 
 If you want to contribute, please read `CONTRIBUTING.md <https://github.com/ugr-sail/sinergym/blob/main/CONTRIBUTING.md>`__ first.
 
 ########################
 Projects using Sinergym
 ########################
 
-The following are some of the projects benefiting from the advantages of Sinergym:
+The following are some of the projects benefiting from the advantages of *Sinergym*:
 
 - `Demosthen/ActiveRL <https://github.com/Demosthen/ActiveRL>`__
 - `VectorInstitute/HV-Ai-C <https://github.com/VectorInstitute/HV-Ai-C>`__
@@ -31,13 +31,13 @@ If you want to appear in this list, do not hesitate to send us a PR and include
 Examples
 ############
 
-The examples can be run if you have your computer or container properly configured (see :ref:`Installation` section) from our notebooks hosted in the `examples <https://github.com/ugr-sail/sinergym/tree/main/examples>`__ folder of the official Sinergym repository.
+The examples can be run if you have your computer or container properly configured (see :ref:`Installation` section) from our notebooks hosted in the `examples <https://github.com/ugr-sail/sinergym/tree/main/examples>`__ folder of the official *Sinergym* repository.
 
 ################
 Citing Sinergym
 ################
 
-If you use Sinergym in your work, please cite our `paper <https://dl.acm.org/doi/abs/10.1145/3486611.3488729>`__::
+If you use *Sinergym* in your work, please cite our `paper <https://dl.acm.org/doi/abs/10.1145/3486611.3488729>`__::
 
     @inproceedings{2021sinergym,
       title={Sinergym: A Building Simulation and Control Framework for Training Reinforcement Learning Agents}, 
@@ -87,6 +87,7 @@ If you use Sinergym in your work, please cite our `paper <https://dl.acm.org/doi
    :caption: Examples
 
    pages/notebooks/basic_example.nblink
+   pages/notebooks/getting_env_information.nblink
    pages/notebooks/change_environment.nblink
    pages/notebooks/default_building_control.nblink
    pages/notebooks/wrappers_examples.nblink