Skip to content

This repository contains code written for Dr. Park's microgrid research at University of Colorado Denver. The purpose is to implement a JADE agent that receives near real-time data from a distributed system of power-grid control/monitoring devices. It should store the data in SQL and send it to a GUI that maps the entire system.

License

Notifications You must be signed in to change notification settings

jake-billings/research-microgrid

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

research-microgrid

screenshot

This repository contains code written for Dr. Park's microgrid research at University of Colorado Denver. The purpose is to implement a JADE agent that receives near real-time data from a distributed system of power-grid control/monitoring devices. It should store the data in SQL and send it to a GUI that maps the entire system.

I approached the problem as event-driven graph data. The power control/monitoring network is represented as a graph. The data is "pushed" to the "data agent" implemented in this repository by other agents.

Each data structure is abstracted in the "abs" package such that the code could theoretically be reused for an entirely different purpose. See Graph and MicrogridGraph.

Contributors

Each class in this repository is assigned a "primary" author via the @author annotation in the class documentation. This primary author should have his/her first and last name listed in the class documentation first. The primary author should be the "person to ask" about the class. If you have questions about a class, this is the person to contact. Other contributors should be listed later in the author list. Contributors should leave contact information here in the README so that future contributors can find it.

Name Primary Contribution Contact Information
Amine Sasse comporthandler package
Jake Billings agents,core,data,live,message,persistence packages [email protected], [email protected], [email protected]

Architecture

Data Flow

  1. Measured by Sender Agent, which interfaces with control hardware Hardware
  2. Sent via JADE INFORM to Receiver Agent
  3. State stored in Receiver agent
  4. Ticker event sends state from Receiver Agent to Frontend Client via Socket.io

class diagram See docs/Spec.docx for more documentation/explanation of the communication protocol. It is important to understand the architecture before working on this repository.

Data is represented as Plain Old Java Objects (POJO) and sents as externalized bytes (using interface Externalizable stored in JADE ACLMessages).

Data Flow

Implementation

Each level of the data flow runs separately. Somewhere, in another repository, there is code that runs on the hardware controller. It sends data via Serial. The agent MicrogridControllerSenderAgent uses the package comporthandler2 to read this data from serial, convert it to a Java object, externalize it, and send via a JADE message to the MicrogridReceiverAgent. The receiver agent reads the object and updates its internal state map of the power grid. Every second, the receiver agent sends its grid state via Socket.IO to the browser. The browser uses Javascript and the Library Vis.js to render the power grid in a current state.

Thus, to run the full system, you need a hardware controller, a sender PC, and a receiver PC.

Steps to Run (some may be omitted if using dummies for testing)

  1. For each PC, follow the installation instructions below.
  2. Run MicrogridReceiverAgent on a dedicated receiver PC.
  3. Open src/resources/frontend/index.html on the receiver PC.
  4. Update MicrogridControllerSenderAgent to point its AID at the dedicated receiver PC.
  5. Run the hardware controller, and connect it to the dedicated sender PC.
  6. Ensure that MicrogridControllerSenderAgent is reading from the correct comport associated with the controller.
  7. Start MicrogridControllerSenderAgent on the sender PC.
  8. Assuming there are no errors, within 30 seconds, data will appear in the browser on the reciever PC.

Many dummy classes are included so that this system can be tested on a single PC. Dummy sender agents send dummy data so that we can test without a real controller or separate sender pc.

Data Analysis (Excel)

If you use the DataLoggingReceiverAgent, all data received by the system will be stored in your local SQL database by the Hibernate JPA implementation. Data can be viewed, processed, filtered, and analyzed using Structured Query Language (SQL). I recommend reading an online tutorial on SQL before attempting to use it to analyze this data.

First, you need an SQL client. There is a command line client built into MySQL. Popular clients include: Microsoft Access, MySQLWorkbench, and DataGrip. I will be using DataGrip since it has an interface similar to that of IntelliJ.

  1. Connect to the local MySQL instance connecting in datagrip 1 connecting in datagrip 2

  2. Open the correct table for the data you want connecting in datagrip 3

  3. Open the console and build your SQL query; here's a template for selecting a single measurement at a single node:

SELECT timestamp,value
FROM microgrid.FloatMicrogridDatum
WHERE _id="microgrid-node-a-g-3"
ORDER BY timestamp ASC LIMIT 5000;

By convention, measurement id's are in the format {{node_id}}-{{measurement_type_id}}. See FloatMicrogridMeasurementType.java and BooleanMicrogridMeasurementType.java for measurement type ids. By convention, floating point id's are positive, and boolean id's are negative.

connecting in datagrip 4

  1. Export your data to CSV for analysis in external tools connecting in datagrip 5

  2. Open your CSV in an external tool and analyze it analyzing in excel

All measurements have the field timestamp. This is field is the number of milliseconds since January 1st, 1970 12:00am when the measurement was recorded. Subtracting the first timestamp from the rest will normalize this field to the number of ms since data recording started.

Data Analysis (R)

  1. R supports accessing MySQL directly, so work from the example script docs/exampleR.R. analyzing in R analyzing in R analyzing in R

Prerequisites

  1. You will need Administrator-level access to a computer
  2. You will need to have cloned this git repository using git clone
  3. You will need to have IntelliJ (or be able to translate my instructions from IntelliJ to whatever you're using)
  4. You will need JDK 1.8 (Not JDK <1.8. NOT JDK >=1.9. You need exactly JDK1.8 (See "Issues" section)
  5. You will need a database installed that is compatible with JDBC. The connector for MySQL is already in pom.xml, so I recommend MySQL.

Setup Javascript Frontend

  1. Install NodeJS Install Node
  2. Check that NodeJS is actually installed node --version, npm --version
  3. Using the NodeJS package manager, install a better package manager, called Yarn. npm i -g yarn
  4. Verify Yarn is installed using yarn -v Verify NPM and Yarn
  5. Move into the frontend project directory using cd src/main/resources/frontend
  6. Install the frontend dependencies using yarn install Yarn Install

Setup JADE/Java

  1. Resolve dependencies using maven (use IntelliJ to do this) mvn install might also work 1a. Open the Maven Toolkit Window Open the Maven Toolkit Window

1b. Open the Maven project by pressing the plus icon the selecting the pom.xml file 1c. Run Maven install to install all of the Java dependencies (JADE, Jackson, netty-socketio, etc...) Maven Install

1d. Run the JUnit tests using Maven to make sure everything installed properly. You should see 0 failures. However, if you see "Failures: n" where n is some number there is a code issue (not a dependency issue). If you don't get test results, then there's probably an issue resolving dependencies or with maven configuration that must be addressed. Maven Test

  1. Run JUnit Tests (again but using IntelliJ test runner instead of Maven) 2a. Open run configurations Open run configuration

2b. Create a new run configuration for the JUnit tests JUnit Run Config

2c. Using the new run configuration, run the JUnit tests to make sure all the tested code works. JUnit Run Config

  1. Create run configurations for the JADE agents 3a. Open run configurations Open run configuration

3b. Create a run configuration for the receiver agent java jade.Boot -gui -local-port 1100 ReceiverAgent:MicrogridReceiverAgent Run Config

JADE GUI

3c. Create a run configuration for the dummy sender agent (for testing purposes) java jade.Boot -gui -local-port 1100 DummySenderAgent:edu.ucdenver.park.microgrid.dummy.DummyMicrogridSenderAgent Run Config

3d. Create a run configuration for the receiver agent AND dummy sender agent (for testing purposes) java jade.Boot -gui -local-port 1100 DummySenderAgent:edu.ucdenver.park.microgrid.dummy.DummyMicrogridSenderAgent Run Config

JADE GUI

3e. Run each agent and ensure that it displays appropriately in the JADE GUI

  1. Database Setup (Required for Storing Data) 4a. Configure database address/credentials in config file. The info in src/main/resources/hibernate.cfg.xml must match your local db configuration. By default, we connect to a local MySQL server as root with no password. 4b. Ensure a database called microgrid exists in MySQL. To do this, run mysql in the terminal to enter the MySQL terminal. Then, run create database microgrid; in the MySQL terminal.

Goals

The scrolling graph may eventually look like this oscilloscope. oscilloscope

The graph should handle large networks. screenshot

Issues

  1. Inherited issue from netty: we use a library to run a socket.io connection to a web browser. This library relies on netty. In Java 9, the keyword "native" is reserved. This breaks behavior in netty. Until netty is fixed, we must use JDK 1.8 instead of 1.9.
  2. McAffee Firewall blocks JADE by default. I encountered this issue in the lab. Somebody installed McAffee security software on the lab computer. None of my JADE messages were getting through, and code changes didn't help. I saw connection timeout errors. Disabling the McAffee firewall fixed the issue.

About

This repository contains code written for Dr. Park's microgrid research at University of Colorado Denver. The purpose is to implement a JADE agent that receives near real-time data from a distributed system of power-grid control/monitoring devices. It should store the data in SQL and send it to a GUI that maps the entire system.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published