22.2.0

Changelog

Mip-Algorithms:

• Creation of mipframework for simplifying writing of python algorithms.
• All SQL queries in framework are now handled by SQLAlchemy (no more vulnerable sql string manipulations).
• Creation of a small python API for HighCharts as well as a HighChart server in Flask for quickly testing them.
• Creation of runner written in python for running locally and debugging algorithms.
• Refactor of Pearson, PCA, LogisticRegression, CalibrationBelt and DescriptiveStatistics to work with framework.
• Adds very detailed logging for above mentioned algorithms (with file rotation).
• Complete test suite for all above mentioned algorithms. Each test suite contains:
• Thousands of automatically generated unit tests for all algorithm methods. This is the so called property-based tests, where random inputs with a lot of corner cases are generated en masse and some property (like no NaNs or infs in result) is verified. The library used is hypothesis.
• 80 algorithm correctness tests. These are full python and thus run much faster than the dockerized versions. Only one worker node is used with no privacy constraints.
• 15 federated tests. These are also full python, they run on 10 worker nodes with no privacy constraints. They are meant to test if the aggregation of local results into master node functions properly.
• 5 integration tests. These run in dockerized form together with Exareme. They are meant to test how the algorithms and exareme integrate correctly. They also should run without privacy constraints.
• Additionally, we created a separate suite of tests to test the privacy mechanism of each algorithm, when it runs in production environment (exareme+docker+privacy).
• New version of Descriptive Statistics algorithm where a single request is made even for multiple datasets. This will speed up loading time in Analysis tab. Old version still there as we decided that they should coexist for some time.
• Fix bug with LogisticRegression where a huge matrix was causing a memory error.
• Many more bugs were caught during testing.
• Add tox.ini file for automated testing
• Descriptive stats: Fix bug where patsy failed for some design matrices with no data at all
• Kaplan Meier bug fix.

Exareme:

• Refactor node handling when worker nodes are down.
• Automated deletion of temp algorithm folders after threshold.
• RMI timeout added when node communication is unresponsive.

22.1.1

Changelog

Mip-Algorithms:

• Small bug fix.

22.1.0

Changelog

Mip-Algorithms:

• Kaplan Meier with multiple levels added.

Exareme:

• Small fixes

22.0.2

Changelog:

• Encoding bug fixed to be able to have greek letters on the metadata.

22.0.1

Changelog:

Mip Algorithms:

• Calibration fix for privacy.

22.0.0

Changelog

Exareme:

• properties.json improved with labels.
• Deployment scripts refactored and improved.
• Improved documentation.
• Bug fixes.

Mip Algorithms:

• Calibration Belt added.
• Multiple Histograms added.
• PCA added.
• CART beta version added.
• Kaplan Meier beta version added.
• Formula added on specific algorithms.
• Tests structure refactored.
• Bug fixes.

v21.3.0

Changelog

Exareme:

• Consul refactored to include pathologies and handle error cases better.
• Multiple CSVs can now be imported from exareme. Documentation also added.
• New parameters in properties.json (valueMin, valueMax, valueEnumerations) for algorithm input restrictions.
• Better way of providing the SQL Query to the algorithms.
• Fixed the database constraints when creating the database.
• Deployment scripts refactored to be more user friendly.
• Bug fixes.

Mip Algorithms:

• Added Naive Bayes Standalone.
• Added multiple histograms algorithm for better front-end response time.
• Some algorithm outputs were beautified.
• Bug fixes.

v21.2.0

Changelog

Exareme:

• Fixes on deployment scripts.
• Fix for logs in portainer when an error happens.

Mip Algorithms:

• Changes to allow Logistic Regression to work on multinomial data once two values are filtered (as opposed to strictly binomial data)
• Changes to allow Pearson correlation to work with variables only (while still working with variables and covariables)
• T-test indepedent working with categorical group variable.

v21.1.0

Changelog

Algorithms:

• Fixed bug in LIST_DATASET.
• Logs in production refactored to show algorithm request and result.
• Beautification on all algorithm parameters.

Exareme:

• New lighter deployment process for local exareme installation.
• Improved Deployment Process for Federated Exareme.
• Bug fixing.

v21.0.0

Changelog

Algorithms:

• TTEST algorithms added.
• ID3 algorithm added.
• KMEANS algorithm added.
• Descriptive Statistics algorithm added.
• Algorithms outputs improved to include all possible visualizations.
• Bug fixing and refactoring.
• New tests added, possibility for automatic test creation also added.

Exareme:

• Multipathologies functionality added in exareme.
• New automated deployment process with Ansible scripts.
• Bootstrap refactoring and improvements.
• New output on exareme's error messages (Privacy Errors, User errors).
• Bug fixing and refactoring on the iterative algorithm execution.
• Small refactoring and bug fixing here and there.