Releases: cropsinsilico/yggdrasil
Releases · cropsinsilico/yggdrasil
New map transformation, composite function wrapping example, and various refactoring of internal methods
New map transformation, composite function wrapping example, and various refactoring of internal methods
Latest
- Added map transformation for converting arrays into map types
- Added support for automatted wrapping of C++ model functions that have arguments with types of std::string, std::vector, or std::map.
- Added composite_function example of wrapping multiple inputs of different types to a function as separate channels and composite_function2 example
of wrapping multiple inputs & outputs to a function as single channels - Added 'count' FileComm parameter to specify the number of times a file should be read
- Refactor geometry serialization classes to use rapidjson methods
- Refactor how additional variables created during function wrapping are handled
- Updated C++ interface to use updated rapidjson that requires allocators for yggdrasil types
- Allow use of with_asan with CMake/Make models
Added support for PyTorch models, sequence files, image files, and Excel files
- Added support for running trained PyTorch models as part of integrations
- Added support for connecting integrations to sequence files in FASTA and FASTQ formats via BioPython
- Added support for connecting integrations to sequence files in SAM, BAM and CRAM formats via pysam
- Added support for connecting integrations to sequence files in BCF and VCF formats via pysam
- Added support for connecting integrations to JPEG, PNG, EPS, BMP, GIF and TIFF formats via Pillow
- Added support for connecting integrations to Excel files via Pandas
- Refactor serialization in preparation of moving communication into C++
- Added CLI for converting files between compatible formats
- Added support for creating Ply and ObjWavefront objects from explicit meshes
- Fix bug in R interface
Migrate to using extended version of rapidjson/python-rapidjson for serialization and add support for models written in Julia
- Added Julia interface with units implemented via Unitful
- Added Julia versions of examples
- Dropped support for Python 3.5
- Changed rapidjson submodule to extended version
- Vendor python-rapidjson extension
- Removed jsonschema requirement
- Updated yggdrasil to use the vendored rapidjson for serialization, validation, normalization, and units
- Removed majority of datatypes submodule. rapidjson should be used for type validation, normalization, and serialization
- Removed C and C++ API for AsciiFile and AsciiTable
- Changed how tables are parsed such that rows in single column tables will be read as arrays unless otherwise specified via the YAML option 'transform: select_scalar'
- Updated utility scripts and package parameters to build extension
- Changed native string types in languages to be used for JSON string values
- In C, C++, and Fortran, if Python objects will be sent/received the model must initialize Python via "init_python_API()" (a subroutine in Fortran).If a model is a wrapped function, this will be added automatically.
- Change all JSON serialization to use rapidjson, removing JSONDecoder, JSONEncoder, & JSONEncoderReadable in the process.
- C++ interface now expects C++ classes for generic, array, object, ply, & obj types. Send methods should pass pointers to these objects. Formatted_io examples for these types have been updated to reflect these changes.
- Unpinned libroadrunner dependency for SBML
- Removed the 'serializer_class' and 'serializer_kwargs' from accepted communication keyword arguments. Serializer classes can be accessed via the seritype string associated with them in the schema and keyword arguments for serializer construction can be passed in a dictionary via 'serializer'.
- Removed 'typedef' attribute from serializer class, 'datatype' should be used instead
- Removed use of 'header_kwargs' keyword argument from serializer 'serialize' method. Use 'metadata' instead.
Bug fixes for Mac M1 chips (arm64) and various improvements to CLI
- Fix bug where yggcompile was compiling some languages multiple times
- Allow yggcompile to be called like yggcc
- Add flag (--with-asan) and YAML option (with_asan) for compiltion with the address sanitizer turned on
- Add flag (--disable-python-c-api) and YAML option (disable_python_c_api) for disabling the Python C API
- Allow for 128 bit types to be absent
- Generalize option for running with a debugging tool via the --with-debugger flag for yggrun or with_debugger YAML option
- Modify verification of file descriptor cleanup in tests to allow for persistence of a socket file descriptor if the socket is closed
- Fix bug in connection tracking of connected models that sometimes prevented integrations from exiting for connections receiving from more than one
model - Update setup script, including option specifically to set up a BioCro-OSR integration environment
- Migrate creation of coveragerc into CLI
- Fix bug where configuration was not completed after call to install from CLI, resulting in second call to install at next import
- Make GNU linkers explicit for simplicity of inheritance from LDLinker
- Allow compilation flag options to be passed to OSR (requires updated version of yggdrasil-enabled OSR)
- Always run reticulate::py_config before importing R source code to prevent segfault
- Allow R interpreter to be specified on the command line during install CLI and use that during dev setup
- Fix bug in Mac SDKROOT configuration where it was being overwritten and stored more than once
- Fix bug where '=' in requirement was interpreted as separator in .ini config parser by changing to use JSON to store the extra requirements
- Change sprintf to snprintf and fix display of bytes when there is no terminating character
- Add CLI utility for installing packages from conda recipes
- Handle conda bug where removing env does not remove the directory and ensure that yggdrasil always installed in a fresh environment
- Clean up leaks in temporary variables
- Add emacs and pre-commit to dev dependencies
- Short cut checking for R dependencies by just checking for the interface
- Update XML for OSR example
- Change deprecated calls to pandas DataFrame.append to concat in TimeSyncModelDriver
Update CI to use mamba, improve dev utilities, & various bug fixes
- Update utility scripts so that requirements can be maintained in a single file with pip requirements.txt file and conda recipe generated via
utils/manage_requirements.py - Consolidate CLI utilities for creating environments
- Add cron job to build docker images periodically to ensure that there is always one available
- Migrate CI to use mamba instead of conda for improved performance
- Update documents to encourage use of mamba and reflect updates to development utilities
- Manage optional dependencies through pip extras and additional conda builds
- Disable flaky tests by default on CI so that tests that fail in teardown are re-run in the second attempt. They can be enabled via the pytes
t plugin flag "--rerun-flaky" - Add utility for tracking memory usage of multitasking processes
- Update remote integration tests to use render.io instance and only run on one CI job to prevent overtaxing it
- Handle failure on CI where MPI was not being installed
- Various updates to comply with updated PEP8 standards
- Add option to track memory usage to integration services manager
- Expand output of ygginfo to include python configuration variables, common executables, and environment variables as well as providing more
detailed information when a language is marked as not configured - Limit SBML testing to single CI job
- Use sysconfig options for compiled languages if they match selected compilation tools
- Fix bug where loading a yaml file with "model_only=True" yielded a result with default inputs and outputs
Minor bug fixes for the R interface and CI
Merge pull request #92 from cropsinsilico/minor Minor bug fixes for the CI and R interface
Migrate tests out of package into pytest fixtures & various bug fixes
- Moved tests out of package to take advantage of pytest conftest.py structure and reduce the size of the package
- Refactored tests to use pytest fixtures instead of the unittest setup/teardown structure
- Removed the yggtest CLI and migrate options into pytest CLI options
- Updated the GHA workflow to use the new pytest based CLI and test parameters so that changing the image used won't change the name of the test required to
merge pull requests - Use lock to prevent parallel compilation for all compiled languages
- Removed 'initial_state' parameter from Transform and Filter schemas as it is unused
- Removed unused yggdrasil.communication.cleanup_comms method
- Updated requirements (pinned libroadrunner to < 2.0.7 and unpinned pytest)
- Updated tests to allow comparison of class objects imported in different ways
- Updated the connection and comm schemas to allow for inputs & vars to include datatypes, fixing a bug where turning on debugging caused the validation of connection parameters to fail
- Updated the connections to allow the input and output patterns to be passed directly
- Added a method for displaying information about serializers and added that information to comm info displays
- Fixed a bug in updating the serializer from messages for ForkComms with a scatter communication pattern
- Added the split_and_merge example of using communication patterns to split and merge messages between multiple inputs/outputs
- Fixed a bug in pandas serializer where strings are now stored as objects by default
- Fixed a bug in table serializer where table parameters were not being passed to the header discovery function, resulting in them being ignored in some cases
Minor updates to support model submission form development
- Added --model_repository option to the integration-service-manager CLI
- Preload models from the model repository into the service manager registry
- Added validation_command option to model schema that can be used to validate a run on completion via the 'validate' runner option
- Added dependencies and additional_dependencies options to model schema that can be used to specify packages that should be installed for a model
- Refactor registration to use constants to prevent failure on circular import and speed up import time
- Migrate constants into the constants module
- Fix various bugs introduced by updates to GHA images and dependencies by pinning or updating use (e.g. jsonschema, libroadrunner, mac version, R version)
Support for REST API based communicators, running integrations as services, and connecting to remote integration services
- Added option to return printStatus string for YggClass subclasses via return_str keyword
- Added classes for managing models as services via Flask or RabbitMQ
- Transitioned from as_function to complete_partial keyword in parse_yaml that can be used for services as well as functions
- Added client side instrumentation for connecting to remote model services and tests
- Added support for registering integrations that can be called locally as services
- Added 'integration-service-manager' CLI for managing service managers
- Added configuration options for services
- Added communicator for use with REST API
- Added ValueEvent class for returning a value with the event
- Added methods for waiting on a function to return True
- Refactored multitasking classes to use slots for improved memory performance
- Added tools for building yggdrasil docker containers and documentation on those tools
- Added repository_url, description, and contact_email parameters to model YAML schema that allows for a repository to be specified in the YAML (in addition
to via the command line as before) - Added model_only and model_submission options to yggdrasil.yamfile.parse_yaml method and YAML validation CLI
- Added generated documentation on command line utilities
- Added a glossary of terms
- Included downloadable versions of the schemas in the documentation
- Moved console_scripts list into a text file
- Note deprecation of the old GUI in the docs for the 2018 & 2019 hackathons
- Added validate_model_submission method for validating model repository submissions
Support for MPI communicators, MPI execution, and pika >= 1.0.0
- Allow models to be run on distributed processes via MPI
- Added support for MPI based comms
- Update the required version of pika to be >=1.0.0 and update the RMQComm/RMQAsyncComm code to use the updated API
- Added C, C++, Fortran, Matlab, R versions of server in rpc_lesson1 example
- Added C, C++, Fortran, Matlab, R versions of server in rpc_lesson2 example
- Added C, C++, Fortran, Matlab, R versions of server in rpc_lesson2b example
- Added C, C++, Fortran, Matlab, R versions of server in rpc_lesson3 example
- Added C, C++, Fortran, Matlab, R versions of server in rpc_lesson3b example
- Added C++, Fortran, and Python versions of client in rpc_lesson3b example (still need to thread the Python version and add R & Matlab versions)
- Fixed bug in yggdevup CLI for missing language directories
- Enhance debug information w/ task status