Contents

Changelog

Changelog

Listed here are the changes between each release of SmartSim and SmartRedis.

Jump to SmartRedis Changelog

SmartSim

0.4.1

Released on June 24, 2022

Description: This release of SmartSim introduces a new experimental feature to help make SmartSim workflows more portable: the ability to run simulations models in a container via Singularity. This feature has been tested on a small number of platforms and we encourage users to provide feedback on its use.

We have also made improvements in a variety of areas: new utilities to load scripts and machine learning models into the database directly from SmartSim driver scripts and install-time choice to use either KeyDB or Redis for the Orchestrator. The RunSettings API is now more consistent across subclasses. Another key focus of this release was to aid new SmartSim users by including more extensive tutorials and improving the documentation. The docker image containing the SmartSim tutorials now also includes a tutorial on online training.

Launcher improvements

Documentation and tutorials

General improvements and bug fixes

Dependency updates

0.4.0

Released on Feb 11, 2022

Description: In this release SmartSim continues to promote ease of use. To this end SmartSim has introduced new portability features that allow users to abstract away their targeted hardware, while providing even more compatibility with existing libraries.

A new feature, Co-located orchestrator deployments has been added which provides scalable online inference capabilities that overcome previous performance limitations in seperated orchestrator/application deployments. For more information on advantages of co-located deployments, see the Orchestrator section of the SmartSim documentation.

The SmartSim build was significantly improved to increase customization of build toolchain and the smart command line inferface was expanded.

Additional tweaks and upgrades have also been made to ensure an optimal experience. Here is a comprehensive list of changes made in SmartSim 0.4.0.

Orchestrator Enhancements:

Emphasize Driver Script Portability:

  • Add ability to create run settings through an experiment (SmartSim-PR110)

  • Add ability to create batch settings through an experiment (SmartSim-PR112)

  • Add automatic launcher detection to experiment portability functions (SmartSim-PR120)

Expand Machine Learning Library Support:

Expand Launcher Setting Options:

  • Add ability to use base RunSettings on a Slurm, PBS, or Cobalt launchers (SmartSim-PR90)

  • Add ability to use base RunSettings on LFS launcher (SmartSim-PR108)

Deprecations and Breaking Changes

General Improvements and Bug Fixes:

Documentation Updates:

0.3.2

Released on August 10, 2021

Description:

0.3.1

Released on May 5, 2021

Description: This release was dedicated to making the install process easier. SmartSim can be installed from PyPi now and the smart cli tool makes installing the machine learning runtimes much easier.

0.3.0

Released on April 1, 2021

Description:

  • initial 0.3.0 (first public) release of SmartSim

SmartRedis

0.3.1

Released on June 24, 2022

Description

Version 0.3.1 adds new functionality in the form of DataSet aggregation lists for pipelined retrieval of data, convenient support for multiple GPUs, and the ability to delete scripts and models from the backend database. It also introduces multithreaded execution for certain tasks that span multiple shards of a clustered database, and it incorporates a variety of internal improvements that will enhance the library going forward.

Detailed Notes

  • Implemented DataSet aggregation lists in all client languages, for pipelined retrieval of data across clustered and non-clustered backend databases. (PR258) (PR257) (PR256) (PR248) New commands are:

    • append_to_list()

    • delete_list()

    • copy_list()

    • rename_list()

    • get_list_length()

    • poll_list_length()

    • poll_list_length_gte()

    • poll_list_length_lte()

    • get_datasets_from_list()

    • get_dataset_list_range()

    • use_list_ensemble_prefix()

  • Implemented multithreaded execution for parallel dataset list retrieval on clustered databases. The number of threads devoted for this purpose is controlled by the new environment variable SR_THERAD_COUNT. The value defaults to 4, but may be any positive integer or special value zero, which will cause the SmartRedis runtime to allocate one thread for each available hardware context. (PR251) (PR246)

  • Augmented support for GPUs by implementing multi-GPU convenience functions for all client languages. (PR254) (PR250) (PR244) New commands are:

    • set_model_from_file_multigpu()

    • set_model_multigpu()

    • set_script_from_file_multigpu()

    • set_script_multigpu()

    • run_model_multigpu()

    • run_script_multigpu()

    • delete_model_multigpu()

    • delete_script_multigpu()

  • Added API calls for all clients to delete models and scripts from the backend database. (PR240) New commands are:

    • delete_script()

    • delete_model()

  • Updated the use of backend RedisAI API calls to discontinue use of deprecated methods for model selection (AI.MODELSET) and execution (AI.MODELRUN) in favor of current methods AI.MODELSTORE and AI.MODELEXECUTE, respectively. (PR234)

  • SmartRedis will no longer call the C runtime method srand() to ensure that it does not interfere with random number generation in client code. It now uses a separate instance of the C++ random number generator. (PR233)

  • Updated the way that the Fortran enum_kind type defined in the fortran_c_interop module is defined in order to better comply with Fortran standard and not interfere with GCC 6.3.0. (PR231)

  • Corrected the spelling of the word “command” in a few error message strings. (PR221)

  • SmartRedis now requires a CMake version 3.13 or later in order to utilize the add_link_options CMake command. (PR217)

  • Updated and improved the documentation of the SmartRedis library. In particular, a new SmartRedis Integration Guide provides an introduction to using the SmartRedis library and integrating it with existing software. (PR261) (PR260) (PR259) (SSPR214)

  • Added clustered Redis testing to automated GitHub check-in testing. (PR239)

  • Updated the SmartRedis internal API for building commands for the backend database. (PR223) This change should not be visible to clients.

  • The SmartRedis example code is now validated through the automated GitHub checkin process. This will help ensure that the examples do not fall out of date. (PR220)

  • Added missing copyright statements to CMakeLists.txt and the SmartRedis examples. (PR219)

  • Updated the C++ test coverage to ensure that all test files are properly executed when running “make test”. (PR218)

  • Fixed an internal naming conflict between a local variable and a class member variable in the DataSet class. (PR215) This should not be visible to clients.

  • Updated the internal documentation of methods in SmartRedis C++ classes with the override keyword to improve compliance with the latest C++ standards. (PR214) This change should not be visible to clients.

  • Renamed variables internally to more cleanly differentiate between names that are given to clients for tensors, models, scripts, datasets, etc., and the keys that are used when storing them in the backend database. (PR213) This change should not be visible to clients.

0.3.0

Released on Febuary 11, 2022

Description

  • Improve error handling across all SmartRedis clients (PR159) (PR191) (PR199) (PR205) (PR206)

    • Includes changes to C and Fortran function prototypes that are not backwards compatible

    • Includes changes to error class names and enum type names that are not backwards compatible

  • Add poll_dataset functionality to all SmartRedis clients (PR184)

    • Due to other breaking changes made in this release, applications using methods other than poll_dataset to check for the existence of a dataset should now use poll_dataset

  • Add environment variables to control client connection and command timeout behavior (PR194)

  • Add AI.INFO command to retrieve statistics on scripts and models via Python and C++ clients (PR197)

  • Create a Dockerfile for SmartRedis (PR180)

  • Update redis-plus-plus version to 1.3.2 (PR162)

  • Internal client performance and API improvements (PR138) (PR141) (PR163) (PR203)

  • Expose Redis FLUSHDB, CONFIG GET, CONFIG SET, and SAVE commands to the Python client (PR139) (PR160)

  • Extend inverse CRC16 prefixing to all hash slots (PR161)

  • Improve backend dataset representation to enable performance optimization (PR195)

  • Simplify SmartRedis build proccess (PR189)

  • Fix zero-length array transfer in Fortran convert_char_array_to_c (PR170)

  • Add continuous integration for all SmartRedis tests (PR165) (PR173) (PR177)

  • Update SmartRedis docstrings (PR200) (PR207)

  • Update SmartRedis documentation and examples (PR202) (PR208) (PR210)

0.2.0

Released on August, 5, 2021

Description

  • Improved tensor memory management in the Python client (PR70)

  • Improved metadata serialization and removed protobuf dependency (PR61)

  • Added unit testing infrastructure for the C++ client (PR96)

  • Improve command execution fault handling (PR65) (PR97) (PR105)

  • Bug fixes (PR52) (PR72) (PR76) (PR84)

  • Added copy, rename, and delete tensor and DataSet commands in the Python client (PR66)

  • Upgrade to RedisAI 1.2.3 (PR101)

  • Fortran and C interface improvements (PR93) (PR94) (PR95) (PR99)

  • Add Redis INFO command execution to the Python client (PR83)

  • Add Redis CLUSTER INFO command execution to the Python client (PR105)

0.1.1

Released on May 5, 2021

Description

  • Compiled client library build and install update to remove environment variables (PR47)

  • Pip install for Python client (PR45)

0.1.0

Released on April 1, 2021

Description

  • Initial 0.1.0 release of SmartRedis

previous

SmartRedis API

next

Code of Conduct