streamparse 3.2.0
This release adds tools to simplify some common deployment scenarios where you need to deploy the same topology to different environments.
Features
- The
par
parameter for theComponent.spec()
method used to set options for components within your topology can now take dictionaries in addition to integers. The keys must be names of environments in yourconfig.json
, and the values are integers as before. This allows you to specify different parallelism hints for components depending on the environment they are deployed to. This is very helpful when one of your environments has more processing power than the other. (PR #326) - Added
--requirements
options tosparse update_virtualenv
andsparse submit
commands so that you can customize the requirements files that are used for your virtualenv, instead of relying on files in yourvirtualenv_specs
directory. (PR #328) - pip is now automatically upgraded to 9.x on the worker nodes and is now run with the flags
--upgrade --upgrade-strategy only-if-needed
to ensure that requirements specified as ranges are upgraded to the same version on all machines, without needlessly upgrading all recursive dependencies. (PR #327)
Fixes
- Fixed an issue where
name
was being used instead ofoverride_name
when calling pre- and post-submit hooks. (10e8ce3) - Docs have been updated to fix some RST rendering issues (issue #321)
- Updated quickstart to clarify which version of Storm is required (PR #315)
- Added information about flux-core dependency to help string for
sparse run
(PR #316)