Skip to content

streamparse 3.2.0

Compare
Choose a tag to compare
@dan-blanchard dan-blanchard released this 03 Nov 19:14
c942cb3

This release adds tools to simplify some common deployment scenarios where you need to deploy the same topology to different environments.

Features

  • The par parameter for the Component.spec() method used to set options for components within your topology can now take dictionaries in addition to integers. The keys must be names of environments in your config.json, and the values are integers as before. This allows you to specify different parallelism hints for components depending on the environment they are deployed to. This is very helpful when one of your environments has more processing power than the other. (PR #326)
  • Added --requirements options to sparse update_virtualenv and sparse submit commands so that you can customize the requirements files that are used for your virtualenv, instead of relying on files in your virtualenv_specs directory. (PR #328)
  • pip is now automatically upgraded to 9.x on the worker nodes and is now run with the flags --upgrade --upgrade-strategy only-if-needed to ensure that requirements specified as ranges are upgraded to the same version on all machines, without needlessly upgrading all recursive dependencies. (PR #327)

Fixes

  • Fixed an issue where name was being used instead of override_name when calling pre- and post-submit hooks. (10e8ce3)
  • Docs have been updated to fix some RST rendering issues (issue #321)
  • Updated quickstart to clarify which version of Storm is required (PR #315)
  • Added information about flux-core dependency to help string for sparse run (PR #316)