Code

Feat/refactor collector#1063

Code

Merged

MischaPanch merged 80 commits intothu-ml:masterthu-ml/tianshou:masterfrom aai-institute:feat/refactor_collectoraai-institute/tianshou-fork:feat/refactor_collectorCopy head branch name to clipboardMar 28, 2024

+1,152-642

Commits

Commits on Feb 23, 2024

• remove self.data, break async tests

  bordeauxred

  committed
  a1e3908

  Copy full SHA for a1e3908

• Remove preprocess_fn, adapt tests.

  bordeauxred

  committed
  72004a1

  Copy full SHA for 72004a1

Commits on Mar 7, 2024

• Removing cur_rollout_batch. WIP

  bordeauxred

  committed
  17d7e8a

  Copy full SHA for 17d7e8a

• Renamings of vars in collect

  

  Michael Panchenko

  committed
  e9a3278

  Copy full SHA for e9a3278

• minor rename

  bordeauxred

  committed
  c6a707e

  Copy full SHA for c6a707e

• Adjusted return type in BasePolicy.forward

  

  Michael Panchenko

  committed
  47bfa8c

  Copy full SHA for 47bfa8c

• More var renamings in collector

  

  Michael Panchenko

  committed
  a5b3601

  Copy full SHA for a5b3601

Commits on Mar 8, 2024

• Enhanced comments in collector

  

  Michael Panchenko

  committed
  6363af8

  Copy full SHA for 6363af8

• Formatting

  

  Michael Panchenko

  committed
  7b37eb1

  Copy full SHA for 7b37eb1

• Collector bugfixes: pass info to obs batch, don't use len on an int

  

  Michael Panchenko

  committed
  49d5648

  Copy full SHA for 49d5648

• Collector bugfix: move breakout of the loop to after updating collections

  

  Michael Panchenko

  committed
  bad9696

  Copy full SHA for bad9696

• Collector bugfix: missing () around walrus

  Show description for db5b9a3
  

  Michael Panchenko

  committed
  db5b9a3

  Copy full SHA for db5b9a3

• Renaming of env and policy used in test, added docstrings

  

  Michael Panchenko

  committed
  d65d80d

  Copy full SHA for d65d80d

• Tests: removed useless repetition of test_collector

  

  Michael Panchenko

  committed
  4641962

  Copy full SHA for 4641962

• Tests: removed more useless repetitions in test_collector

  

  Michael Panchenko

  committed
  7db9fca

  Copy full SHA for 7db9fca

• Trainer: more control over buffer resetting, fixed call to iter

  Show description for 0282b81
  

  Michael Panchenko

  committed
  0282b81

  Copy full SHA for 0282b81

• Fixes in tests - new Collector logic!

  Show description for 9add4f8
  

  Michael Panchenko

  committed
  9add4f8

  Copy full SHA for 9add4f8

• Minor, aesthetic

  

  Michael Panchenko

  committed
  fdede35

  Copy full SHA for fdede35

• Collector: fix persistence between collect iterations, remove reset for n_episodes

  

  Michael Panchenko

  committed
  7b041ad

  Copy full SHA for 7b041ad

Commits on Mar 9, 2024

• Fixed collector test by reinstating data mutation AFTER ADDING TO BUFFER. IMPORTANT - this needs fixing!

  

  Michael Panchenko

  committed
  0fa5148

  Copy full SHA for 0fa5148

• Aesthetic

  

  Michael Panchenko

  committed
  900df81

  Copy full SHA for 900df81

Commits on Mar 11, 2024

• Collector: fixed bug introduced in refactoring in persistence of _pre_collect_obs

  

  Michael Panchenko

  committed
  b2c43ec

  Copy full SHA for b2c43ec

• Rename variable, fix mixup in collectory.collect docstring.

  bordeauxred

  committed
  571ba45

  Copy full SHA for 571ba45

Commits on Mar 12, 2024

• Rename variables in AsyncCollector

  bordeauxred

  committed
  cf58c7c

  Copy full SHA for cf58c7c

• Rename id to env_ids in venv; reformat

  bordeauxred

  committed
  32e8a34

  Copy full SHA for 32e8a34

Commits on Mar 13, 2024

• fix typo and unnecessary mypy type ignore

  bordeauxred

  committed
  c84d459

  Copy full SHA for c84d459

• extract compute_action_policy_hidden in AsyncCollector

  bordeauxred

  committed
  c4a0bec

  Copy full SHA for c4a0bec

• start replacing cur_data_batch, whole_batch

  bordeauxred

  committed
  b3b3ae6

  Copy full SHA for b3b3ae6

Commits on Mar 14, 2024

• refactor async.collect to replace cur_rollout_batch, pull reset hidden state out

  bordeauxred

  committed
  6c3614d

  Copy full SHA for 6c3614d

• add test to collect one episode in asyncCollector

  bordeauxred

  committed
  4da2e13

  Copy full SHA for 4da2e13

Commits on Mar 15, 2024

• Make AsynCollector.collect work

  bordeauxred

  committed
  877fbab

  Copy full SHA for 877fbab

• Add option to reset_before_collect to AsycColl in n_episode case

  bordeauxred

  committed
  5138321

  Copy full SHA for 5138321

• Rewrite AsyncCollector tests

  bordeauxred

  committed
  6868ebf

  Copy full SHA for 6868ebf

• Rewrite AsyncCollector to not use self.current_transition_in_all_batches but explicit attributes instead.

  Show description for eb8119b
  bordeauxred

  committed
  eb8119b

  Copy full SHA for eb8119b

• remove depreciated test.

  bordeauxred

  committed
  b7b9205

  Copy full SHA for b7b9205

Commits on Mar 19, 2024

• ractor collect of asyncCollector, replace self.data by explicit attributes

  bordeauxred

  committed
  0cda877

  Copy full SHA for 0cda877

• factor out _compute_action_policy shared across Collector and AsyncCollector

  bordeauxred

  committed
  1fb64b5

  Copy full SHA for 1fb64b5

• formatting with poe

  bordeauxred

  committed
  2d5087d

  Copy full SHA for 2d5087d

Commits on Mar 20, 2024

• fix some pypy errors

  bordeauxred

  committed
  d879125

  Copy full SHA for d879125

Commits on Mar 21, 2024

• adapt type annotation of reset to match step in venv

  bordeauxred

  committed
  f560164

  Copy full SHA for f560164

• fix some mypy issues

  bordeauxred

  committed
  f28a6fe

  Copy full SHA for f28a6fe

• adapt types of reset in venv_wrappers to match step

  bordeauxred

  committed
  cbe9140

  Copy full SHA for cbe9140

• fixing some more mypy issues

  bordeauxred

  committed
  59ecee5

  Copy full SHA for 59ecee5

• Fixed typing things by casting

  

  Michael Panchenko

  committed
  dc6c2a1

  Copy full SHA for dc6c2a1

Commits on Mar 22, 2024

• fix poe issues, allow to collect fewer episodes than env_num

  bordeauxred

  committed
  ceedd19

  Copy full SHA for ceedd19

• factor out get_values_at_indices_if_not_None

  bordeauxred

  committed
  b53f18e

  Copy full SHA for b53f18e

• black

  bordeauxred

  committed
  a31cdca

  Copy full SHA for a31cdca

• adapt notebooks based upon doc-build

  bordeauxred

  committed
  69f8b2b

  Copy full SHA for 69f8b2b

• fix inconsistent indent in docstring

  bordeauxred

  committed
  d78e920

  Copy full SHA for d78e920

• Typo

  

  Michael Panchenko

  committed
  a91b262

  Copy full SHA for a91b262

• Merge branch 'thuml-master' into feat/refactor_collector

  

  Michael Panchenko

  committed
  52d8a6d

  Copy full SHA for 52d8a6d

• update tests to follow new reset interface

  bordeauxred

  committed
  aad2927

  Copy full SHA for aad2927

• Docstring

  

  Michael Panchenko

  committed
  064088d

  Copy full SHA for 064088d

• Minor fixes in FiniteVectorEnv (created in tests)

  

  Michael Panchenko

  committed
  d6cd083

  Copy full SHA for d6cd083

• Make slicing function private

  

  Michael Panchenko

  committed
  4ebdd22

  Copy full SHA for 4ebdd22

• Added a function reproducing the hacky and wrong behavior of Batch for backwards compat.

  

  Michael Panchenko

  committed
  fb65b10

  Copy full SHA for fb65b10

• Minor, comment

  

  Michael Panchenko

  committed
  da1f78a

  Copy full SHA for da1f78a

• fix multiagent test

  bordeauxred

  committed
  96824f0

  Copy full SHA for 96824f0

• Merge remote-tracking branch 'aai-tian/feat/refactor_collector' into feat/refactor_collector

  bordeauxred

  committed
  10ac920

  Copy full SHA for 10ac920

• Fixed type in test

  

  Michael Panchenko

  committed
  1fa6f9a

  Copy full SHA for 1fa6f9a

• Fixes in high-level interfaces: reset collectors before collecting

  

  Michael Panchenko

  committed
  a94fa71

  Copy full SHA for a94fa71

• Fixes in test: add explicit reset of collectors

  

  Michael Panchenko

  committed
  3b90faf

  Copy full SHA for 3b90faf

• AsyncCollector: Stricter types and enforcement

  

  Michael Panchenko

  committed
  729c60b

  Copy full SHA for 729c60b

Commits on Mar 25, 2024

• Fixes in tests, reset

  bordeauxred

  committed
  48bd641

  Copy full SHA for 48bd641

• poe format

  bordeauxred

  committed
  39bc3cc

  Copy full SHA for 39bc3cc

• Fixes in tests, int instead of np.int in Discrete.n

  bordeauxred

  committed
  be8d7ef

  Copy full SHA for be8d7ef

• Use CollectStats with autogenerate_stats in AsyncCollector, put len to int to keep summary format

  bordeauxred

  committed
  574c6e9

  Copy full SHA for 574c6e9

• Added a method to converge a batch into dict or a list of dicts

  

  Michael Panchenko

  committed
  deb6f78

  Copy full SHA for deb6f78

• Collector: fixed reset behavor for envpool envs

  

  Michael Panchenko

  committed
  abcb44c

  Copy full SHA for abcb44c

• Added explicit iter to batch, fixed mypy issues

  Show description for 572dd68
  

  Michael Panchenko

  committed
  572dd68

  Copy full SHA for 572dd68

• Simplified nullable_slice (no need for @overload)

  

  Michael Panchenko

  committed
  8757d65

  Copy full SHA for 8757d65

Commits on Mar 26, 2024

• Simplified nullable_slice (no need for @overload)

  

  Michael Panchenko

  committed
  09cb1c6

  Copy full SHA for 09cb1c6

• Fixed iter for empty batch

  

  Michael Panchenko

  committed
  bff8e7d

  Copy full SHA for bff8e7d

• Renamed env_ids back to env_id, added a TODO

  

  Michael Panchenko

  committed
  92a7467

  Copy full SHA for 92a7467

• Fixed iter in batch

  

  Michael Panchenko

  committed
  0cde27b

  Copy full SHA for 0cde27b

• Fixed test vectorenv

  

  Michael Panchenko

  committed
  3f305ab

  Copy full SHA for 3f305ab

• Protocols: removed accidental method implementation (copy-paste error)

  

  Michael Panchenko

  committed
  5470a6c

  Copy full SHA for 5470a6c

• Removed runtime checkable of ObsBatchProtocol

  

  Michael Panchenko

  committed
  c04df12

  Copy full SHA for c04df12

• Changelog [skip ci]

  

  Michael Panchenko

  committed
  7e77d08

  Copy full SHA for 7e77d08

• Refactor: replace overload in exploration_noise by generic

  bordeauxred

  committed
  4f47f23

  Copy full SHA for 4f47f23