Code

Fix/job resume#1704

Code

Merged

einari merged 117 commits intomainCratis/Chronicle:mainfrom fix/job-resumeCratis/Chronicle:fix/job-resumeCopy head branch name to clipboardMar 12, 2025

Commits

Commits on Feb 11, 2025

• Split up Observer in partial classes

  woksin

  committed
  6cc83a5

  Copy full SHA for 6cc83a5

• Make JobsManager Reentrant because it should be safe

  woksin

  committed
  09b696f

  Copy full SHA for 09b696f

Commits on Feb 12, 2025

• Add StartJobError type and use it in IJob.Start

  woksin

  committed
  cf11938

  Copy full SHA for cf11938

• Add Job logging for failed startign job step

  woksin

  committed
  d91bb61

  Copy full SHA for d91bb61

• Code cleanup

  woksin

  committed
  7ffcf21

  Copy full SHA for 7ffcf21

• Write EventTypes to ObserverState when Subscribing instead of OnLeave of state machine state. We were just lucky that when starting jobs that we had stored the correct state on the Observer most of…

  Show description for 1653636
  woksin

  committed
  1653636

  Copy full SHA for 1653636

• Wait some time for the observers to reach the desired handled event in integration test because it can occationally fail because GetState is interleaved method

  woksin

  committed
  c809bb9

  Copy full SHA for c809bb9

• Separate job step related methods in Job in its own partial class. Also rework Start so that it is waits for the whole process of starting all jobsteps to be finished before returning. This makes t…

  Show description for 18ce331
  woksin

  committed
  18ce331

  Copy full SHA for 18ce331

• Make IObserver GetState not Interleaved because it was only used in tests and then it is nice to wait for the Observer to be finished processing messages

  woksin

  committed
  457bfd6

  Copy full SHA for 457bfd6

• Simplify JobStep by just having Prepare, Start and Stop semantics. Start behaves as Resume would

  woksin

  committed
  ca415f1

  Copy full SHA for ca415f1

Commits on Feb 13, 2025

• Add Prepared boolean to the JobStepState

  woksin

  committed
  bdcf198

  Copy full SHA for bdcf198

• Make it build

  woksin

  committed
  68c778f

  Copy full SHA for 68c778f

• Remove methods from IJob grain that should not be called from the outside

  woksin

  committed
  428e17a

  Copy full SHA for 428e17a

Commits on Feb 17, 2025

• JobStatus change PreparingStepsForRunning to StartingSteps and add a PreparingJob state

  woksin

  committed
  2b9d6ad

  Copy full SHA for 2b9d6ad

• Improve documentationf

  woksin

  committed
  9874825

  Copy full SHA for 9874825

• Add a StoppedSteps int to JobProgress and change IsCompleted to also check stopped steps

  woksin

  committed
  5cfd973

  Copy full SHA for 5cfd973

• Improve IJob interface

  woksin

  committed
  080beed

  Copy full SHA for 080beed

• make specs build

  woksin

  committed
  2850b2f

  Copy full SHA for 2850b2f

• Remove unused property from JobStepGrainAndRequest

  woksin

  committed
  7367cf3

  Copy full SHA for 7367cf3

• JobStep should not unsubscribe itself

  woksin

  committed
  b468c13

  Copy full SHA for b468c13

• Fix build issue in JobsManager

  woksin

  committed
  bab7158

  Copy full SHA for bab7158

• Add more log messages to JobLogging

  woksin

  committed
  d6793fa

  Copy full SHA for d6793fa

• Move state related things from Job into it's own partial class file

  woksin

  committed
  3c85046

  Copy full SHA for 3c85046

Commits on Feb 18, 2025

• Make it build

  woksin

  committed
  2aa07a3

  Copy full SHA for 2aa07a3

• Make it build

  woksin

  committed
  5b46548

  Copy full SHA for 5b46548

• Fix the job step

  woksin

  committed
  97cdfd4

  Copy full SHA for 97cdfd4

Commits on Feb 19, 2025

• Simplify JobsManager Start to have an overload that does not take JobId

  woksin

  committed
  3bee66b

  Copy full SHA for 3bee66b

• Fix build issues

  woksin

  committed
  17a7c41

  Copy full SHA for 17a7c41

• Small improvements to Jobs

  woksin

  committed
  25f0891

  Copy full SHA for 25f0891

• Fix small issues with Job

  woksin

  committed
  4178015

  Copy full SHA for 4178015

• Start making integration tests that uses strictly jobs and jobs manager directly

  woksin

  committed
  c22ebc2

  Copy full SHA for c22ebc2

Commits on Feb 26, 2025

• WIP integration tests for jobs

  woksin

  committed
  d151ccd

  Copy full SHA for d151ccd

Commits on Feb 27, 2025

• Correectly map JobType to job grain type which is the interface type

  woksin

  committed
  fd92bea

  Copy full SHA for fd92bea

• Catch and log error in JobStep ReportError

  woksin

  committed
  7fc4d88

  Copy full SHA for 7fc4d88

• Use the correct Job Type when getting JobType

  woksin

  committed
  8f12ab0

  Copy full SHA for 8f12ab0

• PerformJobStepError could not be serialized correctly in Orleans

  woksin

  committed
  32df87a

  Copy full SHA for 32df87a

• Test for when a Job is stopped

  woksin

  committed
  a66af21

  Copy full SHA for a66af21

• Use event types from subcsription instead of the State because it is not available

  woksin

  committed
  f2d65bb

  Copy full SHA for f2d65bb

• Integration test

  woksin

  committed
  d0be231

  Copy full SHA for d0be231

• Interleave IObserver GetState

  woksin

  committed
  9ccd0d9

  Copy full SHA for 9ccd0d9

• Handle the case when registerign an observer with no event types, just transition it to disconnected as it can provide no value and it even caused forever looping issues

  woksin

  committed
  186722d

  Copy full SHA for 186722d

• In Job when there are no steps no start call HandleCompletion without waiting for it because it could cause a deadlock depending on how JobsManager.Start was invoked

  woksin

  committed
  f3d6b05

  Copy full SHA for f3d6b05

• INtegration test for registering reactor without any event handlers

  woksin

  committed
  6552ec9

  Copy full SHA for 6552ec9

Commits on Feb 28, 2025

• JobTypes extensions for getting or throwing exception

  woksin

  committed
  0a87f49

  Copy full SHA for 0a87f49

• Implement GetState on

  woksin

  committed
  8fcf25b

  Copy full SHA for 8fcf25b

• WIP:

  woksin

  committed
  9829b22

  Copy full SHA for 9829b22

Commits on Mar 3, 2025

• Do not deactivate on Idle during stop - this will actually collect the grain after the call

  einari

  committed
  a993c93

  Copy full SHA for a993c93

• Make integration test for jobs more reliabable

  woksin

  committed
  d1396b0

  Copy full SHA for d1396b0

• Fix Job grain reference in JobStep

  woksin

  committed
  2ef93dc

  Copy full SHA for 2ef93dc

• Remove unnecessary code from JobStep

  woksin

  committed
  bc44e3a

  Copy full SHA for bc44e3a

• Fix build issue

  woksin

  committed
  9121179

  Copy full SHA for 9121179

Commits on Mar 8, 2025

• Fix bug in Storage.MongoDB Sink where it never set _isReplaying to false

  woksin

  committed
  c9d6cda

  Copy full SHA for c9d6cda

Commits on Mar 9, 2025

• Implement ResumeReplay accross the board for observer service, replay handlers and sinks

  woksin

  committed
  eb748e0

  Copy full SHA for eb748e0

• Rehydrate all JobsManagers when starting up Chronicle

  woksin

  committed
  530be52

  Copy full SHA for 530be52

• Remove JobType from JobStepState as it is not needed

  woksin

  committed
  95bcc01

  Copy full SHA for 95bcc01

• Code refactor

  woksin

  committed
  71ced98

  Copy full SHA for 71ced98

• Correct the JobStepStorage so that it goes the failed collection when it should

  woksin

  committed
  d8530ed

  Copy full SHA for d8530ed

• Correct naming on IJob, Start, Stop, Resume, Remove

  woksin

  committed
  54473a4

  Copy full SHA for 54473a4

• Add ObserverType to all observer job requests

  woksin

  committed
  6f9ed50

  Copy full SHA for 6f9ed50

• Fix ResumeJobs on Observer so that it calls Resume only on the Jobs for the Observer that should be resumed.

  Show description for eebb4d3
  woksin

  committed
  eebb4d3

  Copy full SHA for eebb4d3

• JobsManager extension for starting or resuming job for Observer

  woksin

  committed
  fdc0563

  Copy full SHA for fdc0563

• Don't need ResumeReplay Observer State

  woksin

  committed
  0614d4a

  Copy full SHA for 0614d4a

• Remove ResumeReplaying and transitioning to Replay from Observer Routing state

  woksin

  committed
  6041ab5

  Copy full SHA for 6041ab5

• Code enhance ReplayEvaluator a bit

  woksin

  committed
  e8a7bb0

  Copy full SHA for e8a7bb0

• Simplify Observer Replay state so that it only starts or resumes the replay observer job

  woksin

  committed
  01f32f9

  Copy full SHA for 01f32f9

• Convenience methods for JobStateWithLastHandledEvent for getting Request as ObserverRequest and also getting ObserverDetails

  woksin

  committed
  c702ab2

  Copy full SHA for c702ab2

• Revisit Job and JobStep status

  woksin

  committed
  8b39b9c

  Copy full SHA for 8b39b9c

• Fix JobProgress IsCompleted and add IsStopped.

  woksin

  committed
  b7e9280

  Copy full SHA for b7e9280

• Add ObserverType to ReplayCandidateRequest because it needs it

  woksin

  committed
  a3f8684

  Copy full SHA for a3f8684

• Add missing ObserverType property

  woksin

  committed
  f47e859

  Copy full SHA for f47e859

• Observer.Replay transition to Replay if not in Replaying state

  woksin

  committed
  6884b8e

  Copy full SHA for 6884b8e

Commits on Mar 10, 2025

• Implement CreateCancelledResultFromCurrentState in HandleEventsForPartition

  woksin

  committed
  e8b73d5

  Copy full SHA for e8b73d5

• Fix build issues. Also remove OnBeforePrepareSteps as it is unnecessary

  woksin

  committed
  6741459

  Copy full SHA for 6741459

• Create extension method for getting IJobsManager grain and use it

  woksin

  committed
  1a59939

  Copy full SHA for 1a59939

• Reimplement ReplayObserver job

  woksin

  committed
  6b81086

  Copy full SHA for 6b81086

• Fix JobStepGrainStorageProvider

  woksin

  committed
  93e1c21

  Copy full SHA for 93e1c21

• Fix build issues in JobGrainStorageProvider

  woksin

  committed
  122f53d

  Copy full SHA for 122f53d

• Fix various issues with JobStep

  woksin

  committed
  a629be4

  Copy full SHA for a629be4

• Change method name

  woksin

  committed
  de80907

  Copy full SHA for de80907

• Fix various issues with Job

  woksin

  committed
  c9bd1a8

  Copy full SHA for c9bd1a8

• Job logging

  woksin

  committed
  8d1a552

  Copy full SHA for 8d1a552

• Fix JobsManager

  woksin

  committed
  ec147dd

  Copy full SHA for ec147dd

• Fix runtime issue, Observer will not take dependency on IReplayEvaluator

  woksin

  committed
  4982e42

  Copy full SHA for 4982e42

• Fix integration tests for JobsManager

  woksin

  committed
  04abf28

  Copy full SHA for 04abf28

• Fix Job HandleCompletion

  woksin

  committed
  498bd38

  Copy full SHA for 498bd38

• Integration test for when starting and deleting job

  woksin

  committed
  e76ece2

  Copy full SHA for e76ece2

• Integration tests for stopping and then deleting a started job

  woksin

  committed
  c2856a2

  Copy full SHA for c2856a2

• Make sure that state of a JobStep is not written after it is removed

  woksin

  committed
  df96450

  Copy full SHA for df96450

• Integration test for starting stopping and resuming a job

  woksin

  committed
  fe1d655

  Copy full SHA for fe1d655

• Fix issues found during integration testing.

  woksin

  committed
  9833ee0

  Copy full SHA for 9833ee0

• make test more reliable

  woksin

  committed
  87aeb9f

  Copy full SHA for 87aeb9f

Commits on Mar 11, 2025

• Integration tests for starting and resuming job

  woksin

  committed
  3bd9cbe

  Copy full SHA for 3bd9cbe

• Fix nasty issue with JobGrainStorageProvider. Do not modify state on read

  woksin

  committed
  682019e

  Copy full SHA for 682019e

• Job DeactivateOnIdle after handling completion result

  woksin

  committed
  0f45a2b

  Copy full SHA for 0f45a2b

• Moving DeactivateIdle on Job was a mistake. Moving it back

  woksin

  committed
  c508e88

  Copy full SHA for c508e88

• Greater delays on integration tests

  woksin

  committed
  b1b2b7f

  Copy full SHA for b1b2b7f

• Fix Job clearing state

  woksin

  committed
  1b60d2e

  Copy full SHA for 1b60d2e

• Add integration tests for checking that state is not removed when job fails

  woksin

  committed
  7a7d1ab

  Copy full SHA for 7a7d1ab

• Introduce IsReplaying state to the Observer

  woksin

  committed
  20e8774

  Copy full SHA for 20e8774

• Make specs work

  woksin

  committed
  3e12882

  Copy full SHA for 3e12882

• Merge branch 'main' of github.com:Cratis/Cratis into fix/job-resume

  woksin

  committed
  8d48f99

  Copy full SHA for 8d48f99

• Adapt ProjectionReplayHandler merge

  woksin

  committed
  78d7990

  Copy full SHA for 78d7990

• Fix interfaces

  woksin

  committed
  a3de995

  Copy full SHA for a3de995

Commits on Mar 12, 2025

• Make it build in release mode

  woksin

  committed
  4b7dff5

  Copy full SHA for 4b7dff5

• Revert JobTypes

  woksin

  committed
  fb75d3c

  Copy full SHA for fb75d3c

• Fix JobTypes again so that it stores the JobType as the name of the implementing class, not the interface.

  woksin

  committed
  8956956

  Copy full SHA for 8956956

• Fix broken integration tests

  woksin

  committed
  fdd7e95

  Copy full SHA for fdd7e95

• Remove test that is not implemented yet

  woksin

  committed
  1820f34

  Copy full SHA for 1820f34

• Fix ObserverManager Expiration TimeSpan, it cannot be Max

  woksin

  committed
  56bdd05

  Copy full SHA for 56bdd05

• Changing to using semantic logging

  einari

  committed
  dec8842

  Copy full SHA for dec8842

• Fixing error

  einari

  committed
  f8edb3c

  Copy full SHA for f8edb3c

• DRYing up repeated methods

  einari

  committed
  26e1154

  Copy full SHA for 26e1154

• Wait for the correct state in integration test

  woksin

  committed
  2b7197c

  Copy full SHA for 2b7197c

• Merge branch 'fix/job-resume' of github.com:Cratis/Cratis into fix/job-resume

  woksin

  committed
  c3b2d8c

  Copy full SHA for c3b2d8c

• Fixing logs to use ReplayObserver as name

  einari

  committed
  f888a8d

  Copy full SHA for f888a8d

• Synchronizing JobStatus across contracts and Api and frontend for Workbench

  einari

  committed
  b6bca5f

  Copy full SHA for b6bca5f

• Merge remote-tracking branch 'origin/main' into fix/job-resume

  einari

  committed
  bc197fb

  Copy full SHA for bc197fb