ENG-53369 : GCP collector - Handle the pagination state properly

[rakhimundhada15]

Problem Description

Collector losses the data as state since and until value not set properly during pagination scenario.
Example : If there is lots of pages and collector collected paws_max_pages_per_invocation(default -10) max pages in one call , if there is still nextPage token since and until value should not change but currently it changes after every call . So if there is no more page it will start collecting the until value which is latest. Hence it lost the data for some period

Solution Description

If nextPage available then don’t change since & until value only update next_token else update the since and until pass in next filter.

[al-cibot]

CI Service Job Number 903 Logs - SUCCESS

alps ci job-logs --organization alertlogic --repository paws-collector --job-num 903 -g

Stages Run:

• PR Test

[al-cibot]

CI Service Job Number 904 Logs - SUCCESS

alps ci job-logs --organization alertlogic --repository paws-collector --job-num 904 -g

Stages Run:

• PR Test