[dagster-fivetran] Implement base resync method in FivetranClient #26059
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced Nov 20, 2024
Merged
[5/n][dagster-fivetran] Implement
FivetranWorkspaceData to FivetranConnectorTableProps method
#25797
Merged
Merged
This was referenced Nov 20, 2024
dpeng817
approved these changes
Nov 20, 2024
| connector_id (str): The Fivetran Connector ID. You can retrieve this value from the | ||
| "Setup" tab of a given connector in the Fivetran UI. | ||
| resync_parameters (Optional[Dict[str, List[str]]]): Optional resync parameters to send to the Fivetran API. | ||
| An example payload can be found here: https://fivetran.com/docs/rest-api/connectors#request_7 |
maximearmstrong
force-pushed
the
maxime/rework-fivetran-9
branch
from
8e607df to
6cd5f9e
Compare
maximearmstrong
force-pushed
the
maxime/implement-resync-base-method-fivetran-client
branch
from
cadbe37 to
90123af
Compare
maximearmstrong
force-pushed
the
maxime/rework-fivetran-9
branch
from
6cd5f9e to
8e3c5a9
Compare
maximearmstrong
force-pushed
the
maxime/implement-resync-base-method-fivetran-client
branch
from
90123af to
b81d516
Compare
maximearmstrong
force-pushed
the
maxime/rework-fivetran-9
branch
from
8e3c5a9 to
8e2d5cb
Compare
maximearmstrong
force-pushed
the
maxime/implement-resync-base-method-fivetran-client
branch
from
b81d516 to
004f258
Compare
maximearmstrong
force-pushed
the
maxime/rework-fivetran-9
branch
from
8e2d5cb to
f43b3f6
Compare
maximearmstrong
force-pushed
the
maxime/implement-resync-base-method-fivetran-client
branch
from
004f258 to
03b4f71
Compare
maximearmstrong
force-pushed
the
maxime/rework-fivetran-9
branch
from
f43b3f6 to
f56783b
Compare
maximearmstrong
force-pushed
the
maxime/implement-resync-base-method-fivetran-client
branch
from
03b4f71 to
6855c14
Compare
This was referenced Nov 23, 2024
maximearmstrong
force-pushed
the
maxime/rework-fivetran-9
branch
from
f56783b to
72d546e
Compare
maximearmstrong
force-pushed
the
maxime/implement-resync-base-method-fivetran-client
branch
from
6855c14 to
2021ba0
Compare
maximearmstrong
force-pushed
the
maxime/rework-fivetran-9
branch
from
72d546e to
a069876
Compare
maximearmstrong
force-pushed
the
maxime/implement-resync-base-method-fivetran-client
branch
from
2021ba0 to
3df2067
Compare
maximearmstrong
force-pushed
the
maxime/rework-fivetran-9
branch
from
a069876 to
cec7700
Compare
maximearmstrong
force-pushed
the
maxime/implement-resync-base-method-fivetran-client
branch
from
3df2067 to
22bfe6c
Compare
maximearmstrong
force-pushed
the
maxime/rework-fivetran-9
branch
from
cec7700 to
4e940a4
Compare
maximearmstrong
force-pushed
the
maxime/implement-resync-base-method-fivetran-client
branch
from
22bfe6c to
ffab308
Compare
maximearmstrong
force-pushed
the
maxime/rework-fivetran-9
branch
from
4e940a4 to
5381ecd
Compare
maximearmstrong
force-pushed
the
maxime/implement-resync-base-method-fivetran-client
branch
from
ffab308 to
16121e5
Compare
maximearmstrong
force-pushed
the
maxime/implement-resync-base-method-fivetran-client
branch
from
16121e5 to
8a48241
Compare
maximearmstrong
deleted the
maxime/implement-resync-base-method-fivetran-client
branch
cmpadden
pushed a commit
that referenced
this pull request
Dec 5, 2024
…6059) ## Summary & Motivation This PR reworks legacy resync method and implements it in the `FivetranClient`: - `start_resync` is added based on legacy `start_resync` - `start_resync` leverages `_start_sync` introduced in #25911 - a [resync in Fivetran](https://fivetran.com/docs/rest-api/api-reference/connectors/resync-connector) is historical data sync - the endpoint and result is different, but logic around how to call and handle a resync is the same as a sync. - a resync can be done with or without resync parameters, using a different endpoint. Tests mock the request API calls and make sure that all calls are made. ## How I Tested These Changes Additional unit tests with BK
pskinnerthyme
pushed a commit
to pskinnerthyme/dagster
that referenced
this pull request
Dec 16, 2024
…gster-io#26059) ## Summary & Motivation This PR reworks legacy resync method and implements it in the `FivetranClient`: - `start_resync` is added based on legacy `start_resync` - `start_resync` leverages `_start_sync` introduced in dagster-io#25911 - a [resync in Fivetran](https://fivetran.com/docs/rest-api/api-reference/connectors/resync-connector) is historical data sync - the endpoint and result is different, but logic around how to call and handle a resync is the same as a sync. - a resync can be done with or without resync parameters, using a different endpoint. Tests mock the request API calls and make sure that all calls are made. ## How I Tested These Changes Additional unit tests with BK
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary & Motivation
This PR reworks legacy resync method and implements it in the
FivetranClient:start_resyncis added based on legacystart_resyncstart_resyncleverages_start_syncintroduced in [9/n][dagster-fivetran] Implement base sync methods in FivetranClient #25911Tests mock the request API calls and make sure that all calls are made.
How I Tested These Changes
Additional unit tests with BK