Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Overload HASH transfrom to cheat checks #914

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

Jonathan-Poelhuis
Copy link

I took an easy route, and (using the idea of Damien) overloaded the HASH transform to get past pipelinewise strict hard-coded checking (that also relies on another external package.) Different from Damien's approach, I wrote the new transforms in the transform_utils.py rather than try an "import a package of our own" approach. It's less snappy but potentially simpler.

To make this work, I

  • Added optional external_type and param to the tap.json schema
  • Added parsing of those optional parameters to the config.py (including levels when they're missing)
  • Added a ExternalTransformationType Enum, including valid levels for when the parameters are not given
    • Then overloaded the TransformationType.HASH to have a switch when an external_type is given.
    • The example functions here are just examples to demonstrate how the code should work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant