Skip to content

Commit

Permalink
chore: Update project metadata and other GitLab references (meltano#687)
Browse files Browse the repository at this point in the history
chore: Update project metadata and GitLab references
  • Loading branch information
edgarrmondragon authored Jun 1, 2022
1 parent 715dad6 commit c2d8f41
Show file tree
Hide file tree
Showing 10 changed files with 16 additions and 23 deletions.
2 changes: 1 addition & 1 deletion cookiecutter/tap-template/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ pipx install cookiecutter
Initialize Cookiecutter template directly from Git:

```bash
cookiecutter https://gitlab.com/meltano/sdk --directory="cookiecutter/tap-template"
cookiecutter https://github.com/meltano/sdk --directory="cookiecutter/tap-template"
```

Or locally from an already-cloned `sdk` repo:
Expand Down
2 changes: 1 addition & 1 deletion cookiecutter/target-template/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ pipx install cookiecutter
Initialize Cookiecutter template directly from Git:

```bash
cookiecutter https://gitlab.com/meltano/sdk --directory="cookiecutter/target-template"
cookiecutter https://github.com/meltano/sdk --directory="cookiecutter/target-template"
```

Or locally from an already-cloned `sdk` repo:
Expand Down
2 changes: 1 addition & 1 deletion docs/code_samples.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ or to get more information on what an SDK-based tap or target will look like.
- [target-athena, Community Project led by Andrew Stewart](https://github.com/dataops-tk/target-athena)

To add your project to this list, please
[submit an issue](https://gitlab.com/meltano/meltano/-/issues/new?issue%5Bassignee_id%5D=&issue%5Bmilestone_id%5D=).
[submit an issue](https://github.com/meltano/sdk/issues/new).

## Reusable Code Snippets

Expand Down
4 changes: 2 additions & 2 deletions docs/dev_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,13 +54,13 @@ pipx install poetry
Now you can initialize your new project with the Cookiecutter template for taps:

```bash
cookiecutter https://gitlab.com/meltano/sdk --directory="cookiecutter/tap-template"
cookiecutter https://github.com/meltano/sdk --directory="cookiecutter/tap-template"
```

...or for targets:

```bash
cookiecutter https://gitlab.com/meltano/sdk --directory="cookiecutter/target-template"
cookiecutter https://github.com/meltano/sdk --directory="cookiecutter/target-template"
```

Note that you do not need to create the directory for the tap.
Expand Down
2 changes: 1 addition & 1 deletion docs/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Most likely you should delete the project and start over.

## What are all of the Property options?

The property types are defined in [this file](https://gitlab.com/meltano/sdk/-/blob/main/singer_sdk/typing.py).
The property types are documented in the [JSON Schema builder docs](./typing).
However, if you're using an IDE such as VSCode, you should be able to set up the environment to give you autocompletion prompts or hints.
Ensure your interpreter is set to poetry if you've followed the [Dev Guide](./dev_guide.md).
Checkout this [gif](https://visualstudiomagazine.com/articles/2021/04/20/~/media/ECG/visualstudiomagazine/Images/2021/04/poetry.ashx) for how to change your interpreter.
Expand Down
2 changes: 1 addition & 1 deletion docs/implementation/at_least_once.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,6 @@ FROM {{ source('tap_widgets', 'widgets') }} AS raw

## Future proposals to mitigate record duplication

There is a [feature proposal](https://gitlab.com/meltano/sdk/-/issues/162) for the SDK to optionally store record hashes within the tap's state object and then dedupe record hashes against new records prior to sending data downstream to the target. This would likely be an opt-in behavior for developers and/or users, and it would come at some small performance penalty, as well as a small cost of increased size of the state object. If you are interested in contributing this feature, please see [this issue](https://gitlab.com/meltano/sdk/-/issues/162).
There is a [feature proposal](https://github.com/meltano/sdk/issues/161) for the SDK to optionally store record hashes within the tap's state object and then dedupe record hashes against new records prior to sending data downstream to the target. This would likely be an opt-in behavior for developers and/or users, and it would come at some small performance penalty, as well as a small cost of increased size of the state object. If you are interested in contributing this feature, please see [this issue](https://github.com/meltano/sdk/issues/161).

Note that while this future proposal may resolve the issue of duplicates due to signposts and greater-than-or-equal-to comparison logic, streams will still be subject to record duplication due to interrupted and retried sync operations. Thus, any implementations not using primary keys to dedupe data in the target will always need some plan for a deduplication strategy in their downstream data processing.
10 changes: 5 additions & 5 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ but they can also be used in any data pipeline tool that supports the
`Singer Spec`_.

The SDK project is 100% open source, licensed under the `Apache 2.0`_
permissive license. Please visit our code repo `here in Gitlab`_, where you can
permissive license. Please visit our code repo `here in GitHub`_, where you can
read the `source code`_, `log an issue or feature request`_, and `contribute back`_
to the SDK. We also regularly discuss SDK topics in our `Meltano Slack`_ community,
within the `#singer-tap-development`_ and `#singer-target-development`_ Slack channels.
Expand Down Expand Up @@ -74,11 +74,11 @@ Advanced Topics
.. _Singer Spec: https://hub.meltano.com/singer/spec
.. _Meltano: https://www.meltano.com
.. _integrated with Meltano: https://docs.meltano.com/tutorials/custom-extractor#add-the-plugin-to-your-meltano-project
.. _contribute back: https://gitlab.com/meltano/sdk/-/issues?scope=all&state=opened&label_name[]=Accepting%20Merge%20Requests
.. _source code: https://gitlab.com/meltano/sdk
.. _log an issue or feature request: https://gitlab.com/meltano/sdk/-/issues
.. _contribute back: https://github.com/meltano/sdk/issues?q=is%3Aopen+is%3Aissue+label%3A%22accepting+merge+requests%22
.. _source code: https://github.com/meltano/sdk
.. _log an issue or feature request: https://github.com/meltano/sdk/-/issues
.. _Apache 2.0: https://en.wikipedia.org/wiki/Apache_License
.. _here in Gitlab: https://gitlab.com/meltano/sdk
.. _here in GitHub: https://github.com/meltano/sdk
.. _Meltano Slack: https://join.slack.com/t/meltano/shared_invite/zt-obgpdeba-7yrqKhwyMBfdHDXsZY8G7Q
.. _#singer-tap-development: https://meltano.slack.com/archives/C01PKLU5D1R
.. _#singer-target-development: https://meltano.slack.com/archives/C01RKUVUG4S
Expand Down
2 changes: 1 addition & 1 deletion docs/python_tips.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ environment to another or if they can change at runtime.
### Static example

Here's a simple example of static definitions based on the cookiecutter
[template](https://gitlab.com/meltano/sdk/-/tree/main/cookiecutter/tap-template).
[template](https://github.com/meltano/sdk/tree/main/cookiecutter/tap-template).
This example defines the primary key and replication key as fixed values which will not change.

```python
Expand Down
7 changes: 3 additions & 4 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ authors = ["Meltano Team and Contributors"]
maintainers = ["Meltano Team and Contributors"]
readme = "README.md"
homepage = "https://sdk.meltano.com/en/latest/"
repository = "https://gitlab.com/meltano/sdk"
repository = "https://github.com/meltano/sdk"
documentation = "https://sdk.meltano.com/en/latest/"
keywords = [
"Meltano",
Expand All @@ -19,11 +19,10 @@ classifiers = [
license = "Apache 2.0"

[tool.poetry.urls]
"Issue Tracker" = "https://gitlab.com/meltano/sdk/-/issues"
"Issue Tracker" = "https://github.com/meltano/sdk/issues"
"Twitter" = "https://twitter.com/meltanodata/"
"Changelog" = "https://gitlab.com/meltano/sdk/-/blob/main/CHANGELOG.md"
"Changelog" = "https://github.com/meltano/sdk/blob/main/CHANGELOG.md"
"Slack" = "https://meltano.com/slack"
"GitHub Mirror" = "https://github.com/meltano/sdk"
"Youtube" = "https://www.youtube.com/meltano"

[tool.poetry.dependencies]
Expand Down
6 changes: 0 additions & 6 deletions singer_sdk/sinks/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,6 @@ class Sink(metaclass=abc.ABCMeta):

MAX_SIZE_DEFAULT = 10000

# TODO: Implement schema flattening: https://gitlab.com/meltano/sdk/-/issues/133
# _flattener: Optional[RecordFlattener]
# _MAX_FLATTEN_DEPTH = 0

def __init__(
self,
target: PluginBase,
Expand Down Expand Up @@ -74,8 +70,6 @@ def __init__(
self._batch_dupe_records_merged: int = 0

self._validator = Draft4Validator(schema, format_checker=FormatChecker())
# TODO: Implement schema flattener: https://gitlab.com/meltano/sdk/-/issues/133
# self._flattener = RecordFlattener(max_level=self._MAX_FLATTEN_DEPTH)

def _get_context(self, record: dict) -> dict:
"""Return an empty dictionary by default.
Expand Down

0 comments on commit c2d8f41

Please sign in to comment.