-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
0 parents
commit c0605cf
Showing
19 changed files
with
2,208 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
# To get started with Dependabot version updates, you'll need to specify which | ||
# package ecosystems to update and where the package manifests are located. | ||
# Please see the documentation for all configuration options: | ||
# https://help.github.com/github/administering-a-repository/configuration-options-for-dependency-updates | ||
|
||
version: 2 | ||
updates: | ||
- package-ecosystem: pip | ||
directory: "/" | ||
schedule: | ||
interval: "daily" | ||
commit-message: | ||
prefix: "chore(deps): " | ||
prefix-development: "chore(deps-dev): " | ||
- package-ecosystem: pip | ||
directory: "/.github/workflows" | ||
schedule: | ||
interval: daily | ||
commit-message: | ||
prefix: "ci: " | ||
- package-ecosystem: github-actions | ||
directory: "/" | ||
schedule: | ||
interval: "weekly" | ||
commit-message: | ||
prefix: "ci: " |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
### A CI workflow template that runs linting and python testing | ||
### TODO: Modify as needed or as desired. | ||
|
||
name: Test tap-msaccess | ||
|
||
on: [push] | ||
|
||
jobs: | ||
pytest: | ||
runs-on: ubuntu-latest | ||
env: | ||
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}} | ||
strategy: | ||
matrix: | ||
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"] | ||
steps: | ||
- uses: actions/checkout@v4 | ||
- name: Set up Python ${{ matrix.python-version }} | ||
uses: actions/setup-python@v5 | ||
with: | ||
python-version: ${{ matrix.python-version }} | ||
- name: Install Poetry | ||
run: | | ||
pip install poetry | ||
- name: Install dependencies | ||
run: | | ||
poetry install | ||
- name: Test with pytest | ||
run: | | ||
poetry run pytest |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,136 @@ | ||
# Secrets and internal config files | ||
**/.secrets/* | ||
|
||
# Ignore meltano internal cache and sqlite systemdb | ||
|
||
.meltano/ | ||
|
||
# Byte-compiled / optimized / DLL files | ||
__pycache__/ | ||
*.py[cod] | ||
*$py.class | ||
|
||
# C extensions | ||
*.so | ||
|
||
# Distribution / packaging | ||
.Python | ||
build/ | ||
develop-eggs/ | ||
dist/ | ||
downloads/ | ||
eggs/ | ||
.eggs/ | ||
lib/ | ||
lib64/ | ||
parts/ | ||
sdist/ | ||
var/ | ||
wheels/ | ||
pip-wheel-metadata/ | ||
share/python-wheels/ | ||
*.egg-info/ | ||
.installed.cfg | ||
*.egg | ||
MANIFEST | ||
|
||
# PyInstaller | ||
# Usually these files are written by a python script from a template | ||
# before PyInstaller builds the exe, so as to inject date/other infos into it. | ||
*.manifest | ||
*.spec | ||
|
||
# Installer logs | ||
pip-log.txt | ||
pip-delete-this-directory.txt | ||
|
||
# Unit test / coverage reports | ||
htmlcov/ | ||
.tox/ | ||
.nox/ | ||
.coverage | ||
.coverage.* | ||
.cache | ||
nosetests.xml | ||
coverage.xml | ||
*.cover | ||
*.py,cover | ||
.hypothesis/ | ||
.pytest_cache/ | ||
|
||
# Translations | ||
*.mo | ||
*.pot | ||
|
||
# Django stuff: | ||
*.log | ||
local_settings.py | ||
db.sqlite3 | ||
db.sqlite3-journal | ||
|
||
# Flask stuff: | ||
instance/ | ||
.webassets-cache | ||
|
||
# Scrapy stuff: | ||
.scrapy | ||
|
||
# Sphinx documentation | ||
docs/_build/ | ||
|
||
# PyBuilder | ||
target/ | ||
|
||
# Jupyter Notebook | ||
.ipynb_checkpoints | ||
|
||
# IPython | ||
profile_default/ | ||
ipython_config.py | ||
|
||
# pyenv | ||
.python-version | ||
|
||
# pipenv | ||
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control. | ||
# However, in case of collaboration, if having platform-specific dependencies or dependencies | ||
# having no cross-platform support, pipenv may install dependencies that don't work, or not | ||
# install all needed dependencies. | ||
#Pipfile.lock | ||
|
||
# PEP 582; used by e.g. github.com/David-OConnor/pyflow | ||
__pypackages__/ | ||
|
||
# Celery stuff | ||
celerybeat-schedule | ||
celerybeat.pid | ||
|
||
# SageMath parsed files | ||
*.sage.py | ||
|
||
# Environments | ||
.env | ||
.venv | ||
env/ | ||
venv/ | ||
ENV/ | ||
env.bak/ | ||
venv.bak/ | ||
|
||
# Spyder project settings | ||
.spyderproject | ||
.spyproject | ||
|
||
# Rope project settings | ||
.ropeproject | ||
|
||
# mkdocs documentation | ||
/site | ||
|
||
# mypy | ||
.mypy_cache/ | ||
.dmypy.json | ||
dmypy.json | ||
|
||
# Pyre type checker | ||
.pyre/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,38 @@ | ||
ci: | ||
autofix_prs: true | ||
autoupdate_schedule: weekly | ||
autoupdate_commit_msg: 'chore: pre-commit autoupdate' | ||
|
||
repos: | ||
- repo: https://github.com/pre-commit/pre-commit-hooks | ||
rev: v4.5.0 | ||
hooks: | ||
- id: check-json | ||
exclude: | | ||
(?x)^( | ||
\.vscode/.*\.json | ||
)$ | ||
- id: check-toml | ||
- id: check-yaml | ||
- id: end-of-file-fixer | ||
- id: trailing-whitespace | ||
|
||
- repo: https://github.com/python-jsonschema/check-jsonschema | ||
rev: 0.27.3 | ||
hooks: | ||
- id: check-dependabot | ||
- id: check-github-workflows | ||
|
||
- repo: https://github.com/astral-sh/ruff-pre-commit | ||
rev: v0.1.14 | ||
hooks: | ||
- id: ruff | ||
args: [--fix, --exit-non-zero-on-fix, --show-fixes] | ||
- id: ruff-format | ||
|
||
- repo: https://github.com/pre-commit/mirrors-mypy | ||
rev: v1.8.0 | ||
hooks: | ||
- id: mypy | ||
additional_dependencies: | ||
- types-requests |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
# IMPORTANT! This folder is hidden from git - if you need to store config files or other secrets, | ||
# make sure those are never staged for commit into your git repo. You can store them here or another | ||
# secure location. | ||
# | ||
# Note: This may be redundant with the global .gitignore for, and is provided | ||
# for redundancy. If the `.secrets` folder is not needed, you may delete it | ||
# from the project. | ||
|
||
* | ||
!.gitignore |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
{ | ||
// Use IntelliSense to learn about possible attributes. | ||
// Hover to view descriptions of existing attributes. | ||
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387 | ||
"version": "0.2.0", | ||
"configurations": [ | ||
{ | ||
"name": "tap-msaccess", | ||
"type": "python", | ||
"request": "launch", | ||
"cwd": "${workspaceFolder}", | ||
"program": "tap_msaccess", | ||
"justMyCode": false, | ||
"args": [ | ||
"--config", | ||
".secrets/config.json", | ||
], | ||
}, | ||
] | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,131 @@ | ||
# tap-msaccess | ||
|
||
`tap-msaccess` is a Singer tap for Microsoft Access. | ||
|
||
Built with the [Meltano Tap SDK](https://sdk.meltano.com) for Singer Taps. | ||
|
||
<!-- | ||
Developer TODO: Update the below as needed to correctly describe the install procedure. For instance, if you do not have a PyPi repo, or if you want users to directly install from your git repo, you can modify this step as appropriate. | ||
## Installation | ||
Install from PyPi: | ||
```bash | ||
pipx install tap-msaccess | ||
``` | ||
Install from GitHub: | ||
```bash | ||
pipx install git+https://github.com/ORG_NAME/tap-msaccess.git@main | ||
``` | ||
--> | ||
|
||
## Configuration | ||
|
||
### Accepted Config Options | ||
|
||
<!-- | ||
Developer TODO: Provide a list of config options accepted by the tap. | ||
This section can be created by copy-pasting the CLI output from: | ||
``` | ||
tap-msaccess --about --format=markdown | ||
``` | ||
--> | ||
|
||
A full list of supported settings and capabilities for this | ||
tap is available by running: | ||
|
||
```bash | ||
tap-msaccess --about | ||
``` | ||
|
||
### Configure using environment variables | ||
|
||
This Singer tap will automatically import any environment variables within the working directory's | ||
`.env` if the `--config=ENV` is provided, such that config values will be considered if a matching | ||
environment variable is set either in the terminal context or in the `.env` file. | ||
|
||
### Source Authentication and Authorization | ||
|
||
<!-- | ||
Developer TODO: If your tap requires special access on the source system, or any special authentication requirements, provide those here. | ||
--> | ||
|
||
## Usage | ||
|
||
You can easily run `tap-msaccess` by itself or in a pipeline using [Meltano](https://meltano.com/). | ||
|
||
### Executing the Tap Directly | ||
|
||
```bash | ||
tap-msaccess --version | ||
tap-msaccess --help | ||
tap-msaccess --config CONFIG --discover > ./catalog.json | ||
``` | ||
|
||
## Developer Resources | ||
|
||
Follow these instructions to contribute to this project. | ||
|
||
### Initialize your Development Environment | ||
|
||
```bash | ||
pipx install poetry | ||
poetry install | ||
``` | ||
|
||
### Create and Run Tests | ||
|
||
Create tests within the `tests` subfolder and | ||
then run: | ||
|
||
```bash | ||
poetry run pytest | ||
``` | ||
|
||
You can also test the `tap-msaccess` CLI interface directly using `poetry run`: | ||
|
||
```bash | ||
poetry run tap-msaccess --help | ||
``` | ||
|
||
### Testing with [Meltano](https://www.meltano.com) | ||
|
||
_**Note:** This tap will work in any Singer environment and does not require Meltano. | ||
Examples here are for convenience and to streamline end-to-end orchestration scenarios._ | ||
|
||
<!-- | ||
Developer TODO: | ||
Your project comes with a custom `meltano.yml` project file already created. Open the `meltano.yml` and follow any "TODO" items listed in | ||
the file. | ||
--> | ||
|
||
Next, install Meltano (if you haven't already) and any needed plugins: | ||
|
||
```bash | ||
# Install meltano | ||
pipx install meltano | ||
# Initialize meltano within this directory | ||
cd tap-msaccess | ||
meltano install | ||
``` | ||
|
||
Now you can test and orchestrate using Meltano: | ||
|
||
```bash | ||
# Test invocation: | ||
meltano invoke tap-msaccess --version | ||
# OR run a test `elt` pipeline: | ||
meltano elt tap-msaccess target-jsonl | ||
``` | ||
|
||
### SDK Dev Guide | ||
|
||
See the [dev guide](https://sdk.meltano.com/en/latest/dev_guide.html) for more instructions on how to use the SDK to | ||
develop your own taps and targets. |
Oops, something went wrong.