Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Refresh tap [MFLP-9] #101

Merged
merged 7 commits into from
Dec 14, 2024
Merged

feat: Refresh tap [MFLP-9] #101

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
fd4bc07
feat: Refresh tap
edgarrmondragon Dec 14, 2024
1fa77d8
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Dec 14, 2024
aaf9cb9
Update workflow
edgarrmondragon Dec 14, 2024
e511250
Use uv in CI
edgarrmondragon Dec 14, 2024
fe85dc7
Use `pass_env`
edgarrmondragon Dec 14, 2024
4bbb3a5
Fix linting
edgarrmondragon Dec 14, 2024
ece5988
Skip package installation for Ruff
edgarrmondragon Dec 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 3 additions & 4 deletions .env.template
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
TAP_JIRA_AUTH_FLOW = ''
TAP_JIRA_AUTH_USERNAME = ''
TAP_JIRA_AUTH_PASSWORD = ''
TAP_JIRA_DOMAIN = ''
TAP_JIRA_DOMAIN=''
TAP_JIRA_API_TOKEN=''
TAP_JIRA_EMAIL=''
82 changes: 46 additions & 36 deletions .github/workflows/ci_workflow.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,56 +1,66 @@
### A CI workflow template that runs linting and python testing
### TODO: Modify as needed or as desired.

name: Test tap-jira

on: [push]
on:
push:
branches: [ main ]
paths:
- .github/workflows/ci_workflow.yml
- tap_jira/**
- pyproject.toml
- poetry.lock
- tox.ini
pull_request:
branches: [ main ]
paths:
- .github/workflows/ci_workflow.yml
- tap_jira/**
- pyproject.toml
- poetry.lock
- tox.ini

env:
FORCE_COLOR: "1"

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref }}
cancel-in-progress: true

jobs:
linting:
runs-on: ubuntu-latest
strategy:
matrix:
# Only lint using the primary version used for dev
python-version: ["3.9"]
steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Install pipx and Poetry
run: |
pip install pipx poetry
- name: Run lint command from tox.ini
run: |
pipx run tox -e lint
python-version: 3.x
- uses: astral-sh/setup-uv@v4
with:
version: ">=0.5"
- run: uvx --with tox-uv tox -e lint

pytest:
name: Python ${{ matrix.python-version }} - pytest
runs-on: ubuntu-latest
env:
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}}
strategy:
max-parallel: 2
matrix:
python-version: ["3.9"]
python-version:
- "3.13"
- "3.12"
- "3.11"
- "3.10"
- "3.9"
steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Install Poetry
run: |
pip install poetry
- name: Install dependencies
run: |
poetry install
- uses: astral-sh/setup-uv@v4
with:
version: ">=0.5"
- name: Test with pytest
id: test_pytest
continue-on-error: false
env:
TAP_JIRA_AUTH_FLOW: ${{ secrets.flow }}
TAP_JIRA_AUTH_USERNAME: ${{ secrets.username }}
TAP_JIRA_AUTH_PASSWORD: ${{ secrets.password }}
TAP_JIRA_DOMAIN: ${{ secrets.domain }}
run: |
poetry run pytest --capture=no
TAP_JIRA_DOMAIN: ${{ secrets.TAP_JIRA_DOMAIN }}
TAP_JIRA_API_TOKEN: ${{ secrets.TAP_JIRA_API_TOKEN }}
TAP_JIRA_EMAIL: ${{ secrets.TAP_JIRA_EMAIL }}
run: uvx --with tox-uv tox -e ${{ matrix.python-version }}
12 changes: 4 additions & 8 deletions .pre-commit-config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ ci:

repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
rev: v5.0.0
hooks:
- id: check-json
- id: check-toml
Expand All @@ -14,18 +14,14 @@ repos:
- id: trailing-whitespace

- repo: https://github.com/charliermarsh/ruff-pre-commit
rev: v0.0.263
rev: v0.8.3
hooks:
- id: ruff
args: [--fix, --exit-non-zero-on-fix]

- repo: https://github.com/psf/black
rev: 23.3.0
hooks:
- id: black
- id: ruff-format

- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.2.0
rev: v1.13.0
hooks:
- id: mypy
additional_dependencies:
Expand Down
57 changes: 37 additions & 20 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# `tap-jira`
# tap-jira

tap-jira tap class.

Expand All @@ -14,30 +14,47 @@ Built with the [Meltano Singer SDK](https://sdk.meltano.com).
* `schema-flattening`
* `batch`

## Settings

| Setting | Required | Default | Description |
|:--------------------|:--------:|:-------:|:------------|
| start_date | False | None | Earliest record date to sync |
| end_date | False | None | Latest record date to sync |
| domain | True | None | The Domain for your Jira account, e.g. meltano.atlassian.net |
| api_token | True | None | Jira API Token. |
| Email | True | None | The user email for your Jira account. |
| page_size | False | None | |
| stream_maps | False | None | Config object for stream maps capability. For more information check out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html). |
| stream_map_config | False | None | User-defined config values to be used within map expressions. |
| flattening_enabled | False | None | 'True' to enable schema flattening and automatically expand nested properties. |
| flattening_max_depth| False | None | The max depth to flatten schemas. |
| batch_config | False | None | |

A full list of supported settings and capabilities is available by running: `tap-jira --about`

## Supported Python Versions

* 3.8
* 3.9
* 3.10
* 3.11
* 3.12
* 3.13

## Settings

| Setting | Required | Default | Description |
| :-------------------------------- | :------- | :------ | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| start_date | False | None | Earliest record date to sync |
| end_date | False | None | Latest record date to sync |
| domain | True | None | The Domain for your Jira account, e.g. meltano.atlassian.net |
| api_token | True | None | Jira API Token. |
| email | True | None | The user email for your Jira account. |
| page_size | False | None | |
| page_size.issues | False | 100 | Page size for issues stream |
| include_audit_logs | False | False | Include the audit logs stream |

### Built-in capabilities

| Setting | Required | Default | Description |
| :-------------------------------- | :------- | :------ | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| stream_maps | False | None | Config object for stream maps capability. For more information check out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html). |
| stream_map_config | False | None | User-defined config values to be used within map expressions. |
| faker_config | False | None | Config for the [`Faker`](https://faker.readthedocs.io/en/master/) instance variable `fake` used within map expressions. Only applicable if the plugin specifies `faker` as an addtional dependency (through the `singer-sdk` `faker` extra or directly). |
| faker_config.seed | False | None | Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator |
| faker_config.locale | False | None | One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization |
| flattening_enabled | False | None | 'True' to enable schema flattening and automatically expand nested properties. |
| flattening_max_depth | False | None | The max depth to flatten schemas. |
| batch_config | False | None | Configuration for BATCH message capabilities. |
| batch_config.encoding | False | None | Specifies the format and compression of the batch files. |
| batch_config.encoding.format | False | None | Format to use for batch files. |
| batch_config.encoding.compression | False | None | Compression format to use for batch files. |
| batch_config.storage | False | None | Defines the storage layer to use when writing batch files |
| batch_config.storage.root | False | None | Root path to use when writing batch files. |
| batch_config.storage.prefix | False | None | Prefix to use when writing batch files. |

A full list of supported settings and capabilities is available by running: `tap-jira --about`

## Elastic License 2.0

Expand Down
Loading
Loading