Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Developers can now customize the default logging configuration for their taps/targets by adding default_logging.yml to their package #2432

18 changes: 18 additions & 0 deletions docs/implementation/logging.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,3 +77,21 @@ This will send metrics to a `metrics.log`:
2022-09-29 00:48:53,302 INFO METRIC: {"metric_type": "timer", "metric": "sync_duration", "value": 0.5258760452270508, "tags": {"stream": "countries", "context": {}, "status": "succeeded"}}
2022-09-29 00:48:53,303 INFO METRIC: {"metric_type": "counter", "metric": "record_count", "value": 250, "tags": {"stream": "countries", "context": {}}}
```

## For package developers

If you're developing a tap or target package and would like to customize its logging configuration, you can put a `default_loggging.yml` file in the package root to set the default logging configuration for your package. This file will be used if the `SINGER_SDK_LOG_CONFIG` environment variable is not set:

```
.
├── README.md
├── poetry.lock
├── pyproject.toml
└── tap_example
    ├── __init__.py
    ├── __main__.py
    ├── default_logging.yml # <-- This file will be used if SINGER_SDK_LOG_CONFIG is not set
    ├── client.py
    ├── streams.py
    └── tap.py
```
9 changes: 5 additions & 4 deletions singer_sdk/metrics.py
Original file line number Diff line number Diff line change
Expand Up @@ -394,14 +394,15 @@ def _load_yaml_logging_config(path: Traversable | Path) -> t.Any: # noqa: ANN40
return yaml.safe_load(f)


def _get_default_config() -> t.Any: # noqa: ANN401
def _get_default_config_path() -> Traversable:
"""Get a logging configuration.

Returns:
A logging configuration.
"""
log_config_path = get_package_files("singer_sdk").joinpath("default_logging.yml")
return _load_yaml_logging_config(log_config_path)
filename = "default_logging.yml"
path = get_package_files(__package__) / filename
return path if path.is_file() else get_package_files("singer_sdk") / filename


def _setup_logging(config: t.Mapping[str, t.Any]) -> None:
Expand All @@ -410,7 +411,7 @@ def _setup_logging(config: t.Mapping[str, t.Any]) -> None:
Args:
config: A plugin configuration dictionary.
"""
logging.config.dictConfig(_get_default_config())
logging.config.dictConfig(_load_yaml_logging_config(_get_default_config_path()))

config = config or {}
metrics_log_level = config.get(METRICS_LOG_LEVEL_SETTING, "INFO").upper()
Expand Down
Loading